MODELDRIFT.WATCH

Sep 27 · Day 18

MINOR · GEMINI · QUESTION 33

Gemini named a different first choice for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 26beforePermalink

Picks read from the answer, in order
  1. Identify "Shadow IT" and Rogue Spending
  2. Watch the Pricing Metric Traps
  3. Factor in the "Hidden TCO
  4. Leverage Your Existing Stack

As a VP of Procurement at a 5,000-person enterprise, your lens for selecting an email marketing platform is very different from your CMO’s. While Marketing cares about UI, templates, and campaign builders, you care about Total Cost of Ownership (TCO), tech stack consolidation, data security (CCPA/GDPR), CRM integration, and implementation costs.

At the 5,000-employee mark, you are well past SMB tools like Mailchimp. You need an Enterprise Marketing Automation Platform (MAP).

To give you the best recommendation, the choice splits entirely on your company’s business model (B2B vs. B2C) and your existing CRM infrastructure. Here is the executive breakdown of the platforms you should be taking to RFP.

1. If you are a B2B Company (Focus on Lead Nurturing & Sales Alignment)

In B2B, the email platform must integrate flawlessly with your CRM.

  • •Adobe Marketo Engage:
  • •The Vibe: The gold standard for complex B2B enterprises.
  • •Procurement Note: High licensing cost and usually requires expensive third-party agencies to implement and run. Pricing is based on database size.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot):
  • •The Vibe: The default choice if your enterprise runs on Salesforce Sales Cloud.
  • •Procurement Note: Great opportunity for vendor consolidation and negotiating a bundled master service agreement (MSA) if you already use Salesforce.
  • •Oracle Eloqua:
  • •The Vibe: Highly secure, rigid, and handles massive, complex data architectures.
  • •Procurement Note: Often preferred by highly regulated industries (finance, healthcare). Heavy enterprise pricing, but highly negotiable at the end of Oracle's fiscal quarters.
  • •HubSpot (Enterprise Tier):
  • •The Vibe: The modern, user-friendly alternative.
  • •Procurement Note: HubSpot’s TCO is often significantly lower than Marketo or Salesforce because it requires far fewer external consultants/developers to operate. Marketing teams adopt it faster, meaning higher ROI. Pricing is based on "Marketing Contacts."

2. If you are a B2C/E-commerce Company (Focus on High Volume & Personalization)

In B2C, you need systems that can handle millions of sends, mobile push notifications, and SMS.

  • •Salesforce Marketing Cloud (formerly ExactTarget):
  • •The Vibe: The heavyweight champion for B2C cross-channel journeys.
  • •Procurement Note: Extremely complex. Be prepared to budget an additional 50-100% of the software cost for an integration partner (Deloitte, Accenture, or boutique agencies) just to set it up.
  • •Braze or Iterable:
  • •The Vibe: The modern disruptors. Mobile-first, highly agile, beloved by modern engineering and marketing teams.
  • •Procurement Note: Pricing is usually based on Monthly Active Users (MAU) or message volume. They are easier to implement than legacy Salesforce/Adobe products, saving on implementation fees.
  • •Adobe Campaign:
  • •The Vibe: Best if you are already heavily invested in the Adobe Experience Cloud (AEM, Adobe Analytics).
  • •Procurement Note: Only buy this if you are consolidating an Adobe tech stack; otherwise, it is too heavy and expensive as a standalone tool.

3. The Procurement Strategy: How to Evaluate Them

When your marketing team brings you their preferred vendor, here are the levers you should pull during evaluation and negotiation:

1. Identify "Shadow IT" and Rogue Spending
In a 5k-person enterprise, I guarantee various departments have bought rogue instances of Mailchimp, Constant Contact, or SendGrid on corporate credit cards. Use this migration to mandate consolidation. Audit your expenses, kill the rogue tools, and use that budget to fund the enterprise platform.

2. Watch the Pricing Metric Traps

  • •Contact-based pricing: (e.g., HubSpot, Marketo). You will be charged for every email address in the database, even if you don't email them. Negotiate a "Marketing Contacts only" or active-contact clause.
  • •Volume-based pricing: (e.g., Braze, SFMC). You are charged by the number of emails/messages sent. If marketing scales up successfully, your costs will skyrocket. Negotiate tiered volume discounts upfront.

3. Factor in the "Hidden TCO" (Implementation & Services)
Do not look just at the SaaS license. A $150k Marketo license might require a $100k implementation agency and a $120k/year certified Marketo architect on staff. Ask vendors to provide a realistic estimate of required professional services.

4. Leverage Your Existing Stack
If your 5,000-person company uses Salesforce, Microsoft Dynamics, or Snowflake, prioritize platforms that have native integrations. Custom-building APIs between an email platform and an enterprise ERP/CRM will cost you hundreds of thousands of dollars in IT resources.

Your Next Step:

Schedule a 30-minute alignment meeting with your CMO and ask three questions:

  • 1."Are we buying this for B2B lead generation, or B2C customer retention?"
  • 2."What CRM must this integrate with natively?"
  • 3."Do we have the internal talent to run a complex system, or do we need something with high usability?"

Once you have those answers, you can narrow the RFP down to the Salesforce/Adobe bucket (for legacy enterprise power) or the HubSpot/Braze bucket (for modern usability and lower TCO).

SEP 27afterPermalink

Picks read from the answer, in order
  1. Align with the existing CRM / Data Warehouse
  2. Scrutinize the Pricing Model
  3. Security

As a VP of Procurement at a 5,000-person enterprise, your lens on this purchase is different from your CMO’s. While Marketing cares about drag-and-drop builders and personalization, you care about Total Cost of Ownership (TCO), tech stack consolidation, data security (SOC 2, CCPA, GDPR), and vendor viability.

At your scale, you are no longer looking for an "email marketing platform" (like Mailchimp or Constant Contact). You are looking for an Enterprise Marketing Automation or Customer Engagement Platform.

To give you the best recommendation, the first question you must ask your marketing team is: Are we primarily B2B or B2C? The architecture and pricing models for these two paths are vastly different.

Here is the shortlist of platforms you should evaluate, categorized by your company's business model, followed by a procurement-specific evaluation guide.

1. If Your Company is B2B (Long sales cycles, lead generation, account-based marketing)

In B2B, email marketing must integrate flawlessly with your CRM (likely Salesforce or Microsoft Dynamics). Pricing is typically based on the number of contacts in the database.

  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot):
  • •Best for: Companies already using Salesforce as their CRM.
  • •Procurement view: The native integration saves hundreds of thousands of dollars in custom API development and IT maintenance. However, Salesforce renewals are notorious for aggressive price hikes.
  • •Adobe Marketo Engage:
  • •Best for: Highly complex, global B2B marketing teams with massive databases and advanced lead-scoring needs.
  • •Procurement view: Marketo is the gold standard for B2B, but implementation is expensive. You will likely need to budget for a third-party implementation agency, which can double your first-year TCO.
  • •HubSpot (Enterprise Hub):
  • •Best for: Ease of use and rapid adoption. If your marketing team complains that your current tools are too hard to use, HubSpot is the antidote.
  • •Procurement view: Excellent TCO when considering implementation and training (which are cheaper than Adobe or Salesforce). However, their contact-based pricing scales aggressively as your database grows.

2. If Your Company is B2C (E-commerce, retail, apps, high-volume consumer emails)

In B2C, you are dealing with millions of emails, SMS, and push notifications. Pricing is usually based on message volume (CPM) or Monthly Active Users (MAU).

  • •Salesforce Marketing Cloud (formerly ExactTarget):
  • •Best for: Omnichannel consumer journeys (Email, SMS, App push) connected to complex data architectures.
  • •Procurement view: Extremely powerful, but highly fragmented (you buy "builders" piecemeal—Email Studio, Mobile Studio, etc.). Negotiation requires a very close reading of the contract regarding data storage limits and API call limits, which result in surprise overages.
  • •Braze or Iterable:
  • •Best for: Modern tech, SaaS, or retail companies that rely heavily on mobile apps + email.
  • •Procurement view: These are the modern disruptors eating into Salesforce and Adobe’s market share. They are highly agile, often have lower implementation costs, and feature transparent MAU (Monthly Active User) pricing.
  • •Klaviyo (Enterprise):
  • •Best for: E-commerce (especially if you use Shopify Plus).
  • •Procurement view: Highly specialized for retail/e-commerce. It replaces multiple tools (email, SMS, reviews) which allows for great vendor consolidation.

The Procurement Strategy: How to evaluate and negotiate

When you take these options to your marketing and IT leaders, guide the evaluation using these procurement priorities:

1. Align with the existing CRM / Data Warehouse
Do not let Marketing buy a tool in a silo. If you use Salesforce CRM, lean toward Salesforce Marketing Cloud. If you use Adobe Experience Manager, lean toward Marketo. If your data lives in Snowflake, look at Braze. Native integrations save IT resources and prevent messy middleware costs.

2. Scrutinize the Pricing Model (The "Gotchas")

  • •Contact-based (HubSpot/Marketo): Ensure you are only paying for marketable contacts, not every single contact in your CRM (like unsubscribes or bounced emails).
  • •Volume-based (Salesforce/Braze): Negotiate your CPM (cost per mille/thousand sends) upfront, but also pre-negotiate the overage rates.

3. Budget for Implementation (The Hidden TCO)
At a 5,000-person enterprise, you do not just "turn on" an email platform. You will need a System Integrator (SI) like Deloitte, Slalom, or a specialized boutique agency to migrate your data, warm up your IP addresses, and build the architecture. Rule of thumb: Expect year-one implementation costs to be 50% to 150% of your software licensing cost.

4. Security, Compliance & Deliverability
Because you are in the US, the platform must seamlessly handle CAN-SPAM, CCPA (California privacy), and potentially HIPAA if you are in healthcare. You also need Dedicated IP addresses (crucial for a company of your size to maintain email sender reputation) and robust SSO (Single Sign-On) with Role-Based Access Control (RBAC) so a junior marketer can't accidentally email 5 million people.

Your Next Step

Schedule a 30-minute alignment meeting with your CMO and CIO/CTO.
Ask them:

  • 1.Are we looking to optimize B2B lead gen or B2C consumer engagement?
  • 2.Where does our primary customer data live today (CRM or Data Warehouse)?
  • 3.Are we willing to consolidate other tools (SMS, social posting) into this purchase?

Once they answer those three questions, you can confidently narrow the list to two of the vendors above and begin the RFP process.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-26 permalink

Scored
answered
Received
2026-09-26 09:24:51 UTC
Run
20260926T090007Z-89c0b7
Served by
Google · HTTP 200 · finish stop
Size
5,451 characters · 2,377 tokens out incl. hidden reasoning · 21.8 s
Extracted picks, in order · confidence high
  1. Identify "Shadow IT" and Rogue Spending
  2. Watch the Pricing Metric Traps
  3. Factor in the "Hidden TCO
  4. Leverage Your Existing Stack

Highlighted: the names the extractor took as picks, where they appear verbatim (4 of 4).

As a VP of Procurement at a 5,000-person enterprise, your lens for selecting an email marketing platform is very different from your CMO’s. While Marketing cares about UI, templates, and campaign builders, **you care about Total Cost of Ownership (TCO), tech stack consolidation, data security (CCPA/GDPR), CRM integration, and implementation costs.**
At the 5,000-employee mark, you are well past SMB tools like Mailchimp. You need an **Enterprise Marketing Automation Platform (MAP)**.
To give you the best recommendation, the choice splits entirely on your company’s business model (B2B vs. B2C) and your existing CRM infrastructure. Here is the executive breakdown of the platforms you should be taking to RFP.
---
### 1. If you are a B2B Company (Focus on Lead Nurturing & Sales Alignment)
In B2B, the email platform must integrate flawlessly with your CRM.
* **Adobe Marketo Engage:**
* **The Vibe:** The gold standard for complex B2B enterprises.
* **Procurement Note:** High licensing cost and usually requires expensive third-party agencies to implement and run. Pricing is based on database size.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot):**
* **The Vibe:** The default choice if your enterprise runs on Salesforce Sales Cloud.
* **Procurement Note:** Great opportunity for vendor consolidation and negotiating a bundled master service agreement (MSA) if you already use Salesforce.
* **Oracle Eloqua:**
* **The Vibe:** Highly secure, rigid, and handles massive, complex data architectures.
* **Procurement Note:** Often preferred by highly regulated industries (finance, healthcare). Heavy enterprise pricing, but highly negotiable at the end of Oracle's fiscal quarters.
* **HubSpot (Enterprise Tier):**
* **The Vibe:** The modern, user-friendly alternative.
* **Procurement Note:** HubSpot’s TCO is often significantly lower than Marketo or Salesforce because it requires far fewer external consultants/developers to operate. Marketing teams adopt it faster, meaning higher ROI. Pricing is based on "Marketing Contacts."
### 2. If you are a B2C/E-commerce Company (Focus on High Volume & Personalization)
In B2C, you need systems that can handle millions of sends, mobile push notifications, and SMS.
* **Salesforce Marketing Cloud (formerly ExactTarget):**
* **The Vibe:** The heavyweight champion for B2C cross-channel journeys.
* **Procurement Note:** Extremely complex. Be prepared to budget an additional 50-100% of the software cost for an integration partner (Deloitte, Accenture, or boutique agencies) just to set it up.
* **Braze or Iterable:**
* **The Vibe:** The modern disruptors. Mobile-first, highly agile, beloved by modern engineering and marketing teams.
* **Procurement Note:** Pricing is usually based on Monthly Active Users (MAU) or message volume. They are easier to implement than legacy Salesforce/Adobe products, saving on implementation fees.
* **Adobe Campaign:**
* **The Vibe:** Best if you are already heavily invested in the Adobe Experience Cloud (AEM, Adobe Analytics).
* **Procurement Note:** Only buy this if you are consolidating an Adobe tech stack; otherwise, it is too heavy and expensive as a standalone tool.
---
### 3. The Procurement Strategy: How to Evaluate Them
When your marketing team brings you their preferred vendor, here are the levers you should pull during evaluation and negotiation:
**1. Identify "Shadow IT" and Rogue Spending**
In a 5k-person enterprise, I guarantee various departments have bought rogue instances of Mailchimp, Constant Contact, or SendGrid on corporate credit cards. Use this migration to mandate consolidation. Audit your expenses, kill the rogue tools, and use that budget to fund the enterprise platform.
**2. Watch the Pricing Metric Traps**
* *Contact-based pricing:* (e.g., HubSpot, Marketo). You will be charged for every email address in the database, even if you don't email them. **Negotiate a "Marketing Contacts only" or active-contact clause.**
* *Volume-based pricing:* (e.g., Braze, SFMC). You are charged by the number of emails/messages sent. If marketing scales up successfully, your costs will skyrocket. **Negotiate tiered volume discounts upfront.**
**3. Factor in the "Hidden TCO" (Implementation & Services)**
Do not look just at the SaaS license. A $150k Marketo license might require a $100k implementation agency and a $120k/year certified Marketo architect on staff. Ask vendors to provide a realistic estimate of required professional services.
**4. Leverage Your Existing Stack**
If your 5,000-person company uses Salesforce, Microsoft Dynamics, or Snowflake, prioritize platforms that have native integrations. Custom-building APIs between an email platform and an enterprise ERP/CRM will cost you hundreds of thousands of dollars in IT resources.
### Your Next Step:
Schedule a 30-minute alignment meeting with your CMO and ask three questions:
1. *"Are we buying this for B2B lead generation, or B2C customer retention?"*
2. *"What CRM must this integrate with natively?"*
3. *"Do we have the internal talent to run a complex system, or do we need something with high usability?"*
Once you have those answers, you can narrow the RFP down to the **Salesforce/Adobe** bucket (for legacy enterprise power) or the **HubSpot/Braze** bucket (for modern usability and lower TCO).

End of response · 63 lines · 5,451 characters · sha256 28d3e800e94a9e83663e4d934eb59b881eeee69fa69715a94e69b73a9f8f6aea

After this run

2026-09-27 permalink

Scored
answered
Received
2026-09-27 09:25:27 UTC
Run
20260927T090007Z-7adb14
Served by
Google · HTTP 200 · finish stop
Size
5,856 characters · 2,355 tokens out incl. hidden reasoning · 21.7 s
Extracted picks, in order · confidence high
  1. Align with the existing CRM / Data Warehouse
  2. Scrutinize the Pricing Model
  3. Security

Highlighted: the names the extractor took as picks, where they appear verbatim (3 of 3).

As a VP of Procurement at a 5,000-person enterprise, your lens on this purchase is different from your CMO’s. While Marketing cares about drag-and-drop builders and personalization, **you care about Total Cost of Ownership (TCO), tech stack consolidation, data security (SOC 2, CCPA, GDPR), and vendor viability.**
At your scale, you are no longer looking for an "email marketing platform" (like Mailchimp or Constant Contact). You are looking for an **Enterprise Marketing Automation** or **Customer Engagement Platform**.
To give you the best recommendation, the first question you must ask your marketing team is: **Are we primarily B2B or B2C?** The architecture and pricing models for these two paths are vastly different.
Here is the shortlist of platforms you should evaluate, categorized by your company's business model, followed by a procurement-specific evaluation guide.
---
### 1. If Your Company is B2B (Long sales cycles, lead generation, account-based marketing)
In B2B, email marketing must integrate flawlessly with your CRM (likely Salesforce or Microsoft Dynamics). Pricing is typically based on the **number of contacts** in the database.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot):**
* **Best for:** Companies already using Salesforce as their CRM.
* **Procurement view:** The native integration saves hundreds of thousands of dollars in custom API development and IT maintenance. However, Salesforce renewals are notorious for aggressive price hikes.
* **Adobe Marketo Engage:**
* **Best for:** Highly complex, global B2B marketing teams with massive databases and advanced lead-scoring needs.
* **Procurement view:** Marketo is the gold standard for B2B, but implementation is expensive. You will likely need to budget for a third-party implementation agency, which can double your first-year TCO.
* **HubSpot (Enterprise Hub):**
* **Best for:** Ease of use and rapid adoption. If your marketing team complains that your current tools are too hard to use, HubSpot is the antidote.
* **Procurement view:** Excellent TCO when considering implementation and training (which are cheaper than Adobe or Salesforce). However, their contact-based pricing scales aggressively as your database grows.
### 2. If Your Company is B2C (E-commerce, retail, apps, high-volume consumer emails)
In B2C, you are dealing with millions of emails, SMS, and push notifications. Pricing is usually based on **message volume (CPM)** or **Monthly Active Users (MAU)**.
* **Salesforce Marketing Cloud (formerly ExactTarget):**
* **Best for:** Omnichannel consumer journeys (Email, SMS, App push) connected to complex data architectures.
* **Procurement view:** Extremely powerful, but highly fragmented (you buy "builders" piecemeal—Email Studio, Mobile Studio, etc.). Negotiation requires a very close reading of the contract regarding data storage limits and API call limits, which result in surprise overages.
* **Braze or Iterable:**
* **Best for:** Modern tech, SaaS, or retail companies that rely heavily on mobile apps + email.
* **Procurement view:** These are the modern disruptors eating into Salesforce and Adobe’s market share. They are highly agile, often have lower implementation costs, and feature transparent MAU (Monthly Active User) pricing.
* **Klaviyo (Enterprise):**
* **Best for:** E-commerce (especially if you use Shopify Plus).
* **Procurement view:** Highly specialized for retail/e-commerce. It replaces multiple tools (email, SMS, reviews) which allows for great vendor consolidation.
---
### The Procurement Strategy: How to evaluate and negotiate
When you take these options to your marketing and IT leaders, guide the evaluation using these procurement priorities:
**1. Align with the existing CRM / Data Warehouse**
Do not let Marketing buy a tool in a silo. If you use Salesforce CRM, lean toward Salesforce Marketing Cloud. If you use Adobe Experience Manager, lean toward Marketo. If your data lives in Snowflake, look at Braze. **Native integrations save IT resources and prevent messy middleware costs.**
**2. Scrutinize the Pricing Model (The "Gotchas")**
* **Contact-based (HubSpot/Marketo):** Ensure you are only paying for *marketable* contacts, not every single contact in your CRM (like unsubscribes or bounced emails).
* **Volume-based (Salesforce/Braze):** Negotiate your CPM (cost per mille/thousand sends) upfront, but also pre-negotiate the overage rates.
**3. Budget for Implementation (The Hidden TCO)**
At a 5,000-person enterprise, you do not just "turn on" an email platform. You will need a System Integrator (SI) like Deloitte, Slalom, or a specialized boutique agency to migrate your data, warm up your IP addresses, and build the architecture. **Rule of thumb: Expect year-one implementation costs to be 50% to 150% of your software licensing cost.**
**4. Security, Compliance & Deliverability**
Because you are in the US, the platform must seamlessly handle CAN-SPAM, CCPA (California privacy), and potentially HIPAA if you are in healthcare. You also need Dedicated IP addresses (crucial for a company of your size to maintain email sender reputation) and robust SSO (Single Sign-On) with Role-Based Access Control (RBAC) so a junior marketer can't accidentally email 5 million people.
### Your Next Step
Schedule a 30-minute alignment meeting with your **CMO** and **CIO/CTO**.
Ask them:
1. *Are we looking to optimize B2B lead gen or B2C consumer engagement?*
2. *Where does our primary customer data live today (CRM or Data Warehouse)?*
3. *Are we willing to consolidate other tools (SMS, social posting) into this purchase?*
Once they answer those three questions, you can confidently narrow the list to two of the vendors above and begin the RFP process.

End of response · 63 lines · 5,856 characters · sha256 a48001383dd847d3b60e55422decc89e0eca5df8e6c61e4e913ba3d1cff527c0

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-27 09:00:07 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
6 prior days with data; 1 matched the baseline value (2026-09-26)
Responses
before 2026-09-26 09:24:51 UTC, run 20260926T090007Z-89c0b7; after 2026-09-27 09:25:27 UTC, run 20260927T090007Z-7adb14
Detected
2026-09-27 · run 20260927T090007Z-7adb14, started 2026-09-27 09:00:07 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-27/P29/google/gemini-3.1-pro-preview/pick_change/picks.top/
Event ID
2026-09-27:P29:google/gemini-3.1-pro-preview:pick_change:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.