MODELDRIFT.WATCH

Sep 26 · Day 17

MINOR · GEMINI · QUESTION 33

Gemini named a different first choice for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 25beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot
  4. Braze
  5. Klaviyo

As a VP of Procurement at a 5,000-employee enterprise, your evaluation criteria are fundamentally different from your CMO’s. While Marketing is looking at feature sets, UI, and campaign capabilities, you are looking at Total Cost of Ownership (TCO), tech stack consolidation, security/compliance, scalability, and pricing leverage.

At your scale, you have outgrown SMB tools like standard Mailchimp or Constant Contact. The right platform depends entirely on your company’s business model and your existing CRM.

Here is the executive breakdown of the platforms you should consider, categorized by use case, along with the procurement angles you need to watch out for.

1. If You Are a B2B Company (Complex Sales Cycles, Lead Scoring)

If your company sells to other businesses, your marketing team needs a platform that handles lead nurturing, account-based marketing (ABM), and tight CRM integration.

  • •Adobe Marketo Engage
  • •The Vibe: The gold standard for enterprise B2B marketing. Highly customizable and powerful.
  • •Procurement Angle: Marketo is notorious for requiring highly specialized (and expensive) certified administrators. When calculating TCO, factor in the cost of talent or an outside agency. Adobe is flexible on discounting at the end of their quarters (specifically late November).
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot)
  • •The Vibe: The default choice if your enterprise runs on Salesforce CRM.
  • •Procurement Angle: If you are already spending heavily with Salesforce, this is your best play for tech stack consolidation and volume discounting. Avoid buying this if you use a non-Salesforce CRM (like Microsoft Dynamics); the integration headaches aren't worth it.
  • •HubSpot (Enterprise Tier)
  • •The Vibe: The most user-friendly platform. It has the fastest "time-to-value" and lowest implementation costs.
  • •Procurement Angle: HubSpot prices based on "Marketing Contacts." At an enterprise scale, this can spiral out of control if Marketing doesn't practice good data hygiene. Negotiate aggressive volume tiers for contacts upfront.

2. If You Are a B2C Company (High-Volume, Consumer-Facing)

If you are sending millions of emails to consumers, you need a platform built for massive scale, SMS integration, and real-time behavioral triggers.

  • •Salesforce Marketing Cloud (formerly ExactTarget)
  • •The Vibe: A massive, powerful, highly complex beast used by Fortune 500 B2C brands.
  • •Procurement Angle: Implementation costs can easily equal or exceed year-one licensing costs. You must budget for a systems integrator (like Deloitte, Slalom, or specialized agencies).
  • •Braze or Iterable
  • •The Vibe: The modern, agile competitors to Salesforce. They are API-first and excel at cross-channel (Email, SMS, In-App, Push notifications).
  • •Procurement Angle: Pricing is often based on Monthly Active Users (MAUs) or data events rather than just the number of emails sent. You need your marketing and data teams to accurately forecast API calls and data points, or you will get hit with massive overage fees.

3. If You Are E-Commerce Focused

  • •Klaviyo (Enterprise)
  • •The Vibe: Unmatched integration with Shopify Plus and other enterprise e-commerce platforms.
  • •Procurement Angle: Very straightforward ROI tracking, which makes it easy to justify the cost, but their enterprise pricing requires aggressive negotiation as volume scales.

The "Procurement Lens" Checklist for Your RFP

Before you issue an RFP or sign a contract, bring these points to your CMO or VP of Marketing:

  • 1.The "SSO Tax": Many vendors charge extra for SAML/SSO (Single Sign-On) and advanced role-based access control. At 5,000 employees, your CISO will mandate SSO. Ensure this is included in the base platform price, not as a costly add-on.
  • 2.Dedicated IP vs. Shared IP: At your size, you must have dedicated IP addresses to protect your email deliverability reputation. Ensure the cost of IP warm-up and dedicated IPs is included.
  • 3.Data Privacy & Residency: Ensure the vendor includes DPA (Data Processing Agreements) that cover CCPA, GDPR, and whatever industry-specific compliance you need (e.g., HIPAA for healthcare, GLBA for finance).
  • 4.Implementation & Support: Enterprise software is rarely plug-and-play. Ask the vendor to itemize premium support (SLAs with 1-hour response times) and onboarding costs.

Your Next Step

Do not evaluate platforms in a vacuum. Ask your Marketing leader two questions:

  • 1."What is our primary CRM (Salesforce, Dynamics, custom)?" (Your marketing platform must natively speak to this).
  • 2."How are we measuring pricing: by number of contacts in our database, or by the volume of emails/messages we send?"

Once you have those answers, invite two legacy vendors (e.g., Salesforce/Adobe) and one modern challenger (e.g., Braze/HubSpot) to present.

SEP 26afterPermalink

Picks read from the answer, in order
  1. Identify "Shadow IT" and Rogue Spending
  2. Watch the Pricing Metric Traps
  3. Factor in the "Hidden TCO
  4. Leverage Your Existing Stack

As a VP of Procurement at a 5,000-person enterprise, your lens for selecting an email marketing platform is very different from your CMO’s. While Marketing cares about UI, templates, and campaign builders, you care about Total Cost of Ownership (TCO), tech stack consolidation, data security (CCPA/GDPR), CRM integration, and implementation costs.

At the 5,000-employee mark, you are well past SMB tools like Mailchimp. You need an Enterprise Marketing Automation Platform (MAP).

To give you the best recommendation, the choice splits entirely on your company’s business model (B2B vs. B2C) and your existing CRM infrastructure. Here is the executive breakdown of the platforms you should be taking to RFP.

1. If you are a B2B Company (Focus on Lead Nurturing & Sales Alignment)

In B2B, the email platform must integrate flawlessly with your CRM.

  • •Adobe Marketo Engage:
  • •The Vibe: The gold standard for complex B2B enterprises.
  • •Procurement Note: High licensing cost and usually requires expensive third-party agencies to implement and run. Pricing is based on database size.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot):
  • •The Vibe: The default choice if your enterprise runs on Salesforce Sales Cloud.
  • •Procurement Note: Great opportunity for vendor consolidation and negotiating a bundled master service agreement (MSA) if you already use Salesforce.
  • •Oracle Eloqua:
  • •The Vibe: Highly secure, rigid, and handles massive, complex data architectures.
  • •Procurement Note: Often preferred by highly regulated industries (finance, healthcare). Heavy enterprise pricing, but highly negotiable at the end of Oracle's fiscal quarters.
  • •HubSpot (Enterprise Tier):
  • •The Vibe: The modern, user-friendly alternative.
  • •Procurement Note: HubSpot’s TCO is often significantly lower than Marketo or Salesforce because it requires far fewer external consultants/developers to operate. Marketing teams adopt it faster, meaning higher ROI. Pricing is based on "Marketing Contacts."

2. If you are a B2C/E-commerce Company (Focus on High Volume & Personalization)

In B2C, you need systems that can handle millions of sends, mobile push notifications, and SMS.

  • •Salesforce Marketing Cloud (formerly ExactTarget):
  • •The Vibe: The heavyweight champion for B2C cross-channel journeys.
  • •Procurement Note: Extremely complex. Be prepared to budget an additional 50-100% of the software cost for an integration partner (Deloitte, Accenture, or boutique agencies) just to set it up.
  • •Braze or Iterable:
  • •The Vibe: The modern disruptors. Mobile-first, highly agile, beloved by modern engineering and marketing teams.
  • •Procurement Note: Pricing is usually based on Monthly Active Users (MAU) or message volume. They are easier to implement than legacy Salesforce/Adobe products, saving on implementation fees.
  • •Adobe Campaign:
  • •The Vibe: Best if you are already heavily invested in the Adobe Experience Cloud (AEM, Adobe Analytics).
  • •Procurement Note: Only buy this if you are consolidating an Adobe tech stack; otherwise, it is too heavy and expensive as a standalone tool.

3. The Procurement Strategy: How to Evaluate Them

When your marketing team brings you their preferred vendor, here are the levers you should pull during evaluation and negotiation:

1. Identify "Shadow IT" and Rogue Spending
In a 5k-person enterprise, I guarantee various departments have bought rogue instances of Mailchimp, Constant Contact, or SendGrid on corporate credit cards. Use this migration to mandate consolidation. Audit your expenses, kill the rogue tools, and use that budget to fund the enterprise platform.

2. Watch the Pricing Metric Traps

  • •Contact-based pricing: (e.g., HubSpot, Marketo). You will be charged for every email address in the database, even if you don't email them. Negotiate a "Marketing Contacts only" or active-contact clause.
  • •Volume-based pricing: (e.g., Braze, SFMC). You are charged by the number of emails/messages sent. If marketing scales up successfully, your costs will skyrocket. Negotiate tiered volume discounts upfront.

3. Factor in the "Hidden TCO" (Implementation & Services)
Do not look just at the SaaS license. A $150k Marketo license might require a $100k implementation agency and a $120k/year certified Marketo architect on staff. Ask vendors to provide a realistic estimate of required professional services.

4. Leverage Your Existing Stack
If your 5,000-person company uses Salesforce, Microsoft Dynamics, or Snowflake, prioritize platforms that have native integrations. Custom-building APIs between an email platform and an enterprise ERP/CRM will cost you hundreds of thousands of dollars in IT resources.

Your Next Step:

Schedule a 30-minute alignment meeting with your CMO and ask three questions:

  • 1."Are we buying this for B2B lead generation, or B2C customer retention?"
  • 2."What CRM must this integrate with natively?"
  • 3."Do we have the internal talent to run a complex system, or do we need something with high usability?"

Once you have those answers, you can narrow the RFP down to the Salesforce/Adobe bucket (for legacy enterprise power) or the HubSpot/Braze bucket (for modern usability and lower TCO).

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-25 permalink

Scored
answered
Received
2026-09-25 09:25:46 UTC
Run
20260925T090007Z-f08d1f
Served by
Google · HTTP 200 · finish stop
Size
5,053 characters · 2,412 tokens out incl. hidden reasoning · 22.4 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot
  4. Braze
  5. Klaviyo

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

As a VP of Procurement at a 5,000-employee enterprise, your evaluation criteria are fundamentally different from your CMO’s. While Marketing is looking at feature sets, UI, and campaign capabilities, you are looking at **Total Cost of Ownership (TCO), tech stack consolidation, security/compliance, scalability, and pricing leverage.**
At your scale, you have outgrown SMB tools like standard Mailchimp or Constant Contact. The right platform depends entirely on **your company’s business model** and **your existing CRM**.
Here is the executive breakdown of the platforms you should consider, categorized by use case, along with the procurement angles you need to watch out for.
---
### 1. If You Are a B2B Company (Complex Sales Cycles, Lead Scoring)
If your company sells to other businesses, your marketing team needs a platform that handles lead nurturing, account-based marketing (ABM), and tight CRM integration.
* **Adobe Marketo Engage**
* **The Vibe:** The gold standard for enterprise B2B marketing. Highly customizable and powerful.
* **Procurement Angle:** Marketo is notorious for requiring highly specialized (and expensive) certified administrators. When calculating TCO, factor in the cost of talent or an outside agency. Adobe is flexible on discounting at the end of their quarters (specifically late November).
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot)**
* **The Vibe:** The default choice if your enterprise runs on Salesforce CRM.
* **Procurement Angle:** If you are already spending heavily with Salesforce, this is your best play for tech stack consolidation and volume discounting. Avoid buying this if you use a non-Salesforce CRM (like Microsoft Dynamics); the integration headaches aren't worth it.
* **HubSpot (Enterprise Tier)**
* **The Vibe:** The most user-friendly platform. It has the fastest "time-to-value" and lowest implementation costs.
* **Procurement Angle:** HubSpot prices based on "Marketing Contacts." At an enterprise scale, this can spiral out of control if Marketing doesn't practice good data hygiene. Negotiate aggressive volume tiers for contacts upfront.
### 2. If You Are a B2C Company (High-Volume, Consumer-Facing)
If you are sending millions of emails to consumers, you need a platform built for massive scale, SMS integration, and real-time behavioral triggers.
* **Salesforce Marketing Cloud (formerly ExactTarget)**
* **The Vibe:** A massive, powerful, highly complex beast used by Fortune 500 B2C brands.
* **Procurement Angle:** Implementation costs can easily equal or exceed year-one licensing costs. You *must* budget for a systems integrator (like Deloitte, Slalom, or specialized agencies).
* **Braze** or **Iterable**
* **The Vibe:** The modern, agile competitors to Salesforce. They are API-first and excel at cross-channel (Email, SMS, In-App, Push notifications).
* **Procurement Angle:** Pricing is often based on Monthly Active Users (MAUs) or data events rather than just the number of emails sent. You need your marketing and data teams to accurately forecast API calls and data points, or you will get hit with massive overage fees.
### 3. If You Are E-Commerce Focused
* **Klaviyo (Enterprise)**
* **The Vibe:** Unmatched integration with Shopify Plus and other enterprise e-commerce platforms.
* **Procurement Angle:** Very straightforward ROI tracking, which makes it easy to justify the cost, but their enterprise pricing requires aggressive negotiation as volume scales.
---
### The "Procurement Lens" Checklist for Your RFP
Before you issue an RFP or sign a contract, bring these points to your CMO or VP of Marketing:
1. **The "SSO Tax":** Many vendors charge extra for SAML/SSO (Single Sign-On) and advanced role-based access control. At 5,000 employees, your CISO will mandate SSO. Ensure this is included in the base platform price, not as a costly add-on.
2. **Dedicated IP vs. Shared IP:** At your size, you must have dedicated IP addresses to protect your email deliverability reputation. Ensure the cost of IP warm-up and dedicated IPs is included.
3. **Data Privacy & Residency:** Ensure the vendor includes DPA (Data Processing Agreements) that cover CCPA, GDPR, and whatever industry-specific compliance you need (e.g., HIPAA for healthcare, GLBA for finance).
4. **Implementation & Support:** Enterprise software is rarely plug-and-play. Ask the vendor to itemize premium support (SLAs with 1-hour response times) and onboarding costs.
### Your Next Step
Do not evaluate platforms in a vacuum. Ask your Marketing leader two questions:
1. *"What is our primary CRM (Salesforce, Dynamics, custom)?"* (Your marketing platform **must** natively speak to this).
2. *"How are we measuring pricing: by number of contacts in our database, or by the volume of emails/messages we send?"*
Once you have those answers, invite **two legacy vendors** (e.g., Salesforce/Adobe) and **one modern challenger** (e.g., Braze/HubSpot) to present.

End of response · 53 lines · 5,053 characters · sha256 11bb475f8bff150ced5126b973de0846a4f391bcbf5aff20f4dd14f2ff1fa957

After this run

2026-09-26 permalink

Scored
answered
Received
2026-09-26 09:24:51 UTC
Run
20260926T090007Z-89c0b7
Served by
Google · HTTP 200 · finish stop
Size
5,451 characters · 2,377 tokens out incl. hidden reasoning · 21.8 s
Extracted picks, in order · confidence high
  1. Identify "Shadow IT" and Rogue Spending
  2. Watch the Pricing Metric Traps
  3. Factor in the "Hidden TCO
  4. Leverage Your Existing Stack

Highlighted: the names the extractor took as picks, where they appear verbatim (4 of 4).

As a VP of Procurement at a 5,000-person enterprise, your lens for selecting an email marketing platform is very different from your CMO’s. While Marketing cares about UI, templates, and campaign builders, **you care about Total Cost of Ownership (TCO), tech stack consolidation, data security (CCPA/GDPR), CRM integration, and implementation costs.**
At the 5,000-employee mark, you are well past SMB tools like Mailchimp. You need an **Enterprise Marketing Automation Platform (MAP)**.
To give you the best recommendation, the choice splits entirely on your company’s business model (B2B vs. B2C) and your existing CRM infrastructure. Here is the executive breakdown of the platforms you should be taking to RFP.
---
### 1. If you are a B2B Company (Focus on Lead Nurturing & Sales Alignment)
In B2B, the email platform must integrate flawlessly with your CRM.
* **Adobe Marketo Engage:**
* **The Vibe:** The gold standard for complex B2B enterprises.
* **Procurement Note:** High licensing cost and usually requires expensive third-party agencies to implement and run. Pricing is based on database size.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot):**
* **The Vibe:** The default choice if your enterprise runs on Salesforce Sales Cloud.
* **Procurement Note:** Great opportunity for vendor consolidation and negotiating a bundled master service agreement (MSA) if you already use Salesforce.
* **Oracle Eloqua:**
* **The Vibe:** Highly secure, rigid, and handles massive, complex data architectures.
* **Procurement Note:** Often preferred by highly regulated industries (finance, healthcare). Heavy enterprise pricing, but highly negotiable at the end of Oracle's fiscal quarters.
* **HubSpot (Enterprise Tier):**
* **The Vibe:** The modern, user-friendly alternative.
* **Procurement Note:** HubSpot’s TCO is often significantly lower than Marketo or Salesforce because it requires far fewer external consultants/developers to operate. Marketing teams adopt it faster, meaning higher ROI. Pricing is based on "Marketing Contacts."
### 2. If you are a B2C/E-commerce Company (Focus on High Volume & Personalization)
In B2C, you need systems that can handle millions of sends, mobile push notifications, and SMS.
* **Salesforce Marketing Cloud (formerly ExactTarget):**
* **The Vibe:** The heavyweight champion for B2C cross-channel journeys.
* **Procurement Note:** Extremely complex. Be prepared to budget an additional 50-100% of the software cost for an integration partner (Deloitte, Accenture, or boutique agencies) just to set it up.
* **Braze or Iterable:**
* **The Vibe:** The modern disruptors. Mobile-first, highly agile, beloved by modern engineering and marketing teams.
* **Procurement Note:** Pricing is usually based on Monthly Active Users (MAU) or message volume. They are easier to implement than legacy Salesforce/Adobe products, saving on implementation fees.
* **Adobe Campaign:**
* **The Vibe:** Best if you are already heavily invested in the Adobe Experience Cloud (AEM, Adobe Analytics).
* **Procurement Note:** Only buy this if you are consolidating an Adobe tech stack; otherwise, it is too heavy and expensive as a standalone tool.
---
### 3. The Procurement Strategy: How to Evaluate Them
When your marketing team brings you their preferred vendor, here are the levers you should pull during evaluation and negotiation:
**1. Identify "Shadow IT" and Rogue Spending**
In a 5k-person enterprise, I guarantee various departments have bought rogue instances of Mailchimp, Constant Contact, or SendGrid on corporate credit cards. Use this migration to mandate consolidation. Audit your expenses, kill the rogue tools, and use that budget to fund the enterprise platform.
**2. Watch the Pricing Metric Traps**
* *Contact-based pricing:* (e.g., HubSpot, Marketo). You will be charged for every email address in the database, even if you don't email them. **Negotiate a "Marketing Contacts only" or active-contact clause.**
* *Volume-based pricing:* (e.g., Braze, SFMC). You are charged by the number of emails/messages sent. If marketing scales up successfully, your costs will skyrocket. **Negotiate tiered volume discounts upfront.**
**3. Factor in the "Hidden TCO" (Implementation & Services)**
Do not look just at the SaaS license. A $150k Marketo license might require a $100k implementation agency and a $120k/year certified Marketo architect on staff. Ask vendors to provide a realistic estimate of required professional services.
**4. Leverage Your Existing Stack**
If your 5,000-person company uses Salesforce, Microsoft Dynamics, or Snowflake, prioritize platforms that have native integrations. Custom-building APIs between an email platform and an enterprise ERP/CRM will cost you hundreds of thousands of dollars in IT resources.
### Your Next Step:
Schedule a 30-minute alignment meeting with your CMO and ask three questions:
1. *"Are we buying this for B2B lead generation, or B2C customer retention?"*
2. *"What CRM must this integrate with natively?"*
3. *"Do we have the internal talent to run a complex system, or do we need something with high usability?"*
Once you have those answers, you can narrow the RFP down to the **Salesforce/Adobe** bucket (for legacy enterprise power) or the **HubSpot/Braze** bucket (for modern usability and lower TCO).

End of response · 63 lines · 5,451 characters · sha256 28d3e800e94a9e83663e4d934eb59b881eeee69fa69715a94e69b73a9f8f6aea

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-26 09:00:06 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
6 prior days with data; 1 matched the baseline value (2026-09-25)
Responses
before 2026-09-25 09:25:46 UTC, run 20260925T090007Z-f08d1f; after 2026-09-26 09:24:51 UTC, run 20260926T090007Z-89c0b7
Detected
2026-09-26 · run 20260926T090007Z-89c0b7, started 2026-09-26 09:00:07 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-26/P29/google/gemini-3.1-pro-preview/pick_change/picks.top/
Event ID
2026-09-26:P29:google/gemini-3.1-pro-preview:pick_change:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.