MODELDRIFT.WATCH

Sep 22 · Day 13

MINOR · GEMINI · QUESTION 33

Gemini named a different first choice for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 18beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Braze
  4. Fast Time-to-Value" Enterprise Contender
  5. HubSpot

As a VP of Procurement for a 5,000-employee enterprise, your evaluation of an email marketing platform will be fundamentally different from that of a CMO. While Marketing is looking at features (A/B testing, AI copy generation, templates), your focus is on Total Cost of Ownership (TCO), scalability, security/compliance, vendor consolidation, and integration capability.

At your size, "email marketing" usually falls into two distinct categories. The right tool depends heavily on your company's business model.

Here is a breakdown of the enterprise-grade platforms you should consider for your RFP, categorized by business model and viewed through a procurement lens.

1. If You Are a B2B Company (Long Sales Cycles, Lead Scoring)

If your 5,000 employees are selling software, manufacturing, consulting, or medical devices, you need a "Marketing Automation Platform" (MAP).

  • •Adobe Marketo Engage
  • •The Procurement View: Marketo is the gold standard for enterprise B2B. It handles complex lead scoring and routing seamlessly.
  • •Cost Structure: Priced by database size (number of contacts), not email volume.
  • •Vendor Risk/TCO: High implementation costs. You will likely need to hire a specialized agency or dedicated internal Marketo architect. However, it is highly stable and integrates deeply with Salesforce CRM.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot)
  • •The Procurement View: If your enterprise already uses Salesforce as its CRM, this is usually the easiest procurement win. Consolidating under one Master Services Agreement (MSA) with Salesforce can give you leverage for volume discounting.
  • •Cost Structure: Tiered user licensing and database size.
  • •Vendor Risk/TCO: Predictable. IT and Legal usually approve it quickly because Salesforce’s security and compliance architecture is already vetted.

2. If You Are a B2C Company (High Volume, E-commerce, Retail)

If you are selling to millions of consumers, you need a platform built for massive scale, transactional emails, and cross-channel journeys (SMS, push notifications).

  • •Salesforce Marketing Cloud (SFMC - formerly ExactTarget)
  • •The Procurement View: An absolute powerhouse for B2C. It can handle billions of sends.
  • •Cost Structure: Highly modular. You pay for "studios" (Email Studio, Mobile Studio) and utilization (Super Messages).
  • •Vendor Risk/TCO: Warning on TCO. The license cost is only the beginning. Implementation is notoriously complex, and you should budget at least 1x to 1.5x the software cost for implementation and ongoing managed services.
  • •Braze or Iterable
  • •The Procurement View: These are the modern, agile alternatives to legacy Salesforce/Adobe. They are built for modern data architectures (e.g., pulling directly from Snowflake or AWS).
  • •Cost Structure: Usually based on Monthly Active Users (MAUs) and message volume.
  • •Vendor Risk/TCO: They require a high level of data maturity. If your enterprise data is messy, these platforms will struggle. However, they boast much faster deployment times than Salesforce or Adobe, leading to quicker ROI.

3. The "Fast Time-to-Value" Enterprise Contender

  • •HubSpot (Enterprise Hub)
  • •The Procurement View: Historically seen as an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies, particularly if you want to replace multiple tools (CRM, Email, CMS, Service) with one vendor.
  • •Cost Structure: Marketing contacts + seat licenses. Very transparent pricing compared to Adobe and Salesforce.
  • •Vendor Risk/TCO: The lowest TCO on this list. It is incredibly user-friendly, meaning marketing teams actually adopt it without requiring expensive external consultants or specialized IT support.

The Procurement Evaluation Checklist (What to demand in the RFP)

When you take this to market, do not let Marketing drive the selection based purely on UI. Ensure your RFP mandates the following:

  • 1.Security & Compliance: Since you are a US enterprise, demand proof of SOC 2 Type II compliance, strict CCPA/CPRA data deletion workflows, and HIPAA compliance (if you touch healthcare).
  • 2.Governance & Access: At 5,000 employees, you will likely have dozens of marketers across different regions or business units. The tool must have robust Role-Based Access Control (RBAC) and Single Sign-On (SSO/SAML) capabilities.
  • 3.Hidden Overage Fees: Pay close attention to how they charge for overages. Are you billed for "contacts in the database" (even if you don't email them) or "sends"? If Marketing runs a massive holiday campaign, will you get hit with a 20% penalty fee for exceeding your sending limits?
  • 4.Dedicated IP and Deliverability: At enterprise scale, you must send from dedicated IP addresses. Demand guarantees around deliverability support and IP warming as part of the onboarding SLA.

Your Immediate Next Steps

  • 1.Define the Tech Stack: Ask your CIO/CTO what your core CRM and Data Warehouse are (e.g., Salesforce, Microsoft Dynamics, Snowflake). The email platform must have native integrations with these, or your integration costs will skyrocket.
  • 2.Align with the CMO: Are they optimizing for B2B lead generation, or B2C mass-market personalization?
  • 3.Shortlist 3 Vendors: Based on the above, typically a standard enterprise RFP looks like: Salesforce vs. Adobe vs. HubSpot (for B2B/Hybrid) OR Salesforce vs. Braze vs. Iterable (for B2C).

SEP 22afterPermalink

Picks read from the answer, in order
  1. Salesforce Marketing Cloud Account Engagement
  2. Adobe Marketo Engage
  3. HubSpot
  4. Klaviyo
  5. Modern
  6. Braze

As a VP of Procurement at a 5,000-person enterprise, your lens is different from the Chief Marketing Officer's. While the CMO wants features, automation, and ease of use, you are evaluating Total Cost of Ownership (TCO), scalability, data security (SOC 2/CCPA), vendor viability, and integration into your existing tech stack.

At the 5,000-employee mark, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an enterprise-grade platform.

However, the "right" platform depends entirely on your company's business model. Here is a breakdown of the top enterprise platforms categorized by use case, along with the procurement considerations for each.

1. If you are a B2B Enterprise (Complex Sales Cycles, Lead Scoring)

In B2B, email marketing is about nurturing leads, integrating deeply with a CRM, and routing qualified leads to a sales team.

  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot)
  • •The Vibe: The gold standard if your company already uses Salesforce CRM.
  • •Procurement Lens: It offers seamless integration with Salesforce, reducing middleware costs. However, Salesforce pricing is notoriously rigid, and implementation almost always requires a third-party SI (Systems Integrator). Be prepared to negotiate hard on renewal caps.
  • •Adobe Marketo Engage
  • •The Vibe: The most powerful B2B marketing automation and lead-scoring tool on the market.
  • •Procurement Lens: Marketo is an enterprise workhorse but requires dedicated, certified staff to run it. When evaluating TCO, factor in the cost of hiring Marketo-certified admins (they are expensive) and a lengthy implementation timeline.
  • •HubSpot (Enterprise Tier)
  • •The Vibe: The easiest to use. Marketing teams love it because of its intuitive UI and fast time-to-value.
  • •Procurement Lens: Implementation costs are significantly lower than Salesforce or Adobe. However, HubSpot prices based on "Marketing Contacts." At an enterprise scale, this can result in explosive cost increases if your marketing team isn't meticulous about archiving old contacts.

2. If you are a B2C / Retail / eCommerce Enterprise (High Volume, Transactional)

In B2C, email marketing is about high-volume sends, personalization, cart abandonment, and cross-channel messaging (SMS + Email).

  • •Salesforce Marketing Cloud (SFMC - ExactTarget)
  • •The Vibe: The traditional heavy hitter for massive B2C sends.
  • •Procurement Lens: Highly secure, scalable, and stable. But it is notoriously disjointed on the back end (it’s a Frankenstein of acquisitions). TCO is high due to the need for specialized developers (AMPScript) and expensive modular add-ons.
  • •Klaviyo (Enterprise Tier)
  • •The Vibe: The undisputed king of eCommerce email marketing. If your company sells products online, Marketing will likely ask for this.
  • •Procurement Lens: Fast deployment and incredible ROI for retail. Pricing is very transparent but scales directly with list size and email/SMS volume. Ensure your contract includes volume tier discounts as you grow.

3. The Modern, "Data-First" Alternatives (B2C or Tech/SaaS Companies)

If your enterprise has a massive mobile app presence or relies heavily on a modern data warehouse (like Snowflake or Databricks), traditional platforms might be too slow.

  • •Braze or Iterable
  • •The Vibe: These are modern, cross-channel customer engagement platforms. They handle email, push notifications, SMS, and in-app messaging flawlessly.
  • •Procurement Lens: These are generally more flexible and have better APIs than legacy Adobe/Salesforce products. They price based on "Monthly Active Users" (MAUs) or data points. Warning: They require a mature data infrastructure. If you buy Braze, you might also need to buy a Customer Data Platform (CDP) like Segment or mParticle to feed it data.

The VP of Procurement’s Checklist for Evaluating these Platforms

Before running an RFP or signing a contract, sit down with your CMO and CIO and align on the following:

  • 1.Pricing Model Traps: Are we paying by Volume (number of emails sent), by Database Size (number of contacts stored), or by Active Users? Ensure the pricing model aligns with your growth strategy. (e.g., If you have a massive database but only email them rarely, do not sign a contract that charges by database size).
  • 2.The "Hidden" Implementation Costs: Salesforce and Adobe rarely work out of the box. You will likely need to hire a digital agency (Deloitte, Slalom, or a boutique firm) for 3–6 months to set it up. Factor a $50k–$200k implementation cost into the Year 1 TCO.
  • 3.Tech Stack Consolidation: What are we already paying for? If you have Salesforce CRM, look at Salesforce Marketing Cloud. If you have Adobe Experience Manager, look at Adobe Campaign. The bundled discounts and native integrations will save you money and IT headaches.
  • 4.Deliverability & Compliance: Being in the US, ensure the vendor has robust, automated tools for CAN-SPAM and CCPA compliance. Ask about their dedicated IP warming services.
  • 5.Service Level Agreements (SLAs): Email is often revenue-generating. If the platform goes down on Black Friday (for B2C) or during a major product launch, what are the financial penalties in the contract?

My Recommendation on Next Steps:
Ask your CMO one question: "What is our primary goal: aligning with Sales on complex B2B leads, or driving high-volume B2C consumer transactions?"

If B2B, put Marketo, Pardot, and HubSpot on your RFP shortlist.
If B2C/App, put Braze, Iterable, and Salesforce Marketing Cloud on your shortlist.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:31:35 UTC
Run
20260918T090022Z-92b6f7
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,651 characters · 2,473 tokens out incl. hidden reasoning · 35.6 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Braze
  4. Fast Time-to-Value" Enterprise Contender
  5. HubSpot

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

As a VP of Procurement for a 5,000-employee enterprise, your evaluation of an email marketing platform will be fundamentally different from that of a CMO. While Marketing is looking at features (A/B testing, AI copy generation, templates), your focus is on **Total Cost of Ownership (TCO), scalability, security/compliance, vendor consolidation, and integration capability.**
At your size, "email marketing" usually falls into two distinct categories. The right tool depends heavily on your company's business model.
Here is a breakdown of the enterprise-grade platforms you should consider for your RFP, categorized by business model and viewed through a procurement lens.
---
### 1. If You Are a B2B Company (Long Sales Cycles, Lead Scoring)
*If your 5,000 employees are selling software, manufacturing, consulting, or medical devices, you need a "Marketing Automation Platform" (MAP).*
* **Adobe Marketo Engage**
* **The Procurement View:** Marketo is the gold standard for enterprise B2B. It handles complex lead scoring and routing seamlessly.
* **Cost Structure:** Priced by database size (number of contacts), not email volume.
* **Vendor Risk/TCO:** High implementation costs. You will likely need to hire a specialized agency or dedicated internal Marketo architect. However, it is highly stable and integrates deeply with Salesforce CRM.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot)**
* **The Procurement View:** If your enterprise already uses Salesforce as its CRM, this is usually the easiest procurement win. Consolidating under one Master Services Agreement (MSA) with Salesforce can give you leverage for volume discounting.
* **Cost Structure:** Tiered user licensing and database size.
* **Vendor Risk/TCO:** Predictable. IT and Legal usually approve it quickly because Salesforce’s security and compliance architecture is already vetted.
### 2. If You Are a B2C Company (High Volume, E-commerce, Retail)
*If you are selling to millions of consumers, you need a platform built for massive scale, transactional emails, and cross-channel journeys (SMS, push notifications).*
* **Salesforce Marketing Cloud (SFMC - formerly ExactTarget)**
* **The Procurement View:** An absolute powerhouse for B2C. It can handle billions of sends.
* **Cost Structure:** Highly modular. You pay for "studios" (Email Studio, Mobile Studio) and utilization (Super Messages).
* **Vendor Risk/TCO:** **Warning on TCO.** The license cost is only the beginning. Implementation is notoriously complex, and you should budget at least 1x to 1.5x the software cost for implementation and ongoing managed services.
* **Braze** or **Iterable**
* **The Procurement View:** These are the modern, agile alternatives to legacy Salesforce/Adobe. They are built for modern data architectures (e.g., pulling directly from Snowflake or AWS).
* **Cost Structure:** Usually based on Monthly Active Users (MAUs) and message volume.
* **Vendor Risk/TCO:** They require a high level of data maturity. If your enterprise data is messy, these platforms will struggle. However, they boast much faster deployment times than Salesforce or Adobe, leading to quicker ROI.
### 3. The "Fast Time-to-Value" Enterprise Contender
* **HubSpot (Enterprise Hub)**
* **The Procurement View:** Historically seen as an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies, particularly if you want to replace multiple tools (CRM, Email, CMS, Service) with one vendor.
* **Cost Structure:** Marketing contacts + seat licenses. Very transparent pricing compared to Adobe and Salesforce.
* **Vendor Risk/TCO:** The lowest TCO on this list. It is incredibly user-friendly, meaning marketing teams actually adopt it without requiring expensive external consultants or specialized IT support.
---
### The Procurement Evaluation Checklist (What to demand in the RFP)
When you take this to market, do not let Marketing drive the selection based purely on UI. Ensure your RFP mandates the following:
1. **Security & Compliance:** Since you are a US enterprise, demand proof of **SOC 2 Type II compliance**, strict **CCPA/CPRA** data deletion workflows, and **HIPAA** compliance (if you touch healthcare).
2. **Governance & Access:** At 5,000 employees, you will likely have dozens of marketers across different regions or business units. The tool *must* have robust Role-Based Access Control (RBAC) and Single Sign-On (SSO/SAML) capabilities.
3. **Hidden Overage Fees:** Pay close attention to how they charge for overages. Are you billed for "contacts in the database" (even if you don't email them) or "sends"? If Marketing runs a massive holiday campaign, will you get hit with a 20% penalty fee for exceeding your sending limits?
4. **Dedicated IP and Deliverability:** At enterprise scale, you must send from dedicated IP addresses. Demand guarantees around deliverability support and IP warming as part of the onboarding SLA.
### Your Immediate Next Steps
1. **Define the Tech Stack:** Ask your CIO/CTO what your core CRM and Data Warehouse are (e.g., Salesforce, Microsoft Dynamics, Snowflake). The email platform *must* have native integrations with these, or your integration costs will skyrocket.
2. **Align with the CMO:** Are they optimizing for B2B lead generation, or B2C mass-market personalization?
3. **Shortlist 3 Vendors:** Based on the above, typically a standard enterprise RFP looks like: **Salesforce vs. Adobe vs. HubSpot** (for B2B/Hybrid) OR **Salesforce vs. Braze vs. Iterable** (for B2C).

End of response · 54 lines · 5,651 characters · sha256 db544ecc3e1e8c289de39c0ae7af0fa5556a866e811e94cedcda0919f883e689

After this run

2026-09-22 permalink

Scored
answered
Received
2026-09-22 09:28:39 UTC
Run
20260922T090022Z-7ff13b
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,786 characters · 2,574 tokens out incl. hidden reasoning · 35.1 s
Extracted picks, in order · confidence medium
  1. Salesforce Marketing Cloud Account Engagement
  2. Adobe Marketo Engage
  3. HubSpot
  4. Klaviyo
  5. Modern
  6. Braze

Highlighted: the names the extractor took as picks, where they appear verbatim (6 of 6).

As a VP of Procurement at a 5,000-person enterprise, your lens is different from the Chief Marketing Officer's. While the CMO wants features, automation, and ease of use, you are evaluating **Total Cost of Ownership (TCO), scalability, data security (SOC 2/CCPA), vendor viability, and integration into your existing tech stack.**
At the 5,000-employee mark, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an **enterprise-grade platform**.
However, the "right" platform depends entirely on your company's business model. Here is a breakdown of the top enterprise platforms categorized by use case, along with the procurement considerations for each.
---
### 1. If you are a B2B Enterprise (Complex Sales Cycles, Lead Scoring)
*In B2B, email marketing is about nurturing leads, integrating deeply with a CRM, and routing qualified leads to a sales team.*
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot)**
* **The Vibe:** The gold standard if your company already uses Salesforce CRM.
* **Procurement Lens:** It offers seamless integration with Salesforce, reducing middleware costs. However, Salesforce pricing is notoriously rigid, and implementation almost always requires a third-party SI (Systems Integrator). Be prepared to negotiate hard on renewal caps.
* **Adobe Marketo Engage**
* **The Vibe:** The most powerful B2B marketing automation and lead-scoring tool on the market.
* **Procurement Lens:** Marketo is an enterprise workhorse but requires dedicated, certified staff to run it. When evaluating TCO, factor in the cost of hiring Marketo-certified admins (they are expensive) and a lengthy implementation timeline.
* **HubSpot (Enterprise Tier)**
* **The Vibe:** The easiest to use. Marketing teams love it because of its intuitive UI and fast time-to-value.
* **Procurement Lens:** Implementation costs are significantly lower than Salesforce or Adobe. However, HubSpot prices based on "Marketing Contacts." At an enterprise scale, this can result in explosive cost increases if your marketing team isn't meticulous about archiving old contacts.
### 2. If you are a B2C / Retail / eCommerce Enterprise (High Volume, Transactional)
*In B2C, email marketing is about high-volume sends, personalization, cart abandonment, and cross-channel messaging (SMS + Email).*
* **Salesforce Marketing Cloud (SFMC - ExactTarget)**
* **The Vibe:** The traditional heavy hitter for massive B2C sends.
* **Procurement Lens:** Highly secure, scalable, and stable. But it is notoriously disjointed on the back end (it’s a Frankenstein of acquisitions). TCO is high due to the need for specialized developers (AMPScript) and expensive modular add-ons.
* **Klaviyo (Enterprise Tier)**
* **The Vibe:** The undisputed king of eCommerce email marketing. If your company sells products online, Marketing will likely ask for this.
* **Procurement Lens:** Fast deployment and incredible ROI for retail. Pricing is very transparent but scales directly with list size and email/SMS volume. Ensure your contract includes volume tier discounts as you grow.
### 3. The Modern, "Data-First" Alternatives (B2C or Tech/SaaS Companies)
*If your enterprise has a massive mobile app presence or relies heavily on a modern data warehouse (like Snowflake or Databricks), traditional platforms might be too slow.*
* **Braze** or **Iterable**
* **The Vibe:** These are modern, cross-channel customer engagement platforms. They handle email, push notifications, SMS, and in-app messaging flawlessly.
* **Procurement Lens:** These are generally more flexible and have better APIs than legacy Adobe/Salesforce products. They price based on "Monthly Active Users" (MAUs) or data points. **Warning:** They require a mature data infrastructure. If you buy Braze, you might also need to buy a Customer Data Platform (CDP) like Segment or mParticle to feed it data.
---
### The VP of Procurement’s Checklist for Evaluating these Platforms
Before running an RFP or signing a contract, sit down with your CMO and CIO and align on the following:
1. **Pricing Model Traps:** Are we paying by *Volume* (number of emails sent), by *Database Size* (number of contacts stored), or by *Active Users*? Ensure the pricing model aligns with your growth strategy. (e.g., If you have a massive database but only email them rarely, do not sign a contract that charges by database size).
2. **The "Hidden" Implementation Costs:** Salesforce and Adobe rarely work out of the box. You will likely need to hire a digital agency (Deloitte, Slalom, or a boutique firm) for 3–6 months to set it up. Factor a $50k–$200k implementation cost into the Year 1 TCO.
3. **Tech Stack Consolidation:** What are we already paying for? If you have Salesforce CRM, look at Salesforce Marketing Cloud. If you have Adobe Experience Manager, look at Adobe Campaign. The bundled discounts and native integrations will save you money and IT headaches.
4. **Deliverability & Compliance:** Being in the US, ensure the vendor has robust, automated tools for CAN-SPAM and CCPA compliance. Ask about their dedicated IP warming services.
5. **Service Level Agreements (SLAs):** Email is often revenue-generating. If the platform goes down on Black Friday (for B2C) or during a major product launch, what are the financial penalties in the contract?
**My Recommendation on Next Steps:**
Ask your CMO one question: *"What is our primary goal: aligning with Sales on complex B2B leads, or driving high-volume B2C consumer transactions?"*
If **B2B**, put **Marketo**, **Pardot**, and **HubSpot** on your RFP shortlist.
If **B2C/App**, put **Braze**, **Iterable**, and **Salesforce Marketing Cloud** on your shortlist.

End of response · 55 lines · 5,786 characters · sha256 64adfdddd2f35932fe803ae60bea525d26039c5ff4ccd099376018a69dce082f

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-22 09:00:21 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
7 prior days with data; 2 matched the baseline value (2026-09-15, 2026-09-18)
Responses
before 2026-09-18 09:31:35 UTC, run 20260918T090022Z-92b6f7; after 2026-09-22 09:28:39 UTC, run 20260922T090022Z-7ff13b
Detected
2026-09-22 · run 20260922T090022Z-7ff13b, started 2026-09-22 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-22/P29/google/gemini-3.1-pro-preview/pick_change/picks.top/
Event ID
2026-09-22:P29:google/gemini-3.1-pro-preview:pick_change:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.