MODELDRIFT.WATCH

Sep 23 · Day 14

MINOR · GEMINI · QUESTION 33

Gemini changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 22beforePermalink

Picks read from the answer, in order
  1. Salesforce Marketing Cloud Account Engagement
  2. Adobe Marketo Engage
  3. HubSpot
  4. Klaviyo
  5. Modern
  6. Braze

As a VP of Procurement at a 5,000-person enterprise, your lens is different from the Chief Marketing Officer's. While the CMO wants features, automation, and ease of use, you are evaluating Total Cost of Ownership (TCO), scalability, data security (SOC 2/CCPA), vendor viability, and integration into your existing tech stack.

At the 5,000-employee mark, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an enterprise-grade platform.

However, the "right" platform depends entirely on your company's business model. Here is a breakdown of the top enterprise platforms categorized by use case, along with the procurement considerations for each.

1. If you are a B2B Enterprise (Complex Sales Cycles, Lead Scoring)

In B2B, email marketing is about nurturing leads, integrating deeply with a CRM, and routing qualified leads to a sales team.

  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot)
  • •The Vibe: The gold standard if your company already uses Salesforce CRM.
  • •Procurement Lens: It offers seamless integration with Salesforce, reducing middleware costs. However, Salesforce pricing is notoriously rigid, and implementation almost always requires a third-party SI (Systems Integrator). Be prepared to negotiate hard on renewal caps.
  • •Adobe Marketo Engage
  • •The Vibe: The most powerful B2B marketing automation and lead-scoring tool on the market.
  • •Procurement Lens: Marketo is an enterprise workhorse but requires dedicated, certified staff to run it. When evaluating TCO, factor in the cost of hiring Marketo-certified admins (they are expensive) and a lengthy implementation timeline.
  • •HubSpot (Enterprise Tier)
  • •The Vibe: The easiest to use. Marketing teams love it because of its intuitive UI and fast time-to-value.
  • •Procurement Lens: Implementation costs are significantly lower than Salesforce or Adobe. However, HubSpot prices based on "Marketing Contacts." At an enterprise scale, this can result in explosive cost increases if your marketing team isn't meticulous about archiving old contacts.

2. If you are a B2C / Retail / eCommerce Enterprise (High Volume, Transactional)

In B2C, email marketing is about high-volume sends, personalization, cart abandonment, and cross-channel messaging (SMS + Email).

  • •Salesforce Marketing Cloud (SFMC - ExactTarget)
  • •The Vibe: The traditional heavy hitter for massive B2C sends.
  • •Procurement Lens: Highly secure, scalable, and stable. But it is notoriously disjointed on the back end (it’s a Frankenstein of acquisitions). TCO is high due to the need for specialized developers (AMPScript) and expensive modular add-ons.
  • •Klaviyo (Enterprise Tier)
  • •The Vibe: The undisputed king of eCommerce email marketing. If your company sells products online, Marketing will likely ask for this.
  • •Procurement Lens: Fast deployment and incredible ROI for retail. Pricing is very transparent but scales directly with list size and email/SMS volume. Ensure your contract includes volume tier discounts as you grow.

3. The Modern, "Data-First" Alternatives (B2C or Tech/SaaS Companies)

If your enterprise has a massive mobile app presence or relies heavily on a modern data warehouse (like Snowflake or Databricks), traditional platforms might be too slow.

  • •Braze or Iterable
  • •The Vibe: These are modern, cross-channel customer engagement platforms. They handle email, push notifications, SMS, and in-app messaging flawlessly.
  • •Procurement Lens: These are generally more flexible and have better APIs than legacy Adobe/Salesforce products. They price based on "Monthly Active Users" (MAUs) or data points. Warning: They require a mature data infrastructure. If you buy Braze, you might also need to buy a Customer Data Platform (CDP) like Segment or mParticle to feed it data.

The VP of Procurement’s Checklist for Evaluating these Platforms

Before running an RFP or signing a contract, sit down with your CMO and CIO and align on the following:

  • 1.Pricing Model Traps: Are we paying by Volume (number of emails sent), by Database Size (number of contacts stored), or by Active Users? Ensure the pricing model aligns with your growth strategy. (e.g., If you have a massive database but only email them rarely, do not sign a contract that charges by database size).
  • 2.The "Hidden" Implementation Costs: Salesforce and Adobe rarely work out of the box. You will likely need to hire a digital agency (Deloitte, Slalom, or a boutique firm) for 3–6 months to set it up. Factor a $50k–$200k implementation cost into the Year 1 TCO.
  • 3.Tech Stack Consolidation: What are we already paying for? If you have Salesforce CRM, look at Salesforce Marketing Cloud. If you have Adobe Experience Manager, look at Adobe Campaign. The bundled discounts and native integrations will save you money and IT headaches.
  • 4.Deliverability & Compliance: Being in the US, ensure the vendor has robust, automated tools for CAN-SPAM and CCPA compliance. Ask about their dedicated IP warming services.
  • 5.Service Level Agreements (SLAs): Email is often revenue-generating. If the platform goes down on Black Friday (for B2C) or during a major product launch, what are the financial penalties in the contract?

My Recommendation on Next Steps:
Ask your CMO one question: "What is our primary goal: aligning with Sales on complex B2B leads, or driving high-volume B2C consumer transactions?"

If B2B, put Marketo, Pardot, and HubSpot on your RFP shortlist.
If B2C/App, put Braze, Iterable, and Salesforce Marketing Cloud on your shortlist.

SEP 23afterPermalink

Picks read from the answer, in order
  1. Salesforce Marketing Cloud
  2. Adobe Marketo Engage
  3. Braze
  4. HubSpot Enterprise

As a VP of Procurement at a 5,000-person enterprise, your evaluation criteria are fundamentally different from your Chief Marketing Officer’s. While Marketing is looking at features, personalization, and UI, you are looking at Total Cost of Ownership (TCO), vendor consolidation, security/compliance (CCPA/GDPR/SOC 2), integration costs, and pricing scalability.

At 5,000 employees, you are firmly in the enterprise tier. Standard SMB tools (like Mailchimp or Constant Contact) will present security, governance, and data-silo risks.

To give you the best recommendation, the platform you choose should be dictated by three things: Your primary CRM, your business model (B2B vs. B2C), and the pricing structure you are willing to tolerate.

Here is the executive breakdown of the enterprise email marketing platforms you should consider, viewed through a procurement lens.

Category 1: The Enterprise Marketing Clouds (Best for Vendor Consolidation & B2B)

If your company is heavily reliant on a major CRM, sticking within their ecosystem is often the safest procurement bet for data security and vendor consolidation.

1. Salesforce Marketing Cloud (SFMC) or Account Engagement (formerly Pardot)

  • •Best for: Enterprises already using Salesforce CRM. SFMC is for B2C (high volume, omnichannel); Pardot is for B2B (long sales cycles, lead scoring).
  • •Procurement Angle:
  • •Pros: Vendor consolidation (bundle it into your master Salesforce MSA for better leverage). It meets every enterprise security and compliance standard.
  • •Cons/Risks: Exceptionally high implementation costs. You will almost certainly need to hire a certified System Integrator (SI) agency to set it up, which can cost 50% to 150% of the Year-1 software license cost.

2. Adobe Marketo Engage

  • •Best for: Complex B2B enterprises. It is the gold standard for lead routing, scoring, and account-based marketing.
  • •Procurement Angle:
  • •Pros: Deep integrations with major CRMs (Salesforce, Microsoft Dynamics). Highly reliable.
  • •Cons/Risks: Pricing is heavily based on database size. If your marketing team does not routinely purge dead contacts, your renewal costs will skyrocket. It also requires highly specialized, expensive internal talent to run.

Category 2: The Modern Agile Platforms (Best for User Adoption & B2C/App-based)

If your company has a massive consumer database, a mobile app, or wants a platform that requires less heavy IT involvement, look here.

3. Braze (or Iterable)

  • •Best for: B2C enterprises, SaaS, or companies with a strong mobile app presence. They treat email, SMS, and push notifications equally.
  • •Procurement Angle:
  • •Pros: Highly modern architecture; integrates beautifully with enterprise data warehouses (Snowflake, AWS Redshift).
  • •Cons/Risks: Pricing model. Braze prices on Monthly Active Users (MAUs) or data points/events. You must have a strict SLA with Marketing and IT on what data is being sent to Braze, or you will blow through your data allotment and face massive overage fees.

4. HubSpot Enterprise (Marketing Hub)

  • •Best for: Mid-to-large enterprises looking for an "all-in-one" solution that Marketing teams actually enjoy using.
  • •Procurement Angle:
  • •Pros: Extremely low implementation cost compared to Salesforce or Adobe. High user adoption. Very transparent pricing.
  • •Cons/Risks: HubSpot prices based on "Marketing Contacts." Again, if the marketing team imports 1 million unvetted trade-show leads into the system, your SaaS bill will increase automatically.

🔑 The VP of Procurement’s Negotiation & Evaluation Checklist

When your marketing team brings you their preferred vendor, here is where you can mitigate risk and control costs:

  • 1.Interrogate the Pricing Metric:
  • •Per Seat? (Rare in this category).
  • •Per Contact in Database? (e.g., Marketo, HubSpot). Rule: Negotiate strict database limits and demand Marketing implements automated data-cleansing rules so you aren't paying for unsubscribed/dead contacts.
  • •Volume/CPM (Cost Per Mille)? (Paying per 1,000 emails sent). Rule: Negotiate overage buffers. Ask for a 10% tolerance on volume before overage rates kick in.
  • 2.Implementation & Professional Services:
  • •Do not look at the software cost in a vacuum. Ask the vendor: "Are implementation services included, or do we need a 3rd party?" If a 3rd party is required, ask the vendor to introduce 3 preferred partners so you can run an RFP on the implementation costs.
  • 3.Deliverability Services:
  • •At a 5,000-person enterprise scale, you need a Dedicated IP address. Ensure a Dedicated IP and "Deliverability Consulting" hours are baked into the contract.
  • 4.Governance & Security:
  • •Ensure the platform includes Single Sign-On (SSO) and strict Role-Based Access Control (RBAC). You cannot have an intern accidentally emailing your entire customer base or exporting a CSV of PII.

Recommendation on Next Steps

  • 1.If you are B2B and use Salesforce: Direct your team to evaluate Salesforce Pardot vs. Marketo.
  • 2.If you are B2C with massive data volumes: Direct your team to evaluate Braze vs. Salesforce Marketing Cloud.
  • 3.If marketing wants speed and ease of use, and IT is backlogged: Direct your team to evaluate HubSpot Enterprise.

Tip for the negotiation table: The end of the month/quarter is critical for these publicly traded companies (Salesforce, Adobe, HubSpot, Braze). If you can align your procurement timeline to sign in the final week of their fiscal quarter, you can routinely extract 15-25% discounts on multi-year commitments.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-22 permalink

Scored
answered
Received
2026-09-22 09:28:39 UTC
Run
20260922T090022Z-7ff13b
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,786 characters · 2,574 tokens out incl. hidden reasoning · 35.1 s
Extracted picks, in order · confidence medium
  1. Salesforce Marketing Cloud Account Engagement
  2. Adobe Marketo Engage
  3. HubSpot
  4. Klaviyo
  5. Modern
  6. Braze

Highlighted: the names the extractor took as picks, where they appear verbatim (6 of 6).

As a VP of Procurement at a 5,000-person enterprise, your lens is different from the Chief Marketing Officer's. While the CMO wants features, automation, and ease of use, you are evaluating **Total Cost of Ownership (TCO), scalability, data security (SOC 2/CCPA), vendor viability, and integration into your existing tech stack.**
At the 5,000-employee mark, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an **enterprise-grade platform**.
However, the "right" platform depends entirely on your company's business model. Here is a breakdown of the top enterprise platforms categorized by use case, along with the procurement considerations for each.
---
### 1. If you are a B2B Enterprise (Complex Sales Cycles, Lead Scoring)
*In B2B, email marketing is about nurturing leads, integrating deeply with a CRM, and routing qualified leads to a sales team.*
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot)**
* **The Vibe:** The gold standard if your company already uses Salesforce CRM.
* **Procurement Lens:** It offers seamless integration with Salesforce, reducing middleware costs. However, Salesforce pricing is notoriously rigid, and implementation almost always requires a third-party SI (Systems Integrator). Be prepared to negotiate hard on renewal caps.
* **Adobe Marketo Engage**
* **The Vibe:** The most powerful B2B marketing automation and lead-scoring tool on the market.
* **Procurement Lens:** Marketo is an enterprise workhorse but requires dedicated, certified staff to run it. When evaluating TCO, factor in the cost of hiring Marketo-certified admins (they are expensive) and a lengthy implementation timeline.
* **HubSpot (Enterprise Tier)**
* **The Vibe:** The easiest to use. Marketing teams love it because of its intuitive UI and fast time-to-value.
* **Procurement Lens:** Implementation costs are significantly lower than Salesforce or Adobe. However, HubSpot prices based on "Marketing Contacts." At an enterprise scale, this can result in explosive cost increases if your marketing team isn't meticulous about archiving old contacts.
### 2. If you are a B2C / Retail / eCommerce Enterprise (High Volume, Transactional)
*In B2C, email marketing is about high-volume sends, personalization, cart abandonment, and cross-channel messaging (SMS + Email).*
* **Salesforce Marketing Cloud (SFMC - ExactTarget)**
* **The Vibe:** The traditional heavy hitter for massive B2C sends.
* **Procurement Lens:** Highly secure, scalable, and stable. But it is notoriously disjointed on the back end (it’s a Frankenstein of acquisitions). TCO is high due to the need for specialized developers (AMPScript) and expensive modular add-ons.
* **Klaviyo (Enterprise Tier)**
* **The Vibe:** The undisputed king of eCommerce email marketing. If your company sells products online, Marketing will likely ask for this.
* **Procurement Lens:** Fast deployment and incredible ROI for retail. Pricing is very transparent but scales directly with list size and email/SMS volume. Ensure your contract includes volume tier discounts as you grow.
### 3. The Modern, "Data-First" Alternatives (B2C or Tech/SaaS Companies)
*If your enterprise has a massive mobile app presence or relies heavily on a modern data warehouse (like Snowflake or Databricks), traditional platforms might be too slow.*
* **Braze** or **Iterable**
* **The Vibe:** These are modern, cross-channel customer engagement platforms. They handle email, push notifications, SMS, and in-app messaging flawlessly.
* **Procurement Lens:** These are generally more flexible and have better APIs than legacy Adobe/Salesforce products. They price based on "Monthly Active Users" (MAUs) or data points. **Warning:** They require a mature data infrastructure. If you buy Braze, you might also need to buy a Customer Data Platform (CDP) like Segment or mParticle to feed it data.
---
### The VP of Procurement’s Checklist for Evaluating these Platforms
Before running an RFP or signing a contract, sit down with your CMO and CIO and align on the following:
1. **Pricing Model Traps:** Are we paying by *Volume* (number of emails sent), by *Database Size* (number of contacts stored), or by *Active Users*? Ensure the pricing model aligns with your growth strategy. (e.g., If you have a massive database but only email them rarely, do not sign a contract that charges by database size).
2. **The "Hidden" Implementation Costs:** Salesforce and Adobe rarely work out of the box. You will likely need to hire a digital agency (Deloitte, Slalom, or a boutique firm) for 3–6 months to set it up. Factor a $50k–$200k implementation cost into the Year 1 TCO.
3. **Tech Stack Consolidation:** What are we already paying for? If you have Salesforce CRM, look at Salesforce Marketing Cloud. If you have Adobe Experience Manager, look at Adobe Campaign. The bundled discounts and native integrations will save you money and IT headaches.
4. **Deliverability & Compliance:** Being in the US, ensure the vendor has robust, automated tools for CAN-SPAM and CCPA compliance. Ask about their dedicated IP warming services.
5. **Service Level Agreements (SLAs):** Email is often revenue-generating. If the platform goes down on Black Friday (for B2C) or during a major product launch, what are the financial penalties in the contract?
**My Recommendation on Next Steps:**
Ask your CMO one question: *"What is our primary goal: aligning with Sales on complex B2B leads, or driving high-volume B2C consumer transactions?"*
If **B2B**, put **Marketo**, **Pardot**, and **HubSpot** on your RFP shortlist.
If **B2C/App**, put **Braze**, **Iterable**, and **Salesforce Marketing Cloud** on your shortlist.

End of response · 55 lines · 5,786 characters · sha256 64adfdddd2f35932fe803ae60bea525d26039c5ff4ccd099376018a69dce082f

After this run

2026-09-23 permalink

Scored
answered
Received
2026-09-23 09:23:59 UTC
Run
20260923T090022Z-9b7b3a
Served by
Google · HTTP 200 · finish stop
Size
5,809 characters · 2,768 tokens out incl. hidden reasoning · 24.1 s
Extracted picks, in order · confidence high
  1. Salesforce Marketing Cloud
  2. Adobe Marketo Engage
  3. Braze
  4. HubSpot Enterprise

Highlighted: the names the extractor took as picks, where they appear verbatim (4 of 4).

As a VP of Procurement at a 5,000-person enterprise, your evaluation criteria are fundamentally different from your Chief Marketing Officer’s. While Marketing is looking at features, personalization, and UI, you are looking at **Total Cost of Ownership (TCO), vendor consolidation, security/compliance (CCPA/GDPR/SOC 2), integration costs, and pricing scalability.**
At 5,000 employees, you are firmly in the enterprise tier. Standard SMB tools (like Mailchimp or Constant Contact) will present security, governance, and data-silo risks.
To give you the best recommendation, the platform you choose should be dictated by three things: **Your primary CRM**, your **business model (B2B vs. B2C)**, and the **pricing structure** you are willing to tolerate.
Here is the executive breakdown of the enterprise email marketing platforms you should consider, viewed through a procurement lens.
---
### Category 1: The Enterprise Marketing Clouds (Best for Vendor Consolidation & B2B)
If your company is heavily reliant on a major CRM, sticking within their ecosystem is often the safest procurement bet for data security and vendor consolidation.
#### 1. Salesforce Marketing Cloud (SFMC) or Account Engagement (formerly Pardot)
* **Best for:** Enterprises already using Salesforce CRM. SFMC is for B2C (high volume, omnichannel); Pardot is for B2B (long sales cycles, lead scoring).
* **Procurement Angle:**
* **Pros:** Vendor consolidation (bundle it into your master Salesforce MSA for better leverage). It meets every enterprise security and compliance standard.
* **Cons/Risks:** Exceptionally high implementation costs. You will almost certainly need to hire a certified System Integrator (SI) agency to set it up, which can cost 50% to 150% of the Year-1 software license cost.
#### 2. Adobe Marketo Engage
* **Best for:** Complex B2B enterprises. It is the gold standard for lead routing, scoring, and account-based marketing.
* **Procurement Angle:**
* **Pros:** Deep integrations with major CRMs (Salesforce, Microsoft Dynamics). Highly reliable.
* **Cons/Risks:** Pricing is heavily based on database size. If your marketing team does not routinely purge dead contacts, your renewal costs will skyrocket. It also requires highly specialized, expensive internal talent to run.
---
### Category 2: The Modern Agile Platforms (Best for User Adoption & B2C/App-based)
If your company has a massive consumer database, a mobile app, or wants a platform that requires less heavy IT involvement, look here.
#### 3. Braze (or Iterable)
* **Best for:** B2C enterprises, SaaS, or companies with a strong mobile app presence. They treat email, SMS, and push notifications equally.
* **Procurement Angle:**
* **Pros:** Highly modern architecture; integrates beautifully with enterprise data warehouses (Snowflake, AWS Redshift).
* **Cons/Risks:** **Pricing model.** Braze prices on Monthly Active Users (MAUs) or data points/events. You must have a strict SLA with Marketing and IT on what data is being sent to Braze, or you will blow through your data allotment and face massive overage fees.
#### 4. HubSpot Enterprise (Marketing Hub)
* **Best for:** Mid-to-large enterprises looking for an "all-in-one" solution that Marketing teams actually enjoy using.
* **Procurement Angle:**
* **Pros:** Extremely low implementation cost compared to Salesforce or Adobe. High user adoption. Very transparent pricing.
* **Cons/Risks:** HubSpot prices based on "Marketing Contacts." Again, if the marketing team imports 1 million unvetted trade-show leads into the system, your SaaS bill will increase automatically.
---
### 🔑 The VP of Procurement’s Negotiation & Evaluation Checklist
When your marketing team brings you their preferred vendor, here is where you can mitigate risk and control costs:
1. **Interrogate the Pricing Metric:**
* *Per Seat?* (Rare in this category).
* *Per Contact in Database?* (e.g., Marketo, HubSpot). **Rule:** Negotiate strict database limits and demand Marketing implements automated data-cleansing rules so you aren't paying for unsubscribed/dead contacts.
* *Volume/CPM (Cost Per Mille)?* (Paying per 1,000 emails sent). **Rule:** Negotiate overage buffers. Ask for a 10% tolerance on volume before overage rates kick in.
2. **Implementation & Professional Services:**
* Do not look at the software cost in a vacuum. Ask the vendor: *"Are implementation services included, or do we need a 3rd party?"* If a 3rd party is required, ask the vendor to introduce 3 preferred partners so you can run an RFP on the implementation costs.
3. **Deliverability Services:**
* At a 5,000-person enterprise scale, you need a **Dedicated IP address**. Ensure a Dedicated IP and "Deliverability Consulting" hours are baked into the contract.
4. **Governance & Security:**
* Ensure the platform includes Single Sign-On (SSO) and strict Role-Based Access Control (RBAC). You cannot have an intern accidentally emailing your entire customer base or exporting a CSV of PII.
### Recommendation on Next Steps
1. **If you are B2B and use Salesforce:** Direct your team to evaluate **Salesforce Pardot** vs. **Marketo**.
2. **If you are B2C with massive data volumes:** Direct your team to evaluate **Braze** vs. **Salesforce Marketing Cloud**.
3. **If marketing wants speed and ease of use, and IT is backlogged:** Direct your team to evaluate **HubSpot Enterprise**.
*Tip for the negotiation table:* The end of the month/quarter is critical for these publicly traded companies (Salesforce, Adobe, HubSpot, Braze). If you can align your procurement timeline to sign in the final week of their fiscal quarter, you can routinely extract 15-25% discounts on multi-year commitments.

End of response · 66 lines · 5,809 characters · sha256 1d67589236ab488dcd2e76cf4a5cee37af54b53a6d67be14963c13d6f387faf1

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-23 09:00:22 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
0 added, 2 dropped
Baseline
7 prior days with data; 1 matched the baseline value (2026-09-22)
Responses
before 2026-09-22 09:28:39 UTC, run 20260922T090022Z-7ff13b; after 2026-09-23 09:23:59 UTC, run 20260923T090022Z-9b7b3a
Detected
2026-09-23 · run 20260923T090022Z-9b7b3a, started 2026-09-23 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-23/P29/google/gemini-3.1-pro-preview/pick_change/picks/
Event ID
2026-09-23:P29:google/gemini-3.1-pro-preview:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.