MODELDRIFT.WATCH

Oct 1 · Day 22

MINOR · GEMINI · QUESTION 33

Gemini went back to its earlier list for a large enterprise’s request for an email platform.

Why it’s minor

The change lasted one day before the model went back.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 30the changePermalink

Picks read from the answer, in order
  1. Heavyweights
  2. High-Volume Titans
  3. Modern Challenger

As a VP of Procurement for a 5,000-person enterprise, your lens for selecting an email marketing platform is going to be vastly different from your CMO’s. While Marketing wants features and usability, your focus is on Total Cost of Ownership (TCO), CRM integration, data security (SOC 2, CCPA, GDPR), vendor stability, and scalability.

At the 5,000-employee mark, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an Enterprise Marketing Automation Platform (MAP).

The right choice depends heavily on two factors: Your business model (B2B vs. B2C) and your existing CRM.

Here is the shortlist of platforms you should be evaluating, categorized by use case, along with the procurement considerations for each.

1. The Heavyweights (Best for Complex B2B & Salesforce Ecosystems)

If your company sells to other businesses (B2B) and uses Salesforce as your CRM, these are the industry standards.

  • •Adobe Marketo Engage:
  • •Best for: Highly complex B2B lead scoring, multi-touch attribution, and deep integrations.
  • •Procurement Note: Marketo is incredibly powerful but requires specialized talent to run. The implementation costs and timeline will be high. Pricing is usually based on database size (number of contacts). Negotiate caps on overage fees.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot):
  • •Best for: B2B companies already deeply entrenched in Salesforce.
  • •Procurement Note: Because it is native to Salesforce, IT and RevOps usually love this choice—it keeps data in one ecosystem and reduces integration costs. However, Salesforce pricing is notoriously complex. Ensure you bundle this with your existing CRM contract for maximum leverage.

2. The High-Volume Titans (Best for Enterprise B2C)

If your company sells directly to consumers (Retail, E-commerce, Financial Services) and sends tens of millions of emails a month, you need a B2C-focused platform.

  • •Salesforce Marketing Cloud (SFMC - formerly ExactTarget):
  • •Best for: Massive scale, personalized consumer journeys across email, SMS, and push notifications.
  • •Procurement Note: SFMC is priced by contact volume and API calls. Watch out for "Super Message" pricing, where SMS and email draw from the same pool of credits at different rates. Implementation usually requires an expensive third-party systems integrator (SI).
  • •Braze:
  • •Best for: Modern, mobile-first B2C companies. If your company relies heavily on an app alongside email, Braze is the current market darling.
  • •Procurement Note: Braze is known for agility and high ROI for marketing teams. From a procurement standpoint, they are a modern, cloud-native vendor with predictable pricing models based on Monthly Active Users (MAUs).

3. The Modern Challenger (Best for Usability & Consolidated Stacks)

  • •HubSpot (Enterprise Tier):
  • •Best for: Enterprises looking to consolidate their tech stack (Marketing, Sales, and Support in one platform) or those wanting a platform with high user adoption.
  • •Procurement Note: Historically an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies. Procurement loves HubSpot because it generally has a lower TCO than Salesforce/Adobe, faster implementation times, and requires fewer expensive third-party developers. Pricing is based on "Marketing Contacts."

Your Procurement RFP Checklist

Before signing a vendor, you should align with your CMO and CIO on the following procurement landmines specific to email platforms:

  • 1.The Pricing Model Trap: Platforms price in three ways: Total Database Size (HubSpot, Marketo), Email Send Volume (some SFMC models), or Active Monthly Users (Braze). If your marketing team has a database of 10 million contacts but only emails 2 million of them, a "Total Database" pricing model will cost you a fortune. Negotiate based on active marketable contacts.
  • 2.Implementation Costs: Enterprise email platforms are not plug-and-play. You must budget for an implementation partner (Accenture, Deloitte, or boutique agencies). Rule of thumb: Expect implementation to cost 50% to 150% of your Year 1 software licensing fee.
  • 3.Compliance & Security: Ensure the vendor holds SOC 2 Type II compliance, ISO 27001, and allows for data residency in the US. If you do any business in Europe or California, strict GDPR/CCPA data deletion capabilities must be native to the platform.
  • 4.IP Warming & Deliverability: Moving to a new platform requires "warming up" new IP addresses so your emails don't go to spam. Ensure the vendor includes a dedicated deliverability consultant in your Year 1 contract.

Next Steps for You

Don't evaluate the software first. Go to your CMO and CIO/Head of RevOps and ask these three questions:

  • 1.Are we strictly B2B, B2C, or both?
  • 2.Is this new platform meant to integrate with our current CRM, or replace it?
  • 3.Do we have the in-house technical talent to run a complex system like Adobe/Salesforce, or do we need something with high usability like HubSpot?

Once they answer those, you can narrow this list down to two vendors and pit them against each other in your RFP.

OCT 1backPermalink

Picks read from the answer, in order
  1. Marketing Automation Platform
  2. Understand the Pricing Metric
  3. Factor in the "Hidden" Implementation TCO
  4. Security
  5. Ecosystem Consolidation

As a VP of Procurement at a 5,000-person enterprise, you are not just buying an "email marketing platform." You are likely procuring a full Marketing Automation Platform (MAP) or a Customer Engagement Platform (CEP).

At your scale, the decision should be driven by three things: your company’s business model (B2B vs. B2C), your existing tech stack (specifically your CRM), and Total Cost of Ownership (TCO), which will include heavy implementation and integration costs.

Here is a breakdown of the top enterprise platforms you should evaluate, categorized by your business model, followed by the specific criteria you should use to negotiate the contracts.

1. If Your Company is B2B (Focus on Lead Gen & Long Sales Cycles)

In B2B, email marketing is about nurturing leads until they are ready to talk to a sales rep. The tool must integrate flawlessly with your CRM (likely Salesforce or Microsoft Dynamics).

  • •Adobe Marketo Engage:
  • •Best for: Complex lead scoring, deep CRM integrations, and highly technical marketing teams.
  • •Procurement note: Marketo is notorious for steep learning curves. Factor in the cost of a specialized Marketo admin (salary) or an agency partner for implementation and managed services. Pricing is generally based on the size of the marketable database.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot):
  • •Best for: Companies already deeply entrenched in the Salesforce ecosystem.
  • •Procurement note: If you use Salesforce Sales Cloud, this is often the path of least resistance. However, Salesforce pricing can be modular; watch out for add-on costs for B2B Marketing Analytics or advanced API access.
  • •HubSpot (Enterprise Hub):
  • •Best for: Companies that want power but highly value ease of use. It is taking massive market share from Marketo and Pardot.
  • •Procurement note: HubSpot is aggressively moving upmarket. TCO is often lower than Marketo/Salesforce because implementation is faster and you rarely need high-priced external developers to run it. Watch their pricing tiers regarding "Marketing Contacts."

2. If Your Company is B2C (Focus on High Volume, E-commerce, or Apps)

In B2C, email is about driving immediate purchases, app engagement, and cross-channel personalization (Email + SMS + Push notifications) to millions of consumers.

  • •Salesforce Marketing Cloud (SFMC - formerly ExactTarget):
  • •Best for: High-volume, omni-channel consumer journeys.
  • •Procurement note: Highly complex and expensive to implement. You will need a System Integrator (SI) like Deloitte, Slalom, or a boutique agency to set this up. Negotiate the implementation SOW just as hard as the software licenses.
  • •Braze or Iterable:
  • •Best for: Modern, mobile-first companies (retail, apps, streaming). They are the modern alternatives to legacy Oracle/Salesforce systems.
  • •Procurement note: These tools price based on Monthly Active Users (MAUs) or event volumes, not just a static list of email addresses. You must work closely with marketing and data teams to accurately forecast MAUs, or you will face massive overage penalties.

The Procurement Checklist: How to Evaluate & Negotiate

Before issuing an RFP or signing a contract, align with your CMO/VP of Marketing on the following to protect the business:

1. Understand the Pricing Metric (The Trap)
Vendors price differently. Know exactly what you are paying for:

  • •Per Contact: Are you paying for every email address in the database, or only the "marketable" ones (excluding unsubscribes)?
  • •Per Send/Volume: Are there limits on how many emails you can send a month?
  • •Per User: How many marketing seats do you get?

2. Factor in the "Hidden" Implementation TCO
At a 5,000-employee scale, the software license is only 40-50% of the first-year cost.

  • •You will likely need an integration partner to connect the ESP to your CRM, Data Warehouse (Snowflake, AWS), and ERP.
  • •Advice: Ask the vendor to co-term or heavily discount the first year’s license while the platform is being implemented (which can take 3 to 6 months).

3. Security, Compliance, & Deliverability
As a US enterprise, you must ensure:

  • •CCPA/CPRA Compliance: How easy is it to execute "Right to be Forgotten" requests across the platform?
  • •SOC 2 Type II: Non-negotiable for a company of your size.
  • •Dedicated IPs: At your volume, you need a dedicated sending IP. Ensure IP warm-up services are included in the onboarding SLA.

4. Ecosystem Consolidation
Look at what your marketing team is currently buying piecemeal. Do they have separate contracts for SMS marketing, landing page builders, web analytics, and social media scheduling? A premium platform (like HubSpot or Salesforce) might allow you to cancel 3 or 4 smaller vendor contracts, offsetting the higher cost.

Next Step

I recommend asking your Marketing Leader two questions before looking at vendors:

  • 1."Are we primarily trying to score and hand off B2B leads to our sales team, or are we trying to drive consumer transactions at a massive scale?"
  • 2."What CRM and Data Warehouse must this natively connect to?"

If they say B2B and Salesforce, look at Pardot or HubSpot.
If they say B2C and Snowflake/custom databases, look at Braze or Salesforce Marketing Cloud.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Deviation the day it changed

2026-09-30 permalink

Scored
answered
Received
2026-09-30 09:24:35 UTC
Run
20260930T090007Z-45ce12
Served by
Google · HTTP 200 · finish stop
Size
5,320 characters · 2,277 tokens out incl. hidden reasoning · 19.8 s
Extracted picks, in order · confidence high
  1. Heavyweights
  2. High-Volume Titans
  3. Modern Challenger

Highlighted: the names the extractor took as picks, where they appear verbatim (3 of 3).

As a VP of Procurement for a 5,000-person enterprise, your lens for selecting an email marketing platform is going to be vastly different from your CMO’s. While Marketing wants features and usability, your focus is on **Total Cost of Ownership (TCO), CRM integration, data security (SOC 2, CCPA, GDPR), vendor stability, and scalability.**
At the 5,000-employee mark, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an **Enterprise Marketing Automation Platform (MAP).**
The right choice depends heavily on two factors: **Your business model (B2B vs. B2C)** and **your existing CRM**.
Here is the shortlist of platforms you should be evaluating, categorized by use case, along with the procurement considerations for each.
---
### 1. The Heavyweights (Best for Complex B2B & Salesforce Ecosystems)
If your company sells to other businesses (B2B) and uses Salesforce as your CRM, these are the industry standards.
* **Adobe Marketo Engage:**
* **Best for:** Highly complex B2B lead scoring, multi-touch attribution, and deep integrations.
* **Procurement Note:** Marketo is incredibly powerful but requires specialized talent to run. The implementation costs and timeline will be high. Pricing is usually based on database size (number of contacts). Negotiate caps on overage fees.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot):**
* **Best for:** B2B companies already deeply entrenched in Salesforce.
* **Procurement Note:** Because it is native to Salesforce, IT and RevOps usually love this choice—it keeps data in one ecosystem and reduces integration costs. However, Salesforce pricing is notoriously complex. Ensure you bundle this with your existing CRM contract for maximum leverage.
### 2. The High-Volume Titans (Best for Enterprise B2C)
If your company sells directly to consumers (Retail, E-commerce, Financial Services) and sends tens of millions of emails a month, you need a B2C-focused platform.
* **Salesforce Marketing Cloud (SFMC - formerly ExactTarget):**
* **Best for:** Massive scale, personalized consumer journeys across email, SMS, and push notifications.
* **Procurement Note:** SFMC is priced by contact volume and API calls. **Watch out for "Super Message" pricing**, where SMS and email draw from the same pool of credits at different rates. Implementation usually requires an expensive third-party systems integrator (SI).
* **Braze:**
* **Best for:** Modern, mobile-first B2C companies. If your company relies heavily on an app alongside email, Braze is the current market darling.
* **Procurement Note:** Braze is known for agility and high ROI for marketing teams. From a procurement standpoint, they are a modern, cloud-native vendor with predictable pricing models based on Monthly Active Users (MAUs).
### 3. The Modern Challenger (Best for Usability & Consolidated Stacks)
* **HubSpot (Enterprise Tier):**
* **Best for:** Enterprises looking to consolidate their tech stack (Marketing, Sales, and Support in one platform) or those wanting a platform with high user adoption.
* **Procurement Note:** Historically an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies. Procurement loves HubSpot because it generally has a **lower TCO than Salesforce/Adobe**, faster implementation times, and requires fewer expensive third-party developers. Pricing is based on "Marketing Contacts."
---
### Your Procurement RFP Checklist
Before signing a vendor, you should align with your CMO and CIO on the following procurement landmines specific to email platforms:
1. **The Pricing Model Trap:** Platforms price in three ways: *Total Database Size* (HubSpot, Marketo), *Email Send Volume* (some SFMC models), or *Active Monthly Users* (Braze). If your marketing team has a database of 10 million contacts but only emails 2 million of them, a "Total Database" pricing model will cost you a fortune. Negotiate based on *active* marketable contacts.
2. **Implementation Costs:** Enterprise email platforms are not plug-and-play. You must budget for an implementation partner (Accenture, Deloitte, or boutique agencies). **Rule of thumb:** Expect implementation to cost 50% to 150% of your Year 1 software licensing fee.
3. **Compliance & Security:** Ensure the vendor holds SOC 2 Type II compliance, ISO 27001, and allows for data residency in the US. If you do any business in Europe or California, strict GDPR/CCPA data deletion capabilities must be native to the platform.
4. **IP Warming & Deliverability:** Moving to a new platform requires "warming up" new IP addresses so your emails don't go to spam. Ensure the vendor includes a dedicated deliverability consultant in your Year 1 contract.
### Next Steps for You
Don't evaluate the software first. Go to your **CMO** and **CIO/Head of RevOps** and ask these three questions:
1. *Are we strictly B2B, B2C, or both?*
2. *Is this new platform meant to integrate with our current CRM, or replace it?*
3. *Do we have the in-house technical talent to run a complex system like Adobe/Salesforce, or do we need something with high usability like HubSpot?*
Once they answer those, you can narrow this list down to two vendors and pit them against each other in your RFP.

End of response · 53 lines · 5,320 characters · sha256 c9049fc689a94c386edfe23ab295bbcf33db57930b5a59cc83405c69af2ea28e

Reverted back to baseline

2026-10-01 permalink

Scored
answered
Received
2026-10-01 09:26:12 UTC
Run
20261001T090007Z-857bc9
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,460 characters · 2,304 tokens out incl. hidden reasoning · 33.1 s
Extracted picks, in order · confidence high
  1. Marketing Automation Platform
  2. Understand the Pricing Metric
  3. Factor in the "Hidden" Implementation TCO
  4. Security
  5. Ecosystem Consolidation

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

As a VP of Procurement at a 5,000-person enterprise, you are not just buying an "email marketing platform." You are likely procuring a full **Marketing Automation Platform (MAP)** or a **Customer Engagement Platform (CEP)**.
At your scale, the decision should be driven by three things: **your company’s business model (B2B vs. B2C)**, **your existing tech stack (specifically your CRM)**, and **Total Cost of Ownership (TCO)**, which will include heavy implementation and integration costs.
Here is a breakdown of the top enterprise platforms you should evaluate, categorized by your business model, followed by the specific criteria you should use to negotiate the contracts.
---
### 1. If Your Company is B2B (Focus on Lead Gen & Long Sales Cycles)
In B2B, email marketing is about nurturing leads until they are ready to talk to a sales rep. The tool must integrate flawlessly with your CRM (likely Salesforce or Microsoft Dynamics).
* **Adobe Marketo Engage:**
* **Best for:** Complex lead scoring, deep CRM integrations, and highly technical marketing teams.
* **Procurement note:** Marketo is notorious for steep learning curves. Factor in the cost of a specialized Marketo admin (salary) or an agency partner for implementation and managed services. Pricing is generally based on the size of the marketable database.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot):**
* **Best for:** Companies already deeply entrenched in the Salesforce ecosystem.
* **Procurement note:** If you use Salesforce Sales Cloud, this is often the path of least resistance. However, Salesforce pricing can be modular; watch out for add-on costs for B2B Marketing Analytics or advanced API access.
* **HubSpot (Enterprise Hub):**
* **Best for:** Companies that want power but highly value ease of use. It is taking massive market share from Marketo and Pardot.
* **Procurement note:** HubSpot is aggressively moving upmarket. TCO is often lower than Marketo/Salesforce because implementation is faster and you rarely need high-priced external developers to run it. Watch their pricing tiers regarding "Marketing Contacts."
### 2. If Your Company is B2C (Focus on High Volume, E-commerce, or Apps)
In B2C, email is about driving immediate purchases, app engagement, and cross-channel personalization (Email + SMS + Push notifications) to millions of consumers.
* **Salesforce Marketing Cloud (SFMC - formerly ExactTarget):**
* **Best for:** High-volume, omni-channel consumer journeys.
* **Procurement note:** Highly complex and expensive to implement. You *will* need a System Integrator (SI) like Deloitte, Slalom, or a boutique agency to set this up. Negotiate the implementation SOW just as hard as the software licenses.
* **Braze or Iterable:**
* **Best for:** Modern, mobile-first companies (retail, apps, streaming). They are the modern alternatives to legacy Oracle/Salesforce systems.
* **Procurement note:** These tools price based on **Monthly Active Users (MAUs)** or event volumes, not just a static list of email addresses. You must work closely with marketing and data teams to accurately forecast MAUs, or you will face massive overage penalties.
---
### The Procurement Checklist: How to Evaluate & Negotiate
Before issuing an RFP or signing a contract, align with your CMO/VP of Marketing on the following to protect the business:
**1. Understand the Pricing Metric (The Trap)**
Vendors price differently. Know exactly what you are paying for:
* *Per Contact:* Are you paying for every email address in the database, or only the "marketable" ones (excluding unsubscribes)?
* *Per Send/Volume:* Are there limits on how many emails you can send a month?
* *Per User:* How many marketing seats do you get?
**2. Factor in the "Hidden" Implementation TCO**
At a 5,000-employee scale, the software license is only 40-50% of the first-year cost.
* You will likely need an integration partner to connect the ESP to your CRM, Data Warehouse (Snowflake, AWS), and ERP.
* *Advice:* Ask the vendor to co-term or heavily discount the first year’s license while the platform is being implemented (which can take 3 to 6 months).
**3. Security, Compliance, & Deliverability**
As a US enterprise, you must ensure:
* **CCPA/CPRA Compliance:** How easy is it to execute "Right to be Forgotten" requests across the platform?
* **SOC 2 Type II:** Non-negotiable for a company of your size.
* **Dedicated IPs:** At your volume, you need a dedicated sending IP. Ensure IP warm-up services are included in the onboarding SLA.
**4. Ecosystem Consolidation**
Look at what your marketing team is currently buying piecemeal. Do they have separate contracts for SMS marketing, landing page builders, web analytics, and social media scheduling? A premium platform (like HubSpot or Salesforce) might allow you to cancel 3 or 4 smaller vendor contracts, offsetting the higher cost.
### Next Step
I recommend asking your Marketing Leader two questions before looking at vendors:
1. *"Are we primarily trying to score and hand off B2B leads to our sales team, or are we trying to drive consumer transactions at a massive scale?"*
2. *"What CRM and Data Warehouse must this natively connect to?"*
If they say B2B and Salesforce, look at **Pardot** or **HubSpot**.
If they say B2C and Snowflake/custom databases, look at **Braze** or **Salesforce Marketing Cloud**.

End of response · 64 lines · 5,460 characters · sha256 77b476cadcbc3e4d2a9c530916592423ef6ea7752dc78366c2b23bba7b0d04bd

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-10-01 09:00:06 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
6 prior days with data; 2 matched the baseline value (2026-09-29, 2026-09-30)
Responses
deviation 2026-09-30 09:24:35 UTC, run 20260930T090007Z-45ce12; reverted 2026-10-01 09:26:12 UTC, run 20261001T090007Z-857bc9
Detected
2026-10-01 · run 20261001T090007Z-857bc9, started 2026-10-01 09:00:07 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-10-01/P29/google/gemini-3.1-pro-preview/transient_anomaly/picks.top/
Event ID
2026-10-01:P29:google/gemini-3.1-pro-preview:transient_anomaly:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.