MODELDRIFT.WATCH

Sep 18 · Day 9

MINOR · GEMINI · QUESTION 33

Gemini went back to its earlier list for a large enterprise’s request for an email platform.

Why it’s minor

The change lasted one day before the model went back.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 17the changePermalink

Picks read from the answer, in order
  1. Enterprise Incumbents
  2. Modern Cross-Channel Leaders
  3. Agile Enterprise Option
  4. Hidden Costs
  5. Security & Compliance
  6. Data Warehousing

As a VP of Procurement at a 5,000-person enterprise, your lens on this purchase is fundamentally different from your CMO’s. While Marketing is looking at features, personalization, and UI, you are looking at Total Cost of Ownership (TCO), scalability, security/compliance, integration with your existing tech stack, and vendor viability.

At your size, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an enterprise-grade platform.

However, the "right" platform depends heavily on two missing pieces of information: Are you B2B or B2C? and What CRM are you currently using?

Here is the breakdown of the top enterprise platforms you should evaluate, categorized by business model, along with the specific procurement considerations for each.


1. The Enterprise Incumbents (Best if you want a unified tech stack)

If your company is already heavily invested in a specific CRM, staying within that ecosystem often provides the best leverage for enterprise licensing agreements (ELAs) and minimizes integration costs.

  • •Salesforce Marketing Cloud (SFMC) / Account Engagement (Pardot)
  • •Best for: Enterprises already using Salesforce Sales/Service Cloud. SFMC is for B2C (high volume), while Pardot is for B2B (long sales cycles).
  • •Procurement Intel: Salesforce negotiates aggressively at the end of their quarters (Jan, April, July, Oct). Watch out for their pricing model: they charge for both "Contacts" and "Super Messages" (email volume). Implementation costs are very high—expect to pay an external Systems Integrator (SI) 30% to 50% of your first-year software cost just to set it up.
  • •Adobe Marketo Engage (B2B) / Adobe Campaign (B2C)
  • •Best for: Complex B2B lead scoring (Marketo) or deep B2C data modeling (Campaign).
  • •Procurement Intel: Excellent choice if you already use Adobe Experience Cloud or Adobe Analytics. Marketo licenses by database size (contacts). The platform is incredibly powerful but requires highly specialized, expensive internal talent to run.

2. The Modern Cross-Channel Leaders (Best for B2C, App-Driven, or Product-Led Growth)

If your 5,000-person company is heavily reliant on consumer data, mobile apps, or e-commerce, Marketing will likely ask for one of these.

  • •Braze
  • •Best for: B2C enterprises with a mobile app and massive amounts of real-time data.
  • •Procurement Intel: Braze prices based on Monthly Active Users (MAU) and Data Points. Warning: If Marketing does not manage their data ingestion properly, data point overage fees can be brutal. Ensure you negotiate hard caps or favorable overage tiers.
  • •Iterable
  • •Best for: Very similar to Braze, highly flexible data model, great for B2C.
  • •Procurement Intel: Iterable is locked in a fierce battle with Braze for market share. If Marketing wants Braze, make them evaluate Iterable as well—you will be able to play the two vendors against each other for massive discounts.

3. The Agile Enterprise Option (Best for B2B wanting lower TCO)

  • •HubSpot (Enterprise Tier)
  • •Best for: Mid-to-large B2B enterprises that want an all-in-one system without the massive developer overhead of Salesforce or Adobe.
  • •Procurement Intel: HubSpot’s Enterprise tier is highly capable and boasts a significantly lower Total Cost of Ownership (TCO) because it rarely requires an expensive external agency to implement and manage. However, their "Marketing Contacts" pricing can escalate quickly. Ensure you negotiate a flat-rate tier for contact expansion.

The Procurement Playbook: How to evaluate this purchase

Before issuing an RFP or engaging vendors, sit down with your CMO or VP of Marketing and align on the following:

1. Hidden Costs (The Implementation)
Do not look just at the SaaS license. Ask Marketing: “Who is implementing this?” Enterprise tools like Salesforce Marketing Cloud and Adobe Marketo require certified agencies to set up. You must budget for an implementation fee that could range from $50,000 to $250,000+.

2. The Pricing Model Trap
Platforms price differently. You need to align the vendor's pricing model with your business’s growth trajectory:

  • •Volume-based (Cost per email sent): Good if you have a massive database but rarely email them.
  • •Contact-based (Cost per person in the database): Good if you email a small, highly targeted group very frequently.
  • •Event-based (Cost per data point tracked): Highly variable; requires strict data hygiene.

3. Security & Compliance
At 5,000 employees, risk mitigation is critical. Ensure the vendor has SOC 2 Type II, ISO 27001, and native compliance tools for GDPR, CCPA, and (if applicable to your industry) HIPAA. Look deeply into their data residency—can they host your data in the US specifically?

4. Data Warehousing
If your company is using Snowflake, Databricks, or Amazon Redshift, ask the vendors about "Zero-Copy Architecture" or "Reverse ETL." Modern enterprise platforms (like Braze or Iterable) can now read data directly from your warehouse without duplicating it. This saves immense amounts of money on data storage and security compliance.

Summary Recommendation

  • •If you are B2B and use Salesforce: Buy Salesforce Account Engagement (Pardot) or Marketo.
  • •If you are B2B and want lower total overhead: Buy HubSpot Enterprise.
  • •If you are B2C and rely on an app/real-time data: Buy Braze or Iterable.
  • •If you are B2C and use Salesforce heavily: Buy Salesforce Marketing Cloud.

SEP 18backPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Braze
  4. Fast Time-to-Value" Enterprise Contender
  5. HubSpot

As a VP of Procurement for a 5,000-employee enterprise, your evaluation of an email marketing platform will be fundamentally different from that of a CMO. While Marketing is looking at features (A/B testing, AI copy generation, templates), your focus is on Total Cost of Ownership (TCO), scalability, security/compliance, vendor consolidation, and integration capability.

At your size, "email marketing" usually falls into two distinct categories. The right tool depends heavily on your company's business model.

Here is a breakdown of the enterprise-grade platforms you should consider for your RFP, categorized by business model and viewed through a procurement lens.


1. If You Are a B2B Company (Long Sales Cycles, Lead Scoring)

If your 5,000 employees are selling software, manufacturing, consulting, or medical devices, you need a "Marketing Automation Platform" (MAP).

  • •Adobe Marketo Engage
  • •The Procurement View: Marketo is the gold standard for enterprise B2B. It handles complex lead scoring and routing seamlessly.
  • •Cost Structure: Priced by database size (number of contacts), not email volume.
  • •Vendor Risk/TCO: High implementation costs. You will likely need to hire a specialized agency or dedicated internal Marketo architect. However, it is highly stable and integrates deeply with Salesforce CRM.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot)
  • •The Procurement View: If your enterprise already uses Salesforce as its CRM, this is usually the easiest procurement win. Consolidating under one Master Services Agreement (MSA) with Salesforce can give you leverage for volume discounting.
  • •Cost Structure: Tiered user licensing and database size.
  • •Vendor Risk/TCO: Predictable. IT and Legal usually approve it quickly because Salesforce’s security and compliance architecture is already vetted.

2. If You Are a B2C Company (High Volume, E-commerce, Retail)

If you are selling to millions of consumers, you need a platform built for massive scale, transactional emails, and cross-channel journeys (SMS, push notifications).

  • •Salesforce Marketing Cloud (SFMC - formerly ExactTarget)
  • •The Procurement View: An absolute powerhouse for B2C. It can handle billions of sends.
  • •Cost Structure: Highly modular. You pay for "studios" (Email Studio, Mobile Studio) and utilization (Super Messages).
  • •Vendor Risk/TCO: Warning on TCO. The license cost is only the beginning. Implementation is notoriously complex, and you should budget at least 1x to 1.5x the software cost for implementation and ongoing managed services.
  • •Braze or Iterable
  • •The Procurement View: These are the modern, agile alternatives to legacy Salesforce/Adobe. They are built for modern data architectures (e.g., pulling directly from Snowflake or AWS).
  • •Cost Structure: Usually based on Monthly Active Users (MAUs) and message volume.
  • •Vendor Risk/TCO: They require a high level of data maturity. If your enterprise data is messy, these platforms will struggle. However, they boast much faster deployment times than Salesforce or Adobe, leading to quicker ROI.

3. The "Fast Time-to-Value" Enterprise Contender

  • •HubSpot (Enterprise Hub)
  • •The Procurement View: Historically seen as an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies, particularly if you want to replace multiple tools (CRM, Email, CMS, Service) with one vendor.
  • •Cost Structure: Marketing contacts + seat licenses. Very transparent pricing compared to Adobe and Salesforce.
  • •Vendor Risk/TCO: The lowest TCO on this list. It is incredibly user-friendly, meaning marketing teams actually adopt it without requiring expensive external consultants or specialized IT support.

The Procurement Evaluation Checklist (What to demand in the RFP)

When you take this to market, do not let Marketing drive the selection based purely on UI. Ensure your RFP mandates the following:

  • 1.Security & Compliance: Since you are a US enterprise, demand proof of SOC 2 Type II compliance, strict CCPA/CPRA data deletion workflows, and HIPAA compliance (if you touch healthcare).
  • 2.Governance & Access: At 5,000 employees, you will likely have dozens of marketers across different regions or business units. The tool must have robust Role-Based Access Control (RBAC) and Single Sign-On (SSO/SAML) capabilities.
  • 3.Hidden Overage Fees: Pay close attention to how they charge for overages. Are you billed for "contacts in the database" (even if you don't email them) or "sends"? If Marketing runs a massive holiday campaign, will you get hit with a 20% penalty fee for exceeding your sending limits?
  • 4.Dedicated IP and Deliverability: At enterprise scale, you must send from dedicated IP addresses. Demand guarantees around deliverability support and IP warming as part of the onboarding SLA.

Your Immediate Next Steps

  • 1.Define the Tech Stack: Ask your CIO/CTO what your core CRM and Data Warehouse are (e.g., Salesforce, Microsoft Dynamics, Snowflake). The email platform must have native integrations with these, or your integration costs will skyrocket.
  • 2.Align with the CMO: Are they optimizing for B2B lead generation, or B2C mass-market personalization?
  • 3.Shortlist 3 Vendors: Based on the above, typically a standard enterprise RFP looks like: Salesforce vs. Adobe vs. HubSpot (for B2B/Hybrid) OR Salesforce vs. Braze vs. Iterable (for B2C).
Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Deviation the day it changed

2026-09-17 permalink

Scored
answered
Received
2026-09-17 09:28:19 UTC
Run
20260917T090022Z-136fc3
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,647 characters · 2,567 tokens out incl. hidden reasoning · 36.8 s
Extracted picks, in order · confidence high
  1. Enterprise Incumbents
  2. Modern Cross-Channel Leaders
  3. Agile Enterprise Option
  4. Hidden Costs
  5. Security & Compliance
  6. Data Warehousing

Highlighted: the names the extractor took as picks, where they appear verbatim (6 of 6).

As a VP of Procurement at a 5,000-person enterprise, your lens on this purchase is fundamentally different from your CMO’s. While Marketing is looking at features, personalization, and UI, you are looking at **Total Cost of Ownership (TCO), scalability, security/compliance, integration with your existing tech stack, and vendor viability.**
At your size, you have outgrown SMB tools like Mailchimp or Constant Contact. You need an **enterprise-grade platform**.
However, the "right" platform depends heavily on two missing pieces of information: **Are you B2B or B2C?** and **What CRM are you currently using?**
Here is the breakdown of the top enterprise platforms you should evaluate, categorized by business model, along with the specific procurement considerations for each.
---
### 1. The Enterprise Incumbents (Best if you want a unified tech stack)
If your company is already heavily invested in a specific CRM, staying within that ecosystem often provides the best leverage for enterprise licensing agreements (ELAs) and minimizes integration costs.
* **Salesforce Marketing Cloud (SFMC) / Account Engagement (Pardot)**
* **Best for:** Enterprises already using Salesforce Sales/Service Cloud. SFMC is for B2C (high volume), while Pardot is for B2B (long sales cycles).
* **Procurement Intel:** Salesforce negotiates aggressively at the end of their quarters (Jan, April, July, Oct). Watch out for their pricing model: they charge for both "Contacts" and "Super Messages" (email volume). **Implementation costs are very high**—expect to pay an external Systems Integrator (SI) 30% to 50% of your first-year software cost just to set it up.
* **Adobe Marketo Engage (B2B) / Adobe Campaign (B2C)**
* **Best for:** Complex B2B lead scoring (Marketo) or deep B2C data modeling (Campaign).
* **Procurement Intel:** Excellent choice if you already use Adobe Experience Cloud or Adobe Analytics. Marketo licenses by database size (contacts). The platform is incredibly powerful but requires highly specialized, expensive internal talent to run.
### 2. The Modern Cross-Channel Leaders (Best for B2C, App-Driven, or Product-Led Growth)
If your 5,000-person company is heavily reliant on consumer data, mobile apps, or e-commerce, Marketing will likely ask for one of these.
* **Braze**
* **Best for:** B2C enterprises with a mobile app and massive amounts of real-time data.
* **Procurement Intel:** Braze prices based on Monthly Active Users (MAU) and Data Points. **Warning:** If Marketing does not manage their data ingestion properly, data point overage fees can be brutal. Ensure you negotiate hard caps or favorable overage tiers.
* **Iterable**
* **Best for:** Very similar to Braze, highly flexible data model, great for B2C.
* **Procurement Intel:** Iterable is locked in a fierce battle with Braze for market share. If Marketing wants Braze, make them evaluate Iterable as well—you will be able to play the two vendors against each other for massive discounts.
### 3. The Agile Enterprise Option (Best for B2B wanting lower TCO)
* **HubSpot (Enterprise Tier)**
* **Best for:** Mid-to-large B2B enterprises that want an all-in-one system without the massive developer overhead of Salesforce or Adobe.
* **Procurement Intel:** HubSpot’s Enterprise tier is highly capable and boasts a significantly lower Total Cost of Ownership (TCO) because it rarely requires an expensive external agency to implement and manage. However, their "Marketing Contacts" pricing can escalate quickly. Ensure you negotiate a flat-rate tier for contact expansion.
---
### The Procurement Playbook: How to evaluate this purchase
Before issuing an RFP or engaging vendors, sit down with your CMO or VP of Marketing and align on the following:
**1. Hidden Costs (The Implementation)**
Do not look just at the SaaS license. Ask Marketing: *“Who is implementing this?”* Enterprise tools like Salesforce Marketing Cloud and Adobe Marketo require certified agencies to set up. You must budget for an implementation fee that could range from $50,000 to $250,000+.
**2. The Pricing Model Trap**
Platforms price differently. You need to align the vendor's pricing model with your business’s growth trajectory:
* *Volume-based (Cost per email sent):* Good if you have a massive database but rarely email them.
* *Contact-based (Cost per person in the database):* Good if you email a small, highly targeted group very frequently.
* *Event-based (Cost per data point tracked):* Highly variable; requires strict data hygiene.
**3. Security & Compliance**
At 5,000 employees, risk mitigation is critical. Ensure the vendor has SOC 2 Type II, ISO 27001, and native compliance tools for GDPR, CCPA, and (if applicable to your industry) HIPAA. Look deeply into their data residency—can they host your data in the US specifically?
**4. Data Warehousing**
If your company is using Snowflake, Databricks, or Amazon Redshift, ask the vendors about **"Zero-Copy Architecture" or "Reverse ETL."** Modern enterprise platforms (like Braze or Iterable) can now read data directly from your warehouse without duplicating it. This saves immense amounts of money on data storage and security compliance.
### Summary Recommendation
* If you are **B2B** and use Salesforce: Buy **Salesforce Account Engagement (Pardot)** or **Marketo**.
* If you are **B2B** and want lower total overhead: Buy **HubSpot Enterprise**.
* If you are **B2C** and rely on an app/real-time data: Buy **Braze** or **Iterable**.
* If you are **B2C** and use Salesforce heavily: Buy **Salesforce Marketing Cloud**.

End of response · 61 lines · 5,647 characters · sha256 c4284d92f6169021cd1484ff1c75ec4002bd2df3e06884a94e7f60d6ea1a79e6

Reverted back to baseline

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:31:35 UTC
Run
20260918T090022Z-92b6f7
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,651 characters · 2,473 tokens out incl. hidden reasoning · 35.6 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Braze
  4. Fast Time-to-Value" Enterprise Contender
  5. HubSpot

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

As a VP of Procurement for a 5,000-employee enterprise, your evaluation of an email marketing platform will be fundamentally different from that of a CMO. While Marketing is looking at features (A/B testing, AI copy generation, templates), your focus is on **Total Cost of Ownership (TCO), scalability, security/compliance, vendor consolidation, and integration capability.**
At your size, "email marketing" usually falls into two distinct categories. The right tool depends heavily on your company's business model.
Here is a breakdown of the enterprise-grade platforms you should consider for your RFP, categorized by business model and viewed through a procurement lens.
---
### 1. If You Are a B2B Company (Long Sales Cycles, Lead Scoring)
*If your 5,000 employees are selling software, manufacturing, consulting, or medical devices, you need a "Marketing Automation Platform" (MAP).*
* **Adobe Marketo Engage**
* **The Procurement View:** Marketo is the gold standard for enterprise B2B. It handles complex lead scoring and routing seamlessly.
* **Cost Structure:** Priced by database size (number of contacts), not email volume.
* **Vendor Risk/TCO:** High implementation costs. You will likely need to hire a specialized agency or dedicated internal Marketo architect. However, it is highly stable and integrates deeply with Salesforce CRM.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot)**
* **The Procurement View:** If your enterprise already uses Salesforce as its CRM, this is usually the easiest procurement win. Consolidating under one Master Services Agreement (MSA) with Salesforce can give you leverage for volume discounting.
* **Cost Structure:** Tiered user licensing and database size.
* **Vendor Risk/TCO:** Predictable. IT and Legal usually approve it quickly because Salesforce’s security and compliance architecture is already vetted.
### 2. If You Are a B2C Company (High Volume, E-commerce, Retail)
*If you are selling to millions of consumers, you need a platform built for massive scale, transactional emails, and cross-channel journeys (SMS, push notifications).*
* **Salesforce Marketing Cloud (SFMC - formerly ExactTarget)**
* **The Procurement View:** An absolute powerhouse for B2C. It can handle billions of sends.
* **Cost Structure:** Highly modular. You pay for "studios" (Email Studio, Mobile Studio) and utilization (Super Messages).
* **Vendor Risk/TCO:** **Warning on TCO.** The license cost is only the beginning. Implementation is notoriously complex, and you should budget at least 1x to 1.5x the software cost for implementation and ongoing managed services.
* **Braze** or **Iterable**
* **The Procurement View:** These are the modern, agile alternatives to legacy Salesforce/Adobe. They are built for modern data architectures (e.g., pulling directly from Snowflake or AWS).
* **Cost Structure:** Usually based on Monthly Active Users (MAUs) and message volume.
* **Vendor Risk/TCO:** They require a high level of data maturity. If your enterprise data is messy, these platforms will struggle. However, they boast much faster deployment times than Salesforce or Adobe, leading to quicker ROI.
### 3. The "Fast Time-to-Value" Enterprise Contender
* **HubSpot (Enterprise Hub)**
* **The Procurement View:** Historically seen as an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies, particularly if you want to replace multiple tools (CRM, Email, CMS, Service) with one vendor.
* **Cost Structure:** Marketing contacts + seat licenses. Very transparent pricing compared to Adobe and Salesforce.
* **Vendor Risk/TCO:** The lowest TCO on this list. It is incredibly user-friendly, meaning marketing teams actually adopt it without requiring expensive external consultants or specialized IT support.
---
### The Procurement Evaluation Checklist (What to demand in the RFP)
When you take this to market, do not let Marketing drive the selection based purely on UI. Ensure your RFP mandates the following:
1. **Security & Compliance:** Since you are a US enterprise, demand proof of **SOC 2 Type II compliance**, strict **CCPA/CPRA** data deletion workflows, and **HIPAA** compliance (if you touch healthcare).
2. **Governance & Access:** At 5,000 employees, you will likely have dozens of marketers across different regions or business units. The tool *must* have robust Role-Based Access Control (RBAC) and Single Sign-On (SSO/SAML) capabilities.
3. **Hidden Overage Fees:** Pay close attention to how they charge for overages. Are you billed for "contacts in the database" (even if you don't email them) or "sends"? If Marketing runs a massive holiday campaign, will you get hit with a 20% penalty fee for exceeding your sending limits?
4. **Dedicated IP and Deliverability:** At enterprise scale, you must send from dedicated IP addresses. Demand guarantees around deliverability support and IP warming as part of the onboarding SLA.
### Your Immediate Next Steps
1. **Define the Tech Stack:** Ask your CIO/CTO what your core CRM and Data Warehouse are (e.g., Salesforce, Microsoft Dynamics, Snowflake). The email platform *must* have native integrations with these, or your integration costs will skyrocket.
2. **Align with the CMO:** Are they optimizing for B2B lead generation, or B2C mass-market personalization?
3. **Shortlist 3 Vendors:** Based on the above, typically a standard enterprise RFP looks like: **Salesforce vs. Adobe vs. HubSpot** (for B2B/Hybrid) OR **Salesforce vs. Braze vs. Iterable** (for B2C).

End of response · 54 lines · 5,651 characters · sha256 db544ecc3e1e8c289de39c0ae7af0fa5556a866e811e94cedcda0919f883e689

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-18 09:00:22 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
7 prior days with data; 3 matched the baseline value (2026-09-14, 2026-09-15, 2026-09-17)
Responses
deviation 2026-09-17 09:28:19 UTC, run 20260917T090022Z-136fc3; reverted 2026-09-18 09:31:35 UTC, run 20260918T090022Z-92b6f7
Detected
2026-09-18 · run 20260918T090022Z-92b6f7, started 2026-09-18 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-18/P29/google/gemini-3.1-pro-preview/transient_anomaly/picks.top/
Event ID
2026-09-18:P29:google/gemini-3.1-pro-preview:transient_anomaly:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.