MODELDRIFT.WATCH

Sep 21 · Day 12

MINOR · GEMINI · QUESTION 33

Gemini named a different first choice for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 18beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Braze
  4. Fast Time-to-Value" Enterprise Contender
  5. HubSpot

As a VP of Procurement for a 5,000-employee enterprise, your evaluation of an email marketing platform will be fundamentally different from that of a CMO. While Marketing is looking at features (A/B testing, AI copy generation, templates), your focus is on Total Cost of Ownership (TCO), scalability, security/compliance, vendor consolidation, and integration capability.

At your size, "email marketing" usually falls into two distinct categories. The right tool depends heavily on your company's business model.

Here is a breakdown of the enterprise-grade platforms you should consider for your RFP, categorized by business model and viewed through a procurement lens.


1. If You Are a B2B Company (Long Sales Cycles, Lead Scoring)

If your 5,000 employees are selling software, manufacturing, consulting, or medical devices, you need a "Marketing Automation Platform" (MAP).

  • •Adobe Marketo Engage
  • •The Procurement View: Marketo is the gold standard for enterprise B2B. It handles complex lead scoring and routing seamlessly.
  • •Cost Structure: Priced by database size (number of contacts), not email volume.
  • •Vendor Risk/TCO: High implementation costs. You will likely need to hire a specialized agency or dedicated internal Marketo architect. However, it is highly stable and integrates deeply with Salesforce CRM.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot)
  • •The Procurement View: If your enterprise already uses Salesforce as its CRM, this is usually the easiest procurement win. Consolidating under one Master Services Agreement (MSA) with Salesforce can give you leverage for volume discounting.
  • •Cost Structure: Tiered user licensing and database size.
  • •Vendor Risk/TCO: Predictable. IT and Legal usually approve it quickly because Salesforce’s security and compliance architecture is already vetted.

2. If You Are a B2C Company (High Volume, E-commerce, Retail)

If you are selling to millions of consumers, you need a platform built for massive scale, transactional emails, and cross-channel journeys (SMS, push notifications).

  • •Salesforce Marketing Cloud (SFMC - formerly ExactTarget)
  • •The Procurement View: An absolute powerhouse for B2C. It can handle billions of sends.
  • •Cost Structure: Highly modular. You pay for "studios" (Email Studio, Mobile Studio) and utilization (Super Messages).
  • •Vendor Risk/TCO: Warning on TCO. The license cost is only the beginning. Implementation is notoriously complex, and you should budget at least 1x to 1.5x the software cost for implementation and ongoing managed services.
  • •Braze or Iterable
  • •The Procurement View: These are the modern, agile alternatives to legacy Salesforce/Adobe. They are built for modern data architectures (e.g., pulling directly from Snowflake or AWS).
  • •Cost Structure: Usually based on Monthly Active Users (MAUs) and message volume.
  • •Vendor Risk/TCO: They require a high level of data maturity. If your enterprise data is messy, these platforms will struggle. However, they boast much faster deployment times than Salesforce or Adobe, leading to quicker ROI.

3. The "Fast Time-to-Value" Enterprise Contender

  • •HubSpot (Enterprise Hub)
  • •The Procurement View: Historically seen as an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies, particularly if you want to replace multiple tools (CRM, Email, CMS, Service) with one vendor.
  • •Cost Structure: Marketing contacts + seat licenses. Very transparent pricing compared to Adobe and Salesforce.
  • •Vendor Risk/TCO: The lowest TCO on this list. It is incredibly user-friendly, meaning marketing teams actually adopt it without requiring expensive external consultants or specialized IT support.

The Procurement Evaluation Checklist (What to demand in the RFP)

When you take this to market, do not let Marketing drive the selection based purely on UI. Ensure your RFP mandates the following:

  • 1.Security & Compliance: Since you are a US enterprise, demand proof of SOC 2 Type II compliance, strict CCPA/CPRA data deletion workflows, and HIPAA compliance (if you touch healthcare).
  • 2.Governance & Access: At 5,000 employees, you will likely have dozens of marketers across different regions or business units. The tool must have robust Role-Based Access Control (RBAC) and Single Sign-On (SSO/SAML) capabilities.
  • 3.Hidden Overage Fees: Pay close attention to how they charge for overages. Are you billed for "contacts in the database" (even if you don't email them) or "sends"? If Marketing runs a massive holiday campaign, will you get hit with a 20% penalty fee for exceeding your sending limits?
  • 4.Dedicated IP and Deliverability: At enterprise scale, you must send from dedicated IP addresses. Demand guarantees around deliverability support and IP warming as part of the onboarding SLA.

Your Immediate Next Steps

  • 1.Define the Tech Stack: Ask your CIO/CTO what your core CRM and Data Warehouse are (e.g., Salesforce, Microsoft Dynamics, Snowflake). The email platform must have native integrations with these, or your integration costs will skyrocket.
  • 2.Align with the CMO: Are they optimizing for B2B lead generation, or B2C mass-market personalization?
  • 3.Shortlist 3 Vendors: Based on the above, typically a standard enterprise RFP looks like: Salesforce vs. Adobe vs. HubSpot (for B2B/Hybrid) OR Salesforce vs. Braze vs. Iterable (for B2C).

SEP 21afterPermalink

Picks read from the answer, in order
  1. Braze
  2. Are you B2B or B2C?

As a VP of Procurement at a 5,000-employee enterprise, your evaluation criteria are fundamentally different from those of your CMO. While your marketing team is looking at automation features and drag-and-drop builders, you need to focus on Total Cost of Ownership (TCO), compliance (SOC 2, CCPA, GDPR), integration stability with your existing CRM/ERP, vendor consolidation, and pricing predictability.

At 5,000 employees, you are firmly in the enterprise tier. You should not be looking at SMB tools like Mailchimp or Constant Contact.

To give you the best recommendation, the market must be split based on your company’s business model. Here is the executive breakdown of the platforms you should put on your RFP short-list.


1. If you are a B2B Enterprise (Selling to other businesses)

B2B marketing requires long sales cycles, complex lead scoring, and deep integration with CRM systems (likely Salesforce or Microsoft Dynamics).

  • •Adobe Marketo Engage:
  • •The Vibe: The gold standard for complex B2B enterprise marketing.
  • •Procurement view: Pricing is based on database size (number of contacts). It is highly robust, but TCO is high. You will need to budget for specialized, certified Marketo administrators on your payroll, plus a 3- to 6-month agency implementation.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot):
  • •The Vibe: The default choice if your global sales team already uses Salesforce CRM.
  • •Procurement view: Offers the path of least resistance for IT and legal if you already have a Master Services Agreement (MSA) with Salesforce. However, Salesforce is notorious for modular pricing (charging extra for specific features). Be prepared to negotiate hard on annual uplifts.
  • •HubSpot (Enterprise Tier):
  • •The Vibe: The modern, high-adoption alternative. Historically an SMB tool, their Enterprise tier is now highly competitive.
  • •Procurement view: Lowest TCO. HubSpot requires far less technical headcount to manage than Adobe or Salesforce. Pricing is highly transparent (Marketing Contacts + Seat Licenses). Implementation is significantly faster and cheaper.

2. If you are a B2C Enterprise (Selling to consumers)

B2C marketing requires massive data volume, split-second transactional emails, SMS, push notifications, and high-volume journeys.

  • •Salesforce Marketing Cloud (formerly ExactTarget):
  • •The Vibe: The enterprise B2C behemoth.
  • •Procurement view: Extremely powerful, but a very complex pricing model (based on "Super Messages" or volume of sends/SMS). It requires a massive implementation budget and highly technical staff. If you already spend millions with Salesforce, you can leverage your overall spend for discounts here.
  • •Braze or Iterable:
  • •The Vibe: The modern tech-stack leaders for cross-channel consumer engagement. If your company has a mobile app or digital product, marketing will want one of these.
  • •Procurement view: Highly scalable, usage-based pricing. They play very nicely with modern cloud data warehouses (like Snowflake or AWS Redshift). If you have cloud-commit agreements (like AWS EDP or Google Cloud commits), you can often buy Braze/Iterable through those cloud marketplaces to burn down your commit.

The Procurement Strategy: Where the hidden costs are

When you write the RFP and evaluate these vendors, do not just look at the software licensing fee. Enterprise marketing automation has a massive "Iceberg Effect" regarding TCO.

As the VP of Procurement, force your marketing and IT teams to answer these questions before you sign:

  • 1.Implementation Costs: A $150,000 annual Salesforce or Adobe contract usually requires a $75,000–$150,000 upfront payment to a third-party implementation agency (like Deloitte, Slalom, or a boutique agency) to set it up. HubSpot and Braze typically have lower implementation costs.
  • 2.Headcount/Talent Costs: Will marketing need to hire a $120,000/year "Platform Architect" just to run the software? (For Marketo and Salesforce Marketing Cloud, the answer is usually yes).
  • 3.Pricing Model Traps: Are you being billed on all contacts in your database, or just marketable contacts? (You don't want to pay email marketing fees for a database of unsubscribed customers or internal employee email addresses).
  • 4.Vendor Consolidation: Does buying a platform like HubSpot or Salesforce allow you to retire 3 or 4 smaller shadow-IT tools (like specialized survey tools, landing page builders, or social media schedulers)?

My Recommendation for your next step:

To narrow this down, I would need to know:

  • 1.Are you B2B or B2C?
  • 2.What CRM does your sales team currently use? (e.g., Salesforce, Dynamics, Custom).
  • 3.Roughly how many contacts are in your database? (e.g., 500,000 vs. 50 million).

If you answer those three questions, I can tell you exactly which two platforms you should pit against each other in an RFP to get the best leverage.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:31:35 UTC
Run
20260918T090022Z-92b6f7
Served by
Google AI Studio · HTTP 200 · finish stop
Size
5,651 characters · 2,473 tokens out incl. hidden reasoning · 35.6 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Braze
  4. Fast Time-to-Value" Enterprise Contender
  5. HubSpot

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

As a VP of Procurement for a 5,000-employee enterprise, your evaluation of an email marketing platform will be fundamentally different from that of a CMO. While Marketing is looking at features (A/B testing, AI copy generation, templates), your focus is on **Total Cost of Ownership (TCO), scalability, security/compliance, vendor consolidation, and integration capability.**
At your size, "email marketing" usually falls into two distinct categories. The right tool depends heavily on your company's business model.
Here is a breakdown of the enterprise-grade platforms you should consider for your RFP, categorized by business model and viewed through a procurement lens.
---
### 1. If You Are a B2B Company (Long Sales Cycles, Lead Scoring)
*If your 5,000 employees are selling software, manufacturing, consulting, or medical devices, you need a "Marketing Automation Platform" (MAP).*
* **Adobe Marketo Engage**
* **The Procurement View:** Marketo is the gold standard for enterprise B2B. It handles complex lead scoring and routing seamlessly.
* **Cost Structure:** Priced by database size (number of contacts), not email volume.
* **Vendor Risk/TCO:** High implementation costs. You will likely need to hire a specialized agency or dedicated internal Marketo architect. However, it is highly stable and integrates deeply with Salesforce CRM.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot)**
* **The Procurement View:** If your enterprise already uses Salesforce as its CRM, this is usually the easiest procurement win. Consolidating under one Master Services Agreement (MSA) with Salesforce can give you leverage for volume discounting.
* **Cost Structure:** Tiered user licensing and database size.
* **Vendor Risk/TCO:** Predictable. IT and Legal usually approve it quickly because Salesforce’s security and compliance architecture is already vetted.
### 2. If You Are a B2C Company (High Volume, E-commerce, Retail)
*If you are selling to millions of consumers, you need a platform built for massive scale, transactional emails, and cross-channel journeys (SMS, push notifications).*
* **Salesforce Marketing Cloud (SFMC - formerly ExactTarget)**
* **The Procurement View:** An absolute powerhouse for B2C. It can handle billions of sends.
* **Cost Structure:** Highly modular. You pay for "studios" (Email Studio, Mobile Studio) and utilization (Super Messages).
* **Vendor Risk/TCO:** **Warning on TCO.** The license cost is only the beginning. Implementation is notoriously complex, and you should budget at least 1x to 1.5x the software cost for implementation and ongoing managed services.
* **Braze** or **Iterable**
* **The Procurement View:** These are the modern, agile alternatives to legacy Salesforce/Adobe. They are built for modern data architectures (e.g., pulling directly from Snowflake or AWS).
* **Cost Structure:** Usually based on Monthly Active Users (MAUs) and message volume.
* **Vendor Risk/TCO:** They require a high level of data maturity. If your enterprise data is messy, these platforms will struggle. However, they boast much faster deployment times than Salesforce or Adobe, leading to quicker ROI.
### 3. The "Fast Time-to-Value" Enterprise Contender
* **HubSpot (Enterprise Hub)**
* **The Procurement View:** Historically seen as an SMB tool, HubSpot’s Enterprise tier is now a legitimate player for 5,000-person companies, particularly if you want to replace multiple tools (CRM, Email, CMS, Service) with one vendor.
* **Cost Structure:** Marketing contacts + seat licenses. Very transparent pricing compared to Adobe and Salesforce.
* **Vendor Risk/TCO:** The lowest TCO on this list. It is incredibly user-friendly, meaning marketing teams actually adopt it without requiring expensive external consultants or specialized IT support.
---
### The Procurement Evaluation Checklist (What to demand in the RFP)
When you take this to market, do not let Marketing drive the selection based purely on UI. Ensure your RFP mandates the following:
1. **Security & Compliance:** Since you are a US enterprise, demand proof of **SOC 2 Type II compliance**, strict **CCPA/CPRA** data deletion workflows, and **HIPAA** compliance (if you touch healthcare).
2. **Governance & Access:** At 5,000 employees, you will likely have dozens of marketers across different regions or business units. The tool *must* have robust Role-Based Access Control (RBAC) and Single Sign-On (SSO/SAML) capabilities.
3. **Hidden Overage Fees:** Pay close attention to how they charge for overages. Are you billed for "contacts in the database" (even if you don't email them) or "sends"? If Marketing runs a massive holiday campaign, will you get hit with a 20% penalty fee for exceeding your sending limits?
4. **Dedicated IP and Deliverability:** At enterprise scale, you must send from dedicated IP addresses. Demand guarantees around deliverability support and IP warming as part of the onboarding SLA.
### Your Immediate Next Steps
1. **Define the Tech Stack:** Ask your CIO/CTO what your core CRM and Data Warehouse are (e.g., Salesforce, Microsoft Dynamics, Snowflake). The email platform *must* have native integrations with these, or your integration costs will skyrocket.
2. **Align with the CMO:** Are they optimizing for B2B lead generation, or B2C mass-market personalization?
3. **Shortlist 3 Vendors:** Based on the above, typically a standard enterprise RFP looks like: **Salesforce vs. Adobe vs. HubSpot** (for B2B/Hybrid) OR **Salesforce vs. Braze vs. Iterable** (for B2C).

End of response · 54 lines · 5,651 characters · sha256 db544ecc3e1e8c289de39c0ae7af0fa5556a866e811e94cedcda0919f883e689

After this run

2026-09-21 permalink

Scored
answered
Received
2026-09-21 09:28:12 UTC
Run
20260921T090021Z-989a7c
Served by
Google · HTTP 200 · finish stop
Size
5,091 characters · 2,347 tokens out incl. hidden reasoning · 20.9 s
Extracted picks, in order · confidence medium
  1. Braze
  2. Are you B2B or B2C?

Highlighted: the names the extractor took as picks, where they appear verbatim (2 of 2).

As a VP of Procurement at a 5,000-employee enterprise, your evaluation criteria are fundamentally different from those of your CMO. While your marketing team is looking at automation features and drag-and-drop builders, you need to focus on **Total Cost of Ownership (TCO), compliance (SOC 2, CCPA, GDPR), integration stability with your existing CRM/ERP, vendor consolidation, and pricing predictability.**
At 5,000 employees, you are firmly in the enterprise tier. You should not be looking at SMB tools like Mailchimp or Constant Contact.
To give you the best recommendation, the market must be split based on your company’s business model. Here is the executive breakdown of the platforms you should put on your RFP short-list.
---
### 1. If you are a B2B Enterprise (Selling to other businesses)
B2B marketing requires long sales cycles, complex lead scoring, and deep integration with CRM systems (likely Salesforce or Microsoft Dynamics).
* **Adobe Marketo Engage:**
* **The Vibe:** The gold standard for complex B2B enterprise marketing.
* **Procurement view:** Pricing is based on database size (number of contacts). It is highly robust, but **TCO is high**. You will need to budget for specialized, certified Marketo administrators on your payroll, plus a 3- to 6-month agency implementation.
* **Salesforce Marketing Cloud Account Engagement (formerly Pardot):**
* **The Vibe:** The default choice if your global sales team already uses Salesforce CRM.
* **Procurement view:** Offers the path of least resistance for IT and legal if you already have a Master Services Agreement (MSA) with Salesforce. However, Salesforce is notorious for modular pricing (charging extra for specific features). Be prepared to negotiate hard on annual uplifts.
* **HubSpot (Enterprise Tier):**
* **The Vibe:** The modern, high-adoption alternative. Historically an SMB tool, their Enterprise tier is now highly competitive.
* **Procurement view:** **Lowest TCO.** HubSpot requires far less technical headcount to manage than Adobe or Salesforce. Pricing is highly transparent (Marketing Contacts + Seat Licenses). Implementation is significantly faster and cheaper.
### 2. If you are a B2C Enterprise (Selling to consumers)
B2C marketing requires massive data volume, split-second transactional emails, SMS, push notifications, and high-volume journeys.
* **Salesforce Marketing Cloud (formerly ExactTarget):**
* **The Vibe:** The enterprise B2C behemoth.
* **Procurement view:** Extremely powerful, but a very complex pricing model (based on "Super Messages" or volume of sends/SMS). It requires a massive implementation budget and highly technical staff. If you already spend millions with Salesforce, you can leverage your overall spend for discounts here.
* **Braze** or **Iterable:**
* **The Vibe:** The modern tech-stack leaders for cross-channel consumer engagement. If your company has a mobile app or digital product, marketing will want one of these.
* **Procurement view:** Highly scalable, usage-based pricing. They play very nicely with modern cloud data warehouses (like Snowflake or AWS Redshift). If you have cloud-commit agreements (like AWS EDP or Google Cloud commits), you can often buy Braze/Iterable through those cloud marketplaces to burn down your commit.
---
### The Procurement Strategy: Where the hidden costs are
When you write the RFP and evaluate these vendors, do not just look at the software licensing fee. Enterprise marketing automation has a massive "Iceberg Effect" regarding TCO.
As the VP of Procurement, force your marketing and IT teams to answer these questions before you sign:
1. **Implementation Costs:** A $150,000 annual Salesforce or Adobe contract usually requires a $75,000–$150,000 upfront payment to a third-party implementation agency (like Deloitte, Slalom, or a boutique agency) to set it up. *HubSpot and Braze typically have lower implementation costs.*
2. **Headcount/Talent Costs:** Will marketing need to hire a $120,000/year "Platform Architect" just to run the software? (For Marketo and Salesforce Marketing Cloud, the answer is usually yes).
3. **Pricing Model Traps:** Are you being billed on *all* contacts in your database, or just *marketable* contacts? (You don't want to pay email marketing fees for a database of unsubscribed customers or internal employee email addresses).
4. **Vendor Consolidation:** Does buying a platform like HubSpot or Salesforce allow you to retire 3 or 4 smaller shadow-IT tools (like specialized survey tools, landing page builders, or social media schedulers)?
### My Recommendation for your next step:
To narrow this down, I would need to know:
1. **Are you B2B or B2C?**
2. **What CRM does your sales team currently use?** (e.g., Salesforce, Dynamics, Custom).
3. **Roughly how many contacts are in your database?** (e.g., 500,000 vs. 50 million).
If you answer those three questions, I can tell you exactly which two platforms you should pit against each other in an RFP to get the best leverage.

End of response · 51 lines · 5,091 characters · sha256 cd3a2cf9b45e2f323e7512bf041d59e6271a56226372495e5149793029c06174

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-21 09:00:20 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
7 prior days with data; 3 matched the baseline value (2026-09-14, 2026-09-15, 2026-09-18)
Responses
before 2026-09-18 09:31:35 UTC, run 20260918T090022Z-92b6f7; after 2026-09-21 09:28:12 UTC, run 20260921T090021Z-989a7c
Detected
2026-09-21 · run 20260921T090021Z-989a7c, started 2026-09-21 09:00:21 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-21/P29/google/gemini-3.1-pro-preview/pick_change/picks.top/
Event ID
2026-09-21:P29:google/gemini-3.1-pro-preview:pick_change:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.