MODELDRIFT.WATCH

Sep 18 · Day 9

MINOR · CLAUDE · QUESTION 33

Claude changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 17beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Transactional/API
  3. Salesforce Marketing Cloud Account Engagement
  4. HubSpot Marketing Hub Enterprise

Short answer: for a 5,000-person US enterprise, the realistic shortlist is Adobe Marketo Engage, Salesforce Marketing Cloud Account Engagement, or HubSpot Marketing Hub Enterprise if this is B2B — and Braze, Iterable, or Salesforce Marketing Cloud Engagement if it's B2C/high-volume consumer messaging. But the fork below determines everything, and as procurement your leverage is mostly in the contract structure, not the vendor choice.

First, pin down which problem you're solving

These are four different markets that all get called "email marketing":

Use caseWhat it looks likeVendors
B2B demand genLead scoring, CRM sync, nurture tracks, sales handoffMarketo, Account Engagement (Pardot), HubSpot, Eloqua
B2C lifecycleMillions of sends, behavioral triggers, push/SMS/in-appBraze, Iterable, Klaviyo, SFMC Engagement, Adobe Journey Optimizer
Transactional/APIPassword resets, receipts — dev-owned, not marketingSendGrid, AWS SES, Postmark
Internal commsEmployee newsletters to your 5,000 staffStaffbase, Firstup, Poppulo

If your marketing stakeholder hasn't written this down, that's the first thing to force before an RFP.

B2B recommendations

Marketo Engage is still the enterprise default. Most powerful automation and multi-touch attribution, works with any CRM. Downside: genuinely complex, requires 1–3 dedicated marketing ops FTEs, and Adobe's pricing is opaque and database-size-driven.

Salesforce Marketing Cloud Account Engagement — only compelling if you're deeply committed to Salesforce. Native integration is the whole value prop; feature depth is materially below Marketo. Watch out that it's a different product line from SFMC Engagement, with different architecture.

HubSpot Marketing Hub Enterprise — has become legitimately credible upmarket. Far better UX, much faster implementation (weeks vs. months), single platform if you also want CMS/service. The catch is "marketing contacts" pricing that escalates aggressively as the database grows.

Skip Oracle Eloqua unless you're already locked into the Oracle stack. Declining investment and shrinking partner ecosystem.

The procurement-specific playbook

This is where you'll actually save money:

Understand the pricing metric, because it's where renewal shock happens. Contact-based pricing (Marketo, HubSpot) means your bill grows as marketing does its job. Negotiate overage rates in the initial contract, not at renewal, and get a price-protection cap on renewal — 3–5% or CPI, not "then-current list price."

Budget for implementation at 0.5–1.5× year-one license for Marketo or SFMC. A systems integrator is effectively mandatory. HubSpot is the outlier where self-implementation is realistic.

Kill evergreen auto-renewal and get the notice window to 60–90 days. Email platforms are notorious for this.

Get a data egress clause. These systems hold your suppression lists, engagement history, and unsubscribe records — data you legally need and that's painful to extract. Specify format and timeline.

Time it against vendor fiscal year-end. Salesforce closes January 31, Adobe November 30, HubSpot December 31. Q4 of their fiscal year is your best discount window.

Security review checklist: SOC 2 Type II, ISO 27001, DPA with SCCs if any EU data, subprocessor list, pen test summary, SSO/SAML and SCIM provisioning. For email specifically, also confirm dedicated IP availability and DKIM/SPF/DMARC alignment support.

Multi-year gets you 15–25% but locks you in before you know whether adoption sticks. A two-year deal with a ramped year one is usually the better trade.

The thing worth checking before you buy anything

At 5,000 employees you very likely already have three to six overlapping email tools scattered across business units — a Mailchimp here, a legacy Pardot instance there, someone's SendGrid account on a corporate card. Running a discovery pass on current spend before the RFP frequently surfaces more savings through consolidation than you'll get from negotiating the new contract. It also strengthens your negotiating position, since you can credibly commit to a larger consolidated volume.

If you can tell me whether this is B2B or B2C, what CRM you're on, and whether there's an incumbent you're replacing, I can narrow this to a specific recommendation and sketch out what the evaluation criteria and weighting should look like.

SEP 18afterPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Oracle Eloqua
  5. Braze
  6. Iterable
  7. Klaviyo
  8. Contact/profile-based
  9. MAU-based

Short answer: there's no single right pick — the answer forks hard on one question. But here's how I'd frame it, plus the procurement-specific landmines that matter more than the feature comparison.

The question that determines everything

Are you buying for B2B demand gen or B2C/consumer lifecycle messaging? These are different product categories with different vendors, pricing models, and cost curves. Related: what CRM you're standardized on, and whether you already have a Salesforce or Adobe enterprise agreement (that usually dominates the decision economically).

Rough shortlists

B2B demand gen (lead scoring, nurture, MQL handoff to sales):

  • •Adobe Marketo Engage — the default for complex B2B at your size. Deep, but needs a skilled admin or agency; not self-serve.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot) — the obvious choice if you're a Salesforce shop. Less powerful than Marketo, much less integration friction.
  • •HubSpot Marketing Hub Enterprise — faster time-to-value, genuinely liked by marketers, but gets expensive at large contact volumes and is weaker on complex multi-entity orgs.
  • •Oracle Eloqua — still strong for heavily regulated or approval-heavy environments; smaller talent pool.

B2C / consumer lifecycle (high-volume, cross-channel, mobile):

  • •Braze — strong if push/in-app matter as much as email.
  • •Iterable — similar space, often more flexible commercially.
  • •Klaviyo — if this is ecommerce/DTC-driven.
  • •Salesforce Marketing Cloud Engagement or Adobe Journey Optimizer — if you're consolidating onto an existing ELA and have real data engineering capacity.

Transactional email (receipts, password resets) is a separate buy — SendGrid, Amazon SES, Postmark. Don't let a marketing platform vendor bundle this in at marketing-tier rates.

What I'd actually focus on as procurement

1. Audit before you buy. At 5,000 people you almost certainly have 4–10 email tools already — Mailchimp on a regional marketing card, a Constant Contact instance in HR, whatever the events team uses. Consolidation is usually a bigger win than platform selection, and the shadow-IT inventory is also your compliance exposure. Pull the expense data first.

2. Understand the pricing metric, not the price. This is where the money leaks:

  • •Contact/profile-based (Marketo, HubSpot, Klaviyo): costs grow with database size whether or not you email those people. Negotiate on marketable or active contacts, define the term precisely, and get an annual true-down right — not just true-up.
  • •Send/message volume (SFMC, some B2C tools): watch overage rates. Pre-negotiate tiered overage pricing rather than accepting list rates when you blow through.
  • •MAU-based (Braze): make sure the MAU definition matches how you'll actually use it.

3. Budget for implementation honestly. For Marketo, SFMC, or Adobe, systems-integrator fees commonly run 0.5–2x year-one license. If the business case only pencils out on license cost, it doesn't pencil out.

4. Check what's a separate SKU. Sandboxes, dedicated IPs, deliverability consulting, SMS, additional business units, API call tiers, premium support — these are routinely carved out and quoted later.

5. Contract terms worth fighting for:

  • •Cap renewal uplift (aim 3–5%) for the full term, not just year two
  • •Price hold on additional volume purchased mid-term
  • •Data export rights and transition assistance on exit, in writing
  • •DPA, SOC 2 Type II, subprocessor notification; BAA if you touch PHI
  • •If you ramp, tie payment to actual ramp, not a flat average

6. Time it. Salesforce's fiscal year ends Jan 31, Adobe's around Nov 30. Quarter- and year-end give you real leverage. Run a genuine competitive process even if you know where you'll land — a credible alternative is worth more than any negotiation tactic.

7. Be wary of ELA bundling. Rolling this into an existing Salesforce or Adobe agreement often gets a striking discount, but it consolidates your renewal exposure and kills future leverage. Sometimes still the right call — just price the optionality you're giving up.

What I'd need to give you a real recommendation

  • •B2B, B2C, or both?
  • •Current CRM and whether there's an existing Salesforce/Adobe ELA
  • •Approximate contact database size and monthly send volume
  • •In-house marketing ops headcount (this rules out or rules in Marketo more than anything else)
  • •Regulated industry? (healthcare, financial services, or EU/global data residency needs)
  • •Rough budget envelope

One caveat: pricing and packaging in this category change frequently, and vendors rebrand tiers constantly — treat any numbers above as directional and verify against current quotes.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-17 permalink

Scored
answered
Received
2026-09-17 09:27:42 UTC
Run
20260917T090022Z-136fc3
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
4,494 characters · 3,755 tokens out incl. hidden reasoning · 56.1 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Transactional/API
  3. Salesforce Marketing Cloud Account Engagement
  4. HubSpot Marketing Hub Enterprise

Highlighted: the names the extractor took as picks, where they appear verbatim (4 of 4).

Short answer: for a 5,000-person US enterprise, the realistic shortlist is **Adobe Marketo Engage, Salesforce Marketing Cloud Account Engagement, or HubSpot Marketing Hub Enterprise** if this is B2B — and **Braze, Iterable, or Salesforce Marketing Cloud Engagement** if it's B2C/high-volume consumer messaging. But the fork below determines everything, and as procurement your leverage is mostly in the contract structure, not the vendor choice.
## First, pin down which problem you're solving
These are four different markets that all get called "email marketing":
| Use case | What it looks like | Vendors |
|---|---|---|
| **B2B demand gen** | Lead scoring, CRM sync, nurture tracks, sales handoff | Marketo, Account Engagement (Pardot), HubSpot, Eloqua |
| **B2C lifecycle** | Millions of sends, behavioral triggers, push/SMS/in-app | Braze, Iterable, Klaviyo, SFMC Engagement, Adobe Journey Optimizer |
| **Transactional/API** | Password resets, receipts — dev-owned, not marketing | SendGrid, AWS SES, Postmark |
| **Internal comms** | Employee newsletters to your 5,000 staff | Staffbase, Firstup, Poppulo |
If your marketing stakeholder hasn't written this down, that's the first thing to force before an RFP.
## B2B recommendations
**Marketo Engage** is still the enterprise default. Most powerful automation and multi-touch attribution, works with any CRM. Downside: genuinely complex, requires 1–3 dedicated marketing ops FTEs, and Adobe's pricing is opaque and database-size-driven.
**Salesforce Marketing Cloud Account Engagement** — only compelling if you're deeply committed to Salesforce. Native integration is the whole value prop; feature depth is materially below Marketo. Watch out that it's a different product line from SFMC Engagement, with different architecture.
**HubSpot Marketing Hub Enterprise** — has become legitimately credible upmarket. Far better UX, much faster implementation (weeks vs. months), single platform if you also want CMS/service. The catch is "marketing contacts" pricing that escalates aggressively as the database grows.
**Skip Oracle Eloqua** unless you're already locked into the Oracle stack. Declining investment and shrinking partner ecosystem.
## The procurement-specific playbook
This is where you'll actually save money:
**Understand the pricing metric, because it's where renewal shock happens.** Contact-based pricing (Marketo, HubSpot) means your bill grows as marketing does its job. Negotiate overage rates *in the initial contract*, not at renewal, and get a price-protection cap on renewal — 3–5% or CPI, not "then-current list price."
**Budget for implementation at 0.5–1.5× year-one license** for Marketo or SFMC. A systems integrator is effectively mandatory. HubSpot is the outlier where self-implementation is realistic.
**Kill evergreen auto-renewal** and get the notice window to 60–90 days. Email platforms are notorious for this.
**Get a data egress clause.** These systems hold your suppression lists, engagement history, and unsubscribe records — data you legally need and that's painful to extract. Specify format and timeline.
**Time it against vendor fiscal year-end.** Salesforce closes January 31, Adobe November 30, HubSpot December 31. Q4 of their fiscal year is your best discount window.
**Security review checklist:** SOC 2 Type II, ISO 27001, DPA with SCCs if any EU data, subprocessor list, pen test summary, SSO/SAML and SCIM provisioning. For email specifically, also confirm dedicated IP availability and DKIM/SPF/DMARC alignment support.
**Multi-year gets you 15–25%** but locks you in before you know whether adoption sticks. A two-year deal with a ramped year one is usually the better trade.
## The thing worth checking before you buy anything
At 5,000 employees you very likely already have **three to six** overlapping email tools scattered across business units — a Mailchimp here, a legacy Pardot instance there, someone's SendGrid account on a corporate card. Running a discovery pass on current spend before the RFP frequently surfaces more savings through consolidation than you'll get from negotiating the new contract. It also strengthens your negotiating position, since you can credibly commit to a larger consolidated volume.
If you can tell me whether this is B2B or B2C, what CRM you're on, and whether there's an incumbent you're replacing, I can narrow this to a specific recommendation and sketch out what the evaluation criteria and weighting should look like.

End of response · 48 lines · 4,494 characters · sha256 3d83affc60de36e83d4233372450b8cd72509933f127e581ff5ef68e48f4f071

After this run

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:30:59 UTC
Run
20260918T090022Z-92b6f7
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
4,768 characters · 3,452 tokens out incl. hidden reasoning · 51.8 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Oracle Eloqua
  5. Braze
  6. Iterable
  7. Klaviyo
  8. Contact/profile-based
  9. MAU-based

Highlighted: the names the extractor took as picks, where they appear verbatim (9 of 9).

Short answer: there's no single right pick — the answer forks hard on one question. But here's how I'd frame it, plus the procurement-specific landmines that matter more than the feature comparison.
## The question that determines everything
**Are you buying for B2B demand gen or B2C/consumer lifecycle messaging?** These are different product categories with different vendors, pricing models, and cost curves. Related: what CRM you're standardized on, and whether you already have a Salesforce or Adobe enterprise agreement (that usually dominates the decision economically).
## Rough shortlists
**B2B demand gen** (lead scoring, nurture, MQL handoff to sales):
- **Adobe Marketo Engage** — the default for complex B2B at your size. Deep, but needs a skilled admin or agency; not self-serve.
- **Salesforce Marketing Cloud Account Engagement** (formerly Pardot) — the obvious choice if you're a Salesforce shop. Less powerful than Marketo, much less integration friction.
- **HubSpot Marketing Hub Enterprise** — faster time-to-value, genuinely liked by marketers, but gets expensive at large contact volumes and is weaker on complex multi-entity orgs.
- **Oracle Eloqua** — still strong for heavily regulated or approval-heavy environments; smaller talent pool.
**B2C / consumer lifecycle** (high-volume, cross-channel, mobile):
- **Braze** — strong if push/in-app matter as much as email.
- **Iterable** — similar space, often more flexible commercially.
- **Klaviyo** — if this is ecommerce/DTC-driven.
- **Salesforce Marketing Cloud Engagement** or **Adobe Journey Optimizer** — if you're consolidating onto an existing ELA and have real data engineering capacity.
**Transactional email** (receipts, password resets) is a separate buy — SendGrid, Amazon SES, Postmark. Don't let a marketing platform vendor bundle this in at marketing-tier rates.
## What I'd actually focus on as procurement
**1. Audit before you buy.** At 5,000 people you almost certainly have 4–10 email tools already — Mailchimp on a regional marketing card, a Constant Contact instance in HR, whatever the events team uses. Consolidation is usually a bigger win than platform selection, and the shadow-IT inventory is also your compliance exposure. Pull the expense data first.
**2. Understand the pricing metric, not the price.** This is where the money leaks:
- **Contact/profile-based** (Marketo, HubSpot, Klaviyo): costs grow with database size whether or not you email those people. Negotiate on *marketable* or *active* contacts, define the term precisely, and get an annual true-down right — not just true-up.
- **Send/message volume** (SFMC, some B2C tools): watch overage rates. Pre-negotiate tiered overage pricing rather than accepting list rates when you blow through.
- **MAU-based** (Braze): make sure the MAU definition matches how you'll actually use it.
**3. Budget for implementation honestly.** For Marketo, SFMC, or Adobe, systems-integrator fees commonly run 0.5–2x year-one license. If the business case only pencils out on license cost, it doesn't pencil out.
**4. Check what's a separate SKU.** Sandboxes, dedicated IPs, deliverability consulting, SMS, additional business units, API call tiers, premium support — these are routinely carved out and quoted later.
**5. Contract terms worth fighting for:**
- Cap renewal uplift (aim 3–5%) for the full term, not just year two
- Price hold on additional volume purchased mid-term
- Data export rights and transition assistance on exit, in writing
- DPA, SOC 2 Type II, subprocessor notification; BAA if you touch PHI
- If you ramp, tie payment to actual ramp, not a flat average
**6. Time it.** Salesforce's fiscal year ends Jan 31, Adobe's around Nov 30. Quarter- and year-end give you real leverage. Run a genuine competitive process even if you know where you'll land — a credible alternative is worth more than any negotiation tactic.
**7. Be wary of ELA bundling.** Rolling this into an existing Salesforce or Adobe agreement often gets a striking discount, but it consolidates your renewal exposure and kills future leverage. Sometimes still the right call — just price the optionality you're giving up.
## What I'd need to give you a real recommendation
- B2B, B2C, or both?
- Current CRM and whether there's an existing Salesforce/Adobe ELA
- Approximate contact database size and monthly send volume
- In-house marketing ops headcount (this rules out or rules in Marketo more than anything else)
- Regulated industry? (healthcare, financial services, or EU/global data residency needs)
- Rough budget envelope
One caveat: pricing and packaging in this category change frequently, and vendors rebrand tiers constantly — treat any numbers above as directional and verify against current quotes.

End of response · 56 lines · 4,768 characters · sha256 a5074c0d9cfe8adc8041ed4d0c08a9f149a4b968d0031fdf6ddf68e556813982

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
anthropic/claude-opus-5
pinned anthropic/claude-opus-5-20260723; alias resolved to anthropic/claude-opus-5-20260723 at 2026-09-18 09:00:22 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
6 added, 1 dropped
Baseline
6 prior days with data; 1 matched the baseline value (2026-09-17)
Responses
before 2026-09-17 09:27:42 UTC, run 20260917T090022Z-136fc3; after 2026-09-18 09:30:59 UTC, run 20260918T090022Z-92b6f7
Detected
2026-09-18 · run 20260918T090022Z-92b6f7, started 2026-09-18 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-18/P29/anthropic/claude-opus-5/pick_change/picks/
Event ID
2026-09-18:P29:anthropic/claude-opus-5:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.