MODELDRIFT.WATCH

Sep 20 · Day 11

MINOR · CLAUDE · QUESTION 33

Claude changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 18beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Oracle Eloqua
  5. Braze
  6. Iterable
  7. Klaviyo
  8. Contact/profile-based
  9. MAU-based

Short answer: there's no single right pick — the answer forks hard on one question. But here's how I'd frame it, plus the procurement-specific landmines that matter more than the feature comparison.

The question that determines everything

Are you buying for B2B demand gen or B2C/consumer lifecycle messaging? These are different product categories with different vendors, pricing models, and cost curves. Related: what CRM you're standardized on, and whether you already have a Salesforce or Adobe enterprise agreement (that usually dominates the decision economically).

Rough shortlists

B2B demand gen (lead scoring, nurture, MQL handoff to sales):

  • •Adobe Marketo Engage — the default for complex B2B at your size. Deep, but needs a skilled admin or agency; not self-serve.
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot) — the obvious choice if you're a Salesforce shop. Less powerful than Marketo, much less integration friction.
  • •HubSpot Marketing Hub Enterprise — faster time-to-value, genuinely liked by marketers, but gets expensive at large contact volumes and is weaker on complex multi-entity orgs.
  • •Oracle Eloqua — still strong for heavily regulated or approval-heavy environments; smaller talent pool.

B2C / consumer lifecycle (high-volume, cross-channel, mobile):

  • •Braze — strong if push/in-app matter as much as email.
  • •Iterable — similar space, often more flexible commercially.
  • •Klaviyo — if this is ecommerce/DTC-driven.
  • •Salesforce Marketing Cloud Engagement or Adobe Journey Optimizer — if you're consolidating onto an existing ELA and have real data engineering capacity.

Transactional email (receipts, password resets) is a separate buy — SendGrid, Amazon SES, Postmark. Don't let a marketing platform vendor bundle this in at marketing-tier rates.

What I'd actually focus on as procurement

1. Audit before you buy. At 5,000 people you almost certainly have 4–10 email tools already — Mailchimp on a regional marketing card, a Constant Contact instance in HR, whatever the events team uses. Consolidation is usually a bigger win than platform selection, and the shadow-IT inventory is also your compliance exposure. Pull the expense data first.

2. Understand the pricing metric, not the price. This is where the money leaks:

  • •Contact/profile-based (Marketo, HubSpot, Klaviyo): costs grow with database size whether or not you email those people. Negotiate on marketable or active contacts, define the term precisely, and get an annual true-down right — not just true-up.
  • •Send/message volume (SFMC, some B2C tools): watch overage rates. Pre-negotiate tiered overage pricing rather than accepting list rates when you blow through.
  • •MAU-based (Braze): make sure the MAU definition matches how you'll actually use it.

3. Budget for implementation honestly. For Marketo, SFMC, or Adobe, systems-integrator fees commonly run 0.5–2x year-one license. If the business case only pencils out on license cost, it doesn't pencil out.

4. Check what's a separate SKU. Sandboxes, dedicated IPs, deliverability consulting, SMS, additional business units, API call tiers, premium support — these are routinely carved out and quoted later.

5. Contract terms worth fighting for:

  • •Cap renewal uplift (aim 3–5%) for the full term, not just year two
  • •Price hold on additional volume purchased mid-term
  • •Data export rights and transition assistance on exit, in writing
  • •DPA, SOC 2 Type II, subprocessor notification; BAA if you touch PHI
  • •If you ramp, tie payment to actual ramp, not a flat average

6. Time it. Salesforce's fiscal year ends Jan 31, Adobe's around Nov 30. Quarter- and year-end give you real leverage. Run a genuine competitive process even if you know where you'll land — a credible alternative is worth more than any negotiation tactic.

7. Be wary of ELA bundling. Rolling this into an existing Salesforce or Adobe agreement often gets a striking discount, but it consolidates your renewal exposure and kills future leverage. Sometimes still the right call — just price the optionality you're giving up.

What I'd need to give you a real recommendation

  • •B2B, B2C, or both?
  • •Current CRM and whether there's an existing Salesforce/Adobe ELA
  • •Approximate contact database size and monthly send volume
  • •In-house marketing ops headcount (this rules out or rules in Marketo more than anything else)
  • •Regulated industry? (healthcare, financial services, or EU/global data residency needs)
  • •Rough budget envelope

One caveat: pricing and packaging in this category change frequently, and vendors rebrand tiers constantly — treat any numbers above as directional and verify against current quotes.

SEP 20afterPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Oracle Eloqua
  4. HubSpot Marketing Hub Enterprise
  5. Braze
  6. Iterable
  7. Adobe Journey Optimizer
  8. SAP Emarsys
  9. Deliverability
  10. Timing

The right answer hinges almost entirely on one question: is this B2B demand generation or B2C/consumer lifecycle messaging? Those are different product categories with different vendors. A few other things matter too (existing CRM/martech stack, send volume, whether you need transactional email), but B2B vs. B2C is the main fork.

Here's how the landscape breaks down at your scale:

If it's B2B marketing (lead nurture, ABM, sales handoff)

  • •Adobe Marketo Engage — the default enterprise B2B choice. Deep campaign logic, strong Salesforce sync. Clunky UI, needs a dedicated admin.
  • •Salesforce Marketing Cloud Account Engagement (Pardot) — weaker than Marketo functionally, but if you're a heavy Salesforce shop the native integration and single-vendor consolidation can outweigh that.
  • •Oracle Eloqua — very strong at complex routing and governance. Aging ecosystem; fewer people know it.
  • •HubSpot Marketing Hub Enterprise — the usability outlier. Genuinely faster time-to-value and lower admin burden. Historically criticized for hitting ceilings at large database sizes and complex org structures, though it's moved upmarket significantly. Worth a serious look if your marketing team is small relative to headcount.

If it's B2C/consumer lifecycle (high-volume, cross-channel)

  • •Braze — strong modern choice for cross-channel (email, push, in-app, SMS). Good API/developer story.
  • •Iterable — similar space, often cited as slightly friendlier for marketer self-service.
  • •Salesforce Marketing Cloud Engagement — powerful and very configurable, but implementation-heavy. Budget for a systems integrator.
  • •Adobe Journey Optimizer — only makes sense if you're already committed to Adobe Experience Platform. Otherwise the platform tax is severe.
  • •SAP Emarsys or Klaviyo — if retail/ecommerce specifically.

Procurement-specific things I'd flag

These are where I'd focus your leverage, since the functional evaluation should really be owned by marketing:

Pricing model is the biggest trap. Vendors price on different units — contacts in database, sends per month, monthly active users, or credits. Model your 3-year growth curve against each. A platform that's cheaper at year one can be 2x at year three depending on which metric grows fastest for you. Specifically negotiate: overage rates (not just base rate), a cap on renewal uplift (aim for 3–5%), and what happens if your contact count drops.

Implementation cost is often the larger number. For Salesforce Marketing Cloud or Adobe, SI fees commonly run 1–2x first-year license. Marketo is somewhat less. HubSpot is usually a fraction of that. Get SI quotes during vendor selection, not after — it frequently changes the ranking.

Don't let the license get bundled into an enterprise agreement you can't unwind. Both Salesforce and Adobe will offer attractive bundling. It's real savings, but it compounds switching costs and weakens you at every future renewal.

Diligence checklist: SOC 2 Type II, ISO 27001, DPA with current SCCs, subprocessor list and change-notification terms, data residency, SSO/SAML plus SCIM provisioning, VPAT/accessibility conformance, data export rights and format on termination, and uptime SLA with actual credits (not just a published target). If you're in healthcare or financial services, confirm BAA availability or relevant attestations early — it eliminates several vendors.

Deliverability. Ask about dedicated IP allocation, IP warming support, and whether deliverability consulting is included or a paid add-on. At 5,000 employees you may have multiple business units sending; shared-IP reputation problems become a real operational risk.

Timing. Both Salesforce and Adobe have January 31 fiscal year ends; most others are calendar. End-of-quarter and end-of-year are your best discount windows. Expect 20–40% off list as normal, more on multi-year.

To narrow this down

It would help to know: B2B or B2C? What CRM are you on today? Rough contact database size and monthly send volume? Are you replacing something specific, and what's driving the change? And how large is the marketing ops team who'd actually run this — that single factor often decides between Marketo/SFMC and HubSpot more than any feature comparison.

One honest caveat: the cost figures and discount norms I've described are directional and move over time, and vendor positioning shifts year to year. Treat this as a framework for structuring your RFP rather than final numbers. Your marketing and marketing-ops stakeholders should own the functional scoring — your leverage is in commercial structure, and that's where the money usually is.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:30:59 UTC
Run
20260918T090022Z-92b6f7
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
4,768 characters · 3,452 tokens out incl. hidden reasoning · 51.8 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Oracle Eloqua
  5. Braze
  6. Iterable
  7. Klaviyo
  8. Contact/profile-based
  9. MAU-based

Highlighted: the names the extractor took as picks, where they appear verbatim (9 of 9).

Short answer: there's no single right pick — the answer forks hard on one question. But here's how I'd frame it, plus the procurement-specific landmines that matter more than the feature comparison.
## The question that determines everything
**Are you buying for B2B demand gen or B2C/consumer lifecycle messaging?** These are different product categories with different vendors, pricing models, and cost curves. Related: what CRM you're standardized on, and whether you already have a Salesforce or Adobe enterprise agreement (that usually dominates the decision economically).
## Rough shortlists
**B2B demand gen** (lead scoring, nurture, MQL handoff to sales):
- **Adobe Marketo Engage** — the default for complex B2B at your size. Deep, but needs a skilled admin or agency; not self-serve.
- **Salesforce Marketing Cloud Account Engagement** (formerly Pardot) — the obvious choice if you're a Salesforce shop. Less powerful than Marketo, much less integration friction.
- **HubSpot Marketing Hub Enterprise** — faster time-to-value, genuinely liked by marketers, but gets expensive at large contact volumes and is weaker on complex multi-entity orgs.
- **Oracle Eloqua** — still strong for heavily regulated or approval-heavy environments; smaller talent pool.
**B2C / consumer lifecycle** (high-volume, cross-channel, mobile):
- **Braze** — strong if push/in-app matter as much as email.
- **Iterable** — similar space, often more flexible commercially.
- **Klaviyo** — if this is ecommerce/DTC-driven.
- **Salesforce Marketing Cloud Engagement** or **Adobe Journey Optimizer** — if you're consolidating onto an existing ELA and have real data engineering capacity.
**Transactional email** (receipts, password resets) is a separate buy — SendGrid, Amazon SES, Postmark. Don't let a marketing platform vendor bundle this in at marketing-tier rates.
## What I'd actually focus on as procurement
**1. Audit before you buy.** At 5,000 people you almost certainly have 4–10 email tools already — Mailchimp on a regional marketing card, a Constant Contact instance in HR, whatever the events team uses. Consolidation is usually a bigger win than platform selection, and the shadow-IT inventory is also your compliance exposure. Pull the expense data first.
**2. Understand the pricing metric, not the price.** This is where the money leaks:
- **Contact/profile-based** (Marketo, HubSpot, Klaviyo): costs grow with database size whether or not you email those people. Negotiate on *marketable* or *active* contacts, define the term precisely, and get an annual true-down right — not just true-up.
- **Send/message volume** (SFMC, some B2C tools): watch overage rates. Pre-negotiate tiered overage pricing rather than accepting list rates when you blow through.
- **MAU-based** (Braze): make sure the MAU definition matches how you'll actually use it.
**3. Budget for implementation honestly.** For Marketo, SFMC, or Adobe, systems-integrator fees commonly run 0.5–2x year-one license. If the business case only pencils out on license cost, it doesn't pencil out.
**4. Check what's a separate SKU.** Sandboxes, dedicated IPs, deliverability consulting, SMS, additional business units, API call tiers, premium support — these are routinely carved out and quoted later.
**5. Contract terms worth fighting for:**
- Cap renewal uplift (aim 3–5%) for the full term, not just year two
- Price hold on additional volume purchased mid-term
- Data export rights and transition assistance on exit, in writing
- DPA, SOC 2 Type II, subprocessor notification; BAA if you touch PHI
- If you ramp, tie payment to actual ramp, not a flat average
**6. Time it.** Salesforce's fiscal year ends Jan 31, Adobe's around Nov 30. Quarter- and year-end give you real leverage. Run a genuine competitive process even if you know where you'll land — a credible alternative is worth more than any negotiation tactic.
**7. Be wary of ELA bundling.** Rolling this into an existing Salesforce or Adobe agreement often gets a striking discount, but it consolidates your renewal exposure and kills future leverage. Sometimes still the right call — just price the optionality you're giving up.
## What I'd need to give you a real recommendation
- B2B, B2C, or both?
- Current CRM and whether there's an existing Salesforce/Adobe ELA
- Approximate contact database size and monthly send volume
- In-house marketing ops headcount (this rules out or rules in Marketo more than anything else)
- Regulated industry? (healthcare, financial services, or EU/global data residency needs)
- Rough budget envelope
One caveat: pricing and packaging in this category change frequently, and vendors rebrand tiers constantly — treat any numbers above as directional and verify against current quotes.

End of response · 56 lines · 4,768 characters · sha256 a5074c0d9cfe8adc8041ed4d0c08a9f149a4b968d0031fdf6ddf68e556813982

After this run

2026-09-20 permalink

Scored
answered
Received
2026-09-20 09:26:03 UTC
Run
20260920T090019Z-bbedf3
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
4,704 characters · 2,961 tokens out incl. hidden reasoning · 46.0 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. Oracle Eloqua
  4. HubSpot Marketing Hub Enterprise
  5. Braze
  6. Iterable
  7. Adobe Journey Optimizer
  8. SAP Emarsys
  9. Deliverability
  10. Timing

Highlighted: the names the extractor took as picks, where they appear verbatim (10 of 10).

The right answer hinges almost entirely on one question: **is this B2B demand generation or B2C/consumer lifecycle messaging?** Those are different product categories with different vendors. A few other things matter too (existing CRM/martech stack, send volume, whether you need transactional email), but B2B vs. B2C is the main fork.
Here's how the landscape breaks down at your scale:
## If it's B2B marketing (lead nurture, ABM, sales handoff)
- **Adobe Marketo Engage** — the default enterprise B2B choice. Deep campaign logic, strong Salesforce sync. Clunky UI, needs a dedicated admin.
- **Salesforce Marketing Cloud Account Engagement (Pardot)** — weaker than Marketo functionally, but if you're a heavy Salesforce shop the native integration and single-vendor consolidation can outweigh that.
- **Oracle Eloqua** — very strong at complex routing and governance. Aging ecosystem; fewer people know it.
- **HubSpot Marketing Hub Enterprise** — the usability outlier. Genuinely faster time-to-value and lower admin burden. Historically criticized for hitting ceilings at large database sizes and complex org structures, though it's moved upmarket significantly. Worth a serious look if your marketing team is small relative to headcount.
## If it's B2C/consumer lifecycle (high-volume, cross-channel)
- **Braze** — strong modern choice for cross-channel (email, push, in-app, SMS). Good API/developer story.
- **Iterable** — similar space, often cited as slightly friendlier for marketer self-service.
- **Salesforce Marketing Cloud Engagement** — powerful and very configurable, but implementation-heavy. Budget for a systems integrator.
- **Adobe Journey Optimizer** — only makes sense if you're already committed to Adobe Experience Platform. Otherwise the platform tax is severe.
- **SAP Emarsys** or **Klaviyo** — if retail/ecommerce specifically.
## Procurement-specific things I'd flag
These are where I'd focus your leverage, since the functional evaluation should really be owned by marketing:
**Pricing model is the biggest trap.** Vendors price on different units — contacts in database, sends per month, monthly active users, or credits. Model your 3-year growth curve against each. A platform that's cheaper at year one can be 2x at year three depending on which metric grows fastest for you. Specifically negotiate: overage rates (not just base rate), a cap on renewal uplift (aim for 3–5%), and what happens if your contact count *drops*.
**Implementation cost is often the larger number.** For Salesforce Marketing Cloud or Adobe, SI fees commonly run 1–2x first-year license. Marketo is somewhat less. HubSpot is usually a fraction of that. Get SI quotes *during* vendor selection, not after — it frequently changes the ranking.
**Don't let the license get bundled into an enterprise agreement you can't unwind.** Both Salesforce and Adobe will offer attractive bundling. It's real savings, but it compounds switching costs and weakens you at every future renewal.
**Diligence checklist:** SOC 2 Type II, ISO 27001, DPA with current SCCs, subprocessor list and change-notification terms, data residency, SSO/SAML plus SCIM provisioning, VPAT/accessibility conformance, data export rights and format on termination, and uptime SLA with actual credits (not just a published target). If you're in healthcare or financial services, confirm BAA availability or relevant attestations early — it eliminates several vendors.
**Deliverability.** Ask about dedicated IP allocation, IP warming support, and whether deliverability consulting is included or a paid add-on. At 5,000 employees you may have multiple business units sending; shared-IP reputation problems become a real operational risk.
**Timing.** Both Salesforce and Adobe have January 31 fiscal year ends; most others are calendar. End-of-quarter and end-of-year are your best discount windows. Expect 20–40% off list as normal, more on multi-year.
## To narrow this down
It would help to know: B2B or B2C? What CRM are you on today? Rough contact database size and monthly send volume? Are you replacing something specific, and what's driving the change? And how large is the marketing ops team who'd actually run this — that single factor often decides between Marketo/SFMC and HubSpot more than any feature comparison.
One honest caveat: the cost figures and discount norms I've described are directional and move over time, and vendor positioning shifts year to year. Treat this as a framework for structuring your RFP rather than final numbers. Your marketing and marketing-ops stakeholders should own the functional scoring — your leverage is in commercial structure, and that's where the money usually is.

End of response · 40 lines · 4,704 characters · sha256 5ee2e64c7014eb4e264be634598838ebec64f2003c240afd40044ff508342c4b

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
anthropic/claude-opus-5
pinned anthropic/claude-opus-5-20260723; alias resolved to anthropic/claude-opus-5-20260723 at 2026-09-20 09:00:19 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
4 added, 3 dropped
Baseline
5 prior days with data; 1 matched the baseline value (2026-09-18)
Responses
before 2026-09-18 09:30:59 UTC, run 20260918T090022Z-92b6f7; after 2026-09-20 09:26:03 UTC, run 20260920T090019Z-bbedf3
Detected
2026-09-20 · run 20260920T090019Z-bbedf3, started 2026-09-20 09:00:19 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-20/P29/anthropic/claude-opus-5/pick_change/picks/
Event ID
2026-09-20:P29:anthropic/claude-opus-5:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.