MODELDRIFT.WATCH

Sep 29 · Day 20

MINOR · CLAUDE · QUESTION 33

Claude changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 27beforePermalink

Picks read from the answer, in order
  1. Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Oracle Eloqua
  5. Braze

The right answer depends heavily on use case, and "email marketing" at a 5,000-person enterprise usually means one of four quite different things. Let me lay out the shortlists, then the procurement-specific angles, which is probably where I can be most useful to you.

First, which problem are you sourcing for?

B2B demand gen / lead nurture (tied to CRM):

  • •Marketo Engage (Adobe) — enterprise default for complex B2B. Deep, powerful, steep learning curve, needs dedicated ops headcount.
  • •Salesforce Marketing Cloud Account Engagement (Pardot) — the pragmatic choice if you're Salesforce-centric and want one throat to choke.
  • •HubSpot Marketing Hub Enterprise — far better usability, increasingly credible at your size, lower total cost of ownership. Weaker at very complex data models.
  • •Oracle Eloqua — strong on complex routing/data governance, but declining mindshare; check the roadmap carefully.

B2C / consumer lifecycle messaging at volume:

  • •Braze or Iterable if mobile/push matters; Salesforce Marketing Cloud Engagement if you need heavy enterprise integration; Klaviyo if commerce-driven.

Transactional/API email (receipts, alerts): SendGrid, Amazon SES, Postmark. Cheap, and shouldn't be bundled into a marketing deal.

Internal/employee communications to your 5,000 staff: completely different category — Staffbase, Firstup, Poppulo, ContactMonkey. Don't let a marketing vendor sell you into this.

Procurement-specific things that actually move the number

Understand the pricing metric, because that's where the risk hides. Each vendor meters differently, and the metric determines your exposure:

  • •Marketo and Pardot price primarily on database/contact count — your bill grows as marketing does its job.
  • •Braze prices on monthly active users plus data points — usage-based and can escalate fast.
  • •HubSpot uses marketing contacts plus seats — relatively predictable.
  • •SFMC Engagement uses a "Super Messages" credit model that is genuinely hard to forecast. Get a written consumption model from the rep and stress-test it.

Negotiate the ramp, not just year one. Ask marketing for a three-year contact and send-volume forecast, then pre-buy tiers at a fixed unit rate. Unplanned overages are the single biggest cost surprise in this category.

Non-price terms worth more than discount points:

  • •Capped renewal uplift (target 3–5%, refuse "then-current list price")
  • •Downsizing rights at renewal — many of these contracts default to no decrease; this is negotiable and valuable
  • •Overage priced at the same unit rate as committed volume
  • •Sandbox/test environment included, plus documented API rate limits
  • •Termination assistance and data egress in a usable format, with deletion certification
  • •Super-cap on liability for data breach (multiple of fees, above the general cap) — you're handing a vendor a large marketable-contact database

Compliance diligence: SOC 2 Type II and ISO 27001, current pen test summary, sub-processor list, DPA with CCPA/CPRA terms and SCCs if you touch EU data. If SMS is in scope, TCPA consent handling is a real liability question, not a checkbox. If you sell to government or education, ask for a VPAT.

Implementation is the hidden half of year one. For Marketo or SFMC, expect services at roughly 0.5–2x the license fee. Bid the SI work separately against partners rather than accepting vendor professional services as a package — it's usually cheaper and you keep leverage.

Timing leverage: Salesforce's fiscal year ends January 31, Adobe's November 30, HubSpot's December 31. Quarter-end is good; fiscal year-end is better.

The thing I'd flag

If this is arriving on your desk as a cost-reduction or consolidation exercise, be careful: switching marketing automation platforms is a 6–12 month migration with real revenue risk from broken nurture programs and lost deliverability reputation. The savings often don't cover it. Consolidating into an existing Salesforce or Adobe enterprise agreement is usually a stronger play than a net-new competitive bid.

Conversely, if marketing is asking for a platform and you're running the sourcing event, the highest-value thing you can do is force a requirements document before anyone sees a demo. Vendor demos in this category are excellent and will reshape requirements if they come first.

To give you a sharper recommendation, it would help to know: B2B or B2C? What CRM are you on? Roughly how many contacts and monthly sends? And is there an incumbent you're replacing, or is this greenfield?

SEP 29afterPermalink

Picks read from the answer, in order
  1. Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Eloqua
  5. Braze
  6. Iterable
  7. Klaviyo
  8. Transactional/API
  9. SendGrid
  10. Staffbase

The honest answer is that "email marketing platform" covers three very different procurement categories, and the right shortlist depends on which one your stakeholders actually need. Worth nailing down before you go to market:

  • 1.Marketing/campaign email — outbound to prospects and customers, usually owned by Marketing, tied to CRM
  • 2.Transactional/product email — password resets, receipts, notifications, sent via API by Engineering
  • 3.Internal employee communications — all-hands emails to your 5,000 staff, usually owned by Comms/HR

These have different vendors, different price structures, and different security reviews. If your intake request just said "email marketing platform," it's worth a 20-minute call with the requester to find out which.

Shortlists by scenario

B2B marketing (long sales cycles, lead scoring, CRM-centric)

  • •Marketo Engage (Adobe) — deepest B2B automation; expect significant implementation cost and a specialist admin
  • •Salesforce Marketing Cloud Account Engagement (formerly Pardot) — the default if you're already heavily invested in Salesforce
  • •HubSpot Marketing Hub Enterprise — fastest time-to-value, best usability, less customizable at the edges
  • •Eloqua (Oracle) — strong if you're an Oracle shop, otherwise fading in mindshare

B2C / high-volume consumer lifecycle

  • •Braze — strong cross-channel (email, push, in-app), engineering-friendly
  • •Iterable — similar space, often easier for marketers to self-serve
  • •Klaviyo — dominant in ecommerce/DTC, moving upmarket
  • •Salesforce Marketing Cloud Engagement — powerful, but complex and implementation-heavy

Transactional/API

  • •SendGrid (Twilio), Amazon SES (cheapest by far if you have engineering capacity), Postmark, Mailgun

Internal employee comms

  • •Staffbase, Firstup, PoliteMail, ContactMonkey — these integrate with Outlook/Exchange and give you open/read analytics on internal mail, which general marketing tools handle poorly

Procurement-specific things that bite people here

Pricing model is the main trap. Marketing automation is usually licensed on database size (contacts stored), not emails sent. That means your cost ratchets up as Marketing imports lists, and vendors rarely let you tier back down mid-term. Push for: a contact tier with headroom, explicit language allowing downgrade at renewal, and a defined (cheap) overage rate rather than a forced tier upgrade.

Implementation is often the bigger number. For Marketo or Marketing Cloud, systems-integrator fees commonly run 1–2x first-year license. Get SI quotes before signing the license, and consider making the license contingent on a fixed-fee SOW.

Renewal uplift caps. Get a cap (aim for 3–5%) written in for at least two renewal cycles. Without it, year-four increases of 15–20% are common.

Timing leverage. Salesforce's fiscal year ends January 31; Adobe's ends late November/early December. HubSpot and the mid-market players push hard at calendar quarter-end. Aligning your signature date accordingly is real money.

Bundling. If you already have an Adobe or Salesforce enterprise agreement, adding this SKU into that vehicle at the next true-up is usually cheaper than a standalone deal — but it also lengthens your lock-in and makes future switching harder. Worth a deliberate decision rather than a default.

Security/compliance review items: SOC 2 Type II, ISO 27001, subprocessor list, data residency (relevant if you have EU or UK data subjects — check GDPR/UK GDPR transfer mechanisms), DPA with standard contractual clauses, breach notification SLA, SSO/SCIM support, and audit-log retention. If you're in healthcare or financial services, add HIPAA BAA or applicable regulatory requirements to the RFP.

Deliverability. Ask each vendor for dedicated IP options, IP warming support, and whether deliverability consulting is included or a paid add-on. At your volume this materially affects whether the tool works at all.

Exit terms. Data export format and cost, transition assistance period, and termination-for-convenience rights. Also check auto-renewal notice windows — 60–90 days is typical and easy to miss.

Practical next step

If this is a competitive process, three bidders is the sweet spot — enough for price tension, not so many that stakeholder evaluation collapses. Run a scored RFP with Marketing as the functional evaluator and hold pricing negotiation separately after technical selection, so you're not negotiating against a foregone conclusion.

If you can tell me which of the three use cases this is, and whether you're already on Salesforce, Adobe, or Microsoft Dynamics, I can narrow the shortlist considerably and flag the specific contract clauses that matter most for those vendors.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-27 permalink

Scored
answered
Received
2026-09-27 09:25:06 UTC
Run
20260927T090007Z-7adb14
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
4,633 characters · 3,350 tokens out incl. hidden reasoning · 52.4 s
Extracted picks, in order · confidence medium
  1. Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Oracle Eloqua
  5. Braze

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

The right answer depends heavily on use case, and "email marketing" at a 5,000-person enterprise usually means one of four quite different things. Let me lay out the shortlists, then the procurement-specific angles, which is probably where I can be most useful to you.
## First, which problem are you sourcing for?
**B2B demand gen / lead nurture (tied to CRM):**
- **Marketo Engage (Adobe)** — enterprise default for complex B2B. Deep, powerful, steep learning curve, needs dedicated ops headcount.
- **Salesforce Marketing Cloud Account Engagement (Pardot)** — the pragmatic choice if you're Salesforce-centric and want one throat to choke.
- **HubSpot Marketing Hub Enterprise** — far better usability, increasingly credible at your size, lower total cost of ownership. Weaker at very complex data models.
- **Oracle Eloqua** — strong on complex routing/data governance, but declining mindshare; check the roadmap carefully.
**B2C / consumer lifecycle messaging at volume:**
- **Braze** or **Iterable** if mobile/push matters; **Salesforce Marketing Cloud Engagement** if you need heavy enterprise integration; **Klaviyo** if commerce-driven.
**Transactional/API email (receipts, alerts):** SendGrid, Amazon SES, Postmark. Cheap, and shouldn't be bundled into a marketing deal.
**Internal/employee communications to your 5,000 staff:** completely different category — Staffbase, Firstup, Poppulo, ContactMonkey. Don't let a marketing vendor sell you into this.
## Procurement-specific things that actually move the number
**Understand the pricing metric, because that's where the risk hides.** Each vendor meters differently, and the metric determines your exposure:
- Marketo and Pardot price primarily on **database/contact count** — your bill grows as marketing does its job.
- Braze prices on **monthly active users plus data points** — usage-based and can escalate fast.
- HubSpot uses **marketing contacts plus seats** — relatively predictable.
- SFMC Engagement uses a **"Super Messages" credit model** that is genuinely hard to forecast. Get a written consumption model from the rep and stress-test it.
**Negotiate the ramp, not just year one.** Ask marketing for a three-year contact and send-volume forecast, then pre-buy tiers at a fixed unit rate. Unplanned overages are the single biggest cost surprise in this category.
**Non-price terms worth more than discount points:**
- **Capped renewal uplift** (target 3–5%, refuse "then-current list price")
- **Downsizing rights at renewal** — many of these contracts default to no decrease; this is negotiable and valuable
- **Overage priced at the same unit rate** as committed volume
- **Sandbox/test environment included**, plus documented API rate limits
- **Termination assistance and data egress** in a usable format, with deletion certification
- **Super-cap on liability for data breach** (multiple of fees, above the general cap) — you're handing a vendor a large marketable-contact database
**Compliance diligence:** SOC 2 Type II and ISO 27001, current pen test summary, sub-processor list, DPA with CCPA/CPRA terms and SCCs if you touch EU data. If SMS is in scope, TCPA consent handling is a real liability question, not a checkbox. If you sell to government or education, ask for a VPAT.
**Implementation is the hidden half of year one.** For Marketo or SFMC, expect services at roughly 0.5–2x the license fee. Bid the SI work separately against partners rather than accepting vendor professional services as a package — it's usually cheaper and you keep leverage.
**Timing leverage:** Salesforce's fiscal year ends January 31, Adobe's November 30, HubSpot's December 31. Quarter-end is good; fiscal year-end is better.
## The thing I'd flag
If this is arriving on your desk as a cost-reduction or consolidation exercise, be careful: switching marketing automation platforms is a 6–12 month migration with real revenue risk from broken nurture programs and lost deliverability reputation. The savings often don't cover it. Consolidating *into* an existing Salesforce or Adobe enterprise agreement is usually a stronger play than a net-new competitive bid.
Conversely, if marketing is asking for a platform and you're running the sourcing event, the highest-value thing you can do is force a requirements document before anyone sees a demo. Vendor demos in this category are excellent and will reshape requirements if they come first.
To give you a sharper recommendation, it would help to know: **B2B or B2C?** What CRM are you on? Roughly how many contacts and monthly sends? And is there an incumbent you're replacing, or is this greenfield?

End of response · 48 lines · 4,633 characters · sha256 0d7933fdd7c4d73471ab2e2230b3c78ce5021e8f878690bb1221e7190648ba1f

After this run

2026-09-29 permalink

Scored
answered
Received
2026-09-29 09:26:05 UTC
Run
20260929T090007Z-334891
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
4,836 characters · 2,950 tokens out incl. hidden reasoning · 44.1 s
Extracted picks, in order · confidence medium
  1. Marketo Engage
  2. Salesforce Marketing Cloud Account Engagement
  3. HubSpot Marketing Hub Enterprise
  4. Eloqua
  5. Braze
  6. Iterable
  7. Klaviyo
  8. Transactional/API
  9. SendGrid
  10. Staffbase

Highlighted: the names the extractor took as picks, where they appear verbatim (10 of 10).

The honest answer is that "email marketing platform" covers three very different procurement categories, and the right shortlist depends on which one your stakeholders actually need. Worth nailing down before you go to market:
1. **Marketing/campaign email** — outbound to prospects and customers, usually owned by Marketing, tied to CRM
2. **Transactional/product email** — password resets, receipts, notifications, sent via API by Engineering
3. **Internal employee communications** — all-hands emails to your 5,000 staff, usually owned by Comms/HR
These have different vendors, different price structures, and different security reviews. If your intake request just said "email marketing platform," it's worth a 20-minute call with the requester to find out which.
## Shortlists by scenario
**B2B marketing (long sales cycles, lead scoring, CRM-centric)**
- **Marketo Engage (Adobe)** — deepest B2B automation; expect significant implementation cost and a specialist admin
- **Salesforce Marketing Cloud Account Engagement (formerly Pardot)** — the default if you're already heavily invested in Salesforce
- **HubSpot Marketing Hub Enterprise** — fastest time-to-value, best usability, less customizable at the edges
- **Eloqua (Oracle)** — strong if you're an Oracle shop, otherwise fading in mindshare
**B2C / high-volume consumer lifecycle**
- **Braze** — strong cross-channel (email, push, in-app), engineering-friendly
- **Iterable** — similar space, often easier for marketers to self-serve
- **Klaviyo** — dominant in ecommerce/DTC, moving upmarket
- **Salesforce Marketing Cloud Engagement** — powerful, but complex and implementation-heavy
**Transactional/API**
- **SendGrid (Twilio)**, **Amazon SES** (cheapest by far if you have engineering capacity), **Postmark**, **Mailgun**
**Internal employee comms**
- **Staffbase**, **Firstup**, **PoliteMail**, **ContactMonkey** — these integrate with Outlook/Exchange and give you open/read analytics on internal mail, which general marketing tools handle poorly
## Procurement-specific things that bite people here
**Pricing model is the main trap.** Marketing automation is usually licensed on *database size* (contacts stored), not emails sent. That means your cost ratchets up as Marketing imports lists, and vendors rarely let you tier back down mid-term. Push for: a contact tier with headroom, explicit language allowing downgrade at renewal, and a defined (cheap) overage rate rather than a forced tier upgrade.
**Implementation is often the bigger number.** For Marketo or Marketing Cloud, systems-integrator fees commonly run 1–2x first-year license. Get SI quotes *before* signing the license, and consider making the license contingent on a fixed-fee SOW.
**Renewal uplift caps.** Get a cap (aim for 3–5%) written in for at least two renewal cycles. Without it, year-four increases of 15–20% are common.
**Timing leverage.** Salesforce's fiscal year ends January 31; Adobe's ends late November/early December. HubSpot and the mid-market players push hard at calendar quarter-end. Aligning your signature date accordingly is real money.
**Bundling.** If you already have an Adobe or Salesforce enterprise agreement, adding this SKU into that vehicle at the next true-up is usually cheaper than a standalone deal — but it also lengthens your lock-in and makes future switching harder. Worth a deliberate decision rather than a default.
**Security/compliance review items:** SOC 2 Type II, ISO 27001, subprocessor list, data residency (relevant if you have EU or UK data subjects — check GDPR/UK GDPR transfer mechanisms), DPA with standard contractual clauses, breach notification SLA, SSO/SCIM support, and audit-log retention. If you're in healthcare or financial services, add HIPAA BAA or applicable regulatory requirements to the RFP.
**Deliverability.** Ask each vendor for dedicated IP options, IP warming support, and whether deliverability consulting is included or a paid add-on. At your volume this materially affects whether the tool works at all.
**Exit terms.** Data export format and cost, transition assistance period, and termination-for-convenience rights. Also check auto-renewal notice windows — 60–90 days is typical and easy to miss.
## Practical next step
If this is a competitive process, three bidders is the sweet spot — enough for price tension, not so many that stakeholder evaluation collapses. Run a scored RFP with Marketing as the functional evaluator and hold pricing negotiation separately after technical selection, so you're not negotiating against a foregone conclusion.
If you can tell me which of the three use cases this is, and whether you're already on Salesforce, Adobe, or Microsoft Dynamics, I can narrow the shortlist considerably and flag the specific contract clauses that matter most for those vendors.

End of response · 51 lines · 4,836 characters · sha256 ec756c89ecb5b499199b4fe7aa2aeaa621a3aff41be10de0aaf441c19594e855

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
anthropic/claude-opus-5
pinned anthropic/claude-opus-5-20260723; alias resolved to anthropic/claude-opus-5-20260723 at 2026-09-29 09:00:07 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
5 added, 0 dropped
Baseline
4 prior days with data; 1 matched the baseline value (2026-09-27)
Responses
before 2026-09-27 09:25:06 UTC, run 20260927T090007Z-7adb14; after 2026-09-29 09:26:05 UTC, run 20260929T090007Z-334891
Detected
2026-09-29 · run 20260929T090007Z-334891, started 2026-09-29 09:00:07 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-29/P29/anthropic/claude-opus-5/pick_change/picks/
Event ID
2026-09-29:P29:anthropic/claude-opus-5:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.