MODELDRIFT.WATCH

Sep 26 · Day 17

MINOR · GPT · QUESTION 33

GPT changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 25beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Enterprise B2B with deep Oracle alignment
  3. Salesforce Marketing Cloud
  4. Braze
  5. HubSpot Enterprise

For a 5,000-person enterprise, my default shortlist would be:

Best fitPlatform
B2B demand generation and complex lead nurturingAdobe Marketo Engage
Salesforce-centric enterprise, especially B2C or high-volume campaignsSalesforce Marketing Cloud Engagement
Product-led lifecycle messaging across email, push, SMS, and in-appBraze
Faster deployment and easier administration, with moderate complexityHubSpot Marketing Hub Enterprise
Enterprise B2B with deep Oracle alignmentOracle Eloqua

My recommendation

  • •Choose Marketo if you are primarily a B2B enterprise with long sales cycles, lead scoring, account-based marketing, and complex CRM workflows.
  • •Choose Salesforce Marketing Cloud if Salesforce is your strategic CRM and you need very high scale, sophisticated customer journeys, or multi-business-unit governance.
  • •Choose Braze if email is part of real-time customer engagement for a digital product rather than traditional campaign marketing.
  • •Choose HubSpot Enterprise if ease of use and speed matter more than maximum customization.

If no additional context were available, I would run a final evaluation between Marketo and Salesforce Marketing Cloud, with the existing CRM architecture as the deciding factor.

Procurement requirements to include

Evaluate three-year total cost, not the license quote alone:

  • •Contact, send-volume, API, data-storage, and business-unit overages
  • •Implementation and systems-integrator costs
  • •Dedicated IPs, deliverability support, and migration services
  • •SSO/SAML, SCIM, role-based access, audit logs, and data residency
  • •SOC 2 Type II, ISO 27001, CCPA, GDPR, CAN-SPAM, and DPA terms
  • •CRM/CDP/data-warehouse integration
  • •SLA, support response times, sandbox access, and disaster recovery
  • •Data export rights, deletion assistance, and capped renewal increases
  • •AI training/data-use restrictions and indemnity
  • •Exit assistance and suppression-list portability

Require finalists to complete a paid or tightly scoped proof of concept using three real workflows: one complex journey, one CRM-triggered campaign, and one compliance/data-export scenario. Avoid selecting purely from demos; implementation quality and deliverability operations often matter more than feature-count differences.

SEP 26afterPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Engagement
  3. Braze or HubSpot

Company headcount is less important than audience size, B2B vs. B2C, message volume, CRM stack, and journey complexity.

Practical recommendation

  • •B2B enterprise: Adobe Marketo Engage
    Strong for lead nurturing, scoring, account-based marketing, and complex campaign operations.
  • •B2C or very high-volume, complex journeys—especially if you use Salesforce: Salesforce Marketing Cloud Engagement
    Strong enterprise governance and personalization, but implementation and administration are costly.
  • •Digital product, app, or real-time lifecycle messaging: Braze
    Best fit when email must coordinate with push, in-app, SMS, and behavioral events.
  • •Simpler operations and faster deployment: HubSpot Marketing Hub Enterprise
    Easier to administer, but less suitable for highly customized, multi-business-unit global programs.

My default shortlist for a 5,000-person enterprise

  • 1.Salesforce Marketing Cloud Engagement
  • 2.Adobe Marketo Engage
  • 3.Braze or HubSpot, depending on the use case

If you are a conventional B2B enterprise, I would start with Marketo. If you are B2C and already standardized on Salesforce, I would start with Salesforce Marketing Cloud.

Procurement criteria that should drive the RFP

Weight vendors on:

  • •CRM, CDP, data warehouse, and CMS integration
  • •Contact count, annual sends, peak throughput, and overage pricing
  • •Deliverability services and sender-reputation controls
  • •SSO/SAML, SCIM, RBAC, audit logs, SOC 2, and data retention
  • •CAN-SPAM, CCPA, GDPR, consent, and suppression management
  • •Multi-brand, business-unit, localization, and approval workflows
  • •Implementation and ongoing administrator requirements
  • •API limits and charges for sandboxes, data storage, support, and connectors
  • •Data portability, deletion, transition assistance, and renewal caps

Evaluate three-year total cost, not subscription price alone. Salesforce and Marketo commonly require meaningful implementation, integration, agency, and specialist-administration spend.

I would not make Mailchimp or a similar SMB-focused product the enterprise standard unless your needs are limited to straightforward newsletters.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-25 permalink

Scored
answered
Received
2026-09-25 09:24:37 UTC
Run
20260925T090007Z-f08d1f
Served by
OpenAI · HTTP 200 · finish stop
Size
2,345 characters · 798 tokens out incl. hidden reasoning · 13.4 s
Extracted picks, in order · confidence medium
  1. Adobe Marketo Engage
  2. Enterprise B2B with deep Oracle alignment
  3. Salesforce Marketing Cloud
  4. Braze
  5. HubSpot Enterprise

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

For a 5,000-person enterprise, my default shortlist would be:
| Best fit | Platform |
|---|---|
| B2B demand generation and complex lead nurturing | **Adobe Marketo Engage** |
| Salesforce-centric enterprise, especially B2C or high-volume campaigns | **Salesforce Marketing Cloud Engagement** |
| Product-led lifecycle messaging across email, push, SMS, and in-app | **Braze** |
| Faster deployment and easier administration, with moderate complexity | **HubSpot Marketing Hub Enterprise** |
| Enterprise B2B with deep Oracle alignment | **Oracle Eloqua** |
### My recommendation
- **Choose Marketo** if you are primarily a B2B enterprise with long sales cycles, lead scoring, account-based marketing, and complex CRM workflows.
- **Choose Salesforce Marketing Cloud** if Salesforce is your strategic CRM and you need very high scale, sophisticated customer journeys, or multi-business-unit governance.
- **Choose Braze** if email is part of real-time customer engagement for a digital product rather than traditional campaign marketing.
- **Choose HubSpot Enterprise** if ease of use and speed matter more than maximum customization.
If no additional context were available, I would run a final evaluation between **Marketo and Salesforce Marketing Cloud**, with the existing CRM architecture as the deciding factor.
### Procurement requirements to include
Evaluate three-year total cost, not the license quote alone:
- Contact, send-volume, API, data-storage, and business-unit overages
- Implementation and systems-integrator costs
- Dedicated IPs, deliverability support, and migration services
- SSO/SAML, SCIM, role-based access, audit logs, and data residency
- SOC 2 Type II, ISO 27001, CCPA, GDPR, CAN-SPAM, and DPA terms
- CRM/CDP/data-warehouse integration
- SLA, support response times, sandbox access, and disaster recovery
- Data export rights, deletion assistance, and capped renewal increases
- AI training/data-use restrictions and indemnity
- Exit assistance and suppression-list portability
Require finalists to complete a paid or tightly scoped proof of concept using three real workflows: one complex journey, one CRM-triggered campaign, and one compliance/data-export scenario. Avoid selecting purely from demos; implementation quality and deliverability operations often matter more than feature-count differences.

End of response · 35 lines · 2,345 characters · sha256 70e7b44c7939549703f27262358f799e3f2df43dd25062e6d3beb48c91d3397d

After this run

2026-09-26 permalink

Scored
answered
Received
2026-09-26 09:23:34 UTC
Run
20260926T090007Z-89c0b7
Served by
OpenAI · HTTP 200 · finish stop
Size
2,213 characters · 874 tokens out incl. hidden reasoning · 12.5 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud Engagement
  3. Braze or HubSpot

Highlighted: the names the extractor took as picks, where they appear verbatim (3 of 3).

Company headcount is less important than audience size, B2B vs. B2C, message volume, CRM stack, and journey complexity.
## Practical recommendation
- **B2B enterprise:** **Adobe Marketo Engage**
Strong for lead nurturing, scoring, account-based marketing, and complex campaign operations.
- **B2C or very high-volume, complex journeys—especially if you use Salesforce:** **Salesforce Marketing Cloud Engagement**
Strong enterprise governance and personalization, but implementation and administration are costly.
- **Digital product, app, or real-time lifecycle messaging:** **Braze**
Best fit when email must coordinate with push, in-app, SMS, and behavioral events.
- **Simpler operations and faster deployment:** **HubSpot Marketing Hub Enterprise**
Easier to administer, but less suitable for highly customized, multi-business-unit global programs.
### My default shortlist for a 5,000-person enterprise
1. Salesforce Marketing Cloud Engagement
2. Adobe Marketo Engage
3. Braze or HubSpot, depending on the use case
If you are a conventional **B2B enterprise**, I would start with **Marketo**. If you are **B2C and already standardized on Salesforce**, I would start with **Salesforce Marketing Cloud**.
## Procurement criteria that should drive the RFP
Weight vendors on:
- CRM, CDP, data warehouse, and CMS integration
- Contact count, annual sends, peak throughput, and overage pricing
- Deliverability services and sender-reputation controls
- SSO/SAML, SCIM, RBAC, audit logs, SOC 2, and data retention
- CAN-SPAM, CCPA, GDPR, consent, and suppression management
- Multi-brand, business-unit, localization, and approval workflows
- Implementation and ongoing administrator requirements
- API limits and charges for sandboxes, data storage, support, and connectors
- Data portability, deletion, transition assistance, and renewal caps
Evaluate **three-year total cost**, not subscription price alone. Salesforce and Marketo commonly require meaningful implementation, integration, agency, and specialist-administration spend.
I would not make Mailchimp or a similar SMB-focused product the enterprise standard unless your needs are limited to straightforward newsletters.

End of response · 40 lines · 2,213 characters · sha256 4dda5d723d42f32da7493177a2244f9fbd57c1475ce07526b545eeaa3b263bde

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
openai/gpt-5.6-sol
pinned openai/gpt-5.6-sol-20260709; alias resolved to openai/gpt-5.6-sol-20260709 at 2026-09-26 09:00:06 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
0 added, 2 dropped
Baseline
6 prior days with data; 1 matched the baseline value (2026-09-25)
Responses
before 2026-09-25 09:24:37 UTC, run 20260925T090007Z-f08d1f; after 2026-09-26 09:23:34 UTC, run 20260926T090007Z-89c0b7
Detected
2026-09-26 · run 20260926T090007Z-89c0b7, started 2026-09-26 09:00:07 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-26/P29/openai/gpt-5.6-sol/pick_change/picks/
Event ID
2026-09-26:P29:openai/gpt-5.6-sol:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.