MODELDRIFT.WATCH

Sep 20 · Day 11

MINOR · GPT · QUESTION 33

GPT changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 19beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Mid-complexity B2B
  3. HubSpot Enterprise
  4. Salesforce Marketing Cloud

Employee count alone should not drive the decision; business model, message volume, CRM, and channel complexity matter more.

Practical shortlist

SituationBest starting point
B2B enterprise, complex lead nurturing/account-based marketingAdobe Marketo Engage
Salesforce-centric organization, B2C or broad multichannel journeysSalesforce Marketing Cloud Engagement
Consumer digital product with real-time, behavior-triggered messagingBraze
Mid-complexity B2B, prioritizing usability and fast deploymentHubSpot Marketing Hub Enterprise
Large e-commerce/retail operationKlaviyo or Braze, depending complexity
Highly customized cross-channel orchestrationIterable or Braze

My default recommendation

For a typical 5,000-person B2B enterprise, I would shortlist:

  • 1.Adobe Marketo Engage – strongest fit for sophisticated B2B lead management and nurturing.
  • 2.HubSpot Enterprise – easier administration and faster time to value, but validate scalability, governance, and contact-based pricing.
  • 3.Salesforce Marketing Cloud – prioritize it if Salesforce is already your strategic CRM/data platform, although implementation and operating costs can be substantial.

For a B2C enterprise, start with Braze and Salesforce Marketing Cloud, adding Iterable as a competitive bid.

Procurement criteria to weight

  • •Functional fit: 25% — segmentation, journeys, testing, personalization, approvals
  • •Integration/data architecture: 20% — CRM, CDP, warehouse, APIs, event latency
  • •Security/compliance: 15% — SAML/SSO, SCIM, RBAC, audit logs, SOC 2, ISO 27001, CPRA/CCPA, CAN-SPAM, consent controls
  • •Total cost of ownership: 15% — contacts, sends, data storage, API usage, implementation, support, partner costs
  • •Deliverability: 10% — dedicated IPs, reputation management, suppression handling, advisory support
  • •Usability/governance: 10% — business-unit separation, templates, approval workflows, localization
  • •Vendor viability/service: 5%

Contract risks to negotiate

Avoid pricing based on unrestricted “marketable contacts” without deletion and suppression rules. Cap renewal increases, lock implementation rates, define SLA/service credits, obtain data-export and transition assistance, and require advance notice for subprocessor or product changes.

Bottom line: Choose Marketo for B2B complexity, Braze for consumer lifecycle marketing, or Salesforce Marketing Cloud when Salesforce alignment and broad enterprise orchestration outweigh cost and administrative complexity. I would not make Mailchimp the default platform for an enterprise of this scale.

SEP 20afterPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage

Company size alone isn’t enough, but for a 5,000-person US enterprise I’d use this default shortlist:

Primary use caseBest starting choiceAlternatives
B2B lead generation and nurtureAdobe Marketo EngageSalesforce Account Engagement, HubSpot Enterprise
High-volume B2C campaigns and complex journeysSalesforce Marketing Cloud EngagementAdobe Journey Optimizer, Iterable
Real-time product/mobile lifecycle messagingBrazeIterable
Ease of use and faster deploymentHubSpot Marketing Hub EnterpriseMarketo

My default recommendation

  • •B2B enterprise: Marketo Engage, especially if you need sophisticated lead scoring, nurture programs, multiple business units, and CRM integration.
  • •B2C enterprise: Salesforce Marketing Cloud Engagement if Salesforce is strategic; Braze if customer behavior and real-time digital engagement drive the program.
  • •Limited marketing-operations resources: HubSpot Enterprise, accepting less flexibility at the highest levels of complexity.

Procurement considerations

Run a proof of concept and compare three-year total cost, not license price. Include:

  • •Contact and send-volume overages
  • •Implementation and migration
  • •CRM/CDP/data-warehouse integration
  • •Sandboxes, dedicated IPs, premium support, and required add-ons
  • •Multi-brand and business-unit governance
  • •SOC 2, SSO/SAML, SCIM, audit logs, encryption, and data-retention controls
  • •CAN-SPAM/CCPA support and consent management
  • •Deliverability services and SLA remedies
  • •Price caps at renewal, data-export rights, and exit assistance

Avoid buying a suite mainly because it matches your CRM—the implementation quality, operating model, and data architecture usually matter more.

If forced to choose without further information: Marketo for B2B; Salesforce Marketing Cloud Engagement for B2C.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-19 permalink

Scored
answered
Received
2026-09-19 09:23:48 UTC
Run
20260919T090005Z-86161c
Served by
OpenAI · HTTP 200 · finish stop
Size
2,759 characters · 966 tokens out incl. hidden reasoning · 17.6 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Mid-complexity B2B
  3. HubSpot Enterprise
  4. Salesforce Marketing Cloud

Highlighted: the names the extractor took as picks, where they appear verbatim (4 of 4).

Employee count alone should not drive the decision; business model, message volume, CRM, and channel complexity matter more.
### Practical shortlist
| Situation | Best starting point |
|---|---|
| **B2B enterprise, complex lead nurturing/account-based marketing** | **Adobe Marketo Engage** |
| **Salesforce-centric organization, B2C or broad multichannel journeys** | **Salesforce Marketing Cloud Engagement** |
| **Consumer digital product with real-time, behavior-triggered messaging** | **Braze** |
| **Mid-complexity B2B, prioritizing usability and fast deployment** | **HubSpot Marketing Hub Enterprise** |
| **Large e-commerce/retail operation** | **Klaviyo** or **Braze**, depending complexity |
| **Highly customized cross-channel orchestration** | **Iterable** or **Braze** |
### My default recommendation
For a typical 5,000-person **B2B enterprise**, I would shortlist:
1. **Adobe Marketo Engage** – strongest fit for sophisticated B2B lead management and nurturing.
2. **HubSpot Enterprise** – easier administration and faster time to value, but validate scalability, governance, and contact-based pricing.
3. **Salesforce Marketing Cloud** – prioritize it if Salesforce is already your strategic CRM/data platform, although implementation and operating costs can be substantial.
For a **B2C enterprise**, start with **Braze** and **Salesforce Marketing Cloud**, adding **Iterable** as a competitive bid.
### Procurement criteria to weight
- **Functional fit: 25%** — segmentation, journeys, testing, personalization, approvals
- **Integration/data architecture: 20%** — CRM, CDP, warehouse, APIs, event latency
- **Security/compliance: 15%** — SAML/SSO, SCIM, RBAC, audit logs, SOC 2, ISO 27001, CPRA/CCPA, CAN-SPAM, consent controls
- **Total cost of ownership: 15%** — contacts, sends, data storage, API usage, implementation, support, partner costs
- **Deliverability: 10%** — dedicated IPs, reputation management, suppression handling, advisory support
- **Usability/governance: 10%** — business-unit separation, templates, approval workflows, localization
- **Vendor viability/service: 5%**
### Contract risks to negotiate
Avoid pricing based on unrestricted “marketable contacts” without deletion and suppression rules. Cap renewal increases, lock implementation rates, define SLA/service credits, obtain data-export and transition assistance, and require advance notice for subprocessor or product changes.
**Bottom line:** Choose **Marketo** for B2B complexity, **Braze** for consumer lifecycle marketing, or **Salesforce Marketing Cloud** when Salesforce alignment and broad enterprise orchestration outweigh cost and administrative complexity. I would not make Mailchimp the default platform for an enterprise of this scale.

End of response · 38 lines · 2,759 characters · sha256 6a4edb6bb241b806a13dba562339692d43784b0a638c649c5c9cb610066c0302

After this run

2026-09-20 permalink

Scored
answered
Received
2026-09-20 09:25:17 UTC
Run
20260920T090019Z-bbedf3
Served by
OpenAI · HTTP 200 · finish stop
Size
1,907 characters · 920 tokens out incl. hidden reasoning · 14.8 s
Extracted picks, in order · confidence low
  1. Adobe Marketo Engage

Highlighted: the names the extractor took as picks, where they appear verbatim (1 of 1).

Company size alone isn’t enough, but for a 5,000-person US enterprise I’d use this default shortlist:
| Primary use case | Best starting choice | Alternatives |
|---|---|---|
| **B2B lead generation and nurture** | **Adobe Marketo Engage** | Salesforce Account Engagement, HubSpot Enterprise |
| **High-volume B2C campaigns and complex journeys** | **Salesforce Marketing Cloud Engagement** | Adobe Journey Optimizer, Iterable |
| **Real-time product/mobile lifecycle messaging** | **Braze** | Iterable |
| **Ease of use and faster deployment** | **HubSpot Marketing Hub Enterprise** | Marketo |
### My default recommendation
- **B2B enterprise:** Marketo Engage, especially if you need sophisticated lead scoring, nurture programs, multiple business units, and CRM integration.
- **B2C enterprise:** Salesforce Marketing Cloud Engagement if Salesforce is strategic; Braze if customer behavior and real-time digital engagement drive the program.
- **Limited marketing-operations resources:** HubSpot Enterprise, accepting less flexibility at the highest levels of complexity.
### Procurement considerations
Run a proof of concept and compare **three-year total cost**, not license price. Include:
- Contact and send-volume overages
- Implementation and migration
- CRM/CDP/data-warehouse integration
- Sandboxes, dedicated IPs, premium support, and required add-ons
- Multi-brand and business-unit governance
- SOC 2, SSO/SAML, SCIM, audit logs, encryption, and data-retention controls
- CAN-SPAM/CCPA support and consent management
- Deliverability services and SLA remedies
- Price caps at renewal, data-export rights, and exit assistance
Avoid buying a suite mainly because it matches your CRM—the implementation quality, operating model, and data architecture usually matter more.
If forced to choose without further information: **Marketo for B2B; Salesforce Marketing Cloud Engagement for B2C.**

End of response · 30 lines · 1,907 characters · sha256 79fa2a975b21b59539515e285be2163aba98a6b51fa628f61be7df71a5b1913b

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
openai/gpt-5.6-sol
pinned openai/gpt-5.6-sol-20260709; alias resolved to openai/gpt-5.6-sol-20260709 at 2026-09-20 09:00:19 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
0 added, 3 dropped
Baseline
7 prior days with data; 1 matched the baseline value (2026-09-19)
Responses
before 2026-09-19 09:23:48 UTC, run 20260919T090005Z-86161c; after 2026-09-20 09:25:17 UTC, run 20260920T090019Z-bbedf3
Detected
2026-09-20 · run 20260920T090019Z-bbedf3, started 2026-09-20 09:00:19 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-20/P29/openai/gpt-5.6-sol/pick_change/picks/
Event ID
2026-09-20:P29:openai/gpt-5.6-sol:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.