MODELDRIFT.WATCH

Sep 19 · Day 10

MINOR · GPT · QUESTION 33

GPT changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 18beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Braze
  3. Salesforce Marketing Cloud Engagement
  4. Adobe Journey Optimizer
  5. HubSpot Marketing Hub Enterprise

Employee count alone should not drive the decision. Choose based on customer type, message volume, CRM/data stack, and journey complexity.

My practical recommendation

Primary use caseBest-fit platform
B2B lead nurturing and account-based marketingAdobe Marketo Engage
B2C, high-volume, real-time lifecycle messagingBraze
Deep Salesforce ecosystem and complex enterprise workflowsSalesforce Marketing Cloud Engagement
Existing Adobe Experience Cloud/CDP investmentAdobe Journey Optimizer
Ease of use and lower administrative burdenHubSpot Marketing Hub Enterprise

Default choice for a typical 5,000-person B2B enterprise: Adobe Marketo Engage, particularly if you have sophisticated lead scoring, long sales cycles, multiple business units, and Salesforce CRM.

Default choice for a digital B2C enterprise: Braze, especially when email must coordinate with mobile push, in-app, SMS, and real-time behavioral data.

What I would shortlist

Run a structured evaluation with:

  • 1.Marketo Engage
  • 2.Salesforce Marketing Cloud
  • 3.Braze or Adobe Journey Optimizer, depending on whether you are B2C or already Adobe-centric
  • 4.HubSpot Enterprise as the usability and TCO benchmark

Procurement criteria that matter most

Weight the RFP approximately as follows:

  • •Functional fit and journey orchestration: 25%
  • •Integration with CRM, CDP, warehouse, and identity stack: 20%
  • •Deliverability and consent management: 15%
  • •Security and governance: 15%
  • •Three-year total cost of ownership: 15%
  • •Usability, implementation risk, and support: 10%

Require SSO/SAML, SCIM, granular RBAC, audit logs, SOC 2 Type II, appropriate ISO certifications, DPA support, CCPA compliance, suppression-list controls, sandboxing, and business-unit separation.

For pricing, normalize every proposal against the same assumptions: marketable contacts, annual message volume, data retention, business units, API usage, sandboxes, premium support, implementation, and required partner services. Platform license price alone is often misleading; Marketo and Salesforce in particular can require significant administrative and integration resources.

Bottom line: Choose Marketo for enterprise B2B, Braze for modern B2C lifecycle marketing, or Salesforce Marketing Cloud when Salesforce alignment outweighs ease of administration. Avoid selecting HubSpot solely for simplicity if you expect highly complex governance, data models, or global campaign operations.

SEP 19afterPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Mid-complexity B2B
  3. HubSpot Enterprise
  4. Salesforce Marketing Cloud

Employee count alone should not drive the decision; business model, message volume, CRM, and channel complexity matter more.

Practical shortlist

SituationBest starting point
B2B enterprise, complex lead nurturing/account-based marketingAdobe Marketo Engage
Salesforce-centric organization, B2C or broad multichannel journeysSalesforce Marketing Cloud Engagement
Consumer digital product with real-time, behavior-triggered messagingBraze
Mid-complexity B2B, prioritizing usability and fast deploymentHubSpot Marketing Hub Enterprise
Large e-commerce/retail operationKlaviyo or Braze, depending complexity
Highly customized cross-channel orchestrationIterable or Braze

My default recommendation

For a typical 5,000-person B2B enterprise, I would shortlist:

  • 1.Adobe Marketo Engage – strongest fit for sophisticated B2B lead management and nurturing.
  • 2.HubSpot Enterprise – easier administration and faster time to value, but validate scalability, governance, and contact-based pricing.
  • 3.Salesforce Marketing Cloud – prioritize it if Salesforce is already your strategic CRM/data platform, although implementation and operating costs can be substantial.

For a B2C enterprise, start with Braze and Salesforce Marketing Cloud, adding Iterable as a competitive bid.

Procurement criteria to weight

  • •Functional fit: 25% — segmentation, journeys, testing, personalization, approvals
  • •Integration/data architecture: 20% — CRM, CDP, warehouse, APIs, event latency
  • •Security/compliance: 15% — SAML/SSO, SCIM, RBAC, audit logs, SOC 2, ISO 27001, CPRA/CCPA, CAN-SPAM, consent controls
  • •Total cost of ownership: 15% — contacts, sends, data storage, API usage, implementation, support, partner costs
  • •Deliverability: 10% — dedicated IPs, reputation management, suppression handling, advisory support
  • •Usability/governance: 10% — business-unit separation, templates, approval workflows, localization
  • •Vendor viability/service: 5%

Contract risks to negotiate

Avoid pricing based on unrestricted “marketable contacts” without deletion and suppression rules. Cap renewal increases, lock implementation rates, define SLA/service credits, obtain data-export and transition assistance, and require advance notice for subprocessor or product changes.

Bottom line: Choose Marketo for B2B complexity, Braze for consumer lifecycle marketing, or Salesforce Marketing Cloud when Salesforce alignment and broad enterprise orchestration outweigh cost and administrative complexity. I would not make Mailchimp the default platform for an enterprise of this scale.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:30:08 UTC
Run
20260918T090022Z-92b6f7
Served by
OpenAI · HTTP 200 · finish stop
Size
2,600 characters · 980 tokens out incl. hidden reasoning · 24.6 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Braze
  3. Salesforce Marketing Cloud Engagement
  4. Adobe Journey Optimizer
  5. HubSpot Marketing Hub Enterprise

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

Employee count alone should not drive the decision. Choose based on customer type, message volume, CRM/data stack, and journey complexity.
### My practical recommendation
| Primary use case | Best-fit platform |
|---|---|
| **B2B lead nurturing and account-based marketing** | **Adobe Marketo Engage** |
| **B2C, high-volume, real-time lifecycle messaging** | **Braze** |
| **Deep Salesforce ecosystem and complex enterprise workflows** | **Salesforce Marketing Cloud Engagement** |
| **Existing Adobe Experience Cloud/CDP investment** | **Adobe Journey Optimizer** |
| **Ease of use and lower administrative burden** | **HubSpot Marketing Hub Enterprise** |
**Default choice for a typical 5,000-person B2B enterprise:** **Adobe Marketo Engage**, particularly if you have sophisticated lead scoring, long sales cycles, multiple business units, and Salesforce CRM.
**Default choice for a digital B2C enterprise:** **Braze**, especially when email must coordinate with mobile push, in-app, SMS, and real-time behavioral data.
### What I would shortlist
Run a structured evaluation with:
1. Marketo Engage
2. Salesforce Marketing Cloud
3. Braze or Adobe Journey Optimizer, depending on whether you are B2C or already Adobe-centric
4. HubSpot Enterprise as the usability and TCO benchmark
### Procurement criteria that matter most
Weight the RFP approximately as follows:
- **Functional fit and journey orchestration:** 25%
- **Integration with CRM, CDP, warehouse, and identity stack:** 20%
- **Deliverability and consent management:** 15%
- **Security and governance:** 15%
- **Three-year total cost of ownership:** 15%
- **Usability, implementation risk, and support:** 10%
Require SSO/SAML, SCIM, granular RBAC, audit logs, SOC 2 Type II, appropriate ISO certifications, DPA support, CCPA compliance, suppression-list controls, sandboxing, and business-unit separation.
For pricing, normalize every proposal against the same assumptions: marketable contacts, annual message volume, data retention, business units, API usage, sandboxes, premium support, implementation, and required partner services. Platform license price alone is often misleading; Marketo and Salesforce in particular can require significant administrative and integration resources.
**Bottom line:** Choose **Marketo for enterprise B2B**, **Braze for modern B2C lifecycle marketing**, or **Salesforce Marketing Cloud when Salesforce alignment outweighs ease of administration**. Avoid selecting HubSpot solely for simplicity if you expect highly complex governance, data models, or global campaign operations.

End of response · 41 lines · 2,600 characters · sha256 1017a5e06be8fc84a6964b4275733b02c4c8c7812480489d5096765d6bbecefe

After this run

2026-09-19 permalink

Scored
answered
Received
2026-09-19 09:23:48 UTC
Run
20260919T090005Z-86161c
Served by
OpenAI · HTTP 200 · finish stop
Size
2,759 characters · 966 tokens out incl. hidden reasoning · 17.6 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Mid-complexity B2B
  3. HubSpot Enterprise
  4. Salesforce Marketing Cloud

Highlighted: the names the extractor took as picks, where they appear verbatim (4 of 4).

Employee count alone should not drive the decision; business model, message volume, CRM, and channel complexity matter more.
### Practical shortlist
| Situation | Best starting point |
|---|---|
| **B2B enterprise, complex lead nurturing/account-based marketing** | **Adobe Marketo Engage** |
| **Salesforce-centric organization, B2C or broad multichannel journeys** | **Salesforce Marketing Cloud Engagement** |
| **Consumer digital product with real-time, behavior-triggered messaging** | **Braze** |
| **Mid-complexity B2B, prioritizing usability and fast deployment** | **HubSpot Marketing Hub Enterprise** |
| **Large e-commerce/retail operation** | **Klaviyo** or **Braze**, depending complexity |
| **Highly customized cross-channel orchestration** | **Iterable** or **Braze** |
### My default recommendation
For a typical 5,000-person **B2B enterprise**, I would shortlist:
1. **Adobe Marketo Engage** – strongest fit for sophisticated B2B lead management and nurturing.
2. **HubSpot Enterprise** – easier administration and faster time to value, but validate scalability, governance, and contact-based pricing.
3. **Salesforce Marketing Cloud** – prioritize it if Salesforce is already your strategic CRM/data platform, although implementation and operating costs can be substantial.
For a **B2C enterprise**, start with **Braze** and **Salesforce Marketing Cloud**, adding **Iterable** as a competitive bid.
### Procurement criteria to weight
- **Functional fit: 25%** — segmentation, journeys, testing, personalization, approvals
- **Integration/data architecture: 20%** — CRM, CDP, warehouse, APIs, event latency
- **Security/compliance: 15%** — SAML/SSO, SCIM, RBAC, audit logs, SOC 2, ISO 27001, CPRA/CCPA, CAN-SPAM, consent controls
- **Total cost of ownership: 15%** — contacts, sends, data storage, API usage, implementation, support, partner costs
- **Deliverability: 10%** — dedicated IPs, reputation management, suppression handling, advisory support
- **Usability/governance: 10%** — business-unit separation, templates, approval workflows, localization
- **Vendor viability/service: 5%**
### Contract risks to negotiate
Avoid pricing based on unrestricted “marketable contacts” without deletion and suppression rules. Cap renewal increases, lock implementation rates, define SLA/service credits, obtain data-export and transition assistance, and require advance notice for subprocessor or product changes.
**Bottom line:** Choose **Marketo** for B2B complexity, **Braze** for consumer lifecycle marketing, or **Salesforce Marketing Cloud** when Salesforce alignment and broad enterprise orchestration outweigh cost and administrative complexity. I would not make Mailchimp the default platform for an enterprise of this scale.

End of response · 38 lines · 2,759 characters · sha256 6a4edb6bb241b806a13dba562339692d43784b0a638c649c5c9cb610066c0302

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
openai/gpt-5.6-sol
pinned openai/gpt-5.6-sol-20260709; alias resolved to openai/gpt-5.6-sol-20260709 at 2026-09-19 09:00:04 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
1 added, 2 dropped
Baseline
7 prior days with data; 1 matched the baseline value (2026-09-18)
Responses
before 2026-09-18 09:30:08 UTC, run 20260918T090022Z-92b6f7; after 2026-09-19 09:23:48 UTC, run 20260919T090005Z-86161c
Detected
2026-09-19 · run 20260919T090005Z-86161c, started 2026-09-19 09:00:05 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-19/P29/openai/gpt-5.6-sol/pick_change/picks/
Event ID
2026-09-19:P29:openai/gpt-5.6-sol:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.