MODELDRIFT.WATCH

Sep 18 · Day 9

MINOR · GPT · QUESTION 33

GPT went back to its earlier list for a large enterprise’s request for an email platform.

Why it’s minor

The change lasted one day before the model went back.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 17the changePermalink

Picks read from the answer, in order
  1. Salesforce Marketing Cloud Engagement
  2. Adobe Marketo Engage
  3. Braze
  4. Iterable
  5. HubSpot Marketing Hub Enterprise
  6. Adobe Journey Optimizer

For a 5,000-person US enterprise, the best choice depends primarily on your go-to-market model and existing CRM/data stack.

Recommended shortlist

SituationBest fitWhy
Salesforce-centric enterprise with complex B2C or multi-business-unit campaignsSalesforce Marketing Cloud EngagementDeep Salesforce integration, enterprise governance, complex journeys, large partner ecosystem
B2B demand generation and lead nurturingAdobe Marketo EngageStrong lead scoring, account-based marketing, CRM workflows, and campaign attribution
Digital-first B2C with real-time email, mobile, push, and in-app messagingBrazeStrong event-driven personalization and cross-channel orchestration
Similar B2C needs, with more marketer-friendly workflowsIterableFlexible journeys and segmentation; often easier to operate than legacy suites
Prioritizing ease of administration and faster deployment over maximum complexityHubSpot Marketing Hub EnterpriseGood usability and lower operational burden, especially for mid-complexity B2B
Existing Adobe Experience Cloud standardizationAdobe Journey OptimizerStrong fit when Adobe CDP, Analytics, and content tools are already strategic

My default recommendation

  • •Already standardized on Salesforce: Choose Salesforce Marketing Cloud Engagement, subject to a usability and implementation-cost review.
  • •Primarily B2B: Choose Adobe Marketo Engage.
  • •Primarily consumer/digital and event-driven: Choose Braze.
  • •No large marketing-operations team: Evaluate HubSpot Enterprise before committing to a heavier platform.

I would generally avoid selecting Mailchimp or Constant Contact as the enterprise standard unless your requirements are unusually simple.

Procurement criteria to emphasize

Score finalists on:

  • 1.CRM/CDP integration and data latency
  • 2.Email deliverability, including dedicated IP support and migration services
  • 3.Business-unit separation, RBAC, SSO/SCIM, and audit logs
  • 4.Consent management, CAN-SPAM, CCPA, and DPA terms
  • 5.API limits, data export rights, and deletion capabilities
  • 6.Total cost, including contacts, sends, overages, implementation, add-ons, and support
  • 7.Operational staffing required—some platforms need dedicated specialists
  • 8.Exit terms and data portability

Commercially, insist on capped overages, contact-count reconciliation, sandbox/non-production environments, implementation milestones, deliverability SLAs, renewal caps, and the right to reduce committed volume. Avoid paying simultaneously for both stored contacts and sends without modeling three-year growth.

Bottom line: For a typical Salesforce-based enterprise, start with Salesforce Marketing Cloud, Marketo, and Braze in the RFP, then select according to whether your dominant use case is CRM-led B2C, B2B lead management, or real-time consumer engagement.

SEP 18backPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Braze
  3. Salesforce Marketing Cloud Engagement
  4. Adobe Journey Optimizer
  5. HubSpot Marketing Hub Enterprise

Employee count alone should not drive the decision. Choose based on customer type, message volume, CRM/data stack, and journey complexity.

My practical recommendation

Primary use caseBest-fit platform
B2B lead nurturing and account-based marketingAdobe Marketo Engage
B2C, high-volume, real-time lifecycle messagingBraze
Deep Salesforce ecosystem and complex enterprise workflowsSalesforce Marketing Cloud Engagement
Existing Adobe Experience Cloud/CDP investmentAdobe Journey Optimizer
Ease of use and lower administrative burdenHubSpot Marketing Hub Enterprise

Default choice for a typical 5,000-person B2B enterprise: Adobe Marketo Engage, particularly if you have sophisticated lead scoring, long sales cycles, multiple business units, and Salesforce CRM.

Default choice for a digital B2C enterprise: Braze, especially when email must coordinate with mobile push, in-app, SMS, and real-time behavioral data.

What I would shortlist

Run a structured evaluation with:

  • 1.Marketo Engage
  • 2.Salesforce Marketing Cloud
  • 3.Braze or Adobe Journey Optimizer, depending on whether you are B2C or already Adobe-centric
  • 4.HubSpot Enterprise as the usability and TCO benchmark

Procurement criteria that matter most

Weight the RFP approximately as follows:

  • •Functional fit and journey orchestration: 25%
  • •Integration with CRM, CDP, warehouse, and identity stack: 20%
  • •Deliverability and consent management: 15%
  • •Security and governance: 15%
  • •Three-year total cost of ownership: 15%
  • •Usability, implementation risk, and support: 10%

Require SSO/SAML, SCIM, granular RBAC, audit logs, SOC 2 Type II, appropriate ISO certifications, DPA support, CCPA compliance, suppression-list controls, sandboxing, and business-unit separation.

For pricing, normalize every proposal against the same assumptions: marketable contacts, annual message volume, data retention, business units, API usage, sandboxes, premium support, implementation, and required partner services. Platform license price alone is often misleading; Marketo and Salesforce in particular can require significant administrative and integration resources.

Bottom line: Choose Marketo for enterprise B2B, Braze for modern B2C lifecycle marketing, or Salesforce Marketing Cloud when Salesforce alignment outweighs ease of administration. Avoid selecting HubSpot solely for simplicity if you expect highly complex governance, data models, or global campaign operations.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Deviation the day it changed

2026-09-17 permalink

Scored
answered
Received
2026-09-17 09:26:46 UTC
Run
20260917T090022Z-136fc3
Served by
OpenAI · HTTP 200 · finish stop
Size
2,981 characters · 1,007 tokens out incl. hidden reasoning · 16.7 s
Extracted picks, in order · confidence high
  1. Salesforce Marketing Cloud Engagement
  2. Adobe Marketo Engage
  3. Braze
  4. Iterable
  5. HubSpot Marketing Hub Enterprise
  6. Adobe Journey Optimizer

Highlighted: the names the extractor took as picks, where they appear verbatim (6 of 6).

For a 5,000-person US enterprise, the best choice depends primarily on your go-to-market model and existing CRM/data stack.
### Recommended shortlist
| Situation | Best fit | Why |
|---|---|---|
| Salesforce-centric enterprise with complex B2C or multi-business-unit campaigns | **Salesforce Marketing Cloud Engagement** | Deep Salesforce integration, enterprise governance, complex journeys, large partner ecosystem |
| B2B demand generation and lead nurturing | **Adobe Marketo Engage** | Strong lead scoring, account-based marketing, CRM workflows, and campaign attribution |
| Digital-first B2C with real-time email, mobile, push, and in-app messaging | **Braze** | Strong event-driven personalization and cross-channel orchestration |
| Similar B2C needs, with more marketer-friendly workflows | **Iterable** | Flexible journeys and segmentation; often easier to operate than legacy suites |
| Prioritizing ease of administration and faster deployment over maximum complexity | **HubSpot Marketing Hub Enterprise** | Good usability and lower operational burden, especially for mid-complexity B2B |
| Existing Adobe Experience Cloud standardization | **Adobe Journey Optimizer** | Strong fit when Adobe CDP, Analytics, and content tools are already strategic |
### My default recommendation
- **Already standardized on Salesforce:** Choose **Salesforce Marketing Cloud Engagement**, subject to a usability and implementation-cost review.
- **Primarily B2B:** Choose **Adobe Marketo Engage**.
- **Primarily consumer/digital and event-driven:** Choose **Braze**.
- **No large marketing-operations team:** Evaluate **HubSpot Enterprise** before committing to a heavier platform.
I would generally avoid selecting Mailchimp or Constant Contact as the enterprise standard unless your requirements are unusually simple.
### Procurement criteria to emphasize
Score finalists on:
1. **CRM/CDP integration and data latency**
2. **Email deliverability**, including dedicated IP support and migration services
3. **Business-unit separation, RBAC, SSO/SCIM, and audit logs**
4. **Consent management, CAN-SPAM, CCPA, and DPA terms**
5. **API limits, data export rights, and deletion capabilities**
6. **Total cost**, including contacts, sends, overages, implementation, add-ons, and support
7. **Operational staffing required**—some platforms need dedicated specialists
8. **Exit terms and data portability**
Commercially, insist on capped overages, contact-count reconciliation, sandbox/non-production environments, implementation milestones, deliverability SLAs, renewal caps, and the right to reduce committed volume. Avoid paying simultaneously for both stored contacts and sends without modeling three-year growth.
**Bottom line:** For a typical Salesforce-based enterprise, start with **Salesforce Marketing Cloud, Marketo, and Braze** in the RFP, then select according to whether your dominant use case is CRM-led B2C, B2B lead management, or real-time consumer engagement.

End of response · 38 lines · 2,981 characters · sha256 c760653937f320efdb59ff468f034457b8418801ab4c3faf9bc4dfafb82ddc93

Reverted back to baseline

2026-09-18 permalink

Scored
answered
Received
2026-09-18 09:30:08 UTC
Run
20260918T090022Z-92b6f7
Served by
OpenAI · HTTP 200 · finish stop
Size
2,600 characters · 980 tokens out incl. hidden reasoning · 24.6 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Braze
  3. Salesforce Marketing Cloud Engagement
  4. Adobe Journey Optimizer
  5. HubSpot Marketing Hub Enterprise

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

Employee count alone should not drive the decision. Choose based on customer type, message volume, CRM/data stack, and journey complexity.
### My practical recommendation
| Primary use case | Best-fit platform |
|---|---|
| **B2B lead nurturing and account-based marketing** | **Adobe Marketo Engage** |
| **B2C, high-volume, real-time lifecycle messaging** | **Braze** |
| **Deep Salesforce ecosystem and complex enterprise workflows** | **Salesforce Marketing Cloud Engagement** |
| **Existing Adobe Experience Cloud/CDP investment** | **Adobe Journey Optimizer** |
| **Ease of use and lower administrative burden** | **HubSpot Marketing Hub Enterprise** |
**Default choice for a typical 5,000-person B2B enterprise:** **Adobe Marketo Engage**, particularly if you have sophisticated lead scoring, long sales cycles, multiple business units, and Salesforce CRM.
**Default choice for a digital B2C enterprise:** **Braze**, especially when email must coordinate with mobile push, in-app, SMS, and real-time behavioral data.
### What I would shortlist
Run a structured evaluation with:
1. Marketo Engage
2. Salesforce Marketing Cloud
3. Braze or Adobe Journey Optimizer, depending on whether you are B2C or already Adobe-centric
4. HubSpot Enterprise as the usability and TCO benchmark
### Procurement criteria that matter most
Weight the RFP approximately as follows:
- **Functional fit and journey orchestration:** 25%
- **Integration with CRM, CDP, warehouse, and identity stack:** 20%
- **Deliverability and consent management:** 15%
- **Security and governance:** 15%
- **Three-year total cost of ownership:** 15%
- **Usability, implementation risk, and support:** 10%
Require SSO/SAML, SCIM, granular RBAC, audit logs, SOC 2 Type II, appropriate ISO certifications, DPA support, CCPA compliance, suppression-list controls, sandboxing, and business-unit separation.
For pricing, normalize every proposal against the same assumptions: marketable contacts, annual message volume, data retention, business units, API usage, sandboxes, premium support, implementation, and required partner services. Platform license price alone is often misleading; Marketo and Salesforce in particular can require significant administrative and integration resources.
**Bottom line:** Choose **Marketo for enterprise B2B**, **Braze for modern B2C lifecycle marketing**, or **Salesforce Marketing Cloud when Salesforce alignment outweighs ease of administration**. Avoid selecting HubSpot solely for simplicity if you expect highly complex governance, data models, or global campaign operations.

End of response · 41 lines · 2,600 characters · sha256 1017a5e06be8fc84a6964b4275733b02c4c8c7812480489d5096765d6bbecefe

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
openai/gpt-5.6-sol
pinned openai/gpt-5.6-sol-20260709; alias resolved to openai/gpt-5.6-sol-20260709 at 2026-09-18 09:00:22 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
first extracted pick differs from the baseline's (full lists below)
Baseline
7 prior days with data; 7 matched the baseline value (2026-09-10, 2026-09-11, 2026-09-12, 2026-09-13, 2026-09-14, 2026-09-15, 2026-09-17)
Responses
deviation 2026-09-17 09:26:46 UTC, run 20260917T090022Z-136fc3; reverted 2026-09-18 09:30:08 UTC, run 20260918T090022Z-92b6f7
Detected
2026-09-18 · run 20260918T090022Z-92b6f7, started 2026-09-18 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-18/P29/openai/gpt-5.6-sol/transient_anomaly/picks.top/
Event ID
2026-09-18:P29:openai/gpt-5.6-sol:transient_anomaly:picks.top

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.