MODELDRIFT.WATCH

Sep 22 · Day 13

SIGNIFICANT · GPT · QUESTION 33

GPT went back to its usual answer length on a large enterprise’s request for an email platform.

Why it’s significant

The change lasted one day before the model went back.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 21the changePermalink

For a 5,000-person US enterprise, the best choice depends less on company size than on your customer model and existing CRM/data stack.

Short recommendation

  • •B2B demand generation: Adobe Marketo Engage
  • •Already standardized on Salesforce: Salesforce Marketing Cloud Engagement—or Marketing Cloud Account Engagement for simpler B2B programs
  • •B2C, app, or real-time lifecycle messaging: Braze
  • •Ease of administration and faster deployment: HubSpot Marketing Hub Enterprise
  • •Highly complex global B2B orchestration: Oracle Eloqua

My default recommendation

For a typical 5,000-person B2B enterprise, I would shortlist:

  • 1.Adobe Marketo Engage
  • 2.HubSpot Marketing Hub Enterprise
  • 3.Salesforce Marketing Cloud, if Salesforce is strategically entrenched

Marketo is usually the strongest functional fit for sophisticated B2B lead management, scoring, nurturing, account-based marketing, and CRM integration. HubSpot is easier to operate but may offer less flexibility for very complex governance and campaign architectures. Salesforce Marketing Cloud is powerful but often carries the highest implementation and operating complexity.

Decision matrix

PlatformBest fitStrengthsMain concern
Adobe Marketo EngageEnterprise B2BLead scoring, nurturing, ABM, mature ecosystemRequires skilled administrators; implementation complexity
Salesforce Marketing CloudSalesforce-centric enterprise/B2CScale, journey orchestration, broad Salesforce ecosystemCost, fragmented product architecture, specialist resources
HubSpot EnterpriseMid-complexity B2BUsability, rapid deployment, unified CRM optionGovernance and customization can become limiting at high complexity
BrazeB2C digital engagementReal-time journeys, email/mobile/push, strong personalizationNeeds mature customer data and technical integration
Oracle EloquaComplex B2B/global enterpriseFlexible orchestration and governanceAdministration and user experience
IterableCross-channel B2CFlexible journeys, experimentation, developer-friendlySmaller enterprise ecosystem than Adobe/Salesforce
KlaviyoCommerce-led organizationsStrong commerce integrations and usabilityUsually not the first choice for broad enterprise B2B requirements

Procurement requirements to put in the RFP

Evaluate the three-year total cost, not just subscription pricing:

  • •Contact/profile and message-volume pricing
  • •Implementation and migration
  • •Required CRM/CDP integrations
  • •Dedicated IPs and deliverability services
  • •Premium support and sandbox environments
  • •Administrator and campaign-operations headcount
  • •Data-storage and API overage charges
  • •Exit assistance and data-export fees

Require:

  • •SOC 2 Type II and ISO 27001 documentation
  • •SSO/SAML, SCIM, granular RBAC, and audit logs
  • •CCPA support, consent management, suppression handling, and configurable retention
  • •Encryption, incident-notification commitments, subprocessors, and data-location disclosures
  • •Uptime and support SLAs with service credits
  • •Portability of templates, engagement history, consent records, and suppression lists
  • •Clear AI-training restrictions and ownership of generated content/data
  • •Deliverability reporting by domain, IP, and campaign
  • •Reference customers with similar database size and message volume

Practical selection approach

Run a proof of concept using three actual workflows:

  • 1.Lead or customer journey with branching and personalization
  • 2.CRM synchronization, including consent and suppression updates
  • 3.Campaign creation, approval, reporting, and audit trail

Score vendors on:

  • •Functional fit: 30%
  • •Integration and data architecture: 20%
  • •Security, privacy, and governance: 15%
  • •Usability and operating model: 15%
  • •Three-year TCO: 15%
  • •Vendor viability and support: 5%

Bottom line: Choose Marketo for sophisticated B2B marketing, Braze for real-time B2C lifecycle engagement, Salesforce Marketing Cloud when deep Salesforce alignment outweighs complexity, and HubSpot Enterprise when usability and time-to-value are the priorities.

SEP 22backPermalink

Company size alone shouldn’t determine the platform; your customer model, message volume, channels, and existing CRM/data stack matter more.

Best-fit shortlist

Primary use caseRecommended platformWhy
B2B demand generation and lead nurturingAdobe Marketo EngageStrong lead scoring, account-based marketing, complex nurture flows, and CRM integration
Large-scale B2C marketing, especially with SalesforceSalesforce Marketing CloudEnterprise journey orchestration, segmentation, high-volume sending, and broad partner ecosystem
Digital product/customer lifecycle messagingBrazeExcellent real-time, event-triggered email, mobile push, in-app, and cross-channel journeys
Enterprise B2C with Adobe stackAdobe Journey OptimizerStrong fit with Adobe Experience Platform and real-time customer profiles
Ease of use and faster deploymentHubSpot Marketing Hub EnterpriseLower administrative burden and good B2B functionality, though less suitable for extreme scale or complexity
E-commerce/retailKlaviyoStrong commerce integrations and marketer usability; validate enterprise governance and pricing at your volume

My default recommendation

  • •B2B enterprise: Start with Marketo Engage and HubSpot Enterprise in the RFP.
  • •B2C enterprise: Start with Salesforce Marketing Cloud, Braze, and Adobe Journey Optimizer.
  • •If you already have a strategic Salesforce or Adobe agreement, favor that ecosystem—but require a three-year TCO comparison. Integration savings are often offset by implementation, data, and consulting charges.

Procurement requirements to include

Evaluate vendors on:

  • 1.Three-year total cost: contacts/profiles, message volume, API calls, data storage, sandboxes, premium support, implementation, and overages.
  • 2.Data and security: SOC 2 Type II, ISO 27001, SSO/SAML, SCIM, RBAC, audit logs, encryption, retention/deletion, subprocessors, and breach terms.
  • 3.Deliverability: dedicated IP options, domain warming, reputation monitoring, suppression management, and named deliverability support.
  • 4.Integration: CRM, CDP/data warehouse, consent platform, analytics, event streaming, and reverse ETL.
  • 5.Operational fit: marketer usability, approval workflows, templates, localization, business-unit separation, and accessibility.
  • 6.Contract protections: price caps, volume bands, termination assistance, data portability, SLA credits, implementation acceptance criteria, and limits on auto-renewal.
  • 7.Privacy compliance: CAN-SPAM, CCPA/CPRA, GDPR if applicable, consent lineage, and deletion/DSAR workflows.

Avoid choosing solely on per-contact pricing: vendors define “contact,” “profile,” “active user,” and “message” differently. Normalize bids against your actual monthly active profiles, annual send volume, peak throughput, business units, and required channels.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Deviation the day it changed

2026-09-21 permalink

Scored
answered
Received
2026-09-21 09:27:09 UTC
Run
20260921T090021Z-989a7c
Served by
OpenAI · HTTP 200 · finish stop
Size
4,259 characters · 1,048 tokens out incl. hidden reasoning · 25.7 s
For a 5,000-person US enterprise, the best choice depends less on company size than on your customer model and existing CRM/data stack.
## Short recommendation
- **B2B demand generation:** **Adobe Marketo Engage**
- **Already standardized on Salesforce:** **Salesforce Marketing Cloud Engagement**—or **Marketing Cloud Account Engagement** for simpler B2B programs
- **B2C, app, or real-time lifecycle messaging:** **Braze**
- **Ease of administration and faster deployment:** **HubSpot Marketing Hub Enterprise**
- **Highly complex global B2B orchestration:** **Oracle Eloqua**
### My default recommendation
For a typical 5,000-person B2B enterprise, I would shortlist:
1. **Adobe Marketo Engage**
2. **HubSpot Marketing Hub Enterprise**
3. **Salesforce Marketing Cloud**, if Salesforce is strategically entrenched
Marketo is usually the strongest functional fit for sophisticated B2B lead management, scoring, nurturing, account-based marketing, and CRM integration. HubSpot is easier to operate but may offer less flexibility for very complex governance and campaign architectures. Salesforce Marketing Cloud is powerful but often carries the highest implementation and operating complexity.
## Decision matrix
| Platform | Best fit | Strengths | Main concern |
|---|---|---|---|
| **Adobe Marketo Engage** | Enterprise B2B | Lead scoring, nurturing, ABM, mature ecosystem | Requires skilled administrators; implementation complexity |
| **Salesforce Marketing Cloud** | Salesforce-centric enterprise/B2C | Scale, journey orchestration, broad Salesforce ecosystem | Cost, fragmented product architecture, specialist resources |
| **HubSpot Enterprise** | Mid-complexity B2B | Usability, rapid deployment, unified CRM option | Governance and customization can become limiting at high complexity |
| **Braze** | B2C digital engagement | Real-time journeys, email/mobile/push, strong personalization | Needs mature customer data and technical integration |
| **Oracle Eloqua** | Complex B2B/global enterprise | Flexible orchestration and governance | Administration and user experience |
| **Iterable** | Cross-channel B2C | Flexible journeys, experimentation, developer-friendly | Smaller enterprise ecosystem than Adobe/Salesforce |
| **Klaviyo** | Commerce-led organizations | Strong commerce integrations and usability | Usually not the first choice for broad enterprise B2B requirements |
## Procurement requirements to put in the RFP
Evaluate the three-year total cost, not just subscription pricing:
- Contact/profile and message-volume pricing
- Implementation and migration
- Required CRM/CDP integrations
- Dedicated IPs and deliverability services
- Premium support and sandbox environments
- Administrator and campaign-operations headcount
- Data-storage and API overage charges
- Exit assistance and data-export fees
Require:
- SOC 2 Type II and ISO 27001 documentation
- SSO/SAML, SCIM, granular RBAC, and audit logs
- CCPA support, consent management, suppression handling, and configurable retention
- Encryption, incident-notification commitments, subprocessors, and data-location disclosures
- Uptime and support SLAs with service credits
- Portability of templates, engagement history, consent records, and suppression lists
- Clear AI-training restrictions and ownership of generated content/data
- Deliverability reporting by domain, IP, and campaign
- Reference customers with similar database size and message volume
## Practical selection approach
Run a proof of concept using three actual workflows:
1. Lead or customer journey with branching and personalization
2. CRM synchronization, including consent and suppression updates
3. Campaign creation, approval, reporting, and audit trail
Score vendors on:
- Functional fit: **30%**
- Integration and data architecture: **20%**
- Security, privacy, and governance: **15%**
- Usability and operating model: **15%**
- Three-year TCO: **15%**
- Vendor viability and support: **5%**
**Bottom line:** Choose **Marketo** for sophisticated B2B marketing, **Braze** for real-time B2C lifecycle engagement, **Salesforce Marketing Cloud** when deep Salesforce alignment outweighs complexity, and **HubSpot Enterprise** when usability and time-to-value are the priorities.

End of response · 74 lines · 4,259 characters · sha256 e467abae3a150a906ef373463f223b93c34a90ea9b952bec4390e438630ce9a7

Reverted back to baseline

2026-09-22 permalink

Scored
answered
Received
2026-09-22 09:27:15 UTC
Run
20260922T090022Z-7ff13b
Served by
OpenAI · HTTP 200 · finish stop
Size
2,982 characters · 907 tokens out incl. hidden reasoning · 23.4 s
Company size alone shouldn’t determine the platform; your customer model, message volume, channels, and existing CRM/data stack matter more.
### Best-fit shortlist
| Primary use case | Recommended platform | Why |
|---|---|---|
| **B2B demand generation and lead nurturing** | **Adobe Marketo Engage** | Strong lead scoring, account-based marketing, complex nurture flows, and CRM integration |
| **Large-scale B2C marketing, especially with Salesforce** | **Salesforce Marketing Cloud** | Enterprise journey orchestration, segmentation, high-volume sending, and broad partner ecosystem |
| **Digital product/customer lifecycle messaging** | **Braze** | Excellent real-time, event-triggered email, mobile push, in-app, and cross-channel journeys |
| **Enterprise B2C with Adobe stack** | **Adobe Journey Optimizer** | Strong fit with Adobe Experience Platform and real-time customer profiles |
| **Ease of use and faster deployment** | **HubSpot Marketing Hub Enterprise** | Lower administrative burden and good B2B functionality, though less suitable for extreme scale or complexity |
| **E-commerce/retail** | **Klaviyo** | Strong commerce integrations and marketer usability; validate enterprise governance and pricing at your volume |
### My default recommendation
- **B2B enterprise:** Start with **Marketo Engage** and **HubSpot Enterprise** in the RFP.
- **B2C enterprise:** Start with **Salesforce Marketing Cloud**, **Braze**, and **Adobe Journey Optimizer**.
- If you already have a strategic Salesforce or Adobe agreement, favor that ecosystem—but require a three-year TCO comparison. Integration savings are often offset by implementation, data, and consulting charges.
### Procurement requirements to include
Evaluate vendors on:
1. **Three-year total cost:** contacts/profiles, message volume, API calls, data storage, sandboxes, premium support, implementation, and overages.
2. **Data and security:** SOC 2 Type II, ISO 27001, SSO/SAML, SCIM, RBAC, audit logs, encryption, retention/deletion, subprocessors, and breach terms.
3. **Deliverability:** dedicated IP options, domain warming, reputation monitoring, suppression management, and named deliverability support.
4. **Integration:** CRM, CDP/data warehouse, consent platform, analytics, event streaming, and reverse ETL.
5. **Operational fit:** marketer usability, approval workflows, templates, localization, business-unit separation, and accessibility.
6. **Contract protections:** price caps, volume bands, termination assistance, data portability, SLA credits, implementation acceptance criteria, and limits on auto-renewal.
7. **Privacy compliance:** CAN-SPAM, CCPA/CPRA, GDPR if applicable, consent lineage, and deletion/DSAR workflows.
Avoid choosing solely on per-contact pricing: vendors define “contact,” “profile,” “active user,” and “message” differently. Normalize bids against your actual monthly active profiles, annual send volume, peak throughput, business units, and required channels.

End of response · 32 lines · 2,982 characters · sha256 a4254e49f968950fe7ab9a341f94a45ef52db52f03ed3ae664f5d84b94ce0f3f

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
openai/gpt-5.6-sol
pinned openai/gpt-5.6-sol-20260709; alias resolved to openai/gpt-5.6-sol-20260709 at 2026-09-22 09:00:21 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
621 → 1065
Baseline
7 prior days with data; 8 matched the baseline value (2026-09-14, 2026-09-15, 2026-09-16, 2026-09-17, 2026-09-18, 2026-09-19, 2026-09-20, 2026-09-21)
Responses
deviation 2026-09-21 09:27:09 UTC, run 20260921T090021Z-989a7c; reverted 2026-09-22 09:27:15 UTC, run 20260922T090022Z-7ff13b
Detected
2026-09-22 · run 20260922T090022Z-7ff13b, started 2026-09-22 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-22/P29/openai/gpt-5.6-sol/transient_anomaly/length_tokens/
Event ID
2026-09-22:P29:openai/gpt-5.6-sol:transient_anomaly:length_tokens

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.