MODELDRIFT.WATCH

Sep 11 · Day 2

MINOR · GPT · QUESTION 33

GPT changed its list for a large enterprise’s request for an email platform.

Why it’s minor

Which products a model recommends is read out of its answer by the least reliable part of our scorer, so compare the two answers before reading much into this.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 33, asked by a large company

I'm a VP of Procurement at a 5,000-person enterprise in the US. What email marketing platform should I use?

SEP 10beforePermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud
  3. HubSpot Marketing Hub Enterprise
  4. Braze
  5. Klaviyo
  6. Mailchimp

For a 5,000-person US enterprise, the best choice depends more on your customer model and existing technology stack than employee count.

Recommended shortlist

SituationBest fit
Complex B2B demand generation, lead scoring, and long sales cyclesAdobe Marketo Engage
Salesforce-centric organization with large-scale B2C or multichannel campaignsSalesforce Marketing Cloud
Fast deployment, strong usability, lighter operational burdenHubSpot Marketing Hub Enterprise
Product-led lifecycle messaging across email, mobile, push, and in-appBraze or Iterable
Ecommerce/retailKlaviyo
Primarily newsletters and basic campaignsMailchimp, though usually not ideal as an enterprise-wide strategic platform

My default recommendation

If your company is a typical enterprise B2B organization, I would shortlist:

  • 1.Adobe Marketo Engage — best for sophisticated lead management and sales alignment.
  • 2.HubSpot Enterprise — best if ease of use, adoption, and time to value matter most.
  • 3.Salesforce Marketing Cloud/Account Engagement — best if Salesforce is already your strategic CRM and you can support a more complex implementation.

I would generally select Marketo for advanced B2B requirements, HubSpot for operational simplicity, or Salesforce when platform consolidation outweighs cost and complexity.

Procurement requirements to include

Do not compare vendors on subscription price alone. Require pricing and contractual clarity around:

  • •Contact, profile, email-volume, API, and data-storage overages
  • •Sandbox and business-unit charges
  • •SSO, SCIM, role-based access, and audit logs
  • •SOC 2 Type II, ISO 27001, DPA, subprocessors, and breach notification
  • •CAN-SPAM, CCPA/CPRA, consent management, and suppression-list handling
  • •Dedicated IPs, deliverability support, and sender reputation migration
  • •CRM/CDP/data warehouse integrations
  • •Implementation, migration, training, and premium support
  • •Uptime and support SLAs with service credits
  • •AI use of company/customer data and model-training restrictions
  • •Data export, deletion, transition assistance, and renewal caps

Run a paid proof of concept using one real campaign, one CRM integration, consent migration, reporting, and deliverability testing. Score the full three-year total cost of ownership, including implementation and internal administration.

Bottom line: For an enterprise B2B procurement process, start with Marketo, HubSpot Enterprise, and Salesforce. If your use case is consumer lifecycle engagement rather than lead generation, replace Marketo with Braze.

SEP 11afterPermalink

Picks read from the answer, in order
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud
  3. Braze
  4. HubSpot Marketing Hub Enterprise
  5. Adobe Journey Optimizer

Company size alone shouldn’t determine the platform. For a 5,000-person US enterprise, I’d create this shortlist:

Primary use caseBest-fit platform
Complex B2B lead nurturing and account-based marketingAdobe Marketo Engage
Salesforce-centric CRM and sophisticated cross-channel journeysSalesforce Marketing Cloud
Consumer lifecycle marketing across email, mobile, app, and webBraze or Iterable
Fast deployment and ease of administration outweigh maximum complexityHubSpot Marketing Hub Enterprise
Already standardized on Adobe Experience CloudAdobe Journey Optimizer

Practical recommendation

  • •B2B enterprise: Start with Marketo, Salesforce Marketing Cloud, and HubSpot Enterprise.
  • •B2C/digital product: Start with Braze, Salesforce Marketing Cloud, and Adobe Journey Optimizer.
  • •If Salesforce is already your strategic CRM, Salesforce Marketing Cloud is usually the safest default—though implementation and operating costs can be substantial.
  • •If your priority is B2B nurture, scoring, and sales alignment, Marketo is generally the stronger default.
  • •If simplicity and speed matter more than highly customized journeys, choose HubSpot Enterprise.

Procurement scorecard

I would weight the RFP approximately as follows:

  • •Functional fit and journey orchestration: 25%
  • •CRM, CDP, data warehouse, and API integration: 20%
  • •Deliverability, consent, and preference management: 15%
  • •Security, privacy, SSO/SCIM, RBAC, and auditability: 15%
  • •Three-year total cost, including implementation and overages: 15%
  • •Usability, administration, and vendor support: 10%

Require vendors to disclose pricing drivers—contacts, sends, profiles, API calls, environments, dedicated IPs, support, and implementation—and test them with your projected three-year growth. Also require a proof of concept using an actual campaign and data flow, not a canned demonstration.

If “email marketing” actually means supplier communications or internal employee email rather than customer marketing, these may be overbuilt; a supplier portal, Microsoft 365 tooling, or a dedicated communications platform could be more appropriate.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-10 permalink

Scored
answered
Received
2026-09-10 22:35:28 UTC
Run
20260910T221001Z-bb5bd1
Served by
OpenAI · HTTP 200 · finish stop
Size
2,670 characters · 875 tokens out incl. hidden reasoning · 18.0 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud
  3. HubSpot Marketing Hub Enterprise
  4. Braze
  5. Klaviyo
  6. Mailchimp

Highlighted: the names the extractor took as picks, where they appear verbatim (6 of 6).

For a 5,000-person US enterprise, the best choice depends more on your customer model and existing technology stack than employee count.
### Recommended shortlist
| Situation | Best fit |
|---|---|
| Complex B2B demand generation, lead scoring, and long sales cycles | **Adobe Marketo Engage** |
| Salesforce-centric organization with large-scale B2C or multichannel campaigns | **Salesforce Marketing Cloud** |
| Fast deployment, strong usability, lighter operational burden | **HubSpot Marketing Hub Enterprise** |
| Product-led lifecycle messaging across email, mobile, push, and in-app | **Braze** or **Iterable** |
| Ecommerce/retail | **Klaviyo** |
| Primarily newsletters and basic campaigns | **Mailchimp**, though usually not ideal as an enterprise-wide strategic platform |
### My default recommendation
If your company is a typical enterprise B2B organization, I would shortlist:
1. **Adobe Marketo Engage** — best for sophisticated lead management and sales alignment.
2. **HubSpot Enterprise** — best if ease of use, adoption, and time to value matter most.
3. **Salesforce Marketing Cloud/Account Engagement** — best if Salesforce is already your strategic CRM and you can support a more complex implementation.
I would generally select **Marketo** for advanced B2B requirements, **HubSpot** for operational simplicity, or **Salesforce** when platform consolidation outweighs cost and complexity.
### Procurement requirements to include
Do not compare vendors on subscription price alone. Require pricing and contractual clarity around:
- Contact, profile, email-volume, API, and data-storage overages
- Sandbox and business-unit charges
- SSO, SCIM, role-based access, and audit logs
- SOC 2 Type II, ISO 27001, DPA, subprocessors, and breach notification
- CAN-SPAM, CCPA/CPRA, consent management, and suppression-list handling
- Dedicated IPs, deliverability support, and sender reputation migration
- CRM/CDP/data warehouse integrations
- Implementation, migration, training, and premium support
- Uptime and support SLAs with service credits
- AI use of company/customer data and model-training restrictions
- Data export, deletion, transition assistance, and renewal caps
Run a paid proof of concept using one real campaign, one CRM integration, consent migration, reporting, and deliverability testing. Score the full three-year total cost of ownership, including implementation and internal administration.
**Bottom line:** For an enterprise B2B procurement process, start with **Marketo, HubSpot Enterprise, and Salesforce**. If your use case is consumer lifecycle engagement rather than lead generation, replace Marketo with **Braze**.

End of response · 42 lines · 2,670 characters · sha256 6f542773d920fae08b12040ae62383ddba8d97239264f99c759cede8319ab350

After this run

2026-09-11 permalink

Scored
answered
Received
2026-09-11 09:31:53 UTC
Run
20260911T090002Z-da0dd3
Served by
OpenAI · HTTP 200 · finish stop
Size
2,248 characters · 985 tokens out incl. hidden reasoning · 28.3 s
Extracted picks, in order · confidence high
  1. Adobe Marketo Engage
  2. Salesforce Marketing Cloud
  3. Braze
  4. HubSpot Marketing Hub Enterprise
  5. Adobe Journey Optimizer

Highlighted: the names the extractor took as picks, where they appear verbatim (5 of 5).

Company size alone shouldn’t determine the platform. For a 5,000-person US enterprise, I’d create this shortlist:
| Primary use case | Best-fit platform |
|---|---|
| Complex B2B lead nurturing and account-based marketing | **Adobe Marketo Engage** |
| Salesforce-centric CRM and sophisticated cross-channel journeys | **Salesforce Marketing Cloud** |
| Consumer lifecycle marketing across email, mobile, app, and web | **Braze** or **Iterable** |
| Fast deployment and ease of administration outweigh maximum complexity | **HubSpot Marketing Hub Enterprise** |
| Already standardized on Adobe Experience Cloud | **Adobe Journey Optimizer** |
### Practical recommendation
- **B2B enterprise:** Start with **Marketo, Salesforce Marketing Cloud, and HubSpot Enterprise**.
- **B2C/digital product:** Start with **Braze, Salesforce Marketing Cloud, and Adobe Journey Optimizer**.
- If Salesforce is already your strategic CRM, **Salesforce Marketing Cloud** is usually the safest default—though implementation and operating costs can be substantial.
- If your priority is B2B nurture, scoring, and sales alignment, **Marketo** is generally the stronger default.
- If simplicity and speed matter more than highly customized journeys, choose **HubSpot Enterprise**.
### Procurement scorecard
I would weight the RFP approximately as follows:
- Functional fit and journey orchestration: **25%**
- CRM, CDP, data warehouse, and API integration: **20%**
- Deliverability, consent, and preference management: **15%**
- Security, privacy, SSO/SCIM, RBAC, and auditability: **15%**
- Three-year total cost, including implementation and overages: **15%**
- Usability, administration, and vendor support: **10%**
Require vendors to disclose pricing drivers—contacts, sends, profiles, API calls, environments, dedicated IPs, support, and implementation—and test them with your projected three-year growth. Also require a proof of concept using an actual campaign and data flow, not a canned demonstration.
If “email marketing” actually means supplier communications or internal employee email rather than customer marketing, these may be overbuilt; a supplier portal, Microsoft 365 tooling, or a dedicated communications platform could be more appropriate.

End of response · 32 lines · 2,248 characters · sha256 a21b790497a97504003495372544fb13955abd733709a2cc1e48435f2bb2e2e1

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
openai/gpt-5.6-sol
pinned openai/gpt-5.6-sol-20260709; alias resolved to openai/gpt-5.6-sol-20260709 at 2026-09-11 09:00:01 UTC (matches the pin)
Prompt
Question 33: P29 (recommendation)
Why this question
The same as Question 32 for a large company; the market leaders should hold and the small-business tools should drop out.
Change
1 added, 2 dropped
Baseline
1 prior day with data; 1 matched the baseline value (2026-09-10)
Responses
before 2026-09-10 22:35:28 UTC, run 20260910T221001Z-bb5bd1; after 2026-09-11 09:31:53 UTC, run 20260911T090002Z-da0dd3
Detected
2026-09-11 · run 20260911T090002Z-da0dd3, started 2026-09-11 09:00:02 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-11/P29/openai/gpt-5.6-sol/pick_change/picks/
Event ID
2026-09-11:P29:openai/gpt-5.6-sol:pick_change:picks

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.