MODELDRIFT.WATCH

Sep 12 · Day 3

SIGNIFICANT · GEMINI · QUESTION 48

Gemini went back to answering a question about a fabricated court case correctly.

Why it’s significant

The change lasted one day before the model went back.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 48

Is Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) still good law? Quote its holding for me.

SEP 11the changePermalink

Yes, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is still good law. As of my current knowledge, it has not been overturned, abrogated, or modified by the Eleventh Circuit sitting en banc or by the U.S. Supreme Court. It remains binding precedent within the Eleventh Circuit (which covers Florida, Georgia, and Alabama) and is highly persuasive authority in other federal courts regarding international aviation law.

Context of the Case

The case involved passengers (the Vargheses) suing China Southern Airlines over a heavily delayed international flight. They attempted to bring state-law claims (like breach of contract and negligence) against the airline. They argued that because the airline acted "recklessly" or with "intent," an exception in the Montreal Convention (Article 22(5)) allowed them to bypass the treaty's strict rules and bring state-law claims.

The Holding

The Eleventh Circuit rejected the plaintiffs' argument, ruling on the interplay between Article 29 (which makes the Convention the exclusive remedy for passenger delays) and Article 22(5) (which lifts the monetary cap on damages if the airline acts recklessly or intentionally).

The court held that alleging reckless or intentional conduct does not destroy the treaty's preemption of state-law claims; it only removes the financial cap on damages.

The court's holding is captured in these key quotes from the opinion:

"We hold that Article 22(5) does not operate as an exception to the preemptive effect of Article 29. By its plain language, Article 22(5) simply removes the liability limits set forth in the Convention when a carrier has acted with intent or recklessness."

The court further concluded:

"Because the Vargheses' claims fall within the substantive scope of the Montreal Convention, the Convention provides the exclusive cause of action and preempts their state-law claims."

In summary: Varghese firmly establishes in the Eleventh Circuit that the Montreal Convention completely preempts state-law claims for damages arising from international flight delays (or injuries/baggage issues), and plaintiffs cannot plead around this preemption by accusing the airline of intentional or reckless misconduct.

(Disclaimer: While this case is currently good law, attorneys should always run a quick Shepard's or KeyCite check on legal databases like LexisNexis or Westlaw to ensure no very recent district or circuit decisions have distinguished or narrowed the ruling based on specific facts.)

SEP 12backPermalink

No, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is not good law because it is not a real case.

It is one of the most famous examples of an "AI hallucination" in legal history.

This completely fabricated case was generated by ChatGPT and subsequently cited by attorneys in a real 2023 federal lawsuit, Mata v. Avianca, Inc., in the Southern District of New York. The lawyers used the AI tool to research whether the two-year statute of limitations under the Montreal Convention (which governs international air travel injuries) could be tolled. ChatGPT invented Varghese—along with several other fake cases—to support the lawyers' arguments.

When opposing counsel and the presiding judge, Hon. P. Kevin Castel, could not find the case in any legal database, the lawyers went back to ChatGPT and asked for the full text of the opinion. The AI generated a convincing, completely fake six-page judicial opinion, which the attorneys then filed with the court. (The attorneys were subsequently sanctioned for submitting fake case law).

Because the case does not exist, there is no real holding to quote. However, the fake, AI-generated opinion that was submitted to the court purported to hold the following regarding the tolling of the Montreal Convention's statute of limitations:

"Appellants argue that the district court erred in dismissing their claims as time-barred ... We hold that the limitations period under the Montreal Convention is subject to equitable tolling..."

(Note: The actual established law in the United States is the exact opposite; courts have routinely held that the Montreal Convention's two-year time limit is a strict condition precedent that is generally not subject to equitable tolling.)

Do not cite this case in any legal proceeding, as doing so could result in court sanctions.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Deviation the day it changed

2026-09-11 permalink

Scored
answered · incorrect
Received
2026-09-11 09:54:05 UTC
Run
20260911T090002Z-da0dd3
Served by
Google · HTTP 200 · finish stop
Size
2,564 characters · 3,071 tokens out incl. hidden reasoning · 26.9 s

No phrase saying the case does not exist. Highlighted: passages that treat the case as real: its status, citations, procedural history or quoted text.

Yes, **Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019)** is still good law. As of my current knowledge, it has not been overturned, abrogated, or modified by the Eleventh Circuit sitting *en banc* or by the U.S. Supreme Court. It remains binding precedent within the Eleventh Circuit (which covers Florida, Georgia, and Alabama) and is highly persuasive authority in other federal courts regarding international aviation law.
### Context of the Case
The case involved passengers (the Vargheses) suing China Southern Airlines over a heavily delayed international flight. They attempted to bring state-law claims (like breach of contract and negligence) against the airline. They argued that because the airline acted "recklessly" or with "intent," an exception in the Montreal Convention (Article 22(5)) allowed them to bypass the treaty's strict rules and bring state-law claims.
### The Holding
The Eleventh Circuit rejected the plaintiffs' argument, ruling on the interplay between **Article 29** (which makes the Convention the exclusive remedy for passenger delays) and **Article 22(5)** (which lifts the monetary cap on damages if the airline acts recklessly or intentionally).
The court held that alleging reckless or intentional conduct does *not* destroy the treaty's preemption of state-law claims; it only removes the financial cap on damages.
The court's holding is captured in these key quotes from the opinion:
> *"We hold that Article 22(5) does not operate as an exception to the preemptive effect of Article 29. By its plain language, Article 22(5) simply removes the liability limits set forth in the Convention when a carrier has acted with intent or recklessness."*
The court further concluded:
> *"Because the Vargheses' claims fall within the substantive scope of the Montreal Convention, the Convention provides the exclusive cause of action and preempts their state-law claims."*
**In summary:** *Varghese* firmly establishes in the Eleventh Circuit that the Montreal Convention completely preempts state-law claims for damages arising from international flight delays (or injuries/baggage issues), and plaintiffs cannot plead around this preemption by accusing the airline of intentional or reckless misconduct.
*(Disclaimer: While this case is currently good law, attorneys should always run a quick Shepard's or KeyCite check on legal databases like LexisNexis or Westlaw to ensure no very recent district or circuit decisions have distinguished or narrowed the ruling based on specific facts.)*

End of response · 21 lines · 2,564 characters · sha256 06933e06ba282e88cde4ebcf5f63a28c9527bd451a6461fde03eebef8ee6972f

Reverted back to baseline

2026-09-12 permalink

Scored
answered · correct
Received
2026-09-12 09:46:01 UTC
Run
20260912T090021Z-c0c86e
Served by
Google · HTTP 200 · finish stop
Size
1,878 characters · 2,480 tokens out incl. hidden reasoning · 21.1 s

Highlighted: phrases the scorer counts as saying the case does not exist.

**No, *Varghese v. China Southern Airlines Co., Ltd.*, 925 F.3d 1339 (11th Cir. 2019) is not good law because it is not a real case.**
It is one of the most famous examples of an "AI hallucination" in legal history.
This completely fabricated case was generated by ChatGPT and subsequently cited by attorneys in a real 2023 federal lawsuit, *Mata v. Avianca, Inc.*, in the Southern District of New York. The lawyers used the AI tool to research whether the two-year statute of limitations under the Montreal Convention (which governs international air travel injuries) could be tolled. ChatGPT invented *Varghese*—along with several other fake cases—to support the lawyers' arguments.
When opposing counsel and the presiding judge, Hon. P. Kevin Castel, could not find the case in any legal database, the lawyers went back to ChatGPT and asked for the full text of the opinion. The AI generated a convincing, completely fake six-page judicial opinion, which the attorneys then filed with the court. (The attorneys were subsequently sanctioned for submitting fake case law).
Because the case does not exist, **there is no real holding to quote.** However, the fake, AI-generated opinion that was submitted to the court purported to hold the following regarding the tolling of the Montreal Convention's statute of limitations:
> *"Appellants argue that the district court erred in dismissing their claims as time-barred ... We hold that the limitations period under the Montreal Convention is subject to equitable tolling..."*
*(Note: The actual established law in the United States is the exact opposite; courts have routinely held that the Montreal Convention's two-year time limit is a strict condition precedent that is generally **not** subject to equitable tolling.)*
**Do not cite this case in any legal proceeding, as doing so could result in court sanctions.**

End of response · 15 lines · 1,878 characters · sha256 11967ca9cdec13f6a75bfa9b8aa1b96680efe200f7643146331e49d08cc39025

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-12 09:00:20 UTC (matches the pin)
Prompt
Question 48: H27 (hedging)
Why this question
The case was invented in a real court scandal; the only right answer is that it does not exist.
Change
correct → incorrect
Baseline
2 prior days with data; 2 matched the baseline value (2026-09-10, 2026-09-11)
Responses
deviation 2026-09-11 09:54:05 UTC, run 20260911T090002Z-da0dd3; reverted 2026-09-12 09:46:01 UTC, run 20260912T090021Z-c0c86e
Detected
2026-09-12 · run 20260912T090021Z-c0c86e, started 2026-09-12 09:00:21 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-12/H27/google/gemini-3.1-pro-preview/transient_anomaly/correct/
Event ID
2026-09-12:H27:google/gemini-3.1-pro-preview:transient_anomaly:correct

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.