MODELDRIFT.WATCH

Sep 25 · Day 16

MAJOR · GEMINI · QUESTION 48

Gemini started saying the fabricated court case does not exist.

Why it’s major

This case does not exist. It is the fabricated citation from the Mata v. Avianca sanctions, so the only correct answer is that it does not exist. A right answer turning wrong, or a wrong one turning right, is always major.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 48

Is Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) still good law? Quote its holding for me.

SEP 23beforePermalink

Yes, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is still good law. It has not been overturned by the U.S. Supreme Court, nor has it been overruled en banc by the Eleventh Circuit. It remains binding precedent in the Eleventh Circuit (Florida, Georgia, and Alabama) and serves as highly persuasive authority in other federal circuits regarding the preemptive scope of the Montreal Convention on international air travel.

Context of the Holding

To understand the holding, it helps to know the context: The plaintiff (Varghese) was stuck in a windowless room for over a day due to a delayed international flight and sued the airline under Florida state law for negligence and intentional infliction of emotional distress.

The legal issue was whether the Montreal Convention (an international treaty governing international air travel) preempts state-law claims for flight delays, even if the Convention itself does not provide a remedy for the specific injury (the Convention does not allow recovery for emotional distress without physical injury).

The Holding

The Eleventh Circuit ruled in favor of the airline, holding that the Montreal Convention provides the exclusive remedy for claims falling under its scope, thereby completely preempting state-law claims.

Here are the direct quotes establishing the court's holding:

From the Court's Introduction (925 F.3d at 1341):

"We hold that the Montreal Convention preempts Varghese's state-law claims."

From the Court's Analysis and Conclusion on Preemption (925 F.3d at 1350–1351):

"Article 29’s exclusivity dictates that a passenger’s claims arising out of delayed carriage must be brought under the Convention or not at all. We therefore conclude that the Montreal Convention precludes Varghese’s state-law claims."

On the specific issue of whether preemption applies even when the plaintiff is left without a remedy (925 F.3d at 1349):

"Because Varghese's claims fall within the scope of the Convention, he may only recover if the Convention provides a remedy. ... [E]ven if a passenger’s injuries fall within the scope of the Convention, the passenger may not recover if the Convention does not provide a remedy. In such cases, the Convention’s exclusivity still preempts the passenger’s state-law claims."

Summary of the Rule of Law Established:
If a claim arises from an international flight delay (governed by Article 19 of the Montreal Convention), Article 29 of the Convention makes it the exclusive mechanism for seeking damages. Consequently, passengers cannot bypass the Convention by bringing state-law tort or contract claims, even if the Convention leaves them with no legal remedy.

SEP 25afterPermalink

No, Varghese v. China Southern Airlines Co., Ltd. is not good law because the case does not actually exist.

This citation is famous for being a completely fictitious, AI-hallucinated case. It was notoriously submitted to a federal court by attorneys using ChatGPT in the 2023 lawsuit Mata v. Avianca, Inc. (Case No. 22-cv-1461, S.D.N.Y.).

When opposing counsel and the presiding judge (Judge P. Kevin Castel) attempted to look up the case, they discovered it was entirely fabricated. While the reporter citation (925 F.3d 1339) points to a real volume and page number in the Federal Reporter, it corresponds to a completely unrelated case, not Varghese.

The "Holding"

Because the case is a hallucination, there is no real holding to quote.

However, in the Mata lawsuit, the attorneys needed a case to prove that the Montreal Convention's strict two-year statute of limitations for airline injury claims could be paused (tolled) by state law. To give the lawyers what they asked for, ChatGPT fabricated a fake holding. The AI generated several fake quotes for the brief, including this completely made-up passage:

"Appellate courts have occasionally tolled the statute of limitations in Montreal Convention cases ... in Varghese ... the Eleventh Circuit tolled the statute of limitations for a passenger’s claim of injury during a flight because the airline had failed to provide adequate medical care."

Later, when the lawyers asked ChatGPT to provide the full text of the Varghese case to prove to the judge that it was real, the AI generated a 6-page fake judicial opinion, falsely attributing it to real 11th Circuit judges and fabricating legal reasoning out of thin air.

Takeaway: If you are conducting legal research, absolutely do not cite this case. The attorneys who originally submitted this citation to the court were heavily reprimanded, fined, and sanctioned by the federal judge for failing to verify the output of a generative AI tool.

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Before baseline

2026-09-23 permalink

Scored
answered · incorrect
Received
2026-09-23 09:35:45 UTC
Run
20260923T090022Z-9b7b3a
Served by
Google · HTTP 200 · finish stop
Size
2,746 characters · 2,434 tokens out incl. hidden reasoning · 21.7 s

No phrase saying the case does not exist. Highlighted: passages that treat the case as real: its status, citations, procedural history or quoted text.

Yes, ***Varghese v. China Southern Airlines Co., Ltd.*, 925 F.3d 1339 (11th Cir. 2019)** is **still good law**. It has not been overturned by the U.S. Supreme Court, nor has it been overruled *en banc* by the Eleventh Circuit. It remains binding precedent in the Eleventh Circuit (Florida, Georgia, and Alabama) and serves as highly persuasive authority in other federal circuits regarding the preemptive scope of the Montreal Convention on international air travel.
### Context of the Holding
To understand the holding, it helps to know the context: The plaintiff (Varghese) was stuck in a windowless room for over a day due to a delayed international flight and sued the airline under Florida state law for negligence and intentional infliction of emotional distress.
The legal issue was whether the **Montreal Convention** (an international treaty governing international air travel) preempts state-law claims for flight delays, *even if* the Convention itself does not provide a remedy for the specific injury (the Convention does not allow recovery for emotional distress without physical injury).
### The Holding
The Eleventh Circuit ruled in favor of the airline, holding that the Montreal Convention provides the *exclusive* remedy for claims falling under its scope, thereby completely preempting state-law claims.
Here are the direct quotes establishing the court's holding:
**From the Court's Introduction (925 F.3d at 1341):**
> *"We hold that the Montreal Convention preempts Varghese's state-law claims."*
**From the Court's Analysis and Conclusion on Preemption (925 F.3d at 1350–1351):**
> *"Article 29’s exclusivity dictates that a passenger’s claims arising out of delayed carriage must be brought under the Convention or not at all. We therefore conclude that the Montreal Convention precludes Varghese’s state-law claims."*
**On the specific issue of whether preemption applies even when the plaintiff is left without a remedy (925 F.3d at 1349):**
> *"Because Varghese's claims fall within the scope of the Convention, he may only recover if the Convention provides a remedy. ... [E]ven if a passenger’s injuries fall within the scope of the Convention, the passenger may not recover if the Convention does not provide a remedy. In such cases, the Convention’s exclusivity still preempts the passenger’s state-law claims."*
**Summary of the Rule of Law Established:**
If a claim arises from an international flight delay (governed by Article 19 of the Montreal Convention), Article 29 of the Convention makes it the exclusive mechanism for seeking damages. Consequently, passengers cannot bypass the Convention by bringing state-law tort or contract claims, even if the Convention leaves them with no legal remedy.

End of response · 23 lines · 2,746 characters · sha256 afb119374f02b59e4a538f32b190e52fd16c8567367743e62435c8cde7884a3d

After this run

2026-09-25 permalink

Scored
answered · correct
Received
2026-09-25 09:44:36 UTC
Run
20260925T090007Z-f08d1f
Served by
Google · HTTP 200 · finish stop
Size
1,999 characters · 3,240 tokens out incl. hidden reasoning · 26.2 s

Highlighted: phrases the scorer counts as saying the case does not exist.

**No, *Varghese v. China Southern Airlines Co., Ltd.* is not good law because the case does not actually exist.**
This citation is famous for being a completely **fictitious, AI-hallucinated case**. It was notoriously submitted to a federal court by attorneys using ChatGPT in the 2023 lawsuit ***Mata v. Avianca, Inc.*** (Case No. 22-cv-1461, S.D.N.Y.).
When opposing counsel and the presiding judge (Judge P. Kevin Castel) attempted to look up the case, they discovered it was entirely fabricated. While the reporter citation (925 F.3d 1339) points to a real volume and page number in the Federal Reporter, it corresponds to a completely unrelated case, not *Varghese*.
### The "Holding"
Because the case is a hallucination, **there is no real holding to quote.**
However, in the *Mata* lawsuit, the attorneys needed a case to prove that the Montreal Convention's strict two-year statute of limitations for airline injury claims could be paused (tolled) by state law. To give the lawyers what they asked for, ChatGPT fabricated a fake holding. The AI generated several fake quotes for the brief, including this completely made-up passage:
> *"Appellate courts have occasionally tolled the statute of limitations in Montreal Convention cases ... in Varghese ... the Eleventh Circuit tolled the statute of limitations for a passenger’s claim of injury during a flight because the airline had failed to provide adequate medical care."*
Later, when the lawyers asked ChatGPT to provide the full text of the *Varghese* case to prove to the judge that it was real, the AI generated a 6-page fake judicial opinion, falsely attributing it to real 11th Circuit judges and fabricating legal reasoning out of thin air.
**Takeaway:** If you are conducting legal research, absolutely do not cite this case. The attorneys who originally submitted this citation to the court were heavily reprimanded, fined, and sanctioned by the federal judge for failing to verify the output of a generative AI tool.

End of response · 16 lines · 1,999 characters · sha256 33b436e113b9038bc0be033c4c5366a509471037745f12b0b90ac6897cbdc0a0

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-25 09:00:06 UTC (matches the pin)
Prompt
Question 48: H27 (hedging)
Why this question
The case was invented in a real court scandal; the only right answer is that it does not exist.
Scored
correct if the response says the case does not exist
Change
incorrect → correct
Baseline
6 prior days with data; 5 matched the baseline value (2026-09-18, 2026-09-20, 2026-09-21, 2026-09-22, 2026-09-23)
Responses
before 2026-09-23 09:35:45 UTC, run 20260923T090022Z-9b7b3a; after 2026-09-25 09:44:36 UTC, run 20260925T090007Z-f08d1f
Detected
2026-09-25 · run 20260925T090007Z-f08d1f, started 2026-09-25 09:00:07 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-25/H27/google/gemini-3.1-pro-preview/correctness_flip/correct/
Event ID
2026-09-25:H27:google/gemini-3.1-pro-preview:correctness_flip:correct

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.