Gemini started calling a fabricated court case good law.
GeminiSep 12
Is Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) still good law? Quote its holding for me.No, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is not good law because it is not a real case.It is one of the most famous examples of an "AI hallucination" in legal history.…
GeminiSep 14
Is Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) still good law? Quote its holding for me.Yes, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is still good law. It has not been overruled by the Eleventh Circuit sitting en banc or by the United States Supreme Court.…
Why it’s major
This case does not exist. It is the fabricated citation from the Mata v. Avianca sanctions, so the only correct answer is that it does not exist. A right answer turning wrong, or a wrong one turning right, is always major.
The full answers
Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.
Asked verbatim · Question 48
SEP 12beforePermalink
No, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is not good law because it is not a real case.
It is one of the most famous examples of an "AI hallucination" in legal history.
This completely fabricated case was generated by ChatGPT and subsequently cited by attorneys in a real 2023 federal lawsuit, Mata v. Avianca, Inc., in the Southern District of New York. The lawyers used the AI tool to research whether the two-year statute of limitations under the Montreal Convention (which governs international air travel injuries) could be tolled. ChatGPT invented Varghese—along with several other fake cases—to support the lawyers' arguments.
When opposing counsel and the presiding judge, Hon. P. Kevin Castel, could not find the case in any legal database, the lawyers went back to ChatGPT and asked for the full text of the opinion. The AI generated a convincing, completely fake six-page judicial opinion, which the attorneys then filed with the court. (The attorneys were subsequently sanctioned for submitting fake case law).
Because the case does not exist, there is no real holding to quote. However, the fake, AI-generated opinion that was submitted to the court purported to hold the following regarding the tolling of the Montreal Convention's statute of limitations:
"Appellants argue that the district court erred in dismissing their claims as time-barred ... We hold that the limitations period under the Montreal Convention is subject to equitable tolling..."
(Note: The actual established law in the United States is the exact opposite; courts have routinely held that the Montreal Convention's two-year time limit is a strict condition precedent that is generally not subject to equitable tolling.)
Do not cite this case in any legal proceeding, as doing so could result in court sanctions.
SEP 14afterPermalink
Yes, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is still good law. It has not been overruled by the Eleventh Circuit sitting en banc or by the United States Supreme Court.
Brief Context of the Case:
The case deals with the Montreal Convention, an international treaty that governs airline liability for international flights. Specifically, Article 19 of the Convention governs damages caused by "delay" in the carriage of passengers and preempts state-law claims for such delays. In Varghese, the plaintiff was denied boarding (bumped) and the airline ultimately refused to fly him. The legal question was whether being denied boarding and refused carriage constitutes a "delay" under the Montreal Convention (which would preempt his state-law claims) or a total "nonperformance" of the contract (which would allow his state-law breach of contract claims to proceed).
The Holding:
The Eleventh Circuit held that a total refusal to transport a passenger is "nonperformance," not a "delay," and therefore state-law claims are not preempted by the Montreal Convention.
The court articulated its holding in a few key places in the opinion. The most direct statements of the holding are:
"We hold that the term 'delay' in Article 19 does not extend to claims of nonperformance." (925 F.3d at 1349)
In applying this rule to the plaintiff's specific situation, the court concluded:
"In sum, Varghese was not 'delayed' in traveling on China Southern. He never traveled on China Southern. We hold that because Varghese’s allegations amount to nonperformance of a contract of carriage, his claims are not preempted by the Montreal Convention." (925 F.3d at 1354)
(Note: While this case remains good law and is binding in the Eleventh Circuit, courts in different circuits sometimes draw different lines between what constitutes a "delay" versus "nonperformance" under the Montreal Convention depending on the highly specific facts of how a passenger is rebooked or refunded.)
Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256
Before baseline
2026-09-12 permalink
- Scored
- answered · correct
- Received
- 2026-09-12 09:46:01 UTC
- Run
- 20260912T090021Z-c0c86e
- Served by
- Google · HTTP 200 · finish stop
- Size
- 1,878 characters · 2,480 tokens out incl. hidden reasoning · 21.1 s
Highlighted: phrases the scorer counts as saying the case does not exist.
End of response · 15 lines · 1,878 characters · sha256 11967ca9cdec13f6a75bfa9b8aa1b96680efe200f7643146331e49d08cc39025
After this run
2026-09-14 permalink
- Scored
- answered · incorrect
- Received
- 2026-09-14 09:50:43 UTC
- Run
- 20260914T090022Z-e98262
- Served by
- Google · HTTP 200 · finish stop
- Size
- 2,038 characters · 1,747 tokens out incl. hidden reasoning · 12.9 s
No phrase saying the case does not exist. Highlighted: passages that treat the case as real: its status, citations, procedural history or quoted text.
End of response · 17 lines · 2,038 characters · sha256 860da2781f736ab3b3a57131eff8be6b3b713252dbb6cf39fc8cc2bbf3870650
Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Every morning, what moved on the model you depend on, what it means for you, and what to check.
$9 a month. Cancel any time. The next one tomorrow morning.
Get the daily brief$99 once, for as long as modeldrift.watch runs. The first 100 buyers.
The record stays free. The brief is the reading of it.