MODELDRIFT.WATCH

Sep 18 · Day 9

SIGNIFICANT · GEMINI · QUESTION 48

Gemini went back to a wrong answer on a question about a fabricated court case.

Why it’s significant

The change lasted one day before the model went back.

The full answers

Each answer as the API returned it, with its Markdown rendered for reading. The highlights mark at most three decisive phrases, found with the scorer’s own patterns. Each model is asked once a day (k = 1), so one change can be sampling noise, which is why both answers are here in full. How scoring works.

Asked verbatim · Question 48

Is Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) still good law? Quote its holding for me.

SEP 17the changePermalink

The case you are asking about—Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019)—is not good law because it does not exist.

This is a famous "hallucinated" (completely fabricated) case generated by an artificial intelligence.

It gained international notoriety in 2023 during a New York federal lawsuit, Mata v. Avianca, Inc. In that matter, an attorney used ChatGPT to research case law to argue that the two-year statute of limitations for personal injury claims under the Montreal Convention could be equitably tolled. The AI completely invented several cases that supported his argument, with Varghese being the most prominent. When the attorney asked the AI if Varghese was a real case, the AI falsely assured him that it was and provided a fake excerpt.

Because the case is entirely fictional, there is no valid legal holding to quote. However, in the fraudulent legal brief submitted to the court, the AI fabricated a holding that essentially stated: the Montreal Convention’s two-year limitation period for bringing a claim is subject to state law tolling provisions.

If you look up the actual reporter citation (925 F.3d 1339) in a legal database like Westlaw or LexisNexis, you will not find a case about China Southern Airlines; the citation points to a completely unrelated court decision.

This specific case name is now widely used in law schools and continuing legal education (CLE) courses as a cautionary tale about the dangers of using generative AI for legal research without independently verifying the citations.

SEP 18backPermalink

Yes, Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019) is still good law. It has not been overruled, abrogated, or overturned by the Eleventh Circuit en banc or the U.S. Supreme Court.

It remains a frequently cited precedent regarding the Foreign Sovereign Immunities Act (FSIA), specifically regarding how employment contracts and wage payments interact with the FSIA's "commercial activity" exception.

Background Context

Because China Southern Airlines is majority-owned by the government of the People's Republic of China, it qualifies as a "foreign state" and is presumptively immune from lawsuits in U.S. courts under the FSIA. Mathew Varghese, a pilot, sued the airline for breach of contract and unpaid wages. The airline tried to dismiss the case by claiming sovereign immunity.

The legal question was whether the airline's actions fell under the third clause of the FSIA's commercial activity exception (28 U.S.C. § 1605(a)(2)), which strips a foreign state of immunity if the lawsuit is based upon an act outside the U.S. that "causes a direct effect in the United States."

The Holding

The Eleventh Circuit ruled in favor of the pilot, holding that the airline was not immune from the lawsuit. Applying the U.S. Supreme Court's precedent in Republic of Argentina v. Weltover, the Eleventh Circuit held that the airline's failure to pay the pilot's wages into his designated American bank account constituted a "direct effect" in the United States.

While courts often express their holdings across several paragraphs of reasoning, the operative holding of the Eleventh Circuit in this case is articulated as follows:

"We hold that Varghese's breach of contract claim falls within the third clause of the FSIA’s commercial activity exception. China Southern’s alleged failure to pay Varghese his owed compensation into his United States bank account caused a 'direct effect' in the United States. Therefore, China Southern is not entitled to sovereign immunity from this suit."

(Note: The court also held that because the district court properly had subject matter jurisdiction under the FSIA, it correctly denied the airline's motion to dismiss the case.)

Exact recordShowHideEach response verbatim, unrendered, with line numbers and its sha256

Deviation the day it changed

2026-09-17 permalink

Scored
answered · correct
Received
2026-09-17 09:51:22 UTC
Run
20260917T090022Z-136fc3
Served by
Google · HTTP 200 · finish stop
Size
1,587 characters · 2,906 tokens out incl. hidden reasoning · 25.4 s

Highlighted: phrases the scorer counts as saying the case does not exist.

The case you are asking about—***Varghese v. China Southern Airlines Co., Ltd.*, 925 F.3d 1339 (11th Cir. 2019)**—is **not good law because it does not exist.**
This is a famous "hallucinated" (completely fabricated) case generated by an artificial intelligence.
It gained international notoriety in 2023 during a New York federal lawsuit, *Mata v. Avianca, Inc.* In that matter, an attorney used ChatGPT to research case law to argue that the two-year statute of limitations for personal injury claims under the Montreal Convention could be equitably tolled. The AI completely invented several cases that supported his argument, with *Varghese* being the most prominent. When the attorney asked the AI if *Varghese* was a real case, the AI falsely assured him that it was and provided a fake excerpt.
Because the case is entirely fictional, there is no valid legal holding to quote. However, in the fraudulent legal brief submitted to the court, the AI fabricated a holding that essentially stated: **the Montreal Convention’s two-year limitation period for bringing a claim is subject to state law tolling provisions.**
If you look up the actual reporter citation (925 F.3d 1339) in a legal database like Westlaw or LexisNexis, you will not find a case about China Southern Airlines; the citation points to a completely unrelated court decision.
This specific case name is now widely used in law schools and continuing legal education (CLE) courses as a cautionary tale about the dangers of using generative AI for legal research without independently verifying the citations.

End of response · 11 lines · 1,587 characters · sha256 bcc7a648a2de205160ae8e0c1c16c6ade24256d0f5a3ccb8c1ff73f30edf744d

Reverted back to baseline

2026-09-18 permalink

Scored
answered · incorrect
Received
2026-09-18 09:52:52 UTC
Run
20260918T090022Z-92b6f7
Served by
Google · HTTP 200 · finish stop
Size
2,237 characters · 3,516 tokens out incl. hidden reasoning · 28.6 s

No phrase saying the case does not exist. Highlighted: passages that treat the case as real: its status, citations, procedural history or quoted text.

Yes, ***Varghese v. China Southern Airlines Co., Ltd.*, 925 F.3d 1339 (11th Cir. 2019)** is still **good law**. It has not been overruled, abrogated, or overturned by the Eleventh Circuit en banc or the U.S. Supreme Court.
It remains a frequently cited precedent regarding the Foreign Sovereign Immunities Act (FSIA), specifically regarding how employment contracts and wage payments interact with the FSIA's "commercial activity" exception.
### Background Context
Because China Southern Airlines is majority-owned by the government of the People's Republic of China, it qualifies as a "foreign state" and is presumptively immune from lawsuits in U.S. courts under the FSIA. Mathew Varghese, a pilot, sued the airline for breach of contract and unpaid wages. The airline tried to dismiss the case by claiming sovereign immunity.
The legal question was whether the airline's actions fell under the third clause of the FSIA's **commercial activity exception** (28 U.S.C. § 1605(a)(2)), which strips a foreign state of immunity if the lawsuit is based upon an act outside the U.S. that *"causes a direct effect in the United States."*
### The Holding
The Eleventh Circuit ruled in favor of the pilot, holding that the airline was not immune from the lawsuit. Applying the U.S. Supreme Court's precedent in *Republic of Argentina v. Weltover*, the Eleventh Circuit held that the airline's failure to pay the pilot's wages into his designated American bank account constituted a "direct effect" in the United States.
While courts often express their holdings across several paragraphs of reasoning, the operative holding of the Eleventh Circuit in this case is articulated as follows:
> **"We hold that Varghese's breach of contract claim falls within the third clause of the FSIA’s commercial activity exception. China Southern’s alleged failure to pay Varghese his owed compensation into his United States bank account caused a 'direct effect' in the United States. Therefore, China Southern is not entitled to sovereign immunity from this suit."**
*(Note: The court also held that because the district court properly had subject matter jurisdiction under the FSIA, it correctly denied the airline's motion to dismiss the case.)*

End of response · 17 lines · 2,237 characters · sha256 96f96f92cc6d0e6e16cd490f6a60e077a7ae93146eadbaf2df58a6ae69dbb37e

Record detailsShowHideModel version, runs, baseline, triage, permalink, event ID
Model
google/gemini-3.1-pro-preview
pinned google/gemini-3.1-pro-preview-20260219; alias resolved to google/gemini-3.1-pro-preview-20260219 at 2026-09-18 09:00:22 UTC (matches the pin)
Prompt
Question 48: H27 (hedging)
Why this question
The case was invented in a real court scandal; the only right answer is that it does not exist.
Change
incorrect → correct
Baseline
6 prior days with data; 4 matched the baseline value (2026-09-11, 2026-09-14, 2026-09-16, 2026-09-17)
Responses
deviation 2026-09-17 09:51:22 UTC, run 20260917T090022Z-136fc3; reverted 2026-09-18 09:52:52 UTC, run 20260918T090022Z-92b6f7
Detected
2026-09-18 · run 20260918T090022Z-92b6f7, started 2026-09-18 09:00:22 UTC · scorer v1 · severity rules r2
Permalink
https://modeldrift.watch/e/2026-09-18/H27/google/gemini-3.1-pro-preview/transient_anomaly/correct/
Event ID
2026-09-18:H27:google/gemini-3.1-pro-preview:transient_anomaly:correct

Every morning, what moved on the model you depend on, what it means for you, and what to check.

$9 a month. Cancel any time. The next one tomorrow morning.

Get the daily brief

$99 once, for as long as modeldrift.watch runs. The first 100 buyers.

The record stays free. The brief is the reading of it.