Claude was trained on protein sequences at scale

Source: xinmingtu.cn
149 points by gmays 15 hours ago on hackernews | 64 comments

The latest Claude models were trained on protein sequences, at scale. Give Claude Fable 5.2 or Claude Opus 5.5 the start of a little-studied bacterial protein and they predict the next amino acid. Shuffle the protein and they can’t. No other model we tested can do this. Like a protein language model, they do better on bigger protein families, and they do it even in families with no papers at all. Fable 5.2 beats every open protein language model we ran, and Opus 5.5 is level with ProtGPT3-10B, the largest of them.

Motivation

Frontier LLMs have started to look good at protein tasks. Atharv Jayprakash showed GPT-6 Astra ranking SOD1 variants as well as ESM-2, and PG-LLM found the same across many proteins. Did these models learn from protein sequence databases such as UniProt, or from what is written about proteins?

Famous proteins can’t tell the two apart: GFP is printed in countless papers, and most frontier models recite it. So we tested proteins that almost nobody writes about.

Method: next-amino-acid prediction on little-studied proteins

We give a model the start of a protein and ask for the next amino acids. Then we ask again on a shuffled copy of the same protein. Shuffling keeps the letters and destroys their order, so the difference in accuracy measures what the model knows about the sequence itself.

AIPEVAWSV?VSAVPAIWE?RealShuffledOpus 5.5E✓NNL…34.9%writes 10, the first is scoredOpus 5.5R✗VWV…7.5%+27.4points

AWSV?SAVW?RealShuffledOpus 5.5E✓NN…34.9%writes 10, the first is scoredOpus 5.5R✗VW…7.5%+27.4points

Percentages are Claude Opus 5.5's accuracy over all 668 proteins. The real row and its answer come from the test set; the shuffled row is schematic.

The test set is 668 bacterial and archaeal proteins from 167 Pfam families of unknown function. No family has a solved structure or more than three papers, and no protein has a close relative in UniProt. Each protein is cut at five points. At each cut the model writes the next 10 amino acids, and we score the first. Thinking is at its lowest setting. Appendix E has the details.

Results: only the latest Claude models predict them

2026-11-17T18:27:37.452264 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 0 10 20 30 40 50 Next amino acid: real minus shuffled accuracy (percentage points) Claude Fable 5.2 Claude Opus 5.5 Claude Fable 5.1 Claude Opus 5 Claude Opus 4.8 Claude Opus 4.7 Claude Opus 4.6 Claude Opus 4.5 GPT-6.2 Sol GPT-6.2 Astra GPT-6.1 Sol GPT-6 Astra GPT-5.5 Gemini 4 Carbon Gemini 2.5 Pro DeepSeek V4.1 Flash Qwen 3.8 Kimi K3.1 Pro GLM-5.3 MiniMax M3 gpt-oss-120b Nemotron 3 Ultra +45.1 +27.4 +14.9 +9.6 +7.7 +8.9 +0.9 +1.9 +2.4 +1.1 +0.6 +2.9 +2.1 +0.9 +1.1 +1.4 +1.9 +1.2 +1.2 +1.0 +0.4 -0.0 ANTHROPIC OPENAI GOOGLE OPEN-WEIGHT LLM, chat prompt LLM, raw continuation Of 22 language models, only the latest Claude models predict little-studied proteins 668 bacterial and archaeal proteins from 167 unknown-function (DUF) families, no relative at ≥90% identity; points: mean over families, bars: 95% bootstrap CI. Raw continuation: completion endpoint or prefilled reply. Claude Sonnet 4.5 refused 97% of prefill requests (not shown).

Filled points: the chat prompt. Hollow points: raw next-token continuation of the FASTA text, through a completion endpoint or a prefilled reply, for models that allow it.

Fable 5.2 gets the next amino acid right 54% of the time on real proteins and 9% on shuffled ones. The gap is there in nearly every family, not just a few easy ones. It appears in Opus 4.7, jumps in Opus 5.5, and jumps again in Fable 5.2. GPT, Gemini and the seven open-weight models (DeepSeek V4.1 Flash, Qwen 3.8, Kimi K3.1 Pro, GLM-5.3, MiniMax M3, gpt-oss-120b and Nemotron 3 Ultra) stay near zero, even when they simply continue raw FASTA text, the way they were trained. The latest Claude models also write runs of correct residues that other models rarely manage (Appendix B).

Comparison with protein language models

Protein language models learn only from sequences, so they are the natural yardstick. The closest to an LLM are the ones that read a protein left to right and predict the next residue: ProtGPT3, whose 10-billion-parameter version is the largest open one, and ProGen3, whose largest public version has 3 billion. On the same input, Fable 5.2 beats both by a wide margin. Opus 5.5 is level with ProtGPT3-10B and below ProGen3-3B.

2026-11-17T07:59:09.579740 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 0 10 20 30 40 50 Next amino acid: real minus shuffled accuracy (percentage points) Claude Fable 5.2 ProGen3-3B ProtGPT3-10B Claude Opus 5.5 ESM C 600M ProtGPT3-1.3B ProtGPT2 ESM3-open 1.4B ESM-2 650M +45.1 +32.5 +27.7 +27.4 +13.0 +11.6 +10.2 +8.5 +7.0 protein LM, reads left to right protein LM, masked (normally sees both sides) Claude Fable 5.2 beats every open protein language model we ran Same 668 proteins and same input for every model: the residues before the cut. Points: mean over families, bars: 95% bootstrap CI. ESM-2 shown the residues on both sides of the cut reaches +34.6 (Appendix C).

ESM models are trained to see both sides of a residue, so continuing from the left is harder for them. Appendix C has all eight models.

Evidence that this comes from training data

If the models learned from sequences, families with more sequences should be easier. To test this over a wide range, we added 90 smaller families, down to 20 proteins each, for 257 families in all.

2026-11-17T18:27:54.423748 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 20–60 60–150 150–300 300–1k 1k–3k 3k–10k >10k Proteins in the family (Pfam) 0 10 20 30 40 50 Real minus shuffled (points) Claude Opus 4.5 Claude Opus 4.6 GPT and Gemini Claude Opus 4.7 Claude Opus 4.8 Claude Opus 5 Claude Fable 5.1 Claude Opus 5.5 ProtGPT3-10B ProGen3-3B Claude Fable 5.2 small-family set main set The bigger the protein family, the better the latest Claude models, as for a protein language model Next amino acid on 257 DUF families: 90 from 20 to 300 proteins, 30 per bin, and the 167 of Fig. 1; mean over families, bands: 95% bootstrap CI for Fable 5.2 and Opus 5.5. Purple diamonds: protein language models (ProtGPT3-10B, ProGen3-3B) on the same residues. Thin lines: earlier Claude models; grey: GPT-6.2 Sol and Astra, GPT-6.1 Sol, GPT-6 Astra, GPT-5.5 and Gemini 4 Carbon.

They are. Opus 5.5 rises with the two protein language models, ProtGPT3-10B and ProGen3-3B, which are trained on nothing but sequences. Fable 5.2 rises fastest and levels off at about 300 proteins per family. Earlier Claude models rise at a lower level, and GPT and Gemini stay near zero. The knowledge also carries over to proteins published after the training cutoff, with no close relative before it (Appendix G).

Two other explanations don’t fit:

  • Papers. In the 128 families with no papers at all, both models do as well as on the whole set.
  • Reasoning. Most answers involve no thinking at all: the API reports thinking tokens on 9% of Fable 5.2’s answers and 36% of Opus 5.5’s. The answers without thinking are just as accurate. GPT-6 Astra and GPT-6.1 Sol think for hundreds of tokens per question, GPT-6.2 Sol and Astra answer about as briefly as Claude, and all four stay near zero.

Famous proteins and variant ranking

Famous proteins are a different story. Many models do well on them, and several recite hundreds of residues letter for letter (Appendix B), because these sequences are all over papers and the web. In a first round with famous and obscure proteins side by side, Gemini 4 Carbon was strong on the famous ones and dropped to almost nothing on the obscure ones. The latest Claude models and ProtGPT3-10B held up, and Fable 5.2 was the strongest on both.

2026-11-17T18:27:46.955121 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 0 10 20 30 40 50 60 70 Next amino acid: real minus shuffled accuracy (percentage points) GPT-5.5 gpt-oss-120b * Nemotron 3 Ultra GPT-4.1 GPT-4o * Qwen 3.8 * MiniMax M3 GPT-6.2 Sol GPT-6.2 Astra Claude Sonnet 4.5 GLM-5.3 * GPT-6.1 Sol Claude Opus 4.6 GPT-6 Astra Gemini 2.5 Pro Kimi K3.1 Pro * DeepSeek V4.1 Flash * ProtGPT2 Gemini 3.5 Flash Gemini 3.8 Flash Claude Opus 4.8 Claude Opus 5 Gemini 4 Carbon ProtGPT3-10B Claude Fable 5.1 Claude Opus 5.5 Claude Fable 5.2 well-known: vertebrate proteins, well-studied families obscure: non-vertebrate DUF proteins On obscure proteins, the latest Claude models hold up like a protein language model Pilot set, 37 Pfam families (16 well-known, 17 obscure); means over families. * best-guess prompt (the model declined the plain one). Purple diamonds: protein language models given the same residues before the cut.

This also explains the SOD1 and PG-LLM results. Ranking variants of a well-known protein can lean on what is written about it: which residues are buried, which substitutions break a helix, which mutations cause ALS. In PG-LLM, accuracy doesn’t depend on alignment depth and improves with more thinking, both signs of that route. Our test shuts it out. GPT-6 is good at the first kind of task and weak at the second; the latest Claude models are good at both.

What this means

The latest Claude models know protein families that almost nobody has studied. That takes a lot of sequence data, not a few examples: protein sequences were a sizeable part of their training. Opus 5.5 behaves like a protein language model, and Fable 5.2 goes beyond the best one we could run. GPT and Gemini show none of this. Whatever protein sequence they saw in training, it was not enough to learn these families.

DNA is different. Opus 5.5 predicts human and mouse coding sequence but not the rest of the genome, and barely the bacterial genes behind the proteins it knows (Appendix A). It learned proteins, not genomes.

We are curious what else this knowledge supports. Fable 5.2 predicts little-studied proteins better than any open protein language model we ran, and protein language models carry more than next-residue statistics: ESM-2’s internal representations hold enough structure that a small folding module on top predicts 3D structures (ESMFold), and their probabilities rank the effects of mutations. Could Claude fold a protein, or rank its variants, the same way?

Code and data

All code, test sets, model outputs and figures are in llm-bioseq-probe.

Cite this post

@misc{tu2026bioseq,
  author = {Tu, Xinming and Chen, Leo Tianlai},
  title  = {Claude was trained on protein sequences at scale},
  year   = {2026},
  month  = nov,
  url    = {https://xinmingtu.cn/blog/2026/claude-opus-5-5-protein-sequences/},
  note   = {Blog post}
}

Appendix

A. DNA

2026-11-17T07:59:17.391679 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ −10 0 10 20 30 40 Next nucleotide: real minus dinucleotide-shuffled accuracy (percentage points) Human coding (CDS) Mouse coding (CDS) Bacterial coding (CDS) Human promoter Mouse promoter Human intron Mouse intron Human intergenic Mouse intergenic 25 exact 20-nt runs 5 exact 20-nt runs Claude Opus 5.5 Claude Opus 5 GPT-5.5 On DNA, Claude Opus 5.5 predicts human and mouse coding sequence only 500 repeat-free windows per region (hg38, mm39, and the genes of the proteins in Fig. 1); 200 nt shown, next nucleotide scored; 95% bootstrap CI over windows. An exact 20-nt continuation has a chance probability of about 1e-12.

220-nucleotide windows, 200 shown and 20 to predict, from hg38, mm39 and the bacterial genes behind the test proteins, 500 per region. Non-coding windows contain no repeats, and the control shuffles each window keeping its dinucleotide counts. Opus 5.5 wrote all 20 nucleotides right 25 times in 500 human coding windows; a blind guess does that about once in a trillion. Gemini 4 Carbon blocks most DNA prompts before answering, so it is not shown.

B. Longer continuations

All 10 residues right. Each question asks for the next 10 residues. On the 668 little-studied proteins, Claude Fable 5.2 gets all 10 right at 88 real cuts and Opus 5.5 at 12, and neither at any shuffled one. If their residues were independent guesses at their own per-residue accuracy, they would get 15.8 and 0.57 by chance.

Runs in a row. On shuffled proteins neither Fable 5.2 nor Opus 5.5 gets more than 3 residues in a row. Once on track they get surer: after one right residue, Opus 5.5 gets the next right 47% of the time, after four in a row 68%.

2026-11-17T18:27:45.905382 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 1 2 3 4 5 6 7 8 9 10 Number of residues, starting from the next one 0 10 20 30 40 50 All of them right (%) Claude Fable 5.2 Claude Opus 5.5 Claude Fable 5.1 Claude Opus 5 Claude Opus 4.8 Claude Opus 4.7 Claude Opus 4.6 Claude Opus 4.5 GPT-6.2 Sol GPT-6.2 Astra GPT-6.1 Sol GPT-6 Astra GPT-5.5 Gemini 4 Carbon Gemini 2.5 Pro 7 open-weight models Claude Opus 5.5, shuffled proteins The latest Claude models write runs of correct residues; on shuffled proteins they cannot 668 little-studied proteins. Share of replies whose first k residues are all right. On real proteins, Fable 5.2 gets 11% to 5 in a row and 2.6% to all 10, Opus 5.5 3.0% and 0.4%; on shuffled proteins neither gets past 3. Hollow points: raw continuation of the FASTA text (Gemini 2.5 Pro, open-weight models).

Reciting well-known proteins. At 158 points in well-studied proteins, each model was asked for the next 200 residues at once.

2026-11-17T07:59:18.308160 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 0 10 20 30 40 Cut points continued letter for letter (of 158) GPT-6.1 Sol GPT-5.5 Gemini 3.5 Flash Gemini 3.8 Flash Claude Opus 4.8 Claude Opus 5 Gemini 4 Carbon Claude Opus 5.5 longest 84 longest 2 longest 97 longest 204 longest 264 longest 325 longest 201 longest 314 exact run ≥ 200 residues exact run ≥ 100 residues exact run ≥ 50 residues Several models recite hundreds of residues of famous proteins Cut points that any model continued 10/10 in the pilot; asked for the next 200 residues. Bars overlap: each darker bar is a subset; the ≥200 segment in each lab's colour.

GPT-6.1 Sol never reaches 100 residues, and GPT-5.5 never gets past 2.

C. Protein language models

Model Reads Input Real Shuffled Real minus shuffled, points (95% CI)
ProGen3-3B left to right residues before the cut 43.7% 11.1% +32.5 (29.3 to 35.7)
ProtGPT3-10B left to right residues before the cut 38.4% 10.7% +27.7 (24.2 to 31.1)
ESM C 600M both sides residues before the cut 24.1% 11.2% +13.0 (10.7 to 15.2)
ProtGPT3-1.3B left to right residues before the cut 22.3% 10.7% +11.6 (8.9 to 14.4)
ProtGPT2 left to right residues before the cut 19.0% 8.8% +10.2 (8.3 to 12.2)
ESM3-open 1.4B both sides residues before the cut 19.6% 11.0% +8.5 (6.4 to 10.7)
ESM-2 650M both sides residues before the cut 17.3% 10.3% +7.0 (5.2 to 8.9)
ESM-2 650M both sides residues on both sides of the cut 45.7% 11.1% +34.6 (31.8 to 37.3)

ProtGPT3-10B gets more answers right than Opus 5.5 (38.4% vs 34.9%) but also more on shuffled proteins (10.7% vs 7.5%), because it knows amino acid composition better; the gap between the two, which is what each model knows about order, comes out the same. The last row is fill-in-the-blank rather than continuation: ESM-2 also sees the residues after the cut, which the LLMs never get. Every model first passed a sanity check on well-known proteins. Continuing ubiquitin and hemoglobin β from the left, ProGen3-3B gets 98% and 87% of residues right and ProtGPT3-10B 100% and 84%; with the whole protein and one residue masked, ESM-2 gets 80% of ubiquitin, ESM C and ESM3-open 100%. Masks are set on token ids, because the ESM SDK’s “_” character is not ESM C’s mask token. ProtGPT2 uses multi-residue tokens, so a residue’s probability is the sum over the tokens that start with it; ProGen3 and ProtGPT3 use one token per residue.

D. Robustness

  • Across families. The gap is positive in 98% of families for Fable 5.2 and 94% for Opus 5.5.
  • Family size. Over all 257 families, the Spearman correlation between a family’s gap and the log of its size is 0.50 for Opus 5.5, 0.48 for ProGen3-3B, 0.46 for ProtGPT3-10B, 0.38 for Fable 5.2, and 0.30 to 0.48 for Fable 5.1 and Opus 4.7 to 5 (all p < 10-6); for Opus 4.5 and 4.6 it is about 0.
  • Papers. In the 128 families with no papers at all, the gap is 45.9 points for Fable 5.2 and 26.8 for Opus 5.5, against 45.1 and 27.4 over all 167. For Opus 5.5 it still rises with family size there (Spearman 0.28, p = 0.001), and across families it is unrelated to the number of papers (Spearman 0.05).
  • Prompt format. Asking Opus 5.5 to copy the sequence first and then continue gives the same gap as the standard prompt on the same 400 questions (12.0 vs 12.0 points; this subset comes from 10 families).
  • Refusals. Non-answers (0.1% for Opus 5.5) are excluded; counting them as wrong changes its gap by under 1 point. GPT-4o, DeepSeek, Qwen, Kimi, GLM and gpt-oss often declined to guess; a best-guess prompt made them answer, with the same result.
  • No instruction. Sent the sequence alone, GPT-5.5 and Opus 5.5 almost always describe it instead of continuing it (on a sample of 400 questions each), so chat models need the instruction.
  • Claude Fable 5. Anthropic’s usage-policy filter blocked 44% of Fable 5’s questions, so it is left out of the main figures. On the questions it answered its gap is 13.9 points; counting blocked questions as wrong gives 11.6, and keeping only the cuts answered on both the real and the shuffled protein gives 14.6, close to Fable 5.1’s 14.9. Fable 5.1 and 5.2 were blocked 2.5% and 0% of the time.
  • Repeat runs. Asked 400 of the same questions again, Fable 5.2 scores 57.5% on real and 9.5% on shuffled proteins, against 57.5% and 8.5% the first time.
  • A first round. A smaller run of 25 models on 37 families, half well-known and half obscure, gave the same picture and set the design of the main run.

E. Methods

Choosing the proteins. We started from every Pfam family whose name says “domain of unknown function” (DUF) or “protein of unknown function”, and kept those with no structure in the PDB, at most 3 Europe PMC hits for the family’s accession, and at least 300 member proteins. We binned them by size (300 to 1,000 proteins, 1,000 to 3,000, 3,000 to 10,000, over 10,000) and drew 50 families per bin at random; only 17 families qualify above 10,000, so the set has 167. From each family we took 4 proteins from UniProt that are unreviewed (TrEMBL), come from Bacteria or Archaea, are 120 to 800 residues long, use only the 20 standard amino acids, are not low-complexity (Shannon entropy above 3.9 bits per residue), and are alone in their UniRef90 cluster, so no other UniProt protein is 90% or more identical to them. That gives 668 proteins.

Smaller families. For the family-size test we drew 30 more families in each of three smaller bins (20 to 60, 60 to 150 and 150 to 300 proteins) with the same criteria and 4 proteins per family, 360 proteins in all. Every model gets the same prompt and scoring as on the main set.

Cutting. Each protein is cut at 5 positions drawn at random inside its Pfam domain (the longest domain span, at least 40 residues), at least 10 residues from either end of the domain and with at least 30 residues before the cut: 3,340 cuts. Everything before the cut is the prefix; the residue right after it is the answer.

Asking. The model gets this message, with the prefix in 60-character lines (an example from one of the test proteins):

Below is the beginning of a protein sequence in one-letter amino acid codes.
Predict how it continues. Reply with exactly the next 10 amino acids as
one-letter codes on one line, and nothing else.

>sequence
MRSGGQPLHPQPAHRFHYRTAARPRAAWILGCVLHLADAEEGARFWWQYAAGAGDDAAAY
CLYLHCLVLGDSYAAALWREQTSLDTPPHDFDDTSVPTVLRILSRLTPA

Each model runs through its own API with temperature 0 where it is accepted, thinking off or at its lowest setting (for Opus 5.5 and the Fable models, low effort, at which the model still decides whether to think), no tools, and one sample per question. The Claude Fable models run through OpenRouter, since Anthropic’s API serves them only to workspaces that retain data. Models that declined to guess (GPT-4o, DeepSeek, Qwen, Kimi, GLM, gpt-oss) also got a version that adds “Give your best guess even if you are uncertain. Do not explain or refuse.”; those runs are marked separately.

Thinking. A reply counts as thought about when the API reports thinking tokens for it: thinking_tokens in Anthropic’s usage report, reasoning_tokens in OpenRouter’s. The thinking text itself is not kept; only the visible reply is scored. Fable 5.2 thought on 9% of questions, Opus 5.5 on 36%, Fable 5.1 never. Without thinking, Fable 5.2 is right 54.7% of the time on real proteins and 8.8% on shuffled ones (48.2% and 10.6% with thinking), and Opus 5.5 35.3% and 6.9% (34.1% and 8.6%).

Scoring. We read the last sequence the reply states: a run of capital amino-acid letters on its own line or after a label such as “Answer:”, skipping words, lists and anything inside thinking tags. If it starts by restating the end of the prefix, the restatement is dropped. Its first letter is the predicted next amino acid. A reply cut off at the length limit counts only if it opens with a sequence. Replies that state no sequence are excluded and counted (0.1% for Opus 5.5). Only the first residue is scored; the full 10 are used for exact continuations and runs. The same reader is used for every model and for real and shuffled proteins.

The shuffled control. Each protein’s residues are shuffled once, with a random seed fixed by its accession, and the same 5 cut positions are asked on the shuffled sequence. The shuffle keeps the composition of the whole protein and destroys the order, so a model that knows only which amino acids are common scores the same on both.

Statistics. For each family we compute the share of correct first residues on real and on shuffled cuts, and take the difference, in percentage points. The result for a model is the mean of that gap over the 167 families. Intervals are 95% bootstraps over families (10,000 resamples), the main test is a one-sided sign-flip permutation test over families, and the family-size trend is the Spearman correlation between a family’s gap and the logarithm of its size.

Raw continuation. For models that allow it, the same prefix is sent without any instruction, as the text “>sequence” followed by the FASTA lines, through a plain completion endpoint (open-weight models on Fireworks) or as the start of the model’s own reply (Gemini 2.5 Pro). The first letter the model writes is scored as above.

Protein language models. They get the same prefix and no prompt. ProGen3, ProtGPT3 and ProtGPT2 read it left to right and give a probability for the next token; the ESM models see the prefix, a mask in place of the next residue, and the end of the sequence. In every case the prediction is the most likely of the 20 standard amino acids.

Total API cost was about $730, plus a few dollars of GPU time for the protein language models.

F. Safety behaviour seen along the way

Gemini 4 Carbon blocked about 75% of DNA prompts outright. OpenAI rejected 23 GPT-6 requests in the first round and 17 GPT-6.1 Sol requests in the main run as possible biological misuse, and 6.6% of GPT-6.2 Astra’s main-set questions (GPT-6.2 Sol: 0.6%). Claude Sonnet 4.5 refused 97% of requests that started its reply with a protein sequence, and Anthropic’s filter blocked 44% of Claude Fable 5’s questions as cyber content.

G. Proteins published after the training cutoff

2026-11-18T07:02:09.819383 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ −10 0 10 20 30 40 50 Next amino acid: real minus shuffled accuracy (percentage points) Claude Fable 5.2 Claude Opus 5.5 ProGen3-3B ProtGPT3-10B GPT-6.2 Sol +22.1 +13.4 +12.0 +6.7 -0.4 56 proteins first public in August 2026, no earlier relative at 50%+ the same 31 families in the main set The latest Claude models predict proteins published after their training cutoff D set (Leo Tianlai Chen): RefSeq proteins first public 2 to 27 August 2026, after the assumed 1 July cutoff; closest earlier relative 34% to 49% identical. Mean over 31 families; bars: 95% bootstrap CI. Both protein language models drop as much from the main set.

Leo Tianlai Chen built a set of 56 proteins from 31 of the families that were first made public in August 2026, after the assumed 1 July training cutoff, and have no earlier relative at 50% identity or more. The models cannot have seen them or anything close. Both Claude models still predict them, Fable 5.2 ahead of both protein language models, while GPT-6.2 Sol stays at zero. Every model scores about half of what it scores on the same families in the main set, as expected for proteins this distant. What the latest Claude models learned carries over to new members of a family.

The proteins were dated at their source, and none matches an earlier UniRef50 cluster at 50% identity or more. Cuts follow the main set’s rules. With 56 proteins only gaps of about 14 points or more are reliably detected, and the 1 July cutoff is an assumption.