The latest Claude models were trained on protein sequences, at scale. Give Claude Fable 5.2 or Claude Opus 5.5 the start of a little-studied bacterial protein and they predict the next amino acid. Shuffle the protein and they can’t. No other model we tested can do this. Like a protein language model, they do better on bigger protein families, and they do it even in families with no papers at all. Fable 5.2 beats every open protein language model we ran, and Opus 5.5 is level with ProtGPT3-10B, the largest of them.
Frontier LLMs have started to look good at protein tasks. Atharv Jayprakash
Famous proteins can’t tell the two apart: GFP is printed in countless papers, and most frontier models recite it. So we tested proteins that almost nobody writes about.
We give a model the start of a protein and ask for the next amino acids. Then we ask again on a shuffled copy of the same protein. Shuffling keeps the letters and destroys their order, so the difference in accuracy measures what the model knows about the sequence itself.
The test set is 668 bacterial and archaeal proteins from 167 Pfam families of unknown function. No family has a solved structure or more than three papers, and no protein has a close relative in UniProt. Each protein is cut at five points. At each cut the model writes the next 10 amino acids, and we score the first. Thinking is at its lowest setting. Appendix E has the details.
Fable 5.2 gets the next amino acid right 54% of the time on real proteins and 9% on shuffled ones. The gap is there in nearly every family, not just a few easy ones. It appears in Opus 4.7, jumps in Opus 5.5, and jumps again in Fable 5.2. GPT, Gemini and the seven open-weight models (DeepSeek V4.1 Flash, Qwen 3.8, Kimi K3.1 Pro, GLM-5.3, MiniMax M3, gpt-oss-120b and Nemotron 3 Ultra) stay near zero, even when they simply continue raw FASTA text, the way they were trained. The latest Claude models also write runs of correct residues that other models rarely manage (Appendix B).
Protein language models learn only from sequences, so they are the natural yardstick. The closest to an LLM are the ones that read a protein left to right and predict the next residue: ProtGPT3
ESM models are trained to see both sides of a residue, so continuing from the left is harder for them. Appendix C has all eight models.
If the models learned from sequences, families with more sequences should be easier. To test this over a wide range, we added 90 smaller families, down to 20 proteins each, for 257 families in all.
They are. Opus 5.5 rises with the two protein language models, ProtGPT3-10B and ProGen3-3B, which are trained on nothing but sequences. Fable 5.2 rises fastest and levels off at about 300 proteins per family. Earlier Claude models rise at a lower level, and GPT and Gemini stay near zero. The knowledge also carries over to proteins published after the training cutoff, with no close relative before it (Appendix G).
Two other explanations don’t fit:
Famous proteins are a different story. Many models do well on them, and several recite hundreds of residues letter for letter (Appendix B), because these sequences are all over papers and the web. In a first round with famous and obscure proteins side by side, Gemini 4 Carbon was strong on the famous ones and dropped to almost nothing on the obscure ones. The latest Claude models and ProtGPT3-10B held up, and Fable 5.2 was the strongest on both.
This also explains the SOD1 and PG-LLM results. Ranking variants of a well-known protein can lean on what is written about it: which residues are buried, which substitutions break a helix, which mutations cause ALS. In PG-LLM, accuracy doesn’t depend on alignment depth and improves with more thinking, both signs of that route. Our test shuts it out. GPT-6 is good at the first kind of task and weak at the second; the latest Claude models are good at both.
The latest Claude models know protein families that almost nobody has studied. That takes a lot of sequence data, not a few examples: protein sequences were a sizeable part of their training. Opus 5.5 behaves like a protein language model, and Fable 5.2 goes beyond the best one we could run. GPT and Gemini show none of this. Whatever protein sequence they saw in training, it was not enough to learn these families.
DNA is different. Opus 5.5 predicts human and mouse coding sequence but not the rest of the genome, and barely the bacterial genes behind the proteins it knows (Appendix A). It learned proteins, not genomes.
We are curious what else this knowledge supports. Fable 5.2 predicts little-studied proteins better than any open protein language model we ran, and protein language models carry more than next-residue statistics: ESM-2’s internal representations hold enough structure that a small folding module on top predicts 3D structures (ESMFold
All code, test sets, model outputs and figures are in llm-bioseq-probe.
Cite this post
@misc{tu2026bioseq,
author = {Tu, Xinming and Chen, Leo Tianlai},
title = {Claude was trained on protein sequences at scale},
year = {2026},
month = nov,
url = {https://xinmingtu.cn/blog/2026/claude-opus-5-5-protein-sequences/},
note = {Blog post}
}
220-nucleotide windows, 200 shown and 20 to predict, from hg38, mm39 and the bacterial genes behind the test proteins, 500 per region. Non-coding windows contain no repeats, and the control shuffles each window keeping its dinucleotide counts. Opus 5.5 wrote all 20 nucleotides right 25 times in 500 human coding windows; a blind guess does that about once in a trillion. Gemini 4 Carbon blocks most DNA prompts before answering, so it is not shown.
All 10 residues right. Each question asks for the next 10 residues. On the 668 little-studied proteins, Claude Fable 5.2 gets all 10 right at 88 real cuts and Opus 5.5 at 12, and neither at any shuffled one. If their residues were independent guesses at their own per-residue accuracy, they would get 15.8 and 0.57 by chance.
Runs in a row. On shuffled proteins neither Fable 5.2 nor Opus 5.5 gets more than 3 residues in a row. Once on track they get surer: after one right residue, Opus 5.5 gets the next right 47% of the time, after four in a row 68%.
Reciting well-known proteins. At 158 points in well-studied proteins, each model was asked for the next 200 residues at once.
GPT-6.1 Sol never reaches 100 residues, and GPT-5.5 never gets past 2.
| Model | Reads | Input | Real | Shuffled | Real minus shuffled, points (95% CI) |
|---|---|---|---|---|---|
| ProGen3-3B | left to right | residues before the cut | 43.7% | 11.1% | +32.5 (29.3 to 35.7) |
| ProtGPT3-10B | left to right | residues before the cut | 38.4% | 10.7% | +27.7 (24.2 to 31.1) |
| ESM C 600M | both sides | residues before the cut | 24.1% | 11.2% | +13.0 (10.7 to 15.2) |
| ProtGPT3-1.3B | left to right | residues before the cut | 22.3% | 10.7% | +11.6 (8.9 to 14.4) |
| ProtGPT2 | left to right | residues before the cut | 19.0% | 8.8% | +10.2 (8.3 to 12.2) |
| ESM3-open 1.4B | both sides | residues before the cut | 19.6% | 11.0% | +8.5 (6.4 to 10.7) |
| ESM-2 650M | both sides | residues before the cut | 17.3% | 10.3% | +7.0 (5.2 to 8.9) |
| ESM-2 650M | both sides | residues on both sides of the cut | 45.7% | 11.1% | +34.6 (31.8 to 37.3) |
ProtGPT3-10B gets more answers right than Opus 5.5 (38.4% vs 34.9%) but also more on shuffled proteins (10.7% vs 7.5%), because it knows amino acid composition better; the gap between the two, which is what each model knows about order, comes out the same. The last row is fill-in-the-blank rather than continuation: ESM-2 also sees the residues after the cut, which the LLMs never get. Every model first passed a sanity check on well-known proteins. Continuing ubiquitin and hemoglobin β from the left, ProGen3-3B gets 98% and 87% of residues right and ProtGPT3-10B 100% and 84%; with the whole protein and one residue masked, ESM-2 gets 80% of ubiquitin, ESM C and ESM3-open 100%. Masks are set on token ids, because the ESM SDK’s “_” character is not ESM C’s mask token. ProtGPT2 uses multi-residue tokens, so a residue’s probability is the sum over the tokens that start with it; ProGen3 and ProtGPT3 use one token per residue.
Choosing the proteins. We started from every Pfam family whose name says “domain of unknown function” (DUF) or “protein of unknown function”, and kept those with no structure in the PDB, at most 3 Europe PMC hits for the family’s accession, and at least 300 member proteins. We binned them by size (300 to 1,000 proteins, 1,000 to 3,000, 3,000 to 10,000, over 10,000) and drew 50 families per bin at random; only 17 families qualify above 10,000, so the set has 167. From each family we took 4 proteins from UniProt that are unreviewed (TrEMBL), come from Bacteria or Archaea, are 120 to 800 residues long, use only the 20 standard amino acids, are not low-complexity (Shannon entropy above 3.9 bits per residue), and are alone in their UniRef90 cluster, so no other UniProt protein is 90% or more identical to them. That gives 668 proteins.
Smaller families. For the family-size test we drew 30 more families in each of three smaller bins (20 to 60, 60 to 150 and 150 to 300 proteins) with the same criteria and 4 proteins per family, 360 proteins in all. Every model gets the same prompt and scoring as on the main set.
Cutting. Each protein is cut at 5 positions drawn at random inside its Pfam domain (the longest domain span, at least 40 residues), at least 10 residues from either end of the domain and with at least 30 residues before the cut: 3,340 cuts. Everything before the cut is the prefix; the residue right after it is the answer.
Asking. The model gets this message, with the prefix in 60-character lines (an example from one of the test proteins):
Below is the beginning of a protein sequence in one-letter amino acid codes.
Predict how it continues. Reply with exactly the next 10 amino acids as
one-letter codes on one line, and nothing else.
>sequence
MRSGGQPLHPQPAHRFHYRTAARPRAAWILGCVLHLADAEEGARFWWQYAAGAGDDAAAY
CLYLHCLVLGDSYAAALWREQTSLDTPPHDFDDTSVPTVLRILSRLTPA
Each model runs through its own API with temperature 0 where it is accepted, thinking off or at its lowest setting (for Opus 5.5 and the Fable models, low effort, at which the model still decides whether to think), no tools, and one sample per question. The Claude Fable models run through OpenRouter, since Anthropic’s API serves them only to workspaces that retain data. Models that declined to guess (GPT-4o, DeepSeek, Qwen, Kimi, GLM, gpt-oss) also got a version that adds “Give your best guess even if you are uncertain. Do not explain or refuse.”; those runs are marked separately.
Thinking. A reply counts as thought about when the API reports thinking tokens for it: thinking_tokens in Anthropic’s usage report, reasoning_tokens in OpenRouter’s. The thinking text itself is not kept; only the visible reply is scored. Fable 5.2 thought on 9% of questions, Opus 5.5 on 36%, Fable 5.1 never. Without thinking, Fable 5.2 is right 54.7% of the time on real proteins and 8.8% on shuffled ones (48.2% and 10.6% with thinking), and Opus 5.5 35.3% and 6.9% (34.1% and 8.6%).
Scoring. We read the last sequence the reply states: a run of capital amino-acid letters on its own line or after a label such as “Answer:”, skipping words, lists and anything inside thinking tags. If it starts by restating the end of the prefix, the restatement is dropped. Its first letter is the predicted next amino acid. A reply cut off at the length limit counts only if it opens with a sequence. Replies that state no sequence are excluded and counted (0.1% for Opus 5.5). Only the first residue is scored; the full 10 are used for exact continuations and runs. The same reader is used for every model and for real and shuffled proteins.
The shuffled control. Each protein’s residues are shuffled once, with a random seed fixed by its accession, and the same 5 cut positions are asked on the shuffled sequence. The shuffle keeps the composition of the whole protein and destroys the order, so a model that knows only which amino acids are common scores the same on both.
Statistics. For each family we compute the share of correct first residues on real and on shuffled cuts, and take the difference, in percentage points. The result for a model is the mean of that gap over the 167 families. Intervals are 95% bootstraps over families (10,000 resamples), the main test is a one-sided sign-flip permutation test over families, and the family-size trend is the Spearman correlation between a family’s gap and the logarithm of its size.
Raw continuation. For models that allow it, the same prefix is sent without any instruction, as the text “>sequence” followed by the FASTA lines, through a plain completion endpoint (open-weight models on Fireworks) or as the start of the model’s own reply (Gemini 2.5 Pro). The first letter the model writes is scored as above.
Protein language models. They get the same prefix and no prompt. ProGen3, ProtGPT3 and ProtGPT2 read it left to right and give a probability for the next token; the ESM models see the prefix, a mask in place of the next residue, and the end of the sequence. In every case the prediction is the most likely of the 20 standard amino acids.
Total API cost was about $730, plus a few dollars of GPU time for the protein language models.
Gemini 4 Carbon blocked about 75% of DNA prompts outright. OpenAI rejected 23 GPT-6 requests in the first round and 17 GPT-6.1 Sol requests in the main run as possible biological misuse, and 6.6% of GPT-6.2 Astra’s main-set questions (GPT-6.2 Sol: 0.6%). Claude Sonnet 4.5 refused 97% of requests that started its reply with a protein sequence, and Anthropic’s filter blocked 44% of Claude Fable 5’s questions as cyber content.
Leo Tianlai Chen built a set of 56 proteins from 31 of the families that were first made public in August 2026, after the assumed 1 July training cutoff, and have no earlier relative at 50% identity or more. The models cannot have seen them or anything close. Both Claude models still predict them, Fable 5.2 ahead of both protein language models, while GPT-6.2 Sol stays at zero. Every model scores about half of what it scores on the same families in the main set, as expected for proteins this distant. What the latest Claude models learned carries over to new members of a family.
The proteins were dated at their source, and none matches an earlier UniRef50 cluster at 50% identity or more. Cuts follow the main set’s rules. With 56 proteins only gaps of about 14 points or more are reliably detected, and the 1 July cutoff is an assumption.