Porting one program to 10 languages with an LLM
Claude ported a 500-line Go bank statement analyzer to 10 languages. Same logic, same tests, different stdlib. I watched where it hesitated and asked it to predict its own performance before each port.
Small unknowns multiply
Each Odin stdlib call: ~75% confidence. Sounds fine. Fifteen such calls in one program:
0.7515 = 1.3%
Nim at 95% per call across 12 calls: 54%. One trivial fixup. Odin: 4 web searches, 2 fixups. Go and Python at 100%: zero fixups, zero searches. The math predicted every outcome.
This isn't specific to porting. Any LLM task that chains many "pretty confident" steps hits the same wall. The bottleneck is never one unknown. It's the product of many small ones.
Known gaps are cheap. Unknown APIs are expensive.
I measured "deliberation": fraction of thinking time spent worrying about the language instead of writing code. It doesn't track difficulty. It tracks uncertainty about API names.
JS has no CSV parser, no HTML escape. More stdlib gaps than Odin for this task. But the LLM knows those gaps exist. Quick workaround, move on. 5% deliberation.
Odin has a CSV reader. The LLM wasn't sure of its name. That cost more deliberation than hand-rolling a parser in JS. 80% deliberation.
A missing feature costs lines. An ambiguous feature costs thinking.
Verbosity is free
Go: 540 lines. Odin: 432 lines. Go was faster.
Java 8 was the extreme. One way to declare a list, one way to iterate, one way to aggregate. Near-zero design choices. The LLM doesn't think, just types. 495 lines, 3% deliberation, zero fixups.
Java 8 is the most annoying language for a human and the most effortless for the LLM. Tokens are cheap. Verification pauses are not. The boilerplate is the documentation.
Calibrated on difficulty, not effort
Before each port, the LLM predicted confidence, fixups, and deliberation. Confidence: near-perfect across 8 languages. Fixups: near-perfect. Deliberation: near-perfect, one miss.
Line counts:
| Language | Predicted | Actual | Error |
|---|---|---|---|
| Python | 150–200 | 253 | +27% |
| JS | 260–310 | 524 | +69% |
| Java 8 | 650–800 | 495 | −31% |
It knows what it doesn't know. It doesn't know how long things take to say. When an LLM calls a task "straightforward," it's probably right about the risk and wrong about the cost.
Setup is a language feature
Pascal and OCaml both failed. Neither because of language knowledge. Both produced zero lines.
Pascal attempt 1: read all existing ports before writing. Ran out of context. Attempt 2: started typing immediately.
Hit the output token limit. Pascal's begin/end verbosity exceeded the generation budget.
First attempt: wasted tokens on input. Second attempt: Pascal wastes tokens on output.
OCaml at ~85% confidence never got tested. opam init on Windows hit Cygwin PATH issues. Entire session gone.
Toolchain accessibility is a language feature.