I'd like to see the results for Ada on these same measures. On the theory that the Ada type system covers more classes of errors than other languages, and so AI can self correct better.
Ive been amazed at how well LLMs are at writing Gleam[1] and Lustre[2]. Compared to a mainstream language, there is basically zero gleam code in the training data.
I have no evidence to back this up, but I suspect that languages that are good for humans[3] will be good for LLMs. Compiled, strongly typed, statically typed, immutable, pure functions, pattern matched, memory safe, etc.
[1] https://gleam.run
[2] https://lustre.hexdocs.pm
[3] Yes I realize that languages features that are "good for humans" is a hotly debated topic. That's just my personal list for what I like in a language.
That's not a take I was expecting to find here. I've found most LLMs absolutely dreadful when it comes to Gleam, to the point that I most often disable even inline autocomplete when working in Gleam codebases.
Too often I find them getting pulled into larger ruts in the training data and trying to insert language features that don't exist (ifs, loops, and syntactic constructs) from more popular languages like TypeScript and Rust. Do you not experience other languages getting partially substituted in when you have LLMs write Gleam?
It's not clear to me how useful of a signal replicating existing pieces of well-known software is for this kind of evaluation, given what we know about how effectively LLMs can retrieve data from their training corpus and style-transfer it across different settings (programming languages here). That would explain their convergence in ability across different languages on the tasks in this post. I'd be far more interested in people's real-world experiences.
You are going to have to give more support for those assertions. I write Python every day, and never would I call it a good candidate for the clankers. Pretty much any dynamic language would be ruled out, as there is too much implicit logic which makes it harder to understand what is happening.
Any good LLM service (not just coding-focused ones) will write and run ad hoc code without being asked if your prompt involves lots of data. Gemini and Claude tend to pick Python with maybe some SQLite. Some of that must be due to portability alone, but it also means they'll make sure the model and tooling are good at those.
Cool line of questioning, but one piece of information is pivotal and critically not-yet-included: equivalent accomplishments in each language. For example, if I want to write standard things: web server, memoized fibonnaci, recipe search engine, what's the length-and-density of these outputs for each language? I think that would add in some ~normalization.
Strongly agree- this is how I “evaluated” languages pre-agents. though I suspect this would bias results in favor of whatever has best signal to noise for boilerplate from stackoverflow/reddit , rather than what LLM’s “””reason””” best with. (Presuming those aren’t quite one-and-the-same)
Is there a relationship between how good a programming language is for coding agents and how popular it is among humans? If so, wouldn't Python be the best language for agents, since it's is the most popular (and hence has the most context available for models)?
Pick something slightly esoteric (eg Haskell) and the quality of public code is very high, because you only have enthusiasts writing it. Choose something taught in schools (Python) and you are going to find 10,000 traveling salesmen homework problems and Django todo applications.
Not sure how you thread the needle on the quality vs quantity dynamic.
There is definitely a relationship. But I personally believe that once the training corpus reaches a certain scale, the returns exhibit diminishing marginal effects, to the point that multiplying the data volume cannot surpass something essential inherent in language design. (Asked an LLM to help me with the translation, so forgive my expression)
One thing worth noting is that syntactic density doesn't necessarily mean cheaper because because symbols don't chunk/tokenize as well as plain English
What I see from results like this is that the delta between languages is small enough now that it's hard to justify not not using something like Rust for the performance and correctness benefits if you're using LLMs and it fits the domain
I discovered last week that Fable 5 can write perfect xTensa LX7 assembler code without tools or references. Mind blown.
But, when working on a creative graphics task, the results were best in Lua, middling in integer-only C, and underwhelming in ASM in terms of creative depth.
> Dynamically typed languages generally have a lower LLM token cost than traditional statically typed languages because omitting explicit type declarations makes the code more compact.
If this was true, the programming languages that are very much on the left side of
should be very ideal for LLMs, in particular if they are dynamically typed.
What I can tell you is: I experimented with AI prompts for generating Wolfram (Mathematica) code using some LLMs, and I can tell you that the results were very disappointing: in my experience LLMs have difficulties with programming languages that are
- very concise, and
- for which there is less code publicly available.
Wolfram (Mathematica) is a good example of such a programming language.
I’ve actually found Sol delivers great results with Odin, despite there not being much Odin code available. I think it is able to work well with it because:
- It is a rather simple language
- It has a lot of very useful libraries already built in.
With just a single main.odin file you can do a heck of a lot stuff, which LLMs seem to like.
It’s interesting I’ve been surprised by how well Claude sonnet can write code in a language I’m developing that probably has no code in the training set. It seems like anything with syntax like python/ruby/elixir is pretty LLM friendly, and layering on a HM type system seems to help catch most errors.
> I think it is able to work well with it because:
> - It is a rather simple language - It has a lot of very useful libraries already built in.
> With just a single main.odin file you can do a heck of a lot stuff, which LLMs seem to like.
Also Wolfram/Mathematica has an insane amount of useful libraries already built in (there even exists the saying "Python is 'batteries included', Wolfram is 'spaceship included'"), and also there in a single file you can do a heck of a lot stuff.
On the other hand:
- LLMs tend to hallucinate non-existing function when you ask an LLM to code something in Wolfram that is not commonly done (concerning this point, nevertheless keep in mind that Wolfram is often used for "one-of-a-kind programs", i.e. for writing very specialized programs that have possibly never been done before).
- Wolfram code tends to be quite dense.
- If there is a small mistake in Wolfram code, the code typically simply won't work.
I love this take because I had exactly the opposite idea. I thought the combo of remarkably simple text (not even wrapped) with incredible, full width visualizations was chef's kiss. I really like the balance there personally, but I hear you. Does your browser have Reader Mode or something like that? I don't use those tools personally, but I believe they will recast the text parts into something that renders optimally for reading (ideal font size, number of characters per line, etc....).
Is doing something for yourself really working for free? That's an interesting take. But I can understand why you don't want this for yourself, so enjoy the page in all its splendour as it is already!
I mean, if it really bothers you you could fairly trivially apply picocss or whatever to it using a user stylesheet. That is so little effort that calling it working for free would be disingenuous to say the least.
I have no evidence to back this up, but I suspect that languages that are good for humans[3] will be good for LLMs. Compiled, strongly typed, statically typed, immutable, pure functions, pattern matched, memory safe, etc.
[1] https://gleam.run [2] https://lustre.hexdocs.pm [3] Yes I realize that languages features that are "good for humans" is a hotly debated topic. That's just my personal list for what I like in a language.
Too often I find them getting pulled into larger ruts in the training data and trying to insert language features that don't exist (ifs, loops, and syntactic constructs) from more popular languages like TypeScript and Rust. Do you not experience other languages getting partially substituted in when you have LLMs write Gleam?
What is the best language to have high quality correctness oracles so that the user doesn't have to babysit the LLM and do lots of manual testing?
Which programming languages are most token-efficient? - https://news.ycombinator.com/item?id=46582728 - Jan 2026 (91 comments)
Not sure how you thread the needle on the quality vs quantity dynamic.
What I see from results like this is that the delta between languages is small enough now that it's hard to justify not not using something like Rust for the performance and correctness benefits if you're using LLMs and it fits the domain
But, when working on a creative graphics task, the results were best in Lua, middling in integer-only C, and underwhelming in ASM in terms of creative depth.
If this was true, the programming languages that are very much on the left side of
> https://danuker.go.ro/programming-languages.html#non-math-ma...
> https://danuker.go.ro/programming-languages.html#overall-map
should be very ideal for LLMs, in particular if they are dynamically typed.
What I can tell you is: I experimented with AI prompts for generating Wolfram (Mathematica) code using some LLMs, and I can tell you that the results were very disappointing: in my experience LLMs have difficulties with programming languages that are
- very concise, and
- for which there is less code publicly available.
Wolfram (Mathematica) is a good example of such a programming language.
I guess this ignores languages with type inference? Hindley-Milner and others
- It is a rather simple language - It has a lot of very useful libraries already built in.
With just a single main.odin file you can do a heck of a lot stuff, which LLMs seem to like.
> - It is a rather simple language - It has a lot of very useful libraries already built in.
> With just a single main.odin file you can do a heck of a lot stuff, which LLMs seem to like.
Also Wolfram/Mathematica has an insane amount of useful libraries already built in (there even exists the saying "Python is 'batteries included', Wolfram is 'spaceship included'"), and also there in a single file you can do a heck of a lot stuff.
On the other hand:
- LLMs tend to hallucinate non-existing function when you ask an LLM to code something in Wolfram that is not commonly done (concerning this point, nevertheless keep in mind that Wolfram is often used for "one-of-a-kind programs", i.e. for writing very specialized programs that have possibly never been done before).
- Wolfram code tends to be quite dense.
- If there is a small mistake in Wolfram code, the code typically simply won't work.
https://danluu.com/web-bloat/
> Order food
> Food comes out as raw, unprepared ingredients
> Complain to chef
> Tells me to go cook it myself
> wtf, I'm not here to do work for free
> "Is doing something for yourself really working for free?"
And every person globally should do this as well for any other website that doesn't have a good default reading experience?