Programming is in tension between the Light Side and the Dark Side.
The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!
The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.
Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
I think the big thing is that Lisp is (aside from mutable variable assignment) basically all declarative, rather than the imperative paradigm.
Even without static types, and even allowing macro craziness, there's just such a stronger baseline of declarative and functional thinking, you're off to such a good start in clearer thinking and reasoning about a program.
It isn't nearly as much of a trade-off as people say. Languages like Haskell are both remarkably expressive and provide a lot of safety. (And, of course, languages like Python, Java and Go are the opposite.)
I'm not big on static type checking in most situations -- adequate testing should find the type errors too -- but one place it looks very useful is avoiding data races in multithreaded programs. See Rust.
I have always assumed that Alan Kay compared Lisp to Maxwell's Equations because of the similarity between the interplay of eval and apply in Lisp on the one hand, and the interplay of the electric and magnetic fields in Maxwell's equations on the other.
And you have a good point. Maybe it's a weakness in the metaphor if one can see it either way.
I fundamentally believe that there is no perfect language, instead one merely chooses a set of trade-offs. What's interesting about Lisp is that it is simultaneously well-respected and yet not widely used in productions (compared to C, C++, Python, JavaScript, and even Rust).
It's almost like Lisp chose a particularly extreme set of trade-offs that yield great power but also repel large swathes of programmers. I think PG might say that it repels dumb programmers, but maybe not.
for a while there's been a strong cultural tendency in programming to mistrust people and trust tools (often even blindly because tools are just assumed to make no mistakes), and so expressiveness has been sacrificed for safety. There can be something to this but it's also self-fulfilling, if you strip people of agency of course they'll unlearn to program.
I've always thought it's a misanthropic philosophy and sucked much joy out of programming but also there's something to be said that there's safety in the expressiveness of Lisp. I read an article a while ago that found a robust correlation simply between length of a code base and errors regardless of languages used. And given how succinct and clear Lisp codebases can be that's valuable in itself.
we're kind of at the logical endpoint of this now with gigantic slop codebases that nobody understands just held together by 20 different tools, and if you ask me if I had to pick one of those or something one tenth the size written by a guy or girl who has been writing Lisp for ten years I'd say thank you I'll go with #2
The website seems to have a bug with syntax highlighting. Pieces of code included in the post text are black, you can still see the actual text if you select it with your mouse. Same bug on Chrome desktop and on Safari on iPad
This is the classic example of why you should always set the foreground color when changing the background color. Otherwise you end up with <code> blocks that are black-on-black. It probably works fine if your browser is in dark mode.
A workaround is to open one of the code elements in the browser's debugger and add color: white to the :not(pre) > code style.
There are many articles extolling the virtues of Lisp. I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Articles like this, and the PG articles it references, amount to "if you know, you know". I understand the appeal and I understand the explicit and implicit arguments this article is making.
Computer programming has matured quite a bit in the past 60 years. I would like to see more articles that are more considered in their examination.
As a language Lisp is great (though the ecosystem is limited). It has two flaws-- it's very open-ended so unless you're talented and disciplined you can fall into a rabbit hole of hacking fun Lisp stuff and not actually getting any work done. Other languages have this problem, but Lisp I think more so.
The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.
> I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Standardized Concurrency is basic table-stakes for a language today. CL does not have a standardized async/await or concurrency model. The standard hasn't been updated since 1995 so it will never happen.
And it remains in a state of almost getting to the point of generic collections (like C++, Clojure, many others) using the standard functions, but not quite getting there. There are functions (not generic functions) which operate on sequences, but no standard way to extend what types are considered sequences (as one example). It makes sense that the 1995 version of the spec would be incremental, but without a further development (or consensus amongst the implementations if no official standard) it's difficult to continue moving forward to make better use of the language's generic function capabilities.
That's where Clojure, Julia, and many others have advanced beyond what CL (by the standard) provides. The language needed at least one or two more standards to drive those features forward, but never got them.
There are some truly powerful and unique things about Lisps, but I wish articles like this would stop including REPLs and hot-reloading. The former have been table stakes for interpreted languages (and some compiled ones!) for years, and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
> and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
For what it's worth, hot reload is very widely used in Dart/Flutter. When you are writing UI code using a reactive style framework where rendering the UI appears to be "generate a new UI from scratch on each frame", it's more straightforward to have an intuition about what does and doesn't get reloaded.
It's not perfect, of course. But it works really well for the kind of changes you make when iterating on a user experience.
> The former have been table stakes for interpreted languages
I used to think so too back in the day when I was getting into Clojure. It was much later when I realized that when Lisp people talk about the REPL they’re usually talking not so much about the interactive CLI where you can evaluate commands easily but more so the ability to connect your program to a live session where you can quickly evaluate forms within your text editor and in the context of your running application, which enables much more interactive development than in other interpreted languages.
No. There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages - Ruby, Python, Haskell, etc. Every single stage there in Read-Eval-Print-Loop differs.
Man, it gets so exhausting trying to convince every PL critic who grabs a single (or a couple) of axis of any language and tries to dispute the value of a language without ever understanding the holistic, overall experience working with it.
Like, I don't understand, do people think that tons of Clojurians or Common Lispers who fall in love with the language after decades of working and getting seasoned in literally dozens of different PLs are on some kind of delusional trip or something?
Guys, just take a gander at Clojurians Slack; see what people are working on, what kind of stuff they're building; check their profiles. Many of them are the battle-scarred veterans of coding. Sure, some of them may have wrong opinions, but surely they can't be all wrong, can't they?
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
Smalltalk is not homoiconic, and it's REPL experience is equivalent (I'd argue somewhat better, but that's mostly a tooling thing, see the commercial CL implementations as examples of improvements over SBCL + Emacs + Slime). Homoiconicity is not the trait that makes the CL REPL experience better than others, it's that it includes a very good debugger, the compiler, hot code reloading, the ability to redefine classes and update current instances, and so on.
That's the tooling, not the language, that provides the experience. Nothing about being non-homoiconic prevents other languages from having a comparable (or even better) experience.
Nothing about being statically/dynamically typed. Nothing about being functional/OOP/relational/logic. Nothing about being pure/side-effecty/strict/lazy. Nothing about being compiled/interpreted. Nothing about imperative/procedural/stack-oriented.
You can pick just about any single or (few) aspects about any language. Heck, it doesn't even have to be a programming language and you can find things to complain about.
You know that there are three genuine, true, legit ways to build robust, bug-free, performant software? Three! The problem? Nobody knows what they are and that's why we are all doomed to keep bashing on everyone else's choices and opinions.
"Better experience" is not dictated by "features". Better experience comes, well, with experience.
I suppose it's my own fault. I tried focusing on "holistic, overall experience", yet still picked a single aspect to chime in.
I wrote GGP as a Lisp user and enjoyer—not pretending to deep experience, but definitely am well acquainted. And neither the REPL nor live reloading are important differentiators. Neither is the ability to attach a REPL to a running program: plenty of languages have that built in (Erlang, Ruby), and many others support it through popular third party tooling (e.g. Pyrasite for Python, and—ironically—the class executor for the Clojure REPL for arbitrary non-Clojure JVM programs). Many Lisps’ REPL tooling is very nice, and the language lends itself well to REPL-oriented development. But those aren’t “uniquely Lisp” features as you and many others claim.
Homoiconicity is a "uniquely Lisp" feature and it doesn't seem like you've fully grokked the implications and differences between homoiconic and non-homoiconic. I'm not making this up - every step there in Read-Eval-Print-Loop does differ. That is easily verifiable info.
Tell me your knowledge of Ruby is surface level without telling me...
I used Clojure when it first came out, I've used Common Lisp for years, I've also used languages like Smalltalk and many others.
Yes, Lisp environments are nice, but sometimes I think Lispers are so insular they don't realize that other languages have similar things. R, Julia and Ruby have similar environments. Smalltalk is next level.
And homoiconicity is great for macros and parsing but it's late-binding that enables the live programming behaviour, which isn't exclusive to Lisp.
Also if you'd ever gotten deep into say, SB-ALIEN, you'd know the limitations too; Common Lisp isn't magic, it can't just redefine say, instantiated structs in memory. It relies on pointers then switching references on the fly.
> they don't realize that other languages have similar things
Yes they do. God, how did you (and apparently you're not alone) read my rant and still got it 100% backwards? I specifically hinted about not picking a single aspect of a language - REPL or whatever. What is so confusing about my wording on "holistic, overall experience working with it"?
Apart from your rant, this is the most pertinent part of what you said:
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
And it's false. There's nothing special about a REPL in a homoiconic language. Homoiconicity has nothing to do with a REPL or tools.
I do however think SBCL has a very nice environment with great tools (especially it's compiler) and that's why I've used it a lot over the years, but I really think you underestimate the environments in other languages (including the "holistic, overall experience").
> And it's false. There's nothing special about a REPL
You are wrong. You might be correct about the workflow but still wrong about the semantics. Pedantically, your notion is false, because homoiconism gives the REPL a genuine capability (program-as-manipulable-data across the read/eval boundary) that non-homoiconic REPLs lack.
Not all Lisp dialects have well-developed hot patching systems. The OOP system has to be carefully designed for it. What if a class definition is superseded by a reload, but there are existing instances? The Common Lisp dialect of Lisp has useful answers to questions like this, but not necessarily every Lisp you come across.
Even Bash supports hot-reloading. I've developed modules that are updated in place by sourcing:
I do think it is worth highlighting how many advanced parts of hot-reloading have already been covered in Common Lisp. Same with highlighting how the REPL is largely not used to directly type into, but is instead a very powerful interface for tools to interact with a running image.
But, again agreed that simply these existing are not that notable today.
I must admit - I still don't understand macros. I get that they're code that's generated at compile time. But I don't understand how that's different than a function which evaluates other functions. I guess the latter would actually be evaluated at runtime? I think I get it conceptually but I'm not sure I have the muscle memory to reach for them. Anybody here have an "ah hah!" Moment with macros?
> But I don't understand how that's different than a function which evaluates other functions.
You want a function that takes in arguments, but does not evaluate the arguments when called. So:
func(foobar(), foobar())
Normally, foobar() will be called twice - at the time of the call. With macro expansion, you can ensure that it's not the result of calling foobar that goes into the func, but this expression.
A canonical example is if you want to write an if/then/else function:
if(condition, then_path, else_path)
It's quite possible that the else_path is invalid and will terminate the program if the condition is true. But if you wrote a function this way, it will evaluate both then_path and else_path - not something you want to do in a regular if expression!
Many problems can be solved with either macros or higher-order-functions. The advantage of macros is two-fold (macros also have disadvantages, and Lisp allows you to use either solution; deciding which to use is often a matter of taste):
1. Syntax can be more familiar
2. Performance (to the extent that there is overhead for functions-calling-functions).
For example, let's consider a hypothetical lispy language that doesn't have a short-circuiting "and" operator. Macros would let you implement something to used like this:
(and (foo x) (bar y))
Higher order functions would require you to do something more like:
(and (lambda () (foo x) (lambda () (bar y))
More noise, and more work for the optimizer (or in the worst-case, more work at run-time). Languages that rely heavily on higher-order-functions will tend to have terser syntax for anonymous functions. For example in javascript you might do:
Sure, macros are functions that take functions as input, and produce new functions as output. But they take the function's symbols as input and produce a new set of symbols. So a macro can extend the syntax of the language without having to modify the core language system. Anyway, what's unique about Lisp macros vs say, Rust macros, or C style preprocessors, is "homoiconicity". The data structure that a Lisp macro takes as an input, the lisp code, is the same data structure that the language uses normally (S-expressions, lists...), so writing a macro requires few new language skills compared to writing normal lisp (again, compare writing Rust macros, a dark art in comparison).
If you haven't read it, I'd suggest taking a look at Paul Graham's book On Lisp [0]. He says better, and with more examples than I'd provide in a comment block, what I'd write on the subject. Jump to chapter 8 for his discussion on the topic, referring back to chapter 7 if you find the macro definitions difficult to read.
If it helps, https://taeric.github.io/CodeAsData.html was my attempt at exploring "code as data" and what makes lisp different here. My specific focus was more to point out that "eval" in lisp doesn't just take in a string. But I think the same general points remain.
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
There's a bit of a mental model flip to make maybe.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
one way to see it is that it's a function that runs at compile time. for instance instead of dumping magic numbers/tables in a codebase you could put the code and substitute to their value at compile time.
but also it can change your code, so you get to do all the java annotation magic stuff.
The arguments presented to a macro don't have to be valid code. Your "function evaluating functions" are all individually valid functions you've composed at runtime.
(defmacro foo [code] ,,,)
(foo "<some totally different language in a string>")
=> actual, compiled Clojure function built up by `foo` parsing the string, producing a Clojure list, and calling `(eval the-list)` and returning it.
I passed a string to `foo` in the example, but it could actually be anything the Clojure reader can parse even if it is semantically invalid—arbitrary Clojure data.
Macros are conceptually similar to FEXPRs in that they act like functions that don't evaluate their operands and return code. The (only?) difference is that macro are all expanded before execution/runtime.
You do not need macros for anything. They are not the tool to "extend the language", but instructions for the compiler. And if the system does not even have a compiler, macros are useless, often incredibly stupid.
If you have an interpreter only, no compiler, using macros for metaprogramming anwyay at least prepares you for the eventuality that one day there will be a compiler. The macros will Just Work as before, only the expanded code is now processed by compiling.
Suppose you reject the idea that there will ever be a compiler. Macros are still useful for doing "compiler-like things" in the context of interpretation, like transforming code into something that will interpret better.
We can regard the interpreter as a virtual machine acting on a representation of the code; the macro system lets you manipulate the representation in a pre-computed pass, which has no further cost at run time.
If you have a code expansion pass on top of an interpreter, for supporting macros, you can use that as an excuse to perform built-in code transformations that are not macros; those enable you to have more flexibility in how special forms are implemented.
In TXR Lisp case is a macro (family) which performs certain optimizations like recognizing values in a range and emitting a table switch. This works interpreted or compiled:
It is faster to do some checks and interpret the sys:switch special form to dispatch by a numeric index than to do a large number of exhaustive comparisons.
Ultimately, the way we optimize interpretation is by compiling, but it can still be worth it to have better interpretation here and there.
You don't want to do this kind of optimization at run-time; you don't want the interpreter to be evaluating the condition "are these cases integers (or characters) in a tight range?". That's a property of the syntax in which the cases are constants; it wants to be pre-computed once.
Yes, to a certain degree, although it does take initial effort to make it happen.
LLMs do somewhat okay job when you use them with Lisp, treating it just like any other PL, which is roughly the Unix/pipe model - batch-style. Agent spawns process -> reads stdout/stderr -> spawns next process. State lives in-between the calls and in files. Each tool invocation is stateless.
Things get far more interesting when you give an LLM a true Lisp REPL. LLM stops guessing and starts empirically analyzing current state of things and produces working solution faster, costing far less tokens. And you get to watch it solve things interactively, e.g. I often let AI poke through our UI (via Playwright-driven Clojurescript REPL), while monitoring situation in k8s - through nrepl port, connected to Clojure REPL. It literally interactively walks the DOM, finds the selectors, clicks buttons, etc. All without restarts, complex state management and all.
Current SOTA models work great with Emacs Lisp and Common Lisp. Giving access to the REPL enables them to do an amazing job but they still work well with Unix input/output static code processing paradigm.
Depends very much on your harness and effective use of tools. For vibe-coding, stick to Python.
It can get good with the right setup. I made Codex work with GToolkit (similarly underrepresented, but technically impressive and a seemingly good fit for LLMs), but it took a lot of tweaking of the project structure, extensive instructions in AGENTS.md, and some custom skills (some borrowed from Gt4Llm, the GT's built-in model harness). Out of the box, it burned tokens and took way too long to implement even basic things.
YMMV - as with everything LLM-related - but I think without a similar setup, an agent instructed to write CL/Clj/Scheme/Racket will have the same issues. It might be better in an established, large project - but starting from an empty Git repo, I suspect you'll have to fight your way to productivity.
I've found LLMs to be bad at balancing parenthesis. I've also found them to be less likely to hallucinate library types in dynamic languages, they tend to hallucinate arguments to library functions/methods instead.
Because you treating Lisp just like any other (non-homoiconic) PL. Give an agent a true Lisp REPL to mess around, and you'd be surprised. Things get very interesting. I still don't understand why more people don't do that - isn't that obvious first thing anyone should figure out? Like I can't even imagine working with Lisp without a REPL and structural editing - I'd immediately fail at balancing parens. Why do you expect a [dumber] machine would do any better?
Yeah but apart from SBCL, Viaweb, Hacker News, Emacs, Clojure, Scheme, Racket, garbage collection, macros, homoiconicity, the REPL, S-expressions, symbolic computation, what has Lisp ever done for us?
> [...] among its extensibility, its interactive environment, the REPL, and a lot of other features we haven’t touched yet. It is the combination of all of them that makes Lisp programming what it is.
Agreed. However, although the alternatives are few, they do exist. Today, I'd like to convince you (whether you're OP or a commenter) to give one of them a try. I'm talking about GToolkit[1]: Smalltalk/Pharo-based reimagining of Smalltalk as a productive environment for modern system design, analysis, and implementation. It's based on Pharo and its VM, but with GT-specific extensions and replacements, developed on GitHub in both Smalltalk and Rust.
I used both Common Lisp and Smalltalk over the years for some of my side projects. Technically, the environments are comparable: image-based, live, interactive development is central to both. Lisp is easier to fit into modern workflows: it's still just files on disk. If you ignore the REPL and treat the image as a bundler, you can have a Java- or Python-like development workflow. You can gradually adopt the more interactive ways of working with the codebase. Smalltalk was historically more of an "all or nothing" approach, but nowadays it supports Git-based workflows as first-class, with GToolkit providing additional tools directly. GT is tied to a single IDE, but that IDE is genuinely powerful, pragmatic, and easy to customize: the entire IDE is Smalltalk code[2] that lives in the image beside your code, so you can live-edit any part of it at any time.
The languages differ, most obviously in the object model (single inheritance and message sends in Smalltalk, and multiple inheritance with multimethods in CLOS), but I don't think one is strictly better than the other for the vast majority of code. You just need to structure your code differently. Both systems are very dynamic, so neither is like Java or C++. After quite a few projects in both, I am convinced that this difference doesn't matter.
Both languages have very simple syntax. Smalltalk doesn't have macros; instead, it has an extensible/replaceable compiler for method bodies. This capability is used, for example, for compiling grammars into parsers/lexers in SmaCC. It's not as convenient for control-flow abstraction as CL's macros, but Smalltalk tends to make them out of blocks (lexical closures with non-local exits) plus actions on thisContext. In practice, I never found either language too limiting in what I could express, and both have small, regular basic syntax.
TL;DR: Common Lisp and GToolkit Smalltalk offer comparable technical merits, with the most pronounced difference being GT's built-in, high-quality IDE, which open source Lisp lacks.
The IDE (and the capability to build cross-platform, but natively rendered GUI apps) is the major selling point of GToolkit, but the bigger reason to consider it is social. Lisp had a bit of a renaissance from 2005 to 2015, but ~it has since died down~ (EDIT: After checking a bit, it's more like there are similar levels of development now, but it's just varied, and I personally hit a few areas where nothing new happened for a while. Or in other words, the hopes of CL getting catapulted to mainstream by the wave of adoption didn't materialize, and the adoption seems to have largely plateaued, with a slight upward trend since then.) A lot of great things for Lisp appeared in that time: ASDF and Quicklisp being prime examples. Unfortunately, the momentum was lost, and since then, the rate of development has plummeted. It's still a solid proposition because of its stability, which means a lot of old code still works perfectly well, but it's a double-edged sword: it would take a hundred miracles in a row for CL to get a plausible M:N concurrency story, for example. This isn't true for Smalltalk, and doubly so.
There's a small but active community around Pharo, mostly academics and hobbyists (IME; and it's from a decade ago, so YMMV). Pharo itself is already a "Smalltalk-like" language, and it regularly gets features that are not in original Smalltalk (stateful traits and slots (reification of class and instance variables) are good examples). It continues to evolve, and each version brings notable improvements to the language, the VM, and the class library. In CL, since there are many implementations maintained by various groups and the standard is set in stone, adopting extensions to the standard is incredibly hard and time-consuming. You can, of course, commit to a single implementation, but there are only 2 or 3 implementations that are actively developed and trying to do something "new"; they are all experimental and incomplete. And while SBCL has quite a few nice features, its codebase was a bit hard for me to grok (read: days on end of banging my head against a wall, then giving up; I'm much more comfortable messing with Smalltalk bytecode compiler).
Secondly, GToolkit is driven by a group of coders[3] who have established a profitable consultancy. It's being developed around a single vision that the authors swear helps them achieve their projects' goals. Even if you're not sold on moldable development[4] as a methodology, GToolkit gives you all the tools you might need in a modern development and then adds tons of domain-specific tools and utilities that might be useful after just a bit of adjustment. The whole environment is built to make such adjustments as painless as possible, too.
The only problem I experience with GToolkit is the lack of structured, book-like documentation. The GToolkit book is closer to the PHP wiki or a bundle of tutorials than something like the Rust book. It's not that the docs aren't there - they often are, but finding them effectively is a bit challenging. I found that simply cloning all the repos that GT consists of (tens, if not hundreds, but there's automation for that) and pointing an agent to the directory with them is often enough to quickly find what I'm looking for (if the built-in Spotter fails and I'm too lazy to construct an in-image search).
TL;DR: GToolkit/Pharo move much faster than the CL world, and while the number of maintainers might be similar (honestly hard to estimate, though), the effort on the GT side is more centralized and, in my opinion, heads in a better direction. If you want an extensible, live, dynamic environment that shares many of Common Lisp's strengths, but is more polished, more actively maintained, and feels more modern, take a look at GToolkit.
(Disclaimer: no affiliation, just a programmer who likes to explore unpopular languages).
The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!
The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.
Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
Even without static types, and even allowing macro craziness, there's just such a stronger baseline of declarative and functional thinking, you're off to such a good start in clearer thinking and reasoning about a program.
It's much better to give all the power to the programmer, to allow him to fix his mistakes rather than fantasising about preventing them.
This isn't so far off, considering that some people consider the "Lisp in Lisp" bit from the 1.5 manual to be the "Maxwell's Equations in Software":
https://michaelnielsen.org/ddi/lisp-as-the-maxwells-equation...
Relevant PG: https://paulgraham.com/langdes.html
And you have a good point. Maybe it's a weakness in the metaphor if one can see it either way.
I fundamentally believe that there is no perfect language, instead one merely chooses a set of trade-offs. What's interesting about Lisp is that it is simultaneously well-respected and yet not widely used in productions (compared to C, C++, Python, JavaScript, and even Rust).
It's almost like Lisp chose a particularly extreme set of trade-offs that yield great power but also repel large swathes of programmers. I think PG might say that it repels dumb programmers, but maybe not.
I've always thought it's a misanthropic philosophy and sucked much joy out of programming but also there's something to be said that there's safety in the expressiveness of Lisp. I read an article a while ago that found a robust correlation simply between length of a code base and errors regardless of languages used. And given how succinct and clear Lisp codebases can be that's valuable in itself.
we're kind of at the logical endpoint of this now with gigantic slop codebases that nobody understands just held together by 20 different tools, and if you ask me if I had to pick one of those or something one tenth the size written by a guy or girl who has been writing Lisp for ten years I'd say thank you I'll go with #2
A workaround is to open one of the code elements in the browser's debugger and add color: white to the :not(pre) > code style.
Articles like this, and the PG articles it references, amount to "if you know, you know". I understand the appeal and I understand the explicit and implicit arguments this article is making.
Computer programming has matured quite a bit in the past 60 years. I would like to see more articles that are more considered in their examination.
The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.
Standardized Concurrency is basic table-stakes for a language today. CL does not have a standardized async/await or concurrency model. The standard hasn't been updated since 1995 so it will never happen.
That's where Clojure, Julia, and many others have advanced beyond what CL (by the standard) provides. The language needed at least one or two more standards to drive those features forward, but never got them.
It's been discussed many times on HN.
For what it's worth, hot reload is very widely used in Dart/Flutter. When you are writing UI code using a reactive style framework where rendering the UI appears to be "generate a new UI from scratch on each frame", it's more straightforward to have an intuition about what does and doesn't get reloaded.
It's not perfect, of course. But it works really well for the kind of changes you make when iterating on a user experience.
I used to think so too back in the day when I was getting into Clojure. It was much later when I realized that when Lisp people talk about the REPL they’re usually talking not so much about the interactive CLI where you can evaluate commands easily but more so the ability to connect your program to a live session where you can quickly evaluate forms within your text editor and in the context of your running application, which enables much more interactive development than in other interpreted languages.
Man, it gets so exhausting trying to convince every PL critic who grabs a single (or a couple) of axis of any language and tries to dispute the value of a language without ever understanding the holistic, overall experience working with it.
Like, I don't understand, do people think that tons of Clojurians or Common Lispers who fall in love with the language after decades of working and getting seasoned in literally dozens of different PLs are on some kind of delusional trip or something?
Guys, just take a gander at Clojurians Slack; see what people are working on, what kind of stuff they're building; check their profiles. Many of them are the battle-scarred veterans of coding. Sure, some of them may have wrong opinions, but surely they can't be all wrong, can't they?
Smalltalk is not homoiconic, and it's REPL experience is equivalent (I'd argue somewhat better, but that's mostly a tooling thing, see the commercial CL implementations as examples of improvements over SBCL + Emacs + Slime). Homoiconicity is not the trait that makes the CL REPL experience better than others, it's that it includes a very good debugger, the compiler, hot code reloading, the ability to redefine classes and update current instances, and so on.
That's the tooling, not the language, that provides the experience. Nothing about being non-homoiconic prevents other languages from having a comparable (or even better) experience.
Nothing about being statically/dynamically typed. Nothing about being functional/OOP/relational/logic. Nothing about being pure/side-effecty/strict/lazy. Nothing about being compiled/interpreted. Nothing about imperative/procedural/stack-oriented.
You can pick just about any single or (few) aspects about any language. Heck, it doesn't even have to be a programming language and you can find things to complain about.
You know that there are three genuine, true, legit ways to build robust, bug-free, performant software? Three! The problem? Nobody knows what they are and that's why we are all doomed to keep bashing on everyone else's choices and opinions.
"Better experience" is not dictated by "features". Better experience comes, well, with experience.
I suppose it's my own fault. I tried focusing on "holistic, overall experience", yet still picked a single aspect to chime in.
I used Clojure when it first came out, I've used Common Lisp for years, I've also used languages like Smalltalk and many others.
Yes, Lisp environments are nice, but sometimes I think Lispers are so insular they don't realize that other languages have similar things. R, Julia and Ruby have similar environments. Smalltalk is next level.
And homoiconicity is great for macros and parsing but it's late-binding that enables the live programming behaviour, which isn't exclusive to Lisp.
Also if you'd ever gotten deep into say, SB-ALIEN, you'd know the limitations too; Common Lisp isn't magic, it can't just redefine say, instantiated structs in memory. It relies on pointers then switching references on the fly.
Yes they do. God, how did you (and apparently you're not alone) read my rant and still got it 100% backwards? I specifically hinted about not picking a single aspect of a language - REPL or whatever. What is so confusing about my wording on "holistic, overall experience working with it"?
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
And it's false. There's nothing special about a REPL in a homoiconic language. Homoiconicity has nothing to do with a REPL or tools.
I do however think SBCL has a very nice environment with great tools (especially it's compiler) and that's why I've used it a lot over the years, but I really think you underestimate the environments in other languages (including the "holistic, overall experience").
You are wrong. You might be correct about the workflow but still wrong about the semantics. Pedantically, your notion is false, because homoiconism gives the REPL a genuine capability (program-as-manipulable-data across the read/eval boundary) that non-homoiconic REPLs lack.
Even Bash supports hot-reloading. I've developed modules that are updated in place by sourcing:
I do think it is worth highlighting how many advanced parts of hot-reloading have already been covered in Common Lisp. Same with highlighting how the REPL is largely not used to directly type into, but is instead a very powerful interface for tools to interact with a running image.
But, again agreed that simply these existing are not that notable today.
You want a function that takes in arguments, but does not evaluate the arguments when called. So:
Normally, foobar() will be called twice - at the time of the call. With macro expansion, you can ensure that it's not the result of calling foobar that goes into the func, but this expression.A canonical example is if you want to write an if/then/else function:
It's quite possible that the else_path is invalid and will terminate the program if the condition is true. But if you wrote a function this way, it will evaluate both then_path and else_path - not something you want to do in a regular if expression!1. Syntax can be more familiar
2. Performance (to the extent that there is overhead for functions-calling-functions).
For example, let's consider a hypothetical lispy language that doesn't have a short-circuiting "and" operator. Macros would let you implement something to used like this:
Higher order functions would require you to do something more like: More noise, and more work for the optimizer (or in the worst-case, more work at run-time). Languages that rely heavily on higher-order-functions will tend to have terser syntax for anonymous functions. For example in javascript you might do:[0] https://www.paulgraham.com/onlisptext.html
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
but also it can change your code, so you get to do all the java annotation magic stuff.
I passed a string to `foo` in the example, but it could actually be anything the Clojure reader can parse even if it is semantically invalid—arbitrary Clojure data.
Suppose you reject the idea that there will ever be a compiler. Macros are still useful for doing "compiler-like things" in the context of interpretation, like transforming code into something that will interpret better.
We can regard the interpreter as a virtual machine acting on a representation of the code; the macro system lets you manipulate the representation in a pre-computed pass, which has no further cost at run time.
If you have a code expansion pass on top of an interpreter, for supporting macros, you can use that as an excuse to perform built-in code transformations that are not macros; those enable you to have more flexibility in how special forms are implemented.
In TXR Lisp case is a macro (family) which performs certain optimizations like recognizing values in a range and emitting a table switch. This works interpreted or compiled:
It is faster to do some checks and interpret the sys:switch special form to dispatch by a numeric index than to do a large number of exhaustive comparisons.Ultimately, the way we optimize interpretation is by compiling, but it can still be worth it to have better interpretation here and there.
You don't want to do this kind of optimization at run-time; you don't want the interpreter to be evaluating the condition "are these cases integers (or characters) in a tight range?". That's a property of the syntax in which the cases are constants; it wants to be pre-computed once.
LLMs do somewhat okay job when you use them with Lisp, treating it just like any other PL, which is roughly the Unix/pipe model - batch-style. Agent spawns process -> reads stdout/stderr -> spawns next process. State lives in-between the calls and in files. Each tool invocation is stateless.
Things get far more interesting when you give an LLM a true Lisp REPL. LLM stops guessing and starts empirically analyzing current state of things and produces working solution faster, costing far less tokens. And you get to watch it solve things interactively, e.g. I often let AI poke through our UI (via Playwright-driven Clojurescript REPL), while monitoring situation in k8s - through nrepl port, connected to Clojure REPL. It literally interactively walks the DOM, finds the selectors, clicks buttons, etc. All without restarts, complex state management and all.
It can get good with the right setup. I made Codex work with GToolkit (similarly underrepresented, but technically impressive and a seemingly good fit for LLMs), but it took a lot of tweaking of the project structure, extensive instructions in AGENTS.md, and some custom skills (some borrowed from Gt4Llm, the GT's built-in model harness). Out of the box, it burned tokens and took way too long to implement even basic things.
YMMV - as with everything LLM-related - but I think without a similar setup, an agent instructed to write CL/Clj/Scheme/Racket will have the same issues. It might be better in an established, large project - but starting from an empty Git repo, I suspect you'll have to fight your way to productivity.
Because you treating Lisp just like any other (non-homoiconic) PL. Give an agent a true Lisp REPL to mess around, and you'd be surprised. Things get very interesting. I still don't understand why more people don't do that - isn't that obvious first thing anyone should figure out? Like I can't even imagine working with Lisp without a REPL and structural editing - I'd immediately fail at balancing parens. Why do you expect a [dumber] machine would do any better?
I think that was true last year. In my experience, it’s no longer the case with Claude Code or Codex.
games (Kandria),
Maxima,
a faster pgloader (https://tapoueh.org/blog/2014/05/why-is-pgloader-so-much-fas...),
hackernews (was rewritten to SBCL: https://lisp-journey.gitlab.io/blog/hacker-news-now-runs-on-...), first reddit…
https://lisp-screenshots.org/
https://www.lispworks.com/success-stories/
also Emacs (for another dialect).
(Yeah, OK, he didn't have a programming language...)
phanitathion, roaphs, aqueduphs, mediphine, wphine...
I'm sorry. Couldn't resist - I watch Python and do Lisp. Or the other way around, sometimes. And, no, I'm not from Barthselona.
> [...] among its extensibility, its interactive environment, the REPL, and a lot of other features we haven’t touched yet. It is the combination of all of them that makes Lisp programming what it is.
Agreed. However, although the alternatives are few, they do exist. Today, I'd like to convince you (whether you're OP or a commenter) to give one of them a try. I'm talking about GToolkit[1]: Smalltalk/Pharo-based reimagining of Smalltalk as a productive environment for modern system design, analysis, and implementation. It's based on Pharo and its VM, but with GT-specific extensions and replacements, developed on GitHub in both Smalltalk and Rust.
I used both Common Lisp and Smalltalk over the years for some of my side projects. Technically, the environments are comparable: image-based, live, interactive development is central to both. Lisp is easier to fit into modern workflows: it's still just files on disk. If you ignore the REPL and treat the image as a bundler, you can have a Java- or Python-like development workflow. You can gradually adopt the more interactive ways of working with the codebase. Smalltalk was historically more of an "all or nothing" approach, but nowadays it supports Git-based workflows as first-class, with GToolkit providing additional tools directly. GT is tied to a single IDE, but that IDE is genuinely powerful, pragmatic, and easy to customize: the entire IDE is Smalltalk code[2] that lives in the image beside your code, so you can live-edit any part of it at any time.
The languages differ, most obviously in the object model (single inheritance and message sends in Smalltalk, and multiple inheritance with multimethods in CLOS), but I don't think one is strictly better than the other for the vast majority of code. You just need to structure your code differently. Both systems are very dynamic, so neither is like Java or C++. After quite a few projects in both, I am convinced that this difference doesn't matter.
Both languages have very simple syntax. Smalltalk doesn't have macros; instead, it has an extensible/replaceable compiler for method bodies. This capability is used, for example, for compiling grammars into parsers/lexers in SmaCC. It's not as convenient for control-flow abstraction as CL's macros, but Smalltalk tends to make them out of blocks (lexical closures with non-local exits) plus actions on thisContext. In practice, I never found either language too limiting in what I could express, and both have small, regular basic syntax.
TL;DR: Common Lisp and GToolkit Smalltalk offer comparable technical merits, with the most pronounced difference being GT's built-in, high-quality IDE, which open source Lisp lacks.
The IDE (and the capability to build cross-platform, but natively rendered GUI apps) is the major selling point of GToolkit, but the bigger reason to consider it is social. Lisp had a bit of a renaissance from 2005 to 2015, but ~it has since died down~ (EDIT: After checking a bit, it's more like there are similar levels of development now, but it's just varied, and I personally hit a few areas where nothing new happened for a while. Or in other words, the hopes of CL getting catapulted to mainstream by the wave of adoption didn't materialize, and the adoption seems to have largely plateaued, with a slight upward trend since then.) A lot of great things for Lisp appeared in that time: ASDF and Quicklisp being prime examples. Unfortunately, the momentum was lost, and since then, the rate of development has plummeted. It's still a solid proposition because of its stability, which means a lot of old code still works perfectly well, but it's a double-edged sword: it would take a hundred miracles in a row for CL to get a plausible M:N concurrency story, for example. This isn't true for Smalltalk, and doubly so.
There's a small but active community around Pharo, mostly academics and hobbyists (IME; and it's from a decade ago, so YMMV). Pharo itself is already a "Smalltalk-like" language, and it regularly gets features that are not in original Smalltalk (stateful traits and slots (reification of class and instance variables) are good examples). It continues to evolve, and each version brings notable improvements to the language, the VM, and the class library. In CL, since there are many implementations maintained by various groups and the standard is set in stone, adopting extensions to the standard is incredibly hard and time-consuming. You can, of course, commit to a single implementation, but there are only 2 or 3 implementations that are actively developed and trying to do something "new"; they are all experimental and incomplete. And while SBCL has quite a few nice features, its codebase was a bit hard for me to grok (read: days on end of banging my head against a wall, then giving up; I'm much more comfortable messing with Smalltalk bytecode compiler).
Secondly, GToolkit is driven by a group of coders[3] who have established a profitable consultancy. It's being developed around a single vision that the authors swear helps them achieve their projects' goals. Even if you're not sold on moldable development[4] as a methodology, GToolkit gives you all the tools you might need in a modern development and then adds tons of domain-specific tools and utilities that might be useful after just a bit of adjustment. The whole environment is built to make such adjustments as painless as possible, too.
The only problem I experience with GToolkit is the lack of structured, book-like documentation. The GToolkit book is closer to the PHP wiki or a bundle of tutorials than something like the Rust book. It's not that the docs aren't there - they often are, but finding them effectively is a bit challenging. I found that simply cloning all the repos that GT consists of (tens, if not hundreds, but there's automation for that) and pointing an agent to the directory with them is often enough to quickly find what I'm looking for (if the built-in Spotter fails and I'm too lazy to construct an in-image search).
TL;DR: GToolkit/Pharo move much faster than the CL world, and while the number of maintainers might be similar (honestly hard to estimate, though), the effort on the GT side is more centralized and, in my opinion, heads in a better direction. If you want an extensible, live, dynamic environment that shares many of Common Lisp's strengths, but is more polished, more actively maintained, and feels more modern, take a look at GToolkit.
(Disclaimer: no affiliation, just a programmer who likes to explore unpopular languages).
[1] https://gtoolkit.com/
[2] Some parts are implemented as Rust dynamic libraries and called from Smalltalk via FFI.
[3] https://feenk.com/
[4] https://moldabledevelopment.com/