From TFA: "Eric Schwitzgebel writes that . . . There’s a huge cognitive difference between nodding along while reading something and actually productively generating a text. Two reasons: First, once the text is on the page, it’s easy to passively let the approximate word suffice, rather than thinking about word choice in the same effortful, active way we do when generating prose de novo. Second, as I suggested above, I doubt that human beings, even experts, have a good sense of all the factors that shape word choice -- everything they’re being sensitive to. You would have phrased it slightly differently, and even if you don’t know that, or why, a different signal is sent and received."
This is the best articulation I've seen of why simply reviewing and copy-editing does not provide remotely the same value as writing from scratch. I spent a considerable amount of time over the past two weeks reviewing and improving a work document that was the output of an LLM. Given the number of people involved and the final level of effort, I'm firmly convinced that writing it manually would have been faster and resulted in a higher quality product. Getting the wording right matters.
While I mostly agree with you, what is the real difference between an editor saying, 'when generating prose de novo' sounds pretentious to an American audience so let's use 'when writing from scratch' instead, versus an LLM giving the same advice?
Not that it's necessarily better but it's the kind of thing an editor might pick up and so why would you reject the advice if it's a machine giving it rather than a human?
Use AI to write things for you to read that you wish someone else had written. ‘Give me a summary of the research on this topic’; ‘Write a report on this data to help me make a decision’. ‘Take this transcript of a meeting and write me the email it could have been’.
Don’t use AI to write things that you are producing for someone else to consume.
I am someone else, if AI can write something that I want to and can consume then it can do the same for other’s consumption. Perhaps a disclaimer would be nice, “co-written with ____” but I see no difference between the two though of course I understand what you are trying to convey conceptually
There's a difference between me getting AI to write something I want to read, and getting it to write something you want to read. The difference is that I know me, but I don't know you.
The other difference is that, if I get AI to write something for me, I expect something AI-written. If I am writing something for you, you expect something that I wrote, not something that AI wrote.
Instead of asking AI to write something substansive for you, ask it to critique your writing. Then use your own judgement as to which bits of advice to follow and which to ignore.
The LLM will always give you a full rewrite — don't use it. It always does too much, and persuading it to tone it down is a constant battle.
"AI writing is vague and wrong in hard-to-notice ways"
This one felt to me, at work I lost a lot of time re-reading and correcting AI prose that was launched to "summarize" a collective white paper, and where subtlety and nuance disappeared in ways that harmed the initial ideas/arguments I was writing about, but in sneaky ways.
AI is very bad at properly handling statements that make heavy use of vague quantifiers (e.g. "some", "most") and also commits a lot of pretty serious logical fallacies. It is also bad at handling subtle logical negation, generally.
One of the most egregious negation issues I run into a lot is when I (or someone) makes a statement of the form: "not X" or "X is thus not true", and the AI then proceeds to interpret or summarize this as 'whatever is the opposite of X is the case'". This will cause it to go down a useless path investigating or disputing the opposite of X, which generally has no relevance or bearing on anything.
It also often very harmfully will replace your carefully chosen words with weirdly specific academic operationalizations or formalisms, then again waste huge amounts of text refuting / showing "problems" that result from that formalism, all of which again have no bearing or relevance on the original statement. An example would be you saying something like "intelligence, generally, must surely explain some of the differences in X", and then it will go "actually IQ does not correlate with X", unless you specifically tell it not to conflate psychometric IQ with intelligence generally.
Sometimes this is helpful, but the more specific / technical the domain, the more often you specifically have to prevent it from going down stupid paths that should be obvious given the expert context and wording, because it can seem almost hungry to try to catch you in some kind of insipid 'gotcha'. Much of these issues often clearly arise immediately from the first-pass "reword what the user said" part, given the reasoning traces.
This advice is too coarse. I believe LLMs have made me a better writer of technical reports. I have a keen ear for language, and I can usually tell when a sentence or a paragraph is orbiting around a point rather than landing on it firmly. Some writers insist that unclear prose is dispositive proof of unclear ideas -- a sign the writer has not thought things through. My experience is different: I often have a clear idea in mind that I cannot quite find the right idiom to express. Sometimes I can find it if I keep revising, but staring at a sentence too long can entangle you in a particular linguistic frame that can be hard to escape.
LLMs are helpful because their idiom-list is vast and they are indefatigable and infinitely patient. You can keep iterating on a sentence and it will keep giving you fresh takes. Eventually, whether through careful steering or brute-force iteration, it will come up with something that has the right resonance. Like a word on the tip of your tongue, you recognize it when you hear it.
For me, the end result of this writing-process is often a document where much of the language first came from the agent, but the voice is recognizably mine, and I feel a clear sense of authorship. It is still my work because of the microscopic attention I paid to every word. The marble is the agent's, but the chisel and mallet are in my hands.
> I think so because (1) the writing process is an essential part of the thinking process, (2) AI writing is vague and wrong in hard-to-notice ways, and (3) writing with AI (and not labeling it as such) is rude and misleading. I’ll explain these points in more detail below, but first, a few throat clearings.
1. Agree writing process is essential, but just getting your thoughts down is the start.
2. Human writing is very often vague and wrong in hard to notice ways, but I would say it's more likely to be wrong in easy to notice ways, which is ... better?
3. I think it's rude to not use the best tools to convey the message most clearly and most respectfully of my time. I often use AI to help me write emails and nearly every time it provides more concise well structured versions of what I have to say. 99% of the time I'm just trying to convey a message clearly, that's all. When I write my draft there is often words I can delete or phrases that I can shorten. It's hard to spot but when you have AI point them out it's obvious. This is the role of an editor (check out Stephen King book On Writing which makes this point). Unfortunately I don't have an editor, but I have AI that helps trim the fat, get to the point and save my readers time.
If these tools are available to you and you don't use them, then I think that would be disrespectful. I don't take any solace thinking how much time and effort went into what someone wrote me. If I could save them time and have AI write or edit, and save me time reading it since it would be more concise and better structured, it's a win-win.
Just a couple of days ago, a customer rep at a company we partner with dropped a long doc that had basic reasoning errors once you got into the individual sections.
I would have much preferred if the rep had written a shorter document and reached out about things they were uncertain about. Instead thet wasted my and my team's time as we first tried to make sense of the document on good faith before realizing the problem.
I don't care if people use LLMs per se, but if this is functionally the result, then in this aspect of my work things would go much better if people did not use them. They are a great interface for talking to a machine, but due to the laziness they breed, they are a terrible interface for talking to other humans unless approached with a great deal of discipline.
Yes, you hit on the main problem with LLMs for writing. It doesn't cost you anything. It can do the reasoning for you and creating long messaging with dodgy reasoning.
I think you should write down your thoughts, tell it to make the points you spelled out and be concise and respectful of the reader's time and attention. When I do this, it works very well. But I agree if you just have it respond and reason it tends to often produces bad responses.
I think your point on “cutting down on words”, so to speak, is relative to personal writing style. I find that AI tends to write far more words than I personally do, which is sometimes useful, but in the opposite direction that it works for you.
And also to think in different directions than you would have gone in isolation. Regardless, this is a great heuristic / rule that I will be sharing and keeping in mind.
I can't help but think that some of the arguments here apply equally well to writing _code_ as they to to other substantive writings.
Writing code is thinking, AI code is often vague and wrong in hard-to-notice ways, and the (human) reader of code is the one that pays the cost for this later.
The cost/benefit analysis may still work out differently for code though...
I never use AI for writing but do so regularly for code. They are quite different to me.
My reason: code can be checked objectively. I can run it and confirm it works. I don't get attached to it. I don't feel pride in it (even when I write it by hand). Code just is. It's lifeless, inert, and entirely replaceable.
How do I do the equivalent for prose? How can I tell if my words "work"? Do they clearly convey my ideas to the intended audience? There's an element of subjectivity here forces me to identify personally with the prose.
Code has no such personality. I don't tie my identity or ego to code the same way I would an essay.
I don't find AI helpful for truly constructive or creative prose, it devolves to the median output very quickly and it's almost painful to hear in my inner voice when I read out an AI-ified idea. The "not this but that" formulation is especially bad or cloying attempt at pretending to refine an idea or lightly correct something to start from a position of authority, very annoying.
That said, I have a lot of situations at work where I am asked to simplify something I am an expert in or convey something for a different audience, particularly as I prepare for presentations and I do find it helpful in helping me step down my writing or work. YMMV.
I also skip words a lot and can miss that even after 2-3 editorial passes and it's very helpful at that vs. say normal spellcheck.
Communication between two people is hard enough. Involving another actor as an intermediary rarely helps. It quickly becomes a game of telephone. Doubly so if the intermediary is an LLM.
I made astra write me a short book (40-50 pages) on Syberian cats collecting different kind of info from the owners on the Internet. Unlike me it could mine the data in multiple languages which I specifically asked for. The book has parallel version with references and more nuance.
Is it a great book? It's meh, but not necessarily below average of what you can find in bookstore or a library. But it didn't eat that much of my subscription, took like 2 days, and saved me a lot of time.
I think about this a lot. I started my teenage-to-young-adult life in the literary world as a poet who loved programming, and made a living as a ghostwriter. At some point, I fell in love with mathematics and wound up in ML in research/engineering for the last 8 years or so. So, I've thought a lot about writing and ML and their intersection.
I think one of the underappreciated things about writing "substantive" work is that the work you see at the end isn't the first attempt. And I don't mean the first draft of the piece. I mean that almost always, writers iterate on the same topic many times, either with complete published pieces or abandoned drafts or even just conversations and sessions of unproductive daydreaming. It's a cliche that your best work typically also comes out fastest, but it's not because of divine inspiration, it's because you've whittled the big idea you actually care about down so much in your mind that you instinctively know exactly how to write it.
My experience has been that for people who don't work this way or don't write a lot, LLMs can give them this incredible feeling of leaping straight from inkling to "substantive" writing. And because they haven't built up those muscles or "taste", they don't immediately recognize that it's imprecise and hard to follow.
That's not to say they're not brilliant in their own right, just that they haven't spent a lot of time on this particular thing. Sort of like a very gifted programmer who doesn't have a ton of experience yet (speaking as someone who is gifted at nothing and frequently has to do things they're inexperienced at).
So I don't think the problem is that LLMs just write bad. It's that LLMs are so wonderfully powerful that they allow you to confidently leap forward to create something that is a little beyond your experience.
And that's why I have different reactions to heavily AI generated writing. When it feels like marketing at scale, it grosses me out. But when I feel like it's just someone who is excited to write an idea and maybe doesn't have a lot of experience doing it, I'm not judgemental. My hope is that it makes them more excited about writing, and that trying to make their next piece better will lead them inevitably to start thinking about where the last piece fell short. And if there's some slop along the way, eh, I'm not compelled to read it.
Writing is pretty hard for a lot of people, maybe especially so if they are more non-verbal thinkers, and then doubly so again if one must write not in one's native language.
This is the best articulation I've seen of why simply reviewing and copy-editing does not provide remotely the same value as writing from scratch. I spent a considerable amount of time over the past two weeks reviewing and improving a work document that was the output of an LLM. Given the number of people involved and the final level of effort, I'm firmly convinced that writing it manually would have been faster and resulted in a higher quality product. Getting the wording right matters.
Not that it's necessarily better but it's the kind of thing an editor might pick up and so why would you reject the advice if it's a machine giving it rather than a human?
Don’t use AI to write things that you are producing for someone else to consume.
Elsewhere it’s painful and irksome when you didn’t ask for it
The other difference is that, if I get AI to write something for me, I expect something AI-written. If I am writing something for you, you expect something that I wrote, not something that AI wrote.
The LLM will always give you a full rewrite — don't use it. It always does too much, and persuading it to tone it down is a constant battle.
One of the most egregious negation issues I run into a lot is when I (or someone) makes a statement of the form: "not X" or "X is thus not true", and the AI then proceeds to interpret or summarize this as 'whatever is the opposite of X is the case'". This will cause it to go down a useless path investigating or disputing the opposite of X, which generally has no relevance or bearing on anything.
It also often very harmfully will replace your carefully chosen words with weirdly specific academic operationalizations or formalisms, then again waste huge amounts of text refuting / showing "problems" that result from that formalism, all of which again have no bearing or relevance on the original statement. An example would be you saying something like "intelligence, generally, must surely explain some of the differences in X", and then it will go "actually IQ does not correlate with X", unless you specifically tell it not to conflate psychometric IQ with intelligence generally.
Sometimes this is helpful, but the more specific / technical the domain, the more often you specifically have to prevent it from going down stupid paths that should be obvious given the expert context and wording, because it can seem almost hungry to try to catch you in some kind of insipid 'gotcha'. Much of these issues often clearly arise immediately from the first-pass "reword what the user said" part, given the reasoning traces.
LLMs are helpful because their idiom-list is vast and they are indefatigable and infinitely patient. You can keep iterating on a sentence and it will keep giving you fresh takes. Eventually, whether through careful steering or brute-force iteration, it will come up with something that has the right resonance. Like a word on the tip of your tongue, you recognize it when you hear it.
For me, the end result of this writing-process is often a document where much of the language first came from the agent, but the voice is recognizably mine, and I feel a clear sense of authorship. It is still my work because of the microscopic attention I paid to every word. The marble is the agent's, but the chisel and mallet are in my hands.
1. Agree writing process is essential, but just getting your thoughts down is the start.
2. Human writing is very often vague and wrong in hard to notice ways, but I would say it's more likely to be wrong in easy to notice ways, which is ... better?
3. I think it's rude to not use the best tools to convey the message most clearly and most respectfully of my time. I often use AI to help me write emails and nearly every time it provides more concise well structured versions of what I have to say. 99% of the time I'm just trying to convey a message clearly, that's all. When I write my draft there is often words I can delete or phrases that I can shorten. It's hard to spot but when you have AI point them out it's obvious. This is the role of an editor (check out Stephen King book On Writing which makes this point). Unfortunately I don't have an editor, but I have AI that helps trim the fat, get to the point and save my readers time.
If these tools are available to you and you don't use them, then I think that would be disrespectful. I don't take any solace thinking how much time and effort went into what someone wrote me. If I could save them time and have AI write or edit, and save me time reading it since it would be more concise and better structured, it's a win-win.
I would have much preferred if the rep had written a shorter document and reached out about things they were uncertain about. Instead thet wasted my and my team's time as we first tried to make sense of the document on good faith before realizing the problem.
I don't care if people use LLMs per se, but if this is functionally the result, then in this aspect of my work things would go much better if people did not use them. They are a great interface for talking to a machine, but due to the laziness they breed, they are a terrible interface for talking to other humans unless approached with a great deal of discipline.
I think you should write down your thoughts, tell it to make the points you spelled out and be concise and respectful of the reader's time and attention. When I do this, it works very well. But I agree if you just have it respond and reason it tends to often produces bad responses.
I think “cutting down on words” depends on personal writing style. AI tends to write more than I would, so I usually have the opposite problem.
Only ever use AI to make yourself think harder, and more.
Writing code is thinking, AI code is often vague and wrong in hard-to-notice ways, and the (human) reader of code is the one that pays the cost for this later.
The cost/benefit analysis may still work out differently for code though...
My reason: code can be checked objectively. I can run it and confirm it works. I don't get attached to it. I don't feel pride in it (even when I write it by hand). Code just is. It's lifeless, inert, and entirely replaceable.
How do I do the equivalent for prose? How can I tell if my words "work"? Do they clearly convey my ideas to the intended audience? There's an element of subjectivity here forces me to identify personally with the prose.
Code has no such personality. I don't tie my identity or ego to code the same way I would an essay.
That said, I have a lot of situations at work where I am asked to simplify something I am an expert in or convey something for a different audience, particularly as I prepare for presentations and I do find it helpful in helping me step down my writing or work. YMMV.
I also skip words a lot and can miss that even after 2-3 editorial passes and it's very helpful at that vs. say normal spellcheck.
Is it a great book? It's meh, but not necessarily below average of what you can find in bookstore or a library. But it didn't eat that much of my subscription, took like 2 days, and saved me a lot of time.
Long story short, context matters.
I think one of the underappreciated things about writing "substantive" work is that the work you see at the end isn't the first attempt. And I don't mean the first draft of the piece. I mean that almost always, writers iterate on the same topic many times, either with complete published pieces or abandoned drafts or even just conversations and sessions of unproductive daydreaming. It's a cliche that your best work typically also comes out fastest, but it's not because of divine inspiration, it's because you've whittled the big idea you actually care about down so much in your mind that you instinctively know exactly how to write it.
My experience has been that for people who don't work this way or don't write a lot, LLMs can give them this incredible feeling of leaping straight from inkling to "substantive" writing. And because they haven't built up those muscles or "taste", they don't immediately recognize that it's imprecise and hard to follow.
That's not to say they're not brilliant in their own right, just that they haven't spent a lot of time on this particular thing. Sort of like a very gifted programmer who doesn't have a ton of experience yet (speaking as someone who is gifted at nothing and frequently has to do things they're inexperienced at).
So I don't think the problem is that LLMs just write bad. It's that LLMs are so wonderfully powerful that they allow you to confidently leap forward to create something that is a little beyond your experience.
And that's why I have different reactions to heavily AI generated writing. When it feels like marketing at scale, it grosses me out. But when I feel like it's just someone who is excited to write an idea and maybe doesn't have a lot of experience doing it, I'm not judgemental. My hope is that it makes them more excited about writing, and that trying to make their next piece better will lead them inevitably to start thinking about where the last piece fell short. And if there's some slop along the way, eh, I'm not compelled to read it.