Computers are a fundamental part of that process in modern times.

If you were taking a test to assess how much weight you could lift, and you got a robot to lift 2,000 lbs for you, saying you should pass for lifting 2000 lbs would be stupid. The argument wouldn't make sense. Why? Because the same exact logic applies. The test is to assess you, not the machine.

Just because computers exist, can do things, and are available to you, doesn't mean that anything to assess your capabilities can now just assess the best available technology instead of you.

Like spell check? Or grammar check?

Spell/Grammar check doesn't generate large parts of a paper, it refines what you already wrote, by simply rephrasing or fixing typos. If I write a paragraph of text and run it through spell & grammar check, the most you'd get is a paper without spelling errors, and maybe a couple different phrases used to link some words together.

If I asked an LLM to write a paragraph of text about a particular topic, even if I gave it some references of what I knew, I'd likely get a paper written entirely differently from my original mental picture of it, that might include more or less information than I'd intended, with different turns of phrase than I'd use, and no cohesion with whatever I might generate later in a different session with the LLM.

These are not even remotely comparable.

Assuming the point is how well someone conveys information, then wouldn’t many people better be better at conveying info by using machines as much as reasonable? Why should they be punished for this? Or forced to pretend that they’re not using machines their whole lives?

This is an interesting question, but I think it mistakes a replacement for a tool on a fundamental level.

I use LLMs from time to time to better explain a concept to myself, or to get ideas for how to rephrase some text I'm writing. But if I used the LLM all the time, for all my work, then me being there is sort of pointless.

Because, the thing is, most LLMs aren't used in a way that conveys info you already know. They primarily operate by simply regurgitating existing information (rather, associations between words) within their model weights. You don't easily draw out any new insights, perspectives, or content, from something that doesn't have the capability to do so.

On top of that, let's use a simple analogy. Let's say I'm in charge of calculating the math required for a rocket launch. I designate all the work to an automated calculator, which does all the work for me. I don't know math, since I've used a calculator for all math all my life, but the calculator should know.

I am incapable of ever checking, proofreading, or even conceptualizing the output.

If asked about the calculations, I can provide no answer. If they don't work out, I have no clue why. And if I ever want to compute something more complicated than the calculator can, I can't, because I don't even know what the calculator does. I have to then learn everything it knows, before I can exceed its capabilities.

We've always used technology to augment human capabilities, but replacing them often just means we can't progress as easily in the long-term.

Short-term, sure, these papers could be written and replaced by an LLM. Long-term, nobody knows how to write papers. If nobody knows how to properly convey information, where does an LLM get its training data on modern information? How do you properly explain to it what you want? How do you proofread the output?

If you entirely replace human work with that of a machine, you also lose the ability to truly understand, check, and build upon the very thing that replaced you.