Technology

Surely this is better than the crypto/NFT tech fad. At least there is some output from the generative AI that could be beneficial to the whole of humankind rather than lining a few people's pockets?

While the consumption for AI train can be large, there are arguments to be made for its net effect in the long run.

The article's last section gives a few examples that are interesting to me from an environmental perspective. Using smaller problem-specific models can have a large effect in reducing AI emissions, since their relation to model size is not linear. AI assistance can indeed increase worker productivity, which does not necessarily decrease emissions but we have to keep in mind that our bodies are pretty inefficient meat bags. Last but not least, AI literacy can lead to better legislation and regulation.

Theoretically we could slow down training and coast on fine-tuning existing models. Once the AI's trained they don't take that much energy to run.

Everyone was racing towards "bigger is better" because it worked up to GPT4, but word on the street is that raw training is giving diminishing returns so the massive spending on compute is just a waste now.

Is it me or is there something very facile and dull about Gartner charts? Thinking especially about the “””magic””” quadrants one (wow, you ranked competitors in some area along TWO axes!), but even this chart feels like such a mundane observation that it seems like frankly undeserved advertising for Gartner, again, given how little it actually says.

LLMs don't "know" anything. The true things they say are just as much bullshit as the falsehoods.

I work on LLM's for a big tech company. The misinformation on Lemmy is at best slightly disingenuous, and at worst people parroting falsehoods without knowing the facts. For that reason, take everything (even what I say) with a huge pinch of salt.

LLM's do NOT just parrot back falsehoods, otherwise the "best" model would just be the "best" data in the best fit. The best way to think about a LLM is as a huge conductor of data AND guiding expert services. The content is derived from trained data, but it will also hit hundreds of different services to get context, find real-time info, disambiguate, etc. A huge part of LLM work is getting your models to basically say "this feels right, but I need to find out more to be correct".

With that said, I think you're 100% right. Sadly, and I think I can speak for many companies here, knowing that you're right is hard to get right, and LLM's are probably right a lot in instances where the confidence in an answer is low. I would rather a LLM say "I can't verify this, but here is my best guess" or "here's a possible answer, let me go away and check".

I hate to break this to everyone who thinks that “AI” (LLM) is some sort of actual approximation of intelligence, but in reality, it’s just a fucking fancy ass parrot.

Our current “AI” doesn’t understand anything or have context, it’s just really good at guessing how to say what we want it to say… essentially in the same way that a parrot says “Polly wanna cracker.”

A parrot “talking to you” doesn’t know that Polly refers to itself or that a cracker is a specific type of food you are describing to it. If you were to ask it, “which hand was holding the cracker…?” it wouldn’t be able to answer the question… because it doesn’t fucking know what a hand is… or even the concept of playing a game or what a “question” even is.

It just knows that it makes it mouth, go “blah blah blah” in a very specific way, a human is more likely to give it a tasty treat… so it mushes its mouth parts around until its squawk becomes a sound that will elicit such a reward from the human in front of it… which is similar to how LLM “training models” work.

Oversimplification, but that’s basically it… a trillion-dollar power-grid-straining parrot.

And just like a parrot - the concept of “I don’t know” isn’t a thing it comprehends… because it’s a dumb fucking parrot.

The only thing the tech is good at… is mimicking.

It can “trace the lines” of any existing artist in history, and even blend their works, which is indeed how artists learn initially… but an LLM has nothing that can “inspire” it to create the art… because it’s just tracing the lines like a child would their favorite comic book character. That’s not art. It’s mimicry.

It can be used to transform your own voice to make you sound like most celebrities almost perfectly… it can make the mouth noises, but has no idea what it’s actually saying… like the parrot.

You get it?

Knowing the limits of your knowledge can itself require an advanced level of knowledge.

Sure, you can easily tell about some things, like if you know how to do brain surgery or if you can identify the colour red.

But what about the things you think you know but are wrong about?

Maybe your information is outdated, like you think you know who the leader of a country is but aren't aware that there was just an election.

Or maybe you were taught it one way in school but it was oversimplified to the point of being inaccurate (like thinking you can do physics calculations but end up treating everything as frictionless spheres in gravityless space because you didn't take the follow up class where the first thing they said was "take everything they taught you last year and throw it out").

Or maybe the area has since developed beyond what you thought were the limits. Like if someone wonders if they can hook their phone up to a monitor and another person takes one look at the phone and says, "it's impossible without a VGA port".

Or maybe applying knowledge from one thing to another due to a misunderstanding. Like overhearing a mathematician correcting a colleague that said "matrixes" with "matrices" and then telling people they should watch the Matrices movies.

Now consider that not only are AIs subject to these things themselves, but the information they are trained on is also subject to them and their training set may or may not be curated for that. And the sheer amount of data LLMs are trained on makes me think it would be difficult to even try to curate all that.

Edit: a word

How is a chatbot here better, faster, or more accurate than just a "return this" button on a web page? Chat bots like that take 10x the programming effort and actively make the user experience worse.

That has nothing to do with AI and is strictly a return policy matter. You can get a return in less than 2 minutes by speaking to a human at Home Depot.

Businesses choose to either prioritize customer experience, or not.

I like using it to assist me when I am coding.

It's on the falling edge of the hype curve. It's quite expected, and you're right about where it's headed. It can't do everything people want/expect but it can do some things really well. It'll find its niche and people will continue to refine it and find new uses, but it'll never be the threat/boon folks have been expecting.

People are using it for things it's not good at thinking it'll get better. And it has to an extent. It is technically very capable of writing prose or drawing pictures, but it lacks any semblance of artistry and it always will. I've seen trained elephants paint pictures, but they are interesting for the novelty, not for their expression. AI could be the impetus for more people to notice art and what makes good art special.

That sounds like dumping trash in the oceans so ships can't get through the trash islands easily anymore and become unable to transport more trashy goods. Kinda missing the forest for the trees here.

My shitposting will make AI dumber all on its own; feedback loop not required.

Alternatively, and possibly almost as useful, companies will end up training their AI to detect AI content so that they don't train on AI content. Which would in turn would give everyone a tool to filter out AI content. Personally, I really like the apps that poison images when they're uploaded to the internet.

NFTs anyone?