Fuck AI

1408 readers

498 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

founded 8 months ago

MODERATORS

[email protected]

When A.I.’s Output Is a Threat to A.I. Itself (www.nytimes.com)

submitted 2 months ago by [email protected] to c/[email protected]

4 comments fedilink hide all child comments

I ran an AI startup back in 2017 and this was a huge deal for us and I’ve seen no actual improvement in this problem. NYTimes is spot on IMO

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 10 points 2 months ago (2 children)

This is a threat to LLMs, not AI itself. AI models looking for novel cures for diseases (for just one of many examples) are not trained on random Internet text.

[–] [email protected] 6 points 2 months ago* (last edited 2 months ago) (1 children)

This is a threat to any neural network that is being constantly trained. Hell it's even a problem with our brain's NN. We just call it "believing your own bullshit" or "getting high on your own supply".

The issue with NNs looking for cures or diseases (or anything that isn't trained off of the internet) is that they are basically out of training data. They'll need orders of magnitude more to get better and we just don't have that. We haven't figured out a way to get better off of less data and there's no real movement on that front either.

What we have right now is essentially a culmination of research that has been going on since the 1960's that was finally able to be realized with us figuring out:

We can map our NN variables to a matrix
We can use linear algebra to optimize the loss on that matrix
We can leverage video cards to crunch the linear algebra
We have the largest data set ever created in order to get our loss lower than ever before

[–] [email protected] -3 points 2 months ago* (last edited 2 months ago)

Completely irrelevant. The title and posted article are talking about unintentionally training LLM text generation models with prior output of other AI models. Not having enough training data for other types of models is a completely different problem and not what the article is about.

Nobody is going to "trawl the web for new data to train their next models” (to quote the article) for a model trying to cure diseases.

[–] [email protected] 5 points 2 months ago

AI labeled datasets have this to some extent too if they're not manually checked.