this post was submitted on 31 Aug 2023
596 points (97.9% liked)
Technology
59152 readers
2010 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
How is "don't rely on content you have no right to use" litteraly impossible?
We teach to children that there is a Google filter to include only the CC images (that they should use for their presentations).
Also it's not like we are talking small companies here, a new billion-making industry is being born and it could totally afford contracts with big platforms that would allow to use their content.
This is an article about unlearning data, not about not consuming it in the first place.
LLM's are not storing learned data in it's raw, original form. They are injesting it and building an understanding of language based off of it.
Attempting to peel out that knowledge would be incredibly difficult, if not impossible because there's really no way to identify it.
And we're saying that if peeling out knowledge that someone has a right to have forgotten is difficult or impossible, that knowledge should not have been used to begin with. If enforcement means big tech companies have to throw out models because they used personal information without knowledge or consent, boo fucking hoo, let me find a Lilliputian to build a violin for me to play.
Okay I get it but that's a different argument. Starting fresh only gets you so far. Once am LLM exists and is exposed to the public users can submit any data they like and the LLM has no idea the source.
You could argue then that these models shouldn't be able to use user submitted data but that would be a devastating restriction to the technology and that starts to become a question of whatever we want this tech to exist at all.
A) this article isn't about a big tech company, it's about an academic researcher. B) he had consent to use the data when he trained the model. The participants later revoked their consent to have their data used.
At the time they used the data, they had a right to use it. The participants later revoked their consent for their data to be used, after the model was already trained at an enormous cost.
I have to admit my comment is not really relevant to the article itself (also, I read only the free part of it).
It was more a reaction to the comment above, which felt more generic. My concern about LLMs is that I could never find an auditable list of websites that were crawled, which would be reasonable to ask for, I think.
And the rest of the data Google has been viewing, cataloging and selling back to everyone for years, because they’re legally allowed to do so… you don’t see the irony in that?
Are they selling back scrapped content? I thought it was only user behaviors through the ad network?
About cataloging at least it is opt-out though robot.txt 🤷
EDIT: plus, "we are already doing bad" is never a good argument to continue doing bad, if Google were to be in fault this could get the traction to slap their ass
Google crawls the internet, archives entire actual photos, large snippets (at least) from every website it sees, aggregates it into a different form and serves it back to people for profit. It’s the same business model, different results with the processing of the data.
Google doesn't sell the data they collect... They sell ads and use their data to better target people with said ads. Third parties are paying google to target their ads to the right people.
You go to google because of the data they collected from the open internet. Peoples’ photos, articles they’ve written, books, etc. They aggregate it, process it and serve it back to you alongside ads. They also collect data about you and sell that as well. But no one would go to Google if they hadn’t aggregated, processed and repackaged the internet’s data.
No they don't. Why would they sell the data they use to target ads? If other corporations could just buy the data, they wouldn't need to pay google to target the ads, they'd just buy the data and do it themselves, Google isn't a data broker. They keep the data for them, it would be business suicide if they'd just sell all the data they collect.
https://www.eff.org/deeplinks/2020/03/google-says-it-doesnt-sell-your-data-heres-how-company-shares-monetizes-and
Because the question of what data one has the right to use is a very open legal question right now.
There is absolutely nothing illegal about a person examining publicly accessible artwork or text, learning from it, and attempting to reproduce a similar style. AIs are, in essence, doing basically the same thing. However, the sheer difference in time and scale may warrant a different legal treatment. That has not yet been settled, and it will probably take a fair amount of societal debate and new legislation before we have a definite answer.