this post was submitted on 29 Mar 2024
341 points (93.4% liked)

Technology

59217 readers
2764 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

A Telegram user who advertises their services on Twitter will create an AI-generated pornographic image of anyone in the world for as little as $10 if users send them pictures of that person. Like many other Telegram communities and users producing nonconsensual AI-generated sexual images, this user creates fake nude images of celebrities, including images of minors in swimsuits, but is particularly notable because it plainly and openly shows one of the most severe harms of generative AI tools: easily creating nonconsensual pornography of ordinary people.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 56 points 7 months ago (9 children)

That's a ripoff. It costs them at most $0.1 to do simple stable diffusion img2img. And most people could do it themselves, they're purposefully exploiting people who aren't tech savvy.

[–] [email protected] 56 points 7 months ago* (last edited 7 months ago) (1 children)

I have no sympathy for the people who are being scammed here, I hope they lose hundreds to it. Making fake porn of somebody else without their consent, particularly that which could be mistaken for real if it were to be seen by others, is awful.

I wish everyone involved in this use of AI a very awful day.

[–] [email protected] 14 points 7 months ago (1 children)

Imagine hiring a hit man and then realizing he hired another hit man at half the price. I think the government should compensate them.

[–] [email protected] 15 points 7 months ago (1 children)
[–] [email protected] 5 points 7 months ago

Nested hit man scalpers taking advantage of overpaying client.

[–] [email protected] 45 points 7 months ago

it's a "I don't know tech" tax

[–] [email protected] 44 points 7 months ago (2 children)

The people being exploited are the ones who are the victims of this, not people who paid for it.

[–] [email protected] -5 points 7 months ago (2 children)

There are many victims, including the perpetrators.

[–] [email protected] 30 points 7 months ago (1 children)

It seems like there's a news story every month or two about a kid who kills themselves because videos of them are circulating. Or they're being blackmailed.

I have a really hard time thinking of the people who spend ten bucks making deep fakes of other people as victims.

[–] [email protected] 29 points 7 months ago

That's like 80% of the IT industry.

[–] [email protected] 25 points 7 months ago

IDK, $10 seems pretty reasonable to run a script for someone who doesn't want to. A lot of people have that type of arrangement for a job...

That said, I would absolutely never do this for someone, I'm not making nudes of a real person.

[–] [email protected] 14 points 7 months ago* (last edited 7 months ago)

Scam is another thing. Fuck these people selling.

But fuck dude they aren't taking advantage of anyone buying the service. That's not how the fucking world works. It turns out that even you have money you can post for people to do shit like clean your house or do an oil change.

NOBODY on that side of the equation are bring exploited 🤣

[–] [email protected] 3 points 7 months ago

In my experience with SD, getting images that aren't obviously "wrong" in some way takes multiple iterations with quite some time spent tuning prompts and parameters.

[–] [email protected] -1 points 7 months ago (1 children)

Wait? This is a tool built into stable diffusion?

In regards to people doing it themselves, it might be a bit too technical for some people to setup. But I’ve never tried stable diffusion.

[–] [email protected] 19 points 7 months ago* (last edited 7 months ago) (3 children)

It's not like deep fake pornography is "built in" but Stable Diffusion can take existing images and generate stuff based on it. That's kinda how it works really. The de-facto standard UI makes it pretty simple, even for someone who's not too tech savvy: https://github.com/AUTOMATIC1111/stable-diffusion-webui

[–] [email protected] 8 points 7 months ago

Img2img isn't always spot-on with what you want it to do, though. I was making extra pictures for my kid's bedtime books that we made together and it was really hit or miss. I've even goofed around with my own pictures to turn myself into various characters and it doesn't work out like you want it to much of the time. I can imagine it's the same when going for porn, where you'd need to do numerous iterations and tweaking over and over to get the right look/facsimile. There are tools/SD plugins like Roop which does make transferring over faces with img2img easier and more reliable, but even then it's still not perfect. I haven't messed around with it in several months, so maybe it's better and easier now.

[–] [email protected] 3 points 7 months ago (2 children)

Thanks for the link, I’ve been running some llm locally, and I have been interested in stable diffusion. I’m not sure I have the specs for it at the moment though.

[–] [email protected] 3 points 7 months ago

An iPhone from 2018 can run Stable Diffusion. You can probably run it on your computer. It just might not be very fast.

[–] [email protected] 2 points 7 months ago

By the way, if you’re interested in Stable Diffusion and it turns out your computer CAN’T handle it, there are sites that will let you toy around with it for free, like civitai. They host an enormous number of models and many of them work with the site’s built in generation.

Not quite as robust as running it locally, but worth trying out. And much faster than any of the ancient computers I own.

[–] [email protected] 3 points 7 months ago

It depends on the models you use too. There's specific training models data out there and all you need to do is give it a prompt of "naked" or something and it's scary good at making something realistic in 2 minutes. But yeah, there is a learning curve at setting everything up.

[–] [email protected] -5 points 7 months ago

And mechanics exploit people needing brake jobs. What's your point?