this post was submitted on 19 Jul 2024
124 points (81.0% liked)

Privacy

31991 readers
537 users here now

A place to discuss privacy and freedom in the digital world.

Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.

In this community everyone is welcome to post links and discuss topics related to privacy.

Some Rules

Related communities

Chat rooms

much thanks to @gary_host_laptop for the logo design :)

founded 5 years ago
MODERATORS
 

Fuck this shit, why does every fucking thing need an LLM?

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 53 points 4 months ago* (last edited 4 months ago) (1 children)

At least this one is open-source and quite privacy respecting

[–] [email protected] 28 points 4 months ago (2 children)

Yeah, and if that's the case, it seems like people just hate AI for the sake of it now.

LLM's are actually good at some things. Just not everything.

[–] [email protected] 25 points 4 months ago (1 children)

LLM's are actually good at some things.

Just look at the most recent ecological reports about it and combine them with the AI industry growth plans. You'll get an interesting perspective.

[–] [email protected] -1 points 4 months ago (1 children)

A lot of work has been going into making AIs more energy efficient, both in training and in inference stages. Electricity costs money, so obviously everyone's interested in more efficient AIs. That makes them more profitable.

[–] [email protected] -1 points 4 months ago (3 children)

Still you can't improve it that much. It's like blockchain. Computers always consume a lot of power, no matter how efficient they are.

[–] [email protected] 7 points 4 months ago (1 children)

Funny you should mention blockchains. Ethereum, the second-largest blockchain after Bitcoin, switched from proof-of-work to a proof-of-stake validation system two and a half years ago. That cut its energy use by 99.95%. The "blockchains are inherently a huge waste of energy" narrative is just firmly lodged in the popular view of them now, though, despite it being long proven false.

[–] [email protected] -2 points 4 months ago (1 children)

But that's really good! And also means that cloud based AI is even worse than blockchain in terms of environmental impact.

[–] [email protected] 6 points 4 months ago* (last edited 4 months ago)

It means that even if AI is having more environmental impact right now, there's no reason to say "you can't improve it that much." Maybe you can improve it. As I said previously, a lot of research is being done on exactly that - methods to train and run AIs much more cheaply than it has so far. I see developments along those lines being discussed all the time in AI forums such as /r/localllama.

Much like with blockchains, though, it's really popular to hate AI and "they waste enormous amounts of electricity" is an easy way to justify that. So news of such developments doesn't spread easily.

[–] [email protected] 3 points 4 months ago* (last edited 4 months ago)

You can improve it hugely. These things are very young.

There was a paper recently about removing the need for matrix multiplication from them which is a hugely expensive operation.

Dedicated hardware is also at a very early stage.

[–] [email protected] 2 points 4 months ago

That's simply not true, there are ways to drastically reduce energy usage while increasing efficiency by offloading the work. A company Mythic AI has worked on an analog processor which sifts through the model. On GPU's this is the power hungry process, for example a PC with the NVIDIA 3080 will typically run at about 350w under load.

Their claim now that these analog chips use 1/100th of the energy needed for GPU's. There's a video from Veritasium that goes over the details. It's genuinely effective, and that was a few years ago now before whatever potential growth they've made with their recent funding. It looks like they actually have products available for inquiry now too.

Doesn't seem to be at the consumer level yet unless you want to use servers for AI vs. your home computer, but it's progress. Here's the thing, I'm not particularly for our current implementation of AI but I don't think we should be entirely against all of it either. There are clearly plenty of benefits that people see from them, so giving any option possible for companies like Google to severely draw back their energy consumption seems like the reasonable path forward.

The independent drawbacks to LLMs and generative AI don't mean the technology will stop getting used. It isn't going anywhere (as in, people will use it) so making it more efficient is the obvious solution to mitigating more waste. Advocate for the prohibition of AI, but it's honestly more reckless than advocating for making the business' usage of AI reach a specific energy goal. Forcing these companies to retrofit their servers to run at something ridiculous like 30w per rack is beneficial for them and for us, as they won't pay as much for energy and we all will have less of it wasted.

Wishful thinking of course, but my point is that energy efficient AI, fortunately or unfortunately, exists and it will continue to. Like we can run "AI" on a raspberry pi 4 which takes what, 9 watts? This technology will get more developed every year, and while I'd be extremely surprised to see a Pi4 on its own running a subjectively useful LLM, I can imagine a setup that uses a Pi and some offloading tech to achieve reasonable results.

I'm personally pretty fine with regular people with computers wanting to use AI in whatever way suits them, as long as they aren't trying to sell the results. While the energy consumption isn't ideal, it's a droplet to the servers these companies take. We should definitely make every effort possible towards increasing the efficiency of this tech, if only because it seems insane to me to pretend like AI will just disappear, or let this huge energy suck exist as we hope it begins to fade.

Tl;Dr offload GPU resources to analog chips, force companies to be more efficient simply because hoping AI is going to disappear is reckless.

[–] [email protected] 7 points 4 months ago

They're really good at burning hug amounts of electricity.