242
this post was submitted on 03 Dec 2024
242 points (97.6% liked)
Technology
60090 readers
1796 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If they double up the VRAM with a 24GB card, this would be great for a "self hosted LLM" home server.
3060, 3090 prices have been rising like crazy because Nvidia is vram gouging and AMD inexplicably refuses to compete. Even ancient P40s (double vram 1080 TIs with no display) are getting expensive. 16GB on the A770 is kinda meager, but 24GB is the point where you can fit the Qwen 2.5 32B models that are starting to perform like the big corporate API ones.
And if they could fit 48GB with new ICs... Well, it would sell like mad.
I always wondered who they were making those mid- and low-end cards with a ridiculous amount of VRAM for... It was you.
All this time I thought they were scam cards to fool people who believe that bigger number always = better.
Yeah, AMD and Intel should be running high VRAM SKUs for hobbyists. I doubt it'll cost them that much to double the RAM, and they could mark them up a bit.
I'd buy the B580 if it had 24GB RAM, at 12GB, I'll probably give it a pass because my 6650 XT is still fine.
Don’t you need nvidia cards to run ai stuff?
Nah, ollama works w/ AMD just fine, just need a model w/ enough VRAM.
I'm guessing someone would get Intel to work as well if they had enough VRAM.
Not at all