this post was submitted on 28 Jan 2025
871 points (94.4% liked)
memes
11276 readers
3228 users here now
Community rules
1. Be civil
No trolling, bigotry or other insulting / annoying behaviour
2. No politics
This is non-politics community. For political memes please go to [email protected]
3. No recent reposts
Check for reposts when posting a meme, you can only repost after 1 month
4. No bots
No bots without the express approval of the mods or the admins
5. No Spam/Ads
No advertisements or spam. This is an instance rule and the only way to live.
A collection of some classic Lemmy memes for your enjoyment
Sister communities
- [email protected] : Star Trek memes, chat and shitposts
- [email protected] : Lemmy Shitposts, anything and everything goes.
- [email protected] : Linux themed memes
- [email protected] : for those who love comic stories.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
How, without information on the dataset and the training code?
It's not hard. There's lots of tutorials out there.
Tutorials won't disclose the data used to train the model.
Yes. Wouldn't be a tutorial if it did.
So the models aren't opn source 🙄
Because the tutorials are on point?
I haven't seen any tutorials that include the training data. As you highlight, these would make for poor tutorials. If you know where there are more complete tutorials, I'd appreciate it if you could share them
Training code created by the community always pops up shortly after release. It has happened for every major model so far. Additionally you have never needed the original training dataset to continue training a model.
So, Ocarina of Time is considered open source now, since it's been decompiled by the community, or what?
Community effort and the ability to build on top of stuff doesn't make anything open source.
Also: initial training data is important.
So i am leaning as much as i can here, so bear with me. But it accepts tokenized data and structures it via a transformer as a json file or sun such. The weights are a binary file that’s separate and is used to, well, modify the tokenized data to generate outcomes. As long as you used a compatible tokenization structure, and weights structure, you could create a new training set. But that can be done with any LLM. You can’t pull the data from this just as you can’t make wheat from dissecting bread. But they provide the tools to set your own data, and the way the LLM handles that data is novel, due to being hamstrung by US sanctions. A “necessity is the mother of invention” and all that. Running comparable ai’s on inferior hardware and much smaller budget is what makes this one stand out, not the training data.
It's still not open source. No matter how extendable the weights are.
I mean, this does not help me understand.
https://slrpnk.net/comment/13455788
Edit: this one is a more thorough explanation: https://lemmy.ml/comment/16365208