Office space meme:
"If y'all could stop calling an LLM "open source" just because they published the weights... that would be great."
this post was submitted on 28 Jan 2025
875 points (94.4% liked)
memes
11276 readers
2646 users here now
Community rules
1. Be civil
No trolling, bigotry or other insulting / annoying behaviour
2. No politics
This is non-politics community. For political memes please go to [email protected]
3. No recent reposts
Check for reposts when posting a meme, you can only repost after 1 month
4. No bots
No bots without the express approval of the mods or the admins
5. No Spam/Ads
No advertisements or spam. This is an instance rule and the only way to live.
A collection of some classic Lemmy memes for your enjoyment
Sister communities
- [email protected] : Star Trek memes, chat and shitposts
- [email protected] : Lemmy Shitposts, anything and everything goes.
- [email protected] : Linux themed memes
- [email protected] : for those who love comic stories.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Open source means you can recreate the binaries yourself. Neiter Facebook. Nor the devs of deepseek published which training data they used, nor their training algorithm.
They published the source code needed run the model. It’s open source in the way that anyone can download the model, run it locally, and further build on it.
Training from scratch costs millions.
Open source isn't really applicable to LLM models IMO.
There is open weights (the model), and available training data, and other nuances.
They actually went a step further and provided a very thorough breakdown of the training process, which does mean others could similarly train models from scratch with their own training data. HuggingFace seems to be doing just that as well. https://huggingface.co/blog/open-r1
Edit: see the comment below by BakedCatboy for a more indepth explanation and correction of a misconception I've made
It's worth noting that OpenR1 have themselves said that DeepSeek didn't release any code for training the models, nor any of the crucial hyperparameters used. So even if you did have suitable training data, you wouldn't be able to replicate it without re-discovering what they did.
OSI specifically makes a carve-out that allows models to be considered "open source" under their open source AI definition without providing the training data, so when it comes to AI, open source is really about providing the code that kicks off training, checkpoints if used, and details about training data curation so that a comparable dataset can be compiled for replicating the results.
Thanks for the correction and clarification! I just assumed from the open-r1 post that they gave everything aside from the training data.
Yeah, but not to train it
Yeah, it's about as open source as binary blobs.
So what? You still can gleam something if you know the dataset on which the model has been trained.
If software is hard to compile, can you keep the source code closed and still call software "open source"?
I agree the bad part is that they didn’t provide the script to train the model from scratch.
This is a great starting point for further improvements of the model. Most AI research is done with pretrained weights used as basis. Few are training models completely from scratch. The model is built with Torch, so anyone should be able to fine tune the model on custom data sets.
The runner is open source, the model is not
The service uses both so calling their service open source gives a false impression to 99,99% of users that don't know better.
The model is as far as I know open, even for commercial use. This is in stark contrast with Meta’s models, which have (or had?) a bespoke community license restricting commercial use.
Or is there anything that can’t be done with the DeepSeek model that I’m unaware of?
The model is open, it's not open source!
How is it so hard to understand? The complete source of the model is not open. It's not a hard concept.
Sorry if I'm coming of as rude but I'm getting increasingly frustrated at having to explain a simple combination of two words that is pretty self explanatory.
Ok I understand now why people are upset. There’s a disagreement with terminology.
The source code for the model is open source. It’s defined in PyTorch. The source code for it is available with the MIT license. Anyone can download it and do whatever they want with it.
The weights for the model are open, but it’s not open source, as it’s not source code (or an executable binary for that matter). No one is arguing that the model weights are open source, but there seem to be an argument against that the model is open source.
And even if they provided the source code for the training script (and all its data), it’s unlikely anyone would reproduce the same model weights due to randomness involved. Training model weights is not like compiling an executable, because you’ll get different results every time.
Hey, I have trained several models in pytorch, darknet, tensorflow.
With the same dataset and the same training parameters, the same final iteration of training actually does return the same weights. There's no randomness unless they specifically add random layers and that's not really a good idea with RNNs it wasn't when I was working with them at least. In any case, weights should converge into a very similar point even if randomness is introduced or else the RNN is pretty much worthless.
There’s usually randomness involved with the initial weights and the order the data is processed.
Not enough for it to make results diverge. Randomness is added to avoid falling into local maximas in optimization. You should still end in the same global maxima. Models usualy run until their optimization converges.
As stated, if the randomness is big enough that multiple reruns end up with different weights aka optimized for different maximas, the randomization is trash. Anything worth their salt won't have randomization big enough.
So, going back to my initial point, we need the training data to validate the weights. There are ways to check the performance of a model (quite literally, the same algorithm that is used to evaluate weights in training is them used to evaluate the trained weights post training) the performance should be identical up to a very small rounding error if a rerun with the same data and parameters is used.
A software analogy:
Someone designs a compiler, makes it open source. Make an open runtime for it. 'Obtain' some source code with unclear license. Compiles it with the compiler and releases the compiled byte code that can run with the runtime on free OS. Do you call the program open source? Definitely it is more open than something that requires proprietary inside use only compiler and closed runtine and sometimes you can't access even the binary; it runs on their servers. It depends on perspective.
ps: the compiler takes ages and costs mils in hardware.
edit: typo
I think a more appropriate analogy is if you make an open source game. With the game you have made textures, because what is a game without textured surfaces? You include the binary jpeg images along with the source code.
You’ve made the textures with photoshop, which is a closed source application. The textures also features elements of stock photos. You don’t provide the original stock photos.
Anyone playing the game is free to replace the textures with their own. The game will have a different feel, but it’s still a playable game. Anyone is also free to modify the existing textures.
Would you consider this game closed source?
Would an open-source Windows installer make it open-source? After all, you can replace its .dll files and modify the registry. I guess PrismLauncher also makes Minecraft open-source, you can replace the textures there as well.
If the installer is open source, then that part is open source. It’s maybe not as useful, because it relies on proprietary software to work. On the other hand, so does emulators like Dolphin.
Windows is not open source just because it’s possible to change dll files. Minecraft is not open source just because it’s possible to modify its textures.
Model weights isn’t the equivalent to a proprietary DLL or GameCube ROM. Anyone is free to modify and distribute the model weights however they like - and people are already doing it. Soon enough we will see variations of the model without the Chinese censor for example.
I'm going to take your point to the extreme.
It's only open source if the camera that took the picture that is used in the stock image that was used to create the texture is open source.
You used a fully mechanical camera and chemical flash powder? Better publish that design patent and include the chemistry of the flash powder!
And looking at mobile games like Tacticus, there are loads of people with millions to burn on hobbies
Eh, it seems like it fits to me. We casually refer to all manner of data as "open source" even if we lack the ability to specifically recreate it. It might be technically more accurate to say "open data" but we usually don't, so I can't be too mad at these folks for also not.
There's huge deaths of USGS data that's shared as open data that I absolutely cannot ever replicate.
If we're specifically saying that open source means you can recreate the binaries, then data is fundamentally not able to be open source, since it distinctly lacks any form of executable content.
lol, are you claiming data isn't reproducable? XD
... Did you not read the litteral next phrase in the sentence?
Your definition of open source specified reproducible binaries. From context it's clear that I took issue with your definition, not with the the notion of reproducing data.
Ok, then my definition givenwas too narrow, when I said "reproducable binaries". If data claims to be "open source", then it needs to supply information on how to reproduce it.
Open data has other criteria, I'm sure.
Yes, that's what I said when you opted to take the first half of a sentence out of context.
https://en.wikipedia.org/wiki/Open_data
The common usage of open data is just that it's freely shareable.
Like I said in my initial comment, people frequently use "open source" to refer to it, but it's such a pervasive error that it hardly worth getting too caught up on and practically doesn't count as an error anymore.
Some open data can't be reproduced by anyone who has access to the data.
I was specifically addressing the use of the phrase "open source". And the term "open data" doesn't apply either, since it's not a dataset that's distributed, but rather weights of an LLM with data baked into it. That's neither open source nor open data.