Ok but how is anyone meant to know if you generated your docstrings using copilot?
this post was submitted on 16 May 2024
367 points (97.9% liked)
Technology
59133 readers
2298 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
How do they know that you wrote it yourself and didn't just steal it?
This is a rule to protect themselves. If there is ever a case around this, they can push the blame to the person that committed the code for breaking that rule.
This is the only reason rules exist, not to stop people doing a thing but to be able to enforce or defect responsibility when they do.
I mean, generally rules at least are to strongly discourage people from doing a thing, or to lead to things that WOULD prevent people from doing a thing.
A purely conceptual rule by itself would not magically stop someone from doing a thing, but that's kind of a weird way to think about it.
They'll use AI to detect it.... obviously. ☺️
I'm saddened to use this phrase but it is literally virtue signalling. They have no way of knowing lmao
It’s actually simple to detect: if the code sucks or is written by a bad programmer, and the docstrings are perfect, it’s AI. I’ve seen this more than once and it never fails.
I'm confused, do people really use copilot to write the whole thing and ship it without re reading?
I literally did an interview that went like this:
- Applicant used copilot to generate nontrivial amounts of the code
- Copilot generated the wrong code for a key part of the algorithm; applicant didn't notice
- We pointed it out, they fixed it
- They had to refactor the code a bit, and ended up making the same exact mistake again
- We pointed out the error again...
And that's in an interview, where you should be extra careful to make a good impression...
Not specific to AI but someone flat out told me they didn't even run the code to see it work. They didn't understand why I would or expect that before accepting code. This was someone submitting code to a widely deployed open source project.
So, I would expect the answer is yes or very soon to be yes.
Around me, most beginners who use that don't have the skills to understand or even test what they get. They don't want to learn I guess, ChatGPT is easier.
I recently suspected a new guy was using ChatGPT because everything seemed perfect (grammar, code formatting, classes made with design patterns, etc.) but the code was very wrong. So I did some pair programming with him and asked if we could debug his simple application. He didn't know where the debug button was.
Guilty as charged, ten years into the job and I never learned to use a debugger lol.
Seriously though that's amazing to me I never met one of those... I guess 95% of them will churn out of the industry in less than five years...
Debug button? There is a button that inserts 'printf("%s:%s boop! \n" , __FUNCTION__, __LINE__) ;'?
So your results are biased, because you're not going to see the decent programmers who are just using it to take mundane tasks off their back (like generating boilerplate functions) while staying in control of the logic. You're only ever going to catch the noobs trying to cheat without fully understanding what it is they're doing.
You're only ever going to catch the noobs.
That’s the fucking point. Juniors must learn, not copy paste random stuff. I don’t care what seniors do.
It's also probably to make things slightly simpler from a legal perspective.
That makes sense yes
Are they long, super verbose and often incorrect?
Magic, I guess ?
Because they'll be shit?
Docstrings based on the method signature and literal contents of a method or class are completely pointless, and that's all copilot can do. It can't Intuit anything that docstrings are actually there for.
Definitely not my experience. With a well structured code base it can be pretty uncanny. I think it's context is limited to files that are currently opened in the editor, so that may be your issue if you're coding with just one file open?
GitHub Copilot introduced a new keyword a little while ago, "@workspace", where it can see everything in your project. The code it generates uses all your own functions and variables in your libraries and it figures out how to use them correctly.
There was one time where I totally went "WTF", because it spat out Python. In a C++ project. But those kind of hallucinations are getting more and more rare. The more code you write, the better it gets. It really does become sort of like a "Copilot", sitting there coding alongside you. The mistake people make is assuming it's going to come up with ideas and algorithms for them without spending any mental energy at all.
I'm not trying to shill. I'm not a programmer by trade. Just a hobbyist who started on QBasic in the ancient times. But I've been trying to learn it off and on for the past 30 years, and I've never learned so much and had so much fun as in the last 1.5 with AI help. I can just think of stuff to do, and shit will just flow out now.
Lots of stupid people asking "how would they know?"
That's not the fucking point. The point is that if they catch you they can block future commits and review your past commits for poor quality code. They're setting a quality standard, and establishing consequences for violating it.
If your AI generated code isn't setting off red flags, you're probably fine, but if something stupid slips through and the maintainers believe it to be the result of Generative AI, they will remove your code from the codebase and you from the project.
It's like laws against weapons. If you have a concealed gun on your person and enter a public school, chances are that nobody will know and you'll get away with it over and over again. But if anyone ever notices, you're going to jail, you're getting permanently trespassed from school grounds, and you're probably not going to be allowed to own guns for a while.
And, it's a message to everyone else quietly breaking the rules that they have something to lose if they don't stop.
Lots of stupid people asking "how would they know?"
That's not the fucking point.
Okay, easy there, Chief. We were just trying to figure out how it worked. Sorry.
It was a fair question, but this is just going to turn out like universities failing or expelling people for alleged AI content in papers.
They can't prove it. They try to use AI tools to prove it, but those same tools will say a thesis paper from a decade ago is also AI generated. Pretty sure I saw a story of a professor accusing someone based off a tool having his own past paper fail the same tool
Short of an admission of guilt, it's a witch hunt.
This is a good move for international open source projects, with multiple lawsuits in multiple countries around the globe currently ongoing, the intellectual property nature of code made using AI isn't really secure enough to open yourself up to the liability.
I've done the same internally at our company. You're free to use whatever tool you want but if the tool you use spits out copyrighted code, and the law eventually has decided that model users instead of model trainers are liable for model output, then that's on you buddy.
Yup. We don't allow AI tools on our codebase, but I allow it for interviews. I honestly haven't been impressed by it at all, it just encourages not understanding the code.
Does this mean you have indicated to your employees and/or contractors that you intend to hold them legally liable in the case someone launches litigation against you?
So proud of you NetBSD, this is why I sponsor you, slam dunk for the future. I'm working on a NetBSD hardening script and Rice as we speak, great OS with some fantastically valuable niche applications and I think, a new broad approach I'm cooking up, a University Edition. I did hardening for all the other BSD, I saved the best for last!
[EDIT 5/16/2024 15:04 GMT -7] NetBSD got Odin lang support yesterday. That totally seals the NetBSD deal for me if I can come up with something cool for my workstation with Odin.
If you would like to vote on whether, or by what year, AI will be in the Linux Kernel on Infosec.space:
Thanks for your efforts Elias!
I was hoping they ban it because it’s shit, but banning it for copyright reasons is fine too.
I can understand why a project might want to do this until the law is fully implemented and testing in court, but I can tell most of the people in this thread haven’t actually figured out how to effectively use LLMs productively. They’re not about to replace software engineers, but as a software engineer, tools like GitHub copilot and ChatGPT are excellent at speeding up a workflow. ChatGPT for example is an excellent search engine that can give you a quick understanding of a topic. It’ll generate small amounts of code more quickly than I could write it by hand. Of course I’m still going to review that code to ensure it is to the same quality that hand written code would be, but overall this is still a much faster problem.
The luddites who hate on LLMs would have complained about the first compilers too, because they could write marginally faster assembly by hand.
Same for intellisense, IDEs, Debuggers, linters, static analyzers, dynamic languages, garbage collection, NoSQL databases...
Why use a computer to do the work when I could do myself? /s
I never felt so close to try NetBSD as after reading this 😃
We need to see more of this
Hell yeah! Get that shit… OUTTA HERE!!!
Ok but seriously, that is a very good reason to ban it. Who knows what would happen if the AI just fully ripped someone else’s code off that’s supposed to be like GPL licensed or something. If humans can plagiarize, than AIs can plagiarize.
But also, how are they still using CVS? CVS is so slow and so bad. Even Subversion would be an upgrade.
I get banning for quality, but for potential copyright is pretty stupid.
It's not really stupid at all. See the matrix code example from this article: https://spectrum.ieee.org/ai-code-generation-ownership
You can't really know when the genAI is synthesizing from thousands of inputs or just outright reciting copyrighted code. Not kosher if it's the latter.