this post was submitted on 03 Dec 2023
212 points (89.8% liked)

Technology

34894 readers
417 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.


Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.


Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 5 years ago
MODERATORS
 

AI singer-songwriter 'Anna Indiana' debuted her first single 'Betrayed by this Town' on X, formerly Twitter—and listeners were not too impressed.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 76 points 11 months ago (11 children)

I've been doing a lot of using, testing, and evaluating LLMs and GPT-style models for generating code and text/prose. Some of it is just general use to see how it behaves, some has been explicit evaluation of creative writing, and a bunch of it is code generation to test out how we need to modify our CS curriculum in light of these new tools.

It's an impressive piece of technology, but it's not very creative. It's meh. The results are meh. Which is to be expected since it's a statistical model that's using a large body of prior work to produce a reasonable approximation of what it's seen before. It trends towards the mean, not the best.

[–] [email protected] 9 points 11 months ago (5 children)

and a bunch of it is code generation to test out how we need to modify our CS curriculum in light of these new tools.

I'm curious if you've gotten anything decent out of them. I've tried to use it for tech/code questions, and it's been nothing but disappointment after disappointment. I've tried to use it to get help with new concepts, but it hallucinates like crazy and always give me bad results, some of the time it's so bad that it gives me answers I've already told it we're wrong.

[–] [email protected] 3 points 11 months ago (2 children)

What version are you using?

GPT-4 is quite impressive, and the dedicated code LLMs like Codex and Copilot are as well. The latter must have had a significant update in the past few months, as it's become wildly better almost overnight. If trying it out, you should really do so in an existing codebase it can use as a context to match style and conventions from. Using a blank context is when you get the least impressive outputs from tools like those.

[–] [email protected] 1 points 11 months ago (1 children)

I've used gpt 3/3.5, bing, bard and copilot, and I'm not super stoked. Copilot gave me PS DSC items that don't actually exist, which was my most recent attempt at using a LLM.

I might see about figuring out if it can hook into my vs code instance so it's a bit smarter at some point.

[–] [email protected] 1 points 11 months ago

I might see about figuring out if it can hook into my vs code instance so it's a bit smarter at some point.

There's an official plug-in to do this that takes like 15 minutes to set up.

load more comments (2 replies)
load more comments (7 replies)