Technology

66353 readers

4315 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

[email protected]

606

AI search engines give incorrect answers at an alarming 60% rate, study says (arstechnica.com)

submitted 22 hours ago by [email protected] to c/[email protected]

43 comments fedilink hide all child comments

cross-posted from: https://infosec.pub/post/24994013

CJR study shows AI search services misinform users and ignore publisher exclusion requests.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 15 points 5 hours ago (1 children)

Only yesterday, I searched for a very simple figure, the number of public service agents in a specific administrative region. This is, obviously, public information. There is a government site where you can get it. However I didn't know the exact site, so I searched for it on Google.

Of course, AI summary shows up first, and gives me a confident answer, accurately mirroring my exact request. However the number seems way too low to me, so I go check the first actual search result, the aforementioned official site. Google's shitty assistant took a sentence about a subgroup of agents, and presented it as the total. The real number was clearly given before, and was about 4 times that.

This is just a tidbit of information any human with the source would have identified in a second. How the hell are we supposed to trust AI for complex stuff after that?

[–] [email protected] 1 points 38 minutes ago

The AI models can be hilariously bad even on their own terms.

Yesterday I asked Gemini for a population figure (because I was too lazy to look it up myself). First I asked it:

what is the combined population of place 1, place 2 and place 3

It answered:

Unfortunately, providing an absolutely precise, up-to-the-minute combined population figure is difficult. Population numbers are constantly changing.

To find these figures, I would recommend checking the Office for National Statistics (ONS) website. They provide detailed population statistics for local authorities in England.

On a whim, I asked it again as:

combined population of place 1, place 2 and place 3

And then it gave me the answer sweet as a nut.

Apparently I was being too polite with it, I guess?