Technology

A guy is driving around the back woods of Montana and he sees a sign in front of a broken down shanty-style house: 'Talking Dog For Sale.'

He rings the bell and the owner appears and tells him the dog is in the backyard.

The guy goes into the backyard and sees a nice looking Labrador Retriever sitting there.

"You talk?" he asks.

"Yep" the Lab replies.

After the guy recovers from the shock of hearing a dog talk, he says, "So, what's your story?"

The Lab looks up and says, "Well, I discovered that I could talk when I was pretty young. I wanted to help the government, so I told the CIA. In no time at all they had me jetting from country to country, sitting in rooms with spies and world leaders, because no one figured a dog would be eavesdropping, I was one of their most valuable spies for eight years running... but the jetting around really tired me out, and I knew I wasn't getting any younger so I decided to settle down. I signed up for a job at the airport to do some undercover security, wandering near suspicious characters and listening in. I uncovered some incredible dealings and was awarded a batch of medals. I got married, had a mess of puppies, and now I'm just retired."

The guy is amazed. He goes back in and asks the owner what he wants for the dog.

"Ten dollars" the guy says.

"Ten dollars? This dog is amazing! Why on Earth are you selling him so cheap?"

"Because he's a liar. He's never been out of the yard."

There is an alternative reality out there where LLMs were never marketed as AI and were marketed as random generator.

In that world, tech savvy people would embrace this tech instead of having to constantly educate people that it is in fact not intelligence.

I've already had more than one conversation where people quote AI as if it were a source, like quoting google as a source. When I showed them how it can sometimes lie and explain it's not a primary source for anything I just get that blank stare like I have two heads.

Because you're using it wrong. It's good for generative text and chains of thought, not symbolic calculations including math or linguistics

I think I have seen this exact post word for word fifty times in the last year.

This is a bad example.. If I ask a friend "is strawberry spelled with one or two r's"they would think I'm asking about the last part of the word.

The question seems to be specifically made to trip up LLMs. I've never heard anyone ask how many of a certain letter is in a word. I've heard people ask how you spell a word and if it's with one or two of a specific letter though.

If you think of LLMs as something with actual intelligence you're going to be very unimpressed.. It's just a model to predict the next word.

This is literally just a tokenization artifact. If I asked you how many r’s are in /0x5273/0x7183 you’d be confused too.

It's predictive text on speed. The LLMs currently in vogue hardly qualify as A.I. tbh..

It's like someone who has no formal education but has a high level of confidence and eavesdrops on a lot of random conversations.

I asked mistral/brave AI and got this response:

How Many Rs in Strawberry

The word "strawberry" contains three "r"s. This simple question has highlighted a limitation in large language models (LLMs), such as GPT-4 and Claude, which often incorrectly count the number of "r"s as two. The error stems from the way these models process text through a process called tokenization, where text is broken down into smaller units called tokens. These tokens do not always correspond directly to individual letters, leading to errors in counting specific letters within words.