Tried out the new and popular “Deepseek” LLM with my standard “tell me facts about the author of PCalc” query. At least half were misleading or straight up hallucinations. LLMs are not a suitable technology for looking up facts, and anybody who tells you otherwise is… probably trying to sell you a LLM.
=> View attached media | View attached media | View attached media | View attached media
=> More informations about this toot | More toots from jamesthomson@mastodon.social
I then asked for a list of ten Easter eggs in the app, and every single one was a hallucination, bar the Konami code, which I did actually do.
=> View attached media | View attached media | View attached media | View attached media
=> More informations about this toot | More toots from jamesthomson@mastodon.social
@jamesthomson I wonder if some other calculator had some of these Easter eggs?
It’s very specific.
=> More informations about this toot | More toots from samir@functional.computer
@samir LLMs are great at specific sounding fabrications!
=> More informations about this toot | More toots from jamesthomson@mastodon.social
@jamesthomson It’s really impressive how on-point they are. I’d love to see some of those Easter eggs. (Please don’t actually waste your time though.)
Total nonsense, of course, but I think that’s a feature for a lot of people.
=> More informations about this toot | More toots from samir@functional.computer
@samir @jamesthomson They are plausibility machines. everything they say is plausible - & that's the problem.
As an expert, it takes seconds to debunk the 20% that's obviously shite, 75% is correct, and that last 5% takes you two hours and a load of research, to be sure the plausible lie isn't some obscure fact you didn't know/forgot!
(Not quite the same percentages when it's you asking about your own life, I bet!)
=> More informations about this toot | More toots from Dss@infosec.exchange
@jamesthomson you should to more easter eggs 🤭
=> More informations about this toot | More toots from flo@chaos.social
@jamesthomson I tried the same for my app @boulesscore and I am not even the developer according to chatGPT. Not even one fact was right.
=> More informations about this toot | More toots from jayfm@mstdn.social
@jamesthomson Can you please implement the RPN history? I’m sure AI can help you…
=> More informations about this toot | More toots from luhrman@mastodon.social
@jamesthomson James, I find it absurd that you would CLAIM to know more about PCalc than an LLM. The LLM was trained on REAL FACTS found online. Who are you to question anything about PCalc???
=> More informations about this toot | More toots from ioslife@techhub.social
@ioslife What was I thinking! Notes apology incoming.
=> More informations about this toot | More toots from jamesthomson@mastodon.social
@jamesthomson Konami code no longer in there? Or did I misunderstand the directions? All I get is a beep?
=> More informations about this toot | More toots from uliwitness@chaos.social
@uliwitness I think it should speak and say “secret mode activated” or something. It may only work with a gamepad though!
=> More informations about this toot | More toots from jamesthomson@mastodon.social
@jamesthomson That could be it, I just tried arrow keys on the keyboard.
=> More informations about this toot | More toots from uliwitness@chaos.social
@jamesthomson 55378008 joke has been beaten down to 5318008?? What's the tale to get to that number, or is it just putting that number in the calc and giggling, leaving poor Dolly to handle her pain issues on her own?
I guess we needed more imagination as kids :p
=> More informations about this toot | More toots from crazyeddie@mastodon.social
@jamesthomson I am glad you included the ability to type in the numerical sequence 5318008 and flipping the phone over. So many calculators don't include these necessary features.
=> More informations about this toot | More toots from tvwonder@mastodon.social
@tvwonder The thing is, I actually didn’t :)
=> More informations about this toot | More toots from jamesthomson@mastodon.social
@jamesthomson it's just funny that the LLM said they were features you added. As if you have to specifically allow people to type in a specific sequence of numbers and flip the phone over.
And relevant to your original statement - my daughter is one who has started to use ChatGPT as her primary search engine, no matter what I say about its hallucinations. 🤷🏼♂️
=> More informations about this toot | More toots from tvwonder@mastodon.social
@tvwonder Yeah, hallucinations in search is just not what the world needs right now!
=> More informations about this toot | More toots from jamesthomson@mastodon.social
@jamesthomson exactly
=> More informations about this toot | More toots from tvwonder@mastodon.social This content has been proxied by September (3851b).Proxy Information
text/gemini
