I get a lot of obviously wrong or slightly wrong answers from all LLMs across the board. If you don't double-check you just don't know or you might miss it.
And for the bleeding edge part: My godchild wanted to know the name of a Pokémon from his licensed shirt. One of the first ones, found in every training set. Couldn't get it right, was certain it was the wrong one.
> wanted to know the name of a Pokémon from his licensed shirt.
how did you even ask an AI this question? Took a picture of the shirt, uploaded it, and asked the AI what it was? That's a different class of problem than "answer this graph theory question and include your references: <paste text here>"
And for the bleeding edge part: My godchild wanted to know the name of a Pokémon from his licensed shirt. One of the first ones, found in every training set. Couldn't get it right, was certain it was the wrong one.