TIL answering trivia questions was the pinnacle of human civilization. I guess that’s why the people who win pub quizzes are so prevalent in all governments.
Obligatory note that the way to go is to generate the rationale first, then the answer. (Chain of Thought)
Since it generates left to right, if you start with the answer, it has to first guess the answer “blindly”, then backfill an argument for it. If you generate the argument first, it only needs to figure out the next step of the argument, and it can consult its own argument when answering.
A human would think about the rationale internally first, determine the conclusion, then report the conclusion and present the rationale from cache. But the context window is what GPT uses to think. If the rationale isn’t in the context window, GPT can’t use it.
Also, remember to prepend “Let’s think step by step.” to the answer when prompting!
The interesting thing here is that it is able to provide a rationale. There have been other examples of ML systems doing well at multiple choice questions because people are really bad at designing them (in one case, the longest answer was correct in over 75% of cases and the system learned just to pick the longest one).
Not until ChatGPT starts generating its own clickbait titles.
Hopefully we’d be able to train it not to fall afoul of Betteridge’s Law
Fake humorous clickbait generators have been a thing for ages.
TIL answering trivia questions was the pinnacle of human civilization. I guess that’s why the people who win pub quizzes are so prevalent in all governments.
Obligatory note that the way to go is to generate the rationale first, then the answer. (Chain of Thought)
Since it generates left to right, if you start with the answer, it has to first guess the answer “blindly”, then backfill an argument for it. If you generate the argument first, it only needs to figure out the next step of the argument, and it can consult its own argument when answering.
A human would think about the rationale internally first, determine the conclusion, then report the conclusion and present the rationale from cache. But the context window is what GPT uses to think. If the rationale isn’t in the context window, GPT can’t use it.
Also, remember to prepend “Let’s think step by step.” to the answer when prompting!
Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/abs/2201.11903
That’s very interesting, thanks!
The interesting thing here is that it is able to provide a rationale. There have been other examples of ML systems doing well at multiple choice questions because people are really bad at designing them (in one case, the longest answer was correct in over 75% of cases and the system learned just to pick the longest one).
could do without the pop-over
I don’t think that answering trivia questions correlates to anything useful in practice. This doesn’t make me feel doom.