Vital.AI’s Post

Interesting article. I've also been thinking about how LLMs might be combined with temporal and defeasible reasoning.   Re your suggestion "If we used our LLM to generate logical statements from the prompt...", how can we reliably induce the LLM to correctly employ the formalism and ontology of our reasoner? I.e., what's needed for the LLM to be "good at" this task?

Claude Opus 3 inside Perplexity AI: Based on the information provided, if the standard 3-color traffic light is not displaying green or blue, the possible color(s) it could be displayed are: 1. Red: A steady red light indicates that vehicles must stop before the stop line or crosswalk. 2. Yellow (Amber): A steady yellow light warns drivers that the red light is about to appear, and they should prepare to stop if it is safe to do so. If they are too close to the intersection to stop safely, they may cautiously proceed through the intersection. 3. Red and Yellow (Amber) simultaneously: In some countries, traffic lights may display red and yellow lights together for a brief period before the green light appears. This indicates that the green light is about to appear, and drivers should prepare to start moving. It's important to note that traffic light colors and their meanings can vary slightly between countries, but red, yellow, and green are the most universally used colors in standard 3-color traffic light systems.

This logical misstep by GPT does not seem to be repeatable. I paraphrased the prompt to get the right response from GPT 3.5. I'm more curious to know why the LLM gave an incorrect response in the first place!

Or red AND yellow ;)