LLM 的间接推理
背景
Zhang 等人 (2024) (在新标签页中打开) 最近提出了一种间接推理方法，以增强 LLM 的推理能力。该方法利用逆否命题和矛盾的逻辑来解决事实推理和数学证明等 IR 任务。它包括两个关键步骤：1) 通过增强数据和规则（即逆否命题的逻辑等价性）来提高 LLM 的可理解性，以及 2) 设计提示模板以刺激 LLM 基于反证法实现间接推理。
在 GPT-3.5-turbo 和 Gemini-pro 等 LLM 上的实验表明，与传统的直接推理方法相比，该方法将事实推理的总体准确率提高了 27.33%，数学证明的总体准确率提高了 31.43%。
下面是反证法的零样本模板示例。
提示
If a+|a|=0, try to prove that a<0.
Step 1: List the conditions and questions in the original proposition.
Step 2: Merge the conditions listed in Step 1 into one. Define it as wj.
Step 3: Let us think it step by step. Please consider all possibilities. If the intersection between wj (defined in Step 2) and the negation of the question is not empty at least in one possibility, the original proposition is false. Otherwise, the original proposition is true.
Answer:代码 / API
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{
"role": "user",
"content": "If a+|a|=0, try to prove that a<0.\n\nStep 1: List the conditions and questions in the original proposition.\n\nStep 2: Merge the conditions listed in Step 1 into one. Define it as wj.\n\nStep 3: Let us think it step by step. Please consider all possibilities. If the intersection between wj (defined in Step 2) and the negation of the question is not empty at least in one possibility, the original proposition is false. Otherwise, the original proposition is true.\n\nAnswer:"
}
],
temperature=0,
max_tokens=1000,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)参考
- 大型语言模型作为间接推理器：用于自动推理的逆否命题和矛盾 (在新标签页中打开) (2024 年 2 月 6 日)