论文阅读_4_prompt_《Active Prompting with Chain-of-Thought for Large Language Models》

0-摘要

结合具体任务的有效prompt对生成高质量的大模型回答十分重要。特别是在复杂问题的回答上,带有样例的COTPrompt推理,取得了显著效果。然而当前的COT依赖于人工标注的样本示例,针对不同任务,这些样例不一定有效。本文提出一种动态Prompt,基于具体任务示例的Prompt,让模型适应不同任务。我们提出一种解决方法,从特定任务查询的池子中决定哪些问题是最重要且最有助于(人工)标注的。借鉴于给予不确定性的主动学习思想,我们引入几种评价指标来表征不确定性,以便选择最不确定的问题进行注释。实验证明我们的方法有领先性,在8像推理任务汇总取得了最好成绩。

code:GitHub - shizhediao/active-prompt: Source code for the paper "Active Prompting with Chain-of-Thought for Large Language Models"

1-引言

in-context learning 在对话生成任务中取得好成绩,但是在复杂推理任务中表现不佳。COTprompt在推理任务中取得了好效果,但是COT依赖于人类工程,他需要人工选择有信息量的问题然后用COT答案标注答案。COT对不同的任务也不是总有效。COT原论文汇总,从训练集中或者手工为8个问题制作了示例,由于推理任务有不同难度,不同领域,不同范围,很难知道哪些问题应该标注,哪些问题不应该。但好在为不同任务标注8个实例成本不高,鉴于此,我们提出利用不确定性来确定一些问题来标注。

借鉴给予不确定性的主动学习思想,我们引入几个评价指标来表征对于每个问题模型预测的不确定性。我们提出一种给予不确定性的标注策略,这种策略从下有数据集中选择一系列问题,人工参

### Chain-of-Thought Prompting Mechanism in Large Language Models In large language models, chain-of-thought prompting serves as a method to enhance reasoning capabilities by guiding the model through structured thought processes. This approach involves breaking down complex problems into simpler components and providing step-by-step guidance that mirrors human cognitive processing. The creation of these prompts typically includes selecting examples from training datasets where each example represents part of an overall problem-solving process[^2]. By decomposing tasks into multiple steps, this technique encourages deeper understanding and more accurate predictions compared to traditional methods. For instance, when faced with multi-hop question answering or logical deduction challenges, using such chains allows models not only to generate correct answers but also articulate intermediate thoughts leading up to those conclusions. Such transparency facilitates better interpretability while improving performance on various NLP benchmarks. ```python def create_chain_of_thought_prompt(task_description, examples): """ Creates a chain-of-thought prompt based on given task description and examples. Args: task_description (str): Description of the task at hand. examples (list): List containing tuples of input-output pairs used for demonstration purposes. Returns: str: Formatted string representing the final prompt including both instructions and sample cases. """ formatted_examples = "\n".join([f"Input: {ex[0]}, Output: {ex[1]}" for ex in examples]) return f""" Task: {task_description} Examples: {formatted_examples} Now try solving similar questions following above pattern. """ # Example usage examples = [ ("What color do you get mixing red and blue?", "Purple"), ("If it rains tomorrow, will we have our picnic?", "No") ] print(create_chain_of_thought_prompt("Solve logic puzzles", examples)) ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值