Language Agent Tree Search (LATS), QA (HotpotQA, FEVER, AmbigNQ, TriviaQA) for LATS, Notebook for QA LATS, Strategies Module QA for LATS/Structured Outputs #244
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughA comprehensive new module for the Language Agent Tree Search (LATS) Agent has been implemented. This includes foundational functionality in Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
alckasoc
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- agential/cog/lats/init.py (1 hunks)
- agential/cog/lats/strategies/init.py (1 hunks)
Files skipped from review due to trivial changes (2)
- agential/cog/lats/init.py
- agential/cog/lats/strategies/init.py
Codecov ReportAttention: Patch coverage is
|
There was a problem hiding this comment.
Actionable comments posted: 10
Outside diff range, codebase verification and nitpick comments (2)
agential/cog/lats/functional.py (2)
73-74: Improve string representation.The
__str__method could be improved by including more details about the node's state.def __str__(self): return f"Node(depth={self.depth}, value={self.value:.2f}, visits={self.visits}, thought={self.state['thought']}, action={self.state['action']}, observation={self.state['observation']}, is_terminal={self.is_terminal}, reward={self.reward})"
91-104: Clarify logic for excluding root node's observation.The logic for excluding the root node's observation could be clarified with a comment.
if node.state['observation'] and node.depth != 0: # Exclude the observation from the root node new_segment.append(f"Observation {node.depth}: {node.state['observation']}")
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- agential/cog/lats/functional.py (1 hunks)
Additional comments not posted (4)
agential/cog/lats/functional.py (4)
227-232: Optimize trajectory collection logic.The logic for collecting the trajectory of a node could be optimized.
def collect_trajectory(node): trajectory = [] while node: trajectory.append(str(node)) node = node.parent return '\n'.join(reversed(trajectory))Likely invalid or redundant comment.
54-67: Avoid using global variables.The use of global variables
reflection_mapandfailed_trajectoriescan lead to issues with maintainability and testability. Consider refactoring to avoid using global variables.- global reflection_map - global failed_trajectories - reflection_map = [] - failed_trajectories = []Likely invalid or redundant comment.
76-88: Optimize dictionary conversion.The
to_dictmethod can be optimized by using dictionary comprehensions.def to_dict(self): return { 'state': self.state, 'question': self.question, 'parent': self.parent.to_dict() if self.parent else None, 'children': [child.to_dict() for child in self.children], 'visits': self.visits, 'value': self.value, 'depth': self.depth, 'is_terminal': self.is_terminal, 'reward': self.reward, 'em': self.em, }Likely invalid or redundant comment.
183-193: Avoid using global variables.The use of global variables
failed_trajectoriesshould be avoided to improve maintainability and testability.def get_unique_trajectories(failed_trajectories, num=5): unique_trajectories = [] seen_final_answers = set() for traj in failed_trajectories: final_answer = traj.get('final_answer') if final_answer not in seen_final_answers: unique_trajectories.append(node_trajectory_to_text(traj['trajectory'])) seen_final_answers.add(final_answer) if len(unique_trajectories) >= num: break return unique_trajectoriesLikely invalid or redundant comment.
agential/cog/lats/functional.py
Outdated
Comment on lines
195
to
196
| def completions_with_backoff(**kwargs): | ||
| return openai.ChatCompletion.create(**kwargs) |
There was a problem hiding this comment.
Add error handling.
The completions_with_backoff function lacks error handling for API call failures.
def completions_with_backoff(**kwargs):
try:
return openai.ChatCompletion.create(**kwargs)
except openai.error.OpenAIError as e:
# Handle the error (e.g., log it, retry, etc.)
raise eCommittable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def completions_with_backoff(**kwargs): | |
| return openai.ChatCompletion.create(**kwargs) | |
| def completions_with_backoff(**kwargs): | |
| try: | |
| return openai.ChatCompletion.create(**kwargs) | |
| except openai.error.OpenAIError as e: | |
| # Handle the error (e.g., log it, retry, etc.) | |
| raise e |
agential/cog/lats/functional.py
Outdated
Comment on lines
289
to
296
| def expand_node(node, args, task): | ||
| if node.depth >= 7: | ||
| print("Depth limit reached") | ||
| node.is_terminal = True | ||
| return | ||
| new_nodes = generate_new_states(node, args, task, args.n_generate_sample) | ||
| node.children.extend(new_nodes) | ||
|
|
There was a problem hiding this comment.
Improve depth limit check.
The depth limit check in the expand_node function could be improved by using a constant for the depth limit.
DEPTH_LIMIT = 7
def expand_node(node, args, task):
if node.depth >= DEPTH_LIMIT:
print("Depth limit reached")
node.is_terminal = True
return
new_nodes = generate_new_states(node, args, task, args.n_generate_sample)
node.children.extend(new_nodes)Committable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def expand_node(node, args, task): | |
| if node.depth >= 7: | |
| print("Depth limit reached") | |
| node.is_terminal = True | |
| return | |
| new_nodes = generate_new_states(node, args, task, args.n_generate_sample) | |
| node.children.extend(new_nodes) | |
| DEPTH_LIMIT = 7 | |
| def expand_node(node, args, task): | |
| if node.depth >= DEPTH_LIMIT: | |
| print("Depth limit reached") | |
| node.is_terminal = True | |
| return | |
| new_nodes = generate_new_states(node, args, task, args.n_generate_sample) | |
| node.children.extend(new_nodes) |
agential/cog/lats/functional.py
Outdated
Comment on lines
242
to
286
| def generate_new_states(node, args, task, n): | ||
| global failed_trajectories | ||
| prompt = generate_prompt(node) | ||
| sampled_actions = get_samples(task, prompt, f"Thought {node.depth + 1}: ", n, prompt_sample=args.prompt_sample, stop="Observation") | ||
| tried_actions = [] | ||
|
|
||
| unique_states = {} # Store unique states here | ||
| for action in sampled_actions: | ||
| new_state = node.state.copy() # Make a copy of the parent node's state | ||
|
|
||
| thought_line = next((line.split(":")[1].strip() for line in action.split("\n") if line.startswith(f"Thought {node.depth + 1}")), '') | ||
| action_line = next((line.split(":")[1].strip() for line in action.split("\n") if line.startswith("Action") and ":" in line), None) | ||
|
|
||
| # Use thought and action to form a unique key | ||
| unique_key = f"{thought_line}::{action_line}" | ||
|
|
||
| if unique_key in unique_states: | ||
| continue # Skip if this state already exists | ||
|
|
||
| tried_actions.append(action_line) | ||
|
|
||
| if action_line: | ||
| action_type = action_line.split('[')[0] if '[' in action_line else action_line | ||
| action_param = action_line.split('[')[1].split(']')[0] if '[' in action_line else "" | ||
|
|
||
| obs, r, done, info = step(env, f"{action_type.lower()}[{action_param}]") | ||
|
|
||
| # Update the new state dictionary | ||
| new_state['thought'] = thought_line | ||
| new_state['action'] = action_line | ||
| new_state['observation'] = obs | ||
|
|
||
| new_node = Node(state=new_state, question=node.question, parent=node) | ||
| new_node.is_terminal = r == 1 or done | ||
| new_node.reward = r | ||
| new_node.depth = node.depth + 1 | ||
| if r == 1: | ||
| new_node.em = info.get('em') | ||
| unique_states[unique_key] = new_node # Add this state to unique_states | ||
|
|
||
| if new_node.is_terminal and r == 0: | ||
| trajectory = collect_trajectory(new_node) | ||
| failed_trajectories.append({'trajectory': trajectory, 'final_answer': f"{action_type.lower()}[{action_param}]"}) | ||
|
|
||
| return list(unique_states.values()) # Return unique nodes as a list |
There was a problem hiding this comment.
Avoid using global variables.
The use of global variables failed_trajectories should be avoided to improve maintainability and testability.
def generate_new_states(node, args, task, n):
prompt = generate_prompt(node)
sampled_actions = get_samples(task, prompt, f"Thought {node.depth + 1}: ", n, prompt_sample=args.prompt_sample, stop="Observation")
tried_actions = []
unique_states = {}
for action in sampled_actions:
new_state = node.state.copy()
thought_line = next((line.split(":")[1].strip() for line in action.split("\n") if line.startswith(f"Thought {node.depth + 1}")), '')
action_line = next((line.split(":")[1].strip() for line in action.split("\n") if line.startswith("Action") and ":" in line), None)
unique_key = f"{thought_line}::{action_line}"
if unique_key in unique_states:
continue
tried_actions.append(action_line)
if action_line:
action_type = action_line.split('[')[0] if '[' in action_line else action_line
action_param = action_line.split('[')[1].split(']')[0] if '[' in action_line else ""
obs, r, done, info = step(env, f"{action_type.lower()}[{action_param}]")
new_state['thought'] = thought_line
new_state['action'] = action_line
new_state['observation'] = obs
new_node = Node(state=new_state, question=node.question, parent=node)
new_node.is_terminal = r == 1 or done
new_node.reward = r
new_node.depth = node.depth + 1
if r == 1:
new_node.em = info.get('em')
unique_states[unique_key] = new_node
if new_node.is_terminal and r == 0:
trajectory = collect_trajectory(new_node)
failed_trajectories.append({'trajectory': trajectory, 'final_answer': f"{action_type.lower()}[{action_param}]"})
return list(unique_states.values())Committable suggestion was skipped due to low confidence.
agential/cog/lats/functional.py
Outdated
Comment on lines
161
to
181
| def node_trajectory_to_text(node_string): | ||
| lines = node_string.split('\n') | ||
| formatted_lines = [] | ||
| for line in lines: | ||
| try: | ||
| depth = int(line.split(",")[0].split("=")[1].strip()) | ||
| thought = line.split(", thought=")[1].split(", action=")[0].strip() | ||
| action = line.split(", action=")[1].split(", observation=")[0].strip() | ||
| observation = line.split(", observation=")[1].split(")")[0].strip() | ||
| except IndexError: | ||
| continue | ||
|
|
||
| if depth != 0: | ||
| if thought: | ||
| formatted_lines.append(f"Thought {depth}: {thought}") | ||
| if action: | ||
| formatted_lines.append(f"Action {depth}: {action}") | ||
| if observation: | ||
| formatted_lines.append(f"Observation {depth}: {observation}") | ||
|
|
||
| return '\n'.join(formatted_lines) |
There was a problem hiding this comment.
Improve error handling for IndexError.
The error handling for IndexError could be improved by logging the error or providing more context.
except IndexError:
# Log the error or provide more context
continueCommittable suggestion was skipped due to low confidence.
agential/cog/lats/functional.py
Outdated
Comment on lines
211
to
225
| def get_samples(task, x, y, n_generate_sample, prompt_sample, stop): | ||
| global failed_trajectories | ||
| global reflection_map | ||
| unique_trajectories = get_unique_trajectories(failed_trajectories) | ||
| if len(unique_trajectories) > len(reflection_map) and len(unique_trajectories) < 4: | ||
| print("generating reflections") | ||
| reflection_map = task.generate_self_reflection(unique_trajectories, x) | ||
| if prompt_sample == 'standard': | ||
| prompt = task.standard_prompt_wrap(x, y) | ||
| elif prompt_sample == 'cot': | ||
| prompt = task.cot_prompt_wrap(x, y, reflection_map) | ||
| else: | ||
| raise ValueError(f'prompt_sample {prompt_sample} not recognized') | ||
| samples = gpt(prompt, n=n_generate_sample, stop=stop) | ||
| return [y + _ for _ in samples] |
There was a problem hiding this comment.
Avoid using global variables.
The use of global variables failed_trajectories and reflection_map should be avoided to improve maintainability and testability.
def get_samples(task, x, y, n_generate_sample, prompt_sample, stop):
unique_trajectories = get_unique_trajectories(failed_trajectories)
if len(unique_trajectories) > len(reflection_map) and len(unique_trajectories) < 4:
print("generating reflections")
reflection_map = task.generate_self_reflection(unique_trajectories, x)
if prompt_sample == 'standard':
prompt = task.standard_prompt_wrap(x, y)
elif prompt_sample == 'cot':
prompt = task.cot_prompt_wrap(x, y, reflection_map)
else:
raise ValueError(f'prompt_sample {prompt_sample} not recognized')
samples = gpt(prompt, n=n_generate_sample, stop=stop)
return [y + _ for _ in samples]Committable suggestion was skipped due to low confidence.
agential/cog/lats/functional.py
Outdated
Comment on lines
8
to
30
| def _build_standard_prompt(): | ||
| pass | ||
|
|
||
| def _prompt_standard(): | ||
| pass | ||
|
|
||
| def _build_cot_prompt(): | ||
| pass | ||
|
|
||
| def _prompt_cot(): | ||
| pass | ||
|
|
||
| def _build_reflection_prompt(): | ||
| pass | ||
|
|
||
| def _prompt_reflection(): | ||
| pass | ||
|
|
||
| def _build_value_prompt(): | ||
| pass | ||
|
|
||
| def _prompt_value(): | ||
| pass |
There was a problem hiding this comment.
Add TODO comments or implement the functions.
The functions _build_standard_prompt, _prompt_standard, _build_cot_prompt, _prompt_cot, _build_reflection_prompt, _prompt_reflection, _build_value_prompt, and _prompt_value are placeholders and do not contain any implementation. Add TODO comments or implement these functions.
def _build_standard_prompt():
# TODO: Implement this function
pass
def _prompt_standard():
# TODO: Implement this function
pass
def _build_cot_prompt():
# TODO: Implement this function
pass
def _prompt_cot():
# TODO: Implement this function
pass
def _build_reflection_prompt():
# TODO: Implement this function
pass
def _prompt_reflection():
# TODO: Implement this function
pass
def _build_value_prompt():
# TODO: Implement this function
pass
def _prompt_value():
# TODO: Implement this function
passCommittable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def _build_standard_prompt(): | |
| pass | |
| def _prompt_standard(): | |
| pass | |
| def _build_cot_prompt(): | |
| pass | |
| def _prompt_cot(): | |
| pass | |
| def _build_reflection_prompt(): | |
| pass | |
| def _prompt_reflection(): | |
| pass | |
| def _build_value_prompt(): | |
| pass | |
| def _prompt_value(): | |
| pass | |
| def _build_standard_prompt(): | |
| # TODO: Implement this function | |
| pass | |
| def _prompt_standard(): | |
| # TODO: Implement this function | |
| pass | |
| def _build_cot_prompt(): | |
| # TODO: Implement this function | |
| pass | |
| def _prompt_cot(): | |
| # TODO: Implement this function | |
| pass | |
| def _build_reflection_prompt(): | |
| # TODO: Implement this function | |
| pass | |
| def _prompt_reflection(): | |
| # TODO: Implement this function | |
| pass | |
| def _build_value_prompt(): | |
| # TODO: Implement this function | |
| pass | |
| def _prompt_value(): | |
| # TODO: Implement this function | |
| pass |
agential/cog/lats/functional.py
Outdated
Comment on lines
199
to
209
| def gpt(prompt, model="gpt-3.5-turbo", temperature=1.0, max_tokens=100, n=1, stop=None) -> list: | ||
| messages = [{"role": "user", "content": prompt}] | ||
| outputs = [] | ||
|
|
||
| while n > 0: | ||
| cnt = min(n, 20) | ||
| n -= cnt | ||
| res = completions_with_backoff(model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, n=cnt, stop=stop) | ||
| outputs.extend([choice["message"]["content"] for choice in res["choices"]]) | ||
|
|
||
| return outputs |
There was a problem hiding this comment.
Add error handling.
The gpt function lacks error handling for API call failures.
def gpt(prompt, model="gpt-3.5-turbo", temperature=1.0, max_tokens=100, n=1, stop=None) -> list:
messages = [{"role": "user", "content": prompt}]
outputs = []
while n > 0:
cnt = min(n, 20)
n -= cnt
try:
res = completions_with_backoff(model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, n=cnt, stop=stop)
outputs.extend([choice["message"]["content"] for choice in res["choices"]])
except openai.error.OpenAIError as e:
# Handle the error (e.g., log it, retry, etc.)
raise e
return outputsCommittable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def gpt(prompt, model="gpt-3.5-turbo", temperature=1.0, max_tokens=100, n=1, stop=None) -> list: | |
| messages = [{"role": "user", "content": prompt}] | |
| outputs = [] | |
| while n > 0: | |
| cnt = min(n, 20) | |
| n -= cnt | |
| res = completions_with_backoff(model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, n=cnt, stop=stop) | |
| outputs.extend([choice["message"]["content"] for choice in res["choices"]]) | |
| return outputs | |
| def gpt(prompt, model="gpt-3.5-turbo", temperature=1.0, max_tokens=100, n=1, stop=None) -> list: | |
| messages = [{"role": "user", "content": prompt}] | |
| outputs = [] | |
| while n > 0: | |
| cnt = min(n, 20) | |
| n -= cnt | |
| try: | |
| res = completions_with_backoff(model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, n=cnt, stop=stop) | |
| outputs.extend([choice["message"]["content"] for choice in res["choices"]]) | |
| except openai.error.OpenAIError as e: | |
| # Handle the error (e.g., log it, retry, etc.) | |
| raise e | |
| return outputs |
agential/cog/lats/functional.py
Outdated
Comment on lines
234
to
241
| def step(env, action): | ||
| attempts = 0 | ||
| while attempts < 10: | ||
| try: | ||
| return env.step(action) | ||
| except requests.exceptions.Timeout: | ||
| attempts += 1 | ||
|
|
There was a problem hiding this comment.
Improve retry logic.
The retry logic in the step function could be improved by adding a delay between retries and handling other potential exceptions.
import time
def step(env, action):
attempts = 0
while attempts < 10:
try:
return env.step(action)
except requests.exceptions.Timeout:
attempts += 1
time.sleep(1) # Add a delay between retries
except requests.exceptions.RequestException as e:
# Handle other potential exceptions
raise eCommittable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def step(env, action): | |
| attempts = 0 | |
| while attempts < 10: | |
| try: | |
| return env.step(action) | |
| except requests.exceptions.Timeout: | |
| attempts += 1 | |
| import time | |
| def step(env, action): | |
| attempts = 0 | |
| while attempts < 10: | |
| try: | |
| return env.step(action) | |
| except requests.exceptions.Timeout: | |
| attempts += 1 | |
| time.sleep(1) # Add a delay between retries | |
| except requests.exceptions.RequestException as e: | |
| # Handle other potential exceptions | |
| raise e |
agential/cog/lats/functional.py
Outdated
Comment on lines
107
to
159
| def select_node(node): | ||
| """ | ||
| Returns current node if it has no children. | ||
|
|
||
| Otherwise, it checks the current node's children. | ||
| - Returns a terminal child node with reward 1 if the current node does not have all terminal children nodes. | ||
| - If the current node has all terminal children nodes, it cuts the current node and all of its children nodes from the tree. | ||
| - If neither of the 2 above are satisfied, then it selects the highest UCT value non-terminal child node and continues looping. | ||
| """ | ||
|
|
||
| # Enters while loop iff the current node exists and has children. | ||
| # Otherwise, it returns the current node. | ||
| while node and node.children: | ||
|
|
||
| # A terminal node is defined as a node with reward 1 or it's done (finishes with an answer). | ||
| terminal_children = [child for child in node.children if child.is_terminal] | ||
| terminal_status = [child.is_terminal for child in node.children] | ||
|
|
||
| # UPDATE: If all the current node's children are finished, then move up to the current node's parent. | ||
| # If all children of current node are terminal, move up to current node's parent. | ||
| # This cuts out all terminal children and the current node from the tree. | ||
| if len(terminal_children) == len(node.children): | ||
| if node.parent: | ||
| node.parent.children.remove(node) | ||
| node = node.parent | ||
| continue | ||
|
|
||
| # c | ||
| # / \ | ||
| # g b | ||
| # \ | ||
| # d | ||
|
|
||
| # Given that the current node does not have all terminal children, | ||
| # - Return the first terminal child node of the current node with reward 1. | ||
| # - Defaults to None if no terminal child node with reward 1 exists. | ||
| node_with_reward_1 = next((child for child in terminal_children if child.reward == 1), None) | ||
| if node_with_reward_1: | ||
| return node_with_reward_1 | ||
|
|
||
| # Given that the current node does not have all terminal children AND | ||
| # Given the current node does not have a terminal child node with reward 1, | ||
| # - Of the current node's non-terminal children, get the child node with the highest UCT value. | ||
| # - Defaults to None if no non-terminal children exist. | ||
| node = max((child for child in node.children if not child.is_terminal), key=lambda child: child.uct(), default=None) | ||
|
|
||
| # Given that the current node does not have all terminal children AND | ||
| # Given the current node does not have a terminal child node with reward 1, | ||
| # - This while loop should never run. The line of code above will be a non-terminal node. | ||
| while node.is_terminal and node.reward != 1: # while False and <> -> False | ||
| node = max((child for child in node.parent.children if not child.is_terminal), key=lambda child: child.uct(), default=None) | ||
|
|
||
| return node # This will return None if all paths from the root are exhausted |
There was a problem hiding this comment.
Simplify nested while loop.
The nested while loop in the select_node function could be simplified to improve readability.
while node and node.children:
terminal_children = [child for child in node.children if child.is_terminal]
if len(terminal_children) == len(node.children):
if node.parent:
node.parent.children.remove(node)
node = node.parent
continue
node_with_reward_1 = next((child for child in terminal_children if child.reward == 1), None)
if node_with_reward_1:
return node_with_reward_1
node = max((child for child in node.children if not child.is_terminal), key=lambda child: child.uct(), default=None)
while node and node.is_terminal and node.reward != 1:
node = max((child for child in node.parent.children if not child.is_terminal), key=lambda child: child.uct(), default=None)
return nodeCommittable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def select_node(node): | |
| """ | |
| Returns current node if it has no children. | |
| Otherwise, it checks the current node's children. | |
| - Returns a terminal child node with reward 1 if the current node does not have all terminal children nodes. | |
| - If the current node has all terminal children nodes, it cuts the current node and all of its children nodes from the tree. | |
| - If neither of the 2 above are satisfied, then it selects the highest UCT value non-terminal child node and continues looping. | |
| """ | |
| # Enters while loop iff the current node exists and has children. | |
| # Otherwise, it returns the current node. | |
| while node and node.children: | |
| # A terminal node is defined as a node with reward 1 or it's done (finishes with an answer). | |
| terminal_children = [child for child in node.children if child.is_terminal] | |
| terminal_status = [child.is_terminal for child in node.children] | |
| # UPDATE: If all the current node's children are finished, then move up to the current node's parent. | |
| # If all children of current node are terminal, move up to current node's parent. | |
| # This cuts out all terminal children and the current node from the tree. | |
| if len(terminal_children) == len(node.children): | |
| if node.parent: | |
| node.parent.children.remove(node) | |
| node = node.parent | |
| continue | |
| # c | |
| # / \ | |
| # g b | |
| # \ | |
| # d | |
| # Given that the current node does not have all terminal children, | |
| # - Return the first terminal child node of the current node with reward 1. | |
| # - Defaults to None if no terminal child node with reward 1 exists. | |
| node_with_reward_1 = next((child for child in terminal_children if child.reward == 1), None) | |
| if node_with_reward_1: | |
| return node_with_reward_1 | |
| # Given that the current node does not have all terminal children AND | |
| # Given the current node does not have a terminal child node with reward 1, | |
| # - Of the current node's non-terminal children, get the child node with the highest UCT value. | |
| # - Defaults to None if no non-terminal children exist. | |
| node = max((child for child in node.children if not child.is_terminal), key=lambda child: child.uct(), default=None) | |
| # Given that the current node does not have all terminal children AND | |
| # Given the current node does not have a terminal child node with reward 1, | |
| # - This while loop should never run. The line of code above will be a non-terminal node. | |
| while node.is_terminal and node.reward != 1: # while False and <> -> False | |
| node = max((child for child in node.parent.children if not child.is_terminal), key=lambda child: child.uct(), default=None) | |
| return node # This will return None if all paths from the root are exhausted | |
| def select_node(node): | |
| """ | |
| Returns current node if it has no children. | |
| Otherwise, it checks the current node's children. | |
| - Returns a terminal child node with reward 1 if the current node does not have all terminal children nodes. | |
| - If the current node has all terminal children nodes, it cuts the current node and all of its children nodes from the tree. | |
| - If neither of the 2 above are satisfied, then it selects the highest UCT value non-terminal child node and continues looping. | |
| """ | |
| # Enters while loop iff the current node exists and has children. | |
| # Otherwise, it returns the current node. | |
| while node and node.children: | |
| # A terminal node is defined as a node with reward 1 or it's done (finishes with an answer). | |
| terminal_children = [child for child in node.children if child.is_terminal] | |
| terminal_status = [child.is_terminal for child in node.children] | |
| # UPDATE: If all the current node's children are finished, then move up to the current node's parent. | |
| # If all children of current node are terminal, move up to current node's parent. | |
| # This cuts out all terminal children and the current node from the tree. | |
| if len(terminal_children) == len(node.children): | |
| if node.parent: | |
| node.parent.children.remove(node) | |
| node = node.parent | |
| continue | |
| # c | |
| # / \ | |
| # g b | |
| # \ | |
| # d | |
| # Given that the current node does not have all terminal children, | |
| # - Return the first terminal child node of the current node with reward 1. | |
| # - Defaults to None if no terminal child node with reward 1 exists. | |
| node_with_reward_1 = next((child for child in terminal_children if child.reward == 1), None) | |
| if node_with_reward_1: | |
| return node_with_reward_1 | |
| # Given that the current node does not have all terminal children AND | |
| # Given the current node does not have a terminal child node with reward 1, | |
| # - Of the current node's non-terminal children, get the child node with the highest UCT value. | |
| # - Defaults to None if no non-terminal children exist. | |
| node = max((child for child in node.children if not child.is_terminal), key=lambda child: child.uct(), default=None) | |
| while node and node.is_terminal and node.reward != 1: | |
| node = max((child for child in node.parent.children if not child.is_terminal), key=lambda child: child.uct(), default=None) | |
| return node # This will return None if all paths from the root are exhausted |
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- agential/cog/lats/functional.py (1 hunks)
- agential/cog/lats/prompts.py (1 hunks)
Additional context used
GitHub Check: codecov/patch
agential/cog/lats/functional.py
[warning] 3-5: agential/cog/lats/functional.py#L3-L5
Added lines #L3 - L5 were not covered by tests
[warning] 8-9: agential/cog/lats/functional.py#L8-L9
Added lines #L8 - L9 were not covered by tests
[warning] 11-12: agential/cog/lats/functional.py#L11-L12
Added lines #L11 - L12 were not covered by tests
[warning] 14-15: agential/cog/lats/functional.py#L14-L15
Added lines #L14 - L15 were not covered by tests
[warning] 17-18: agential/cog/lats/functional.py#L17-L18
Added lines #L17 - L18 were not covered by tests
[warning] 20-21: agential/cog/lats/functional.py#L20-L21
Added lines #L20 - L21 were not covered by tests
[warning] 23-24: agential/cog/lats/functional.py#L23-L24
Added lines #L23 - L24 were not covered by tests
[warning] 26-27: agential/cog/lats/functional.py#L26-L27
Added lines #L26 - L27 were not covered by tests
[warning] 29-30: agential/cog/lats/functional.py#L29-L30
Added lines #L29 - L30 were not covered by tests
[warning] 51-52: agential/cog/lats/functional.py#L51-L52
Added lines #L51 - L52 were not covered by tests
[warning] 54-66: agential/cog/lats/functional.py#L54-L66
Added lines #L54 - L66 were not covered by tests
[warning] 68-71: agential/cog/lats/functional.py#L68-L71
Added lines #L68 - L71 were not covered by tests
[warning] 73-74: agential/cog/lats/functional.py#L73-L74
Added lines #L73 - L74 were not covered by tests
[warning] 76-89: agential/cog/lats/functional.py#L76-L89
Added lines #L76 - L89 were not covered by tests
[warning] 92-92: agential/cog/lats/functional.py#L92
Added line #L92 was not covered by tests
[warning] 104-104: agential/cog/lats/functional.py#L104
Added line #L104 was not covered by tests
[warning] 107-108: agential/cog/lats/functional.py#L107-L108
Added lines #L107 - L108 were not covered by tests
[warning] 113-117: agential/cog/lats/functional.py#L113-L117
Added lines #L113 - L117 were not covered by tests
[warning] 128-130: agential/cog/lats/functional.py#L128-L130
Added lines #L128 - L130 were not covered by tests
[warning] 136-136: agential/cog/lats/functional.py#L136
Added line #L136 was not covered by tests
[warning] 141-142: agential/cog/lats/functional.py#L141-L142
Added lines #L141 - L142 were not covered by tests
[warning] 144-144: agential/cog/lats/functional.py#L144
Added line #L144 was not covered by tests
[warning] 146-156: agential/cog/lats/functional.py#L146-L156
Added lines #L146 - L156 were not covered by tests
[warning] 158-164: agential/cog/lats/functional.py#L158-L164
Added lines #L158 - L164 were not covered by tests
[warning] 166-166: agential/cog/lats/functional.py#L166
Added line #L166 was not covered by tests
[warning] 168-178: agential/cog/lats/functional.py#L168-L178
Added lines #L168 - L178 were not covered by tests
[warning] 180-181: agential/cog/lats/functional.py#L180-L181
Added lines #L180 - L181 were not covered by tests
[warning] 184-186: agential/cog/lats/functional.py#L184-L186
Added lines #L184 - L186 were not covered by tests
[warning] 188-192: agential/cog/lats/functional.py#L188-L192
Added lines #L188 - L192 were not covered by tests
[warning] 194-194: agential/cog/lats/functional.py#L194
Added line #L194 was not covered by tests
Additional comments not posted (17)
agential/cog/lats/functional.py (13)
8-30: Add TODO comments or implement the functions.The functions
_build_standard_prompt,_prompt_standard,_build_cot_prompt,_prompt_cot,_build_reflection_prompt,_prompt_reflection,_build_value_prompt, and_prompt_valueare placeholders and do not contain any implementation. Add TODO comments or implement these functions.Tools
GitHub Check: codecov/patch
[warning] 8-9: agential/cog/lats/functional.py#L8-L9
Added lines #L8 - L9 were not covered by tests
[warning] 11-12: agential/cog/lats/functional.py#L11-L12
Added lines #L11 - L12 were not covered by tests
[warning] 14-15: agential/cog/lats/functional.py#L14-L15
Added lines #L14 - L15 were not covered by tests
[warning] 17-18: agential/cog/lats/functional.py#L17-L18
Added lines #L17 - L18 were not covered by tests
[warning] 20-21: agential/cog/lats/functional.py#L20-L21
Added lines #L20 - L21 were not covered by tests
[warning] 23-24: agential/cog/lats/functional.py#L23-L24
Added lines #L23 - L24 were not covered by tests
[warning] 26-27: agential/cog/lats/functional.py#L26-L27
Added lines #L26 - L27 were not covered by tests
[warning] 29-30: agential/cog/lats/functional.py#L29-L30
Added lines #L29 - L30 were not covered by tests
49-52: Avoid using global variables.The use of global variables
reflection_mapandfailed_trajectoriesshould be avoided to improve maintainability and testability.Tools
GitHub Check: codecov/patch
[warning] 51-52: agential/cog/lats/functional.py#L51-L52
Added lines #L51 - L52 were not covered by tests
54-74: Handle division by zero in UCT calculation.The
uctmethod does not handle the case whereself.parent.visitsis zero, which can lead to a division by zero error.def uct(self): if self.visits == 0 or self.parent.visits == 0: return self.value return self.value / self.visits + np.sqrt(2 * np.log(self.parent.visits) / self.visits)Tools
GitHub Check: codecov/patch
[warning] 54-66: agential/cog/lats/functional.py#L54-L66
Added lines #L54 - L66 were not covered by tests
[warning] 68-71: agential/cog/lats/functional.py#L68-L71
Added lines #L68 - L71 were not covered by tests
[warning] 73-74: agential/cog/lats/functional.py#L73-L74
Added lines #L73 - L74 were not covered by tests
76-89: LGTM!The
generate_promptfunction is well-defined and correctly implemented.Tools
GitHub Check: codecov/patch
[warning] 76-89: agential/cog/lats/functional.py#L76-L89
Added lines #L76 - L89 were not covered by tests
92-144: Simplify nested while loop.The nested while loop in the
select_nodefunction could be simplified to improve readability.while node and node.children: terminal_children = [child for child in node.children if child.is_terminal] if len(terminal_children) == len(node.children): if node.parent: node.parent.children.remove(node) node = node.parent continue node_with_reward_1 = next((child for child in terminal_children if child.reward == 1), None) if node_with_reward_1: return node_with_reward_1 node = max((child for child in node.children if not child.is_terminal), key=lambda child: child.uct(), default=None) while node and node.is_terminal and node.reward != 1: node = max((child for child in node.parent.children if not child.is_terminal), key=lambda child: child.uct(), default=None) return nodeTools
GitHub Check: codecov/patch
[warning] 92-92: agential/cog/lats/functional.py#L92
Added line #L92 was not covered by tests
[warning] 104-104: agential/cog/lats/functional.py#L104
Added line #L104 was not covered by tests
[warning] 107-108: agential/cog/lats/functional.py#L107-L108
Added lines #L107 - L108 were not covered by tests
[warning] 113-117: agential/cog/lats/functional.py#L113-L117
Added lines #L113 - L117 were not covered by tests
[warning] 128-130: agential/cog/lats/functional.py#L128-L130
Added lines #L128 - L130 were not covered by tests
[warning] 136-136: agential/cog/lats/functional.py#L136
Added line #L136 was not covered by tests
[warning] 141-142: agential/cog/lats/functional.py#L141-L142
Added lines #L141 - L142 were not covered by tests
[warning] 144-144: agential/cog/lats/functional.py#L144
Added line #L144 was not covered by tests
146-166: Improve error handling for IndexError.The error handling for
IndexErrorcould be improved by logging the error or providing more context.except IndexError: # Log the error or provide more context continueTools
GitHub Check: codecov/patch
[warning] 146-156: agential/cog/lats/functional.py#L146-L156
Added lines #L146 - L156 were not covered by tests
[warning] 158-164: agential/cog/lats/functional.py#L158-L164
Added lines #L158 - L164 were not covered by tests
[warning] 166-166: agential/cog/lats/functional.py#L166
Added line #L166 was not covered by tests
168-178: LGTM!The
get_unique_trajectoriesfunction is well-defined and correctly implemented.Tools
GitHub Check: codecov/patch
[warning] 168-178: agential/cog/lats/functional.py#L168-L178
Added lines #L168 - L178 were not covered by tests
180-181: Add error handling.The
completions_with_backofffunction lacks error handling for API call failures.def completions_with_backoff(**kwargs): try: return openai.ChatCompletion.create(**kwargs) except openai.error.OpenAIError as e: # Handle the error (e.g., log it, retry, etc.) raise eTools
GitHub Check: codecov/patch
[warning] 180-181: agential/cog/lats/functional.py#L180-L181
Added lines #L180 - L181 were not covered by tests
184-194: Add error handling.The
gptfunction lacks error handling for API call failures.def gpt(prompt, model="gpt-3.5-turbo", temperature=1.0, max_tokens=100, n=1, stop=None) -> list: messages = [{"role": "user", "content": prompt}] outputs = [] while n > 0: cnt = min(n, 20) n -= cnt try: res = completions_with_backoff(model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, n=cnt, stop=stop) outputs.extend([choice["message"]["content"] for choice in res["choices"]]) except openai.error.OpenAIError as e: # Handle the error (e.g., log it, retry, etc.) raise e return outputsTools
GitHub Check: codecov/patch
[warning] 184-186: agential/cog/lats/functional.py#L184-L186
Added lines #L184 - L186 were not covered by tests
[warning] 188-192: agential/cog/lats/functional.py#L188-L192
Added lines #L188 - L192 were not covered by tests
[warning] 194-194: agential/cog/lats/functional.py#L194
Added line #L194 was not covered by tests
196-210: Avoid using global variables.The use of global variables
failed_trajectoriesandreflection_mapshould be avoided to improve maintainability and testability.
219-226: Improve retry logic.The retry logic in the
stepfunction could be improved by adding a delay between retries and handling other potential exceptions.import time def step(env, action): attempts = 0 while attempts < 10: try: return env.step(action) except requests.exceptions.Timeout: attempts += 1 time.sleep(1) # Add a delay between retries except requests.exceptions.RequestException as e: # Handle other potential exceptions raise e
227-271: Avoid using global variables.The use of global variables
failed_trajectoriesshould be avoided to improve maintainability and testability.
274-281: Improve depth limit check.The depth limit check in the
expand_nodefunction could be improved by using a constant for the depth limit.DEPTH_LIMIT = 7 def expand_node(node, args, task): if node.depth >= DEPTH_LIMIT: print("Depth limit reached") node.is_terminal = True return new_nodes = generate_new_states(node, args, task, args.n_generate_sample) node.children.extend(new_nodes)agential/cog/lats/prompts.py (4)
1-3: LGTM!The
STANDARD_PROMPTtemplate is well-defined and correctly implemented.
6-49: LGTM!The
COT_PROMPTtemplate is well-defined and correctly implemented.
51-98: LGTM!The
COT_PROMPT_FEEDBACKtemplate is well-defined and correctly implemented.
101-153: LGTM!The
REFLECTION_PROMPTtemplate is well-defined and correctly implemented.
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- agential/cog/lats/functional.py (1 hunks)
- agential/cog/lats/prompts.py (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- agential/cog/lats/functional.py
- agential/cog/lats/prompts.py
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- agential/cog/lats/functional.py (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- agential/cog/lats/functional.py
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- agential/cog/lats/functional.py (1 hunks)
- agential/cog/lats/tmp.ipynb (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- agential/cog/lats/functional.py
Additional comments not posted (5)
agential/cog/lats/tmp.ipynb (5)
20-22: Ensure environment variables are loaded securely.The
load_dotenvfunction is used to load environment variables. Ensure that sensitive information such as API keys is stored securely and not hard-coded.
24-33: Verify the usage of thestopparameter.The
stopparameter is set to".". Ensure that this is the intended behavior and that it does not conflict with the model's expected input.
53-53: LGTM! Verify the model initialization output.The
ChatOpenAImodel is correctly initialized. Ensure the output matches the expected configuration.
62-68: LGTM! Verify the message format and response retrieval.The message is correctly formatted and the response is retrieved. Ensure the response content matches the expected output.
99-99: LGTM! Verify the response output.The response from the
ChatOpenAImodel is correctly outputted. Ensure the output matches the expected response.
There was a problem hiding this comment.
Actionable comments posted: 3
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- agential/cog/lats/functional.py (1 hunks)
- agential/cog/lats/prompts.py (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- agential/cog/lats/functional.py
Additional comments not posted (3)
agential/cog/lats/prompts.py (3)
1-3: LGTM!The
LATS_INSTRUCTION_HOTPOTQAprompt is clear and well-structured.
156-276: LGTM!The
VALUE_PROMPT_REASONING_FEEDBACKprompt is clear and well-structured.
218-276: LGTM!The
VALUE_PROMPT_REASONINGprompt is clear and well-structured.
agential/cog/lats/prompts.py
Outdated
Comment on lines
6
to
49
| COT_PROMPT = """ | ||
| Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: | ||
| (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. | ||
| (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage. | ||
| (3) Finish[answer], which returns the answer and finishes the task. | ||
| After each observation, provide the next Thought and next Action. Here are some examples: | ||
|
|
||
| Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? | ||
| Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area. | ||
| Action 1: Search[Colorado orogeny] | ||
| Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas. This took place from 1780 to 1650 million years ago (Mya), during the Paleoproterozoic (Statherian Period). It is recorded in the Colorado orogen, a >500-km-wide belt of oceanic arc rock that extends southward into New Mexico. The Colorado orogeny was likely part of the larger Yavapai orogeny. | ||
| Thought 2: It does not mention the eastern sector of the Colorado orogeny. I need to look up eastern sector. | ||
| Action 2: Lookup[eastern sector] | ||
| Observation 2: (Result 1 / 1) The Colorado orogen, formerly called the Colorado province, is a >500-km-wide belt of oceanic arc rock (1.78\u20131.65 Ga) that extends southward into New Mexico and composes a major part of the Proterozoic provinces of southwestern United States. This transcontinental collisional event occurred during the Paleoproterozoic (Statherian Period).[1] The Wyoming sector of the Colorado orogeny was formerly called the Medicine Bow orogeny. The eastern sector extends into the High Plains and is called the Central Plains orogeny. The boundary between the Colorado orogeny and the Wyoming craton is the Cheyenne belt, a 5-km-wide mylonitic shear zone that verges northward. The Cheyenne belt transects and cuts off the south edge of the older Trans-Hudson orogeny.[2] | ||
| Thought 3: The eastern sector of Colorado orogeny extends into the High Plains, so High Plains is the area. I need to search High Plains and find its elevation range. | ||
| Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regions: | ||
| Thought 4: I need to search High Plains (United States). | ||
| Action 4: Search[High Plains (United States)] | ||
| Observation 4: The High Plains are a subregion of the Great Plains, mainly in the Western United States, but also partly in the Midwest states of Nebraska, Kansas, and South Dakota, generally encompassing the western part of the Great Plains before the region reaches the Rocky Mountains. The High Plains are located in eastern Montana, southeastern Wyoming, southwestern South Dakota, western Nebraska, eastern Colorado, western Kansas, eastern New Mexico, western Oklahoma, and to just south of the Texas Panhandle.[4] The southern region of the Western High Plains ecology region contains the geological formation known as Llano Estacado which can be seen from a short distance or on satellite maps.[5] From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3] | ||
| Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft. | ||
| Action 5: Finish[1,800 to 7,000 ft] | ||
|
|
||
| Question: Guitars for Wounded Warriors is an album that was recorded in the village in which New York county? | ||
| Thought 1: I need to search Guitars for Wounded Warriors, then find in which New York county it was recorded. | ||
| Action 1: Search[Guitars for Wounded Warriors] | ||
| Observation 1: Guitars for Wounded Warriors is the sixth solo studio album by guitarist Brian Tarquin, released in August 2014 by Cleopatra Records. In 2016, it received a Best Compilation Album nomination by the Independent Music Awards. All tracks were recorded at Tarquin's Jungle Room Studios in New Paltz (village), New York. Being moved by the lack of support for returning veterans through his life Tarquin decided to compose a heavy instrumental rock album as a way to show his appreciation to all veterans. So once again he enlisted top notch players to guest with him on the album, Billy Sheehan on Black Hawk, Ron \"Bumblefoot\" Thal and Reb Beach on Taliban Terror, Steve Morse on Freedom, Chris Poland on Charlie Surfs and Hunting, Chuck Loeb on Escape Kabul, Hal Lindes on Sand & Blood, Gary Hoey on 5 Klicks To Hell and Baghdad, Alex De Rosso Dokken on Broken Arrow, and The Flyin' Ryan Brothers on Surgical Strike. The entire album was engineered, produced and composed by Tarquin especially for each one of the guest guitarists. Partial proceeds are donated to the Fisher House Foundation from sales. | ||
| Thought 2: Guitars for Wounded Warriors was recorded in New Paltz (village), New York. I need to search New Paltz (village), New York and find which county it is in | ||
| Action 2: Search[New Paltz (village), New York] | ||
| Observation 2: New Paltz is a village in Ulster County located in the U.S. state of New York. It is approximately 80 miles (130 km) north of New York City and 70 miles (110 km) south of Albany. The population was 7,324 at the 2020 census.[3] | ||
| Thought 3: New Paltz (village), New York is in Ulster County, so the answer is Ulster County. | ||
| Action 3: Finish[Ulster County] | ||
|
|
||
| Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? | ||
| Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after. | ||
| Action 1: Search[Milhouse] | ||
| Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. | ||
| Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after". | ||
| Action 2: Lookup[named after] | ||
| Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. | ||
| Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon. | ||
| Action 3: Finish[President Richard Nixon] | ||
|
|
||
| {input} | ||
| """ |
There was a problem hiding this comment.
Fix the typo in the example section.
There is a missing newline character before "Observation 3: High Plains refers to one of two distinct land regions."
- Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regions:
+ Action 3: Search[High Plains]
+ Observation 3: High Plains refers to one of two distinct land regions:Committable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| COT_PROMPT = """ | |
| Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: | |
| (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. | |
| (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage. | |
| (3) Finish[answer], which returns the answer and finishes the task. | |
| After each observation, provide the next Thought and next Action. Here are some examples: | |
| Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? | |
| Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area. | |
| Action 1: Search[Colorado orogeny] | |
| Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas. This took place from 1780 to 1650 million years ago (Mya), during the Paleoproterozoic (Statherian Period). It is recorded in the Colorado orogen, a >500-km-wide belt of oceanic arc rock that extends southward into New Mexico. The Colorado orogeny was likely part of the larger Yavapai orogeny. | |
| Thought 2: It does not mention the eastern sector of the Colorado orogeny. I need to look up eastern sector. | |
| Action 2: Lookup[eastern sector] | |
| Observation 2: (Result 1 / 1) The Colorado orogen, formerly called the Colorado province, is a >500-km-wide belt of oceanic arc rock (1.78\u20131.65 Ga) that extends southward into New Mexico and composes a major part of the Proterozoic provinces of southwestern United States. This transcontinental collisional event occurred during the Paleoproterozoic (Statherian Period).[1] The Wyoming sector of the Colorado orogeny was formerly called the Medicine Bow orogeny. The eastern sector extends into the High Plains and is called the Central Plains orogeny. The boundary between the Colorado orogeny and the Wyoming craton is the Cheyenne belt, a 5-km-wide mylonitic shear zone that verges northward. The Cheyenne belt transects and cuts off the south edge of the older Trans-Hudson orogeny.[2] | |
| Thought 3: The eastern sector of Colorado orogeny extends into the High Plains, so High Plains is the area. I need to search High Plains and find its elevation range. | |
| Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regions: | |
| Thought 4: I need to search High Plains (United States). | |
| Action 4: Search[High Plains (United States)] | |
| Observation 4: The High Plains are a subregion of the Great Plains, mainly in the Western United States, but also partly in the Midwest states of Nebraska, Kansas, and South Dakota, generally encompassing the western part of the Great Plains before the region reaches the Rocky Mountains. The High Plains are located in eastern Montana, southeastern Wyoming, southwestern South Dakota, western Nebraska, eastern Colorado, western Kansas, eastern New Mexico, western Oklahoma, and to just south of the Texas Panhandle.[4] The southern region of the Western High Plains ecology region contains the geological formation known as Llano Estacado which can be seen from a short distance or on satellite maps.[5] From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3] | |
| Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft. | |
| Action 5: Finish[1,800 to 7,000 ft] | |
| Question: Guitars for Wounded Warriors is an album that was recorded in the village in which New York county? | |
| Thought 1: I need to search Guitars for Wounded Warriors, then find in which New York county it was recorded. | |
| Action 1: Search[Guitars for Wounded Warriors] | |
| Observation 1: Guitars for Wounded Warriors is the sixth solo studio album by guitarist Brian Tarquin, released in August 2014 by Cleopatra Records. In 2016, it received a Best Compilation Album nomination by the Independent Music Awards. All tracks were recorded at Tarquin's Jungle Room Studios in New Paltz (village), New York. Being moved by the lack of support for returning veterans through his life Tarquin decided to compose a heavy instrumental rock album as a way to show his appreciation to all veterans. So once again he enlisted top notch players to guest with him on the album, Billy Sheehan on Black Hawk, Ron \"Bumblefoot\" Thal and Reb Beach on Taliban Terror, Steve Morse on Freedom, Chris Poland on Charlie Surfs and Hunting, Chuck Loeb on Escape Kabul, Hal Lindes on Sand & Blood, Gary Hoey on 5 Klicks To Hell and Baghdad, Alex De Rosso Dokken on Broken Arrow, and The Flyin' Ryan Brothers on Surgical Strike. The entire album was engineered, produced and composed by Tarquin especially for each one of the guest guitarists. Partial proceeds are donated to the Fisher House Foundation from sales. | |
| Thought 2: Guitars for Wounded Warriors was recorded in New Paltz (village), New York. I need to search New Paltz (village), New York and find which county it is in | |
| Action 2: Search[New Paltz (village), New York] | |
| Observation 2: New Paltz is a village in Ulster County located in the U.S. state of New York. It is approximately 80 miles (130 km) north of New York City and 70 miles (110 km) south of Albany. The population was 7,324 at the 2020 census.[3] | |
| Thought 3: New Paltz (village), New York is in Ulster County, so the answer is Ulster County. | |
| Action 3: Finish[Ulster County] | |
| Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? | |
| Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after. | |
| Action 1: Search[Milhouse] | |
| Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. | |
| Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after". | |
| Action 2: Lookup[named after] | |
| Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. | |
| Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon. | |
| Action 3: Finish[President Richard Nixon] | |
| {input} | |
| """ | |
| COT_PROMPT = """ | |
| Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: | |
| (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. | |
| (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage. | |
| (3) Finish[answer], which returns the answer and finishes the task. | |
| After each observation, provide the next Thought and next Action. Here are some examples: | |
| Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? | |
| Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area. | |
| Action 1: Search[Colorado orogeny] | |
| Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas. This took place from 1780 to 1650 million years ago (Mya), during the Paleoproterozoic (Statherian Period). It is recorded in the Colorado orogen, a >500-km-wide belt of oceanic arc rock that extends southward into New Mexico. The Colorado orogeny was likely part of the larger Yavapai orogeny. | |
| Thought 2: It does not mention the eastern sector of the Colorado orogeny. I need to look up eastern sector. | |
| Action 2: Lookup[eastern sector] | |
| Observation 2: (Result 1 / 1) The Colorado orogen, formerly called the Colorado province, is a >500-km-wide belt of oceanic arc rock (1.78\u20131.65 Ga) that extends southward into New Mexico and composes a major part of the Proterozoic provinces of southwestern United States. This transcontinental collisional event occurred during the Paleoproterozoic (Statherian Period).[1] The Wyoming sector of the Colorado orogeny was formerly called the Medicine Bow orogeny. The eastern sector extends into the High Plains and is called the Central Plains orogeny. The boundary between the Colorado orogeny and the Wyoming craton is the Cheyenne belt, a 5-km-wide mylonitic shear zone that verges northward. The Cheyenne belt transects and cuts off the south edge of the older Trans-Hudson orogeny.[2] | |
| Thought 3: The eastern sector of Colorado orogeny extends into the High Plains, so High Plains is the area. I need to search High Plains and find its elevation range. | |
| Action 3: Search[High Plains] | |
| Observation 3: High Plains refers to one of two distinct land regions: | |
| Thought 4: I need to search High Plains (United States). | |
| Action 4: Search[High Plains (United States)] | |
| Observation 4: The High Plains are a subregion of the Great Plains, mainly in the Western United States, but also partly in the Midwest states of Nebraska, Kansas, and South Dakota, generally encompassing the western part of the Great Plains before the region reaches the Rocky Mountains. The High Plains are located in eastern Montana, southeastern Wyoming, southwestern South Dakota, western Nebraska, eastern Colorado, western Kansas, eastern New Mexico, western Oklahoma, and to just south of the Texas Panhandle.[4] The southern region of the Western High Plains ecology region contains the geological formation known as Llano Estacado which can be seen from a short distance or on satellite maps.[5] From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3] | |
| Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft. | |
| Action 5: Finish[1,800 to 7,000 ft] | |
| Question: Guitars for Wounded Warriors is an album that was recorded in the village in which New York county? | |
| Thought 1: I need to search Guitars for Wounded Warriors, then find in which New York county it was recorded. | |
| Action 1: Search[Guitars for Wounded Warriors] | |
| Observation 1: Guitars for Wounded Warriors is the sixth solo studio album by guitarist Brian Tarquin, released in August 2014 by Cleopatra Records. In 2016, it received a Best Compilation Album nomination by the Independent Music Awards. All tracks were recorded at Tarquin's Jungle Room Studios in New Paltz (village), New York. Being moved by the lack of support for returning veterans through his life Tarquin decided to compose a heavy instrumental rock album as a way to show his appreciation to all veterans. So once again he enlisted top notch players to guest with him on the album, Billy Sheehan on Black Hawk, Ron \"Bumblefoot\" Thal and Reb Beach on Taliban Terror, Steve Morse on Freedom, Chris Poland on Charlie Surfs and Hunting, Chuck Loeb on Escape Kabul, Hal Lindes on Sand & Blood, Gary Hoey on 5 Klicks To Hell and Baghdad, Alex De Rosso Dokken on Broken Arrow, and The Flyin' Ryan Brothers on Surgical Strike. The entire album was engineered, produced and composed by Tarquin especially for each one of the guest guitarists. Partial proceeds are donated to the Fisher House Foundation from sales. | |
| Thought 2: Guitars for Wounded Warriors was recorded in New Paltz (village), New York. I need to search New Paltz (village), New York and find which county it is in | |
| Action 2: Search[New Paltz (village), New York] | |
| Observation 2: New Paltz is a village in Ulster County located in the U.S. state of New York. It is approximately 80 miles (130 km) north of New York City and 70 miles (110 km) south of Albany. The population was 7,324 at the 2020 census.[3] | |
| Thought 3: New Paltz (village), New York is in Ulster County, so the answer is Ulster County. | |
| Action 3: Finish[Ulster County] | |
| Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? | |
| Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after. | |
| Action 1: Search[Milhouse] | |
| Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. | |
| Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after". | |
| Action 2: Lookup[named after] | |
| Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. | |
| Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon. | |
| Action 3: Finish[President Richard Nixon] | |
| {input} | |
| """ |
agential/cog/lats/prompts.py
Outdated
Comment on lines
51
to
98
| COT_PROMPT_FEEDBACK = '''You are also an advanced reasoning agent that can improve based on self refection. Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: | ||
| (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. | ||
| (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage. | ||
| (3) Finish[answer], which returns the answer and finishes the task. | ||
| After each observation, provide the next Thought and next Action. Here are some examples: | ||
|
|
||
| Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? | ||
| Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area. | ||
| Action 1: Search[Colorado orogeny] | ||
| Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas. This took place from 1780 to 1650 million years ago (Mya), during the Paleoproterozoic (Statherian Period). It is recorded in the Colorado orogen, a >500-km-wide belt of oceanic arc rock that extends southward into New Mexico. The Colorado orogeny was likely part of the larger Yavapai orogeny. | ||
| Thought 2: It does not mention the eastern sector of the Colorado orogeny. I need to look up eastern sector. | ||
| Action 2: Lookup[eastern sector] | ||
| Observation 2: (Result 1 / 1) The Colorado orogen, formerly called the Colorado province, is a >500-km-wide belt of oceanic arc rock (1.78\u20131.65 Ga) that extends southward into New Mexico and composes a major part of the Proterozoic provinces of southwestern United States. This transcontinental collisional event occurred during the Paleoproterozoic (Statherian Period).[1] The Wyoming sector of the Colorado orogeny was formerly called the Medicine Bow orogeny. The eastern sector extends into the High Plains and is called the Central Plains orogeny. The boundary between the Colorado orogeny and the Wyoming craton is the Cheyenne belt, a 5-km-wide mylonitic shear zone that verges northward. The Cheyenne belt transects and cuts off the south edge of the older Trans-Hudson orogeny.[2] | ||
| Thought 3: The eastern sector of Colorado orogeny extends into the High Plains, so High Plains is the area. I need to search High Plains and find its elevation range. | ||
| Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regions: | ||
| Thought 4: I need to search High Plains (United States). | ||
| Action 4: Search[High Plains (United States)] | ||
| Observation 4: The High Plains are a subregion of the Great Plains, mainly in the Western United States, but also partly in the Midwest states of Nebraska, Kansas, and South Dakota, generally encompassing the western part of the Great Plains before the region reaches the Rocky Mountains. The High Plains are located in eastern Montana, southeastern Wyoming, southwestern South Dakota, western Nebraska, eastern Colorado, western Kansas, eastern New Mexico, western Oklahoma, and to just south of the Texas Panhandle.[4] The southern region of the Western High Plains ecology region contains the geological formation known as Llano Estacado which can be seen from a short distance or on satellite maps.[5] From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3] | ||
| Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft. | ||
| Action 5: Finish[1,800 to 7,000 ft] | ||
|
|
||
| Question: Guitars for Wounded Warriors is an album that was recorded in the village in which New York county? | ||
| Thought 1: I need to search Guitars for Wounded Warriors, then find in which New York county it was recorded. | ||
| Action 1: Search[Guitars for Wounded Warriors] | ||
| Observation 1: Guitars for Wounded Warriors is the sixth solo studio album by guitarist Brian Tarquin, released in August 2014 by Cleopatra Records. In 2016, it received a Best Compilation Album nomination by the Independent Music Awards. All tracks were recorded at Tarquin's Jungle Room Studios in New Paltz (village), New York. Being moved by the lack of support for returning veterans through his life Tarquin decided to compose a heavy instrumental rock album as a way to show his appreciation to all veterans. So once again he enlisted top notch players to guest with him on the album, Billy Sheehan on Black Hawk, Ron \"Bumblefoot\" Thal and Reb Beach on Taliban Terror, Steve Morse on Freedom, Chris Poland on Charlie Surfs and Hunting, Chuck Loeb on Escape Kabul, Hal Lindes on Sand & Blood, Gary Hoey on 5 Klicks To Hell and Baghdad, Alex De Rosso Dokken on Broken Arrow, and The Flyin' Ryan Brothers on Surgical Strike. The entire album was engineered, produced and composed by Tarquin especially for each one of the guest guitarists. Partial proceeds are donated to the Fisher House Foundation from sales. | ||
| Thought 2: Guitars for Wounded Warriors was recorded in New Paltz (village), New York. I need to search New Paltz (village), New York and find which county it is in | ||
| Action 2: Search[New Paltz (village), New York] | ||
| Observation 2: New Paltz is a village in Ulster County located in the U.S. state of New York. It is approximately 80 miles (130 km) north of New York City and 70 miles (110 km) south of Albany. The population was 7,324 at the 2020 census.[3] | ||
| Thought 3: New Paltz (village), New York is in Ulster County, so the answer is Ulster County. | ||
| Action 3: Finish[Ulster County] | ||
|
|
||
| Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? | ||
| Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after. | ||
| Action 1: Search[Milhouse] | ||
| Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. | ||
| Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after". | ||
| Action 2: Lookup[named after] | ||
| Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. | ||
| Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon. | ||
| Action 3: Finish[President Richard Nixon] | ||
|
|
||
| You have attempted to answer the following question before and failed, either because your reasoning for the answer was incorrect or the phrasing of your response did not exactly match the answer. The following reflection(s) give a plan to avoid failing to answer the question in the same way you did previously. Use them to improve your strategy of correctly answering the given question. | ||
|
|
||
| {trajectories} | ||
| When providing the thought and action for the current trial, that into account these failed trajectories and make sure not to repeat the same mistakes and incorrect answers. | ||
|
|
||
| {input} | ||
| ''' |
There was a problem hiding this comment.
Fix the typo in the feedback section.
There is a typo where "self refection" should be "self reflection."
- You are also an advanced reasoning agent that can improve based on self refection.
+ You are also an advanced reasoning agent that can improve based on self reflection.Committable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| COT_PROMPT_FEEDBACK = '''You are also an advanced reasoning agent that can improve based on self refection. Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: | |
| (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. | |
| (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage. | |
| (3) Finish[answer], which returns the answer and finishes the task. | |
| After each observation, provide the next Thought and next Action. Here are some examples: | |
| Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? | |
| Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area. | |
| Action 1: Search[Colorado orogeny] | |
| Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas. This took place from 1780 to 1650 million years ago (Mya), during the Paleoproterozoic (Statherian Period). It is recorded in the Colorado orogen, a >500-km-wide belt of oceanic arc rock that extends southward into New Mexico. The Colorado orogeny was likely part of the larger Yavapai orogeny. | |
| Thought 2: It does not mention the eastern sector of the Colorado orogeny. I need to look up eastern sector. | |
| Action 2: Lookup[eastern sector] | |
| Observation 2: (Result 1 / 1) The Colorado orogen, formerly called the Colorado province, is a >500-km-wide belt of oceanic arc rock (1.78\u20131.65 Ga) that extends southward into New Mexico and composes a major part of the Proterozoic provinces of southwestern United States. This transcontinental collisional event occurred during the Paleoproterozoic (Statherian Period).[1] The Wyoming sector of the Colorado orogeny was formerly called the Medicine Bow orogeny. The eastern sector extends into the High Plains and is called the Central Plains orogeny. The boundary between the Colorado orogeny and the Wyoming craton is the Cheyenne belt, a 5-km-wide mylonitic shear zone that verges northward. The Cheyenne belt transects and cuts off the south edge of the older Trans-Hudson orogeny.[2] | |
| Thought 3: The eastern sector of Colorado orogeny extends into the High Plains, so High Plains is the area. I need to search High Plains and find its elevation range. | |
| Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regions: | |
| Thought 4: I need to search High Plains (United States). | |
| Action 4: Search[High Plains (United States)] | |
| Observation 4: The High Plains are a subregion of the Great Plains, mainly in the Western United States, but also partly in the Midwest states of Nebraska, Kansas, and South Dakota, generally encompassing the western part of the Great Plains before the region reaches the Rocky Mountains. The High Plains are located in eastern Montana, southeastern Wyoming, southwestern South Dakota, western Nebraska, eastern Colorado, western Kansas, eastern New Mexico, western Oklahoma, and to just south of the Texas Panhandle.[4] The southern region of the Western High Plains ecology region contains the geological formation known as Llano Estacado which can be seen from a short distance or on satellite maps.[5] From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3] | |
| Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft. | |
| Action 5: Finish[1,800 to 7,000 ft] | |
| Question: Guitars for Wounded Warriors is an album that was recorded in the village in which New York county? | |
| Thought 1: I need to search Guitars for Wounded Warriors, then find in which New York county it was recorded. | |
| Action 1: Search[Guitars for Wounded Warriors] | |
| Observation 1: Guitars for Wounded Warriors is the sixth solo studio album by guitarist Brian Tarquin, released in August 2014 by Cleopatra Records. In 2016, it received a Best Compilation Album nomination by the Independent Music Awards. All tracks were recorded at Tarquin's Jungle Room Studios in New Paltz (village), New York. Being moved by the lack of support for returning veterans through his life Tarquin decided to compose a heavy instrumental rock album as a way to show his appreciation to all veterans. So once again he enlisted top notch players to guest with him on the album, Billy Sheehan on Black Hawk, Ron \"Bumblefoot\" Thal and Reb Beach on Taliban Terror, Steve Morse on Freedom, Chris Poland on Charlie Surfs and Hunting, Chuck Loeb on Escape Kabul, Hal Lindes on Sand & Blood, Gary Hoey on 5 Klicks To Hell and Baghdad, Alex De Rosso Dokken on Broken Arrow, and The Flyin' Ryan Brothers on Surgical Strike. The entire album was engineered, produced and composed by Tarquin especially for each one of the guest guitarists. Partial proceeds are donated to the Fisher House Foundation from sales. | |
| Thought 2: Guitars for Wounded Warriors was recorded in New Paltz (village), New York. I need to search New Paltz (village), New York and find which county it is in | |
| Action 2: Search[New Paltz (village), New York] | |
| Observation 2: New Paltz is a village in Ulster County located in the U.S. state of New York. It is approximately 80 miles (130 km) north of New York City and 70 miles (110 km) south of Albany. The population was 7,324 at the 2020 census.[3] | |
| Thought 3: New Paltz (village), New York is in Ulster County, so the answer is Ulster County. | |
| Action 3: Finish[Ulster County] | |
| Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? | |
| Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after. | |
| Action 1: Search[Milhouse] | |
| Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. | |
| Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after". | |
| Action 2: Lookup[named after] | |
| Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. | |
| Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon. | |
| Action 3: Finish[President Richard Nixon] | |
| You have attempted to answer the following question before and failed, either because your reasoning for the answer was incorrect or the phrasing of your response did not exactly match the answer. The following reflection(s) give a plan to avoid failing to answer the question in the same way you did previously. Use them to improve your strategy of correctly answering the given question. | |
| {trajectories} | |
| When providing the thought and action for the current trial, that into account these failed trajectories and make sure not to repeat the same mistakes and incorrect answers. | |
| {input} | |
| ''' | |
| COT_PROMPT_FEEDBACK = '''You are also an advanced reasoning agent that can improve based on self reflection. Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: | |
| (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. | |
| (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage. | |
| (3) Finish[answer], which returns the answer and finishes the task. | |
| After each observation, provide the next Thought and next Action. Here are some examples: | |
| Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? | |
| Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area. | |
| Action 1: Search[Colorado orogeny] | |
| Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas. This took place from 1780 to 1650 million years ago (Mya), during the Paleoproterozoic (Statherian Period). It is recorded in the Colorado orogen, a >500-km-wide belt of oceanic arc rock that extends southward into New Mexico. The Colorado orogeny was likely part of the larger Yavapai orogeny. | |
| Thought 2: It does not mention the eastern sector of the Colorado orogeny. I need to look up eastern sector. | |
| Action 2: Lookup[eastern sector] | |
| Observation 2: (Result 1 / 1) The Colorado orogen, formerly called the Colorado province, is a >500-km-wide belt of oceanic arc rock (1.78\u20131.65 Ga) that extends southward into New Mexico and composes a major part of the Proterozoic provinces of southwestern United States. This transcontinental collisional event occurred during the Paleoproterozoic (Statherian Period).[1] The Wyoming sector of the Colorado orogeny was formerly called the Medicine Bow orogeny. The eastern sector extends into the High Plains and is called the Central Plains orogeny. The boundary between the Colorado orogeny and the Wyoming craton is the Cheyenne belt, a 5-km-wide mylonitic shear zone that verges northward. The Cheyenne belt transects and cuts off the south edge of the older Trans-Hudson orogeny.[2] | |
| Thought 3: The eastern sector of Colorado orogeny extends into the High Plains, so High Plains is the area. I need to search High Plains and find its elevation range. | |
| Action 3: Search[High Plains]Observation 3: High Plains refers to one of two distinct land regions: | |
| Thought 4: I need to search High Plains (United States). | |
| Action 4: Search[High Plains (United States)] | |
| Observation 4: The High Plains are a subregion of the Great Plains, mainly in the Western United States, but also partly in the Midwest states of Nebraska, Kansas, and South Dakota, generally encompassing the western part of the Great Plains before the region reaches the Rocky Mountains. The High Plains are located in eastern Montana, southeastern Wyoming, southwestern South Dakota, western Nebraska, eastern Colorado, western Kansas, eastern New Mexico, western Oklahoma, and to just south of the Texas Panhandle.[4] The southern region of the Western High Plains ecology region contains the geological formation known as Llano Estacado which can be seen from a short distance or on satellite maps.[5] From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3] | |
| Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft. | |
| Action 5: Finish[1,800 to 7,000 ft] | |
| Question: Guitars for Wounded Warriors is an album that was recorded in the village in which New York county? | |
| Thought 1: I need to search Guitars for Wounded Warriors, then find in which New York county it was recorded. | |
| Action 1: Search[Guitars for Wounded Warriors] | |
| Observation 1: Guitars for Wounded Warriors is the sixth solo studio album by guitarist Brian Tarquin, released in August 2014 by Cleopatra Records. In 2016, it received a Best Compilation Album nomination by the Independent Music Awards. All tracks were recorded at Tarquin's Jungle Room Studios in New Paltz (village), New York. Being moved by the lack of support for returning veterans through his life Tarquin decided to compose a heavy instrumental rock album as a way to show his appreciation to all veterans. So once again he enlisted top notch players to guest with him on the album, Billy Sheehan on Black Hawk, Ron \"Bumblefoot\" Thal and Reb Beach on Taliban Terror, Steve Morse on Freedom, Chris Poland on Charlie Surfs and Hunting, Chuck Loeb on Escape Kabul, Hal Lindes on Sand & Blood, Gary Hoey on 5 Klicks To Hell and Baghdad, Alex De Rosso Dokken on Broken Arrow, and The Flyin' Ryan Brothers on Surgical Strike. The entire album was engineered, produced and composed by Tarquin especially for each one of the guest guitarists. Partial proceeds are donated to the Fisher House Foundation from sales. | |
| Thought 2: Guitars for Wounded Warriors was recorded in New Paltz (village), New York. I need to search New Paltz (village), New York and find which county it is in | |
| Action 2: Search[New Paltz (village), New York] | |
| Observation 2: New Paltz is a village in Ulster County located in the U.S. state of New York. It is approximately 80 miles (130 km) north of New York City and 70 miles (110 km) south of Albany. The population was 7,324 at the 2020 census.[3] | |
| Thought 3: New Paltz (village), New York is in Ulster County, so the answer is Ulster County. | |
| Action 3: Finish[Ulster County] | |
| Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? | |
| Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after. | |
| Action 1: Search[Milhouse] | |
| Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. | |
| Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after". | |
| Action 2: Lookup[named after] | |
| Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. | |
| Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon. | |
| Action 3: Finish[President Richard Nixon] | |
| You have attempted to answer the following question before and failed, either because your reasoning for the answer was incorrect or the phrasing of your response did not exactly match the answer. The following reflection(s) give a plan to avoid failing to answer the question in the same way you did previously. Use them to improve your strategy of correctly answering the given question. | |
| {trajectories} | |
| When providing the thought and action for the current trial, that into account these failed trajectories and make sure not to repeat the same mistakes and incorrect answers. | |
| {input} | |
| ''' |
agential/cog/lats/prompts.py
Outdated
Comment on lines
101
to
153
| REFLECTION_PROMPT = '''You are an advanced reasoning agent that can improve based on self refection. You will be given a previous reasoning trial in which you were given access to an Docstore API environment and a question to answer. You were unsuccessful in answering the question either because you guessed the wrong answer with Finish[<answer>], or you used up your set number of reasoning steps. In a few sentences, Diagnose a possible reason for failure and devise a new, concise, high level plan that aims to mitigate the same failure. Use complete sentences. | ||
| Here are some examples: | ||
| Previous Trial: | ||
| Question: The Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of what? | ||
| Thought 1: I need to search Rome Protocols, find the three Prime Ministers, then find what they were assassinated as part of. | ||
| Action 1: Search[Rome Protocols] | ||
| Observation 1: The Rome Protocols were a series of three international agreements concluded in Rome on 17 March 1934 between the governments of Austria, Hungary and Italy. They were signed by Italian Prime Minister Benito Mussolini, Austrian Chancellor Engelbert Dollfuss and Hungarian Prime Minister Gyula GΓΆmbΓΆs. All of the protocols went into effect on 12 July 1934 and were registered in League of Nations Treaty Series on 12 December 1934. | ||
| Thought 2: The three Prime Ministers were Benito Mussolini, Engelbert Dollfuss, and Gyula GΓΆmbΓΆs. I need to search Benito Mussolini and find what he was assassinated as part of. | ||
| Action 2: Search[Benito Mussolini] | ||
| Observation 2: Benito Amilcare Andrea Mussolini (UK: MU(U)SS-Ι-LEE-nee, US: MOOSS-, Italian: [beΛniΛto aΛmilkare anΛdrΙΛa mussoΛliΛni]; 29 July 1883 β 28 April 1945) was an Italian politician and journalist who founded and led the National Fascist Party (PNF). He was Prime Minister of Italy from the March on Rome in 1922 until his deposition in 1943, as well as "Duce" of Italian fascism from the establishment of the Italian Fasces of Combat in 1919 until his summary execution in 1945 by Italian partisans. As dictator of Italy and principal founder of fascism, Mussolini inspired and supported the international spread of fascist movements during the inter-war period.Mussolini was originally a socialist politician and a journalist at the Avanti! newspaper. In 1912, he became a member of the National Directorate of the Italian Socialist Party (PSI), but he was expelled from the PSI for advocating military intervention in World War I, in opposition to the party's stance on neutrality. In 1914, Mussolini founded a new journal, Il Popolo d'Italia, and served in the Royal Italian Army during the war until he was wounded and discharged in 1917. Mussolini denounced the PSI, his views now centering on Italian nationalism instead of socialism, and later founded the fascist movement which came to oppose egalitarianism and class conflict, instead advocating "revolutionary nationalism" transcending class lines. On 31 October 1922, following the March on Rome (28β30 October), Mussolini was appointed prime minister by King Victor Emmanuel III, becoming the youngest individual to hold the office up to that time. After removing all political opposition through his secret police and outlawing labor strikes, Mussolini and his followers consolidated power through a series of laws that transformed the nation into a one-party dictatorship. Within five years, Mussolini had established dictatorial authority by both legal and illegal means and aspired to create a totalitarian state. In 1929, Mussolini signed the Lateran Treaty with the Holy See to establish Vatican City. | ||
| Mussolini's foreign policy aimed to restore the ancient grandeur of the Roman Empire by expanding Italian colonial possessions and the fascist sphere of influence. In the 1920s, he ordered the Pacification of Libya, instructed the bombing of Corfu over an incident with Greece, established a protectorate over Albania, and incorporated the city of Fiume into the Italian state via agreements with Yugoslavia. In 1936, Ethiopia was conquered following the Second Italo-Ethiopian War and merged into Italian East Africa (AOI) with Eritrea and Somalia. In 1939, Italian forces annexed Albania. Between 1936 and 1939, Mussolini ordered the successful Italian military intervention in Spain in favor of Francisco Franco during the Spanish Civil War. Mussolini's Italy initially tried to avoid the outbreak of a second global war, sending troops at the Brenner Pass to delay Anschluss and taking part in the Stresa Front, the Lytton Report, the Treaty of Lausanne, the Four-Power Pact and the Munich Agreement. However, Italy then alienated itself from Britain and France by aligning with Germany and Japan. Germany invaded Poland on 1 September 1939, resulting in declarations of war by France and the UK and the start of World War II. | ||
| On 10 June 1940, Mussolini decided to enter the war on the Axis side. Despite initial success, the subsequent Axis collapse on multiple fronts and eventual Allied invasion of Sicily made Mussolini lose the support of the population and members of the Fascist Party. As a consequence, early on 25 July 1943, the Grand Council of Fascism passed a motion of no confidence in Mussolini; later that day King Victor Emmanuel III dismissed him as head of government and had him placed in custody, appointing Pietro Badoglio to succeed him as Prime Minister. After the king agreed to an armistice with the Allies, on 12 September 1943 Mussolini was rescued from captivity in the Gran Sasso raid by German paratroopers and Waffen-SS commandos led by Major Otto-Harald Mors. Adolf Hitler, after meeting with the rescued former dictator, then put Mussolini in charge of a puppet regime in northern Italy, the Italian Social Republic (Italian: Repubblica Sociale Italiana, RSI), informally known as the SalΓ² Republic, causing a civil war. In late April 1945, in the wake of near total defeat, Mussolini and his mistress Clara Petacci attempted to flee to Switzerland, but both were captured by Italian communist partisans and summarily executed by firing squad on 28 April 1945 near Lake Como. The bodies of Mussolini and his mistress were then taken to Milan, where they were hung upside down at a service station to publicly confirm their demise. | ||
| Thought 3: Benito Mussolini was assassinated as part of the collapse of the Axis powers during World War II. So the Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of World War II. | ||
| Action 3: Finish[World War II] | ||
|
|
||
| Reflection: I searched one of the prime ministers involved in the signing, then attemted to answer right away. I should have searched each of the prime ministers, then looked up 'death' on each of their pages in order to get more information before answering. | ||
|
|
||
| Previous Trial: | ||
| Question: Kam Heskin plays Paige Morgan in a 2004 film directed by who? | ||
| Thought 1: I need to search Kam Heskin and Paige Morgan, find the 2004 film they are in, then find the director of the film. | ||
| Action 1: Search[Kam Heskin] | ||
| Observation 1: Kam Erika Heskin (born May 8, 1973) is an American actress. She began her career playing Caitlin Deschanel on the NBC daytime soap opera Sunset Beach (1998β1999), before appearing in films Planet of the Apes (2001) and Catch Me If You Can (2002). Heskin went on in 2003 to play Elizabeth Bennet in an independent film Pride & Prejudice: A Latter-Day Comedy, Robin in Charmed (2004), and Paige Morgan in The Prince and Me film franchise (2006β2010). | ||
| Thought 2: Kam Heskin plays Paige Morgan in The Prince and Me film franchise. I need to search The Prince and Me film franchise and find the director of the 2004 film. | ||
| Action 2: Search[The Prince and Me film franchise] | ||
| Observation 2: Could not find [The Prince and Me film franchise]. Similar: ['Kam Heskin', 'Prince of Persia', 'Harry Potter and the Half-Blood Prince (film)', 'Prince of Persia: The Sands of Time (film)', 'The Chronicles of Narnia (film series)', 'Majestic Prince (franchise)', 'Kingsman (franchise)', 'The Little Prince (2015 film)', 'Shrek (franchise)', 'Harry Potter (film series)'] | ||
| Thought 3: I need to search The Prince and Me (2004 film) instead. | ||
| Action 3: Search[The Prince and Me (2004 film)] | ||
| Observation 3: Could not find [The Prince and Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Kam Heskin', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince of Persia', 'Brooklynn Prince', 'Prince (musician)'] | ||
| Thought 4: I need to search The Prince & Me (2004 film). | ||
| Action 4: Search[The Prince & Me (2004 film)] | ||
| Observation 4: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | ||
| Thought 5: I need to search The Prince & Me (2004 film) instead. | ||
| Action 5: Search[The Prince & Me (2004 film)] | ||
| Observation 5: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | ||
| Thought 6: I need to search The Prince & Me (2004 film). | ||
| Action 6: Search[The Prince & Me (2004 film)] | ||
| Observation 6: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | ||
|
|
||
| Reflection: I got stuck in a loop where I kept trying to search 'The Prince & Me (2004 film)' but the page could not be found. Instead I should have tried to search the similar results that had a similar name to see and they were made in 2004. | ||
|
|
||
| Previous Trial: | ||
| Question: The Oberoi family is part of a hotel company that has a head office in what city? | ||
| Thought 1: I need to search the Oberoi family and find the city where the hotel company's head office is located. | ||
| Action 1: Search[Oberoi family] | ||
| Observation 1: The Oberoi Group is a luxury hotel group with its head office in New Delhi, India.[2] Founded in 1934, the company owns and operates 32 luxury hotels and two river cruise ships in 7 countries, primarily under its Oberoi Hotels & Resorts and Trident brands.[3] The group also operates The Oberoi Centre for Learning and Development, which is regarded as one of Asia's top institutions for hospitality education.[4]. The foundations of the Oberoi Group date back to 1934 when The Rai Bahadur Mohan Singh Oberoi, the founder of the group, bought two properties: the Maidens in Delhi and the Clarke's in Shimla.[5] In the following years Oberoi, assisted by his two sons, Tilak Raj Singh Oberoi and Prithvi Raj Singh Oberoi (P.R.S. Oberoi | ||
| Thought 2: The Oberoi Group has its head office in New Delhi, India. I just need to finish with New Delhi. | ||
| Action 2: Finish[New Delhi] | ||
| Observation 2: Episode finished, reward = 0 | ||
|
|
||
| Reflection: I appeared to have retrieved the correct information about The Oberoi Family and the location of it's head office, and provided a corresponding answer. However this answer does not exactly match the ground truth answer so I should try a different wording, such as Delhi. | ||
|
|
||
| Previous trial: | ||
| {trajectory}Reflection:''' |
There was a problem hiding this comment.
Fix the typo in the reflection section.
There is a typo where "attemted" should be "attempted."
- Reflection: I searched one of the prime ministers involved in the signing, then attemted to answer right away.
+ Reflection: I searched one of the prime ministers involved in the signing, then attempted to answer right away.Committable suggestion
βΌοΈ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| REFLECTION_PROMPT = '''You are an advanced reasoning agent that can improve based on self refection. You will be given a previous reasoning trial in which you were given access to an Docstore API environment and a question to answer. You were unsuccessful in answering the question either because you guessed the wrong answer with Finish[<answer>], or you used up your set number of reasoning steps. In a few sentences, Diagnose a possible reason for failure and devise a new, concise, high level plan that aims to mitigate the same failure. Use complete sentences. | |
| Here are some examples: | |
| Previous Trial: | |
| Question: The Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of what? | |
| Thought 1: I need to search Rome Protocols, find the three Prime Ministers, then find what they were assassinated as part of. | |
| Action 1: Search[Rome Protocols] | |
| Observation 1: The Rome Protocols were a series of three international agreements concluded in Rome on 17 March 1934 between the governments of Austria, Hungary and Italy. They were signed by Italian Prime Minister Benito Mussolini, Austrian Chancellor Engelbert Dollfuss and Hungarian Prime Minister Gyula GΓΆmbΓΆs. All of the protocols went into effect on 12 July 1934 and were registered in League of Nations Treaty Series on 12 December 1934. | |
| Thought 2: The three Prime Ministers were Benito Mussolini, Engelbert Dollfuss, and Gyula GΓΆmbΓΆs. I need to search Benito Mussolini and find what he was assassinated as part of. | |
| Action 2: Search[Benito Mussolini] | |
| Observation 2: Benito Amilcare Andrea Mussolini (UK: MU(U)SS-Ι-LEE-nee, US: MOOSS-, Italian: [beΛniΛto aΛmilkare anΛdrΙΛa mussoΛliΛni]; 29 July 1883 β 28 April 1945) was an Italian politician and journalist who founded and led the National Fascist Party (PNF). He was Prime Minister of Italy from the March on Rome in 1922 until his deposition in 1943, as well as "Duce" of Italian fascism from the establishment of the Italian Fasces of Combat in 1919 until his summary execution in 1945 by Italian partisans. As dictator of Italy and principal founder of fascism, Mussolini inspired and supported the international spread of fascist movements during the inter-war period.Mussolini was originally a socialist politician and a journalist at the Avanti! newspaper. In 1912, he became a member of the National Directorate of the Italian Socialist Party (PSI), but he was expelled from the PSI for advocating military intervention in World War I, in opposition to the party's stance on neutrality. In 1914, Mussolini founded a new journal, Il Popolo d'Italia, and served in the Royal Italian Army during the war until he was wounded and discharged in 1917. Mussolini denounced the PSI, his views now centering on Italian nationalism instead of socialism, and later founded the fascist movement which came to oppose egalitarianism and class conflict, instead advocating "revolutionary nationalism" transcending class lines. On 31 October 1922, following the March on Rome (28β30 October), Mussolini was appointed prime minister by King Victor Emmanuel III, becoming the youngest individual to hold the office up to that time. After removing all political opposition through his secret police and outlawing labor strikes, Mussolini and his followers consolidated power through a series of laws that transformed the nation into a one-party dictatorship. Within five years, Mussolini had established dictatorial authority by both legal and illegal means and aspired to create a totalitarian state. In 1929, Mussolini signed the Lateran Treaty with the Holy See to establish Vatican City. | |
| Mussolini's foreign policy aimed to restore the ancient grandeur of the Roman Empire by expanding Italian colonial possessions and the fascist sphere of influence. In the 1920s, he ordered the Pacification of Libya, instructed the bombing of Corfu over an incident with Greece, established a protectorate over Albania, and incorporated the city of Fiume into the Italian state via agreements with Yugoslavia. In 1936, Ethiopia was conquered following the Second Italo-Ethiopian War and merged into Italian East Africa (AOI) with Eritrea and Somalia. In 1939, Italian forces annexed Albania. Between 1936 and 1939, Mussolini ordered the successful Italian military intervention in Spain in favor of Francisco Franco during the Spanish Civil War. Mussolini's Italy initially tried to avoid the outbreak of a second global war, sending troops at the Brenner Pass to delay Anschluss and taking part in the Stresa Front, the Lytton Report, the Treaty of Lausanne, the Four-Power Pact and the Munich Agreement. However, Italy then alienated itself from Britain and France by aligning with Germany and Japan. Germany invaded Poland on 1 September 1939, resulting in declarations of war by France and the UK and the start of World War II. | |
| On 10 June 1940, Mussolini decided to enter the war on the Axis side. Despite initial success, the subsequent Axis collapse on multiple fronts and eventual Allied invasion of Sicily made Mussolini lose the support of the population and members of the Fascist Party. As a consequence, early on 25 July 1943, the Grand Council of Fascism passed a motion of no confidence in Mussolini; later that day King Victor Emmanuel III dismissed him as head of government and had him placed in custody, appointing Pietro Badoglio to succeed him as Prime Minister. After the king agreed to an armistice with the Allies, on 12 September 1943 Mussolini was rescued from captivity in the Gran Sasso raid by German paratroopers and Waffen-SS commandos led by Major Otto-Harald Mors. Adolf Hitler, after meeting with the rescued former dictator, then put Mussolini in charge of a puppet regime in northern Italy, the Italian Social Republic (Italian: Repubblica Sociale Italiana, RSI), informally known as the SalΓ² Republic, causing a civil war. In late April 1945, in the wake of near total defeat, Mussolini and his mistress Clara Petacci attempted to flee to Switzerland, but both were captured by Italian communist partisans and summarily executed by firing squad on 28 April 1945 near Lake Como. The bodies of Mussolini and his mistress were then taken to Milan, where they were hung upside down at a service station to publicly confirm their demise. | |
| Thought 3: Benito Mussolini was assassinated as part of the collapse of the Axis powers during World War II. So the Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of World War II. | |
| Action 3: Finish[World War II] | |
| Reflection: I searched one of the prime ministers involved in the signing, then attemted to answer right away. I should have searched each of the prime ministers, then looked up 'death' on each of their pages in order to get more information before answering. | |
| Previous Trial: | |
| Question: Kam Heskin plays Paige Morgan in a 2004 film directed by who? | |
| Thought 1: I need to search Kam Heskin and Paige Morgan, find the 2004 film they are in, then find the director of the film. | |
| Action 1: Search[Kam Heskin] | |
| Observation 1: Kam Erika Heskin (born May 8, 1973) is an American actress. She began her career playing Caitlin Deschanel on the NBC daytime soap opera Sunset Beach (1998β1999), before appearing in films Planet of the Apes (2001) and Catch Me If You Can (2002). Heskin went on in 2003 to play Elizabeth Bennet in an independent film Pride & Prejudice: A Latter-Day Comedy, Robin in Charmed (2004), and Paige Morgan in The Prince and Me film franchise (2006β2010). | |
| Thought 2: Kam Heskin plays Paige Morgan in The Prince and Me film franchise. I need to search The Prince and Me film franchise and find the director of the 2004 film. | |
| Action 2: Search[The Prince and Me film franchise] | |
| Observation 2: Could not find [The Prince and Me film franchise]. Similar: ['Kam Heskin', 'Prince of Persia', 'Harry Potter and the Half-Blood Prince (film)', 'Prince of Persia: The Sands of Time (film)', 'The Chronicles of Narnia (film series)', 'Majestic Prince (franchise)', 'Kingsman (franchise)', 'The Little Prince (2015 film)', 'Shrek (franchise)', 'Harry Potter (film series)'] | |
| Thought 3: I need to search The Prince and Me (2004 film) instead. | |
| Action 3: Search[The Prince and Me (2004 film)] | |
| Observation 3: Could not find [The Prince and Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Kam Heskin', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince of Persia', 'Brooklynn Prince', 'Prince (musician)'] | |
| Thought 4: I need to search The Prince & Me (2004 film). | |
| Action 4: Search[The Prince & Me (2004 film)] | |
| Observation 4: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | |
| Thought 5: I need to search The Prince & Me (2004 film) instead. | |
| Action 5: Search[The Prince & Me (2004 film)] | |
| Observation 5: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | |
| Thought 6: I need to search The Prince & Me (2004 film). | |
| Action 6: Search[The Prince & Me (2004 film)] | |
| Observation 6: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | |
| Reflection: I got stuck in a loop where I kept trying to search 'The Prince & Me (2004 film)' but the page could not be found. Instead I should have tried to search the similar results that had a similar name to see and they were made in 2004. | |
| Previous Trial: | |
| Question: The Oberoi family is part of a hotel company that has a head office in what city? | |
| Thought 1: I need to search the Oberoi family and find the city where the hotel company's head office is located. | |
| Action 1: Search[Oberoi family] | |
| Observation 1: The Oberoi Group is a luxury hotel group with its head office in New Delhi, India.[2] Founded in 1934, the company owns and operates 32 luxury hotels and two river cruise ships in 7 countries, primarily under its Oberoi Hotels & Resorts and Trident brands.[3] The group also operates The Oberoi Centre for Learning and Development, which is regarded as one of Asia's top institutions for hospitality education.[4]. The foundations of the Oberoi Group date back to 1934 when The Rai Bahadur Mohan Singh Oberoi, the founder of the group, bought two properties: the Maidens in Delhi and the Clarke's in Shimla.[5] In the following years Oberoi, assisted by his two sons, Tilak Raj Singh Oberoi and Prithvi Raj Singh Oberoi (P.R.S. Oberoi | |
| Thought 2: The Oberoi Group has its head office in New Delhi, India. I just need to finish with New Delhi. | |
| Action 2: Finish[New Delhi] | |
| Observation 2: Episode finished, reward = 0 | |
| Reflection: I appeared to have retrieved the correct information about The Oberoi Family and the location of it's head office, and provided a corresponding answer. However this answer does not exactly match the ground truth answer so I should try a different wording, such as Delhi. | |
| Previous trial: | |
| {trajectory}Reflection:''' | |
| REFLECTION_PROMPT = '''You are an advanced reasoning agent that can improve based on self refection. You will be given a previous reasoning trial in which you were given access to an Docstore API environment and a question to answer. You were unsuccessful in answering the question either because you guessed the wrong answer with Finish[<answer>], or you used up your set number of reasoning steps. In a few sentences, Diagnose a possible reason for failure and devise a new, concise, high level plan that aims to mitigate the same failure. Use complete sentences. | |
| Here are some examples: | |
| Previous Trial: | |
| Question: The Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of what? | |
| Thought 1: I need to search Rome Protocols, find the three Prime Ministers, then find what they were assassinated as part of. | |
| Action 1: Search[Rome Protocols] | |
| Observation 1: The Rome Protocols were a series of three international agreements concluded in Rome on 17 March 1934 between the governments of Austria, Hungary and Italy. They were signed by Italian Prime Minister Benito Mussolini, Austrian Chancellor Engelbert Dollfuss and Hungarian Prime Minister Gyula GΓΆmbΓΆs. All of the protocols went into effect on 12 July 1934 and were registered in League of Nations Treaty Series on 12 December 1934. | |
| Thought 2: The three Prime Ministers were Benito Mussolini, Engelbert Dollfuss, and Gyula GΓΆmbΓΆs. I need to search Benito Mussolini and find what he was assassinated as part of. | |
| Action 2: Search[Benito Mussolini] | |
| Observation 2: Benito Amilcare Andrea Mussolini (UK: MU(U)SS-Ι-LEE-nee, US: MOOSS-, Italian: [beΛniΛto aΛmilkare anΛdrΙΛa mussoΛliΛni]; 29 July 1883 β 28 April 1945) was an Italian politician and journalist who founded and led the National Fascist Party (PNF). He was Prime Minister of Italy from the March on Rome in 1922 until his deposition in 1943, as well as "Duce" of Italian fascism from the establishment of the Italian Fasces of Combat in 1919 until his summary execution in 1945 by Italian partisans. As dictator of Italy and principal founder of fascism, Mussolini inspired and supported the international spread of fascist movements during the inter-war period.Mussolini was originally a socialist politician and a journalist at the Avanti! newspaper. In 1912, he became a member of the National Directorate of the Italian Socialist Party (PSI), but he was expelled from the PSI for advocating military intervention in World War I, in opposition to the party's stance on neutrality. In 1914, Mussolini founded a new journal, Il Popolo d'Italia, and served in the Royal Italian Army during the war until he was wounded and discharged in 1917. Mussolini denounced the PSI, his views now centering on Italian nationalism instead of socialism, and later founded the fascist movement which came to oppose egalitarianism and class conflict, instead advocating "revolutionary nationalism" transcending class lines. On 31 October 1922, following the March on Rome (28β30 October), Mussolini was appointed prime minister by King Victor Emmanuel III, becoming the youngest individual to hold the office up to that time. After removing all political opposition through his secret police and outlawing labor strikes, Mussolini and his followers consolidated power through a series of laws that transformed the nation into a one-party dictatorship. Within five years, Mussolini had established dictatorial authority by both legal and illegal means and aspired to create a totalitarian state. In 1929, Mussolini signed the Lateran Treaty with the Holy See to establish Vatican City. | |
| Mussolini's foreign policy aimed to restore the ancient grandeur of the Roman Empire by expanding Italian colonial possessions and the fascist sphere of influence. In the 1920s, he ordered the Pacification of Libya, instructed the bombing of Corfu over an incident with Greece, established a protectorate over Albania, and incorporated the city of Fiume into the Italian state via agreements with Yugoslavia. In 1936, Ethiopia was conquered following the Second Italo-Ethiopian War and merged into Italian East Africa (AOI) with Eritrea and Somalia. In 1939, Italian forces annexed Albania. Between 1936 and 1939, Mussolini ordered the successful Italian military intervention in Spain in favor of Francisco Franco during the Spanish Civil War. Mussolini's Italy initially tried to avoid the outbreak of a second global war, sending troops at the Brenner Pass to delay Anschluss and taking part in the Stresa Front, the Lytton Report, the Treaty of Lausanne, the Four-Power Pact and the Munich Agreement. However, Italy then alienated itself from Britain and France by aligning with Germany and Japan. Germany invaded Poland on 1 September 1939, resulting in declarations of war by France and the UK and the start of World War II. | |
| On 10 June 1940, Mussolini decided to enter the war on the Axis side. Despite initial success, the subsequent Axis collapse on multiple fronts and eventual Allied invasion of Sicily made Mussolini lose the support of the population and members of the Fascist Party. As a consequence, early on 25 July 1943, the Grand Council of Fascism passed a motion of no confidence in Mussolini; later that day King Victor Emmanuel III dismissed him as head of government and had him placed in custody, appointing Pietro Badoglio to succeed him as Prime Minister. After the king agreed to an armistice with the Allies, on 12 September 1943 Mussolini was rescued from captivity in the Gran Sasso raid by German paratroopers and Waffen-SS commandos led by Major Otto-Harald Mors. Adolf Hitler, after meeting with the rescued former dictator, then put Mussolini in charge of a puppet regime in northern Italy, the Italian Social Republic (Italian: Repubblica Sociale Italiana, RSI), informally known as the SalΓ² Republic, causing a civil war. In late April 1945, in the wake of near total defeat, Mussolini and his mistress Clara Petacci attempted to flee to Switzerland, but both were captured by Italian communist partisans and summarily executed by firing squad on 28 April 1945 near Lake Como. The bodies of Mussolini and his mistress were then taken to Milan, where they were hung upside down at a service station to publicly confirm their demise. | |
| Thought 3: Benito Mussolini was assassinated as part of the collapse of the Axis powers during World War II. So the Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of World War II. | |
| Action 3: Finish[World War II] | |
| Reflection: I searched one of the prime ministers involved in the signing, then attempted to answer right away. I should have searched each of the prime ministers, then looked up 'death' on each of their pages in order to get more information before answering. | |
| Previous Trial: | |
| Question: Kam Heskin plays Paige Morgan in a 2004 film directed by who? | |
| Thought 1: I need to search Kam Heskin and Paige Morgan, find the 2004 film they are in, then find the director of the film. | |
| Action 1: Search[Kam Heskin] | |
| Observation 1: Kam Erika Heskin (born May 8, 1973) is an American actress. She began her career playing Caitlin Deschanel on the NBC daytime soap opera Sunset Beach (1998β1999), before appearing in films Planet of the Apes (2001) and Catch Me If You Can (2002). Heskin went on in 2003 to play Elizabeth Bennet in an independent film Pride & Prejudice: A Latter-Day Comedy, Robin in Charmed (2004), and Paige Morgan in The Prince and Me film franchise (2006β2010). | |
| Thought 2: Kam Heskin plays Paige Morgan in The Prince and Me film franchise. I need to search The Prince and Me film franchise and find the director of the 2004 film. | |
| Action 2: Search[The Prince and Me film franchise] | |
| Observation 2: Could not find [The Prince and Me film franchise]. Similar: ['Kam Heskin', 'Prince of Persia', 'Harry Potter and the Half-Blood Prince (film)', 'Prince of Persia: The Sands of Time (film)', 'The Chronicles of Narnia (film series)', 'Majestic Prince (franchise)', 'Kingsman (franchise)', 'The Little Prince (2015 film)', 'Shrek (franchise)', 'Harry Potter (film series)'] | |
| Thought 3: I need to search The Prince & Me (2004 film) instead. | |
| Action 3: Search[The Prince & Me (2004 film)] | |
| Observation 3: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Kam Heskin', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince of Persia', 'Brooklynn Prince', 'Prince (musician)'] | |
| Thought 4: I need to search The Prince & Me (2004 film). | |
| Action 4: Search[The Prince & Me (2004 film)] | |
| Observation 4: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | |
| Thought 5: I need to search The Prince & Me (2004 film) instead. | |
| Action 5: Search[The Prince & Me (2004 film)] | |
| Observation 5: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | |
| Thought 6: I need to search The Prince & Me (2004 film). | |
| Action 6: Search[The Prince & Me (2004 film)] | |
| Observation 6: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince'] | |
| Reflection: I got stuck in a loop where I kept trying to search 'The Prince & Me (2004 film)' but the page could not be found. Instead I should have tried to search the similar results that had a similar name to see and they were made in 2004. | |
| Previous Trial: | |
| Question: The Oberoi family is part of a hotel company that has a head office in what city? | |
| Thought 1: I need to search the Oberoi family and find the city where the hotel company's head office is located. | |
| Action 1: Search[Oberoi family] | |
| Observation 1: The Oberoi Group is a luxury hotel group with its head office in New Delhi, India.[2] Founded in 1934, the company owns and operates 32 luxury hotels and two river cruise ships in 7 countries, primarily under its Oberoi Hotels & Resorts and Trident brands.[3] The group also operates The Oberoi Centre for Learning and Development, which is regarded as one of Asia's top institutions for hospitality education.[4]. The foundations of the Oberoi Group date back to 1934 when The Rai Bahadur Mohan Singh Oberoi, the founder of the group, bought two properties: the Maidens in Delhi and the Clarke's in Shimla.[5] In the following years Oberoi, assisted by his two sons, Tilak Raj Singh Oberoi and Prithvi Raj Singh Oberoi (P.R.S. Oberoi | |
| Thought 2: The Oberoi Group has its head office in New Delhi, India. I just need to finish with New Delhi. | |
| Action 2: Finish[New Delhi] | |
| Observation 2: Episode finished, reward = 0 | |
| Reflection: I appeared to have retrieved the correct information about The Oberoi Family and the location of it's head office, and provided a corresponding answer. However this answer does not exactly match the ground truth answer so I should try a different wording, such as Delhi. | |
| Previous trial: | |
| {trajectory}Reflection:''' |
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- agential/cog/lats/functional.py (1 hunks)
- agential/cog/lats/prompts.py (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- agential/cog/lats/functional.py
- agential/cog/lats/prompts.py
β¦nto vincent/lats
β¦nto vincent/lats
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 13284360 | Triggered | Generic High Entropy Secret | 9a275cb | agential/cog/lats/tmp.ipynb | View secret |
| 13284360 | Triggered | Generic High Entropy Secret | 990f319 | agential/cog/lats/tmp.ipynb | View secret |
| 13284361 | Triggered | OpenAI API Key | f4e84d2 | agential/cog/lats/tmp.ipynb | View secret |
| 13284361 | Triggered | OpenAI API Key | ddd98f5 | agential/cog/lats/tmp.ipynb | View secret |
π Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
π¦ GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
alckasoc
changed the title
alckasoc
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
π€ Reasoning
Explain the purpose of this PR...
π§ Changes
Describe the changes made...
β PR Checklist
Summary by CodeRabbit
New Features
Documentation