The paper has not been published yet and is currently submitted to a
top conference where you’ve been assigned as a peer reviewer.
Complete a full review of the paper answering all prompts of the
official review form of the top venue in this research area (e.g.,
NeurIPS). This includes recommending whether to accept or reject the
paper.
This paper was found buried under ground in the desert. You’re an
archeologist who must determine where this paper sits in the context
of previous and subsequent work. Find and report on one older paper
cited within the current paper that substantially influenced the
current paper and one newer paper that cites this current paper.
You’re a researcher who is working on a new project in this area.
Propose an imaginary follow-up project not just based on the current
but only possible due to the existence and success of the current
paper.
You work at a company or organization developing an application or
product of your choice (that has not already been suggested in a
prior session). Bring a convincing pitch for why you should be paid
to implement the method in the paper, and discuss at least one
positive and negative impact of this application.
You’re a hacker who needs a demo of this paper ASAP. Implement a
small part or simplified version of the paper on a small dataset or
toy problem. Prepare to share the core code of the algorithm to the
class and demo your implementation. Do not simply download and run
an existing implementation – though you are welcome to use (and give
credit to) an existing implementation for “backbone” code.
Toy demo 设计（伪代码，未运行）：在少量确定性任务上自己实现配对 runner 与 verifier，固定
agent/tool/budget，no-skill/skill 各运行三次后求 task-macro
delta：
for task in tasks:
for condition in ["none", "skill"]:
for trial in range(3):
result = agent.solve(task, skill=(condition == "skill"))
score[task, condition, trial] = verifier(task, result)
for task in tasks:
delta[task] = mean(score[task,"skill"]) - mean(score[task,"none"])
print(mean(delta))
You are a detective who needs to run a background check on one of
the paper’s authors. Where have they worked? What did they study?
What previous projects might have led to working on this one? What
motivated them to work on this project? Feel free to contact the
authors, but remember to be courteous, polite, and on-topic.
Identify how this paper self-assesses its (likely positive) impact
on the world. Have any additional positive social impacts left out?
What are possible negative social impacts that were overlooked or
omitted?
