Safety & permission
Distinguish access from authority. Respect changing consent, spending limits, and the right to say no.
Personal agents live with context. Evaluate how they remember, act, and earn trust as people, plans, and the world keep changing.
Use personal apps on your phone and computer. Wearables capture voice, movement and other signals. Together, they connect one person to services and workspaces across the digital world.
Choose a domain to see its apps and workflows below.
Explore how devices, personal apps and services connect.
Contribute realistic app workflows that agents can use.
Turn a believable personal situation into a demanding evaluation. Follow the history, the intervention, and the consequences over time.
↔ Swipe the timeline to explore all dates. Tap a dot or app to pause and inspect it.
We seek tasks that are difficult for a meaningful reason, with clear goals and a reasonable path to success.
Distinguish access from authority. Respect changing consent, spending limits, and the right to say no.
Recognize emotional cues. Offer support when it helps, and give the user space when they need it.
Notice when plans need attention. Choose when to act, ask, wait, or let the user focus.
Carry useful context across sessions. Reconcile what used to be true with what a person wants now.
Handle ambiguous speakers, interrupted speech, and incomplete sensor signals with appropriate uncertainty.
Coordinate shared plans while keeping each person’s private context and delegated authority separate.
Two complementary tracks. Environment contributors make actions possible. Task contributors make those actions worth evaluating.
Own an app or website experience, from the first interaction to the consequences of the last.
Start with an available environment and a plausible personal story. Make success concrete and testable.
Describe the domain, the missing capability, and the environment you will use or build.
Develop a small, convincing slice before widening its scope. Include a demonstration.
Review realism, reproducibility, privacy, and solvability. Treat evaluation defects as defects.
Study fresh runs and revise the task. Report limitations and keep versions comparable.
Invented or carefully transformed scenarios are welcome. Keep private messages, account details, and third-party personal information out of public submissions. A believable story does not require exposing a real person.
Personal Agents' Last Exam is an open research effort toward challenging personal agent evaluation. The goal is to measure useful behavior over time: completing a person’s task while respecting their context, permissions, and changing circumstances.
Yes, and No. If a real experience inspires a task, remove identifying details and protect everyone involved before sharing it. A synthetic scenario is valuable when it is internally consistent and realistic with real user insights.
Yes, a hosted environment can be proposed when full redistribution is unsuitable. It should provide documented evaluation access, reproducible resets, stable behavior, and independent review. Clearly distinguish publicly available components from hosted ones.
A strong task requires a meaningful capability, such as reconciling new consent with an old plan. Essential facts must be discoverable, multiple safe solutions should be accepted, and a reasonable reference solution should work. Hidden requirements and broken tools do not establish difficulty.