Harbor district · A fictional world08:12
PeoplePersonal agentsA changing world
A dynamic world
Personal Agents' Last Exam
The open-source benchmark version ofPersonalAgentEval

A whole life.
An agent
alongside.

Personal agents live with context. Evaluate how they remember, act, and earn trust as people, plans, and the world keep changing.

Personal agents that grow with you.
Illustrative world
January
Environment track

Apps and websites.
One connected world.

Use personal apps on your phone and computer. Wearables capture voice, movement and other signals. Together, they connect one person to services and workspaces across the digital world.

Devices & sensorsInternal · Basic toolsExternal · Service domains

Choose a domain to see its apps and workflows below.

Lena, an illustrated fictional personLena
↔
Personal AgentPersonal Agent

One life, together.

Explore how devices, personal apps and services connect.

Contribute realistic app workflows that agents can use.

The digital environment library

Apps & websites

Explore the collection

Public product interface reference · not an evaluation rollout

View original public source ↗
02 Task track

The task unfolds.
So does the world.

Turn a believable personal situation into a demanding evaluation. Follow the history, the intervention, and the consequences over time.

A living task, unfolding over time
UserDevicesAgentPeople + agentsWorld
Everyday traces flow as small dots; app icons mark core task actions. Twelve days unfold in seconds. Two key moments include screenshots.

↔ Swipe the timeline to explore all dates. Tap a dot or app to pause and inspect it.

Illustrative scenario
Research directions

Looking for challenging tasks and important scenarios.
Trust is earned over time.

We seek tasks that are difficult for a meaningful reason, with clear goals and a reasonable path to success.

◈

Safety & permission

Distinguish access from authority. Respect changing consent, spending limits, and the right to say no.

⌁

Emotion and Feeling

Recognize emotional cues. Offer support when it helps, and give the user space when they need it.

◷

Proactive assistance

Notice when plans need attention. Choose when to act, ask, wait, or let the user focus.

∞

Memory & relationships

Carry useful context across sessions. Reconcile what used to be true with what a person wants now.

≋

Speech & wearables

Handle ambiguous speakers, interrupted speech, and incomplete sensor signals with appropriate uncertainty.

⋈

People & other agents

Coordinate shared plans while keeping each person’s private context and delegated authority separate.

Build PALE together

Bring a world.
Or a challenge inside one.

Two complementary tracks. Environment contributors make actions possible. Task contributors make those actions worth evaluating.

01 Environment contributor

Build the places agents use.

Own an app or website experience, from the first interaction to the consequences of the last.

  1. 01A high-fidelity app or website with complete core workflows: navigation, search, edits, confirmations, cancellations, and realistic failure states.
  2. 02An adapter for PALE’s evaluation interface, with documented actions, observations, permissions, and errors that agree with the visible app.
  3. 03A repeatable reset, outcome checks, and tests for permissions and side effects.
  4. 04Setup documentation and a clear license or hosted access plan.
Explore the project ↗
02 Task contributor

Build the moments that matter.

Start with an available environment and a plausible personal story. Make success concrete and testable.

  1. 01A person, a meaningful goal, relevant history, and explicit boundaries.
  2. 02A sequence of world changes and interactions with a coherent cause and effect.
  3. 03Outcome checks, a reasonable reference solution, and progressively helpful hints.
  4. 04Independent trial runs, a failure analysis, and a privacy review.
Explore the project ↗
01 · ProposeExplain the contribution.

Describe the domain, the missing capability, and the environment you will use or build.

02 · BuildMake one complete experience.

Develop a small, convincing slice before widening its scope. Include a demonstration.

03 · ReviewTest the claim independently.

Review realism, reproducibility, privacy, and solvability. Treat evaluation defects as defects.

04 · ImproveLearn from observed behavior.

Study fresh runs and revise the task. Report limitations and keep versions comparable.

Real user insights are part of the contributions.

Invented or carefully transformed scenarios are welcome. Keep private messages, account details, and third-party personal information out of public submissions. A believable story does not require exposing a real person.

A few good questions.

What is PALE trying to measure?

Personal Agents' Last Exam is an open research effort toward challenging personal agent evaluation. The goal is to measure useful behavior over time: completing a person’s task while respecting their context, permissions, and changing circumstances.

Do I need real user data?

Yes, and No. If a real experience inspires a task, remove identifying details and protect everyone involved before sharing it. A synthetic scenario is valuable when it is internally consistent and realistic with real user insights.

Can an environment be hosted instead of fully open?

Yes, a hosted environment can be proposed when full redistribution is unsuitable. It should provide documented evaluation access, reproducible resets, stable behavior, and independent review. Clearly distinguish publicly available components from hosted ones.

What makes a task challenging rather than confusing?

A strong task requires a meaningful capability, such as reconciling new consent with an old plan. Essential facts must be discoverable, multiple safe solutions should be accepted, and a reasonable reference solution should work. Hidden requirements and broken tools do not establish difficulty.

Help build the next exam
for agents in everyday life.

PALE on GitHub ↗