nkl: a common task language for human and artificial agents
1Introduction
Behavioral tasks are a central tool for evaluating human, and now artificial, intelligence. Cognitive science, neuroscience, psychology, and other fields offer rich experimental traditions, but each specifies its tasks in its own languages and tools.
We propose nkl as a common language that unifies many of those tasks, and allows them to be easily deployed to any agent, human or model, without modification.
2Tasks as functions
nkl describes tasks as functions over three streams of data:
nkl functions are written in Python, from a small set of primitives:
task.pyimport nkl@nkl.functionasync def hello_world() -> dict[str, str]: await nkl.show(nkl.Text("Hello world! Press 'a' or 'h'")) key = await nkl.next(nkl.filter({'a', 'h'}, nkl.keys.stream())) if key == "a": await nkl.show("ai!") else: await nkl.show("humans!") await nkl.sleep(1000) return {"response": key}Human
AI
3Matched behavioral testing
Any agent that performs a nkl task is faced with the same perception–action loop:
nkl records every frame, key, and pointer event, supporting comparisons between models and humans at multiple scales.
4Deploy
Deploy a single nkl task, without modification, using one command for each setting:
- In the lab
nkl play task.py- On Prolific
nkl bundle task.py- To AI agents
nkl sim task.py
Scan to see this poster online and learn more (mil@mit.edu):