Multi Agent Protocol for AI Scientist

Socrates is an open-source multi-agent protocol built around a specific structural constraint: one agent acts, one agent questions, and the questioning agent is hardcoded to never answer, never direct, and never touch tools. The Scientist is stateless, executes code, and reads and writes a shared environment. The Socrates advisor is stateful across sessions — it remembers prior questions — and blocks execution until it issues an explicit [APPROVED] signal. The workflow is a loop: Scientist proposes a plan, Socrates interrogates it with questions only, Scientist revises or proceeds once approved, then runs the experiment.

The differentiating constraint is the advisor’s deliberate incapacity. Most multi-agent setups allow every agent to suggest solutions, which means the planning agent can anchor on bad ideas that go unchallenged. By stripping the advisor of the ability to give answers or directives, the protocol forces the Scientist to surface its own assumptions rather than inherit the advisor’s. The docs describe this as unlocking latent knowledge already present in the Scientist — the questions expose gaps the agent would otherwise paper over.

The protocol fits AI researchers running MLE-bench-style experiments or building evaluation harnesses for research agents. It is not architected for production pipelines: there is no API surface, no hosted option in the repository, and the two-agent structure is fixed. Teams that need branching based on intermediate outputs, parallel agent tracks, or integration with external services will find the protocol’s rigidity a hard constraint rather than a feature. Self-hosting requires working from the repository directly, with dependencies managed via conda and a requirements file.