Methodology · Growth Readiness v0.2.3
How we measure agent readiness.
Bloom’s Growth Readiness measures agent scaffolding, not LLM model quality. Claude, GPT, Gemini, and open-source models all start from the same baseline; what varies is the harness, tools, context, and skills around them. The full spec is open-source and citation-anchored to current agent-evaluation research from Anthropic, Stanford, Princeton, and Meta.
What this measures
Scaffolding readiness — the configuration of the agent: what tools it can call, what skills it has installed, what project context is loaded, what memory layers persist.
The output is a single composite percentage (0–100%) plus three axes (Insight / Create / Distribute) and a list of missing capabilities. Each missing capability ships with a paste-back remediation prompt the agent can apply to itself, then re-run for the lift.
We grade the agent’s setup, not its run-time behavior. This is deliberate: behavior compounds with model choice, and Bloom is model-agnostic by design.
What this does not measure
- LLM model quality — Claude vs GPT-4 vs Gemini all start at the same baseline. Tribe missions cover outcome quality separately.
- Latency, cost, token usage — these are operational concerns, not growth-readiness signals.
- Run-time correctness — covered by Tribe mission verification (deterministic graders, schema validators, citation re-scans), not by setup-audit.
The readiness percentage
9 capability primitives matched against a fixed TARGET_PROFILE. Cross-harness fair by construction: every harness maps to the same target, declaring its native equivalent (Bash → shell-or-equiv, Task tool → sub-agents, etc.). Reasonable Claude Code, Hermes, Codex CLI, OpenClaw setups all converge to similar readiness percentages for similar capability.
| Capability | Axis | Per-harness detection |
|---|---|---|
Web search Can the agent query the open web for buyer-intent signal (Reddit, HN, Google Trends, Brave/Tavily/Perplexity, etc.)? webSearch | insight |
|
URL fetch Can the agent fetch arbitrary URLs to read landing pages, competitor sites, schema markup? webFetch | insight |
|
File system read/write Can the agent write artifacts (llms.txt, JSON-LD, markdown pages) to disk? fileSystemRW | create |
|
Structured LLM output Can the agent produce schema-valid structured output (JSON-LD FAQPage, ItemList, Schema.org)? llmStructured | create |
|
Persistent memory (project or higher) Does state survive across runs? `none` / `session` / `project` / `cross_session` (ordinal). persistentMem | insight |
|
Project context file Is there a project-level brief (CLAUDE.md / instructions.md) describing product + customers + competitors? projectContext | insight |
|
Sub-agent / parallel reasoning Can the agent spawn parallel sub-agents (e.g. one mining intents, one drafting, one validating)? subAgents | distribute |
|
Shell exec or arbitrary-tool gateway Can the agent invoke arbitrary CLI tools (gh, curl, schema validators, IndexNow ping)? shellOrEquiv | distribute |
|
Bloom skill installed Does the agent have any bloom-* playbook installed (Visibility, Cite Boost, Comparison Page Generator, etc.)? bloomSkillInstalled | create |
|
Readiness = matched / total × 100, rounded. Per-axis level = matched-in-axis / total-in-axis × 100. persistentMem uses ordinal comparison (project or cross_session both meet a project target). Verification confidence is displayed separately so declared-only high readiness is not mistaken for fully probed capability.
Academic anchors
We compose validated research. Each design decision below is anchored to peer-reviewed or published industry work; we cite the paper, name the lift, and stay narrow about what we adopt.
Demystifying evals for AI agents
Anthropic 2026
Anthropic Engineering · Anthropic Engineering Blog · 2026
Used for: Grade what the agent produced, not the path it took. Bloom grades scaffolding (a stable property of the agent) instead of run-time behavior (which compounds with model choice). 20-50 simple tasks drawn from real failures — small N is sufficient at this stage.
https://www.anthropic.com/engineering/demystifying-evals-for-ai-agentsHELM — Holistic Evaluation of Language Models
Liang et al. 2023+
Liang et al., Stanford CRFM · Stanford CRFM · 2023+
Used for: Multi-specific-metric philosophy: many narrow primitive metrics communicate more than a single composite. Bloom v0.2.0 exposes 9 primitives + per-axis breakdown; the headline percentage is a derived view, not the source of truth.
https://crfm.stanford.edu/helm/autoresearch — modify · verify · keep · discard · repeat
Karpathy autoresearch
Andrej Karpathy · GitHub · —
Used for: Immutable versioned eval pattern. v0.1.0 scorer is frozen; v0.2.3 ships alongside; old reports remain reproducible from their stored signed payloads. Readiness signatures (HMAC) are version-prefixed.
https://github.com/karpathy/autoresearchCo-Gym — Towards Building Collaborative Agents
Shao et al. 2024
Shao, Yu, Zhang, Zhao, Welleck (Stanford OVAL) · arxiv 2412.15701 · Dec 2024
Used for: Bidirectional human-agent checkpoints outperform autonomous execution by 60-86% on real-world tasks. Bloom’s draft/approve flow on high-risk missions (publish to builder’s domain / repo / social) implements this pattern.
https://arxiv.org/abs/2412.15701GAIA — A Benchmark for General AI Assistants
Mialon et al. 2023
Mialon et al., Meta AI · arxiv 2311.12983 · 2023
Used for: Outcome-based agent benchmarking with deterministic verification. Bloom uses this philosophy for Tribe missions (artifacts must validate against schema.org / format spec / HTTP-HEAD checks) — not for setup-audit, which is structural by design.
https://arxiv.org/abs/2311.12983AgentBench — Evaluating LLMs as Agents
AgentBench (THUDM)
Liu et al., Tsinghua · ICLR 2024 · 2024
Used for: Task-outcome agent eval across 8 environments. Companion frame for GAIA. Mission-side outcome verification adopts the deterministic-grader pattern.
https://arxiv.org/abs/2308.03688Holistic Agent Leaderboard
HAL 2025
Princeton CITP / HAL contributors · arxiv 2510.11977 · Oct 2025
Used for: Reliability-first benchmark consolidation. Bloom’s v0.2.0 stays narrow (configuration, single readout) to avoid the leaderboard pollution that HAL documents in earlier multi-benchmark work.
https://arxiv.org/abs/2510.11977SkillsBench — Benchmarking How Well Agent Skills Work Across Diverse Tasks
SkillsBench 2026
SkillsBench contributors · arxiv 2602.12670 · 2026
Used for: Containerized skill verification with deterministic graders. Cite Boost’s 30-day citation re-scan pipeline adapts the deterministic-verification pattern from skill-level to outcome-level.
https://arxiv.org/abs/2602.12670SHADE-Arena — Sabotage Detection in LLM Agents
Anthropic SHADE 2024
Kutasov et al., Anthropic · Anthropic Research · 2024
Used for: Adversarial scaffolding-stress framing. Bloom’s anti-gaming middleware (rate limits, fresh-agent + all-high-confidence detection, content allowlists) draws on the threat-model patterns documented here.
https://assets.anthropic.com/m/4fb35becb0cd87e1/original/SHADE-Arena-Paper.pdfKnown limitations
- Structural, not predictive. v0.2.0 measures whether the agent could ship growth artifacts well, not whether it will. Cohort outcome data (Tribe mission completions, Cite Boost 30-day re-scans) calibrate predictive validity over time.
- Self-report. Capability primitives are agent-declared. Cheap proofs (CLAUDE.md content snippet, sentinel URL fetch) mitigate but do not fully prevent spoofing. V1 trust threshold is “agent reports its own setup honestly”; gaming has no payoff because Tribe missions verify deterministically downstream.
- Opinionated TARGET_PROFILE. We picked 9 primitives. Other groups may pick differently; we expose the spec so they can fork or critique.
- v0.2.0 default; v0.1.0 frozen. Old reports remain reproducible from their stored signed payloads. Future versions will register alongside, never mutate the past.
Open-source
Full scoring spec, capability detection rules, and reference implementation:
github.com/bloomprotocol/growth-readiness-specSpec under MIT. Issues / forks / methodology critiques welcome.
Run the audit on your agent
Paste this one line into Claude Code, Hermes, OpenClaw, or Codex. The agent runs the readiness loop locally with its own tokens; Bloom never sees keys or reasoning.
Fetch https://bloomprotocol.ai/readiness.md, show me the approval preflight, then run the Growth Readiness Report end-to-endOr start from the homepage →