Systems Architecture

A map of everything.
Working.

The Cognitive Twin

A cognitive twin is designed to scan the environment, index every tool, map every project, and set up the work so it can run with less babysitting. It is built to be the meta-layer that the other stacks are designed to sit on top of.

01

The System Scan

What was found.

The environment is a working surface. Not a storage dump — a living pipeline. The twin starts by making an honest inventory of what's actually there, in plain text, so the rest of the layers have something concrete to reason over.

Music Production

DAW · Notation · Sample library · Sampler · Synth · FX chain · Generative audio
music-corpus audio-pipeline practice-tracker repertoire-db

AI Audio

STT · Audio analysis · Audio editing · LLM APIs · Browser audio · Voice synthesis
transcription-dashboard handwriting-ocr audio-summarizer

AI Agents

Code assistant · Coder · Generalist · Reviewer · Verifier
memory/user.md memory/agent.md

Visual / Web

WebGL · Build tool · UI framework · Image gen · Video gen · NLE · Video upscale
shader-sketches gen-art long-horizon web-lab

Knowledge

Note system · Reference corpus · Notation · Code assistant
notes-source.md notes-status.log

GitHub

gh CLI · Git · Deploy targets · Static hosting
series-site prompt-library jazz-lexicon

"I want to transcribe this recording" should route to the transcription pipeline, not require re-discovering which tools are installed.

$ twin scan --environment
Scanning filesystem...
Found package.json: 12 projects
Found requirements.txt: 3 Python envs
Found brew list: 47 packages
✓ Scanner complete: 6 domains indexed
Written to memory/twin-index.json
02

Architecture

Four layers. Each one does one thing.

The twin is not a single agent. It's a layered system — scan, model, reason, orchestrate — where each layer feeds the next and none of them require a human in the loop once the work is defined.

Scanner — Index the environment

Runs on demand or on a cron. Traverses the filesystem at startup — reads package.json, requirements.txt, agent skills, installed tools (brew, pip, npm), GitHub repos, Obsidian vault index. Outputs a structured snapshot of every tool, project, and capability available right now. If a new package appears, the scanner finds it on the next run.

Model — Persist the inventory as memory

The scan output writes to a durable layer — memory/user.md for cross-project preferences, memory/agent.md for agent behavior, and a project-level twin index for domain-specific context. This is what allows the twin to know "I already have a transcription pipeline" without re-running discovery. The model is the memory of the environment.

Reasoner — Match work to tools

Given a job — "transcribe this recording", "build a landing page for X", "run the weekly report" — the reasoner reads the model and selects the right tools in the right order. It's not an LLM doing freeform reasoning from scratch; it's a routing decision constrained by the actual inventory. If Whisper and pYIN are installed, route audio transcription there. If not, route to the API. The reasoner picks from what's actually available.

Orchestrator — Set up and run the work process

Once the reasoner picks the path, the orchestrator sets up the process — writes a subagent prompt, schedules a cron, triggers a webhook, or hands off to the next agent in the chain. The orchestrator also handles failure modes: if a step fails, it retries, rolls back, or escalates to the human digest with a diff. The orchestrator is the part that does the work. The other three layers exist so it always knows what it's working with.

03

Designed Processes

What the twin is built to run.

The twin doesn't just describe the environment — it acts on it. These are the process templates it runs, the way an operating system runs its daemons.

Research Swarm

Running

Scheduled subagent that polls a paper feed (arXiv, conference proceedings) and a code feed (GitHub trending, releases) on a fixed cadence. Writes a summary into a tracked docs folder; the published site rebuilds on push.

Deploy Gate

Active

Fires on every web deploy. Visits the live URL, asserts HTTP 200, checks the rendered DOM for expected strings, fails on console errors, retries the deploy if the check fails. The twin verifies its own output.

Memory Maintenance

Active

At the end of a working session, the twin appends durable learnings to memory/user.md (cross-project rules), memory/agent.md (agent behavior), or a project-level AGENTS.md (repo-specific context). The next agent that spins up reads the model and starts with working context.

E2E Regression Gate

Scheduled

After any sprint (multi-step code change), the twin runs the project's end-to-end test before reporting done. Typecheck + build passing don't count. The test must pass. If it fails, the sprint isn't done.

The twin is not a dashboard. A dashboard shows you what's happening. The twin acts on what it finds. If a scheduled job fails three times in a row, the twin doesn't just tell you — it flags it, rolls back the broken deploy, and puts the diagnostic in your next digest.

04

Twin Instances

Domain-specific twins built from the scan.

The general scanner produces domain-specific instantiations. Each twin knows its domain cold and runs the work without asking.

Music Twin

Index: DAW projects, recording sessions, scores, sample libraries, patches, presets. Knows the full instrument and FX chain. Routes requests to the right environment — generative audio for demos, DAW for production, corpus tools for analysis.

Logic Pro Ableton Sibelius pYIN

Transcription Twin

Index: live recordings, transcription output, charts, practice logs. Scans the transcription stack directories. Keeps the pitch-detection + notation pipeline warm. Routes new recordings to the ingest → transcribe → chart flow automatically.

Whisper pYIN MusicXML LilyPond

Web Twin

Index: all deployed pages, deploy targets, hosting providers. Keeps every spoke linked to the hub. Monitors every deploy with the page-check gate. Notifies (or auto-reverts) on failure.

Vercel GitHub Pages Cloudflare Netlify

Research Twin

Index: paper feeds, code trending, jazz reference corpus, audio corpora, knowledge base. Runs the research swarm. Keeps a domain-specific dashboard current. Alerts when a new relevant paper or tool appears.

arXiv Papers With Code GitHub Trending JSTOR

Persistent Agent Loop

Live view of the agent-loop memory core (.agent/state.json). Checkpoints before and after every step; run rewrites this dashboard on pause/exit and every 10 iterations, so the panel below needs no manual regeneration.

⚡

Toolchain

What the twin is built from.

Filesystem Traversal

stdlib scanning

GitHub CLI

Repo & deploy management

Code Assistant

+ specialized subagents

Memory Layers

user · agent · AGENTS.md

Scheduled Reminders

Cron & triggers

Subagent Swarms

Parallel task execution

Deploy Gate

Headless verification

E2E Regression

Test runner

Python Introspection

Package discovery

npm Parsing

Manifest reading

Job Runner

Scheduled execution

JSON Memory

Durable state

Nine stacks. One philosophy. Zero babysitting.