Garage

Every embodied AI project I looked at was writing the same thing from scratch: a script to run a policy in a simulator, save what happened, and hope the saved format is still readable in three months. The scripts multiply, none of them agree, and the data outlives the code that produced it.

So the design starts from a different question — not “how do I collect a dataset” but “what has to be true for this data to still be usable later?” That turns into three separated planes. A control plane decides what should be collected: resolved configs, simulator capabilities, checkpoint strategies, task mixtures, reset distributions, quality gates. An execution plane decides how it runs fast: schedulers, process-backed environment workers, GPU-vectorized environments, batched actor inference, action chunks, writer backpressure. A data plane writes raw step events first and treats everything else — validators, metrics, rendered artifacts, training views — as derived.

The last one is the actual opinion. The raw event log is canonical; training formats are views over it. If you materialize the training format directly, you have silently thrown away the provenance, and every later question — why did this run fail, which checkpoint produced this, can I reproduce it — becomes unanswerable. Keeping raw and deriving views costs storage and buys the ability to change your mind.

Backends register capabilities instead of being special-cased, so adding a simulator, an actor or a storage format is a matter of declaring what it can do rather than editing a monolith.

I kept running into the same failure: a paper I found genuinely interesting, explained to a non-specialist, lands as nothing. The ideas survive translation badly, and the format most people actually read — a phone-sized image with eight pages of text — is nothing like a paper.

So I built the translation as a pipeline rather than as a habit. It takes a paper and produces a cover line, eight content pages with an accompanying layout plan, the post body, and a tagging strategy, then renders the whole thing to HTML that can be screenshotted. Each stage is a separate skill with its own prompt and its own examples: one decides what the piece is about and who it is for, one drafts, one refines, one handles layout, one writes the post body.

The step I did not expect to need is the one I now find most interesting: a de-AI-ing pass. Draft output is fluent and completely lifeless — it has the cadence of something generated, and readers detect it instantly even when they cannot name what is wrong. That stage does nothing but rewrite for human rhythm, checking its own output against examples of writing that sounds like a person. It is the stage that most affects whether anyone reads to the end, and it is entirely about voice rather than information.

Building it changed how I think about writing generally. Most of the quality is decided before any drafting happens — in choosing what the piece is actually claiming and who is supposed to care.

Working with coding agents on research, I noticed I was re-typing the same instructions every session: how to map a research landscape before diving in, how to chase implementation detail rather than stopping at the abstract, how a figure should be built so it belongs in a paper.

That is a sign the knowledge belongs in a file, not in my head. So the pack splits what I was saying into three installable skills — one for concept-level landscape mapping (what and why), one for implementation-level detail (how), and one dedicated to publication figures. Plus a notes convention so that what an agent learns in one session is still available in the next.

The part I keep thinking about is the last one. A session that ends with the context window is a session whose learning is thrown away. Writing findings down as they happen is not bookkeeping — it is the difference between an assistant that starts over every morning and one that compounds.

The premise is that the useful version of an AI document assistant is the one that can see everything you have — which is exactly the version you should be least willing to upload. So the whole thing is built local-first: an Electron shell over a local store, with a Python backend doing the work on your own machine.

That constraint makes synchronization the hard part rather than an afterthought. The design keeps a local database as the source of truth and syncs patches rather than documents, with the cloud side holding only metadata for search. Edits made offline reconcile when the connection returns, and a background scheduler handles monitoring and cleanup. The rule that falls out of it — metadata may leave the machine, content does not — is what makes the rest of the architecture decidable.

Most of what I learned came from the unglamorous parts: what happens when the same document is edited in two places, what a file picker is allowed to reach, how an export path becomes a security hole if you let the renderer choose it. None of that is interesting until it breaks, and all of it is what separates a demo from something you would actually keep your own files in.

This is about fifty lines of code, written a few weeks after AutoGPT appeared. It adds one command: search Semantic Scholar, filtered by year and field of study, ranked by citations, and hand the results back to the agent.

It is here because of the date rather than the code. In the spring of 2023 the interesting question about agents was not whether they could plan — it was what they could reach. An agent with a search box gets you plausible-sounding summaries; an agent that can query a real bibliographic index gets you something checkable. Giving a model a good tool was a bigger lever than giving it a better prompt.

Two years later I was building an agent that selects and runs causal analysis methods. The through-line is the same one: the intelligence is in the loop, but the usefulness is in what the loop is allowed to touch.

Every project started the same way: copy last project’s training script, change the model, break the data loader, fix it, forget which experiment used which learning rate. The work that mattered was a small fraction of what I was typing.

Inspired by how MMSegmentation handles this, shallowmind reduces a project to a single config.py. Everything else is disentangled modules over PyTorch Lightning — model families kept general on purpose (encoder-decoder for classification and regression, VAE, GAN, DQN), with backbones, embeddings and heads composed rather than hard-coded, and datasets plugged in the same way.

What I actually got out of it was not the time saved. It was that building the abstraction forced me to notice which parts of my work were genuinely different each time and which were ritual. Most of it was ritual.

A handful of other people forked it and used it for their own work, which remains a nicer outcome than anything I expected from a tool I wrote for an audience of one.

Situation puzzles — turtle soup, in Chinese — work like this: you are told how a story begins and how it ends, and you have to reconstruct the middle by asking yes-or-no questions. The host knows the whole story and may only answer yes or no.

That structure is a nice test for a model, because it demands two things at once. One model has to invent a story with a coherent hidden middle given only its two ends. Another has to answer arbitrary guesses about that middle truthfully, without leaking it — a model that answers too helpfully destroys the game instantly, and one that answers inconsistently makes it unwinnable.

It was a course final project and it is not polished. I keep it on the list because it is the only thing here I built purely because it would be fun if it worked, and because the constraint it turns on — be truthful, be minimal, do not volunteer — has stayed interesting to me far longer than the puzzle format has.