I have a fantasy series with my name on it. It's called Underfoot, and it's planned for six books. Four of them are finished, 133 chapters in all, and two are done as full audiobooks with a cast of about thirty voices. I have not read a word of any of it.
That was on purpose. I am listening to it in the truck, one chapter at a time, the same way anyone else would. Every status report I got back while it was being written had to be free of plot: chapter counts, runtimes, casting names, dollar figures, nothing else. The planning documents carry "do not read" warnings. The chapter text for books two through four was deliberately never put in the same Google Drive folder as the audio, so I couldn't open one by accident.
It turned out to have a practical cost too. Searching the project by keyword became dangerous, because search pulls up planning documents full of spoilers and drops them in a chat window I'm looking at. Chapter lookups had to go through exact file paths from a directory listing instead.
This post is about how the thing got built, because the engineering is more interesting than the fact that an AI wrote a novel.
The setup
The work ran in Claude Cowork, which gives Claude a persistent Linux sandbox in the cloud with a shell, a filesystem, and the ability to call outside APIs. The model was Opus 5. This is a different setup from the Claude Code work I've written about before, where the agent is running against a codebase on my machine.
Three things about that environment drove nearly every decision that followed.
The sandbox is ephemeral. It gets recycled without warning, and it did that twice during this project. Every helper function that lived only in memory went with it.
The durable layer was a claude.ai Project, which is a document store attached to the conversation that survives across sessions. It held the research, the series bible, all 133 chapter manuscripts, the cover prompts, and a production reference document that let a fresh session rebuild the whole pipeline from nothing. The conversation ran out of context something like fifteen times. The Project was the memory.
Everything external went through MCP connectors: Google Drive for delivery, ElevenLabs for the voices, the OpenAI image API for covers, and a remote workbench for the heavy processing. Each of those has its own way of failing, and a lot of the work below is about surviving them.
Research before fiction
The project didn't start with writing. It started with a genre teardown.
Eight progression fantasy series I already liked got analyzed one at a time, each with its own breakdown document. For the most reviewed of them, over a thousand reader reviews were mined to separate what readers say they want from what they actually praise and complain about. Those eight breakdowns were then synthesized into a single document describing the shared skeleton of the genre, along with the specific ways long series go bad: stat screen bloat, settlement management turning into spreadsheet homework, the protagonist drifting away from the rest of the cast.
Buried in that synthesis was a section called "audio-first design rules," written before a single word of fiction existed. It said that every recurring character needs a distinct verbal fingerprint, something a text to speech voice can actually differentiate. It said the voice inventory has to be budgeted, with the core cast getting real voices and side characters getting speech patterns instead. It said system messages have to be sparse, and numbers should only show up when they change a decision. It said chapters should end on scene turns and be sized for a commute.
That section is the reason the rest of this worked. The books were written to be rendered. The low density of system messages is why rendering stayed cheap. The clean speaker attribution is why automated voice routing was possible at all. The chapter lengths are why each one fit into a single batch. Every one of those is a technical win that traces back to a design decision made on day one, in a research document, before there was anything to render.
Four books came out of it: 26 chapters, then 35, then 36, then 36. Books one and two took four days. Books three and four took a day each, once the world rules and the cast were stable enough that drafting was mostly execution.
The pipeline
By the time it stabilized, the render pipeline ran 61 chapters in this form.
A chapter gets pulled out of the Project as markdown. A Python script walks it paragraph by paragraph and emits a segment list, a JSON array where each entry is a voice, a piece of text, and a pause type. A typical chapter produces 25 to 50 segments. The walker handles scene breaks, bracketed system notifications routed by keyword, per chapter overrides for things the generic rules can't catch, and quoted dialogue split out of the narration and matched against a hand written list of voices in the order the quotes appear.
Then, before anything costs money, a quote count runs. If that hand written voice list is one entry short of the actual number of quotes in the chapter, the segmenter throws an error partway through, possibly after burning credits. That check caught a miscount on the very last chapter of book two.
After that, four parallel workers call ElevenLabs and write each segment to a cache directory. Every segment is measured for loudness and boosted if it's quiet. Typed silences get inserted between segments, from 0.15 seconds inside a paragraph up to two seconds at a scene break. Everything is concatenated, the book cover is muxed in as album art, ID3 tags are written, and the file goes to the right Drive folder. Then the duration is confirmed, the credit meter is read, and a line goes into a progress ledger.
Feedback becoming infrastructure
I mentioned once that one character sounded quiet.
The easy answer would have been to re-render that chapter. Instead it turned into a measurement step: check the loudness of every segment against the median narrator level, boost anything more than 2 dB under, cap the boost at 12 dB so artifacts don't get amplified. It was applied backward through book one and across all of book two.
That step has since corrected hundreds of segments that nobody ever had to notice. It also turned out the problem was real and systematic, not a fluke. That particular voice renders 5 to 15 dB below the narrator every time, and three others drop on short lines.
This is the pattern I'd take away from the whole project. When a person notices a defect, don't fix the instance. Ask what class of defect it belongs to and automate the check.
The same instinct produced the strangest piece of the system. Moving a several thousand character JSON payload from the local container into the remote workbench occasionally corrupted a single character in transit. One flipped character means a parse failure if you're lucky, and a wrong voice ID rendering an entire chapter in the wrong voice if you're not, which costs real money to discover. So the payload gets base64 encoded, split into pieces, split again into 125 character lines, and every line gets a checksum with an MD5 over the whole thing. On the far side, a line that fails its checksum gets repaired by brute forcing single character substitutions, which fixes the overwhelmingly common case without a retransmit. The final MD5 is verified before anything is spent.
That sounds absurd for moving a JSON file around. It also saved more time than anything else in the project.
What broke
The network backed storage the pipeline used for caching died in the middle of a chapter and never came back. Every helper had to be rewritten to use local disk instead. The book cover lived only on that mount, and it got recovered by pulling the embedded album art back out of an MP3 that had already been delivered.
The sandbox recycled twice, taking every helper function with it. Both times the fix was a rebuild from the production reference document, which is exactly why that document exists.
The remote disk filled up on chapter 35 of 35. Two things made that worse than it should have been. The assembly step returned a perfectly plausible success value, with the real error only visible in stderr, so it looked like it worked. And the remote shell tool became unusable, because it stages a script file to disk before running it, so the obvious cleanup tool failed for the same reason cleanup was needed.
There were the usual API surprises too. The TTS response nests the audio URL two levels deep. Drive download takes a camelCase parameter while upload takes a snake_case one. The workbench swallows printed output, so the value you want has to be the last expression in the cell rather than something you print.
The numbers
Book two is 35 chapters and 3 hours 59 minutes 54 seconds of finished audio. It cost 121,386 ElevenLabs credits, which at the rate I bought them works out to $22.09. That's about 63 cents a chapter, or $5.52 per finished hour. The covers ran about 25 cents each, and all three landed on the first attempt, which says less about luck than about the fact that each cover prompt was a written document derived from that book's synopsis before anything was spent.
For comparison, a professionally narrated audiobook of that length usually runs somewhere between $1,500 and $4,000. That is not a fair comparison. A good narrator brings interpretation and timing and emotional judgment that no amount of segment routing replicates. But the gap is two orders of magnitude, and that's the actual story.
One early estimate had the burn rate at about 12,600 credits a chapter, which projected to running out around chapter 29. That number was inflated by retries during an unstable stretch. Sampling the meter every few chapters showed the real steady state was around 3,300, roughly a quarter of the first guess. I got warned I'd need to top up and then told I wouldn't. Annoying message to receive, better than either surprise.
Where the human stayed
I made every casting decision. When a significant character showed up, I'd get three or four preview clips and pick one. Two rules came out of that and got written into the casting document: distinctive but natural beats generic epic, and no cartoonish kid voices.
I approved every cover. I set the content boundaries. I never reviewed segmentation logic, tuned an ffmpeg flag, or picked a pause duration.
That split is the only reason a six book series with a thirty voice cast was manageable. It's also worth saying that about a third of the cast was assigned on a single early appearance and never re-auditioned, and two of those characters turned out to carry a lot more weight later than they did when I cast them. Casting a series you haven't finished writing means some of your casting is a bet.
Listening to it
Chapters are going up on my YouTube channel as I release them, starting with book one, The Gutter King. Here's chapter one.
New chapters land on the full playlist on Monday, Wednesday and Friday.
Books three and four are written and waiting. Together they're 72 chapters, which projects to roughly eight or nine more hours of audio for about $46. Before book three can be rendered I owe it three auditions, including proper ones for the two characters I cast provisionally and then gave a lot more to do.
Book five and six aren't written yet. I don't know what happens in any of it.