Case study · iOS · on-device LLM · 2026

Lucy
Offline AI

An iPhone app that answers questions about one Burning Man camp out of ten years of its own group chat, with a three-billion-parameter language model on the phone and no network at all.

The problem

Preservation Society is a theme camp: a few hundred people, a decade of know-how, one week a year when all of it matters. Which socket the lag bolts take. Where the water barrels went. The answers exist, buried in 27,000 WhatsApp messages, and the person who knows is asleep or not coming this year.

A chatbot over the chat history, then. Three things make that hard here:

No network. The knowledge, the search index and the model all have to be on the phone before the trip.
The knowledge is conversation. "The padlock hammer lives in the van" is three messages, two years apart, one of them a joke.
A wrong answer costs a walk in a whiteout. Nobody can check. A language model left to itself will send someone to the wrong truck, fluently.

0bytes of network per answer

3.11 GBlanguage model, on the phone

27,163WhatsApp messages read

2%answers inventing a fact, measured

The one rule

A language model is a guesser of the next word. It has read the internet and none of this chat, and it cannot tell what it knows from what sounds right. So it is not asked to know anything.

Retrieval supplies the facts. The model only phrases them. A name, a date, a number, a place: each has to come from a database row. A claim the model made up is a bug.

General knowledge stays the model's own. She may know the desert is hot. She may not guess which van the water is in.

Two halves

The expensive thinking happens once, on a Mac, before the event. The phone only searches a catalogue and reads it aloud.

Build time · Mac + cloud, before the burn

Raw history27k messages · 144 documents, cut from 627 · 291 people

EnrichmentGemini turns conversation into dated claims, each citing its messages

Embeddingsa 768-number vector per claim, so meaning is searchable too

Gatesevery claim evidenced · every claim embedded · nothing personal ships

↻ cached by content, so a rebuild pays only for messages that changed

↓ knowledge.db · 17.6 MB of SQLite, inside the app

Run time · iPhone, offline

Questiontyped, or spoken to an on-device recogniser

Retrievalword search plus meaning search over facts, people, documents, stories, past turns

Fact sheetthe matching rows, ranked, with dates and sources

Gemma 4llama.cpp on the phone, phrases the sheet and nothing else

↻ every turn is journalled: question, facts given, answer. The journal is the test set.

Build time

Enrichment is the step that matters. A large model reads the messages and decides what the camp knows: that a phrase is a thing with a name, that a thread contains a decision, that a joke is a tradition. It writes short dated claims, each stored beside an evidence row pointing at the messages it came from. 1,745 facts, 8,020 evidence rows, and a trigger that refuses a claim without one.

The model was chosen by measuring. On the same corpus, Gemini's strongest model found 149 named things; its fast one found 64. The hundred it missed were the camp's culture, not its inventory.

Embeddings cover what word search misses: "med kit" never matches "first aid". A 300M-parameter encoder turns every claim into 768 numbers, a point in a space where similar meanings sit near each other. The same encoder runs on the phone.

Documents were cut 77%, with a test rather than a rule: a document may go only if every distinctive word in it survives elsewhere. Three sensible folder rules each failed it, one by deleting the only document containing "tourniquet".

The build guide with the camp plan figure — A document that stayed: the build manual, converted from Word with its 15 figures intact. No paraphrase of a site plan helps someone placing a truck.

Run time

Ask "how do we get water delivered?" Stop words go; water delivered stays. Those terms hit five tables at once, scored by match and discounted by age. The question is embedded on the CPU and the two nearest claims above a 0.45 floor join in. What matched becomes the only world the model sees:

FACTS YOU MAY USE:
- Water comes by truck on Tuesday and Friday;
  book it on the shared sheet (2024) [from our manual]
About Doris, our vehicle:
- carries the water barrels to playa [said 2022-08-19]

QUESTION: how do we get water delivered?

Illustrative rows, not the camp's.

The prompt around it is five rules and no examples. It once had examples, and she pasted them back as answers.

Memory is retrieval too. No transcript rides in the prompt. Every turn is written to a second database and comes back like a fact when it matches, labelled as conversation, never as truth. That reaches Tuesday's answer on Friday, which no context window does.

The model is Gemma 4 E2B, 4-bit, through llama.cpp. Phones with less RAM get Gemma 3 1B. The two use different chat markers, and the app finds out which by tokenising a probe, not by reading a filename. For a week the markers were hardcoded wrong and every answer was fluent anyway.

A real exchange with Lucy, names blurred — A real exchange, on-device. The number came from a row; the second answer from the camp's own record of a running joke. Names blurred.

Measured, not judged

The phone's journal replays. The camp's real questions and their exact fact sheets go through the real model under any prompt, and a scorer checks each answer for a name, number or place that was not in the sheet.

Forty real questions, same facts. With the rules: 2% of answers invent a camp particular, median 15 words. Without them: 82% invent, median 56 words.

That is the trade-off in one line. The rule buys truth and pays in voice; the flat answers people dislike are the price of the 2%. It is also why a fine-tune on the camp's own voice did not ship. It learned the register and invented more, and the number said no.

Where it broke

Each passed its tests. The failure mode in this system is silence.

Meaning search shipped and did nothing: On the GPU the encoder returned NaN. NaN clears no threshold, so it added zero rows to every answer, with no error. No test had ever run the encoder, because it needs a 333 MB model. It runs on the CPU now, and a live test loads the real one.
Then it worked, and made answers worse: Its threshold came from one good question. Printed out, the similarities split cleanly at 0.45, and 30% of the facts turned out to be formulaic roster lines that any question with a name lands on. Floor raised, cap set to two rows.
The roster was missing for seven months: The installer needed a flag, printed one line without it, and nobody read the line. A gate now refuses any build where a table shrank more than 25% against the last one a human accepted.

Privacy by ownership

What ships is decided by whose a fact is, not what it looks like. A campmate's home address never leaves the Mac; the camp's warehouse address must ship, because "where is the warehouse" is a real question. A second, independent scanner reads the final database before every release. It has overruled the builder once, on ten phone numbers a safe-looking refactor exposed.

Same rule on this page: the app's own chrome and the city's public listings only. The camp and its people stay in the dust.

The people directory, names blurred — 309 campers, searchable by what they are known for. Blurred here; on the phone they are the point.

What she does

Hold to talkOn-device speech, network path disabled. Answers as text you can read in a tent.

Grounded chatWho owns what, where things live, what the camp decided and when.

Memory across rebootsEvery exchange stored and searchable; "Earlier" pages the old thread back in.

Capture for the campPhotos and voice notes, transcribed on the phone, synced when coverage appears.

The paperworkManual, illustrated build guide and shift schedule, searchable, in the app.

The whole city4,224 published events by day and hour, every camp, every art piece.

The Memory tab with the camera viewfinder — Memory: shoot or hold to talk, and the note is queued for the camp.

The city's event listings — The city's events, folded by day and hour.

Sync and shipping

A small FastAPI and Postgres service takes notes whenever a phone finds coverage. The note's id is minted on the phone, so a retry on a dead connection cannot duplicate it. Feed order is assigned under a lock, so a late commit cannot be skipped. Photos are stored by content and never deleted, because two phones photographing the same sign share one file.

One developer, forty testers, no staging in the desert. 199 unit tests, a harness that renders any screen headlessly, resource checks that open the built app rather than the source tree, and unattended TestFlight uploads whose build number is the commit count.

Searching the art listing — Search reaches into descriptions: "dodecahedron" finds a piece by what it is.

Stack

Swift / SwiftUI llama.cpp Gemma 4 E2B · Q4_K_M Gemma 3 1B fallback EmbeddingGemma 300M SQLite + vectors on-device speech Python pipeline Gemini enrichment FastAPI PostgreSQL Caddy GCE TestFlight via ASC API xcodegen

The code is public at github.com/kaymo-ai/Lucy, without the camp's data.

Built with heavy use of Claude Code as a pair.