Instrument before you launch

Logging, crash tracking, OpenTelemetry traces and usage dashboards went into Locava before there was a launch date, with BigQuery behind them as the warehouse. Doing it in that order felt like procrastination at the time. It is the single most useful decision I made on that product, and the reason is not the one I expected.

The expected reason, which is real but secondary

I thought instrumentation was for debugging production. It is, and that part matters more when you are alone: there is no one to page, so the only way to find out why a feed is empty for one user is to be able to ask. Traces across the Fastify handler, the Firestore read and the Redis lookup turn that from a guess into a query.

The actual reason

What instrumentation really bought was the end of roadmap arguments. Every product discussion on a small team resolves to one of two things: a question with an answer nobody has looked up, or a genuine disagreement about values. Those need completely different conversations, and without data they are indistinguishable, so all of them turn into the second kind and get settled by whoever is most confident.

"Do people use collections?" is not an opinion. Once the event is in BigQuery it is a query, it takes a minute, and the answer ends the discussion. What is left afterwards is the small set of real disagreements, and those are worth the meeting.

What went in, in what order

1. 01Structured logs with a request id, propagated from the client. Without the id you have lines; with it you have stories.
2. 02Crash reporting on both clients, before any feature work. A crash you do not know about is indistinguishable from a user who left.
3. 03OpenTelemetry traces across the backend, so the Firestore read and the Redis lookup are spans rather than guesses about which one is slow.
4. 04Product events into BigQuery, with a warehouse from the start rather than a dashboard tool that owns the data.

The fourth is the one I would defend to somebody who thinks this is overkill for a pre-launch app. A dashboard product that stores your events answers the questions it has features for. A warehouse answers the question you thought of this morning, including the ones about last spring, which is the entire point of having kept the data.

The part that is genuinely overkill

I over-instrumented the media pipeline. Every stage of upload, resize and thumbnail generation emitted a span, and for months the only thing I learned from them was that the pipeline worked. If I did it again I would instrument the boundaries, the client edge and the storage write, and add the middle only after something in the middle went wrong.

The rule I would actually give: instrument every boundary you cannot see across, and nothing else, until something forces you to. Boundaries are where the unknowns are. The inside of a function you wrote last week is not a boundary.

The other half of the same argument is the test suite: 777 test files across the mobile client and the backend. Tests and telemetry solve the same problem from opposite ends, which is that one person cannot hold a production system in their head. Tests tell you what broke before you shipped it; telemetry tells you what broke after. Neither substitutes for the other, and a solo project with only one of them is a solo project that has chosen which class of failure to be surprised by.