Journal · May 2025

Instrumentation debt in mobile apps

Web products can often ship an event on a Thursday. Mobile products negotiate with stores, devices that never update, and two platforms that only pretend to be the same product.

Instrumentation debt is the gap between the taxonomy you wrote and the binaries still in pockets. A standard that assumes `PaymentFailed` has a reason code is fiction until both stores have a build containing that property, and until a painful percentage of users have installed it. In the meantime your App Analytics will report a tidy event with a null property, and someone will build a dashboard on the nulls.

iOS and Android teams often implement the same verb with different enthusiasm. One platform fires on tap; the other fires on successful navigation. Funnels then “prove” that one operating system converts better. Funnel Forensics exists partly to embarrass this class of finding before it reaches a quarterly deck.

SDK upgrades are another lien. A major version changes identity merge behaviour. Your historical retention is not comparable, even if the chart looks continuous. We teach students to put SDK version in the readout the way a lab notes a reagent lot. It is not pedantry. It is how you avoid claiming that a year of product work moved a line that was moved by a library.

There is also the old binary. Users who declined updates for eleven months are still in your denominator. If you measure a new onboarding event among them, you will invent a drop. Segment by app version or admit that the population includes people who never saw the work.

Priya Nandakumar’s limitation speech on night one of Cohort Instrumentation is about this debt. We can draft a perfect standard in eleven weeks. We cannot force a release train. If your platform freeze lasts until September, start the inventory now, and schedule the hearings after events can actually change. Otherwise you will graduate with literature.

Related reading: Why event names rot. Rooms: course list.