A Logging Standard That Never Gives Away PII
Photo: Negative Space · StockSnap (CC0 1.0), via Openverse
There's a moment in every project when you're sitting in a log file at three in the morning trying to reconstruct why a particular user couldn't sign in. In that moment, you find out very quickly whether your own logs are any good. Either you find the relevant line in seconds, or you scroll through tens of thousands of entries of free text, half-formatted exceptions, and lines nobody can make sense of anymore.
Xircuit processes health data. That sharpens the problem in both directions: I need logs that are informative enough to diagnose real problems, and at the same time, under no circumstances can these logs contain anything that would make a person identifiable or reveal their health status. This week is about how we poured these two requirements into a single, binding logging standard.
Why Free-Text Logs Don't Scale
In the beginning, everyone logs however seems sensible to them at the time. One person writes "User logged in", the next writes "login ok for account", a third stuffs an entire stack trace into an info message. As long as a single service is running, that works. But once you're operating multiple services, workers, and an app across several platforms, searching turns into a guessing game. You never quite know which string you're actually supposed to filter for.
The real problem isn't the volume of logs, but their lack of structure. A log entry that can only be found through full-text search is, in the worst case, not findable at all. And in a distributed system orchestrated by Aspire, whose telemetry converges into a central observability pipeline, that findability is exactly the decisive factor. So we didn't want better log text — we wanted a convention that makes every single line attributable without having to think about it.
A Tag Scheme: Domain.Process.Layer
The core of the standard is a structured tag that every log line carries with it. It follows the scheme <Domain>.<Process>.<Layer>. The domain describes the business area, the process the concrete workflow, the layer the technical level at which the entry originates. A tag like Auth.Login.Service tells you immediately what it's about, without me having to read a single word of free text.
The payoff is that attribution no longer has to be guessed from the message text. Instead of searching for vague phrasing, I filter by a stable, predictable prefix. All lines of a domain, all lines of a process across multiple services, all lines of a particular layer — every one of these views is just one filter away. In a structured logging world like .NET's, this fits in seamlessly: the tag is its own indexable field, not an interpolated piece of text.
This discipline sounds petty, but it fundamentally changes debugging. You stop reading logs and start querying them.
The Hard Rule: Never Log PII
The second pillar is a rule with no room for discretion: personal or health-related information never ends up in Xircuit logs. No email addresses, no names, no measurement values, no health data of any kind. This rule is deliberately phrased as absolute, because every exception immediately becomes a gray area, and gray areas are unacceptable when it comes to health data.
Concretely, that means we work consistently with opaque identifiers instead of the actual values. I don't log that a particular person recorded a particular blood glucose reading, but that an operation took place for an anonymous identifier. For diagnostics, that's almost always enough, because what I need for troubleshooting is the flow and its outcome, not the content of the data. The value itself belongs in the database, behind access control and encryption, and never in a log line that potentially travels through multiple systems.
Photo: Rawpixel (CC0 1.0), via Openverse
AuthLog.Forbid: Making Denials Traceable
One area where the need for informativeness and the need for data minimization particularly clash is authorization. When access is denied, I want to be able to understand later why — but this is exactly the place where the temptation is greatest to log precisely the identity and context that should never be written into a log.
For that, Xircuit has a dedicated helper: AuthLog.Forbid. It's the central point through which every authorization denial is logged. It records which rule applied and in what context the decision was made, but it does so fundamentally without PII. Because all denials run through the same helper, they're uniformly formatted, uniformly tagged, and can be analyzed as a group. If a series of access attempts suddenly starts failing at the same spot, I see the pattern without having to read a single line that would allow an inference about a real person.
The real value lies in the fact that a denial is no longer silent. An "access denied" with no traceable reason is just as frustrating in operation as a silent failure. AuthLog.Forbid turns every rejection into an explainable, yet privacy-compliant, entry.
Boundary Logging at the Auth Boundaries
Sign-in problems are the most thankless kind of error, because they hit the user right at the door and because so many moving parts are involved. That's why we treat the boundaries of the authentication flow separately. Login, token refresh, and account recovery each have their own boundary logging that makes the transition across these boundaries diagnosable.
The idea behind it is that at any boundary, the question always stands: did the request come in cleanly, and did a clean response go back out? If both ends are logged, a failed login can be pinpointed precisely — does it break off at entry, somewhere in the middle, or only when the new token is issued. Especially when interacting with an external identity provider, this pinpointing is worth its weight in gold, because without clean boundaries you spend hours figuring out which side of the boundary the problem is even on. And here too, it applies without compromise that actual credentials have no business being in the logs; what's logged is the course of events, not the content.
A Standard Every New Service Inherits from Day One
The best part of all this would be worthless if it only existed in my head. That's why the logging standard is documented as a convention, not as a loose collection of habits. Anyone setting up a new service follows it from the very first commit, instead of laboriously retrofitting it later.
This decision was made deliberately. Logging discipline can't be retrofitted into a grown system without it hurting — the tags are missing in a thousand places, the PII leaks have already happened, and nobody dares touch the old lines anymore. By having the standard apply from the beginning, doing the right thing becomes the path of least resistance. New services inherit the convention instead of reinventing it, and the entire observability pipeline stays consistent no matter how many more building blocks get added.
What's Next?
Next week: database-backed feature flags that let us toggle functionality at runtime and pause workers within seconds.
Comments
No comments yet. Be the first to share your thoughts!
Leave a Comment