References and reconciliation

A StoredFile is destroyed only when nobody needs it any more, and FileReference is what answers "who needs it". This page describes how that link is created and let go of, how the module keeps a cheap answer to "is anybody holding this", and the nightly routine that repairs that answer when it drifts.

The link belongs to the consumer

A reference is a pair of opaque strings: ownerScope names the consumer holding the file (nexio.attachment, gryd.report-execution) and ownerKey names one row inside that consumer. The module stores both exactly as they arrive. It does not validate that the owner exists, does not keep a table of scopes and never translates a scope into a business name — doing any of that would be GrydFiles claiming knowledge of a consumer's model.

Two rules shape everything else:

• A reference may only be created over an Available file. Nothing is referenceable before the antivirus verdict (RN-GF-01). Every other state refuses with a code of its own, so a screen can tell "still being checked" from "we found something in this".
• Releasing is an append, not a delete. releasedAt and releasedBy are written and the row stays. Who held a file, and until when, is part of the evidence a retention dispute turns on.

Registering the same pair twice is an operation rather than an error: the second call returns the row the first one made. Releasing twice answers the same as the call that released. A consumer may reprocess a message or re-run a job without having to know which of the two it is doing.

The counter is a cross-check; the table is the truth

Two denormalised fields live on StoredFile beside the child rows:

Field	What it holds
referenceCount	How many FileReference rows of this file are still active.
lastReferenceReleasedAt	When the release that emptied the count happened.

They exist so the retention sweep can select purge candidates in an index scan over (status, referenceCount, lastReferenceReleasedAt) instead of aggregating the child table once per candidate. That matters more than it looks: releasing is an append, so the child table only ever grows, and a per-candidate aggregation would get more expensive every day the platform runs.

The two fields are one fact in two columns, and the pairing is an invariant rather than a convention:

• while the count is positive, the date is null;
• when the count reaches zero, the date is the most recent release;
• a file nobody ever referenced has a count of zero and no date, because there is no release to date.

A release that leaves the file still held writes nothing — nothing about when the file was last let go of has changed. A new reference that brings the count back from zero to one clears the date, because the file stopped being a purge candidate and the clock it was counting down from stopped meaning anything.

Making that a strict invariant is what lets the routine below treat any other combination as damage instead of having to guess whether somebody meant it.

The counter never authorises a deletion. It selects candidates; the decision to destroy content is confirmed against the active rows every time. A count of zero is enough to make a file a candidate and never enough to run the delete.

The nightly reconciliation

ReferenceReconciliationJob (gryd-files.reference-reconciliation, queue files) runs at 03:00 and returns both fields to what the child rows say.

It works in two phases. A projection asks the database which records disagree with their own children — answered without materialising a single aggregate — and only those are then loaded with their history and repaired. On the common run it reads one query and loads nothing at all.

The recomputation itself lives on the aggregate, in StoredFile.ReconcileReferences, and not in the job. It is the same rule that adding and releasing a reference apply, and a second copy of it in a job would be one edit away from disagreeing with the first — with the job's copy winning, because the job is the one that overwrites.

Like the expiry sweep, it runs with the tenant filter off: a tenant-scoped reconciliation would leave every record it could not see diverging forever, which is the opposite of a safety net. Product files, which belong to no tenant, are covered for the same reason.

Reading its output

A quiet run logs one informational line saying every record agreed with its rows.

A correction is logged at warning level, per record and in total, and it is a defect signal. Repairing the pair does not remove whatever wrote it wrong, so the number to watch is not the size of a single run but whether corrections keep coming back. Each line carries the file id and the before and after of both fields — never an object key or a bucket name (RN-GF-07).

The two directions fail differently, and both are worth naming:

• A count stuck above zero means the file is never even a candidate for the retention sweep. It is kept for as long as the system runs, silently, and the only symptom is storage that does not shrink.
• A count wrongly at zero, or a date older than it should be, points the other way: the file becomes a candidate earlier than it should, and the grace period is the last thing standing between it and deletion.

Because a divergence takes a defect, a lost race or somebody editing the database by hand, a run that corrects nothing is the expected result. Nightly is deliberate: the pair it repairs is read by the retention job, whose grace period is measured in days, so a divergence that survives a few hours changes no outcome — while a scan every few minutes would spend a full aggregate query to find nothing.

Who is holding a file

StoredFile.ActiveReferences reads the child rows, not the counter, and returns the pairs exactly as they were written. It is what fills the body of a purge refused by an active reference: the response lists the ownerScope values still pointing at the file, distinct and ordered.

The list is the file's own holders. A reference on another record sharing the same blob through dedupe does not refuse this file's purge — that reference is the other record's custody — it only keeps the shared bytes alive (see Deciding the fate of shared bytes in the dedupe page).

Scopes, not keys. The scope names the consumer that has to be talked to; the key names one row inside it, and a file held by four hundred rows of the same consumer would otherwise answer with four hundred lines that all say the same thing.

---

The decisions behind this page are in ADR 0010; the contract is in the specification.