The Comparison That Never Fired
A guard stopped automatic scoring from relabelling quotations a buyer had already awarded. It was written, reviewed and correct in intent, and it never once fired — because one side of the comparison was lowercase and the other side was not.
The protection was in the code, in the right place, doing the right thing. It had also never run.
Quotations against a request can be scored automatically, ranked, and labelled — this one analysed, that one the best quote. It saves a buyer the arithmetic of comparing six quotes across price, lead time and whatever else matters.
Automatic labelling has an obvious hazard. A buyer who has already made a decision — approved a quotation, awarded part of one, checked a delivery in — must not have that decision overwritten by a scoring run that happens afterwards.
So a guard was written. Before scoring, check whether any quotation on this request carries a status meaning a human has already acted, and if so, stop.
Why it never fired
The statuses meaning "a human has acted" were held as constants, written in lower case. The code that approves a quotation writes them in title case.
The comparison was strict, so it asked whether the stored status was exactly the constant. It never was.
A strict comparison here never fired, and every awarded request was re-scored anyway.
Read that as a description of what a reviewer would have seen. The constant list is correct. The check is in the right place, before anything is written. The intent is documented in a comment directly above it. Every element is right, and the composition does nothing at all — because two pieces of code, written at different times for different purposes, chose different capitalisation for the same word.
The shape of this defect
It is worth separating three things that get muddled when this kind of bug is described.
| Situation | How visible is it? |
|---|---|
| No guard at all | Visible. A reviewer asks what stops this from overwriting a decision. |
| A guard that throws | Visible immediately. It fails loudly the first time it is wrong. |
| A guard that can never be true | Invisible. It reads as protection and behaves as absence. |
The third is the worst of the three and the only one that survives review, because reviewing means reading, and the code reads correctly. Nothing about a strict comparison between a constant and a column suggests that the two are written in different cases — you would have to know what the other code path writes, and that code path is somewhere else.
Where to expect this in your own systems
Anywhere a value is written by one component and compared by another. Status strings are the classic case, because they are usually introduced as constants and then written by hand somewhere as a literal. The tell is that the two sides have different authors. Where a comparison decides whether something is protected, the useful discipline is to assert that the guard fires — a test proving it stops the operation, rather than a test proving the operation works.
The second layer, which held
One reason this did not cause more damage than it did is that the guard was not the only protection.
Even when analysis runs, the updates that relabel quotations exclude protected statuses at the point of writing — in the query itself, comparing without regard to case. So a scoring run reaching an already-approved quotation would still not have relabelled it.
That is defence in depth doing what it is for: the outer check failed silently and the inner one was still standing. It is also the reason the failure went unnoticed, which is the honest other half of the observation — a redundant protection that saves you also hides the fact that the first one is gone.
What was actually lost was work rather than correctness. Every awarded request was re-scored on every run, spending a language model call and a set of database writes on a decision that had already been made.
Two details worth borrowing
Rejected is protected from relabelling but does not stop the run. A rejected quotation is excluded from the updates, so scoring never resurrects it into "analysed" or "best quote". But its presence does not prevent the analysis from running, because rejecting one quote out of six is not a decision about the request — the buyer is still choosing among the rest.
That distinction is precise and easy to get wrong in either direction. Treat rejected as finalising, and one early rejection freezes the scoring for the whole request. Treat it as unprotected, and a quote somebody deliberately dismissed comes back labelled as the best one.
The guard is a parameter, defaulting to on. A caller can deliberately re-score a finalised request by asking for it explicitly. Protection by default with an explicit override is the right way round, and it means the guard does not have to be weakened for the one legitimate case where somebody wants a re-run.
Four questions to ask about any automatic labelling
Can automatic scoring change a status a person set?
A good answer sounds like
No, and here is the guard.
What ours actually is
No — protected at the point of writing as well as by a check before the run. Ask for both: a single check is one silent failure away from nothing.
Is there a test proving the guard stops the operation?
A good answer sounds like
Yes, asserting it refuses.
What ours actually is
The failure described here is exactly what such a test catches, and exactly what a test of the happy path does not. Worth asking of any protection anywhere.
Does a rejected quote stop the analysis or just resist relabelling?
A good answer sounds like
Resists relabelling, does not stop the run.
What ours actually is
Resists relabelling. Rejecting one quote of six is not a decision about the request, and treating it as one would freeze the comparison.
Can you deliberately re-score something already decided?
A good answer sounds like
Yes, explicitly, and it is not the default.
What ours actually is
Yes, by asking for it. Protection by default with an explicit override beats weakening the guard for one legitimate case.
What AWRA OpsHub does today
- Automatic scoring and ranking of quotations against a request, labelling those analysed and identifying a best quote.
- A guard preventing analysis from running once a person has approved, partly awarded or checked in a quotation on that request — now comparing without regard to case, so it fires.
- Case-insensitive exclusion of protected statuses in the update queries themselves, so a scoring run cannot relabel a decided quotation even if it reaches one.
- Rejected quotations excluded from relabelling while not blocking the analysis, so dismissing one quote does not freeze the comparison of the rest.
- Request-level statuses that mean the request is finalised excluded from relabelling.
- The skip behaviour exposed as a parameter defaulting to on, so a deliberate re-score is possible without weakening the guard.
- Analysis that fails soft on a missing provider or a failed request, so scoring being unavailable never blocks a request from proceeding.
More we can add to your workspace
- A test asserting the guard refuses, proving the protection stops the operation rather than proving the operation works — which is the test that would have caught this on the day it was written.
- One shared source for status values, so the code writing a status and the code comparing it cannot choose different capitalisation for the same word.
- A record of when a request was last scored and what changed, so a re-score is visible rather than silently replacing the previous labels.
- A count of scoring runs against a request, which is what would have made the wasted re-analysis of awarded requests noticeable.
- A note on the request when scoring was skipped because a person had acted, so unchanged labels read as a deliberate skip rather than as a failed run.
Where we point you to a specialist
- We hold that automatic scoring never overwrites a decision a person has recorded, and we would keep both layers of that protection even though either alone would usually be enough. The failure described here is precisely why: a single guard is one silent mismatch away from no guard, and nothing about the silence announces itself.
- We would decline to let a scoring run relabel a rejected quotation. Somebody dismissed it deliberately, and a system that brings it back as the best quote has overridden a judgement rather than assisted one.
- Where a procurement regime prescribes how quotations must be evaluated and by whom, that regime governs and automatic scoring is a preparatory aid rather than the evaluation. Which rules bind a given procurement is a question for your own advisers.
A shared source for status values, and a test that asserts each guard refuses rather than that the operation succeeds, are the two that prevent this class of defect rather than fixing one instance of it. Both are contained work with effects well beyond this service.
Anything above that you need, we can build for you
Everything listed above as something we can add describes what ships in the standard product today — it is a starting point, not a limit on what AWRA OpsHub can do for your organisation. Kenya's eTIMS integration and its maintained payroll engine are both in the product because clients needed them and commissioned them; neither appeared by itself, and the same door is open for whatever you just read about. One qualification so this is worth what it claims: a small number of things on this blog we deliberately leave to a specialist rather than build — a statutory ledger we will not sign our name to, a rule that would decide a tax question for you, a clinical or member-funds record that belongs in a regulated system — and where that is true the post says so in those words. Everything else is a scope, a timeline and a price.
The operational work, which is what most commissions actually are
An extra approval stage in a chain that does not match the standard one, a custom field set on employees or assets that only your sector needs, an expiry that has to block an order rather than send an email, a report your board asks for in a shape nothing produces, or a scanner or weighbridge feeding the goods-in door. These are the commissions we are asked for most often and the smallest ones we quote — and unlike a revenue-authority pipeline, none of them waits on a regulator.
The module-shaped additions, which are the ones readers ask for most often
A price list with real discount authority, a customer-facing quotation that expires, a bill of materials or recipe costing, a staff advance that is issued, acquitted and chased, a member or unit ledger, a matching rule that holds a payment. Each of these is a build rather than a setting, and each has been quoted before — a bigger piece of work than a custom field, with a written spec and a date instead of a roadmap slide.
The report, document or pack nothing currently produces
The board pack in the shape your board actually asks for, a donor or funder layout, an invoice or receipt template carrying what your regulator or your customer expects, a dataset the report builder cannot reach yet. Usually the fastest thing on this list to deliver, because the data is already in the system.
Systems, rails and hardware you already run
The accounting package, CRM, online store, core banking or custom database you intend to keep — connected through our API so a fact is entered once and appears everywhere it is needed. Plus the physical edge: a scanner, a scale, a weighbridge or a till peripheral feeding the door it belongs to.
How it works: you describe the requirement, we return a written scope, timeline and cost, and once agreed it is built into your environment and maintained as part of the product. Nothing here waits on a regulator or a published specification, which is why operational builds are the ones we quote fastest. Tell us the requirement that would otherwise rule us out — that is a better first conversation than a demo.
Tell us what your operation needsOur take
The bug is small and the lesson is not. A guard that can never evaluate true is the worst kind of protection to have, because it is indistinguishable from a working one at every point a person would look — the constant list is right, the placement is right, the intent is documented, and the comparison is a single line that reads correctly. What made it survivable here was a second layer written independently at the query level, which is defence in depth working exactly as intended and also the reason nobody noticed for a while. The discipline that would have caught it on day one is a test asserting the guard refuses. Almost every test suite proves the operation works; comparatively few prove that the thing supposed to stop it actually stops it, and a protection nobody has watched fire is a protection nobody has evidence of.
Pick one guard in your own code and prove it fires
Not that the operation works — that the guard refuses. Write the case it is supposed to stop and assert that it is stopped. It takes twenty minutes and it is the only way to distinguish a protection that works from one that reads correctly and has never once evaluated true.
Talk through quotation analysisFrequently asked questions
What exactly went wrong?
A check meant to stop automatic scoring from relabelling quotations a buyer had already acted on compared the stored status against a list of constants. The constants were lower case; the code that approves a quotation writes them in title case; and the comparison was strict. So it never matched, the check never stopped anything, and every awarded request was re-scored on every run.
Did that overwrite anybody's decisions?
No, because a second protection held. The queries that relabel quotations exclude protected statuses at the point of writing, comparing without regard to case, so an already-approved quotation would not have been relabelled even when the analysis ran. What was lost was work rather than correctness — a language model call and a set of writes spent on requests that had already been decided.
Why did code review not catch it?
Because everything a reviewer reads is correct. The list of protected statuses is right, the check sits before anything is written, the intent is documented in a comment directly above it, and the comparison is one clear line. Noticing the defect requires knowing what a different code path, written at a different time, writes into that column — and that code is somewhere else entirely.
Is a rejected quotation protected?
From relabelling, yes — scoring will not turn a rejected quote into an analysed one or a best quote. But a rejection does not stop the analysis running, and that distinction is deliberate. Rejecting one quote out of six is not a decision about the request; the buyer is still comparing the rest, and treating an early rejection as finalising would freeze the scoring for everybody.
Can we deliberately re-score a request that has been awarded?
Yes. The skip behaviour is a parameter that defaults to on, so a caller can ask for a re-score explicitly. That is the right shape: protection by default with a deliberate override, rather than weakening the guard to accommodate the one case where somebody genuinely wants a re-run.
How do we avoid this in our own code?
Two habits. Keep one source for a value that is written in one place and compared in another, so the two cannot choose different capitalisation for the same word. And write a test that asserts the guard refuses, rather than a test that asserts the operation succeeds. Almost every suite proves the happy path; a guard nobody has watched fire is a guard nobody has evidence of.
