The Score That Has Not Read the Certificate
A supplier prequalification application can be scored automatically, with a rationale, a per-factor breakdown and risk flags. What the score has not done is open a single uploaded certificate — the model reads text, so it scores what was typed and whether the documents are present.
The application scores well. The tax compliance certificate is attached, expired, and belongs to a company with a similar name.
Supplier prequalification is a real bottleneck, and an unglamorous one. A public application form collects a company's details, its registrations, its references and its certificates. Somebody then reads all of it and decides whether this supplier can be invited to quote. At a hundred applications, that is somebody's month.
So an automatic first pass is genuinely valuable, and this one produces more than a number: a score, a written rationale, a breakdown naming each factor with a comment, and risk flags carrying severity levels. It also records which provider produced it, which matters more than it sounds and we will come back to.
What it has not done is read the certificates.
Present is not the same as valid
The models behind this evaluation are text-only. They receive the typed application data and information about which documents were uploaded. They do not open a file.
So the document dimension of the score is completeness: were the required attachments provided. Which is a genuinely useful thing to know, and is a different question from every question a reviewer actually has about a certificate.
What the evaluation can and cannot establish about an attachment
| Question | Answered |
|---|---|
| Was a document uploaded for this requirement? | Yes |
| Is it still in date? | No |
| Is it issued to this company? | No |
| Is it the kind of certificate that was asked for? | No |
| Is it genuine? | No |
| Does it say what the application claims it says? | No |
Built and maintained Configurable by you, not maintained by us Not built
Every row after the first needs a model that can look at the document. This is stated openly in the code rather than being something we discovered by testing it, and it is the single most important thing to know before letting a score influence a decision.
A blank page uploaded under the right filename scores exactly as well as the certificate it is pretending to be.
That sentence is the whole risk, and it points at where the human review effort should go. The typed data has been assessed. The documents have been counted. Reviewing means opening the attachments — which is the slow part, and is now the part that has not been touched.
The uncomfortable dynamic to guard against
An automated score changes what a reviewer looks at even when everybody knows its limits. A high score signals "this one is fine", and the attachments get a quicker glance than they would have received with no score at all. The value here is real and it is concentrated in triage — deciding what order to review in — rather than in reducing what each review involves. Where a score is used to shorten a review, it has shortened the half it never performed.
What it does assess, and does well
The typed application is substantial: company details, registrations, declared capabilities, references, years trading, and whatever else the form collects. Reading that as a whole and forming a judgement about coherence and risk is exactly the kind of task a language model is suited to, and better suited to than a scoring rubric of the sort most prequalification systems use.
A rubric awards points for facts. A model can notice that the declared turnover is inconsistent with the declared headcount, that a reference is at the same address as the applicant, or that a company claiming fifteen years of experience was registered last year. Those are the observations a rubric structurally cannot make, and they are the ones an experienced reviewer makes in the first minute.
The output is shaped so those observations survive: risk flags carry a severity, and the breakdown attaches a comment to each factor rather than only a number. A reviewer can therefore disagree with a specific point, which is the difference between a score they can use and a score they must either accept or ignore.
Failing soft, on purpose
Three things can go wrong: no provider is configured, the request fails, or the response comes back in a shape that cannot be read. All three produce the same outcome — no evaluation, and the reviewer sees that none is available.
Prequalification carries on regardless. The application can be reviewed, approved or rejected by a person exactly as it could before any of this existed.
This is the correct arrangement for an advisory feature and it is worth being explicit about why. An automated assessment that can block a workflow has quietly stopped being advisory — the moment an application cannot proceed because a model was unavailable, the model is in the approval path. Failing to nothing keeps the human decision primary in the failure case as well as the success case.
And a partial score would be worse than none. A score computed from a truncated or half-parsed response looks exactly like a complete one, and there is no field on it saying how much of the assessment actually happened.
Recording which provider scored it
The result carries the name of the provider that produced it, and this deserves more attention than it usually gets.
Where a chain of providers runs in failover order, two applications assessed a week apart may have been assessed by different models with different behaviour. Without recording which, a set of scores looks like one consistent measurement and is not. With it, an unexplained shift in scoring can be checked against a change in which provider was answering — and a supplier disputing a score can be told what assessed them.
A score with no record of what produced it is a number, not a measurement.
What AWRA OpsHub does today
- Automatic advisory scoring of a supplier prequalification application, producing a score, a written rationale, a per-factor breakdown with comments, and risk flags carrying severity levels.
- The score and risk flags persisted on the application, so a later reviewer sees what the assessment concluded rather than only its number.
- The provider that produced each evaluation recorded alongside it, so scores are traceable to what generated them.
- A failover chain of language providers, so one being unavailable does not remove the feature.
- Assessment of the typed application as a whole, which can notice inconsistency between declared facts in a way a points rubric structurally cannot.
- Document completeness assessed — whether the required attachments were provided.
- A soft failure on every error path — no provider, a failed request, an unreadable response — leaving the reviewer with no evaluation rather than a partial one, and prequalification proceeding normally.
- The whole evaluation advisory, with a person still approving or rejecting every application.
- Distinct log entries for an absent provider and an unparseable response, so a feature that has quietly stopped working is diagnosable.
More we can add to your workspace
- Reading the contents of an uploaded certificate, which needs a model that can look at a document rather than only read text. This is the one that changes what the score means.
- Checking an expiry date on an attached document, so a certificate that is present and out of date is distinguishable from a current one.
- Matching the company named on a certificate to the applicant, which is the check a reviewer performs first and the score currently cannot.
- Verification against an issuing authority, so a registration or tax number is confirmed at source rather than assessed as plausible.
- A confidence level on the score, so an assessment made from a sparse application is distinguishable from one made from a complete one.
- Re-evaluation when an application is amended, so a score always describes the version of the application in front of the reviewer.
- Scoring consistency across providers, so a failover to a second model does not shift where the numbers sit.
Where we point you to a specialist
- We hold that this stays advisory and that a person approves or rejects every supplier. Prequalification decides who may be paid by your organisation, and a model that has read no certificate is not equipped to make that decision even when its assessment of the typed application is good.
- We would decline to let an evaluation block an application from proceeding. An assessment that can stop a workflow has stopped being advisory, and an applicant delayed because a provider was unreachable has been failed by a feature that exists to help.
- Where a public procurement regime prescribes how suppliers must be prequalified, what evidence must be assessed and by whom, that regime governs and we implement what it requires. An automated score is not a substitute for a prescribed evaluation, and which regime binds you is a question for your own advisers.
- A score is presented only when the whole assessment completed, and we would keep it that way. An assessment computed from a truncated response looks identical on screen to a complete one, so the honest options are the full number or none.
Reading the uploaded documents is the piece that changes what this feature is, and it is a well-defined change: a vision-capable model, the same output shape, and expiry and company-name checks arriving with it. If prequalification volume is your bottleneck, it is the one to scope.
From counting documents to reading them
The pipeline around the evaluation is built: applications, documents, the scoring call, the persisted result, the reviewer's screen. What changes the value of it is what the model can see.
A model that opens the file
The single change everything else here depends on. Same output shape, same advisory position, a score that has actually looked at the certificate.
Expiry and company name
The two checks a reviewer performs first, and the two that catch most of what a completeness count misses.
A confidence level
So a score from a thin application reads differently from one from a thorough one, rather than both arriving as a bare number.
We publish scope, not dates.
Scope supplier evaluationFive questions for any vendor offering automated supplier scoring
Does the assessment read the uploaded documents?
A good answer sounds like
A straight yes or no.
What ours actually is
No. It scores the typed application and whether documents are present. Ask this first and insist on a plain answer — "it evaluates the documents" is compatible with counting them.
Does it check whether a certificate has expired?
A good answer sounds like
Yes, and here is where the date is shown.
What ours actually is
Not today. This is the question that separates reading from counting, and it is harder to answer vaguely than the previous one.
Can the score block or approve an application?
A good answer sounds like
No, a person decides.
What ours actually is
No. Advisory throughout, and the evaluation failing leaves prequalification working normally.
Do you record which model produced a score?
A good answer sounds like
Yes, on every result.
What ours actually is
Yes. Worth asking wherever a failover chain exists — otherwise a set of scores from different models looks like one consistent measurement.
What does a reviewer see when the assessment fails?
A good answer sounds like
Nothing, clearly labelled as unavailable.
What ours actually is
That no evaluation is available. Be wary of a system that shows a partial score, which reads identically to a complete one.
Our take
This is a useful feature described accurately in its own source, which is rarer than it should be — the limitation at the centre of this article is written down in the code rather than being something we found by probing it. Used for triage it earns its place: a hundred applications ordered by assessed risk, with flags naming what looked wrong, is a materially better starting point than a hundred applications in the order they arrived. Used to shorten a review it is actively dangerous, because the half it shortens is the half it never performed. The document check is completeness, and the gap between a document being present and a document being valid is where every prequalification failure we have ever heard about actually lives. Until a model can open the file, treat the score as a reading order and open every certificate yourself.
Decide what the score is allowed to change
Before switching this on, agree one thing with whoever reviews applications: does the score change the order of review, or the depth of it? The first is where the value is. The second is where the risk is, and the distinction is worth writing down before the first hundred applications rather than after.
Talk through prequalificationFrequently asked questions
Does the AI read our suppliers' certificates?
No. The models used are text-only, so the evaluation scores the typed application data and whether the required documents were uploaded. It never opens an attachment. This is stated openly in the code rather than being something we discovered, and it means a certificate that is present, expired and issued to a different company scores exactly as well as a valid one.
What does the evaluation actually produce?
A score, a written rationale, a breakdown naming each factor with a comment against it, risk flags carrying severity levels, and the identity of the provider that produced the assessment. The breakdown matters more than the number: it lets a reviewer disagree with a specific point rather than having to accept or ignore the whole thing.
Can the score approve or reject a supplier?
No. It is advisory throughout and a person makes every decision. That extends to the failure case, which is deliberate: if no provider is configured, the request fails, or the response is unreadable, the reviewer simply sees that no evaluation is available and prequalification proceeds exactly as it would have without the feature. An assessment able to block a workflow has stopped being advisory.
Why record which provider produced the score?
Because the providers run in failover order, so two applications assessed a week apart may have been assessed by different models with different behaviour. Without that record a set of scores looks like one consistent measurement when it is not, an unexplained shift in scoring cannot be traced, and a supplier disputing their assessment cannot be told what produced it.
How should we use it without over-trusting it?
Use it to decide the order of review rather than the depth. A hundred applications sorted by assessed risk, with flags naming what looked inconsistent, is a much better starting point than a hundred in arrival order. What it must not do is shorten the review of a high-scoring application, because the attachments are the part it has not examined and they are where prequalification failures actually occur.
What would it take to check the documents properly?
A model capable of looking at a document rather than only reading text, which is a defined change rather than a research problem. The pipeline around it already exists — applications, attachments, the scoring call, the persisted result and the reviewer's screen — so the work is the model and the checks that become possible with it, principally the expiry date and whether the company named on the certificate is the applicant.
