Most of the people filling in those annual vendor risk assessments are doing their best with limited time and imperfect information. The problem is not dishonesty. The problem is that questionnaires are structurally incapable of answering the questions that matter most for supply chain risk.

This became clear to me when we ran an assessment for a large financial services firm. Their top-tier supplier had a clean questionnaire, a current SOC 2 Type II report, and an ISO 27001 certification issued eight months prior. Our platform found 23 outdated software components on the infrastructure serving APIs directly to the client's production environment. Several of those components had known exploits. None of them were covered by the audit scope.

The supplier had not lied. The questionnaire had not lied. But the resulting risk picture of 'this supplier is secure' was false.

The Fundamental Problem with Self-Assessment

Questionnaires ask suppliers to describe their security controls. That description is filtered through the supplier's own understanding of their environment, their interpretation of the questions, their appetite for disclosure, and the bandwidth of whoever is filling in the form. Even with the best intentions, the output is a representation of how the supplier perceives their security posture, not a measurement of it.

This matters because perception and reality diverge constantly in complex systems. A supplier may have excellent patch management policies and genuinely believe their production systems are up to date, while a legacy integration environment that predates the current security program runs unpatched software on a subdomain nobody visits anymore. The questionnaire captures the policy. Our platform finds the subdomain.

The gap between policy and reality is systemic:

What Evidence-Based Risk Assessment Looks Like

The alternative is not to kill questionnaires. There are things questionnaires measure well:  

• Organizational policies

• Incident response procedures

• Contractual commitments

• Insurance coverage

But these are inputs to the broader risk picture. The primary instrument should be technical evidence: live, observable, continuous data about what the supplier's digital environment actually looks like and how it connects to your organization.

This requires a different starting point. Instead of asking the supplier to describe their environment, you observe it. You map their external digital surface — the applications, APIs, infrastructure, and third-party connections visible from the internet, like an attacker would do.  

You score the hygiene of the specific components that connect to your systems, not the supplier's overall posture with a generic security rating. You understand the nature and depth of the digital relationship: is this a supplier with a single DNS record, or one whose JavaScript is executing on your customer-facing applications?

The difference in risk exposure between those two scenarios is enormous. The questionnaire will not tell you which one you have.

The Liability Dimension

There is a governance dimension to this that boards and risk committees are beginning to take seriously. When a breach occurs via a third party and the post-mortem reveals that the affected organization had the technical means to discover the vulnerability and chose not to deploy them (relying instead on questionnaires and certifications) the legal and regulatory exposure is significant.

Regulatory frameworks including NIS2, DORA, and emerging supply chain security legislation are moving in the direction of technical due diligence requirements. The expectation is shifting from 'we asked the supplier about their security' to 'we continuously monitored the technical posture of our most critical supplier relationships.' The questionnaire alone will not satisfy regulators who have access to the same discovery tools we do.

In conversations with GRC leaders across financial services, transport, and government, a consistent theme emerged:  

Technical discovery raises awareness, which raises the obligation to act. Organizations that are about to invest seriously in supply chain risk management need to be prepared for that accountability.

The Cost of False Confidence

Beyond the legal dimension, there is a resource allocation problem. Supply chain risk teams spend significant hours collecting questionnaires, chasing responses, reviewing certifications, and maintaining risk registers that are based on supplier self-reporting. That effort produces a compliance artefact. It does not produce an accurate risk picture.

If those same resources were redirected to evidence-based assessment, such as technical discovery, continuous monitoring, integration with remediation workflows, the risk picture would be fundamentally different. And the time spent would produce genuine security improvements.

A More Honest Starting Point

If you are rebuilding or maturing your TPRM program, start by asking: what do we actually know, from direct technical observation, about the state of our digital supply chain?

First, map the real connections: understand which suppliers are embedded in your production environment, which components they are serving, and what the technical hygiene of those components looks like today.  

Then, layer in questionnaires and certifications as context for the technical findings, as evidence to either validate or challenge what the data shows.

TPRM as a discipline is not broken beyond repair, but it has been measuring the wrong thing for too long. Luckily, the technology now exists to measure the right thing.

‍