APPROACH
From a defined objective
to evidence.
Credible evaluation requires more than identifying an unusual response. A potential failure must be examined in context, supporting evidence must be preserved, and the outcome must reflect what the evaluation actually establishes.
Five stages, one question: what does the evidence establish?
Specify the system, behavior, and constraints under evaluation.
Construct relevant evaluation attempts against the defined objective.
Record the system’s responses and the context needed to interpret them.
Review candidate failures and seek independent verification where supported.
Present evidence-backed findings or mark the outcome inconclusive when the evidence is insufficient.
These stages describe the intended product approach under development. It is not a representation that every stage is currently operational. Availability of specific capabilities should be confirmed before planning an evaluation.
PRECISE LANGUAGE
Four terms, and the
differences between them.
- Observed behavior
- What the system actually did in the context of the evaluation — the recorded response and the conditions around it.
- Candidate failure
- A response that appears unusual, unsafe, or out of step with the objective. A candidate is a signal to examine, not a conclusion.
- Evidence-supported finding
- A conclusion that the preserved evidence supports: the context, the observed behavior, and — where supported — independent verification.
- Inconclusive outcome
- A result recorded when the available evidence does not establish a clear conclusion. It remains visible rather than being forced into a pass-or-fail claim.
EVALUATION PRINCIPLES
Evidence matters more
than an assumption.
Context matters.
An AI system must be considered in the context in which it operates: its instructions, information sources, available tools, and intended constraints.
A candidate is not a conclusion.
An unusual or potentially unsafe response is not automatically a verified vulnerability. The evidence must support the conclusion.
Uncertainty must remain visible.
When an evaluation does not establish a clear result, the outcome should remain inconclusive rather than be forced into a pass-or-fail claim.
THESE PRINCIPLES SHAPE HOW PROJECTILE IS BEING BUILT — WHAT IT RECORDS, WHAT IT CLAIMS, AND WHAT IT LEAVES OPEN.
GO DEEPER