EVALUATION #IM-0042
ILLUSTRATIVE EXAMPLE — NOT LIVE DATA
- Evaluation
- #IM-0042
- Target
- Customer Support Agent
- Attack
- Tool Abuse
- Observation
- Unauthorized tool invocation
- Judge
- Verifier
- 7/8 reproductions
- Severity
- HIGH
- Evidence
- Captured
PROJECTILE / PROTOTYPE
PROJECTILE
Test beyond
the expected path.
PROJECTILE challenges intelligent systems to expose potential weaknesses, examines candidate failures, and preserves the evidence needed to assess what occurred.
IN PROTOTYPE — PUBLIC BETA PLANNED FOR JANUARY 2027 · NOT ALL DESCRIBED CAPABILITIES ARE OPERATIONAL TODAY
HOW AN EVALUATION IS INTENDED TO UNFOLD
From objective to outcome,
with room to change direction.
FIG. 00401 / 06
ILLUSTRATIVE EVALUATION SEQUENCE — NOT A LIVE PRODUCT TRACE
01 / DEFINE
Define the objective
The evaluation begins with a defined target behavior and a clear scope. The objective determines what is being examined and which constraints matter.
CORE CAPABILITIES
Built to press
on what holds.
Find vulnerabilities
within an hour.
PROJECTILE is designed to help teams surface and investigate potential weaknesses quickly, with a target of finding vulnerabilities within an hour under suitable evaluation conditions. Evaluation time depends on the target system, scope, configuration, and test conditions.
PROJECTILE is currently in the prototype stage.
Public beta is planned for January 2027. The beta timeline is a plan, not a guaranteed release date.
CAPABILITY SET / 04
- 01
30,000+ base strategies
Evaluations draw on a substantial base of more than 30,000 strategies for challenging intelligent systems — a wide foundation that testing and self-learning build on.
- 02
Self-learning
PROJECTILE learns from attacks and their outcomes, using results to sharpen subsequent testing and strategy selection rather than repeating a fixed sequence.
- 03
Multi-turn adaptive stress testing
Evaluations run across multi-turn interactions and adapt their direction in response to observed behaviour — because failures often emerge through accumulated context, changing instructions, and tool use rather than isolated prompts.
- 04
Continuous coverage, 24/7
Designed to support continuous evaluation coverage rather than one-off manual testing, so a system can be examined as it changes — not only on the day someone checks.
THE RECORD
What a finding
looks like.
Every candidate is judged, reproduced by the independent verifier, and only then trusted.
The record below shows the structure of an evaluation result. It is an illustrative example — not live data, and not a real evaluation of a real system.
- Candidate
- A potential issue identified during evaluation, requiring assessment. Not a confirmed finding.
- Verified finding
- A candidate supported by the applicable verification and evidence process.
- Inconclusive
- Insufficient evidence to establish the result confidently — recorded as such, not forced.
RESEARCH MODE
Built for security research,
not just dashboards.
Research runs track everything needed to reproduce, compare, and defend a result.
- 01
Benchmarks & Baselines
Versioned benchmark suites with static, single-pass, and autonomous baselines — so evaluation approaches can be compared and changes tracked across runs.
- 02
Novelty
Distinguishes findings the engine surfaced for the first time from weaknesses already represented in known evaluation results. Novelty is not a claim of global uniqueness across all models or research.
- 03
Reproduction
Verification trials and reproduction rates are recorded for every confirmed finding, making results more useful for investigation and comparison.
- 04
Provenance
Model, provider, runner version, git commit, seeds, and evidence hashes are persisted — the context needed to understand and reproduce a result.
- 05
Cost
Token and dollar budgets are enforced and reported per run, per attack, and per finding, so resource consumption is visible at every level of an evaluation.
- 06
Target Versions
Results are pinned to the exact target version they were produced against, separating genuine behaviour changes from differences in the system under test.
- 07
Exports
JSON and CSV exports for downstream analysis and reporting, so findings can flow into existing research and review workflows.
- 08
Evidence Hashes
Evidence artifacts are hash-addressed, supporting integrity checks and independent auditing. Hashing supports auditability; it does not by itself establish the truth of a finding.
INTENDED METHODOLOGY
What the process is
designed to establish.
The descriptions below explain the intended methodology. They are not a representation that every capability is currently implemented. Availability of specific capabilities should be confirmed before planning an evaluation.
- 01
Objective definition and scope
Each evaluation starts from an explicit objective: the system, the behavior of interest, and the constraints that matter. A clear scope keeps the result interpretable.
- 02
Evaluation attempts and observed responses
Relevant attempts are constructed against the objective, and the system’s actual responses are recorded rather than assumed.
- 03
Adaptive testing and changes in direction
Where supported, observed behavior informs where further examination may be useful. This is an intended capability under development, not a fixed checklist.
- 04
Candidate failure identification
An unusual or potentially unsafe response is identified as a candidate — a signal to examine, not a verified failure.
- 05
Contextual examination
A candidate is reviewed in the context of the objective, the surrounding instructions and information, and the observed sequence of behavior.
- 06
Evidence preservation
The responses and context needed to interpret a result are preserved, so an outcome can be assessed rather than merely asserted.
- 07
Independent verification where supported
Where the implementation supports it, candidate failures are re-examined independently to check whether the evidence supports the conclusion.
- 08
Evidence-backed and inconclusive outcomes
A finding is reported only when the evidence supports it. When it does not, the outcome remains visibly inconclusive rather than being forced.
See it for yourself —
privately.
PROJECTILE is demonstrated privately, in controlled settings. Request a demonstration for your research or enterprise team, or contact DYMORPIC to discuss the evaluation context and current product capabilities.
PROJECTILE is currently in the prototype stage.
Public beta is planned for January 2027. The beta timeline is a plan, not a guaranteed release date.