EVALUATION CONTEXTS
Different systems
fail in different ways.
The risks associated with an AI system depend on its role, the information it receives, the actions it can take, and the constraints around it. Evaluation should reflect those conditions rather than treat every application as the same problem.
The contexts on this page describe evaluation concerns relevant to DYMORPIC’s product direction. They do not imply that every scenario is currently supported by PROJECTILE. Product scope and availability must be confirmed.
SYSTEMS COVERED
Customer-service assistants, conversational sales tools, product recommendation systems, and order or account-support workflows.
Conversational systems can behave differently as context accumulates across a conversation. Evaluation should examine the interaction as a whole, including changing instructions, untrusted inputs, and the constraints the application is expected to maintain.
RELEVANT EVALUATION CONCERNS
- AMulti-turn behavior.
- BConflicting instructions.
- CPolicy adherence.
- DUntrusted user-provided content.
- EInconsistent answers across an interaction.
- FActions taken through connected tools, where applicable.
HOW PROJECTILE IS INTENDED TO HELP
Objective-driven evaluation can help teams examine how a conversational system behaves across relevant interactions, identify candidate failures, and determine what evidence is needed to assess them.
SYSTEMS COVERED
Enterprise knowledge assistants, internal search, document-based copilots, and AI features integrated into business applications.
When an AI system works across internal knowledge sources and business tools, its behavior depends on how information and instructions interact. Evaluation should examine whether the system respects its intended constraints and whether its conclusions are supported by the available context.
RELEVANT EVALUATION CONCERNS
- AInstructions embedded in untrusted documents.
- BConflicting source material.
- CUnsupported conclusions.
- DContext boundaries.
- EInteractions between retrieved information, application instructions, and tools.
HOW PROJECTILE IS INTENDED TO HELP
Evaluation can focus on how instructions, retrieved context, and connected capabilities influence the behavior of the wider application.
SYSTEMS COVERED
Customer-facing financial assistants, research copilots, document-processing systems, and other AI-supported operational workflows where appropriate.
Financial and insurance workflows can depend on precise handling of instructions, source material, and defined operating constraints. Evaluation should examine whether observed behavior remains consistent with the intended use of the application.
RELEVANT EVALUATION CONCERNS
- AUnsupported conclusions.
- BConflicting source material.
- CPolicy and instruction conflicts.
- DInconsistent handling of context.
- EResponses that exceed the defined task or information boundaries.
HOW PROJECTILE IS INTENDED TO HELP
Defined evaluation objectives can help teams examine relevant behavior and record evidence for further review.
SYSTEMS COVERED
Administrative assistants, research synthesis, document workflows, and AI-supported knowledge processes where appropriate.
In healthcare and life-sciences contexts, AI-supported workflows may need to work with incomplete or conflicting information. Evaluation can examine how a system handles those conditions and whether its conclusions remain within the scope of available evidence.
RELEVANT EVALUATION CONCERNS
- AConclusions exceeding available evidence.
- BConflicting sources.
- CUntrusted inputs.
- DInstruction consistency.
- ESeparation between supplied information and generated conclusions.
HOW PROJECTILE IS INTENDED TO HELP
Defined evaluation objectives can help teams examine relevant behavior and preserve evidence for subsequent review.
SYSTEMS COVERED
Repository-aware coding agents, issue-to-change workflows, code review assistants, and developer tools with access to files or commands.
Coding agents operate across repositories, issue descriptions, tools, code changes, and command execution. Evaluation must account for how instructions and permissions affect behavior throughout a development task.
RELEVANT EVALUATION CONCERNS
- AConflicting task requirements.
- BUntrusted repository or issue content.
- CInstruction drift during multi-step work.
- DTool and permission boundaries.
- EActions outside the defined task.
HOW PROJECTILE IS INTENDED TO HELP
Evaluation can examine behavior across a development task, including the relationship between instructions, available tools, observed actions, and the intended boundaries.
SYSTEMS COVERED
Browsing agents, research assistants, synthesis tools, and systems operating across multiple sources.
Browser and research agents encounter information that can be irrelevant, contradictory, misleading, or untrusted. Evaluation should examine whether the system maintains its task constraints while interpreting information from multiple sources.
RELEVANT EVALUATION CONCERNS
- AUntrusted web content.
- BConflicting information.
- CTask integrity across navigation steps.
- DUnsupported conclusions.
- EConfusion between information being read and instructions being followed.
HOW PROJECTILE IS INTENDED TO HELP
Relevant evaluation objectives can examine how a research or browser agent interprets source material, maintains constraints, and reaches conclusions.
SYSTEMS COVERED
Document review, contract analysis, research assistants, and internal professional-services knowledge tools.
Document-intensive workflows often require careful treatment of source material and defined instructions. Evaluation should examine whether the system preserves the intended scope of a task and whether conclusions are supported by the available information.
RELEVANT EVALUATION CONCERNS
- AConclusions insufficiently supported by source material.
- BConflicting document instructions.
- CContext drift.
- DInconsistent application of defined constraints.
- EUnsupported generalizations.
HOW PROJECTILE IS INTENDED TO HELP
Evaluation can be structured around the workflow’s expected behavior, the source material available to the system, and the evidence needed to assess a candidate failure.
SYSTEMS COVERED
Workflows involving multiple agents, staged planning and execution, or several components contributing to one outcome.
When several agents or stages contribute to an outcome, failures can emerge between steps rather than within a single response. Evaluation should consider how instructions, context, and decisions move through the wider workflow.
RELEVANT EVALUATION CONCERNS
- AConflicting objectives.
- BInstruction handoffs.
- CInconsistent state or context.
- DFailure propagation between stages.
- EAccumulated assumptions.
HOW PROJECTILE IS INTENDED TO HELP
Evaluation can consider not only individual responses, but also the behavior that emerges as information and decisions pass between stages.
A context that isn’t listed?
These categories describe where DYMORPIC’s evaluation work is directed. If your system combines several of these characteristics, the most useful next step is a direct conversation about the evaluation context.
CONTACT DYMORPIC