Data handling
Understand evaluation destinations, telemetry content, capture settings, and retained evidence.
Processing and storage are different
Evaluation sends the submitted input and optional output to the configured Nyraxis API. A detector or judge may process that content at another configured model endpoint. A metadata-only storage setting does not remove these processing steps or promise zero network egress.
Keep synthetic data in initial tests. Review your evaluator, model, telemetry, alert, and export destinations before using workload content.
Capture settings
Organization settings establish the capture ceiling. Requests may restrict capture further; they cannot opt into full content when the organization permits only metadata.
| Setting | What it controls |
|---|---|
Organization capture_mode: "metadata" | Retains allowed technical evidence while withholding workload content fields. |
Organization capture_mode: "full" | Permits retained content for applicable records; individual requests can still reduce it. |
Request capture_mode: "metadata" | Restricts that evaluation's stored content. |
Request no_trace: true | Suppresses that evaluation's trace recording, while evaluation and other operational accounting still occur. |
Python instrument_capture_content=False | Requests suppression of prompt/completion attributes in supported instrumentation. |
These controls have different purposes. For example, instrumentation suppression does not stop direct evaluation from sending text, and no_trace is not a promise that no operational record exists anywhere.
What a metadata-only trace can show
Depending on the operation and instrumentor, evidence can include agent and trace IDs, timings, status, model identity, token counts, findings counts, and policy versions. Names derived from prompts may become generic operation labels. Retained violation descriptions and details can be withheld or reduced.
Do not expect the original prompt or tool arguments to be recoverable from metadata-only records. A retest needs new input or a deliberately retained regression payload under the application's data policy.
Reports and alerts
Review an exported report before sharing it. Export behavior follows current capture rules, and older stored content may need a separate retention or remediation process. Policy mode changes do not retroactively change old evaluation outcomes.
For webhook or Slack delivery, review the recipient, configuration, and actual delivered fields. Retention, alert delivery, storage capture, and external processor destinations are independent operational controls.
Rollout checklist
- Record approved API, model/judge, OTLP, alert, and report destinations.
- Confirm capture mode and which roles can change it.
- Inspect actual traces and exports for content fields using synthetic markers.
- Establish retention and deletion procedures for current and historical records.
- Verify the integration's error logging does not print prompts, credentials, or raw API error bodies.
See self-hosted operations and compliance evidence for related scope.