A support assistant gives a wrong answer. You enable conversation capture to see what the model received, and the next trace contains a private ticket and account details returned by a tool. Those details now live in your telemetry backend too.
Before enabling capture, decide what to record, who can read it, and when to delete it. OpenTelemetry cannot determine which data is sensitive in your application. Its guidance recommends minimizing collection and reviewing instrumentation output.
Write down the recording policy
For that assistant, I would start here. The retention periods are examples. Choose yours for the data and investigation needs.
| Data | Recording rule | Access and retention |
|---|---|---|
| Metadata such as duration, token counts, model alias, prompt version, outcome code, trace ID | Allow only explicitly approved metadata fields. Keep values free of customer content. | Service operators; delete after 30 days. |
| User messages, model responses, retrieved text, tool arguments and results | Exclude from normal telemetry. Capture selected fields only through a separate, reviewed diagnostic path for a named investigation. | Assigned investigators; delete after 24 hours. |
| Credentials, access tokens, connection strings | Never retain, including inside captured text. | Drop the diagnostic sample if these cannot be removed reliably. |
The service owner maintains the table and configuration. Diagnostic capture needs explicit opt-in specifying the investigation ID, permitted fields, destination, and expiry. Configure destination deletion too: stopping capture does not delete records. Apply the same access and retention rules to exports and investigation copies.
Enforce it before recording
Build records from allowed fields instead of serializing whole chat requests and removing private content afterward. Redact permitted diagnostic content before it reaches a logger, span event, exporter, or persistent queue. If redaction fails, skip the sample and record a controlled failure code.
Use Collector filtering and redaction as an additional safeguard before backend export. They cannot undo raw data reaching the Collector, possibly across a network. Review exception messages and automatic SDK instrumentation. One content-capture setting cannot govern every logging path. Sampling reduces the number of captured conversations but it does not make a retained conversation safe.
Before using real data, test successful and failed calls with a fake secret and email address. Check exported spans, logs, and exception events for those markers. Verify capture expiry and scheduled deletion too.
Use this policy for user or business content, even in local development. Experiments using only synthetic conversations can use simpler rules. Investigations requiring unredacted customer content need a separately authorized evidence store with its own controls, outside this diagnostic path.