Audit Logs for LLM Traffic

Compliance auditors evaluating LLM deployments ask three questions: what data was sent to the model, what the model returned, and what safeguards were in place when the request was made. Standard application logging captures some of this, but typically misses the middleware context — which guardrails were active, which model was selected, whether the request was cached or went to the provider. Stockyard's audit module captures the full request lifecycle in a single record: prompt, completion, model, provider, active middleware configuration, and a tamper-evident hash linking each record to the one before it.

The hash chain is what separates audit logging from regular logging. Each audit entry includes a SHA-256 hash of the previous entry's content. Modifying or deleting a historical record breaks the chain, which is immediately detectable through the verification endpoint. This satisfies the immutability requirement that SOC 2, HIPAA, and financial regulators expect from audit trails. The entire chain is verifiable with a single API call — no external blockchain, no third-party attestation service, just cryptographic math applied to a SQLite database on your server.

Why LLM traffic needs audit logs

Regulated industries cannot deploy LLM features without proving what the model said, when it said it, and what safeguards were in place. Healthcare, finance, legal, and government teams need an audit trail that survives legal discovery.

Standard application logging is not enough. LLM audit logs need to capture the full request and response, the model and provider used, which guardrails were active, whether PII was detected and redacted, and a tamper-evident chain of custody.

How Stockyard handles audit

The Brand product (formerly Trust) maintains a hash-chained audit ledger. Every entry includes a SHA-256 hash of the previous entry, making the chain tamper-evident. If any record is modified or deleted, the chain breaks and the tampering is detectable.

Audit entries capture: the full prompt and completion (optionally redacted), the model and provider, active middleware modules, PII detection results, content policy decisions, latency and token counts, and the requesting user or API key.

Self-hosted matters for compliance

Sending audit data to a SaaS observability tool creates a new compliance surface. Now your audit trail lives on someone else's infrastructure, subject to their security posture and data handling policies.

Stockyard keeps everything in embedded SQLite on your server. The audit ledger, the traces, the cost data. Your compliance team can point auditors at a server you control, not a third-party dashboard.