Turn AI tool-call logs into reviewable evidence

Define a canonical tool-call event

Use a versioned schema rather than free-form application logs. The event should carry execution and attempt identifiers, actor and tenant, tool name and version, normalized operation, input digest, authorization decision, approval reference, timestamps, outcome, result digest, and external resource identifiers.

Emit a requested or authorized event before execution and a completed, denied, failed, or unknown event afterward. An unknown outcome is important when a timeout occurs after the remote system may already have accepted the action.

Connect the record to the external effect

The model response is not proof that a tool changed anything. Capture the transaction, message, job, document, or object identifier returned by the target system and record the service account or delegated identity used. Where possible, retain an idempotency key and a target-system timestamp.

For read tools, log the source and retrieval scope without copying entire confidential documents. For write tools, include before-and-after references or a change digest so a reviewer can determine what the action affected.

Add integrity without claiming that hashes solve authorization

Canonicalize the event fields before hashing. Sequence links or Merkle batches can make later modification and omission detectable, especially when roots or signed manifests are stored separately. A verifier should reject changed fields, missing links, an unexpected schema version, or an untrusted signature.

Integrity does not prove that the call was appropriate. Authorization rules, approval policy, least privilege, and target-system controls remain necessary and should be referenced by identifier in the evidence.

Test the failure paths an auditor will ask about

Exercise denials, expired credentials, malformed input, downstream timeout, duplicate retry, partial completion, delayed event delivery, queue replay, export failure, and retention expiry. Reconcile the agent event count with target-system activity so silent evidence loss cannot appear as a successful run.

• Verify a normal event and a deliberately modified event.
• Confirm duplicate delivery does not create a second accepted evidence record.
• Prove tenant filters prevent cross-customer retrieval and export.
• Record reviewer identity and the exact evidence version used for a decision.