Lattice
Skip to content
← Fix a symptom

Band III · Control · “unreliable, unmeasured”

How do I stop PII showing up in LLM traces and prompts?

Short answer

PII in traces is almost always a logging choice, not a model bug. Redact or block before text hits your observability vendor, enforce the same policy at the gateway for every provider, and add output validators for fields that must never leave the system. Then sample production traces in an eval that fails when known patterns appear.

By Kunj Shah · Tool facts verified · method

Check in this order

  1. 01

    Apply one redaction policy on every model call

    Layer 02 · Routing & Gateways

    Multiple SDK paths mean multiple leak paths. The gateway is where provider keys and policy meet.

    • Portkey
      Use when
      A managed gateway with guardrails and analytics already bundled.
      Skip when
      Prompts cannot leave your infrastructure.
    • LiteLLM
      Use when
      The widest provider coverage, deployed the least committal way.
      Skip when
      You want routing policy to live in a SaaS console.
  2. 02

    Detect and strip PII before prompts are stored

    Layer 07 · Guardrails & Safety

    Prompt management and tracing replay what you sent. Scrub at ingress, not when someone exports a CSV months later.

    • Microsoft Presidio
      Use when
      Detecting and anonymising PII in text and images.
      Skip when
      Your data is already pseudonymised at source.
    • Guardrails AI
      Use when
      Validators that check output against a schema you define.
      Skip when
      You need free-form moderation rather than structural checks.
  3. 03

    Add conversational rails for fields users should never paste

    Layer 07 · Guardrails & Safety

    When users paste secrets anyway, flow-level rails can refuse or mask before the model sees them.

    • NeMo Guardrails
      Use when
      Constraining conversational flow with programmable rails.
      Skip when
      You need deep semantic moderation — it is not a classifier.
    • Invariant Guardrails
      Use when
      Guardrails as code, enforced inline in the call path.
      Skip when
      Self-hosting is a hard requirement.
  4. 04

    Audit what your tracing tool actually retains

    Layer 09 · Evaluation & Observability

    Many 'debug' modes store full prompts by default. Turn retention down before you add more guardrails upstream.

    • Langfuse
      Use when
      Self-hostable tracing, prompts and evals in one place.
      Skip when
      You want a fully managed product with a support contract.
    • Arize Phoenix
      Use when
      OpenTelemetry-native evaluation you can run yourself.
      Skip when
      You want a vendor support contract behind it.
  5. 05

    Run a regression set that fails on synthetic PII patterns

    Layer 09 · Evaluation & Observability

    Policy without a test decays the first time someone adds a new tool integration.

    • promptfoo
      Use when
      Declarative red-teaming and regression tests that run in CI.
      Skip when
      You need a managed UI rather than a test runner.
    • DeepEval
      Use when
      pytest-style evaluation you can run next to your unit tests.
      Skip when
      You need a platform rather than a library.

Looks like a fix, is not

  • Asking the model in the system prompt not to log sensitive data.
  • Deleting traces after the fact while leaving full prompts in the pipeline.
  • Buying a new observability tool without changing what gets sent to it.

Quick answers

What should I check first?
Apply one redaction policy on every model call. Multiple SDK paths mean multiple leak paths. The gateway is where provider keys and policy meet.
What looks like a fix but is not?
Asking the model in the system prompt not to log sensitive data. Deleting traces after the fact while leaving full prompts in the pipeline. Buying a new observability tool without changing what gets sent to it.