Haystack LLM Orchestration in a TEE

What is this?

We deployed Haystack inside a Trusted Execution Environment (TEE). The full LLM orchestration pipeline runs within hardware-protected confidential VMs so that your prompts and data stay private even from the cloud provider.

TEEs provide a tamper-proof secure boundary where your LLM pipelines run. Everything operates within hardware-protected memory that guarantees the following:

• End-to-end privacy: Your data is encrypted all the time, even during compute when it's being processed by the agent. No one, including the host can see your data.
• Tamper-proof execution: TEEs prevent unauthorized access or modification of running code and data without detection.
• Verifiable Workloads: TEEs provide a proof of what code is running inside them, and can attest to the exact version of Haystack that is being run

Try it now (1 deployed)

Run Haystack pipelines directly from your browser:

curl -X POST https://haystack-demo.lunal.dev/simple_chat/run \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What are TEEs?"
  }'

What is Lunal?

Lunal is the trusted compute company that makes TEEs simple, usable, and scalable. We provide unified software and infrastructure for deploying AI workloads in TEEs with zero configuration.

Learn more about Lunal and why secure AI needs TEEs.