OpenPCC: Open and Confidential LLM Serving on Commodity TEEs
2026-06-09 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors explain that big AI models, like those used in chatbots and image generators, need powerful cloud computers to work, which means users send their private info to these cloud services. Because this info can be sensitive, keeping it safe is really important. The authors looked at existing solutions but found they rely on special hardware that isn't widely available and have other issues. They created OpenPCC, a secure cloud system using common Trusted Execution Environments (TEEs) instead of special hardware, and tested it with an AI model to show it works well and keeps data safe.
Large Language ModelsCloud Inference ServicePrivacy ProtectionTrusted Execution EnvironmentsOpenPCCConfidential ComputingLlama-3Proprietary HardwareSecure Cloud ComputingvLLM
Authors
Haoling Zhou, Shixuan Zhao, Chao Wang, Zhiqiang Lin
Abstract
Generative AI applications such as personal AI agents, image generators, and chat assistants offer advanced capabilities to improve user experience. Behind the scenes, Large Language Models (LLMs) that power these services require a massive amount of computation and are usually deployed in the cloud, available as APIs, meaning that a user's request has to be sent to a Cloud Inference Service (CIS) for processing. However, the strong capabilities of LLM also mean that user's requests now contain much more personal sensitive or enterprise confidential information, demanding equally strong protection in CIS. While early industry efforts such as Apple Private Cloud Compute (PCC) and Google Private AI Compute have emerged to show the potential of secure CIS, they are not adoptable for deployment by others due to their reliance on proprietary hardware and closed ecosystem. In addition, they all suffer from their own design glitches that can undermine the ambitious goal of bringing in true privacy protection to end users. In this paper, we present our analysis of the fundamental requirements of building a secure yet open CIS. We then present OpenPCC, a Confidential CIS framework that does not rely on proprietary hardware but instead uses commercially available TEEs. We implement an open-source prototype and characterize it end-to-end on a Llama-3 8B vLLM workload, separating OpenPCC's own cost from the underlying TEE hardware. Our analysis and evaluation demonstrated the feasibility and security of the system.