Securing Local LLM Endpoints: Reverse Proxying vLLM with Nginx and Let's Encrypt SSL
When serving Large Language Models using frameworks like vLLM or Ollama, the default HTTP server exposes unencrypted API endpoints on port 8000 or 11434. Exposing raw model endpoints directly to the open internet without TLS encryption or authentication creates severe security risks.
In this guide, we will configure an Nginx Reverse Proxy with Certbot SSL to secure our vLLM endpoint behind HTTPS with rate-limiting and access protection.
1. Installing Nginx and Certbot on Ubuntu
Start by updating system packages and installing Nginx along with Certbot for SSL management:
sudo apt update && sudo apt install -y nginx certbot python3-certbot-nginx
2. Configuring the Nginx Reverse Proxy Block
Create a dedicated Nginx configuration file for your AI endpoint at /etc/nginx/sites-available/vllm.conf:
# Rate limiting zone for API requests
limit_req_zone $binary_remote_addr zone=llm_limit:10m rate=10r/s;
server {
server_name api.yourdomain.com;
location / {
limit_req zone=llm_limit burst=20 nodelay;
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Streaming support for Server-Sent Events (SSE) token generation
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 600s;
}
}
3. Enabling the Site & Obtaining SSL Certificates
Enable the site configuration, test syntax, and run Certbot to request an automated Let's Encrypt TLS certificate:
# Symlink to sites-enabled
sudo ln -s /etc/nginx/sites-available/vllm.conf /etc/nginx/sites-enabled/
# Test Nginx configuration
sudo nginx -t
# Reload Nginx service
sudo systemctl reload nginx
# Request HTTPS Certificate
sudo certbot --nginx -d api.yourdomain.com
Summary
Your vLLM endpoint is now protected behind HTTPS TLS 1.3 encryption, buffered against DDoS attacks via Nginx rate-limiting, and configured for zero-buffering response streaming.
Comments