Optimizing Redis Server Performance and Kernel Tuning on Linux

Redis is an essential component in modern cloud architectures, acting as an in-memory database, cache layer, and message broker. However, running Redis with default out-of-the-box settings on high-traffic Linux hosts can lead to latency spikes and memory allocation failures.

1. Tuning Linux Kernel Memory Parameters

When Redis writes snapshot dumps (RDB) to disk, it forks background child processes. If the Linux kernel lacks sufficient virtual memory headroom, forks fail. Fix this by setting memory overcommit rules in /etc/sysctl.conf:

# Open sysctl config
sudo nano /etc/sysctl.conf

# Add the following kernel performance parameters:
vm.overcommit_memory = 1
net.core.somaxconn = 65535

# Apply changes live without rebooting
sudo sysctl -p

2. Disabling Transparent Huge Pages (THP)

Transparent Huge Pages degrade Redis latency and increase memory usage significantly. Disable THP permanently by creating a systemd service at /etc/systemd/system/disable-thp.service:

[Unit]
Description=Disable Transparent Huge Pages (THP) for Redis
DefaultDependencies=no
After=sysinit.target local-fs.target
Before=redis-server.service

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'

[Install]
WantedBy=basic.target

Enable the service:

sudo systemctl daemon-reload
sudo systemctl enable --now disable-thp.service

3. Configuring MaxMemory and Eviction Policies

Edit /etc/redis/redis.conf to prevent Redis from consuming 100% of host RAM:

# Cap Redis memory usage to 75% of available server RAM
maxmemory 4gb

# Automatically evict least recently used (LRU) keys when maxmemory is hit
maxmemory-policy allkeys-lru

Comments

Production Log Management with

Production Log Management on Linux: Mastering logrotate for High-Uptime Servers Unmanaged log growth is one of the most common causes of unexpected production outages. When application or web server logs fill up host disk partitions, database transactions fail, services crash, and systems become un-writeable. Linux provides a built-in utility called logrotate to automatically compress, rotate, and archive old log files. 1. Understanding logrotate Structure System logrotate configurations are split between two main locations: /etc/logrotate.conf — Global baseline configurations. /etc/logrotate.d/ — Service-specific configuration files (e.g., Nginx, Docker, custom apps). 2. Creating a Custom Application Log Rule Create a rule file for a custom production web service at /etc/logrotate.d/jam-app : /var/log/jam-app/*.log { daily missingok rotate 14 compress delaycompress notifempty create 0640 www-data www-data sharedscripts ...

Building High-Performance AI Infrastructure on Linux: From NVIDIA Drivers to vLLM Production Deployments

The modern Machine Learning landscape relies entirely on Linux infrastructure. While high-level framework developments like PyTorch, Hugging Face, and LangChain dominate developer discussions, the actual engine serving low-latency inference at scale sits directly on Linux kernels, GPU acceleration drivers, and containerized serving engines. In this guide, we will step through the core architecture required to turn a raw Linux server (Ubuntu/Debian or RHEL/Rocky) into a high-throughput, OpenAI-compatible AI inference endpoint using vLLM and Docker . 1. Prerequisites & Linux Kernel Preparation Before running any Large Language Model (LLM) inference engine, your host operating system must be properly configured with NVIDIA proprietary drivers and the NVIDIA Container Toolkit to allow Docker containers direct access to host GPU acceleration. Step 1.1: Verify GPU Hardware Ensure your host system correctly detects all installed NVIDIA GPUs: Bash # Check PCI bus for recognized NVI...

Automated PostgreSQL Backups to S3 via Bash & Cron

Automated PostgreSQL Backups: A Production Bash & Cron Blueprint Database backup failures are catastrophic when disaster strikes. Relying on manual database exports leads to data loss and unverified recovery points. Setting up an automated pipeline that dumps PostgreSQL tables, compresses the archive using gzip , and syncs the output to remote object storage guarantees data durability. In this guide, we will build an automated shell script for PostgreSQL backups and schedule it via system cron jobs. 1. Prerequisites & AWS CLI Setup First, ensure the AWS CLI tool and PostgreSQL client utilities are installed on your Linux host environment: sudo apt update && sudo apt install -y postgresql-client awscli gzip Configure credentials for an IAM user with Write permissions to your S3 bucket: aws configure 2. Writing the Production Backup Script Create a backup script at /usr/local/bin/pg_backup.sh : sudo nano /usr/local/bin/pg_backup.sh Paste the foll...

InfraScale AI

Search This Blog

Optimizing Redis Infrastructure