TechSetupGuides
Beginnerrediscachedatabasenosqlin-memorycachingsession-storemessage-queueaivectors

Redis - In-Memory Data Store

Install and configure Redis, the fastest in-memory data store for caching, real-time analytics, session management, and AI workloads — covering installation, configuration, data types, and common use cases.

  1. Step 1

    Overview

    Redis is an open-source in-memory data store that serves as a cache, database, message broker, and streaming engine. It's one of the most popular real-time data platforms, powering everything from simple caching to complex AI applications.

    Key capabilities:

    • Caching: Sub-millisecond latency for frequently accessed data
    • Session Store: Distributed session management for web applications
    • Data Structure Server: Strings, lists, sets, hashes, sorted sets, JSON, and more
    • NoSQL Database: Key-value, document, and time series storage
    • Search & Query: Full-text search, vector search, geospatial queries
    • Message Broker: Pub/sub, streams, and queue operations
    • AI/ML: Vector storage for RAG, semantic caching, and LLM memory

    Why Redis:

    • Speed: In-memory storage with sub-millisecond latency
    • Flexibility: Rich data types and operations
    • Simplicity: Easy-to-use command set and protocols
    • Reliability: Battle-tested at massive scale
    • Extensibility: Modules API for custom functionality
    Official site: https://redis.io
    GitHub: https://github.com/redis/redis (74K+ stars)
    Documentation: https://redis.io/docs
  2. Step 2

    Quick Installation Options

    Multiple installation methods available depending on your environment:

    # Option 1: Docker (recommended for development)
    docker run -d -p 6379:6379 redis:latest
    
    # Option 2: Homebrew (macOS/Linux)
    brew install redis
    brew services start redis
    
    # Option 3: APT (Debian/Ubuntu)
    sudo apt update
    sudo apt install redis-server
    sudo systemctl start redis-server
    
    # Option 4: YUM (RHEL/CentOS/Fedora)
    sudo yum install redis
    sudo systemctl start redis
    
    # Option 5: Snap
    sudo snap install redis --classic
    
    # Verify installation
    redis-cli ping
    # Should return: PONG
  3. Step 3

    Build from Source (Ubuntu 22.04/24.04)

    For the latest features or custom builds, compile Redis from source. This gives you full control over build options and enables all data structures including modules.

    # Install dependencies
    sudo apt update
    sudo apt install -y gcc make wget curl
    
    # Clone Redis repository
    git clone --depth 1 https://github.com/redis/redis.git
    cd redis
    
    # Build with all features (including modules)
    make BUILD_WITH_MODULES=yes
    
    # Run tests (optional but recommended)
    make test
    
    # Install system-wide (with modules)
    sudo make install PREFIX=/usr/local
    
    # Copy config file
    sudo cp redis.conf /etc/redis/
    sudo mkdir -p /var/lib/redis
    sudo chown -R $USER:$USER /etc/redis /var/lib/redis
    
    # Start Redis
    redis-server /etc/redis/redis.conf
    
    # Verify
    redis-cli ping
    # Output: PONG
    
    # Check version
    redis-cli INFO server | grep redis_version
  4. Step 4

    Configuration

    Redis uses a simple configuration file. Here are the most important settings to customize:

    Key configuration options:

    • port: TCP port (default: 6379)
    • bind: IP addresses to listen on
    • requirepass: Password for AUTH command
    • maxmemory: Memory limit
    • maxmemory-policy: Eviction policy when memory is full
    • save: RDB snapshot settings
    • appendonly: AOF persistence enabled/disabled
    • dir: Working directory for data files
    # Basic secure configuration for production
    port 6379
    bind 127.0.0.1
    protected-mode yes
    requirepass your-secure-password-here
    
    # Memory management
    maxmemory 2gb
    maxmemory-policy allkeys-lru
    
    # Persistence
    save 900 1
    save 300 10
    save 60 10000
    appendonly yes
    appendfsync everysec
    
    # Performance
    tcp-backlog 511
    timeout 0
    tcp-keepalive 300
    
    # Data directory
    dir ./
    
    # Database count
    databases 16
  5. Step 5

    Basic Commands

    Redis provides a rich command set. Start with redis-cli to interact with your Redis instance:

    String operations (most basic type):

    • SET key value - Store a value
    • GET key - Retrieve a value
    • INCR key - Increment a numeric value by 1
    • INCRBY key increment - Increment by specific amount
    • DEL key - Delete a key
    • EXPIRE key seconds - Set key expiration

    Hash operations (field-value pairs):

    • HSET key field value - Set hash field
    • HGET key field - Get hash field
    • HGETALL key - Get all fields
    • HDEL key field - Delete field

    List operations (ordered lists):

    • LPUSH key value - Push to left
    • RPOP key - Pop from right
    • LRANGE key start stop - Get range

    Set operations (unique unordered values):

    • SADD key member - Add member
    • SMEMBERS key - Get all members
    • SISMEMBER key member - Check membership
    # Start Redis CLI
    redis-cli
    
    # Basic string operations
    redis> SET mykey "Hello World"
    OK
    redis> GET mykey
    "Hello World"
    redis> EXPIRE mykey 60
    (integer) 1
    
    # Counter (great for tracking)
    redis> SET page_views 0
    OK
    redis> INCR page_views
    (integer) 1
    redis> INCRBY page_views 100
    (integer) 101
    
    # Hash (user data)
    redis> HSET user:1000 name "Alice"
    (integer) 1
    redis> HSET user:1000 email "alice@example.com"
    (integer) 1
    redis> HGETALL user:1000
    1) "name"
    2) "Alice"
    3) "email"
    4) "alice@example.com"
    
    # List (queue)
    redis> LPUSH tasks "task1"
    (integer) 1
    redis> LPUSH tasks "task2"
    (integer) 2
    redis> RPOP tasks
    "task1"
    
    # Set (unique items)
    redis> SADD tags:post:1 "redis" "caching" "database"
    (integer) 3
    redis> SMEMBERS tags:post:1
    1) "redis"
    2) "caching"
    3) "database"
    redis> SISMEMBER tags:post:1 "redis"
    (integer) 1
    
    # Sorted Set (leaderboard)
    redis> ZADD leaderboard 100 "Alice"
    (integer) 1
    redis> ZADD leaderboard 150 "Bob"
    (integer) 1
    redis> ZADD leaderboard 120 "Charlie"
    (integer) 1
    redis> ZRANGE leaderboard 0 -1 WITHSCORES
    1) "Alice"
    2) "100"
    3) "Charlie"
    4) "120"
    5) "Bob"
    6) "150"
    
    # Exit
    redis> EXIT
  6. Step 6

    JSON Data Type

    Redis JSON enables storing and querying JSON documents directly without serialization overhead. Use JSONPath expressions for nested access and modifications.

    Note: JSON support requires Redis to be built with modules (BUILD_WITH_MODULES=yes).

    # Enable JSON module if needed (after building with modules)
    # Then use JSON commands:
    
    # Store JSON document
    redis> JSON.SET user:1000 $ '{"name":"Alice","age":30,"skills":["redis","nodejs"]}'
    OK
    
    # Get entire document
    redis> JSON.GET user:1000
    {"name":"Alice","age":30,"skills":["redis","nodejs"]}
    
    # Get specific field
    redis> JSON.GET user:1000 $.name
    ["Alice"]
    
    # Update nested value
    redis> JSON.SET user:1000 $.age 31
    OK
    
    # Add array element
    redis> JSON.ARRAPPEND user:1000 $.skills "python"
    (integer) 4
    
    # Query with aggregation
    redis> JSON.GET user:1000 '.*'
  7. Step 7

    Stream Data Type

    Redis Streams are append-only logs with consumer groups — perfect for message brokers, event sourcing, and real-time data pipelines.

    # Create stream with messages
    redis> XADD mystream * field1 "value1" field2 "value2"
    "1635791234567-0"
    redis> XADD mystream * field1 "value3" field2 "value4"
    "1635791234568-0"
    
    # Read stream
    redis> XREAD COUNT 2 BLOCK 0 STREAMS mystream 0
    1) 1) "mystream"
    2)  1)  1) "1635791234567-0"
         2)  1) "field1"
              2) "value1"
              3) "field2"
              4) "value2"
         2)  1) "1635791234568-0"
              2)  1) "field1"
                  2) "value3"
                  3) "field2"
                  4) "value4"
    
    # Create consumer group
    redis> XGROUP CREATE mystream mygroup $ MKSTREAM
    OK
    
    # Read from consumer group
    redis> XREADGROUP GROUP mygroup consumer1 COUNT 1 STREAMS mystream >
    1) 1) "mystream"
    2)  1)  1) "1635791234567-0"
         2)  1) "field1"
              2) "value1"
              3) "field2"
              4) "value2"
    
    # Acknowledge message
    redis> XACK mystream mygroup "1635791234567-0"
    (integer) 1
  8. Step 8

    Pub/Sub Messaging

    Redis Pub/Sub enables lightweight messaging between clients. Publishers send messages to channels, subscribers receive messages from those channels.

    # Terminal 1: Subscribe to channel
    redis-cli
    > SUBSCRIBE news
    Reading messages on 'news'
    
    # Terminal 2: Publish message
    redis-cli
    > PUBLISH news "Breaking news: Redis is awesome!"
    (integer) 1
    
    # Terminal 1 receives:
    # 1) "subscribe"
    # 2) "news"
    # 3) "Breaking news: Redis is awesome!"
    
    # Pattern subscriptions (multiple channels)
    redis-cli
    > PSUBSCRIBE news:* sports:*
    Reading messages on 'news:*' and 'sports:*'
    
    redis-cli
    > PUBLISH news:local "Local news update"
    (integer) 1
    > PUBLISH news:world "World news update"
    (integer) 1
  9. Step 9

    Transactions with WATCH/MULTI

    Redis transactions ensure atomic execution of multiple commands. WATCH enables optimistic locking for concurrent access control.

    # Simple transaction
    redis> MULTI
    OK
    redis> INCR counter:1
    QUEUED
    redis> INCR counter:2
    QUEUED
    redis> SET mykey "transactional value"
    QUEUED
    redis> EXEC
    1) (integer) 1
    2) (integer) 1
    3) "OK"
    
    # Watch for changes (optimistic locking)
    redis> WATCH mykey
    OK
    redis> MULTI
    OK
    redis> INCR mykey
    QUEUED
    redis> EXEC
    1) (integer) 2
    
    # If another client modified mykey between WATCH and EXEC:
    # EXEC returns nil (transaction aborted)
    
    # Discard transaction
    redis> MULTI
    OK
    redis> SET key "value"
    QUEUED
    redis> DISCARD
    OK
    # Nothing was executed
  10. Step 10

    Lua Scripts

    Execute Lua scripts atomically on the Redis server. Perfect for complex operations that need to be atomic or for reducing network round-trips.

    # Simple Lua script (increment with condition)
    redis> EVAL "if tonumber(ARGV[1]) > 0 then return redis.call('INCR', KEYS[1]) else return redis.call('DECR', KEYS[1]) end" 1 mycounter 5
    (integer) 11
    
    # Complex: atomic check-and-set
    redis> EVAL "if not redis.call('EXISTS', KEYS[1]) then redis.call('SET', KEYS[1], ARGV[1], 'EX', ARGV[2]); return 1; else return 0; end" 1 cache:key "value" 3600
    (integer) 1
    
    # Store and reuse scripts (SHA1)
    redis> SCRIPT LOAD "return redis.call('GET', KEYS[1])"
    "9a0568f33435c3a0e6c5e7c6e8e9e1e2e3e4e5e6"
    redis> EVALSHA "9a0568f33435c3a0e6c5e7c6e8e9e1e2e3e4e5e6" 1 mykey
  11. Step 11

    Client Libraries

    Redis provides official and community client libraries for most programming languages. Here are examples in popular languages:

    Python (redis-py):

    import redis
    r = redis.Redis(host='localhost', port=6379, db=0)
    r.set('key', 'value')
    print(r.get('key'))
    

    Node.js (node-redis):

    const redis = require('redis');
    const client = redis.createClient();
    await client.set('key', 'value');
    console.log(await client.get('key'));
    

    Go (go-redis):

    import "github.com/go-redis/redis/v8"
    rdb := redis.NewClient(&redis.Options{Addr: ":6379"})
    rdb.Set(ctx, "key", "value", 0)
    val, _ := rdb.Get(ctx, "key").Result()
    

    Java (Jedis):

    Jedis jedis = new Jedis("localhost", 6379);
    jedis.set("key", "value");
    String value = jedis.get("key");
    
    Python: pip install redis
    Node.js: npm install redis
    Go: go get github.com/go-redis/redis/v8
    Java: Maven - org.redisson:redisson
    C#: dotnet add package StackExchange.Redis
    PHP: composer require predis/predis
    Ruby: gem install redis
    
    Full list: https://redis.io/docs/latest/develop/clients/
  12. Step 12

    Common Use Cases

    1. Session Store: Store user sessions with automatic expiration:

    # Python Flask example
    from flask import Flask, session
    app = Flask(__name__)
    app.secret_key = 'your-secret'
    app.config.update(
        SESSION_COOKIE_SECURE=True,
        SESSION_COOKIE_HTTPONLY=True,
    )
    # Configure Redis for sessions
    from flask_session import Session
    app.config['SESSION_TYPE'] = 'redis'
    app.config['SESSION_REDIS'] = redis.from_url('redis://localhost:6379')
    

    2. Leaderboard (Gaming/Social):

    # Add score
    zadd('leaderboard', {player_id: score})
    
    # Get top 10
    zrevrange('leaderboard', 0, 9, withscores=True)
    
    # Get player rank
    zrevrank('leaderboard', player_id)
    

    3. Caching Database Queries:

    def get_user(user_id):
        cache_key = f'user:{user_id}'
        cached = redis.get(cache_key)
        if cached:
            return json.loads(cached)
        # Query database
        user = db.query('SELECT * FROM users WHERE id = ?', user_id)
        redis.setex(cache_key, 3600, json.dumps(user))
        return user
    

    4. Rate Limiting:

    def rate_limit(key, max_requests, window_seconds):
        count = redis.incr(f'ratelimit:{key}')
        if count == 1:
            redis.expire(f'ratelimit:{key}', window_seconds)
        return count <= max_requests
    

    5. Job Queue:

    # Producer
    redis.lpush('jobs', json.dumps(job_data))
    
    # Consumer
    while True:
        job = redis.rpop('job', timeout=1)
        if job:
            process_job(job)
    

    6. Real-time Analytics:

    # Count page views
    redis.incr('views:page:home')
    
    # Unique visitors (HyperLogLog)
    redis.pfadd('unique:visitors', user_id)
    redis.pfcount('unique:visitors')
    
    # Percentile calculations (T-Digest module)
    TDIGEST.ADD latency:api 123.45
    TDIGEST.QUANTILE latency:api 0.5 0.9 0.95
    

    7. Distributed Lock:

    import redis.lock
    with redis.lock.Lock(
        client=redis,
        name='critical-resource',
        timeout=10,
        blocking=True,
        blocking_timeout=5
    ) as lock:
        # Critical section
        pass
    

    8. AI/Vector Search:

    # Store vector embeddings
    ft.create_index(
        ['vector'],
        schema=[Field('vector', 'VECTOR', 'HNSW', 'FLOAT32')]
    )
    
    # Semantic similarity search
    ft.search('(@vector:[KNN 10 @vector $bvec])',
              params={'bvec': binary_vector},
              return_fields=['__vector_score'])
    
    # Installation for examples
    pip install redis flask redis-py
    
    # For vector search (Redis Stack)
    pip install redis-stack
    
    # For AI workloads
    pip install redisvl
  13. Step 13

    Persistence Options

    Redis offers two main persistence mechanisms:

    RDB (Redis Database Backup):

    • Snapshot-based persistence
    • Faster restart times
    • Single file, compact
    • Risk of data loss between snapshots

    AOF (Append Only File):

    • Logs every write operation
    • More durable
    • Larger file size
    • Slower restart

    Best practice: Enable both for safety + performance.

    # Redis config (redis.conf)
    
    # RDB snapshots
    save 900 1      # Save after 900 sec if 1 key changed
    save 300 10     # Save after 300 sec if 10 keys changed
    save 60 10000   # Save after 60 sec if 10000 keys changed
    dbfilename dump.rdb
    dir ./
    
    # AOF persistence
    appendonly yes
    appendfilename "appendonly.aof"
    appendfsync everysec   # fsync every second (balance)
    # appendfsync always     # fsync on every write (safest)
    # appendfsync no         # let OS handle fsync (fastest)
    
    # AOF rewrite (compact AOF)
    aof-use-rdb-preamble yes
    auto-aof-rewrite-percentage 100
    auto-aof-rewrite-min-size 64mb
    
    # Load priority on restart
    stop-writes-on-bgsave-error no
    rdb-save-background-saves-enabled yes
  14. Step 14

    Security Hardening

    Redis is often left unprotected by default. Here's how to secure it:

    Essential security measures:

    1. Always use a password (requirepass)
    2. Bind to localhost or specific IPs
    3. Enable protected-mode
    4. Rename or disable dangerous commands
    5. Use TLS for encryption
    6. Keep Redis updated
    # Security hardening (redis.conf)
    
    # Network
    bind 127.0.0.1
    protected-mode yes
    port 6379
    
    # Authentication
    requirepass "strong-random-password-min-32-chars"
    
    # Rename dangerous commands
    rename-key FLUSHALL ""
    rename-key FLUSHDB ""
    rename-key CONFIG ""
    rename-key DEBUG ""
    rename-key SHUTDOWN ""
    
    # TLS (optional but recommended)
    tls-port 6379
    tls-cert-file /etc/redis/redis.crt
    tls-key-file /etc/redis/redis.key
    tls-ca-cert-file /etc/redis/ca.crt
    
    # Resource limits
    maxmemory 2gb
    maxmemory-policy allkeys-lru
    maxclients 10000
    
    # Disable dangerous modules
    loadmodule /path/to/module.so
  15. Step 15

    Monitoring & Debugging

    Monitor Redis performance and debug issues with built-in commands and external tools:

    Built-in commands:

    • INFO - Server information and stats
    • MONITOR - Live command monitoring (use carefully)
    • SLOWLOG - Slow query log
    • DBSIZE - Key count

    External tools:

    • Redis Insight (GUI)
    • Redis Commander
    • Prometheus + Grafana
    # Check server info
    redis-cli INFO
    
    # Memory usage
    redis-cli INFO memory | grep used_memory_human
    # Output: used_memory_human:256.19Mb
    
    # Key count
    redis-cli DBSIZE
    # Output: 1523
    
    # Slow queries
    redis-cli SLOWLOG GET 10
    
    # Memory per key (Redis 4.0+)
    redis-cli --bigkeys
    
    # Search for keys pattern
    redis-cli --scan --pattern "user:*" | head -20
    
    # Delete expired keys manually (if needed)
    redis-cli KEYS "expired:*" | xargs redis-cli DEL
    
    # Monitor connections
    redis-cli INFO clients
    
    # Check persistence status
    redis-cli INFO persistence
    
    # Top memory consumers (requires redis-memory-command)
    redis-cli MEMORY STATS
  16. Step 16

    High Availability & Replication

    Redis supports master-replica replication for high availability and read scaling:

    Replication setup:

    • One master (read/write)
    • Multiple replicas (read-only, failover candidates)
    • Automatic failover with Redis Sentinel
    • Native clustering with Redis Cluster
    # Setup replica (redis.conf)
    replicaof master-IP master-port
    masterauth "master-password"
    
    # Start replica
    redis-server /etc/redis/replica.conf
    
    # Verify replication
    redis-cli INFO replication
    # Output:
    # role:master
    # connected_slaves:2
    # slave0:ip=10.0.0.2,port=6379,state=online
    # slave1:ip=10.0.0.3,port=6379,state=online
    
    # Check replica lag
    redis-cli -h replica-IP latency histogram
    
    # Test failover (manual)
    redis-cli -h master-IP SLAVEOF NO ONE
    redis-cli -h replica-IP INFO replication
    
    # Redis Cluster (3 masters + 3 replicas)
    redis-server --cluster-enabled yes --cluster-config-file nodes.conf
    redis-cli --cluster create \n    127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \n    127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \n    --cluster-replicas 1
  17. Step 17

    Cloud Alternatives

    For production deployments without self-management overhead, consider managed Redis services:

    Cloud providers:

    • AWS ElastiCache - Fully managed Redis
    • Google Cloud Memorystore - Redis on GCP
    • Azure Cache for Redis - Enterprise features
    • Redis Cloud - From Redis Ltd.

    Benefits:

    • Automatic backups
    • Built-in monitoring
    • Easy scaling
    • Managed failover
    • Security patches
    AWS ElastiCache: https://aws.amazon.com/elasticache/
    Google Memorystore: https://cloud.google.com/memorystore
    Azure Cache: https://azure.microsoft.com/en-us/products/cache-for-redis/
    Redis Cloud: https://redis.io/cloud/
  18. Step 18

    Resources & Next Steps

    Documentation:

    Community:

    Tools:

    Next guides:

    • Redis Sentinel for high availability
    • Redis Cluster for horizontal scaling
    • Redis Stack with Search, Vector, and TimeSeries modules
    • Production hardening and monitoring
    GitHub: https://github.com/redis/redis
    Official site: https://redis.io
    Documentation: https://redis.io/docs
    Redis Stack (with modules): https://redis.io/docs/latest/develop/oss/redis-stack/

Feature requests

Sign in to suggest features or vote on existing ones.

No feature requests yet.

Discussion

0 people marked this as worked·Sign in to mark your own.

Sign in to join the discussion.

No comments yet.