1. Introduction

django-redis is a BSD Licensed, full featured Redis cache/session backend for Django.

1.1. Why use django-redis?

Because:

  • In active development.

  • Modular client system (pluggable clients).

  • Master-Slave support in the default client.

  • Complete battery of tests.

  • Used in production in several projects as cache and session storage.

  • Supports infinite timeouts.

  • Python 3 support in same code base.

  • Facilities for raw access to Redis client/connection pool.

  • Highly configurable (can emulate memcached exception behavior, for example).

  • Unix sockets supported by default.

  • Pluggable parsers.

1.2. Supported versions

  • Latest stable version: 3.6.2

  • Previous stable version: 3.5.2

1.3. Requirements

  • Python 2.7, 3.3 or 3.4

  • Django 1.4, 1.5, 1.6 and 1.7

  • redis-py >= 2.9.0

2. User guide

2.1. Installation

The simplest way to use django-redis in your project is to install it with pip:

pip install django-redis

2.2. Configure as cache backend

To start using django-redis, you should change your Django cache settings to something like:

CACHES = {
    'default': {
        'BACKEND': 'redis_cache.cache.RedisCache',
        'LOCATION': '127.0.0.1:6379:1',
        'OPTIONS': {
            'CLIENT_CLASS': 'redis_cache.client.DefaultClient',
            'PASSWORD': 'secretpassword',  # Optional
        }
    }
}

In this example, we are using a standard TCP/IP socket connection to Redis running at localhost, using database 1. In OPTIONS, we have specified the pluggable client we want to use; in this case the default one. You can omit this part since that client is used by default if one is not specified.

Note
If you want use Unix sockets, change the LOCATION value to something like "unix:/path/to/socket:1".

2.3. Configure as session backend

Django can by default use any cache backend as session backend and you benefit from that by using django-redis as backend for session storage without installing any additional and unnecesary backends:

SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
SESSION_CACHE_ALIAS = 'default'

3. Advanced usage

3.1. Pickle version

For almost all values, django-redis uses pickle to serialize objects and by default uses the latest pickle protocol version. If you want set a custom version you can do it with the PICKLE_VERSION key in OPTIONS:

'OPTIONS': {
    'PICKLE_VERSION': -1  # Use the latest protocol version
}

3.2. Socket timeout

You can set socket timeout using the SOCKET_TIMEOUT key in the OPTIONS dict:

'OPTIONS': {
    'SOCKET_TIMEOUT': 5,  # in seconds
}

3.3. Memcached exceptions behavior

In some situations, when Redis is only used for cache, you do not want exceptions when Redis is down. This is default behavior in the memcached backend and it can be emulated in django-redis.

You can ignore exceptions globally (for all configured Redis cache clients) by setting DJANGO_REDIS_IGNORE_EXCEPTIONS to True in your settings.

Additionally, you can set it individually for each client in their OPTIONS dict:

'OPTIONS': {
    'IGNORE_EXCEPTIONS': True,
}

3.4. Infinite timeout

django-redis comes with infinite timeouts support, since before Django added it in version 1.6.

If used with Django before version 1.6, django-redis interprets a 0 timeout value as infinite. In Django >= 1.6 it also supports None as timeout value.

These statements are then equivalent:
cache.set('key', 'value', timeout=0)
cache.set('key', 'value', timeout=None)

3.5. Get ttl (time-to-live) from key

With redis, you can access to ttl of any stored key, for it, django-redis exposes ttl function.

It returns:

  • ttl value for any volatile key (any key that has expiration)

  • 0 for expired and not existent keys

  • None for keys that does not have expiration

Simple search keys by pattern
>>> from django.core.cache import cache
>>> cache.set('foo', 'value', timeout=25)
>>> cache.ttl('foo')
25
>>> cache.ttl('not-existent')
0

3.6. Scan & Delete keys in bulk

django-redis comes with some additions methods that hels for search or delete keys using glob patterns.

Simple search keys by pattern
>>> from django.core.cache import cache
>>> cache.keys('foo_*')
['foo_1', 'foo_2']

On simple, search it return all matched values, in databases with big amount of keys this isn’t suitable method. For it, you can use iter_keys function that works like keys function but uses redis>=2.8 server side cursors.

Search using server side cursors
>>> from django.core.cache import cache
>>> cache.iter_keys('foo_*')
<generator object algo at 0x7ffa9c2713a8>
>>> next(cache.iter_keys('foo_*'))
"foo_1"

For delete keys, you should use delete_pattern that has the same glob pattern syntax like keys function and returns a number of deleted keys.

Example use of delete_pattern
>>> from django.core.cache import cache
>>> cache.delete_pattern('foo_*')

3.7. Redis native commands

django-redis has limited support for some Redis atomic operations, such as the commands SETNX and INCR.

You can use the SETNX command through the backend `set() method with the nx parameter:

Example:
>>> from django.core.cache import cache
>>> cache.set('key', 'value1', nx=True)
True
>>> cache.set('key', 'value2', nx=True)
False
>>> cache.get('key')
'value1'

Also, incr and decr methods uses redis atomic operations when value that contains a key is suitable for it.

3.8. Raw client access

In some situations your application requires access to a raw Redis client to use some advanced features that aren’t exposed by the Django cache interface. To avoid storing another setting for creating a raw connection, django-redis exposes functions with which you can obtain a raw client reusing the cache connection string: get_redis_connection(alias).

>>> from redis_cache import get_redis_connection
>>> con = get_redis_connection('default')
>>> con
<redis.client.Redis object at 0x2dc4510>
Warning
Not all pluggable clients support this feature.

3.9. Connection pools

Behind the scenes, django-redis uses the underlying redis-py connection pool implementation, and exposes a simple way to configure it or directly customize a connection/connection pool creation for backend.

The default redis-py behavior is to not close connections, recycling them when possible.

3.9.1. Configure default connection pool

The default connection pool is simple. You can only customize the maximum number of connections in the pool, by setting CONNECTION_POOL_KWARGS in the CACHES setting:

CACHES = {
    'default': {
        'BACKEND': 'redis_cache.cache.RedisCache',
        ...
        'OPTIONS': {
            'CONNECTION_POOL_KWARGS': {'max_connections': 100}
        }
    }
}

You can verify how many connections the pool has opened with the following snippet:

from django.core.cache import get_cache
redis_cache = get_cache('default')  # Use the name you have defined for Redis in settings.CACHES
redis = redis_cache.raw_client
connection_pool = redis.connection_pool
print('Created connections so far: %d' % connection_pool._created_connections)

3.9.2. Use your own connection pool subclass

Sometimes you want to use your own subclass of the connection pool. This is possible with django-redis using the CONNECTION_POOL_CLASS parameter in the backend options.

myproj/mypool.py
from redis.connection import ConnectionPool

class MyOwnPool(ConnectionPool):
    # Just doing nothing, only for example purpose
    pass
settings.py
# Omitting all backend declaration boilerplate code.

'OPTIONS': {
    'CONNECTION_POOL_CLASS': 'myproj.mypool.MyOwnPool',
}

3.9.3. Customize connection factory

If none of the previous methods satisfies you, you can get in the middle of the django-redis connection factory process and customize or completely rewrite it.

By default, django-redis creates connections through the redis_cache.pool.ConnectionFactory class that is specified in the global Django setting DJANGO_REDIS_CONNECTION_FACTORY.

Partial interface of ConnectionFactory class
# Note: Using Python 3 notation for code documentation ;)

class ConnectionFactory(object):
    def get_connection_pool(self, params:dict):
        # Given connection parameters in the `params` argument,
        # return new connection pool.
        # It should be overwritten if you want do something
        # before/after creating the connection pool, or return your
        # own connection pool.
        pass

    def get_connection(self, params:dict):
        # Given connection parameters in the `params` argument,
        # return a new connection.
        # It should be overwritten if you want to do something
        # before/after creating a new connection.
        # The default implementation uses `get_connection_pool`
        # to obtain a pool and create a new connection in the
        # newly obtained pool.
        pass

    def get_or_create_connection_pool(self, params:dict):
        # This is a high layer on top of `get_connection_pool` for
        # implementing a cache of created connection pools.
        # It should be overwritten if you want change the default
        # behavior.
        pass

    def make_connection_params(self, host:str, port:int, db:int) -> dict:
        # The responsibility of this method is to convert basic connection
        # parameters and other settings to fully connection pool ready
        # connection parameters.
        pass

    def connect(self, host:str, port:int, db:int):
        # This is really a public API and entry point for this
        # factory class. This encapsulates the main logic of creating
        # the previously mentioned `params` using `make_connection_params`
        # and creating a new connection using the `get_connection` method.
        pass

3.10. Pluggable parsers

redis-py (the Python Redis client used by django-redis) comes with a pure Python Redis parser that works very well for most common task, but if you want some performance boost, you can use hiredis.

hiredis is a Redis client written in C and it has its own parser that can be used with django-redis.

'OPTIONS': {
    'PARSER_CLASS': 'redis.connection.HiredisParser',
}

3.11. Pluggable clients

3.11.1. Default client

In addition to what has been explained previously, the default client comes with master-slave connection support. To use master-slave configuration in your project, you should change your LOCATION value to something like this:

'LOCATION': [
    '127.0.0.1:6379:1',
    '127.0.0.1:6378:1',
]
Warning
Master-Slave setup is not heavily tested in production environments.

3.11.2. Shard client

This pluggable client implements client-side sharding. It inherits almost all functionality from the default client. To use it, change your cache settings to something like this:

CACHES = {
    'default': {
        'BACKEND': 'redis_cache.cache.RedisCache',
        'LOCATION': [
            '127.0.0.1:6379:1',
            '127.0.0.1:6379:2',
        ],
        'OPTIONS': {
            'CLIENT_CLASS': 'redis_cache.client.ShardClient',
        }
    }
}
Warning
Shard client is still experimental, so be careful when using it in production environments.

3.11.3. Herd client

This pluggable client helps dealing with the thundering herd problem. You can read more about it on Wikipedia.

As with previous pluggable clients, it inherits all functionality from the default client, adding some additional methods for getting/setting keys.

Example setup
 CACHES = {
    'default': {
        'BACKEND': 'redis_cache.cache.RedisCache',
        'LOCATION': '127.0.0.1:6379:1',
        'OPTIONS': {
            'CLIENT_CLASS': 'redis_cache.client.HerdClient',
        }
    }
}

This client exposes additional settings:

  • CACHE_HERD_TIMEOUT: Set default herd timeout. (Default value: 60s)

3.11.4. Auto Failover client

This pluggable experimental client offers a simple failover algorithm if the main Redis server is down.

To use it, you should change your cache settings to something like this:

CACHES = {
    'default': {
        'BACKEND': 'redis_cache.cache.RedisCache',
        'LOCATION': '127.0.0.1:6379:1/127.0.0.2:6379:1',
        'OPTIONS': {
            'CLIENT_CLASS': 'redis_cache.client.SimpleFailoverClient',
        }
    }
}

The big difference is that each key in the LOCATION setting list can contain two connection strings separated by "/". A second connection string works as a failover server.

4. License

Copyright (c) 2011-2014 Andrei Antoukh <niwi@niwi.be>
Copyright (c) 2011 Sean Bleier

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
   derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.