Redis Notes

Russell Bateman
July 2019

Redis is an in-memory, key-value store. It may be thought of as a NoSQL database. It's clusterable and offers built-in replication. It's open source. It also offers a bit more than a mere key-value store as it supports multiple data structures.

String values
Lists
Sets (with scores)
Sorted sets
Hashes
Bitmaps (special-semantics strings)
Hyperlogs (ibid)
Geospatial indices

There is hardly a programming language for which there's no Redis API.

Some links:

Installation

russ@nargothrond ~/dev $ mkdir redis
russ@nargothrond ~/dev $ cd redis
russ@nargothrond ~/dev/redis $ wget http://download.redis.io/redis-stable.tar.gz
--2019-07-03 15:13:23--  http://download.redis.io/redis-stable.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2014657 (1.9M) [application/x-gzip]
Saving to: ‘redis-stable.tar.gz’

redis-stable.tar.gz           100%[=================================================>]   1.92M  2.20MB/s    in 0.9s

2019-07-03 15:13:24 (2.20 MB/s) - 'redis-stable.tar.gz' saved [2014657/2014657]

russ@nargothrond ~/dev/redis $ tar -zxf redis-stable.tar.gz
russ@nargothrond ~/dev/redis $ cd redis-stable/
russ@nargothrond ~/dev/redis/redis-stable $ make
cd src && make all
make[1]: Entering directory '/home/russ/dev/redis/redis-stable/src'
    CC Makefile.dep
    rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-rdb redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep dict-benchmark
    (cd ../deps && make distclean)
    make[2]: Entering directory '/home/russ/dev/redis/redis-stable/deps'
    .
    .
    .
make[1]: Leaving directory '/home/russ/dev/redis/redis-stable/src'

Launching the Redis server

russ@nargothrond ~/dev/redis/redis-stable $ cd src
russ@nargothrond ~/dev/redis/redis-stable/src $ ./redis-cli ping
Could not connect to Redis at 127.0.0.1:6379: Connection refused
russ@nargothrond ~/dev/redis/redis-stable/src $ ./redis-server &
[1] 15159
russ@nargothrond ~/dev/redis/redis-stable/src $ 15159:C 03 Jul 2019 15:20:53.356 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
15159:C 03 Jul 2019 15:20:53.356 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=15159, just started
15159:C 03 Jul 2019 15:20:53.356 # Warning: no config file specified, using the default config. In order to specify a config file use ./redis-server /path/to/redis.conf
15159:M 03 Jul 2019 15:20:53.358 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
       _.-``    `.  `_.  ''-._           Redis 5.0.5 (00000000/0) 64 bit
   .-`` .-```.  ```\/    _.,_ ''-._
  (    '      ,       .-`  | `,    )     Running in standalone mode
  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
  |    `-._   `._    /     _.-'    |     PID: 15159
  `-._    `-._  `-./  _.-'    _.-'
  |`-._`-._    `-.__.-'    _.-'_.-'|
  |    `-._`-._        _.-'_.-'    |           http://redis.io
   `-._    `-._`-.__.-'_.-'    _.-'
  |`-._`-._    `-.__.-'    _.-'_.-'|
  |    `-._`-._        _.-'_.-'    |
   `-._    `-._`-.__.-'_.-'    _.-'
       `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

15159:M 03 Jul 2019 15:20:53.359 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15159:M 03 Jul 2019 15:20:53.359 # Server initialized
15159:M 03 Jul 2019 15:20:53.359 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15159:M 03 Jul 2019 15:20:53.359 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15159:M 03 Jul 2019 15:20:53.359 * Ready to accept connections
russ@nargothrond ~/dev/redis/redis-stable/src $ disown

Launching the command-line Redis client against the server

russ@nargothrond ~/dev/redis/redis-stable/src $ ./redis-cli ping
PONG
russ@nargothrond ~/dev/redis/redis-stable/src $ ./redis-cli
127.0.0.1:6379> SET mykey somevalue
OK
127.0.0.1:6379> GET mykey
"somevalue"
127.0.0.1:6379> SET age 64
OK
127.0.0.1:6379> GET age
"64"
127.0.0.1:6379> INCR age
(integer) 65
127.0.0.1:6379> GET age
"65"
127.0.0.1:6379> EXISTS poop
(integer) 0
127.0.0.1:6379> GET poop
(nil)

A Java-based consumer

The highlighted lines show what I'm using Redis for. In short, I'm de-identifying documents containing personal medical information which is illegal to expose. For age, I'm calculating an offset by which I'll skew the age of a particular patient in every separate document (hence the use of Redis), but I always skew it by the same amount lest the documents no longer correlate to tell an accurate story. Later, I'll skew names, streets, city, state, various identifiers, etc. storing them in Redis in order to de-identify all associated patient documents accurately.

RandomUtilities.java:

package com.windofkeltia.utilities;

import java.util.Random;

public class RandomUtilities
{
  public static int getRandomIntegerInRange( int bottom, int top )
  {
    if( bottom > top )
      throw new IllegalArgumentException( "Range: top must be greater than bottom" );

    Random random = new Random();
    return random.nextInt( ( top - bottom ) + 1 ) + bottom;
  }

  public static long getRandomLongInRange( long bottom, long top )
  {
    return bottom + ( long ) ( Math.random() * ( top - bottom ) );
  }
}

RedisTest.java:

package com.windofkeltia.redis;

import java.time.Duration;

import org.junit.Test;

import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

import com.windofkeltia.utilities.RandomUtilities;

public class RedisTest
{
  @Test
  public void test()
  {
    JedisPoolConfig poolConfig = buildPoolConfig();
    JedisPool       jedisPool  = new JedisPool( poolConfig, "localhost" );

    try( Jedis pairs = jedisPool.getResource() )
    {
      final int AGE_OFFSET = RandomUtilities.getRandomIntegerInRange( 0, 5 );
      int delta = Integer.parseInt( "44" ) + AGE_OFFSET;

      pairs.set( "44", delta + "" );
      String ageForThisOne = pairs.get( "44" );

      System.out.println( "            AGE OFFSET: " + AGE_OFFSET );
      System.out.println( "Deidentified age delta: " + ageForThisOne );
    }
  }

  private JedisPoolConfig buildPoolConfig()
  {
    final JedisPoolConfig poolConfig = new JedisPoolConfig();
    poolConfig.setMaxTotal( 128 );
    poolConfig.setMaxIdle( 128 );
    poolConfig.setMinIdle( 16 );
    poolConfig.setTestOnBorrow( true );
    poolConfig.setTestOnReturn( true );
    poolConfig.setTestWhileIdle( true );
    poolConfig.setMinEvictableIdleTimeMillis( Duration.ofSeconds( 60 ).toMillis() );
    poolConfig.setTimeBetweenEvictionRunsMillis( Duration.ofSeconds( 30 ).toMillis() );
    poolConfig.setNumTestsPerEvictionRun( 3 );
    poolConfig.setBlockWhenExhausted( true );
    return poolConfig;
  }
}

Output. This will change each time the test is run because a new, random offset from the actual age is chosen.

            AGE OFFSET: 5
Deidentified age delta: 49

Redis Java clients

Jedis, is a tiny client, not thread-safe (reentrancy left to consumer), conceived to be easy to use.

<dependency>
  <groupId>redis.clients</groupId>
  <artifactId>jedis</artifactId>
  <version>3.1.0</version>
</dependency>

While Baeldung's tutorial gives complex code (see A Java-based consumer, above), Jedis is really very, very simple to code to, as simple as this operation which attaches to Redis, establishes a value for key "foo" and is able to fetch that value back:

Jedis jedis = new Jedis( "localhost" );
jedis.set( "foo", "bar" );
String value = jedis.get( "foo" );

Redisson, a high-performance, asynchronous and lock-free client that uses the Netty NIO client-server framework.

<dependency>
  <groupId>org.redisson</groupId>
  <artifactId>redisson</artifactId>
  <version>3.11.1</version>
</dependency>

Lettuce, interface plus support for reactive streams. It uses Netty which makes its API much heavier than others, but better suited for sharing a connection across multiple threads. See also Baeldung: Introduction to Lettuce—the Java Redis Client.

<dependency>
  <groupId>io.lettuce</groupId>
  <artifactId>lettuce-core</artifactId>
  <version>5.1.0</version>
</dependency>

Redis vs. Kafka

(When using Redis publish/subscribe.)

Redis is push-based, meaning it's a mostly fire-and-forget system where a message is delivered immediately to all consumers and the data is kept nowhere. Redis has memory limitations too and is degraded in its performance by the number of producers and consumers.

Kafka is a high-throughput, distributed log that can be used as a queue. It is pull-based, requiring consumers (subscribers) to ask for messages as soon as they are ready to handle them. Any number of producers and consumers can be active any time. Kafka provides persistence for the messages in its queues.

Use Redis if...

you don't care that messages (data) are lost,
speed is what you're after,
the amount of data exchanged isn't huge.

Use Kafka instead if...

reliability is crucial,
you need to safeguard messages (i.e.: keep copies even after consumption),
speed is not so crucial,
you're dealing with large amounts of data.