Tuesday, June 11, 2013

Jedis - Java client for Redis

There are huge amount of clients for Redis written in different programming languages.

I've already written some thoughts about Redis and today we will talk about one of the most popular Java client for Redis - Jedis. Jedis is a blazingly small and sane Redis java client and was conceived to be EASY to use.

One of the main pros of Jedis is using all Redis command names as the names of the functions and it really simplifies working with Jedis.

There's no big deal to install Redis server. You just have to download the distributive, unarchive it and run the redis-server. More information you can find here and here.

So, let's assume that you've already installed Redis and able to ping it using redis-cli. It means you are ready to start working with Jedis.

Simple connection to Redis can be established in the following way:

    private static final int PORT = 6379;
    private static final String HOST_NAME = "localhost";
    // single instance
    public static Jedis JEDIS = new Jedis(HOST_NAME, PORT);


It's very simple, isn't it? Another way to establish connection is to use JedisPool - pool of connections:

private void init() {
            GenericObjectPool.Config poolConfig = new GenericObjectPool.Config();
            poolConfig.maxActive = 100;
            poolConfig.maxIdle = 10;
            poolConfig.minIdle = 2;
            poolConfig.maxWait = 100;
            poolConfig.testWhileIdle = true;
            poolConfig.testOnBorrow = true;
            poolConfig.testOnReturn = true;
            poolConfig.minEvictableIdleTimeMillis = 10000;
            poolConfig.timeBetweenEvictionRunsMillis = 5000;
            poolConfig.numTestsPerEvictionRun = 10;
            // create JEDIS pool
            this.jedisPool = new JedisPool(poolConfig, HOST_NAME, PORT);

}


Frankly speaking using the pool is preferable and I'd recommend to use the pool in production. In order to obtain Jedis object from the pool you should write something like this:

    public Jedis getCache() {
        return this.getJedisPool().getResource();
    }

    public void returnCache(Jedis cache) {
        this.getJedisPool().returnResource(cache);
    }


Don't forget about returning the Jedis instance back to the pool after doing some staff with it.

You can write few simple tests to see how Jedis is working and try to play with some Redis commands:

    @Test
    public void testSet() {
        Jedis jedis = connectionFactory.getCache();
        jedis.set("abc", "cba");
        Assert.assertEquals("cba", jedis.get("abc"));
        connectionFactory.returnCache(jedis);
    }


    @Test
    public void testDelete() {
        Jedis jedis = connectionFactory.getCache();
        jedis.set("abc", "cba");
        jedis.del("abc");
        Assert.assertNull(jedis.get("abc"));
        connectionFactory.returnCache(jedis);
    }


    @Test
    public void testAddToList() {
        Jedis jedis = connectionFactory.getCache();
        jedis.rpush("items", "111");
        jedis.rpush("items", "222");
        //check the list size
        Assert.assertEquals(new Long(2), jedis.llen("items"));
        connectionFactory.returnCache(jedis);
    }


As you can see it's very easy to connect to the Redis server using Jedis client.
It's intuitive, clear and simple, but the most important thing is - it's fast. You might want to take a look at Demo project and try to run a test for bulk loading.
You will be impressed. Hopefully, you'll have a fun.

Monday, June 10, 2013

Some thoughts about Redis

In this article I would like to shed light on what the Redis is. I will also explain when you may want to use it from my opinion and say few words about pros and cons of Redis.

I met with this storage few months ago during solving the task which required from me to use a distributed cache. Choosing the key/value store was not so simple task as I thought, because of our specific needs. Besides that the main requirements were stability, availability, scalability and, of course, speed (>60 000 ops/sec). Providing a simple and understandable API and not being a "monster" like MongoDB would also be a plus. So we decided to choose the winner from the following list: Couchbase (Couchbase/Memcached buckets), Memcached, Hazelcast and Redis.

We started our investigation from the Hazelcast, because we've been already using it in many other projects where it demonstrates pretty good results. We wrote simple test application trying to load 20 millions objects to the cache with 10 threads and after putting 7-8 millions objects Hazelcast has gone crazy. Sometimes it happened after loading 10 millions objects, but keeping in mind that we have to store up to 50 millions objects in storage, we decided to switch to the next candidate.

Then we were trying to see how both data buckets provided by Couchbase are working. From the first look Couchbase is awesome. It has a lot of different options like clustering, auto-failover, map/reduce and persistence for Couchbase buckets, asynchronous operations and many other things. Sounds good, yeah? Then we run our test and it quickly put everything in its place. The Couchbase Admin page showed incredible numbers for remote connection 50 000-60 000 ops/sec and >100 000 ops/sec for local connection, however only 40-60% of all data we were trying to load to the bucket was there in the end of the test. All other operations failed because of timeout. The Couchbase experts explained it by the fact that we didn't check the result of set/add operations. Yes, it was true, because when you add this check it means you are using synchronous operations instead of asynchronous and performance degradation is very impressive (<1000 ops/sec for remote connection and 20 000 - 30 000 ops/sec for local connection). So, taking into account that our distributed cache must handle huge amount of operations during bulk load correctly and in a very fast way, we decided to find luck somewhere else.

The next step was to choose between Memcached and Redis. It was our last hope... 

Memcached - free & open source, high-performance, distributed memory object caching system - was developed very long time ago and is very popular till now. It has only one disadvantage - it's a simple key/value store without an ability to work with different data structures. So, if you know that you won't need to store, for example, Sorted Set in the cache for sure, you might want to choose Memcached. Redis is a little bit more than that...

Thus, Redis became our choice and, fortunately, it meets all our needs. 

Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.

You can do "almost" the same operations with strings, lists, sets as you usually do working with simple String, ArrayList or HashSet in Java. Unfortunately, "almost" means not really, pardon the pun. For example, there is no ability to check whether the list contains some element or not. Strange? Yes, it's very strange. You wanna another "funny" example? Well, it's simple - you can get a range of elements from the sorted set but cannot do that using a regular one. Let's imagine that you don't want to use different data structures and want to use Redis as a simple key/value store, like a simple map. Another surprises will appear, be sure. For example it's not so simple to iterate over this map or get the ranges. I would say that it's even impossible to do without creating an additional key where you will store a list or a set of keys. What do you think about ability to specify the DB not by name and just by index? Why it's like that? - I have no idea. You can read about all operations provided by Redis here.

But let's finish telling bad things about Redis, because I have a feeling that you already don't like it. The good news are - it's stable, you can configure master-slave communication. It's very fast - some benchmarks say that it's even faster than memcached, but I would say that it's pretty much the similar. If you have enough memory you can implement missed functionality by creating additional lists, sets and so on. It also has a very good community. So basically it can do everything you might want to do with data in the cache and my suggestion is to play with it a little bit and you will understand what's what. Don't give up.

In the next article I will provide small example demonstrating how to use Jedis - Java client for Redis.

Notice that all tests in this article have been run on a specific environment and you probably will get different results testing all these storages on your custom environment.


Thursday, June 6, 2013

Few words about me and this blog

Greetings. If you might know I've just moved to Germany and started working here as a Java Software Engineer at Rakuten Deutchland GmbH. So, the new epoch for me has started...



Before it happened, I've been working on different positions at Intersog and Lohika - software companies located in Ukraine. Frankly speaking it was an excellent experience and great time. I was lucky to work on several startups as well as on a "real" enterprise HP product which is called Application Lyfecycle Management. During this period I was able to meet, work  and learn from the engineers which can be called a true GURU.
 
However, I feel that now is my turn to share the knowledge and through this blog I will try to provide as many interesting thoughts and articles as I can. Hope it'll help somebody.

I would also want to take a moment and talk about what kind of frameworks and technologies I like and usually use. My love number one is Spring framework and all infrastructure around it. I would recommend to take a look at the following blog SpringByExample. So, I think there are going to be several posts about Spring added in coming weeks, at least I will try to write something interesting. It was a time when I had written several JSF applications and to be honest I like it much more than GWT or so on. Of course I cannot live without utility libraries developed by apache and google. In addition to what I've already said I would add ORM frameworks, caching libraries, NoSQL and many-many other technologies. Someday this blog will contain articles about all these staff and even more.

Small announcement: the first technical article would be - "Some thoughts about Redis". To be continued...