Showing posts with label Redis. Show all posts
Showing posts with label Redis. Show all posts

Redis client testing

I am a TDD guy. If I don't write some test cases, even for simple stuff, I feel uneasy. So, when I wrote the unique and the shared Radis wrappers for the previous posts, I let some (very basic) tests drive my code development. I guess it could be useful to have a look to them. The full C++ code is on github, you would need Google Test, Redis and its hiredis plugin, and a C++11-compliant compiler to build it. It won't be difficult to refactor the code for a different test framework (better if xUnit-based), and for a less modern C++ compiler, using Boost (or another library) for smart pointer support.

Firstly, I have written a tiny wrapper class to Google Test, just to make the code less messy, in Tester.h:
class Tester
{
public:
    Tester(int argc, char** argv) { testing::InitGoogleTest(&argc, argv); } // 1

    int run() { return RUN_ALL_TESTS(); } // 2
};
1. Before running any test, we should initialize the environment. What a better place than a constructor to do that?
2. I am happy with a very basic usage of Google Test, I would simply run all the tests.

Then I have written a simple main function, that creates an object Tester, and run it:
Tester(argc, argv).run();
I don't even use the return value from run(), I rely on the output generated by Google Test to let the user understand how the testing behaved.

Finally, I put in a source file all the tests I want to be performed. Here are some of them:
TEST(TestConnectShared, BadPort) // 1
{
    TTR::SharedRedis redis(1234); // 2
    ASSERT_FALSE(redis.isConnected()); // 3
}

TEST(TestConnectShared, Vanilla) // 4
{
    TTR::SharedRedis redis;
    ASSERT_TRUE(redis.isConnected());
}

TEST(TestCopyShared, Copy) // 5
{
    TTR::SharedRedis redis;
    ASSERT_TRUE(redis.isConnected());
    TTR::SharedRedis r2 = redis;
    ASSERT_TRUE(r2.isConnected());
}
1. The TEST() macro requires a test case name (here is TestConnectShared) and a test function name (BadPort). These names should make clear what we are testing here. In this case, I want to check the shared Redis connection behavior when I pass a wrong port number. Notice that the prerequisite is that the Redis server is up and running on localhost, default port.
2. I have put SharedRedis in a namespace named TTR (short for ThisThread Redis), that's way I use that prefix here.
3. I expect isConnected() to return false, so I assert it. If, unexpectedly, I get a valid connection to Redis, this test would fail.
4. The vanilla test on a shared Redis connection would try to create a default connection to Redis. If isConnected() does not return true, I should assume the test has failed.
5. A simple test to check if I can actually copy a shared Redis connection, and if the copy is still connected to Redis.

If you run those tests when a Redis server is not running on your localhost (and accepting connections on its default port), you should expect a number of failures. Actually, it would (wrongly) succeed only the tests expecting a failure, as the shown TestConnectShared-BadPort. Otherwise I would expect an all green lights scenario.

Go to the full post

Connecting to multiple Redis servers

In the previous post, I have showed how I designed a simple C++ wrapper class that uses a standard C++11 unique_ptr to help working with the hiredis context in such a scenario. My solution has a few limitation, first of them, it doesn't allow to connect to more than a Redis server. Often this is "as designed", still sometimes it is just a nuisance. In any case it is not difficult to redesign the code to allow connections to many Redis servers.

I have written a class named SharedRedis that wraps the hiredis functionality, storing the Redis connection in a C++11 standard shared_ptr smart pointer. You could find the full include file on github.
typedef std::shared_ptr<redisContext> SharedContext; // 1

class SharedRedis
{
    enum { DEFAULT_PORT = 6379 }; // 2
public:
    SharedRedis(const std::string& server, int port = DEFAULT_PORT); // 3
    SharedRedis(int port = DEFAULT_PORT) : SharedRedis("localhost", port) {}

    // ...
private:
    // ...
    SharedContext spContext_;
};
1. To keep the code more readable, I typedef'ed the shared pointer to the Redis context.
2. In this implementation, the user should specify host and port for the Redis server, by default the standard Redis port is used.
3. The only substantial difference to the UniqueRedis implementation (beside the usage of a different smart pointer) is that the object is freely built by the user. No static initializer, but just normal ctors are called to create a new object.

There is not much to say about the implementation, that is very close to what we have seen for the UniqueRedis version. You could see the full source code on github. Maybe it could be worthy to spend a few words on the shared smart pointer construction/destruction.
SharedRedis::SharedRedis(const std::string& server, int port)
{
    // ...
    spContext_ = SharedContext(context, std::bind(&SharedRedis::free, this)); // 1
}

void SharedRedis::free() // 2
{
    redisFree(spContext_.get());
}
1. In the class constructor, the context smart pointer is initialized passing the raw context pointer (we have ensured before that it is a valid value) and the function that has be called on its destruction. Here I use a method of the same class, I could have passed the native radisFree() function, but I wanted to add some stuff to its implementation (actually, just some logging, but I guess it is interesting to show the idea). That's way I had to bind the SharedRedis::free() function to the this pointer, so that the SharedContext destructor could be able to call the right function on the right object.
2. Here is the function that we want to be called on the SharedContext destruction.

There is still something missing. Probably, if we have many Redis connection we want to play with, we should think of storing them in a collection (maybe a map, using as key the host/port), so to have a centralized place to manage them consistently. But I guess this could be seen in another post.

Go to the full post

A unique Redis connection

I need to connect to Redis from a C++ application, and I plan to use hiredis, the Redis official C client, to do it. I want to use just one Redis connection, so I designed my C++ wrapper as a singleton. I would connect once, as I get the first request to Redis, and I would disconnect at the end of the application run.

The environment setup is done in a previous post, including the Makefile description, that you could download from github.

The access to Redis is ruled through a class, UniqueRedis, that has a minimal public interface. There is a static method to get the unique connection instance, a utility method to check if currently the connection is available, and a getter/setter couple. The full source code for the header file is on github:
typedef std::unique_ptr<redisContext, std::function<void(redisContext*)>> UniqueContext; // 1

class UniqueRedis
{
public:
    static UniqueRedis& instance() // 2
    {
        static UniqueRedis redis; // 3
        return redis;
    }

    bool isConnected() { return spContext_ ? true : false; } // 4

    void set(const std::string& key, const std::string& value); // 5
    std::string get(const std::string& key);
private:
    UniqueRedis(); // 6
    void free(); // 7

    UniqueRedis(const UniqueRedis&) = delete; // 8
    const UniqueRedis& operator=(const UniqueRedis&) = delete;

    UniqueContext spContext_; // 9
};
1. I don't want to manage directly the Redis context, I delegate instead a C++11 smart pointer to do the dirty job for me instead. I have chosen unique_ptr because I don't want it to be copied around, and I am giong to use it in the flavor that let the user passing a deleter to it, so that the smart pointer could also take care of calling that cleanup function when needed.
2. The user of this Redis wrapper, should call this static method to gain access to its unique object.
3. If you are using a compiler supporting the C++11 standard, as GNU GCC, initializing a local static variable is implicitely thread-safe.
4. This method returns true if the private smart pointer has been initialized, meaning that a connection to Redis has been already estabilished.
5. Setter and getter. The code is shown below.
6. A UniqueRedis object could be created only through the static initializer defined above.
7. Utility function to cleanup the Redis connection.
8. Copy ctor and assignment operator are explicitely deleted from the class interface.
9. The standard unique_ptr for the Redis context.

The source code for the cpp file is on github, too. Here are some stripped down and commented parts of it:
const std::string REDIS_HOST = "localhost"; // 1
const int REDIS_PORT = 6379;

UniqueRedis::UniqueRedis()
{
    redisContext* context = redisConnect(REDIS_HOST.c_str(), REDIS_PORT);

    if(context->err) // 2
    {
        redisFree(context);
        return;
    }

    spContext_ = UniqueContext(context, std::bind(&UniqueRedis::free, this)); // 3
}

void UniqueRedis::set(const std::string& key, const std::string& value) // 4
{
    void* reply = redisCommand(spContext_.get(), "SET %b %b", key.c_str(), key.length(), value.c_str(), value.length());
    if(reply)
    {
        freeReplyObject(reply);
        return;
    }
}

std::string UniqueRedis::get(const std::string& key)
{
    redisReply* reply = static_cast<redisReply*>(redisCommand(spContext_.get(), "GET %b", key.c_str(), key.length()));
    if(!reply)
        return "";

    std::string result = reply->str ? reply->str : "";
    freeReplyObject(reply);
    return result;
}
1. It would be a smarter idea to have Redis host and port in a configuration file, and fetch the actual values from there. But for the moment having them defined as constants will do.
2. If the hiredis function redisConnect() can't create a good connection to Redis using the passed host and port, I simply cleanup the context and return. You could decide to implement a more aggressive behavior (typically, throwing an exception).
3. We store the valid Redis context in the class private smart pointer. We pass as deleter the address to the member function that would free the context (not shown here, it boils down to call redisFree() for the raw Redis context pointer).
4. The call to redisCommand() is wrapped by set() and get(). As you can see, they work in a very similar way. Notice that I use the binary flag (%b) so that I could pass whatever comes from the user without troubles.

Go to the full post

Redis hello world

Once your environment is set up for Redis, it is easy to use the hiredis C interface to work with it. Here I write and use a minimal, shamelessly almost-C, bunch of functions to connect, set, get, and finally disconnect from a Redis server.

I am writing in C++, but here you won't see much of a difference from a plain C implementation. This is to keep the example focused on the hiredis features. I plan to refactor this code to have some C++ fun in a next post.

What I want to do, is connecting to a Redis server (assuming it runs locally on the default port), store on it a key/value pair, and immediately fetch it back. Something like this:
redisContext* ctx = TTR::connect("localhost", 6379); // 1
if(ctx)
{
    std::string key("key"); // 2
    std::string value("value");

    TTR::set(ctx, key, value); // 3
    std::string cache = TTR::get(ctx, key); // 4
    if(value == cache)
    {
        std::cout << "Value stored and fetched correctly" << std::endl;
    }
    else
    {
        std::cout << "Something weird happened" << std::endl;
    }

    TTR::disconnect(ctx); // 5
}
1. redisContext is the hiredis structure that keeps the context for a connection to Redis. Namespace is one of the few C++ features I'm using here, all my wrapper functions are in a namespace named TTR, so to avoid any name clash. The TTR::connect() function is one of them and, as you should expect, it establish a connection to the specified Redis server, or returns NULL in case of failure.
2. The key/value pair that I want to store on Redis.
3. Push a key/value to the server.
4. Fetch the value from the Radis server for the same key I have already used for the set(). I wouldn't ever expect the fetched value being different from the original one.
5. Do not forget to disconnect!

Writing code like that in the real world is asking for trouble. Converting my bunch of free functions in a class is easy, straightforward and immediately pays off, saving the nuisance of passing around the Redis context and be forced of taking care of its disposal. But I'll do it another time.

Let's now see how I have implemented my functions - remember that all of them are in the TTR namespace.

The most interesting one is connect():
redisContext* connect(const std::string& server, int port)
{
    std::string sport = server + ":" + boost::lexical_cast<std::string>(port); // 1

    if(!port || server.empty()) // 2
    {
        std::cout << "Can't connect to Redis [" + sport + "]" << std::endl;
        return NULL;
    }

    redisContext* context = redisConnect(server.c_str(), port); // 3
    if(!context) // paranoid
    {
        std::cout << "No memory for Redis on " + sport << std::endl;
        return NULL;
    }

    if(context->err) // 4
    {
        std::cout << "Can't connect to Redis on " + sport + " - " + context->errstr << std::endl;

        redisFree(context); // 5
        return NULL;
    }

    std::cout << "Connected to Redis [" + sport + "]" << std::endl;
    return context;
}
1. I concatenate the server name to the port number for debugging purpose, notice the usage of the handy boost lexical cast to convert an integer to a C++ string.
2. Test for valid user input. It could, and probably should, be more strict. But you get the idea.
3. Call the hiredis connection function. It would fail, returning NULL, for an out of memory problem. This is quite improbable. Still, better safe than sorry.
4. When there is a trouble connecting to the Redis server, we have it reported in the fields err and errstr on the Redis context object returned. If this is the case, I log a message, release the context, and return a fat NULL to the caller.
5. Remember, any context returned by redisConnect() has to be cleaned up calling redisFree().

The disconnect() is so boring that one would happily hide it in a class destructor:
void disconnect(redisContext* context)
{
    if(context) // 1
    {
        std::cout << "Disconnecting from Redis" << std::endl;
        redisFree(context);
    }
}
1. Passing a NULL to redisFree could result in a disaster (AKA a segmentation fault), so, I'd better check for it.

The real stuff, setting and getting a value on Redis, is done by these functions:
void set(redisContext* context, const std::string& key, const std::string& value) // 1
{
    if(!context) // 2
        return;

    void* reply = // 3
        redisCommand(context, "SET %b %b", key.c_str(), key.length(), value.c_str(), value.length());
    if(reply)
    {
        freeReplyObject(reply);
        return;
    }

    // unexpected
    std::cout << "No reply from Redis" << std::endl;
}

std::string get(redisContext* context, const std::string& key)
{
    if(!context)
        return "";

    redisReply* reply = static_cast<redisReply*>(redisCommand(context, "GET %b", key.c_str(), key.length()));
    if(!reply)
        return "";

    std::string result = reply->str ? reply->str : ""; // 4
    freeReplyObject(reply);
    return result;
}
1. We don't care about the Redis reply, any failure in setting a value for a specified key for the passed Redis context is (almost) silently ignored.
2. As we have already seen, Redis doesn't check if the context we pass to it is good or not. To avoid an unpleasent segmentation fault, it's better to check it ourself.
3. Here I don't care of what is the actual reply of the Redis server, I only check if it actually emits an answer and, if so, I clean it up.
As you can see, redisCommand() is designed to be similar to the standard C fprintf() function. First argument is the Redis context on which the call is performed, Than we have a string, containing the name of the actual operation (here is SET) and any required parameter, identified by a percent flag. If you know that you are about to send plain strings with no special character in it, you can use the "%s" placeholder. By I want to play safe, so I use the "%b", that allows binary strings to be sent. In this case I have to double the number of subsequent arguments, passing both the relative C-string and its size.
4. When I GET from Redis, I am much more interested in the reply object, so I cast the redisCommand() return value (a void pointer) to a pointer to its actual type, redisReply. I need its str field, that contains the value that Redis stores for the key passed as GET parameter, so firstly I check if the reply is not NULL, then I copy its str content in a C++ string, so that I can safely clean the reply object up before returning.

Go to the full post

Preparing to write a Redis client

I have to add some caching capabilities to a C++ module in my current project. There are a few viable alternatives, but we already have a Redis server available, and we don't have any compelling reason to look to anything else. So, it is almost done, I have just to download Redis source code, and preparing the environment to write a tiny test client.

There are a few ways to install Redis, I downloaded it from the official site, redis.io, following the instructions you could find there. In a matter of minutes I had a plain Redis server up and running and I checked it through redis-cli (Redis command line interface utility).

There are plenty of available Redis clients for different programming languages and you typically want to get an existing one matching your elected language. But in C++ case, I see only one alternative in the list proposed by Redis, and there are a few reasons not to peek it up. The code in the repository is quite old (2/3 years), and in the list it is not marked as recommended. So I fell back to hiredis, the official C client, included in the distribution (you could find it under deps/hiredis).

All I need is at hand, I just have to write a Makefile (you can download it from github), and then I am ready to write some code:
#
# Makefile for hiredis client
#

MY_NAME := hrc
MY_SRCS := $(wildcard *.cpp)
MY_OBJS := ${MY_SRCS:.cpp=.o}

MY_INCLUDE_DIRS := /usr/local/include/hiredis
MY_LIBRARY_DIRS := /usr/local/lib
MY_LIBRARIES := hiredis

CXXFLAGS += $(foreach includedir,$(MY_INCLUDE_DIRS),-I$(includedir))
CXXFLAGS += -Wall -g -std=c++11
LDFLAGS += $(foreach librarydir,$(MY_LIBRARY_DIRS),-L$(librarydir))
LDLIBS += $(foreach library,$(MY_LIBRARIES),-l$(library))

.PHONY: all clean

all: $(MY_NAME)

$(MY_NAME): $(MY_OBJS)
    $(LINK.cc) -o $(MY_NAME) $(MY_OBJS) $(LDLIBS)

clean:
    @- rm -rf $(MY_OBJS) $(MY_NAME)
My client is going to be named hrc (as MY_NAME shows), it is written in C++, and the code is contained in .cpp files (MY_SRCS). Any cpp file would have a matching object file identified by the "o" extension (MY_OBJS).

The only custom directory I want to include now is the one for the hiredis client (MY_INCLUDE_DIRS), and I want the make tool to check just in one custom directory for non standard library (MY_LIBRARY_DIRS), and the only custom library I currently want (MY_LIBRARIES) is hiredis. You would probably ask makefile to look in other directories. Remember that the actual library names are decorated versions for the bare name you should put in the makefile. In this case, Redis compilation has generated an archive named libhiredis.a, for static linkage, and a shareable object named libhiredis.so.

Besides, the linker would search for the actual versioned file, as maintained by ldd. In the worst case scenario, it could happens that the linker would complain for missing a versioned so. You could fix this by ldconfig, or even by hand, creating a reference to the missing file through symlink.

I add to the CXXFLAGS firstly the dependencies for include directories and then a bunch of option - I want all the warnings up (-Wall), the executable filled with debug information (-g), and the code generated to support the latest C++ standard (-std=c++11).

In the LDFLAGS I add the dependencies for library directories, and in LDLIBS the actual requested library names.

An then I defines a few targets. A couple of phony ones, "all" and "clean", as tradition wants, and the one that actually creates my target.

Notice that I don't specify explicitly the name of the C++ compiler, but I rely on the make tool to be smart enough to deduce it from the environment. For this reason I use LINK.cc, that in my case contains the expected g++.

In the next post, I am going to write some C++ code meant to be compiled with this Makefile.

Go to the full post