Pages

Reading about ZeroMQ

I have just got this book, ZeroMQ - Use ZeroMQ and learn how to apply different message patterns, written by Faruk Akgul for Packt Publishing, and I about to read it. Some more substantial feedback in the near future.

It is a bit under my level, since it is thought for C developers at their first experience with ZeroMQ, but it is a while I don't actually work with this fine framework, and I felt it would be nice to have a refresh of the basic concepts starting from a different perspective.

Browsing the index, you could see how the book is structured to provide an introduction to ZeroMQ, letting you know how to write a simple C client-server application, describing how it works. Chapter two is mainly dedicated to a couple of messaging patterns, pub-sub and pipeline, and there is also a section dedicated to Valgrind, and how to use it to detect memory leaks on 0MQ. Chapter three gives more details on what a ZeroMQ socket is, and how to use it. In its second part, an introduction to CZMQ, the high(er) level C wrapper to the standard ZeroMQ library, is given. Chapter four delves a bit more on some more advanced topics.

The code in the book has been written for ZMQ version 3.2, and CZMQ 1.3.1; for what I have seen, the building instruction are provided only for the GCC compiler (version 4.7.2 is cited, I quite confident the newest 4.8 would be alright).

On Windows, Visual C++ is the suggested compiler. I couldn't see any detail on how to set MSVC for ZeroMQ in the book, still I could assure you that it is very easy to build a ZMQ solution in that environment too. If you need an hint, have a look at this ancient post of mine, that should be still valid.

Go to the full post

Matching bracket

Think of a string (it could be very long) composed exclusively by round brackets (aka parentheses). We want to check if it is correct, in the sense that each open bracket should have a matching close one. So, a string starting with a ")" is considered wrong, while the "()()()" string is a good one. Nested brackets are allowed, so "((()())(()()))" is a good sequence.

It is not difficult to think to a solution, still, you should not forget that unharmful looking hint on the possible large input size.

I bumped into this problem when having a coffee-break with colleagues. We were talking about interview questions, and someone remembered this one. In a few minutes we envisioned a couple of possible solutions, neither of them looked satisfactory. After a bit more of talking, and we found out a third candidate, that looked quite good.

First attempt

Naively, we could iteratively remove each "()" from the input string, until we could not find anymore such pattern. After that, we simply check if the string is empty. The good thing about this solution is that is easy to describe and implement. Less convincing is that we should copy the input to a buffer, and then perform on it a lot of splitting and splicing of substrings.

Let's see a possible C++ implementation of this idea:
bool simpleChecker(const std::string& input)
{
  std::string buffer(input);

  if(buffer.empty()) // 1
    return true;

  if(buffer.length() % 2) // 2
    return false;

  while(true)
  {
    std::string::size_type pos = buffer.find("()"); // 3
    if(pos == std::string::npos)
      break;

    buffer.replace(pos, 2, ""); // 4
  }

  return buffer.empty(); // 5
}
1. The requisites do not say if an empty string should be considered valid or not. Let's assume it should.
2. If we have an odd number of elements, the string is obviously invalid.
3. We stop looping if we can't find a "()" element.
4. When we find a "()", we replace it with an empty string (i.e., we remove it).
5. If the string is empty, the validation succeeded.

This code works fine, even for moderately large strings. In the best case, a monotonous sequence of "()()()()", the standard string replace() function is so nicely written that on my machine it takes a handful of millisecs to check an input of thousands characters. On the other side, it is easy to spot a sequence that gives troubles to replace(). The test case here below takes me something around one second on a 50K string:
TEST(CPPSimple, FiftyKWorstCase)
{
  const int SIZE = 50 * 1024;

  char buffer[SIZE + 1];
  for(int i = 0; i < SIZE / 2; ++i)
    buffer[i] = '(';
  for(int i = SIZE / 2; i < SIZE; ++i)
    buffer[i] = ')';
  buffer[SIZE] = '\0';

  EXPECT_TRUE(simpleChecker(buffer));
}
And his bigger brother, 250K sized, takes something like half a minute.

Recursive approach

The main issue of the first try is that it modifies the string. A better approach would be checking the string for each matching parenthesis, without doing any change. We read the first character in the passed string, it should be an '(', if the next one is a ')', cool, we have completed a sequence, we can move to the third character. Otherwise, we call recursively the same function, to extract a subsequence. It should return the length of the found sequence, or zero, if the subsequence is not valid. The caller would check the returned value, and move in the string accordingly, to see if the next character closes its sequence, or if it has to call again itself to check if we have another subsequence.

As you can see, this solution is more contrived, but promises to be more performant. Still, it has a big issue. In the same worst case as seen above, we have such a huge number of recursive function calls that it could easily lead to corruption in the memory stack.

Just count them

Yes, just count the brackets. After all, what we want is having the same number of open and close parenthesis.

Actually, we have also to take care that we don't want to see a close bracket when there is nothing to close, but it is easy to take care of it. Each time we have an open bracket, we put it on a stack, and we pop out one of them as we get a close bracket. If we get a closing bracket before any opening one has been scanned, the stack is empty, and so we know the string is not correct.

Here is how I have implemented this solution:
bool stackChecker(const char* input)
{
  int tracker = 0; // 1

  for(int i = 0; input[i] != '\0'; ++i)
  {
    switch(input[i])
    {
    case '(': // 2
      ++tracker;
      break;
    case ')': // 3
      if(tracker)
        --tracker;
      else
        return false;
    } // 4
  }

  return tracker ? false : true; // 5
}
1. I use this integer as the stack I talked above. Since I am not interested in the actual position of the brackets, I don't need a real stack, I just need to keep track of how many open brackets have been read.
2. I see an open bracket in the string, put it in the stack, and go to the next character.
3. Close bracket. If the stack is not empty, it matches with the last open bracket in the stack, I pop it and I am ready to check the next one. If the stack is empty, the input string is not valid.
4. What if the string contains anything else? I assumed a "don't care" behavior, but possibly the string could be considered invalid. If that is the case, a default section should be added to the switch, to always return false.
5. The complete string has been scanned, if no open bracket is left in the stack, we consider it valid.

Go to the full post

Redis client testing

I am a TDD guy. If I don't write some test cases, even for simple stuff, I feel uneasy. So, when I wrote the unique and the shared Radis wrappers for the previous posts, I let some (very basic) tests drive my code development. I guess it could be useful to have a look to them. The full C++ code is on github, you would need Google Test, Redis and its hiredis plugin, and a C++11-compliant compiler to build it. It won't be difficult to refactor the code for a different test framework (better if xUnit-based), and for a less modern C++ compiler, using Boost (or another library) for smart pointer support.

Firstly, I have written a tiny wrapper class to Google Test, just to make the code less messy, in Tester.h:
class Tester
{
public:
    Tester(int argc, char** argv) { testing::InitGoogleTest(&argc, argv); } // 1

    int run() { return RUN_ALL_TESTS(); } // 2
};
1. Before running any test, we should initialize the environment. What a better place than a constructor to do that?
2. I am happy with a very basic usage of Google Test, I would simply run all the tests.

Then I have written a simple main function, that creates an object Tester, and run it:
Tester(argc, argv).run();
I don't even use the return value from run(), I rely on the output generated by Google Test to let the user understand how the testing behaved.

Finally, I put in a source file all the tests I want to be performed. Here are some of them:
TEST(TestConnectShared, BadPort) // 1
{
    TTR::SharedRedis redis(1234); // 2
    ASSERT_FALSE(redis.isConnected()); // 3
}

TEST(TestConnectShared, Vanilla) // 4
{
    TTR::SharedRedis redis;
    ASSERT_TRUE(redis.isConnected());
}

TEST(TestCopyShared, Copy) // 5
{
    TTR::SharedRedis redis;
    ASSERT_TRUE(redis.isConnected());
    TTR::SharedRedis r2 = redis;
    ASSERT_TRUE(r2.isConnected());
}
1. The TEST() macro requires a test case name (here is TestConnectShared) and a test function name (BadPort). These names should make clear what we are testing here. In this case, I want to check the shared Redis connection behavior when I pass a wrong port number. Notice that the prerequisite is that the Redis server is up and running on localhost, default port.
2. I have put SharedRedis in a namespace named TTR (short for ThisThread Redis), that's way I use that prefix here.
3. I expect isConnected() to return false, so I assert it. If, unexpectedly, I get a valid connection to Redis, this test would fail.
4. The vanilla test on a shared Redis connection would try to create a default connection to Redis. If isConnected() does not return true, I should assume the test has failed.
5. A simple test to check if I can actually copy a shared Redis connection, and if the copy is still connected to Redis.

If you run those tests when a Redis server is not running on your localhost (and accepting connections on its default port), you should expect a number of failures. Actually, it would (wrongly) succeed only the tests expecting a failure, as the shown TestConnectShared-BadPort. Otherwise I would expect an all green lights scenario.

Go to the full post

Connecting to multiple Redis servers

In the previous post, I have showed how I designed a simple C++ wrapper class that uses a standard C++11 unique_ptr to help working with the hiredis context in such a scenario. My solution has a few limitation, first of them, it doesn't allow to connect to more than a Redis server. Often this is "as designed", still sometimes it is just a nuisance. In any case it is not difficult to redesign the code to allow connections to many Redis servers.

I have written a class named SharedRedis that wraps the hiredis functionality, storing the Redis connection in a C++11 standard shared_ptr smart pointer. You could find the full include file on github.
typedef std::shared_ptr<redisContext> SharedContext; // 1

class SharedRedis
{
    enum { DEFAULT_PORT = 6379 }; // 2
public:
    SharedRedis(const std::string& server, int port = DEFAULT_PORT); // 3
    SharedRedis(int port = DEFAULT_PORT) : SharedRedis("localhost", port) {}

    // ...
private:
    // ...
    SharedContext spContext_;
};
1. To keep the code more readable, I typedef'ed the shared pointer to the Redis context.
2. In this implementation, the user should specify host and port for the Redis server, by default the standard Redis port is used.
3. The only substantial difference to the UniqueRedis implementation (beside the usage of a different smart pointer) is that the object is freely built by the user. No static initializer, but just normal ctors are called to create a new object.

There is not much to say about the implementation, that is very close to what we have seen for the UniqueRedis version. You could see the full source code on github. Maybe it could be worthy to spend a few words on the shared smart pointer construction/destruction.
SharedRedis::SharedRedis(const std::string& server, int port)
{
    // ...
    spContext_ = SharedContext(context, std::bind(&SharedRedis::free, this)); // 1
}

void SharedRedis::free() // 2
{
    redisFree(spContext_.get());
}
1. In the class constructor, the context smart pointer is initialized passing the raw context pointer (we have ensured before that it is a valid value) and the function that has be called on its destruction. Here I use a method of the same class, I could have passed the native radisFree() function, but I wanted to add some stuff to its implementation (actually, just some logging, but I guess it is interesting to show the idea). That's way I had to bind the SharedRedis::free() function to the this pointer, so that the SharedContext destructor could be able to call the right function on the right object.
2. Here is the function that we want to be called on the SharedContext destruction.

There is still something missing. Probably, if we have many Redis connection we want to play with, we should think of storing them in a collection (maybe a map, using as key the host/port), so to have a centralized place to manage them consistently. But I guess this could be seen in another post.

Go to the full post

A unique Redis connection

I need to connect to Redis from a C++ application, and I plan to use hiredis, the Redis official C client, to do it. I want to use just one Redis connection, so I designed my C++ wrapper as a singleton. I would connect once, as I get the first request to Redis, and I would disconnect at the end of the application run.

The environment setup is done in a previous post, including the Makefile description, that you could download from github.

The access to Redis is ruled through a class, UniqueRedis, that has a minimal public interface. There is a static method to get the unique connection instance, a utility method to check if currently the connection is available, and a getter/setter couple. The full source code for the header file is on github:
typedef std::unique_ptr<redisContext, std::function<void(redisContext*)>> UniqueContext; // 1

class UniqueRedis
{
public:
    static UniqueRedis& instance() // 2
    {
        static UniqueRedis redis; // 3
        return redis;
    }

    bool isConnected() { return spContext_ ? true : false; } // 4

    void set(const std::string& key, const std::string& value); // 5
    std::string get(const std::string& key);
private:
    UniqueRedis(); // 6
    void free(); // 7

    UniqueRedis(const UniqueRedis&) = delete; // 8
    const UniqueRedis& operator=(const UniqueRedis&) = delete;

    UniqueContext spContext_; // 9
};
1. I don't want to manage directly the Redis context, I delegate instead a C++11 smart pointer to do the dirty job for me instead. I have chosen unique_ptr because I don't want it to be copied around, and I am giong to use it in the flavor that let the user passing a deleter to it, so that the smart pointer could also take care of calling that cleanup function when needed.
2. The user of this Redis wrapper, should call this static method to gain access to its unique object.
3. If you are using a compiler supporting the C++11 standard, as GNU GCC, initializing a local static variable is implicitely thread-safe.
4. This method returns true if the private smart pointer has been initialized, meaning that a connection to Redis has been already estabilished.
5. Setter and getter. The code is shown below.
6. A UniqueRedis object could be created only through the static initializer defined above.
7. Utility function to cleanup the Redis connection.
8. Copy ctor and assignment operator are explicitely deleted from the class interface.
9. The standard unique_ptr for the Redis context.

The source code for the cpp file is on github, too. Here are some stripped down and commented parts of it:
const std::string REDIS_HOST = "localhost"; // 1
const int REDIS_PORT = 6379;

UniqueRedis::UniqueRedis()
{
    redisContext* context = redisConnect(REDIS_HOST.c_str(), REDIS_PORT);

    if(context->err) // 2
    {
        redisFree(context);
        return;
    }

    spContext_ = UniqueContext(context, std::bind(&UniqueRedis::free, this)); // 3
}

void UniqueRedis::set(const std::string& key, const std::string& value) // 4
{
    void* reply = redisCommand(spContext_.get(), "SET %b %b", key.c_str(), key.length(), value.c_str(), value.length());
    if(reply)
    {
        freeReplyObject(reply);
        return;
    }
}

std::string UniqueRedis::get(const std::string& key)
{
    redisReply* reply = static_cast<redisReply*>(redisCommand(spContext_.get(), "GET %b", key.c_str(), key.length()));
    if(!reply)
        return "";

    std::string result = reply->str ? reply->str : "";
    freeReplyObject(reply);
    return result;
}
1. It would be a smarter idea to have Redis host and port in a configuration file, and fetch the actual values from there. But for the moment having them defined as constants will do.
2. If the hiredis function redisConnect() can't create a good connection to Redis using the passed host and port, I simply cleanup the context and return. You could decide to implement a more aggressive behavior (typically, throwing an exception).
3. We store the valid Redis context in the class private smart pointer. We pass as deleter the address to the member function that would free the context (not shown here, it boils down to call redisFree() for the raw Redis context pointer).
4. The call to redisCommand() is wrapped by set() and get(). As you can see, they work in a very similar way. Notice that I use the binary flag (%b) so that I could pass whatever comes from the user without troubles.

Go to the full post

GTest on Ubuntu

The Google C++ Testing Framework (better known as Google Test, or even GTest), has been designed to be as platform independent as possible. There are only a few little difference in setting it up on any specific environment, as you could see comparing this post with the one I wrote some time ago on how to set GTest up for Windows, and this one, focused on Linux (in a Debian flavor).

First step is always the same: go to Google code and download the Google Test zipped archive.

Do you already have cmake at hand? The Google Test package provides a few specific building files for some environments, the other ones are generated on the fly by this tool. Have a look at the official CMake site for more information. Here we'll be happy enough getting cmake from the standard repository:
sudo apt-get install cmake
Once you have unzipped the GTest package, you should see in its base directory a file named CMakeLists.txt, we'll pass it to cmake to generate a proper Makefile tailored on the current system characteristics.
cmake CMakeLists.txt
Do you see a newly generated Makefile? Good, make it up. As result you should get (among the other stuff) a couple of static libraries: libgtest.a and libgtest_main.a

I symlink-ed them in /usr/local/lib, then I symlink-ed the include/gtest directory to /usr/local/include. Now I am ready to write a tiny test application.

Here is the Makefile that I am going to use:
MY_NAME := gt
MY_SRCS := $(wildcard *.cpp)
MY_OBJS := ${MY_SRCS:.cpp=.o}

MY_INCLUDE_DIRS := /usr/local/include
MY_LIBRARY_DIRS := /usr/local/lib
MY_LIBRARIES := gtest

CXXFLAGS += $(foreach includedir,$(MY_INCLUDE_DIRS),-I$(includedir))
CXXFLAGS += -Wall -g -pthread
LDFLAGS += $(foreach librarydir,$(MY_LIBRARY_DIRS),-L$(librarydir))
LDLIBS += $(foreach library,$(MY_LIBRARIES),-l$(library))

.PHONY: all clean

all: $(MY_NAME)

$(MY_NAME): $(MY_OBJS)
    $(LINK.cc) -o $(MY_NAME) $(MY_OBJS) $(LDLIBS)

clean:
    @- rm -rf $(MY_OBJS) $(MY_NAME)
Notice that among the C++ flags it has been specified -pthread, since GTest requires the POSIX thread library. If you forget to specify it, you would get a bunch of undefined reference to pthreads symbols.

I reused the source code as defined in my old post about GTest on Windows that, as expected, works smoothly in this new context. The only (really minor) change that I applied is in the main function. Here I want to always run the tests and then give the control to the normal application execution path:
int main(int argc, char* argv[])
{
    Tester(argc, argv).run();

    std::cout << "Normal program behavior" << std::endl;
}
If you run it, you would get the output from GTest, where a test (TestIncrease.NormalBehavior) succeedes and another one (TestIncrease.ErrorTest) fails.

Alternatively, you could remove the C++ main function source from the build, and use instead the standard bootstrap functionality that GTest makes available. To do that, just add gtest_main among the libraries in the MY_LIBRARIES list.

Gary (thank you!) suggests me to stress a couple of points, very useful if you are new in this area:
  • Indentation in Makefile is done with tabs!
  • In the UNIX family of Operating Systems, the file separator is the slash ('/'), be careful not to use the backslash ('\')!

Go to the full post

Redis hello world

Once your environment is set up for Redis, it is easy to use the hiredis C interface to work with it. Here I write and use a minimal, shamelessly almost-C, bunch of functions to connect, set, get, and finally disconnect from a Redis server.

I am writing in C++, but here you won't see much of a difference from a plain C implementation. This is to keep the example focused on the hiredis features. I plan to refactor this code to have some C++ fun in a next post.

What I want to do, is connecting to a Redis server (assuming it runs locally on the default port), store on it a key/value pair, and immediately fetch it back. Something like this:
redisContext* ctx = TTR::connect("localhost", 6379); // 1
if(ctx)
{
    std::string key("key"); // 2
    std::string value("value");

    TTR::set(ctx, key, value); // 3
    std::string cache = TTR::get(ctx, key); // 4
    if(value == cache)
    {
        std::cout << "Value stored and fetched correctly" << std::endl;
    }
    else
    {
        std::cout << "Something weird happened" << std::endl;
    }

    TTR::disconnect(ctx); // 5
}
1. redisContext is the hiredis structure that keeps the context for a connection to Redis. Namespace is one of the few C++ features I'm using here, all my wrapper functions are in a namespace named TTR, so to avoid any name clash. The TTR::connect() function is one of them and, as you should expect, it establish a connection to the specified Redis server, or returns NULL in case of failure.
2. The key/value pair that I want to store on Redis.
3. Push a key/value to the server.
4. Fetch the value from the Radis server for the same key I have already used for the set(). I wouldn't ever expect the fetched value being different from the original one.
5. Do not forget to disconnect!

Writing code like that in the real world is asking for trouble. Converting my bunch of free functions in a class is easy, straightforward and immediately pays off, saving the nuisance of passing around the Redis context and be forced of taking care of its disposal. But I'll do it another time.

Let's now see how I have implemented my functions - remember that all of them are in the TTR namespace.

The most interesting one is connect():
redisContext* connect(const std::string& server, int port)
{
    std::string sport = server + ":" + boost::lexical_cast<std::string>(port); // 1

    if(!port || server.empty()) // 2
    {
        std::cout << "Can't connect to Redis [" + sport + "]" << std::endl;
        return NULL;
    }

    redisContext* context = redisConnect(server.c_str(), port); // 3
    if(!context) // paranoid
    {
        std::cout << "No memory for Redis on " + sport << std::endl;
        return NULL;
    }

    if(context->err) // 4
    {
        std::cout << "Can't connect to Redis on " + sport + " - " + context->errstr << std::endl;

        redisFree(context); // 5
        return NULL;
    }

    std::cout << "Connected to Redis [" + sport + "]" << std::endl;
    return context;
}
1. I concatenate the server name to the port number for debugging purpose, notice the usage of the handy boost lexical cast to convert an integer to a C++ string.
2. Test for valid user input. It could, and probably should, be more strict. But you get the idea.
3. Call the hiredis connection function. It would fail, returning NULL, for an out of memory problem. This is quite improbable. Still, better safe than sorry.
4. When there is a trouble connecting to the Redis server, we have it reported in the fields err and errstr on the Redis context object returned. If this is the case, I log a message, release the context, and return a fat NULL to the caller.
5. Remember, any context returned by redisConnect() has to be cleaned up calling redisFree().

The disconnect() is so boring that one would happily hide it in a class destructor:
void disconnect(redisContext* context)
{
    if(context) // 1
    {
        std::cout << "Disconnecting from Redis" << std::endl;
        redisFree(context);
    }
}
1. Passing a NULL to redisFree could result in a disaster (AKA a segmentation fault), so, I'd better check for it.

The real stuff, setting and getting a value on Redis, is done by these functions:
void set(redisContext* context, const std::string& key, const std::string& value) // 1
{
    if(!context) // 2
        return;

    void* reply = // 3
        redisCommand(context, "SET %b %b", key.c_str(), key.length(), value.c_str(), value.length());
    if(reply)
    {
        freeReplyObject(reply);
        return;
    }

    // unexpected
    std::cout << "No reply from Redis" << std::endl;
}

std::string get(redisContext* context, const std::string& key)
{
    if(!context)
        return "";

    redisReply* reply = static_cast<redisReply*>(redisCommand(context, "GET %b", key.c_str(), key.length()));
    if(!reply)
        return "";

    std::string result = reply->str ? reply->str : ""; // 4
    freeReplyObject(reply);
    return result;
}
1. We don't care about the Redis reply, any failure in setting a value for a specified key for the passed Redis context is (almost) silently ignored.
2. As we have already seen, Redis doesn't check if the context we pass to it is good or not. To avoid an unpleasent segmentation fault, it's better to check it ourself.
3. Here I don't care of what is the actual reply of the Redis server, I only check if it actually emits an answer and, if so, I clean it up.
As you can see, redisCommand() is designed to be similar to the standard C fprintf() function. First argument is the Redis context on which the call is performed, Than we have a string, containing the name of the actual operation (here is SET) and any required parameter, identified by a percent flag. If you know that you are about to send plain strings with no special character in it, you can use the "%s" placeholder. By I want to play safe, so I use the "%b", that allows binary strings to be sent. In this case I have to double the number of subsequent arguments, passing both the relative C-string and its size.
4. When I GET from Redis, I am much more interested in the reply object, so I cast the redisCommand() return value (a void pointer) to a pointer to its actual type, redisReply. I need its str field, that contains the value that Redis stores for the key passed as GET parameter, so firstly I check if the reply is not NULL, then I copy its str content in a C++ string, so that I can safely clean the reply object up before returning.

Go to the full post

Preparing to write a Redis client

I have to add some caching capabilities to a C++ module in my current project. There are a few viable alternatives, but we already have a Redis server available, and we don't have any compelling reason to look to anything else. So, it is almost done, I have just to download Redis source code, and preparing the environment to write a tiny test client.

There are a few ways to install Redis, I downloaded it from the official site, redis.io, following the instructions you could find there. In a matter of minutes I had a plain Redis server up and running and I checked it through redis-cli (Redis command line interface utility).

There are plenty of available Redis clients for different programming languages and you typically want to get an existing one matching your elected language. But in C++ case, I see only one alternative in the list proposed by Redis, and there are a few reasons not to peek it up. The code in the repository is quite old (2/3 years), and in the list it is not marked as recommended. So I fell back to hiredis, the official C client, included in the distribution (you could find it under deps/hiredis).

All I need is at hand, I just have to write a Makefile (you can download it from github), and then I am ready to write some code:
#
# Makefile for hiredis client
#

MY_NAME := hrc
MY_SRCS := $(wildcard *.cpp)
MY_OBJS := ${MY_SRCS:.cpp=.o}

MY_INCLUDE_DIRS := /usr/local/include/hiredis
MY_LIBRARY_DIRS := /usr/local/lib
MY_LIBRARIES := hiredis

CXXFLAGS += $(foreach includedir,$(MY_INCLUDE_DIRS),-I$(includedir))
CXXFLAGS += -Wall -g -std=c++11
LDFLAGS += $(foreach librarydir,$(MY_LIBRARY_DIRS),-L$(librarydir))
LDLIBS += $(foreach library,$(MY_LIBRARIES),-l$(library))

.PHONY: all clean

all: $(MY_NAME)

$(MY_NAME): $(MY_OBJS)
    $(LINK.cc) -o $(MY_NAME) $(MY_OBJS) $(LDLIBS)

clean:
    @- rm -rf $(MY_OBJS) $(MY_NAME)
My client is going to be named hrc (as MY_NAME shows), it is written in C++, and the code is contained in .cpp files (MY_SRCS). Any cpp file would have a matching object file identified by the "o" extension (MY_OBJS).

The only custom directory I want to include now is the one for the hiredis client (MY_INCLUDE_DIRS), and I want the make tool to check just in one custom directory for non standard library (MY_LIBRARY_DIRS), and the only custom library I currently want (MY_LIBRARIES) is hiredis. You would probably ask makefile to look in other directories. Remember that the actual library names are decorated versions for the bare name you should put in the makefile. In this case, Redis compilation has generated an archive named libhiredis.a, for static linkage, and a shareable object named libhiredis.so.

Besides, the linker would search for the actual versioned file, as maintained by ldd. In the worst case scenario, it could happens that the linker would complain for missing a versioned so. You could fix this by ldconfig, or even by hand, creating a reference to the missing file through symlink.

I add to the CXXFLAGS firstly the dependencies for include directories and then a bunch of option - I want all the warnings up (-Wall), the executable filled with debug information (-g), and the code generated to support the latest C++ standard (-std=c++11).

In the LDFLAGS I add the dependencies for library directories, and in LDLIBS the actual requested library names.

An then I defines a few targets. A couple of phony ones, "all" and "clean", as tradition wants, and the one that actually creates my target.

Notice that I don't specify explicitly the name of the C++ compiler, but I rely on the make tool to be smart enough to deduce it from the environment. For this reason I use LINK.cc, that in my case contains the expected g++.

In the next post, I am going to write some C++ code meant to be compiled with this Makefile.

Go to the full post

When an iterator is not dereferencable

Say that I am using an STL container (for instance, a queue) in your code. Somewhere I want to get an element from it, calling front(). Say that I forget to check if there is actually anything in it before grabbing it, and I am so unluck to find out that it is empty. What it is going to happens?

Accordingly to the C++ standard, no one knows. It is one of those undefined behavior that should worry you so much to be very careful in writing C++ code.

If you are developing on Visual Studio, and you application is built in debug mode, the disaster is shielded by an assertion failure that pops up a window with some debug information that should help you to find the offending line (just press the retry button, and debug your code).

If you want to see what happens in these cases, you could try this code:
#include <queue>

// ...
void f()
{
// ...
    std::queue<int> qi;

//    qi.front(); // deque iterator not dereferencable
//    qi.pop(); // deque empty before pop

// ...
}
1. If you uncomment this line, you would get a "deque iterator not dereferencable" assertion, since you are trying to dereference an iterator to an empty queue.
2. Trying to popping an empty queue results in a "deque empty before pop" assertion.

[edit: Jon commented on google plus to this post, adding information for g++ and STLport. It is a very long time since the last time I have used the latter, but g++ is everyday matter, and Jon is right, I'd better complete the discussion with some words on the behavior of the Free Software Foundation compiler]

To check what happens on g++, compile the code in debug mode, defining the constant _GLIBCXX_DEBUG, and run the resulting application.

For both call you will get the same error message, "attempt to access an element in an empty container.", the execution will be aborted, and you will get some information on the objects involved in the operation.

Go to the full post

Httpd virtual hosts

I wanted to manage a couple of web sites, let's call them one.dd and two.dd, with my Apache Web Server, and I wanted them to live on the same machine, sharing the same IP address. We know that to do that in the real life, I could not choose randomly a fancy name, like I have just said, but I have to register a proper name under well known limitations. But if I play just on my local machine(s), I can forget about that, and being free and foolish. Still I have to follow a few basic rules.

I am working on a Debian box, on a Apache httpd 2.2 built from scratch, downloading the package from the official Apache site. I reckon you can adapt very easily what I have done to your current setup.

Setting the hosts

The operating system should be aware of the names I want to use on the current machine. This is done in a text file, typically (for *x environments) named /etc/hosts. There we see, among the other things, the standard mapping between 127.0.0.1 and localhost, and we are about to extend it to add our two host names:
127.0.0.1 localhost one.dd two.dd
Setting the httpd configuration

Apache has to know how to manage our virtual hosts, too. The standard http configuration file, conf/httpd.conf, has a commented line that, when activated, includes the specific configuration file for virtual hosts.
# Virtual hosts
#Include conf/extra/httpd-vhosts.conf
It is usually considered a better idea to let the provided example alone, and work on a different file.

This is my virtual host configuration file:
# Virtual Hosts
NameVirtualHost *

<VirtualHost *>
    DocumentRoot /site/www/one.dd
    ServerName www.one.dd
</VirtualHost>

<VirtualHost *>
    DocumentRoot /site/www/two.dd
    ServerName www.two.dd
</VirtualHost>
I guess this is the simplest configuration file one could conceive.

The directive NameVirtualHost says to Apache that we want to attach one or more virtual hosts to the specified address/port. Here I passed a star to it, meaning "anything you get to this point". Usually you want to be more choosy. Besides, I didn't specify any port number. In this case, Apache assumes I expect it to use the one specified in the Listen directive.

Then I have a VirtualHost block for each host I want to define. If anything not matching with the ServerName's specified is getting here, the first one is considered as the default one.

The DocumentRoot says to Apache which directory to use as root for the site. I have created the specified document root directories, and put in both of them an index.html file.

Looks easy, doesn't it? Still, even at this basic level, there are a few thing that could go wrong. And the resulting error messages could look cryptical.

Wrong!

If NameVirtualHost is not matching with any VirtualHost (a different port number is enough) Apache doesn't know what to do of that directive, and a "NameVirtualHost has no VirtualHosts" warning is issued at startup.

I have already noted that if the NameVirtualHost port is not explicitly given, the one specified in the Listen directive is used. But you should ensure to keep the same convention for the associated VirtualHost, too. Otherwise you could get a "VirtualHost mixing * ports and non-* ports with a NameVirtualHost address is not supported, proceeding with undefined results".

Go to the full post

Iterating over an Apache apr_table_t

A common data structure that is very useful to have at hand when working with a web server, is an associative array where both key and value are strings. If Apache httpd was developed in C++, they would have probably used an STL unordered_map, but here we are dealing with pure C, so an internal data structure named apr_table_t has been designed expressly for this scope, with a bunch of associated functions for manage it.

Here I am going to write an example that uses apr_table_do() to loop over all the elements in an Apache table.

What I want to do is writing an Apache2 module that generates as output an HTML page listing all the properties in the request header.

If we have a look to the apr_tables.h, we'll find this couple of interesting lines:
typedef int (apr_table_do_callback_fn_t)(
    void* rec, const char* key, const char* value);

int apr_table_do(apr_table_do_callback_fn_t* comp,
    void* rec, const apr_table_t* t, ...);
The apr_table_do() gets as first parameter a callback function, then the module request record, and the Apache table we want to loop on. Finally we specify which tables elements we are interested in, or a NULL if we want to go through all of them.

Here is the function I want to use as callback, a simple output of the current key-value pair:
int print(void* rec, const char* key, const char* value)
{
    request_rec* r = static_cast<request_rec*>(rec); // 1
    ap_rprintf(r, "%s: %s<br />\n", key, value); // 2

    return 1; // 3
}
1. Tiny nuisance, the request_rec is seen by the callback prototype as a void pointer - to allow more flexibility, I reckon - so we need to cast it back to its original type. I was about to check the cast result, but in the end I decided that was a bit too paranoid for such a basic example.
2. Dump the pair to the HTML response that the module is generating.
3. And finally return a non-zero value, to mean success.

In the handler, I'll have something like:
int handler(request_rec* r)
{
    // ...
    apr_table_do(print, r, r->headers_in, NULL);

    // ...
    return OK;
}
The full C++ source code is on github. You should compile it, possibly using a make file like the one showed in the previous post, and make the resulting shared object available to Apache.

In the httpd configuration file, we should explain to Apache how to map a request to the server to a call for our module, and how to load the module:
<Location /info>
    SetHandler info
</Location>

# ...

LoadModule info_module modules/mod_info.so
And what it is left to do, to have the new module available, it is just stop and start your Apache server.

Go to the full post

Makefile for C++ Apache module

The Apache web server (AKA httpd, or just Apache) is written in C language, but this is not a compelling reason for us to write our modules in the same language. And, as you could expect, it is pretty easy to use the C++ language instead.

Converting the minimal Hello World and the simple example from C to C++ (actually g++ 4.4.5 on Linux Debian for Apache 2.2) took a minimal effort.

What I had to do was adding an explicit include directive for http_protocol.h, to let the less forgiving C++ compiler to properly check against a few functions. Not doing it was leading to these errors:
error: ‘ap_set_content_type’ was not declared in this scope
error: ‘ap_rputs’ was not declared in this scope
error: ‘ap_rprintf’ was not declared in this scope
Besides, I also removed the static specification for all the local function, and put them instead in an unnamed namespace.

Finally I wrote this Makefile:
all: mod_hello.so

mod_hello.o : mod_hello.cpp
    g++ -c -I/path/to/apache22/include -fPIC mod_hello.cpp

mod_hello.so : mod_hello.o
    g++ -shared -o mod_hello.so mod_hello.o

clean:
    rm -rf mod_hello.o mod_hello.so
To build the object I called g++ with a few options:
-c because I don't want it to run the linker, its output should be the object file.
-I to specify the apache include directory (put there your actual one).
-fPIC is due to the fact that we are about to create a shared object, so we need g++ to generate position-independent code.

The actual generation of the shared object is accomplished by second call to g++, this time specifying as options:
-shared to let it know that a shared object is what we want.
-o to specify the output file name.

Remember that in a Makefile you should put TAB and only TAB (no white spaces at all!), if you don't want to get a puzzling error like this:
Makefile:6: *** missing separator.  Stop.

Go to the full post