Trond Norbye's Weblog

Manage Gearmand and Drizzle with SMF

If you are running Gearman or Drizzle on Solaris, you may want to let SMF start and monitor the services. I just pushed service definitions and and scripts to install them a couple of days ago.

If you look in the scripts directory in Gearman, you will see a script named smf_install.sh. If you run this script it will define a user and group named gearmand, create the Gearman authorizations and profile before a service named gearman is defined. To start the Gearman service all you need to do is to run:

trond@storm ~> svcadm enable gearman

For Drizzle you need to look in the support-files/smf directory for a script named install.sh. To start Drizzle all you need to do is to run:

trond@storm ~> svcadm enable drizzle

Posted at 06:06PM May 25, 2009 by trond in OpenSolaris  |  Comments[0]

Connection pooling libmemcached

A while back I looked at the Memcached UDF for MySQL, and noticed that it didn't use libmemcached in an optimal way. In order to work in a multithreaded environment it used the following pattern:

   memcached_st* clone = memcached_clone(NULL, memc);

   ... memcached operations using the clone ---   

   memcached_free(clone);

Well, that doesn't look bad, does it? Well, it isn't that bad, but if you look at the network traffic you will see that we end up connecting / disconnecting to the involved memcached servers every time, and memcached is not optimized for "single-shot" connections.

So how should you solve this? Well, you should reuse your clones! And luckily for you, you don't have to reinvent the wheel. Yesterday I pushed a patch to libmemcached introducing a new library: libmemcachedutil. The intention of that library is to put utility functions built on top of libmemcached that you might want to use in your application, and the first routine there is the pool functionality.

So let's write some code using the new library:

#include <pthread.h>
#include <stdbool.h>
#include <signal.h>

#include "libmemcached/memcached_util.h"


static volatile bool run = true;

static void sig_handler(int sig) {
    assert(sig == SIGINT);
    run = false;
}

static void* my_application_thread(void *arg) 
{
  memcached_pool_st* pool = arg;

  while (run) {
    memcached_return rc;
    memcached_st* mem = memcached_pool_pop(pool, true, &rc);

    if (mem != NULL) {
      ... use the memcached handle for whatever you want! ...

      /* Return the instance to the pool */
      if (memcached_pool_push(pool, mem) != MEMCACHED_SUCCESS) {
        fprintf(stderr, "Failed to release the memcached instance!\n");
      }
    } else {
      fprintf(stderr, "Failed to get the memcached instance from pool!\n");
    }
  }

  return NULL;
}

int main(int argc, char** argv)
{
  memcached_st* memc = memcached_create(NULL);
  if (memc == NULL) {
    fprintf(stderr, "Failed to create memcached instance\n");
    return 1;
  }

  if (memcached_server_add(memc, "localhost", 11211) != MEMCACHED_SUCCESS) {
    fprintf(stderr, "Failed to add localhost to the server pool\n");
    memcached_free(memc);
    return 1;
  }
 
  memcached_pool_st* pool= memcached_pool_create(memc, 5, 10);
  if (pool == NULL) {
    fprintf(stderr, "Failed to create connection pool\n");
    memcached_free(memc);
    return 1;
  }

  signal(SIGINT, sig_handler);

  /* create 10 threads to use the pool */
  pthread tid[10];
  for (int x= 0; x < 10; ++x) {
    pthread_create(&tid[x], NULL, my_application_thread, pool);
  }

  for (int x= 0; x < 10; ++x) {
    pthread_join(&tid[x], NULL);
  }

  /* Release allocated resources */
  memcached_pool_destroy(pool);
  memcached_free(memc);

  return 0;
}

Posted at 10:26AM May 12, 2009 by trond in Memcached  |  Comments[0]