Writing sequential test scripts with node

Today I was trying to create a node.js script to test a HTTP service but the test required multiple steps.   I gave it a try by using async module to "symplify" that code and that's the ugly code I came up with.

I'm not an expert in js/node, feel free to comment if I'm doing something wrong, I'm more than happy to learn.

(inflightSession and create are two helper functions that I have)

Test using node + jasmine: 

it("should accept valid sessionId", function(done) {
      async.waterfall([
          inflightSession,

          function(sessionId, callback) {
            create({ 'sessionId':sessionId }, callback);
          }
      ], function(error, response) {
          expect(response.statusCode).to.equal(200);
          done()
    });
  });

Same test using python + unittest:

def create_ok_test():
    session_id = inflight_session()
    response = create({ 'sessionId': session_id })

    assert_equals(200, response.status_code)

Same test using node ES6 generators (yield keyword):

it("should accept valid sessionId", function*() {
      var sessionId = yield inflightSession();
      var response = yield create({ 'sessionId': sessionId });
      expect(response.statusCode).to.equal(200);
  });

Honestly the code in python is way more readable than the existing node code, and still better even when comparing it with the new node generators.    Anway definitely looks like a promising way to move forward in the node community.   Some comments:

ES6 generators are available under a flag in node 0.11 and are supposed to be included in 0.12.

yield is a common keyword in other languages (i.e. python, C#) to exit from a function but keeping the state of that function so that you can resume the execution later.

function* is the syntax to define a generator function (a function using yield inside).

You need a runner supporting those generator functions (in this example jasmine needs to add support for it), basically calling the generator.next and waiting for the result (the result should be a promise or similar object) before calling generator.next again.

UPDATE: As I´m somehow forced to use node, I ended up creating a helper function and my tests are now like this

itx("should accept valid sessionId", inflightSession, function(sessionId, done) {
      create({ 'sessionId':sessionId }, function(error, response) {
          expect(response.statusCode).to.equal(200);
          done()
      });
});

Distributed Load Testing: Conclussions (5/5)

Let's recap what we have done in these series and try to get some conclusions.   The steps or achievements are these ones:

1. Find and test a distributed load testing tool in python: locust.
2. Extend locust for custom (non-HTTP) protocol testing.
3. Use Instant Servers to run the locust master and slaves.
4. Implement a simple way to autostop the machines when they are not being used based on the locust logs and instant servers stop API.
5. Create a template for the slaves to be easily cloned.   Use instant servers tags to define groups.
6. Fix the python Instant Server SDK and extend it with new authentication and clone features.
7. Extend locust interface adding a button to spawn machines in instant servers directly from the locust web interface.

Today I like even more python and the testing tools based in scripting instead of complex UIs.  This project gave me the oportunity to discover locust and Instant Servers and highly recommend people to use them for this kind of use case, it was very easy and a lot of fun using and combining those technologies.  Hopefully I can get more time for deeper integration of virtual machines in locust (with a good control UI and perhaps support for other providers).

Distributed Load Testing: py-smartdc and spawing slaves from locust (4/5)

After all the previous work I was able to spawn slaves easily from the Instant Servers interface (just clicking Clone) or with the command line tools but I wanted to go further and explore the extension capabilities of locust to add some very simple support to the locust web page to create the slaves.

How to clone and tag an Instant Servers machine with python

I have to recognize that my first instinct was to try to create a python Instant Servers SDK,  I even created the github repo and built the skeleton of the SDK, but 10 mins later somebody told me how stupid I was because there was already an official python SDK :(

Ok, that's great I though, but when I tried to use it I realize that it was not compatible with Instant Servers.   The "problem" is that the SDKs are maintained by joyent and even if Instant Servers is the same infrastructure, the API version is not exactly the same and the existing python SDK doesn't work with Telefonica infrastructure.

The solution was easy and I created my fork and pull requested a patch [1] that still hasn't been merged.   Feel free to use my fork! [2]  In addition I added another patch to support username&password authentication [3], and another one to add support for cloning machines

Once that's solved creating a machine by cloning the template instance we built in the previous post is very easy:

from smartdc import DataCenter, TELEFONICA_LOCATIONS

mad = DataCenter(location='eu-mad-1',
              known_locations=TELEFONICA_LOCATIONS,
              login='is0012', password='HNSnFAkc', api_version='6.5')

template_found = False
for machine in mad.machines():
        tags = machine.get_tags()

        if tags.get('locust') == 'slave-template':
                new_machine = machine.clone()
                new_machine.add_tags(locust='slave')
                print new_machine.name + ' ' + str(new_machine.get_tags())
                template_found = True

if not template_found:
        print 'slave-template instance not found'

How to integrate spawning machines in locust

Locust is easily extensible in the testing scripts as we saw in the previous post but also in the UI.   It is easy to add functionality to the website but modifying the template and adding more HTTP routes to process new API requests.  In my case I added a new button to spawn an Instant Servers machine and a route to process that request:

In locust/templates/index.html: 

                <div class="top_box box_stop box_running" id="box_reset">
                    <a href="/stats/reset">Reset Stats</a></br>
                    {% if is_distributed %}
                        <a href="/cloud/create">Spawn Slave</a>
                    {% endif %}
                </div>


In a new file locust/cloud.py:

from smartdc import DataCenter, TELEFONICA_LOCATIONS
from locust import web

mad = DataCenter(location='eu-mad-1',
              known_locations=TELEFONICA_LOCATIONS,
              login='', password='', api_version='6.5')

@web.app.route("/cloud/create")
def cloud_create():

    template_found = False
    for machine in mad.machines():
        tags = machine.get_tags()

        if tags.get('locust') == 'slave-template':
                new_machine = machine.clone()
                new_machine.add_tags(locust='slave')
                return new_machine

    return None

You can find my locust fork in [5], don't forget to change your login and password in the cloud.py file.

The result is this simple new button with the functionality required to spawn new machines automatically configured as locust slaves connected to the master for distributed testing.

[1] https://github.com/atl/py-smartdc/pull/9
[2] https://github.com/ggarber/py-smartdc/
[3] https://github.com/atl/py-smartdc/pull/10
[4] https://github.com/atl/py-smartdc/pull/11
[5] https://github.com/ggarber/locust

Distributed Load Testing: Using Instant Servers for semi-automated slaves spawning (3/5)

As I mention in the first post of this series virtual machines provides a very dynamic and cheap way to create slave nodes for load testing.  There are different providers out there but I was interested on using Instant Servers (based on Joyent technology) because it is really simple to start with and it is fun to explore new solutions instead of using always the boring Amazon infrastructure.

The three small features I was interested on implementing with Instant Servers were:

• Simplify the creation of machines preconfigured to be used of locust slaves for my load testing
• Starting the slaves automatically on the machine startup so that the master detects them and is able to schedule jobs
• Stopping the machine automatically when it is not used for any test for some time to make sure we don't waste our money when forgetting to stop those unused machines

Creation of machines

To simplify the creation of machines I decided to build an instance with all the required packages and use it as template to clone the actual slave nodes.    To be able to find this instance later automatically I used Instant Server tags.  As far as I know unfortunately there is no UI for tagging in Instant Servers but you could use a script similar to mine [1].

This machine needs to have python, locust, the test file (locustfile.py), all the basic packages (make, gcc) and the packages required to start locust automatically (see next point).

Auto starting slaves

To make sure that the slaves are started during machine startup and that we start multiple instances in every box (because locust is single threaded and I want to use multicore machines) I used supervisord with the following (hopefully autoexplicative) configuration.   Tune the numprocs parameter depending on the machine you are using.

/etc/supervisor/conf.d/locust.conf

[program:locust]
command=locust -f /root/locustfile.py --master-host=81.45.23.221 --slave
stderr_logfile = /var/log/supervisord/locust-stderr.log
stdout_logfile = /var/log/supervisord/locust-stdout.log
process_name=%(program_name)s_%(process_num)02d
numprocs=4

Auto stopping machines

To monitor the usage of a machine I could be using the extensive Analytics API in instant servers but I decided to use the locust log file for simplicity.   I created a python script [2] monitoring log files for activity and if there is no activity for some minutes it invokes the stop Instant Servers API to shut down that instance.

Note: It was not possible to use the existing python SDK with Instant Servers and I had to fork it and made a small modification.  More details in next post.


PD. If I ever mention the word "cloud" in any of this posts, please feel free to insult me, I will deserve it.

[1] https://gist.github.com/ggarber/8381238
[2] https://gist.github.com/ggarber/8381263

Scala I. New years's plan

Over the last years I have been trying to learn one new cool language, but because of my lack of consistency I've been jumping constantly between Erlang, Go and Scala, reading some tutorials, writting some examples... but never getting a real deep understanding of any of them.

It is always good to do plans for the new year, I promise to go for running much more and to read more non technical books, so this year I really want to master one of those languages because I think it is healthy to learn new things, because it is usually a lot of fun and because I think I need it to improve the way I develop software at work.

By the way, I recommend the book Seven Languages on Seven Weeks if you are trying to make a decision like mine.

So, which one to choose....  (spoiler, I have decided to learn Scala).. these are my arguments...

Go is very interesting but who else apart of Google is using Go for serious projects?  In addition it looks like a mixture of python and C and I want something different.

Erlang is really cool and probably the best language/framework for the type of applications I usually build as part of my daily job.   But it is not strongly typed, there are no good tools and doc and it is very difficult to engage people on it.   I tried for the last two months and I give up.

Scala is slightly less cool because it has some Java inheritance, but because of that it is the easier one to introduce in a Java organization.   The language is modern (support for functional programming is basic) and expressive and combined with Akka framework you are close to Erlang features.

So, let's go, hopefully next post will be on some cool Scala feature.

Distributed load testing: Extending locus.io for connection oriented services (2/5)

I have to admit that when talking about testing scripts I'm a python fan because of the simplicity, low overhead and availability of libraries (Ruby could be a good option too).  So, the first think I did when started to build my distributed load testing environment was searching google for "python distributed load testing" and as usual google didn't dissapoint me and that was the beginning of fun with locust!

locust is python testing framework where you write your tests in python and the execution can be scheduled in one or multiple machines.    It has facilities for HTTP testing but can be extended to other services and provides a web interface to get easy access to the progress and results of the tests.

After digging for a while in the documentation (not that good) and the source code (nice code) I decided to give it a try and create my first non HTTP test.   When creating a Locust class (that class defines the configuration of the tests) you can specify a client to override the default HTTP client:

class MyLocust(Locust):
    task_set = MyTaskSet
    min_wait = 500
    max_wait = 500

    def __init__(self):
        super(MyLocust, self).__init__()

        self.client = MyProtocolLocustClient(host=self.host)

class MyProtocolLocustClient(object):
    def __init__(self, host):
        self.host = host
        self.connection = MyProtocolConnection(host)

    def ping(self):
        request_meta = {}
        request_meta["start_time"] = time.time()
        request_meta["method"] = 'MESSAGE'
        request_meta["name"] = 'PING'

        self.connection.send_message(self.connection.id(), "HI")

        try:
            response = self.connection.recv_message()

            if payload != "HI":
                raise Exception()

            request_meta["response_time"] = (time.time() - request_meta["start_time"]) * 1000

            events.request_success.fire(
                    method=request_meta["method"],
                    name=request_meta["name"],
                    response_time=request_meta["response_time"],
                    response_length=0
                )
        except Exception as e:
            events.request_failure.fire(
                    method=request_meta["method"],
                    name=request_meta["name"],
                    response_time=0,
                    exception=e,
                    response=None,
                )


That client object is available to all the locust TaskSets that you will define later with your specific tests.  For example:

class MyTaskSet(TaskSet):
    @task
    def my_task(l):
        l.client.ping()

And that's all, just put that code in a locustfile.py file and run it from the command line:
 locust -f locustfile.py -H xxxx.yyyy.com

And open a browser in http://localhost:8089 to start the test and get the results and you should get something like this:

Xamarin and Google Glass together

For the latest weeks I've been playing around with Google Glass, amazing technology!!! and for about a year I've been following Xamarin and using it for small projects/hackathons because I think that it is the PERFECT way to develop mobile apps.   Today it is time to try to use both together!!!

Let's start with some introductions:

Google Glass runs Android 4.0.4 and you can easily run any android app [1]  (thanks song for introducing me to Google Glass world).   You can run any standard android app, Google just released a specific SDK (the GDK) that you can also use with Xamarin if you want deeper integration with Google Glasses.

Xamarin is a cross platform framework allowing you to create mobile apps (e.g. for android) in C#/.NET will using the native UI framework in each platform.

Now, let's start having fun:

1. Download the standard Xamarin package and open Xamarin Studio

2. Create an Android project targeting an android version <= 4.0.4

3. Disable the system status bar and the application title for your Application with this code:
            Window.AddFlags(WindowManagerFlags.Fullscreen);
            RequestWindowFeature(WindowFeatures.NoTitle);

4. Use standard android controls, buttons, textviews...

5. Try to follow google guidelines if you want to create a coherent experience [2]

Screen on Google Glass

I don't have a Xamarin license to create something much better (because of the limitations of Starter licence) but if somebody gets one for me I'll be happy to build something much better to show the powerful combination of Xamarin and Google Glass :)

[1] http://songz.quora.com/How-to-run-Android-Apps-on-Google-Glass
[2] https://developers.google.com/glass/design/index