Distributed Load Testing: Conclussions (5/5)

Let's recap what we have done in these series and try to get some conclusions.   The steps or achievements are these ones:

1. Find and test a distributed load testing tool in python: locust.
2. Extend locust for custom (non-HTTP) protocol testing.
3. Use Instant Servers to run the locust master and slaves.
4. Implement a simple way to autostop the machines when they are not being used based on the locust logs and instant servers stop API.
5. Create a template for the slaves to be easily cloned.   Use instant servers tags to define groups.
6. Fix the python Instant Server SDK and extend it with new authentication and clone features.
7. Extend locust interface adding a button to spawn machines in instant servers directly from the locust web interface.

Today I like even more python and the testing tools based in scripting instead of complex UIs.  This project gave me the oportunity to discover locust and Instant Servers and highly recommend people to use them for this kind of use case, it was very easy and a lot of fun using and combining those technologies.  Hopefully I can get more time for deeper integration of virtual machines in locust (with a good control UI and perhaps support for other providers).

Distributed load testing: Extending locus.io for connection oriented services (2/5)

I have to admit that when talking about testing scripts I'm a python fan because of the simplicity, low overhead and availability of libraries (Ruby could be a good option too).  So, the first think I did when started to build my distributed load testing environment was searching google for "python distributed load testing" and as usual google didn't dissapoint me and that was the beginning of fun with locust!

locust is python testing framework where you write your tests in python and the execution can be scheduled in one or multiple machines.    It has facilities for HTTP testing but can be extended to other services and provides a web interface to get easy access to the progress and results of the tests.

After digging for a while in the documentation (not that good) and the source code (nice code) I decided to give it a try and create my first non HTTP test.   When creating a Locust class (that class defines the configuration of the tests) you can specify a client to override the default HTTP client:

class MyLocust(Locust):
    task_set = MyTaskSet
    min_wait = 500
    max_wait = 500

    def __init__(self):
        super(MyLocust, self).__init__()

        self.client = MyProtocolLocustClient(host=self.host)

class MyProtocolLocustClient(object):
    def __init__(self, host):
        self.host = host
        self.connection = MyProtocolConnection(host)

    def ping(self):
        request_meta = {}
        request_meta["start_time"] = time.time()
        request_meta["method"] = 'MESSAGE'
        request_meta["name"] = 'PING'

        self.connection.send_message(self.connection.id(), "HI")

        try:
            response = self.connection.recv_message()

            if payload != "HI":
                raise Exception()

            request_meta["response_time"] = (time.time() - request_meta["start_time"]) * 1000

            events.request_success.fire(
                    method=request_meta["method"],
                    name=request_meta["name"],
                    response_time=request_meta["response_time"],
                    response_length=0
                )
        except Exception as e:
            events.request_failure.fire(
                    method=request_meta["method"],
                    name=request_meta["name"],
                    response_time=0,
                    exception=e,
                    response=None,
                )


That client object is available to all the locust TaskSets that you will define later with your specific tests.  For example:

class MyTaskSet(TaskSet):
    @task
    def my_task(l):
        l.client.ping()

And that's all, just put that code in a locustfile.py file and run it from the command line:
 locust -f locustfile.py -H xxxx.yyyy.com

And open a browser in http://localhost:8089 to start the test and get the results and you should get something like this:

Creating "REST" APIs with django

I've been trying to implement an API for our service with some django based frameworks for the last week and to be honest the experience hasn't been as placent as I expected.

These are the frameworks that I evaluated and the reasons that made me get away or make use of them:

• piston: the project looks abandoned and it is not compatible with latest django version (because of automatic csrf checks).  It seems to be very used with hundreds of forks but none of them convinced me.   Its strongest points are OAuth support and being used for the bitbucket API.

• tastypie: the project is actively maintained and looks very nice but I had some problems as soon as I tried to implement things a bit different than usual.  I didn't like the concept of hydrating/dehydrating objects. In addition it has some external dependencies and I didn't like very much the way it is coded when I looked at the source code.

• djangorestframework: it looks less featured but simpler and actively maintained.   It also took me some time to implement things that were a bit different from the usual use case, but it was finally my choice.  It source code is clean and easy to understand.

During this evaluation process I also found this table with info about different  implementations that could be useful.   There are also a lot of stackoverflow questions about this issue.