Distributed Load Testing: py-smartdc and spawing slaves from locust (4/5)

After all the previous work I was able to spawn slaves easily from the Instant Servers interface (just clicking Clone) or with the command line tools but I wanted to go further and explore the extension capabilities of locust to add some very simple support to the locust web page to create the slaves.

How to clone and tag an Instant Servers machine with python

I have to recognize that my first instinct was to try to create a python Instant Servers SDK,  I even created the github repo and built the skeleton of the SDK, but 10 mins later somebody told me how stupid I was because there was already an official python SDK :(

Ok, that's great I though, but when I tried to use it I realize that it was not compatible with Instant Servers.   The "problem" is that the SDKs are maintained by joyent and even if Instant Servers is the same infrastructure, the API version is not exactly the same and the existing python SDK doesn't work with Telefonica infrastructure.

The solution was easy and I created my fork and pull requested a patch [1] that still hasn't been merged.   Feel free to use my fork! [2]  In addition I added another patch to support username&password authentication [3], and another one to add support for cloning machines

Once that's solved creating a machine by cloning the template instance we built in the previous post is very easy:

from smartdc import DataCenter, TELEFONICA_LOCATIONS

mad = DataCenter(location='eu-mad-1',
              known_locations=TELEFONICA_LOCATIONS,
              login='is0012', password='HNSnFAkc', api_version='6.5')

template_found = False
for machine in mad.machines():
        tags = machine.get_tags()

        if tags.get('locust') == 'slave-template':
                new_machine = machine.clone()
                new_machine.add_tags(locust='slave')
                print new_machine.name + ' ' + str(new_machine.get_tags())
                template_found = True

if not template_found:
        print 'slave-template instance not found'

How to integrate spawning machines in locust

Locust is easily extensible in the testing scripts as we saw in the previous post but also in the UI.   It is easy to add functionality to the website but modifying the template and adding more HTTP routes to process new API requests.  In my case I added a new button to spawn an Instant Servers machine and a route to process that request:

In locust/templates/index.html: 

                <div class="top_box box_stop box_running" id="box_reset">
                    <a href="/stats/reset">Reset Stats</a></br>
                    {% if is_distributed %}
                        <a href="/cloud/create">Spawn Slave</a>
                    {% endif %}
                </div>


In a new file locust/cloud.py:

from smartdc import DataCenter, TELEFONICA_LOCATIONS
from locust import web

mad = DataCenter(location='eu-mad-1',
              known_locations=TELEFONICA_LOCATIONS,
              login='', password='', api_version='6.5')

@web.app.route("/cloud/create")
def cloud_create():

    template_found = False
    for machine in mad.machines():
        tags = machine.get_tags()

        if tags.get('locust') == 'slave-template':
                new_machine = machine.clone()
                new_machine.add_tags(locust='slave')
                return new_machine

    return None

You can find my locust fork in [5], don't forget to change your login and password in the cloud.py file.

The result is this simple new button with the functionality required to spawn new machines automatically configured as locust slaves connected to the master for distributed testing.

[1] https://github.com/atl/py-smartdc/pull/9
[2] https://github.com/ggarber/py-smartdc/
[3] https://github.com/atl/py-smartdc/pull/10
[4] https://github.com/atl/py-smartdc/pull/11
[5] https://github.com/ggarber/locust