on Apr 25, 2013 by Ramona
Getting our Selenium tests to run faster can be complicated at times, because there are a number of factors to be taken into consideration. Debugging these tests is time consuming and the end result is almost always the same one - a large part of the tests needs to be refactored.
pip install -r requirements.txtand you’re good to go. It is recommended that you use a virtualenv, so that you avoid any conflicts with software already installed on your computer.
on Apr 25, 2013 by Danu
I’ve been watching both presentations that Carl Meyer held at Pycon 2012/13 and I highly recommend them if you want a deep dive into writing tests with django. They outline some very good principles for writing effective and maintainable tests. They also highlight a suite of test utilities and frameworks which help you in writing better tests. Among the others, Webtest caught my attention via django-webtest for writing integration/functional tests.
tests/__init__.pyand for rest of the goodness that nose offers.
on Feb 18, 2013 by Ramona
Selenium is great by itself, but its API grows every day - it’s not obvious anymore whether to use “type” or “typeKeys”.
- Download PhantomJS from here and extract it.
- Add the folder to the system path and run phantomjs --version in a new command prompt. You should see the actual version (just to verify there are no errors so far).
- On Linux (Ubuntu) - command line:
cd /usr/local/share wget http://phantomjs.googlecode.com/files/phantomjs-1.8.1-linux-x86_64.tar.bz2 tar xjf phantomjs-1.8.1-linux-x86_64.tar.bz2 sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-x86_64/bin/phantomjs /usr/local/share/phantomjs sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-x86_64/bin/phantomjs /usr/local/bin/phantomjs sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-x86_64/bin/phantomjs /usr/bin/phantomjs
It is important that you have installed version 1.8*, because GhostDriver is not integrated into PhantomJS’s older versions.
- Start GhostDriver by setting phantomjs --webdriver=PORT, where PORT can be any available port. You should see a successful message, indicating that GhostDriver is running on port PORT.
- Configure your test environment to use PhantomJS:
For example, create a folder structure like:
- headless_testing, containing a subfolder test, where I will add my selenium scripts written with Python
Add the following two files inside test folder:
- base_test.py (but delete the first two lines - there's no need for these as we will place all py files in the same folder):
Modify the base_test.py file as follows:
- replace port 8080 with the port where GhostDriver is running for you (line 59)
- replace open() with open() (line 40)
- Add your own selenium python script to the same test folder. Make sure your methods are named as test_*, so that you can run them using unittest module.
Add a config.ini file in the test folder, containing
driver=phantomjs.In this way, you’re telling Selenium it should use the headless PhantomJS browser. If you don't want your config.ini file to be mixed up with the python scripts, you could add it to a separate folder, and modify the base_test.py file accordingly.
- pip install discover ( the test discovery mechanism and load_tests protocol for unittest backported from Python 2.7 to work with Python 2.4 or more recent (including Python 3).)
python -m unittest discover -s test --pattern=test*.py
- Log in to Travis with GitHub and enable service hooks for the repositories you want added to Travis. More details here.
- On your local machine, drill inside headless_testing folder and add a file called .travis.yml - this is the config file that will tell Travis how to run your selenium python scripts:
language: python python: - "2.7" install: "pip install -r requirements.txt --use-mirrors" script: - "python -m unittest discover -s test --pattern=test*.py" before_script: - "python test/start_ghostdriver.py"
- Add this to the requirements.txt file (separate lines): selenium==2.28.0 and discover==0.4.0
- The start_ghostdriver.py script is a script I created in order to be able to start GhostDriver and get the control back (so that you are able to run other commands from the command line as well). The content of this script is:
import os #os.system('phantomjs --webdriver=PORT 2>&1 > /dev/null &')
- Travis comes with PhantomJS configured, so no other action is needed in this direction.
- Make sure to add the start_ghostdriver.py file inside headless_testing/test folder. What it does is that it starts a process in the background and it captures the output in the log.txt file. Adding the & ensures the fact that you will gain control back in the command line. Only two lines will be written inside the log file, so you shouldn’t worry about it too much.
- Push the content of the headless_testing folder to GitHub (you might want to use a .gitignore file as well).
on Feb 01, 2013 by Danu
When running a functional test, you fire up “browser” and do the "same" actions as a real user (or API client). There are different “browsers” for testing your applications, some of them are real, like selenium and some of them are less real, like django test client. Depending on the context, each of them has its pros and cons.
- selenium is a browser-driving library, opens a real browser and tests rendered HTML alongside with behavior of Web pages. Write selenium tests for Ajax, other JS/server interactions.
A complete test suite should contain both test types.
Functional tests and Django
Write functional/integration/system (has more names than it needs) tests for views. Unit testing the view is hard because views have many dependencies (templates, db, middleware, url routing etc).
You should definitely have django functional tests but chances are you should have fewer than you have now. See the software testing pyramid by Alister Scott which provides solid approach to automated testing and shows the mix of testing a team should aim for.
- test that the whole integrated system works, catch regressions
- they tend to be slow
- will catch bugs that unit tests will not, but it's harder to debug
- write fewer (more unit tests)
So what can you test with django-test client ?
- the correct view is executed for a given url
- simulate post, get, head, put etc. requests
- the returned content has the expected values (you can use beautiful soup, lxml or html5lib for parsing the content)
Example of functional test with lettuce and django test client
Feature: Register In order to get access to app A user should be able to register Scenario: User registers Given I go to the "/register/" URL | username | email | password1 | password2 | | danul | firstname.lastname@example.org | test123 | test123 | And I submit the data Then I should see "Check your email" And I should receive an email at "email@example.com" with the subject "Activate your djangoproject.com account - you have 7 days!" And I activate the account Then I should see "Congratulations!" Scenario: Users login Given following users exist | username | password | | danu | test123 | | lulu | test123 | When I go to the "/login/" URL And I login as user "danu" Then I should see "Welcome, danu. Thanks for logging in."
import re from bs4 import BeautifulSoup from lettuce import * from django.core import mail from nose.tools import assert_equals from django.contrib.auth.models import User response = world.html = for data in step.hashes: world.data = data world.response = html = expected_text = . activation_url = world.response = for user_hash in step.hashes: user_data = password = user, created = if not . : world.response =
import logging from django.conf import settings from django.core.management import call_command from django.test import Client from django.db import connection #from django.test.utils import setup_test_enviroment, teardown_test_environment from lettuce import * world.browser = try: from south.management.commands import patch_for_test_db_setup except ImportError: pass # Setup test environment / called by harvest.py #setup_test_enviroment() world.testdb = # Tear Down the test environment / called by harvest.py #teardown_test_enviroment()
What's next ?
Be a good person and write functional tests. Functional testing is something that every app needs, no testing strategy is complete without high-level tests to ensure the entire programming system works together.
Source code for this article:
on Jan 14, 2013 by Danu
We’ll use Firefox by default since it has webdriver support built-in. If you want to use Chrome you’ll need to download the chrome driver separately.
First let's extend django default settings with lettuce specific configurations. This way we avoid running tests against dev db and declare another db to run tests against. This is actually bad, the lack of test database integration with django and the fact that lettuce server is used over LiveServerTestCase may cause reliability issues in the future (see django test database). Add
lettuce.django to INSTALLED_APPS. You can also specify the app names in LETTUCE_APPS to run tests against, so that you won’t need to type the app name as command-line parameters.
from settings import * DATABASES = INSTALLED_APPS += ( 'lettuce.django', ) LETTUCE_APPS = ( 'registerap', )
Lettuce will look for a features folder inside every installed app. For each app you want to create lettuce tests you have to create a sub folder called features. Our features are stored under
registerapp folder. The "feature" file contains scenarios, while ".py" file contains python code executed for steps used in scenarios.
Feature: Register In order to get access to app A user should be able to register Scenario: User registers Given I go to the "/register" URL When I fill in "username" with "danul" And I fill in "email" with "firstname.lastname@example.org" And I fill in "password1" with "test123" And I fill in "password2" with "test123" And I press "submit" Then I should see "Check your email" And I should receive an email at "email@example.com" with the subject "Activate your djangoproject.com account - you have 7 days!" And I activate the account Then I should see "Congratulations!" Scenario: User logs in successfully Given I go to the "/login" URL When I fill in "username" with "danul" And I fill in "password" with "test123" And I press "submit" Then I should see "Welcome, danul. Thanks for logging in."
We are importing
django_url utility function so we can easily access web pages within steps using relative urls. We also importing the email backend that sends mails to a multiprocessing queue. Because Django server runs in a different process than lettuce you are not able to use the
EMAIL_BACKEND specified in settings file, not to mention that
harvest.py is using
django.test.utils.setup_test_environment that overrides email backend to
import re from lettuce import step, world from lettuce.django import django_url from lettuce.django import mail from nose.tools import assert_equals @step(u'I go to the "(.*)" URL') : world.response = world.browser.get(django_url(url)) @step(u'I fill in "(.*)" with "(.*)"') : world.browser.find_element_by_name(field).send_keys(value) @step(u'I press "(.*)"') : button = world.browser.find_element_by_id(button_id) button.click() @step(u'I should see "(.*)"') : h1 = world.browser.find_element_by_tag_name('h1') assert_equals(h1.text, expected_response) @step(u'And I should receive an email at "([^"]*)" with the subject "([^"]*)"') : message = mail.queue.get(True, timeout=5) world.email_body = message.body assert_equals(message.subject, subject) assert_equals(message.recipients(), [address]) @step(u'And I activate the account') : activation_url = re.findall( r'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', world.email_body ) world.response = world.browser.get(activation_url)
Its about setup and teardown. We setup the browser instance using webdriver. We prepare the test environment by running the django
syncdb command which creates the database tables as well loading any fixtures named
initial_data. Teardown test environment by destroying the test database. Different hooks and commands can be used, for e.g: before each scenario load specific fixtures using
from django.core.management import call_command from django.db import connection from django.conf import settings from lettuce import before, after, world from selenium import webdriver world.browser =
Running lettuce tests
All that’s left is running the tests:
python manage.py harvest tests/lettuce_selenium_tests --settings=conf.settings_lettuce
Source code for this project can be found on github: