QuickStart

If you just want to use flunc, the quickest way to install it is using easy-install:

easy_install flunc 

virtualenv myflunc
cd myflunc
source bin/activate
easy_install flunc

If you want to develop flunc itself, you can install from source:

virtualenv myflunc
source myflunc/bin/activate
mkdir myflunc/src
cd myflunc/src
svn co https://svn.openplans.org/svn/flunc/trunk flunc
cd flunc
python ./setup.py develop

Example Flunc tests

Examples of tests and suites can be found at https://svn.openplans.org/svn/opencore/trunk/ftests

Tests, Suites and Configurations

Flunc uses browser simulation to do rudimentary functional testing over the web using Twill scripts chained together by a simple harness. The testing harness understands 3 things: tests, suites and configurations.

Tests

Tests are just simple twill scripts. The tester recognizes files with the extension '.twill' as tests

go ${base_url}/login_form
fv login_form __ac_name $user
fv login_form __ac_password $password
submit submit
find 'You are now logged in'

Suites

Suites may simply be newline separated lists of test names (and possibly suite names). When run, the tests listed are run in the order specified. The tester recognizes files with the extension '.tsuite' as suites.

create_user
login
create_project
edit_project
logout
destroy_project
destroy_user

Configurations

Configurations are also twill scripts, but by convention they contain only statements which set the value of variables used in tests. The tester recognizes files with the extension '.conf' as configurations.

setglobal user      'tester'
setglobal password  'tester1'

Stringing Things Together

The test runner will run suites and tests in the order specified on the command line. When a suite is encountered, if a configuration with a corresponding name is found, the configuration is loaded, eg all.tsuite and all.conf.

In addition, a configuration file may be specified on the command line using the -c option. This configuration will partially override any suite configurations that are loaded, eg this configuration might only override $user.

Flunc only looks for configuration files for suites, not for individual twill files.

Example Usage

$ flunc --help

show available command line options

$ flunc -l

list test suites

$ flunc all

run the suite 'all.tsuite' against the default host using the default configuration

$ flunc -c basic create_user login create_project destroy_project destroy_user

run the specified tests specified in order using 'basic.conf' for configuration

$ flunc -t http://localhost:8080/some_portal all

run 'all.tsuite' against the portal specified (set base_url to the url given)

Creating New Tests

Since simple tests are just twill scripts, all you need to do to create a new test is create a file that ends in .twill and put in a few twill commands.

By default flunc will look for tests in the ftests folder within the flunc egg. You can run tests in the current
directory with the -p. command line flag.

The following asserts that we can load http://www.google.com.

Often when creating tests, it is helpful to abstract out specific test data into twill variables. This includes data like:
username, password, base url. The goal is to make it easy to run the tests in an slightly different context.

Builtin Variables

Flunc always defines a handfull of twill variables which are often useful in writing retargetable tests.

$ echo "go \${base_url}" > go_somewhere.twill
$ echo "url \${base_url}" >> go_somewhere.twill
$ flunc -p. go_somewhere -t http://www.yahoo.com

[*] scanning for tests in '.'
[*] Running against http://www.yahoo.com, host: www.yahoo.com path=
[*] running test: go_somewhere
==> at http://www.yahoo.com
[*] All Tests Passed!

Debugging Tests

There are some useful command line options that aid while debugging tests

-i, --interactive-debug

Fall into twill shell on error

-d DUMP_FILE, --dump-html=DUMP_FILE

dump current HTML to file specified on error. specify - for stdout.

-w, --show-error-in-browser

show dumped HTML in a web browser on error, forces interactive mode

-v, --verbose

Display stack traces from twill

-F, --ignore-failures

continue running tests after failures

-C, --cleanup-only

run only the suite cleanup and not the tests

-X, --no-cleanup

do not run cleanup handlers for suites

Various combinations of these options can allow of interesting possibilities. Here are some useful examples:

$ flunc -i all

Drop into an interactive shell on the first failure. The twill shell provides excellent commands to help debug the problem. Some useful ones to start off with are: showforms, showlinks, show, save_html, and find.

$ flunc -w all

Drop into an interactive shell on the first failure and display the current page in a web browser

$ flunc all -wF

This command will launch a web browser with the page it failed with on error, as well as drop you into an interactive twill shell. You can fix the issue in the twill script, while verifying visually what went wrong through your browser. After making the correction, the runner will continue execution of the remaining suites starting with the next test.

$ flunc -C basic

Runs the cleanup for the basic suite without executing any of the tests

Advanced Usage

Test Arguments

It is sometimes useful to provide arguments to a test inside a suite to avoid repeating a series of similar steps. To support this, suites support a simple argument passing syntax to override variables in a test. The assignments apply only during the execution of the particular test. Values may be strings or refer to global twill variables which have been assigned in configurations. n.b this syntax currently has no meaning when applied to suites.

login(user='joe')
do_something # as_joe
logout

login(user=${admin_user})
do_something_fancy(blah=${somearg}, foo='joop') # as admin
logout

Cleanup Handlers

flunc will recognize suites which end in '_cleanup' as cleanup handlers, eg 'policy_cleanup.tsuite' as the cleanup handler for 'policy.tsuite'. Cleanup handlers will always run after a suite regardless of failure. In interactive mode, cleanup handlers are not run until the user has exited the twill shell.