Writing testcases for EdenTest

The F.I.R.S.T principle of Test Driven Development

• Fast: run (subset of) tests quickly (since you'll be running them all the time)
• Independent: no tests depend on others, so can run any subset in any order
• Repeatable: run N times, get same result (to help isolate bugs and enable automation)
• Self-checking: test can automatically detect if passed (no human checking of output)
• Timely: written before writing the code

EdenTest has been written using these principles. Whenever using EdenTest to create a new testcase, it is imperative to make sure that F.I.R.S.T mantra is followed.

How to write good testcases?

Before beginning to write a testcase, please go through the article ​How to write good testcases for Robot Framework?. It is a short and descriptive article that will help you a great deal to write precise and useful testcases.

Writing a testcase for EdenTest

Before writing your first testcase, please make sure that you have EdenTest set up, you understand the directory structure and the know-how of how to run EdenTest and see its results. If not, go through EdenTest.

As EdenTest is built to make it easy to do ATDD which is a practice in which the acceptance criteria is decided, with examples, and then distilled into a set of concrete acceptance tests before development begins. To say it simply, ATDD is like:

1. Define (or refactor) acceptance test(s) from user stories
2. Automate the acceptance test
3. Run the acceptance test
4. If fail: refactor code, then repeat 3. If pass: development stops, back to 1.

What we have to do?

Let us assume you have to implement a simple login form. There are a few possible cases:

1. Login is successful
2. Login is unsuccessful. Because -
   • Invalid email
   • Invalid password
   • Both are invalid
   • Data entered fails validation check (for eg: empty fields etc)

Let us write the testcase for the first case i.e. Login is successful. What is expected when the login will be successful:

• Browser is opened to the login page
• Valid email and valid password is entered
• Submit button is clicked
• A message Logged in shows up confirming the user is logged in.

Let us translate this user story into the testcase. (and marvel at the beauty of Robot Framework as it is so very easy. :D)

Writing the testcase

Follow these steps:

• Create a file named login_functionality.robot inside the folder /tests/implementation/testsuites.
• Write the testcase below in it

*** Settings ***
Resource  ../resources/main.robot


*** Test Cases ***
Login with valid email and valid passwd should be successful
   #opens the browser to the argument url
    Go To  http://${SERVER}/eden/default/user/login
    #puts arg2 in textbox(arg1)
    Input Text  auth_user_email  admin@example.com
    Input Text  auth_user_password  testing
    #Clicks the submit button
    Click Button  xpath=//input[@class='btn' and @value='Login']
    #Checks if the text 'Logged in' is there or not
    Page Should Contain  Logged in

Note: Please make sure that the various arguments are separated by two spaces.

• Run the testsuite login_functionality.robot. (Using EdenTest)

python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A login_functionality 

• Open report.html to see the outcome.

Understanding the testcase

• The testcase is in a separate file and has two sections - *** Setttings *** and *** Test Cases *** itself.
• As you can see, the name of the testcase is very descriptive and describes the objective.
• Open Browser, Input Text, Click Button and Page Should Contain are all keywords (functions) implemented in Selenium2Library.
• Every testcase and a suite has a setup and teardown. The common setup/teardown function is defined in the file main.txt file. It is run by default on every test defined under testsuites folder. One can create test setup/teardown as well by setting them in the *** Settings *** section.

The beauty of writing tests this way is that they are easy to read and understand and that is one of aims as well, that even someone without the technical know-how can read and write tests.

If you have understood how the above above testcase is written, try writing the testcase Login with invalid email and valid passwd should fail. It should be fairly simple.

To understand the selenium keywords visit ​Selenium2Library Keyword Documentation and the documentation of keywords implemented under EdenTest in tests/docs.

Features of EdenTest

Settings inspection

You can fetch deployment settings from the server (only if you have admin permissions for that server) by doing the following:

    ${deployment settings} =  Get Deployment Settings  template
    ${TEMPLATE}=  Get From Dictionary  ${deployment settings}  template

For this to work, there should a function defined as get_template() defined in s3cfg.py

Database Inspection

You can do database inspection inside EdenTest (only for local servers). Keywords doing database querying, assertions etc should be defined in edentest_database_local.py. There are already a few helper keywords defined there to make it easy to do querying. Eg:

 Row count is equals x  (s3db.asset_asset.id > 25)  16

This would fail/pass depending wether the query is true or false.

Tests Repeatability

Tests are made repeatable by executing the following steps in order:

1. A copy of the database is made during the test build as the original database might change while the test runs.
2. The original database is restored from the copy during the test tear down, thus making the tests repeatable.

Note: The test repeatability flag is off by default and can be turned on in the 000_config.py file. Also, the repeatability feature is currently only implemented for sqlite and postgres database.

Deliver traceback of internal error pages

If there is a ticket generated, EdenTest gives back the traceback inside the logs and shows it on the console. If there is no traceback generated a screenshot is taken instead of the error and stored in the report.

Advanced Test Design

Please see the article Advanced Test Design.