EdenTest

Testing is not about finding and fixing defects in the code - it's about preventing them from being introduced into the code the first place.

EdenTest is a Robot Framework based test framework used for automated testing in Sahana Eden. It is located in the eden codebase under directory eden/tests.

About Robot Framework

​Robot Framework is a Python-based, extensible keyword-driven (keywords are methods in Robot Framework) test automation framework used for testing. It provides many features that make writing tests, reading their results and finding errors hassle free. Some of its key features are:

• Enables easy-to-use tabular syntax for creating test cases in a uniform way.
• Provides ability to create reusable higher-level keywords from the existing keywords.
• Provides easy-to-read result reports and logs in HTML format.
• Is platform and application independent.

Installation guidelines

EdenTest requires three python packages.

• ​robotframework
• ​robotframework-seleniumlibrary
• ​robotframework-databaselibrary
• ​Python Requests

All four packages mentioned above are available with two most commonly used python package installation tools, namely, ​pip and ​easy_install.

Installation using pip

pip install robotframework
pip install robotframework-seleniumlibrary
pip install robotframework-databaselibrary
pip install requests

Installation using easy_install

easy_install robotframework
easy_install robotframework-seleniumlibrary
easy_install robotframework-databaselibrary
easy_install requests

If you have successfully installed the packages, you should be able to run EdenTest. If for some reason, you can not install the packages using either of the two methods mentioned above, please visit the three links given above. Manual install instructions for each package are given there.

You can also run the script edentest_setup.py inside /tests/ to install all the dependencies.

Directory structure

Before running EdenTest, it is important to have a mental picture of the directory structure of EdenTest and the importance of its sub-directories.

/tests
├── docs
│   ├── auth.html
│   │   ...
│   ├── main.html
│   └── widgets.html
├── execution
│   ├── libs
│   │   ├── edentest_database_local.py
│   │   └── edentest_robot.py
│   └── settings
│       ├── config.example.py
│       └── config.py
│
└── implementation
    ├── libs
    ├── resources
    │   ├── auth.robot
    │   │   ...
    │   ├── main.robot
    │   └── widgets.robot
    └── testsuites
        ├── __init__.robot
        │
        ├── hrm
        ├── org
        ├── project
        ├── ...
        │
        └── smoke_tests

We distinguish between the implementation and the execution of the tests.

• docs: It contains the documentation of keywords in html format. The name of the file is same as the keyword file present in resources sub-directory. The documentation is extracted from the text written next to the [Documentation] setting below the keyword definition.
• execution: Execution directory contains two sub folders, settings and libs. settings contains the settings local to your system to run the tests. It has a file config.example.py using which config.py file is created. config.py is not under version control. It has settings variables like server address, valid username, valid password etc. libs contains the python library files that created for EdenTest eg: edentest_database.py. It might be also used to store ready-made libraries to ensure they are kept stable for example the selenium2library.
• implementation: It contains the implementation aspects.
  • resources: It is imperative from a test-design point of view to have higher-level keywords. These are kept in the resource files. The keywords are stored in a file as per the function they serve for eg: Keyword Login To Eden is kept in auth.robot.
    • main.robot: It is the center resource file. All the resource files like auth.robot, settings like config.py are imported in this file. Further, this file is imported in the tests.
  • testsuites: Testsuites directory contains all the different testsuites differentiated based on the basis of functionality they test eg: hrm. Inside each testcase, there is at least one .robot file which contains the testcases.

Using EdenTest

Start Sahana Eden

1. Make sure your have Sahana Eden running.

2. Populate the database

Delete the contents of databases/* , errors/* and sessions/* in your eden directory.

E.g. for Linux, cd to your eden directory and do:

 rm databases/* sessions/* errors/*

On Windows, use Explorer to delete the contents of the eden\databases, eden\errors, and eden\sessions folders.

To re-populate the database, for either Linux or Windows (using a command prompt window), cd to the web2py directory, and do:

python web2py.py -S eden -M -R applications/eden/static/scripts/tools/noop.py

Creating config.py

Before running the tests, there has to exist a config.py file inside the execution directory. It contains the various settings variables that are imported by EdenTest. Copy the config.example.py into config.py by [assuming the current working directory is /tests/]

cp execution/settings/config.example.py execution/settings/config.py

Edit the config.py file according to your requirements.

Changes made to the config.example.py in the main branch would not be reflected in your config.py file locally. So, a possible point of error could that you have a stale config.py file. Check ​config.example.py to see if you have an updated file.

Running EdenTest

EdenTest testcases are run from the command line. They are run from inside web2py. The command used to run the tests is python web2py.py --no-banner -M -S eden -R applications/eden/tests/edentest.py -A <name of the testsuite>. All the command line arguments to EdenTest are given after -A argument. Give it --help to know about more command line options.

The end result is, by default, an ​output file in XML format and an HTML ​report and a ​log. These three files are created in the current working directory by default.

• report.html: Report files contain an overview of the test execution results in HTML format. They have statistics based on tags and executed test suites, as well as a list of all executed test cases. When both reports and logs are generated, the report has links to the log file for easy navigation to more detailed information. It is easy to see the overall test execution status from report, because its background color is green, if all critical tests pass, and bright red otherwise.

• log.html: Log files contain details about the executed test cases in HTML format. They have a hierarchical structure showing test suite, test case and keyword details. Log files are needed nearly every time when test results are to be investigated in detail. Even though log files also have statistics, reports are better for getting an higher-level overview.
• output.xml: Output files contain all the test execution results in machine readable XML format.

Some common usages

[Assuming the current working directory is the location of web2py]

• Run all the testsuites

  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py #no argument
  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A testsuites #testsuites folder as argument
  

This will run all the testsuites found under the testsuites folder.

• Run a specific testsuite file. Just give the name of the file present inside the testsuites folder as an argument. eg: hrm

  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A hrm
  

• Change the execution speed of selenium

  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A hrm --variable DELAY:10 
  

Similarly, other variables like BASEURL, HOMEPAGE etc can be manipulated from the command line. -V flag can be used to give a path of a file containing a list of variables. The format of the file should be similar to config.py.

• Run a specific testcase

  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A org -t Create_Organisation 
  

-t flag is used to select the testcase.

• Run a specific testsuite

  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A hrm
  

  -s flag is disabled in EdenTest

• Change the output directory

  python web2py.py --no-banner -M -S eden  -R applications/eden/tests/edentest.py -A hrm -d [path to the directory] 
  

To read the results after running the tests, open report.html in your browser.

Writing a testcase

Please see the tutorial How to write a testcase for EdenTest?

For more advanced usage, please refer to Advanced Test Design.