Continuous Integration

Current Setup

CI Server's cron runs the tests every 24 hours

• Using Web2Py from Tue Jun 11 08:00:48 - Version 2.5.1-stable+timestamp.2013.06.11.08.00.05
• Using latest Trunk Eden
• Using IFRC, default, SandyRelief, DRMP, CRMT templates
• Using SQLite
• Using Chrome

Results are stored as HTML files in the filesystem which can be accessed via the Eden instance under test:

• ​http://82.71.213.53/RESULTS

• Also, the results are being mailed to the mailing list - ​Sahana Eden CI Server.
• You can simply follow the above link and click on "Join group to post" to join the mailing list.
• The CI Server sends out a daily test report email to this list.
• You can manage your subscription by clicking on "My Settings" tab on the top right corner.

/usr/local/bin/testsuite

• Updates Web2Py to current Trunk (from github.com/web2py/web2py)
  • Currently disabled
• Updates Eden to current Trunk (from github.com/flavour/eden)
• Cleans database & prepopulates
• Runs Functional Tests (Selenium)
• Runs Smoke Tests
• Runs Roles Tests
  • Currently disabled
• Mails out test results (using /usr/local/bin/mailTestResults.sh)
• The code for /usr/local/bin/testsuite and /usr/local/bin/mailTestResults.sh can be found in the attachments.

NB Tests require Python 2.7, so Pythonbrew is used to allow this to coexist with the system Python 2.6

To run the test suite manually, run:

/usr/local/bin/testsuite

To just run smoke tests:

/usr/local/bin/web2pystarter
cd /home/web2py
xvfb-run -a python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --html-path /home/web2py/applications/eden/static/test_smoke/ --suite smoke --force-debug --link-depth 16 -V 4

Start Window based applications (x11):

• If the GUI window of the application is of no interest to you:

  xvfb-run -a *test commands here*
  

  For example :

  xvfb-run -a python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A -V 2
  

  Note: Xvfb basically makes a virtual display which isn't displayed anywhere. Xserver itself actually requires a real display
• If you want to see the GUI interface of the application
  • If the machine you use to ssh into the server, has X Server installed, then you can access the GUI window by using the -X flag when loggin into the server

    ssh -X youraccount@82.71.213.53
    

    And now, if you simply execute any application which has a Xserver attached to it, it will be displayed on your screen.

Plans for Future

• Move mailing script out of a personal user directory
• Move cron commands to system /etc/crontab rather than root's crontab
• Fix Reporting of Smoke Tests
• Enable Roles Tests & fix any issues which caused it to be disabled
• Run Unit Tests
• More templates (default, etc)
• More databases
• More browsers
• More web2py versions
  • e.g. 'Current Stable' (as-defined by Eden) & 'Trunk'
  • having errors in Trunk Web2pY which aren't seen in 'Current Stable' homes us in on where problems lie
• Install an Eden instance to act as a CI server
  • Move the cron scripts to being run by Eden Scheduler
  • Have Eden manage the mailing of results & manage subscriptions
  • Have Eden store the results in the database for Graphical analysis
  • Same Eden instance can be Sunflower which replaces Trac for tickets
  • Have Eden analyse results intelligently - e.g. logging tickets for Web2Py issues when web2py version is seen as the cause
  • AUT should be possible on remote servers
    • e.g. Spawn AWS instances using ​https://github.com/flavour/spawn-eden
      • AMIs built for Cherokee/PostgreSQL & Apache/MySQL to speed-up bootstrapping of instances
    • Can do Performance Testing
    • Can do Tests on servers with User Data?
  • This replaces the old install of Jenkins

Setup Notes & Discussions (to be ported to Wiki):

• ​https://docs.google.com/document/d/1T29cF3kkK4lIAVdiATfE8a-tLm_Tyh3_dIh6FJWY7sU/edit