Changes between Version 1 and Version 2 of DeveloperGuidelines/Testing/Load


Ignore:
Timestamp:
09/26/11 12:16:28 (13 years ago)
Author:
Fran Boon
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • DeveloperGuidelines/Testing/Load

    v1 v2  
    55
    66Testing should normally be done from a separate machine. [InstallationGuidelines/Amazon Amazon] is great as a platform for accessing a lof of resources for a short period of time.
     7 * A 'Micro' instance is sufficient for testing 100 concurrent users (the dynamic parts of the site)
     8 * A 'High CPU' instance is required for testing 6,500 concurrent users (the static parts of the site)
     9
     10== Server Setup ==
     11Tsung can include results on Server Load if SNMP is enabled:
     12{{{
     13apt-get install -y snmpd
     14vim /etc/snmp/snmpd.conf
     15#agentAddress  udp:127.0.0.1:161
     16agentAddress udp:161,udp6:[::1]:161
     17#rocommunity public  default    -V systemonly       
     18rocommunity public 0.0.0.0/0
     19
     20/etc/init.d/snmpd restart
     21}}}
     22
     23Notes:
     24 * Log files can fill-up quickly, especially when testing a large number of users for the static parts of the site
     25 * Database will need resetting to clear any transactions made
     26 * Reboot between tests to reset the RAM/Swap
     27
     28== Client Setup ==
     29
     30=== Installation ===
     31{{{
     32apt-get install -y erlang gnuplot libtemplate-perl make lrzsz vim
     33wget http://tsung.erlang-projects.org/dist/tsung-1.4.1.tar.gz
     34tar zxvf tsung-1.4.1.tar.gz
     35cd tsung-1.4.1
     36./configure
     37make
     38make install
     39ssh localhost
     40yes to accept hostkey
     41exit
     42}}}
     43
     44=== Static Tests ===
     45It is recommended to make the homepage of the site static. The performance of this page can be tested using {{{~/.tsung/tsung.xml}}}:
     46{{{
     47<?xml version="1.0"?>
     48<!DOCTYPE tsung SYSTEM "/usr/share/tsung/tsung-1.0.dtd">
     49<tsung loglevel="notice" version="1.0">
     50
     51  <!-- Client side setup -->
     52  <clients>
     53    <client host="localhost" cpu="8"/>
     54  </clients>
     55 
     56  <!-- Server side setup -->
     57<servers>
     58  <server host="10.171.46.7" port="80" type="tcp"></server>
     59</servers>
     60
     61  <!-- to start OS monitoring (cpu, network, memory) -->
     62  <monitoring>
     63    <monitor host="10.171.46.7" type="snmp"></monitor>
     64  </monitoring>
     65 
     66  <load duration="30" unit="minute">
     67   <arrivalphase phase="1" duration="30" unit="minute">
     68     <users maxnumber="10000" arrivalrate="10" unit="second"></users>
     69   </arrivalphase>
     70   <!-- Will run fully-loaded for 13 minutes -->
     71  </load>
     72
     73  <options>
     74   <option type="ts_http" name="user_agent">
     75    <user_agent probability="80">Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Galeon/1.3.21</user_agent>
     76    <user_agent probability="20">Mozilla/5.0 (Windows; U; Windows NT 5.2; fr-FR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4</user_agent>
     77   </option>
     78  </options>
     79
     80  <!-- start a session for a http user. the probability is the
     81  frequency of this type of session. The sum of all session's
     82  probabilities must be 100 -->
     83
     84 <sessions>
     85  <session name="static-homepage" probability="100" type="ts_http">
     86
     87    <!-- Since we use maxnumber, we need each client to loop indefinitely to sustain the concurrency -->
     88    <for from="1" to="1000" var="i">
     89        <!-- full url with server name, this overrides the "server" config value -->
     90
     91        <!-- Visit Home page -->
     92        <request> <http url="/" method="GET" version="1.1"></http> </request>
     93        <!-- Doesn't follow the 301 -->
     94        <request> <http url="/eden/static/index.html" method="GET" version="1.1"></http> </request>
     95        <request> <http url="/eden/static/styles/S3/sahana.min.css" method="GET" version="1.1"></http> </request>
     96        <!-- Moved to CDN
     97        <request> <http url="/eden/static/scripts/web2py/jquery-1.6.2.min.js" method="GET" version="1.1"></http> </request>
     98        -->
     99        <request> <http url="/eden/static/img/la/cityofla_logo.png" method="GET" version="1.1"></http> </request>
     100        <request> <http url="/eden/static/img/la/logo.png" method="GET" version="1.1"></http> </request>
     101        <request> <http url="/eden/static/img/la/mayor_antonio_vilaraigosa.jpg" method="GET" version="1.1"></http> </request>
     102        <request> <http url="/eden/static/img/la/la_seal.png" method="GET" version="1.1"></http> </request>
     103        <request> <http url="/eden/static/img/sahanasmall_05.png" method="GET" version="1.1"></http> </request>
     104        <request> <http url="/eden/static/img/la/bg.png" method="GET" version="1.1"></http> </request>
     105        <request> <http url="/eden/static/img/la/footer_end.png" method="GET" version="1.1"></http> </request>
     106        <request> <http url="/eden/static/img/la/header_bg.png" method="GET" version="1.1"></http> </request>
     107        <request> <http url="/eden/static/img/la/nav_divider.png" method="GET" version="1.1"></http> </request>
     108        <request> <http url="/eden/static/img/la/open_quote.png" method="GET" version="1.1"></http> </request>
     109        <request> <http url="/eden/static/img/la/close_quote.png" method="GET" version="1.1"></http> </request>
     110        <request> <http url="/eden/static/img/la/donate_home.jpg" method="GET" version="1.1"></http> </request>
     111        <request> <http url="/eden/static/img/la/paper_corner.png" method="GET" version="1.1"></http> </request>
     112        <request> <http url="/eden/static/img/la/bottom_paper_shadow.png" method="GET" version="1.1"></http> </request>
     113        <request> <http url="/eden/static/img/la/arrow_bullet_large.png" method="GET" version="1.1"></http> </request>
     114        <request> <http url="/eden/static/img/la/volunteer_home.jpg" method="GET" version="1.1"></http> </request>
     115        <request> <http url="/eden/static/img/la/paper_corner_right.png" method="GET" version="1.1"></http> </request>
     116
     117        <thinktime value="10" random="true"></thinktime>
     118
     119    </for>
     120
     121  </session>
     122 </sessions>
     123</tsung>
     124}}}
     125
     126=== Dynamic Tests ===
     127A set of transactions can be performed with an increasing level of load {{{~/.tsung/tsung.xml}}}:
     128{{{
     129<?xml version="1.0"?>
     130<!DOCTYPE tsung SYSTEM "/usr/share/tsung/tsung-1.0.dtd">
     131<tsung loglevel="notice" version="1.0">
     132
     133  <!-- Client side setup -->
     134  <clients>
     135    <client host="localhost" use_controller_vm="true"/>
     136  </clients>
     137 
     138  <!-- Server side setup -->
     139<servers>
     140  <server host="10.171.46.7" port="80" type="tcp"></server>
     141</servers>
     142
     143  <!-- to start OS monitoring (cpu, network, memory) -->
     144  <monitoring>
     145    <monitor host="10.171.46.7" type="snmp"></monitor>
     146  </monitoring>
     147 
     148  <load duration="30" unit="minute">
     149   <arrivalphase phase="1" duration="5" unit="minute">
     150     <users maxnumber="5" arrivalrate="1" unit="second"></users>
     151   </arrivalphase>
     152   <arrivalphase phase="2" duration="5" unit="minute">
     153      <!-- Add another 5 to bring the total to 10 -->
     154      <users maxnumber="5" arrivalrate="1" unit="second"></users>
     155   </arrivalphase>
     156   <arrivalphase phase="3" duration="5" unit="minute">
     157      <!-- Add another 15 to bring the total to 25 -->
     158      <users maxnumber="15" arrivalrate="1" unit="second"></users>
     159   </arrivalphase>
     160   <arrivalphase phase="4" duration="5" unit="minute">
     161      <!-- Add another 25 to bring the total to 50 -->
     162      <users maxnumber="25" arrivalrate="1" unit="second"></users>
     163   </arrivalphase>
     164   <arrivalphase phase="5" duration="5" unit="minute">
     165      <!-- Add another 50 to bring the total to 100 -->
     166      <users maxnumber="50" arrivalrate="1" unit="second"></users>
     167   </arrivalphase>
     168   <!-- Will run fully-loaded for 9 minutes -->
     169  </load>
     170
     171  <options>
     172   <option name="file_server" id='userdb' value="/root/.tsung/users.csv"/>
     173   <option type="ts_http" name="user_agent">
     174    <user_agent probability="80">Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Galeon/1.3.21</user_agent>
     175    <user_agent probability="20">Mozilla/5.0 (Windows; U; Windows NT 5.2; fr-FR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4</user_agent>
     176   </option>
     177  </options>
     178
     179  <!-- start a session for a http user. the probability is the
     180  frequency of this type of session. The sum of all session's
     181  probabilities must be 100 -->
     182
     183 <sessions>
     184  <session name="dynamic-with-progressive-load" probability="100" type="ts_http">
     185
     186    <!-- Since we use maxnumber, we need each client to loop indefinitely to sustain the concurrency -->
     187    <for from="1" to="1000" var="i">
     188        <!-- full url with server name, this overrides the "server" config value -->
     189
     190        <!-- Visit Home page -->
     191        <request> <http url="/" method="GET" version="1.1"></http> </request>
     192        <!-- Doesn't follow the 301 -->
     193        <request> <http url="/eden/static/index.html" method="GET" version="1.1"></http> </request>
     194        <request> <http url="/eden/static/styles/S3/sahana.min.css" method="GET" version="1.1"></http> </request>
     195        <!-- Moved to CDN
     196        <request> <http url="/eden/static/scripts/web2py/jquery-1.6.2.min.js" method="GET" version="1.1"></http> </request>
     197        -->
     198        <request> <http url="/eden/static/img/la/cityofla_logo.png" method="GET" version="1.1"></http> </request>
     199        <request> <http url="/eden/static/img/la/logo.png" method="GET" version="1.1"></http> </request>
     200        <request> <http url="/eden/static/img/la/mayor_antonio_vilaraigosa.jpg" method="GET" version="1.1"></http> </request>
     201        <request> <http url="/eden/static/img/la/la_seal.png" method="GET" version="1.1"></http> </request>
     202        <request> <http url="/eden/static/img/sahanasmall_05.png" method="GET" version="1.1"></http> </request>
     203        <request> <http url="/eden/static/img/la/bg.png" method="GET" version="1.1"></http> </request>
     204        <request> <http url="/eden/static/img/la/footer_end.png" method="GET" version="1.1"></http> </request>
     205        <request> <http url="/eden/static/img/la/header_bg.png" method="GET" version="1.1"></http> </request>
     206        <request> <http url="/eden/static/img/la/nav_divider.png" method="GET" version="1.1"></http> </request>
     207        <request> <http url="/eden/static/img/la/open_quote.png" method="GET" version="1.1"></http> </request>
     208        <request> <http url="/eden/static/img/la/close_quote.png" method="GET" version="1.1"></http> </request>
     209        <request> <http url="/eden/static/img/la/donate_home.jpg" method="GET" version="1.1"></http> </request>
     210        <request> <http url="/eden/static/img/la/paper_corner.png" method="GET" version="1.1"></http> </request>
     211        <request> <http url="/eden/static/img/la/bottom_paper_shadow.png" method="GET" version="1.1"></http> </request>
     212        <request> <http url="/eden/static/img/la/arrow_bullet_large.png" method="GET" version="1.1"></http> </request>
     213        <request> <http url="/eden/static/img/la/volunteer_home.jpg" method="GET" version="1.1"></http> </request>
     214        <request> <http url="/eden/static/img/la/paper_corner_right.png" method="GET" version="1.1"></http> </request>
     215
     216        <thinktime value="10" random="true"></thinktime>
     217
     218        <!-- Visit Donate page -->
     219        <request> <http url="/eden/don/index" method="GET" version="1.1"></http> </request>
     220        <request> <http url="/eden/static/scripts/ext/resources/css/ext-theme.min.css" method="GET" version="1.1"></http> </request>
     221        <request> <http url="/eden/static/scripts/S3/S3.min.js" method="GET" version="1.1"></http> </request>
     222        <request> <http url="/eden/static/scripts/ext/src/locale/ext-lang-en.js" method="GET" version="1.1"></http> </request>
     223        <request> <http url="/eden/static/scripts/S3/jquery.hoverIntent.minified.js" method="GET" version="1.1"></http> </request>
     224        <request> <http url="/eden/static/img/la/logo-laepf.png" method="GET" version="1.1"></http> </request>
     225        <request> <http url="/eden/static/img/la/logo_arc.png" method="GET" version="1.1"></http> </request>
     226        <request> <http url="/eden/static/img/la/logo_wvi.png" method="GET" version="1.1"></http> </request>
     227        <request> <http url="/eden/static/img/la/logo_sa.png" method="GET" version="1.1"></http> </request>
     228        <request> <http url="/eden/static/img/la/corner_box.png" method="GET" version="1.1"></http> </request>
     229        <request> <http url="/eden/static/img/la/arrow_bullet_medium.png" method="GET" version="1.1"></http> </request>
     230        <request> <http url="/eden/static/img/la/donate_donate.jpg" method="GET" version="1.1"></http> </request>
     231        <request> <http url="/eden/static/img/la/volunteer_donate.jpg" method="GET" version="1.1"></http> </request>
     232        <request> <http url="/eden/static/img/colorbox/controls.png" method="GET" version="1.1"></http> </request>
     233        <request> <http url="/eden/static/img/colorbox/border.png" method="GET" version="1.1"></http> </request>
     234        <request> <http url="/eden/static/img/colorbox/loading_background.png" method="GET" version="1.1"></http> </request>
     235        <request> <http url="/eden/static/img/jquery-ui/ui-anim_basic_16x16.gif" method="GET" version="1.1"></http> </request>
     236        <request> <http url="/eden/static/media/closebox.png" method="GET" version="1.1"></http> </request>
     237        <request> <http url="/eden/static/img/la/popup_donate.png" method="GET" version="1.1"></http> </request>
     238        <request> <http url="/eden/static/img/la/popup_donate_bg.png" method="GET" version="1.1"></http> </request>
     239
     240        <thinktime value="10" random="true"></thinktime>
     241
     242        <!-- Visit Volunteer page -->
     243        <request> <http url="/eden/vol/req_skill" method="GET" version="1.1"></http> </request>
     244        <request> <http url="/eden/static/img/la/logo_cert.png" method="GET" version="1.1"></http> </request>
     245        <request> <http url="/eden/static/img/la/logo_dhv.png" method="GET" version="1.1"></http> </request>
     246        <request> <http url="/eden/static/img/la/laworks.png" method="GET" version="1.1"></http> </request>
     247        <request> <http url="/eden/static/img/la/logo_phev.png" method="GET" version="1.1"></http> </request>
     248        <request> <http url="/eden/static/img/la/vcla.png" method="GET" version="1.1"></http> </request>
     249        <request> <http url="/eden/static/img/icon-xls.png" method="GET" version="1.1"></http> </request>
     250        <request> <http url="/eden/static/img/RSS_16.png" method="GET" version="1.1"></http> </request>
     251        <request> <http url="/eden/static/img/kml_icon.png" method="GET" version="1.1"></http> </request>
     252        <request> <http url="/eden/static/scripts/S3/s3.dataTables.min.js" method="GET" version="1.1"></http> </request>
     253        <request> <http url="/eden/static/img/la/sub_nav_shadow.png" method="GET" version="1.1"></http> </request>
     254        <request> <http url="/eden/static/img/la/light_blue_button_end.png" method="GET" version="1.1"></http> </request>
     255        <request> <http url="/eden/static/img/la/light_blue_button.png" method="GET" version="1.1"></http> </request>
     256        <request> <http url="/eden/static/img/la/popup_footer.png" method="GET" version="1.1"></http> </request>
     257        <request> <http url="/eden/static/img/la/popup_header.png" method="GET" version="1.1"></http> </request>
     258        <request> <http url="/eden/static/img/la/external_link.png" method="GET" version="1.1"></http> </request>
     259        <request> <http url="/eden/static/img/la/heading_shadow.png" method="GET" version="1.1"></http> </request>
     260        <request> <http url="/eden/static/img/jquery.dataTables/sort_asc.jpg" method="GET" version="1.1"></http> </request>
     261        <request> <http url="/eden/static/img/jquery.dataTables/sort_both.jpg" method="GET" version="1.1"></http> </request>
     262        <request> <http url="/eden/static/img/la/pagination_sprite.png" method="GET" version="1.1"></http> </request>
     263        <request> <http url="/eden/static/img/la/apply_arrow_head.png" method="GET" version="1.1"></http> </request>
     264        <request> <http url="/eden/static/img/la/apply_arrow_tail.png" method="GET" version="1.1"></http> </request>
     265
     266        <thinktime value="10" random="true"></thinktime>
     267
     268        <!-- Visit Registration page -->
     269        <request>
     270            <!-- Collect the formkey from the registration form (not the sign-in form!) -->
     271            <dyn_variable name="_formkey" xpath="//form[@id='regform']//input[@name='_formkey']/@value" />
     272            <http url="/eden/vol/register" method="GET" version="1.1"></http>
     273        </request>
     274        <request> <http url="/eden/static/scripts/S3/jquery.validate.min.js" method="GET" version="1.1"></http> </request>
     275        <request> <http url="/eden/static/scripts/S3/jquery.pstrength.1.3.min.js" method="GET" version="1.1"></http> </request>
     276        <request> <http url="/eden/static/img/help_off.gif" method="GET" version="1.1"></http> </request>
     277        <request> <http url="/eden/static/img/la/long_red_button.png" method="GET" version="1.1"></http> </request>
     278        <request> <http url="/eden/static/img/jquery.cluetip/wait.gif" method="GET" version="1.1"></http> </request>
     279
     280        <thinktime value="10" random="true"></thinktime>
     281
     282        <!-- Register -->
     283        <setdynvars sourcetype="file" fileid="userdb" delimiter="," order="iter">
     284            <var name="username" />
     285            <!--<var name="password" />-->
     286        </setdynvars>
     287        <request subst="true">
     288            <http url="/eden/vol/register" method="POST" version="1.1" content_type='application/x-www-form-urlencoded'
     289                  contents="_formname=register&amp;_formkey=%%__formkey%%&amp;first_name=Test&amp;middle_name=&amp;last_name=Test&amp;email=%%_username%%&amp;password=eden&amp;password_two=eden&amp;language=en&amp;phone=123456789&amp;phone_type=SMS&amp;address1=1%20Main%20Street&amp;city=Los%20Angeles&amp;location_id=251&amp;zip=91024&amp;eighteen=on&amp;citizen=on">
     290            </http>
     291        </request>
     292        <!-- Next Page -->
     293        <request>
     294            <http url="/eden/vol/skill/create" method="GET" version="1.1">
     295                <www_authenticate userid="%%_username%%" passwd="eden"/>
     296            </http>
     297        </request>
     298        <request> <http url="/eden/static/img/jquery-ui/ui-bg_highlight-soft_100_eeeeee_1x100.png" method="GET" version="1.1"></http> </request>
     299        <request> <http url="/eden/static/img/jquery-ui/ui-bg_glass_100_f6f6f6_1x400.png" method="GET" version="1.1"></http> </request>
     300        <request> <http url="/eden/static/img/jquery-ui/ui-icons_ef8c08_256x240.png" method="GET" version="1.1"></http> </request>
     301
     302        <thinktime value="10" random="true"></thinktime>
     303
     304        <!-- Apply for a task -->
     305        <request>
     306            <http url="/eden/vol/req/application/create?skill_id=1" method="GET" version="1.1">
     307                <www_authenticate userid="%%_username%%" passwd="eden"/>
     308            </http>
     309        </request>
     310        <request>
     311            <http url="/eden/vol/req/7/application/create" method="GET" version="1.1">
     312                <www_authenticate userid="%%_username%%" passwd="eden"/>
     313            </http>
     314        </request>
     315        <request> <http url="/eden/static/img/dialog-warning.png" method="GET" version="1.1"></http> </request>
     316
     317        <thinktime value="10" random="true"></thinktime>
     318
     319        <!-- Logout -->
     320        <request> <http url="/eden/default/user/logout" method="GET" version="1.1"></http> </request>
     321
     322        <thinktime value="10" random="true"></thinktime>
     323
     324    </for>
     325
     326  </session>
     327 </sessions>
     328</tsung>
     329}}}
     330
     331=== Run Tests ===
     332(replace 20xxx with the name of the folder created)
     333{{{
     334tsung start
     335cd ~/.tsung/log/20xxx
     336/usr/lib/tsung/bin/tsung_stats.pl
     337cd ..
     338tar cvf 20xxx.tar 20xxx/
     339gzip -9 20xxx.tar
     340sz 20xxx.tar.gz
     341rm -rf 20xxx*
     342}}}
     343
     344=== Analyse Results ===
     345Open report.html in a web browser:
     346 * Check that there are minimal errors
     347 * Check the Response time (Request is per-request, Transaction is for the whole script)
     348
     349----
     350DeveloperGuidelinesTesting