DeveloperGuidelines
= Tips =
[[TOC]]

== Regular Expressions Debugging ==
https://regex101.com/
== Python debugging ==
* Use the [DeveloperGuidelines/Eclipse Eclipse IDE] to be able to set breakpoints and inspect expressions.
* WSGI likes print statements to go to {{{sys.stderr}}} not {{{sys.stdout}}}: http://code.google.com/p/modwsgi/wiki/DebuggingTechniques
* use Web2Py shell: http://www.vimeo.com/879939
* use [http://ipython.scipy.org/moin/ IPython] for interactive exploration (on Windows, install [http://ipython.scipy.org/moin/PyReadline/Intro PyReadline])
* use [http://groups.google.com/group/web2py/browse_thread/thread/cc13960a5079b2d5# FirePy] to see logging output to Console (to view with Firebug)
* Log to a File:
{{{
cp /home/web2py/examples/logging.example.conf /home/web2py/logging.conf
vi /home/web2py/logging.conf
# Add PyDAL
keys=root,rocket,markdown,web2py,rewrite,cron,app,welcome,pydal
[logger_pydal]
level=DEBUG
handlers=rotatingFileHandler
qualname=pyDAL
propagate=0
# Don't break uwsgi: Remove ConsoleHandler (from all) & log to stderr
handlers=rotatingFileHandler
args=(sys.stderr,)
}}}
 * http://groups.google.com/group/web2py/browse_thread/thread/e20d0bd2e542aa14
* Debug Tools: http://docs.python.org/library/debug.html
* Use [http://stackoverflow.com/questions/2654113/python-how-to-get-the-callers-method-name-in-the-called-method Inspect] to check which function called the one you're troubleshooting:
{{{
import inspect
curframe = inspect.currentframe()
calframe = inspect.getouterframes(curframe, 2)
current.log.debug("caller name: %s" % calframe[1][3])
}}}
 * Debug SQL Queries:
{{{
vi gluon/dal.py
class BaseAdapter()
    def select():
        def response(sql):
            if "searchphrase" in sql:
                current.log.debug("Query: %s" % query)
                current.log.debug("SQL: %s" % sql)
}}}
 * Debug Views:
{{{
vi gluon/globals.py
    def write(self, data, escape=True):
        try:
            ...All Old Code ...
        except Exception, error:
            # Put Eclipse breakpoint on the below line
            exc_type, exc_value, exc_traceback = sys.exc_info()
            pass
}}}
* [http://aymanh.com/python-debugging-techniques Python Debugging Techniques]
 * Insert these statements into your code
  * Get a Python shell up to explore interactively: {{{import code; code.interact(locals=locals()}}}
  * Get a full stack trace: {{{import pdb; pdb.set_trace()}}}
* winpdb: http://groups.google.com/group/web2py/msg/ee46125b7c93fdd4
* WingIDE: http://www.wingware.com/
 * Using with Web2Py: http://www.wingware.com/doc/howtos/web2py
* [DeveloperGuidelines/Eclipse Eclipse] IDE
* Unicode:
 * "Decode early, Unicode everywhere, Encode late": http://farmdev.com/talks/unicode/
 * http://boodebr.org/main/python/all-about-python-and-unicode
* Python Design Patterns:
 * http://www.youtube.com/watch?v=Er5K_nR5lDQ
 * http://videolectures.net/youtube_martelli_python/

=== Reserved Keywords ===
* 'request' -> web2py internal use
* 'key' as a db row name -> MySQL confuses it with internal keyword KEY
* http://dev.mysql.com/doc/mysqld-version-reference/en/mysqld-version-reference-reservedwords-5-0.html
* http://www.postgresql.org/docs/8.1/static/sql-keywords-appendix.html

== Web2Py ==
* [wiki:Web2pyInternalsAndTips Web2py Internals and Tips]
* http://web2py.com/examples/default/docs
* Prevent HTML tags being escaped in views: {{{{{=XML("<p>Not escaped!</p>")}}}}}
* HTML Helpers: http://web2py.com/AlterEgo/default/show/217
* Arbitrary HTML tags using TAG, e.g. TAG.LEGEND('a','b',_c='d')
* HTML Helpers can be navigated/manipulated like the DOM: http://groups.google.com/group/web2py/browse_thread/thread/ac045f3b1d3846d9
* https://web2py.com/web2py_wiki/default/wiki/tips
* https://web2py.com/web2py_wiki/default/wiki/widgets
* Pagination:
 * Basic server-side pagination: http://web2py.com/AlterEgo/default/show/95
 * Grid to replace SQLTABLE with server-side pagination included: http://www.web2pyslices.com/main/slices/take_slice/39
* New syntax for low-level database record create/update/delete: http://groups.google.com/group/web2py/browse_thread/thread/d79a6e52824d4221
* Default field values with crud.create(): http://groups.google.com/group/web2py/browse_thread/thread/717e2349f0f8e93d
* Edit multiple records at once: http://groups.google.com/group/web2py/browse_thread/thread/401b4c192871194c/a96cc8c7923a8565#
* Prevent multiple concurrent logins: http://groups.google.com/group/web2py/browse_thread/thread/4e1c62a0d9255160
* Process uploaded files manually (e.g. to unzip KMZ): http://groups.google.com/group/web2py/browse_thread/thread/63809c06cac3b0d5
* Access DB models from a Python script: http://groups.google.com/group/web2py/browse_thread/thread/147b47069a9c1ea2
* Custom Forms
 * Debug by setting the return to just a plain dict() (=> can read values of vars in 'request')
 * Latest code method: http://groups.google.com/group/web2py/browse_thread/thread/1d7c1e31b5f66534
  * Suggested workflow for customising HTML: http://groups.google.com/group/web2py/browse_thread/thread/95e571e49b7d77ed/8c998b226f5a0437
 * less retyping of code in view using SQLFORM extensions: http://groups.google.com/group/web2py/browse_thread/thread/2d8cc57352d158bd
 * Simple version: http://groups.google.com/group/web2py/browse_thread/thread/7245cab0df469d5f
 * Alternate approach: http://www.wellbehavedsystems.co.uk/web2py/examples/
 * Helper field2html(): http://groups.google.com/group/web2py/browse_thread/thread/1012c11c977aa05e
 * Multiple per page: http://groups.google.com/group/web2py/browse_thread/thread/57e452d186d48fa4
 * Values retrieved from DB are stored for re-referencing:  http://mdp.cti.depaul.edu/AlterEgo/default/show/205
 * Summary of MVC options: http://groups.google.com/group/web2py/browse_thread/thread/7da38e0d32d01076
 * form.accepts AJAX submission: http://groups.google.com/group/web2py/browse_thread/thread/e583a1e377cc7dea
* Lambda tricks: http://p-nand-q.com/python/stupid_lambda_tricks.html
* THIS_NOT_IN_DB custom validator: http://groups.google.com/group/web2py/browse_thread/thread/27b14433976c0540?pli=1
* IS_IN_DB (...,order_by=...): http://groups.google.com/group/web2py/browse_thread/thread/5fe6e343d115590d/2a8c38fccb2cf3cf#2a8c38fccb2cf3cf
* Multiple field validation example:
{{{
db.Wedding.location.requires = [IS_NOT_EMPTY(), IS_NOT_IN_DB(db(db.Wedding.weddingDateTime == request.vars.weddingDateTime), 'Wedding.location', error_message="Date/Time for Venue is Booked")]
}}}
* CGI co-existence: http://groups.google.com/group/web2py/browse_thread/thread/1520b84d144f0b0c
* Inline Images in T2 CRUD: http://groups.google.com/group/web2py/browse_thread/thread/d954bf883baf7adc/3b55a819b425a741
* PIL to create thumbnails:
 * http://groups.google.com/group/web2py/browse_thread/thread/3523f9a12edc6836
 * http://groups.google.com/group/web2py/browse_thread/thread/380cfb61a1fa1596
* Download Images with original filename:
 * http://groups.google.com/group/web2py/browse_thread/thread/484a2d04bdd7606f
 * http://groups.google.com/group/web2py/browse_thread/thread/1df80dbf52d401fb
* TEXTAREA options: http://groups.google.com/group/web2py/browse_thread/thread/482ecab0878689d5
* nicEdit integration: http://groups.google.com/group/web2py/msg/ff491af9382f6c39
* Multi-column unique constraint: http://groups.google.com/group/web2py/browse_thread/thread/f5ab94cc7d511e0b
* Modified preorder Tree Traversal (for large Hierarchical trees): http://groups.google.com/group/web2py/browse_thread/thread/70efddab62dfe73
* Hierarchical References: http://groups.google.com/group/web2py/browse_thread/thread/a4f57635d4a05f83
* JSON datetime serialization: https://web2py.com/web2py_wiki/default/wiki/JSONdatetime
* Enter key in ajaxified forms: http://groups.google.com/group/web2py/browse_thread/thread/bae7b19ff35b896b#
* Limiting access to a controller function to just certain IPs: http://groups.google.com/group/web2py/msg/df000860c707d3d3
* Auto-complete (for when Dropdowns get too long):
 * AutoComplete guidelines proposal
 * use the [wiki:ajaxS3 ajaxS3] calls: ''getS3'', ''getJSONS3'', or ''postS3''
 * use the [wiki:IS_ONE_OF_EMPTY IS_ONE_OF_EMPTY] validator
 * http://trac.sahanapy.org/browser/views/msg/email_outbox_create.html
 * http://groups.google.com/group/web2py/browse_thread/thread/e1034df0091b5bfd
 * http://groups.google.com/group/web2py/browse_thread/thread/f6e0170273b14241
 * http://bassistance.de/jquery-plugins/jquery-plugin-autocomplete/ (Also: http://plugins.jquery.com/project/autocompletex)
 * http://dyve.net/jquery/?autocomplete (Older version: http://www.pengoworks.com/workshop/jquery/autocomplete.htm)
 * http://code.google.com/p/js-autocomplete/
 * http://github.com/ReinH/jquery-autocomplete/tree/master (Also: http://plugins.jquery.com/project/jq-autocomplete)
 * http://plugins.jquery.com/project/YA_AutoComplete
* EPB: http://groups.google.com/group/web2py/browse_thread/thread/c090560680b65db2
* Implementing Plone's Indexing & Workflow: http://groups.google.com/group/web2py/browse_thread/thread/496edd88f6298653
* Importing Legacy Data: http://groups.google.com/group/web2py/browse_thread/thread/5bd8f85b21444858
* Upload progress: http://www.web2pyslices.com/main/slices/take_slice/10
* Use the StatusBar for dynamic status updates
* http://code.google.com/p/web2pyorm/

== HTML, CSS, and Javascript debugging ==

Each of the major browsers has tools available for viewing HTML page structure and form data, Javascript code, CSS.  Some also support Javascript debugging, including setting breakpoints and viewing values in variables.

* Firefox: Install the [https://addons.mozilla.org/en-US/firefox/addon/firebug/ Firebug] plugin.  Start Firebug by clicking the bug icon in the upper right.  After the tools open in the lower half of the page, load the page you want to debug.

* Chrome: Open the [https://developers.google.com/chrome-developer-tools/ Chrome developer tools] with Chrome menu (three bar icon) -> Tools -> Developer tools.

* Internet Explorer: Open the [http://msdn.microsoft.com/library/ie/bg182326%28v=vs.85%29 IE developer tools] with F12, or Tools (gear icon) -> F12 Developer Tools.  Note that you can test behavior of pages under earlier versions of IE by setting your current IE to behave as earlier versions.  In the developer tools menu bar, click Browser Mode and select the desired version.

Other tools that may be of interest:

* use [http://www.sprymedia.co.uk/article/Visual+Event Visual Event] to see which events are active.
* use Venkman: http://www.mozilla.org/projects/venkman/
* use Fiddler for compressed files: http://www.yuiblog.com/blog/2008/06/27/fiddler
* http://pastebin.me (allows edits of HTML & previews)
* http://jsfiddle.net (allows edits of JS & previews)
* http://yura.thinkweb2.com/domlint/
 * http://www.jslint.com/lint.html
* IE6 cheatsheet: http://www.virtuosimedia.com/tutorials/ultimate-ie6-cheatsheet-how-to-fix-25-internet-explorer-6-bugs

== !JavaScript ==
* http://trac.openlayers.org/wiki/Documentation#BeforeGettingStarted--TheTechnologiesBehindOpenLayers
* http://eloquentjavascript.net
* Dynamically load additional JS: http://unixpapa.com/js/dyna.html
* [http://aymanh.com/9-javascript-tips-you-may-not-know#String_Concatenation_vs._Array.join array.join instead of string concatenation] (& other tips)
* Ensure variable names use only Word characters:
 * Python: {{{variable_name = re.sub('\W', '_', string)}}}
 * JS: {{{var RE_JSVAR = new RegExp("\W", "g"); var variable_name = string.replace(RE_JSVAR, '_');}}}
* Loop timings: http://blogs.sun.com/greimer/entry/best_way_to_code_a
* Closures: http://www.javascriptkit.com/javatutors/closures.shtml
* Memory Leaks: http://www.javascriptkit.com/javatutors/closuresleak/index.shtml
* Console cd to iframe:
{{{
cd(frames[0])
...
cd(top)
}}}

=== jQuery ===
* http://www.smashingmagazine.com/2008/09/16/jquery-examples-and-best-practices/
* http://www.tvidesign.co.uk/blog/improve-your-jquery-25-excellent-tips.aspx
* http://james.padolsey.com/javascript/things-you-may-not-know-about-jquery/
* http://15daysofjquery.com/
* http://jqueryfordesigners.com/jquery-tabs/
* http://devsnippets.com/reviews/using-jquery-to-style-design-elements-20-impressive-plugins.html
* http://www.mapbender.org/JavaScript_pitfalls:_null,_false,_undefined,_NaN
* http://www.learningjquery.com/2009/03/43439-reasons-to-use-append-correctly
* Web2Py HTML Helper: http://groups.google.com/group/web2py/browse_thread/thread/ec63ee9af3ee659f
* jqGrid in Web2Py: http://groups.google.com/group/web2py/browse_thread/thread/2caa0e4e0161bb8d

=== Ext ===
* http://extjs.com/learn/Tutorial:Introduction_to_Ext_2.0
* http://extjs.com/learn/Manual:Basic_Application_Design
* http://extjs.com/learn/Tutorial:Writing_a_Big_Application_in_Ext
* http://extjs.com/forum/showthread.php?t=26728
* http://extjs.com/learn/Ext_FAQ_Debugging
* http://www.extjs.com/blog/2009/05/13/introducing-ext-direct/
 * http://blog.extjs.eu/know-how/mastering-ext-direct-part-1/

== HTML/CSS ==
* http://enhance.qd-creative.co.uk/2008/06/28/the-misunderstood-div/
* http://enhance.qd-creative.co.uk/2008/10/14/7-things-you-can-do-so-your-users-wont-leave/
* Usability: http://www.useit.com
* Progressive Enhancement (better than Graceful Degradation):
 * http://developer.yahoo.com/yui/articles/gbs/
 * http://accessites.org/site/2007/02/graceful-degradation-progressive-enhancement/
 * http://www.alistapart.com/articles/understandingprogressiveenhancement
  * http://www.alistapart.com/articles/progressiveenhancementwithcss
  * http://www.alistapart.com/articles/progressiveenhancementwithjavascript
 * http://webtips.dan.info/graceful.html

== SQL ==
Test queries on multiple databases:
* http://sqlfiddle.com
Basics of Design:
* http://www.simple-talk.com/sql/database-administration/ten-common-database-design-mistakes/
* Normalization:
 * http://msdn.microsoft.com/en-us/library/ms191178%28SQL.100%29.aspx
 * http://www.tomjewett.com/dbdesign/dbdesign.php?page=normalize.php
* http://www.sqlteam.com/article/database-design-and-modeling-fundamentals
* http://msdn.microsoft.com/en-us/library/dd129510%28VS.85%29.aspx
Language:
* http://www.intermedia.net/support/sql/sqltut.asp
* http://sqlcourse.com/intro.html
* http://www.w3schools.com/sql/default.asp
* http://www.codinghorror.com/blog/2007/10/a-visual-explanation-of-sql-joins.html
More Advanced:
* http://www.tomjewett.com/dbdesign/dbdesign.php?page=intro.html
* http://www.w3schools.com/Sql/sql_join.asp
* http://en.wikipedia.org/wiki/Join_(SQL
* http://www.plus2net.com/sql_tutorial/sql_linking_table.php
* http://en.wikipedia.org/wiki/Entity-attribute-value_model
 * http://pypi.python.org/pypi/eav-django
* Object-Relational Mapping: http://www.objectmatter.com/vbsf/docs/maptool/ormapping.html
* Modified preorder Tree Traversal (for large Hierarchical trees): http://www.sitepoint.com/print/hierarchical-data-database/
* GAE Many<>Many: http://arbingersys.blogspot.com/2008/04/google-app-engine-better-many-to-many_30.html
* Django ORM & JOINs: http://slott-softwarearchitect.blogspot.com/2009/07/object-models-and-relational-joins.html
* Especially useful for USB (minimise disk access): http://web.utk.edu/~jplyon/sqlite/SQLite_optimization_FAQ.html

== XSLT: XML, XSL, XPath ==

We use XSLT (Extensible Stylesheet Language Transformations) to import external structured data, such as
CSV files, spreadsheets, GIS data, and CAP messages into Eden, and also to export Eden data into external data formats.

XSL Stylesheets are themselves XML documents that specify a conversion between one data format and another.
XPath (XML Path Language) is used to select elements from the source document to act on.  Our stylesheets
convert between the internal [wiki:S3XRC/S3XML S3XML] format and the desired external format.

Stylesheets for various data formats are found in the directory:
[https://github.com/flavour/eden/blob/master/static/formats eden/static/formats]

Support for XSLT is provided by [http://lxml.de/ lxml], which is a Python wrapper for libxml2 and libxslt.
This currently supports the XML 1.0 standard, not 2.0, and there are functions available in 2.0 that are not in 1.0,
so be aware of this when you are consulting XML / XSLT references.

=== Eden documentation ===

* [wiki:S3XRC/S3XML S3XML]
* [wiki:XsltTemplates XsltTemplates]
=== Online references ===

* W3 standards
 * [http://www.w3.org/standards/techs/xslt#w3c_all List of XSLT-related standards]
 * [http://www.w3.org/standards/xml/transformation XSLT overview]
 * [http://www.w3.org/TR/xslt XSLT 1.0]
 * [http://www.w3.org/TR/xpath-functions/ XPath and XQuery functions and operators]
* [http://msdn.microsoft.com/en-us/library/ms256069%28v=vs.110%29.aspx MSDN XSLT Reference]
* [http://www.dpawson.co.uk/xsl/sect1/sect1.html Dave Pawson's tutorials]

=== Books ===

* Evan Lenz, [http://shop.oreilly.com/product/9780596100087.do XSLT 1.0 Pocket Reference], O'Reilly
* Michael Kay, [http://shop.oreilly.com/product/9780470192740.do XSLT Programmer's Reference], WROX Press (note link is to a later edition that covers XSLT 2.0)
=== Tools ===

Try out your xsl stylesheets with: [https://github.com/flavour/eden/blob/master/static/scripts/tools/xsltransform.py xsltransform.py]



== Visualization ==
* http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen.html
* http://www.ted.com/talks/david_mccandless_the_beauty_of_data_visualization.html

== Misc ==
* Bugmail add-on for Thunderbird: https://addons.mozilla.org/en-US/thunderbird/addon/9584

----
DeveloperGuidelines