wiki:DeveloperGuidelines/Tips

Context Navigation

DeveloperGuidelines

Tips

Regular Expressions Debugging
Python debugging
1. Reserved Keywords
Web2Py
HTML, CSS, and Javascript debugging
JavaScript
1. jQuery
2. Ext
HTML/CSS
SQL
XSLT: XML, XSL, XPath
Visualization
Misc

Regular Expressions Debugging

https://regex101.com/

Python debugging

Use the Eclipse IDE to be able to set breakpoints and inspect expressions.
WSGI likes print statements to go to sys.stderr not sys.stdout: http://code.google.com/p/modwsgi/wiki/DebuggingTechniques
use Web2Py shell: http://www.vimeo.com/879939
use IPython for interactive exploration (on Windows, install PyReadline)
use FirePy to see logging output to Console (to view with Firebug)

Log to a File:

cp /home/web2py/examples/logging.example.conf /home/web2py/logging.conf
vi /home/web2py/logging.conf
# Add PyDAL
keys=root,rocket,markdown,web2py,rewrite,cron,app,welcome,pydal
[logger_pydal]
level=DEBUG
handlers=rotatingFileHandler
qualname=pyDAL
propagate=0
# Don't break uwsgi: Remove ConsoleHandler (from all) & log to stderr
handlers=rotatingFileHandler
args=(sys.stderr,)

http://groups.google.com/group/web2py/browse_thread/thread/e20d0bd2e542aa14

Debug Tools: http://docs.python.org/library/debug.html

Use Inspect to check which function called the one you're troubleshooting:

import inspect
curframe = inspect.currentframe()
calframe = inspect.getouterframes(curframe, 2)
current.log.debug("caller name: %s" % calframe[1][3])

Debug SQL Queries:

vi gluon/dal.py
class BaseAdapter()
    def select():
        def response(sql):
            if "searchphrase" in sql:
                current.log.debug("Query: %s" % query)
                current.log.debug("SQL: %s" % sql)

Debug Views:

vi gluon/globals.py
    def write(self, data, escape=True):
        try:
            ...All Old Code ...
        except Exception, error:
            # Put Eclipse breakpoint on the below line
            exc_type, exc_value, exc_traceback = sys.exc_info()
            pass

Python Debugging Techniques
- Insert these statements into your code
  - Get a Python shell up to explore interactively: import code; code.interact(locals=locals()
  - Get a full stack trace: import pdb; pdb.set_trace()
winpdb: http://groups.google.com/group/web2py/msg/ee46125b7c93fdd4
WingIDE: http://www.wingware.com/
- Using with Web2Py: http://www.wingware.com/doc/howtos/web2py
Eclipse IDE
Unicode:
- "Decode early, Unicode everywhere, Encode late": http://farmdev.com/talks/unicode/
- http://boodebr.org/main/python/all-about-python-and-unicode
Python Design Patterns:
- http://www.youtube.com/watch?v=Er5K_nR5lDQ
- http://videolectures.net/youtube_martelli_python/

Reserved Keywords

'request' -> web2py internal use
'key' as a db row name -> MySQL confuses it with internal keyword KEY
http://dev.mysql.com/doc/mysqld-version-reference/en/mysqld-version-reference-reservedwords-5-0.html
http://www.postgresql.org/docs/8.1/static/sql-keywords-appendix.html

Web2Py

Web2py Internals and Tips
http://web2py.com/examples/default/docs
Prevent HTML tags being escaped in views: {{=XML("<p>Not escaped!</p>")}}
HTML Helpers: http://web2py.com/AlterEgo/default/show/217
Arbitrary HTML tags using TAG, e.g. TAG.LEGEND('a','b',_c='d')
HTML Helpers can be navigated/manipulated like the DOM: http://groups.google.com/group/web2py/browse_thread/thread/ac045f3b1d3846d9
https://web2py.com/web2py_wiki/default/wiki/tips
https://web2py.com/web2py_wiki/default/wiki/widgets
Pagination:
- Basic server-side pagination: http://web2py.com/AlterEgo/default/show/95
- Grid to replace SQLTABLE with server-side pagination included: http://www.web2pyslices.com/main/slices/take_slice/39
New syntax for low-level database record create/update/delete: http://groups.google.com/group/web2py/browse_thread/thread/d79a6e52824d4221
Default field values with crud.create(): http://groups.google.com/group/web2py/browse_thread/thread/717e2349f0f8e93d
Edit multiple records at once: http://groups.google.com/group/web2py/browse_thread/thread/401b4c192871194c/a96cc8c7923a8565#
Prevent multiple concurrent logins: http://groups.google.com/group/web2py/browse_thread/thread/4e1c62a0d9255160
Process uploaded files manually (e.g. to unzip KMZ): http://groups.google.com/group/web2py/browse_thread/thread/63809c06cac3b0d5
Access DB models from a Python script: http://groups.google.com/group/web2py/browse_thread/thread/147b47069a9c1ea2
Custom Forms
- Debug by setting the return to just a plain dict() (=> can read values of vars in 'request')
- Latest code method: http://groups.google.com/group/web2py/browse_thread/thread/1d7c1e31b5f66534
  - Suggested workflow for customising HTML: http://groups.google.com/group/web2py/browse_thread/thread/95e571e49b7d77ed/8c998b226f5a0437
- less retyping of code in view using SQLFORM extensions: http://groups.google.com/group/web2py/browse_thread/thread/2d8cc57352d158bd
- Simple version: http://groups.google.com/group/web2py/browse_thread/thread/7245cab0df469d5f
- Alternate approach: http://www.wellbehavedsystems.co.uk/web2py/examples/
- Helper field2html(): http://groups.google.com/group/web2py/browse_thread/thread/1012c11c977aa05e
- Multiple per page: http://groups.google.com/group/web2py/browse_thread/thread/57e452d186d48fa4
- Values retrieved from DB are stored for re-referencing: http://mdp.cti.depaul.edu/AlterEgo/default/show/205
- Summary of MVC options: http://groups.google.com/group/web2py/browse_thread/thread/7da38e0d32d01076
- form.accepts AJAX submission: http://groups.google.com/group/web2py/browse_thread/thread/e583a1e377cc7dea
Lambda tricks: http://p-nand-q.com/python/stupid_lambda_tricks.html
THIS_NOT_IN_DB custom validator: http://groups.google.com/group/web2py/browse_thread/thread/27b14433976c0540?pli=1
IS_IN_DB (...,order_by=...): http://groups.google.com/group/web2py/browse_thread/thread/5fe6e343d115590d/2a8c38fccb2cf3cf#2a8c38fccb2cf3cf

Multiple field validation example:

db.Wedding.location.requires = [IS_NOT_EMPTY(), IS_NOT_IN_DB(db(db.Wedding.weddingDateTime == request.vars.weddingDateTime), 'Wedding.location', error_message="Date/Time for Venue is Booked")]

CGI co-existence: http://groups.google.com/group/web2py/browse_thread/thread/1520b84d144f0b0c
Inline Images in T2 CRUD: http://groups.google.com/group/web2py/browse_thread/thread/d954bf883baf7adc/3b55a819b425a741
PIL to create thumbnails:
- http://groups.google.com/group/web2py/browse_thread/thread/3523f9a12edc6836
- http://groups.google.com/group/web2py/browse_thread/thread/380cfb61a1fa1596
Download Images with original filename:
- http://groups.google.com/group/web2py/browse_thread/thread/484a2d04bdd7606f
- http://groups.google.com/group/web2py/browse_thread/thread/1df80dbf52d401fb
TEXTAREA options: http://groups.google.com/group/web2py/browse_thread/thread/482ecab0878689d5
nicEdit integration: http://groups.google.com/group/web2py/msg/ff491af9382f6c39
Multi-column unique constraint: http://groups.google.com/group/web2py/browse_thread/thread/f5ab94cc7d511e0b
Modified preorder Tree Traversal (for large Hierarchical trees): http://groups.google.com/group/web2py/browse_thread/thread/70efddab62dfe73
Hierarchical References: http://groups.google.com/group/web2py/browse_thread/thread/a4f57635d4a05f83
JSON datetime serialization: https://web2py.com/web2py_wiki/default/wiki/JSONdatetime
Enter key in ajaxified forms: http://groups.google.com/group/web2py/browse_thread/thread/bae7b19ff35b896b#
Limiting access to a controller function to just certain IPs: http://groups.google.com/group/web2py/msg/df000860c707d3d3
Auto-complete (for when Dropdowns get too long):
- AutoComplete guidelines proposal
- use the ajaxS3 calls: getS3, getJSONS3, or postS3
- use the IS_ONE_OF_EMPTY validator
- http://trac.sahanapy.org/browser/views/msg/email_outbox_create.html
- http://groups.google.com/group/web2py/browse_thread/thread/e1034df0091b5bfd
- http://groups.google.com/group/web2py/browse_thread/thread/f6e0170273b14241
- http://bassistance.de/jquery-plugins/jquery-plugin-autocomplete/ (Also: http://plugins.jquery.com/project/autocompletex)
- http://dyve.net/jquery/?autocomplete (Older version: http://www.pengoworks.com/workshop/jquery/autocomplete.htm)
- http://code.google.com/p/js-autocomplete/
- http://github.com/ReinH/jquery-autocomplete/tree/master (Also: http://plugins.jquery.com/project/jq-autocomplete)
- http://plugins.jquery.com/project/YA_AutoComplete
EPB: http://groups.google.com/group/web2py/browse_thread/thread/c090560680b65db2
Implementing Plone's Indexing & Workflow: http://groups.google.com/group/web2py/browse_thread/thread/496edd88f6298653
Importing Legacy Data: http://groups.google.com/group/web2py/browse_thread/thread/5bd8f85b21444858
Upload progress: http://www.web2pyslices.com/main/slices/take_slice/10
Use the StatusBar for dynamic status updates
http://code.google.com/p/web2pyorm/

HTML, CSS, and Javascript debugging

Each of the major browsers has tools available for viewing HTML page structure and form data, Javascript code, CSS. Some also support Javascript debugging, including setting breakpoints and viewing values in variables.

Firefox: Install the Firebug plugin. Start Firebug by clicking the bug icon in the upper right. After the tools open in the lower half of the page, load the page you want to debug.

Chrome: Open the Chrome developer tools with Chrome menu (three bar icon) -> Tools -> Developer tools.

Internet Explorer: Open the IE developer tools with F12, or Tools (gear icon) -> F12 Developer Tools. Note that you can test behavior of pages under earlier versions of IE by setting your current IE to behave as earlier versions. In the developer tools menu bar, click Browser Mode and select the desired version.

Other tools that may be of interest:

use Visual Event to see which events are active.
use Venkman: http://www.mozilla.org/projects/venkman/
use Fiddler for compressed files: http://www.yuiblog.com/blog/2008/06/27/fiddler
http://pastebin.me (allows edits of HTML & previews)
http://jsfiddle.net (allows edits of JS & previews)
http://yura.thinkweb2.com/domlint/
- http://www.jslint.com/lint.html
IE6 cheatsheet: http://www.virtuosimedia.com/tutorials/ultimate-ie6-cheatsheet-how-to-fix-25-internet-explorer-6-bugs

JavaScript

http://trac.openlayers.org/wiki/Documentation#BeforeGettingStarted--TheTechnologiesBehindOpenLayers
http://eloquentjavascript.net
Dynamically load additional JS: http://unixpapa.com/js/dyna.html
array.join instead of string concatenation (& other tips)
Ensure variable names use only Word characters:
- Python: variable_name = re.sub('\W', '_', string)
- JS: var RE_JSVAR = new RegExp("\W", "g"); var variable_name = string.replace(RE_JSVAR, '_');
Loop timings: http://blogs.sun.com/greimer/entry/best_way_to_code_a
Closures: http://www.javascriptkit.com/javatutors/closures.shtml
Memory Leaks: http://www.javascriptkit.com/javatutors/closuresleak/index.shtml
Console cd to iframe:
```
cd(frames[0])
...
cd(top)
```

jQuery

Ext

HTML/CSS

SQL

Test queries on multiple databases:

http://sqlfiddle.com

Basics of Design:

Language:

More Advanced:

http://www.tomjewett.com/dbdesign/dbdesign.php?page=intro.html
http://www.w3schools.com/Sql/sql_join.asp
http://en.wikipedia.org/wiki/Join_(SQL
http://www.plus2net.com/sql_tutorial/sql_linking_table.php
http://en.wikipedia.org/wiki/Entity-attribute-value_model
- http://pypi.python.org/pypi/eav-django
Object-Relational Mapping: http://www.objectmatter.com/vbsf/docs/maptool/ormapping.html
Modified preorder Tree Traversal (for large Hierarchical trees): http://www.sitepoint.com/print/hierarchical-data-database/
GAE Many<>Many: http://arbingersys.blogspot.com/2008/04/google-app-engine-better-many-to-many_30.html
Django ORM & JOINs: http://slott-softwarearchitect.blogspot.com/2009/07/object-models-and-relational-joins.html
Especially useful for USB (minimise disk access): http://web.utk.edu/~jplyon/sqlite/SQLite_optimization_FAQ.html

XSLT: XML, XSL, XPath

We use XSLT (Extensible Stylesheet Language Transformations) to import external structured data, such as CSV files, spreadsheets, GIS data, and CAP messages into Eden, and also to export Eden data into external data formats.

XSL Stylesheets are themselves XML documents that specify a conversion between one data format and another. XPath (XML Path Language) is used to select elements from the source document to act on. Our stylesheets convert between the internal S3XML format and the desired external format.

Stylesheets for various data formats are found in the directory: eden/static/formats

Support for XSLT is provided by lxml, which is a Python wrapper for libxml2 and libxslt. This currently supports the XML 1.0 standard, not 2.0, and there are functions available in 2.0 that are not in 1.0, so be aware of this when you are consulting XML / XSLT references.

Eden documentation

Online references

Books

Evan Lenz, XSLT 1.0 Pocket Reference, O'Reilly
Michael Kay, XSLT Programmer's Reference, WROX Press (note link is to a later edition that covers XSLT 2.0)

Tools

Try out your xsl stylesheets with: xsltransform.py

Visualization

Misc

Bugmail add-on for Thunderbird: https://addons.mozilla.org/en-US/thunderbird/addon/9584