Version 27 (modified by Dominic König, 5 years ago) ( diff )


Python 2/3 Compatibility

This guideline documents coding conventions to achieve hybrid Python-2.7/Python-3.5 compatibility in Sahana.

It's a working document that will be added to as we move towards full Python-3 compatibility.


No print statements

Don't use the print statement anywhere:

print "example" # deprecated

You shouldn't use the print-function either, because it can clash with uWSGI:

print("example") # not good

...but it can be tolerated in CLI scripts which don't run in the WSGI environment.

Best option for debug output is to use sys.stderr.write:

import sys
sys.stderr.write("example\n") # better

...or the logger (as it can be configured globally to write to a log file instead of the system console):

current.log.debug("example") # even better

Use as-syntax for catching exceptions

When catching exceptions in a variable, don't use the comma-syntax:

except Exception, e: # deprecated

Instead, use the as-keyword:

except Exception as e: # new standard

No exec statements

In Python-3, the exec statement has become a function exec(). Python-2.7 accepts the function-syntax as well, so we use it throughout:

# Deprecated:
exec pyexpr
# New Standard:

NB the Python-2.7 exec statement also accepts a 3-tuple as parameter exec(expr, globals, locals) which syntax is equivalent to the exec-function in Python-3.


No implicit package-relative imports

Python-3 does not search for modules relative to the current module in the same package - unless explicitly indicated by leading . or .. in the module path.

from s3datetime import s3_format_datetime # inside modules/s3, not working in Python-3

Python-2.7 would search relative to the current module, but on the other hand, it supports the explicit-relative syntax as well.

So we decide that only explicit paths shall be used in imports.

To import a module in the same package (e.g. within s3), either use explicit-relative syntax:

from .s3datetime import s3_format_datetime # inside modules/s3, preferred variant

...or an absolute path relative to modules (or the global python path):

from s3.s3datetime import s3_format_datetime # inside modules/s3, acceptable alternative

Outside of modules/s3, you should always import from the top-level of the s3 package (because the package structure may change over time):

from s3 import s3_format_datetime # outside modules/s3

Alternative Imports

As the locations and names of some libraries have changed in Python-3, we use the compatibility module (modules/ to implement suitable alternatives. Similarily, the compat module provides alternatives for other objects such as types, functions and certain common patterns.

Where an object name is provided by modules/, it must be imported from there if used.

The following objects are provided by s3compat:

Name Import Comments/Caveats
PY2from s3compat import PY2Constant indicating whether we're currently running on Py2, should only be used if alternatives cannot be generalized
StringIOfrom s3compat import StringIO
basestringfrom s3compat import basestring
unichrfrom s3compat import unichr
STRING_TYPESfrom s3compat import STRING_TYPESmaps to tuple of all known string types: (str,unicode) in Py2, (str,) in Py3, for type checking
longfrom s3compat import longsame as int in Py3, so can occasionally lead to redundancy
INTEGER_TYPESfrom s3compat import INTEGER_TYPESmaps to tuple of all known integer types: (int,long) in Py2, (int,) in Py3, for use with isinstance
urllib2from s3compat import urllib2maps to urllib.requests in Py3, which contains only part of Py2's urllib2 - some urllib2 objects therefore need to be imported separately (see below)
HTTPErrorfrom s3compat import HTTPErrorreplaces urllib2.HTTPError in Py2
urlopenfrom s3compat import urlopenreplaces urllib.urlopen and urllib2.urlopen in Py2
urlencodefrom s3compat import urlencodereplaces urllib.urlencode in Py2
HTMLParserfrom s3compat import HTMLParserreplaces HTMLParser.HTMLParser in Py2

No dict.iteritems, iterkeys or itervalues

Python-3 does no longer have dict.iteritems(). We use dict.items() instead:

# Deprecated:
for x, y in d.iteritems():
# Compatible:
for x, y in d.items():

NB In Python-2, dict.items() returns a list of tuples, but in Python-3 it returns a dict view object that is sensitive to changes of the dict. If a list is required, or the dict is changed inside the loop, the result of dict.items() must be converted into a list explicitly.

The same applies to dict.iterkeys() (use dict.keys() instead) and dict.itervalues() (use dict.values() instead).

Map and Zip return generators

In Python-3, the map() and zip() functions return generator objects rather than lists. This is fine when we want to iterate over the result, especially when there is a chance to break out of the loop early.

But where lists are required, the return value must be converted explicitly using the list constructor.

# This is fine:
for item in map(func, values):
# This could be wrong:
result = map(func, values)
# This is better:
result = list(map(func, values))
# This could be even better:
result = [func(v) for v in values]

NB For building a list from a single iterable argument, we prefer list comprehensions over map() for readability and speed.

Note: See TracWiki for help on using the wiki.