This is an implementation of: BluePrintSynchronisation and BluePrintZeroConf.

Evaluators guide

Current Status

• All webservices are implemented. They can be consumed using JSON and JSON-RPC.
• Daemon which automates synchronization is functional though its status can not be confirmed without testing Zeroconf with it.

ToDo

• Daemon as cronjob is not initialized automatically: ​http://groups.google.com/group/web2py/browse_thread/thread/b8a55655c7a651b8
• Zeroconf doesn't seems to be very promising. Extensive testing should be done.
• Automatic Synchronization is limited to SahanaPy. SahanaPHP port should be implemented.

Evaluation Guide

0. Install python 2.5 or 2.6 (2.6 passes all json-rpc tests, whilst 2.5 fails a couple)
1. Install json-rpc from ​http://json-rpc.org/wiki/python-json-rpc
2. Install Bazaar
3. bzr branch lp:~mdipierro/web2py/devel web2py
4. cd web2py/applications
5. bzr branch lp:~hasanatkazmi/sahana/p2psync
6. cd p2psync/cron
7. start web2py 8 start daemonX: python daemonX.py
8. Replicate same procedure on another machine
9. Synchronization module should list other server in Administration | Synchronisation | Sync Partners. After sometime, synchronization history should list sync activity. Logs are also maintained in {{cron/synclogs}}}.

WEB SERVICES API

This API is used by daemonX which drives the automatic p2p syncing between clients

Supported web services

• JSON
• JSON-RPC

ToDo (partial work has been done)

• XML
• XML-RPC

Service Proxy

JSON: ​http://localhost:8000/sahana/admin/call/json

e.g. ​http://localhost:8000/sahana/admin/call/json/getdata?timestamp=0&.........

JSON-RPC: ​http://server's-ip:port/sahana/admin/call/jsonrpc replace json and jsonrpc with xml and xmlrpc respectively for xml

Available functions

putdata(uuid, username, password, nicedbdump)

This function is used to insert data in the system. args:

uuid (required):

uuid of the machine which is calling, uuid of machine is 16 character unique string. In the case when web services client is also a Sahana instance, uuid will be generated and stored by deamonX.

Username, password (required):

Used for authentication purposes. Both are strings. A user must be registered at the host machine with data alteration privileges. e.g. Administrator of the system can put data in the system.

nicedbdump (required):

nicedbdump can be best illustrated using diagrammatically representation. If [] represents a python list then: nicedump = [ each element of this list is another list representing a database table

[BR

table name,
[ comma separated table attributes as string]
[ each element in this list is a list which represents a row in table

[comma separated row values]
[]
..
..

]

]
[]
[]
..
..
..

]

Note that if you pass a table using nicedbdump which is not present in database, it will be simply ignored. If nicedbdump is not formated properly then an error string will be returned. Following situations will raise an error: If nicedbdump is not a list. If nicedbdump is not list of lists. Each list in nicedbdump represents a table, say n:

If n does not exactly has 3 elements. If first of these three elements is not a string data type. If second of these three elements, say s, is not a list:

if s in this list is not a string

If third of these three elements, say t, is not a list:

if each element in t is not a list, let such an element be r:

if number of elements in r is not equal to number of elements in s

If a table (s in the case described above) is not having 'id', 'uuid' and 'modified_on' as attribute. 'id' is unique id for each row in table. This 'uuid' is different from the 'uuid' which daemonX maintains, this is row uuid. 'modified_on' represents the last time data was modified (or created it not altered after creation) Note that only that data (referenced by row uuid) which has never life then the one in database will be added. In case of absence of that uuid in database, that data is be added. return:

If user is authenticated and nicedbdump is successfully parsed, data will be added to the database and True will be returned. On the other hand, in case of error, error message will be returned as String.

getdata(uuid, username, password, timestamp = None)

returns data as nicedbdump defined in putdata. Data after timestamp time will returned, if None is passes as timestamp, then that data which has been added to the system after last getdata call from uuid will be returned. Args:

uuid (required):

uuid of the machine which is calling, uuid of machine is 16 character unique string. In the case when web services client is also a Sahana instance, uuid will be generated and stored by deamonX.

username, password (required except for local machine):

Both are strings. used for authentication, user must have privileges for reading the database. If service is called from local machine (i.e. with IP 127.0.0.1) username and password are ignored and user is given access. e.g. deamonX accesses this function locally without providing username and password.

Timestamp (optional):

timestamp is of string type. It should be like “YYYY-MM-DD HH:MM:SS”. If timestamp = null is passes, system will automatically return data after last getdata operation between uuid and machine. deamonX uses this setting

return:

In case of error (like failure to authenticate), error message as string will be returned. If successful, then nicedbdump will be returned which is described above.

Example code

Note: This example is in Python, but you can write a client in any language of your choice.

from jsonrpc import ServiceProxy, JSONRPCException
#jsonrpc needs simplejson which we have to install first
s = ServiceProxy("http://localhost:8000/sahana/admin/call/jsonrpc")
try:
	nicedbdump = s.getdata("machinename12345","email@lums.edu.pk", "myPassword")
	if type(result) == str:
		#it means there is an error, now result has error messege
		pass
	else:
		#result is list type for sure, 
		#result is nicedbdump type
    		
	putit = s.putdata("machinename12345", "email@lums.edu.pk", "myPassword", nicedbdump)
	if putit == True:
		#data sucessfully sent, parsed and processes at server (but in this case no data will be added because you just queried data and sent it back)
		pass
	else:
		#its an error
		pass
except JSONRPCException, e:
	print repr(e.error)

Choosing ZeroConf for Network discovery

Automatic synchronization between servers require automatic service discovery. We had two major options to choose from:

1. ZeroConf
2. Mesh4x

ZeroConf & Mesh4x solve different problems. They don't overlap in functionality at all:

• ZeroConf provides a solution to automatic discovery.
• Mesh4x provides a solution to the data sync.

We were more interested in Zeroconf because:

1. ZeroConf has Python library but Mesh4x doesn't. I means double work was required if we go with Mesh4x.
2. We just needed automatic discovery of service because we wanted to use web services, so that foreign developers can also use Restful API
3. Mesh4x required java daemon, which meant adding jre in the package which would double Sahana package size.

daemonX: Daemon which runs automatic synchronization

We created a daemon which calls web services listed above. DaemonX uses ZeroConf libraries available at ​http://www.amk.ca/python/zeroconf Note that ZeroConf is not being maintained after Dec 2006. daemonX also requires installing jsonrpc libraries from ​http://json-rpc.org/wiki/python-json-rpc for processing JSON.

Very initial tests of daemonX using Zeroconf are below expectations. Using a GPRS moderm as network source, Zeroconf library has thrown errors. More testing needs to be done before making any final statement.