[Gpe-list] gpesyncd uids

[Graham Cobb]

On Tuesday 16 March 2010 19:34:51 Robert Chéramy wrote:
> I'm working on a syncing program for gpe-calendar (todo and contacts
> should come next). My aim is to sync (in both ways) gpe-calendar between
> my Nokia n810 and my PC.

It is up to you, but I would suggest putting your effort into contributing to 
one of the projects which plan to solve this problem properly: either 
OpenSync or SyncEvolution.  Both could do with some help.  After some years 
wrestling with this, I have discovered that synchronisation is a hard 
problem -- that is why several projects have started and either failed or had 
to be substantially rewritten and why there is currently no working open 
source sync framework!  If all you need is something quick and dirty then I 
recommend taking the Opensync 0.23 code and just using that -- it works quite 
well, particularly with only two devices.

In practical terms, I think the most effective way to solve your problem would 
be to create a SyncML Client for GPE, as an alternative to gpesyncd.  The 
code already knows how to handle the vFormat objects, of course, so a SyncML 
client should be quite straightforward -- it would just mean implementing the 
SyncML protocol (I presume libsyncml would help with that).  That would then 
immediately work with several synchronisation frameworks, including 
SyncEvolution and OpenSync.  That would be a generally useful contribution, 
not just for your project.

> The command uidlist will return a list of pairs (uid,
> unix_last_modification_time) [2]
> The trouble is, the uid which is returned is the sqlite primary key
> which is not the "universal" UID returned by get [2].

That is deliberate.  The problem of deciding whether an entry has changed, or 
is new, or has been deleted and recreated, or has been changed in  two places 
is quite hard enough without mixing up the concept of the object's UID and 
how you access it through the protocol.  And dealing with (relatively small) 
integers is a lot easier than dealing with arbitrary strings.

The local "UID" used in gpesyncd only has meaning for the operations you are 
doing on the GPE device.  If a particular local UID has disappeared, it means 
the user has deleted it.  If a new local UID has appeared, it means the entry 
has been created.  If the local UID is still around, it is the same entry 
even though the user may have changed it (completely).

The vFormat UID tells you something different.  In particular, if a sync 
process has copied the event from some other device, the vFormat UID will be 
the same (if the local UID has changed then the event was deleted and was 
then recreated by a sync from somewhere else).  Also, GPE does not enforce 
that the vFormat UID really is unique.  How your sync program uses the 
difference between these UIDs is up to it: but together they give it more 
information to help it decide how to handle changes.

I recommend that, if you want to rely on the vFormat UID as the basis for your 
sync decisions, you keep a table of mappings between gpesyncd local UIDs and 
vFormat UIDs.  That is what all existing sync programs do, as far as I know.

> The internal primary key will never be the same on two devices, so it is
> useless for syncing. So I need to modify gpesyncd for my needs, and I
> have following options (in order of preference) :
> a) modify all commands so it will return the "universal" uid. This may
> break other synchronisation scripts (opensync ... but does it work anyway?)

No.  Don't do that.  The OpenSync GPE plugins work perfectly well, thank you, 
and I would not appreciate you breaking them.  And, by the way, backwards 
compatability is important -- make sure any output changes you make are not 
visible to earlier clients which do not know about them.  That means only add 
new commands or require some particular command or command line option to be 
specified to trigger the change.

> b) add new commands (uidlist2, get2, set2...)
> c) add a command line option to switch the behaviour of gpesync between
> "sqlite" ID and "uinversal" UID.

If you do this, I would recommend that you create new commands.  That way, if 
anyone else finds them useful, they can check the VERSION output and start 
using the new commands if they are present.  Personally, for the new 
UIDLISTWITHUID command I would add a third parameter to the UIDLIST output, 
so that the output is a triple (local UID, modification time, vFormat UID).  
I would also add GETBYUID, MODIFYBYUID, DELBYUID.  The hardest part may be 
getting MODIFYBYUID to work out how to parse the command line to separate the 
UID and the "BEGIN:VCARD" (I don't remember what the restrictions are on the 
format of a UID).

> What do you think ?

The most important thing: if you make changes which are backwards compatible 
make sure you increment the minor version number used in the VERSION command.  
If you make non-compatible changes (please don't!), increment the major 
version number.  That way the plugins can adapt if necessary and produce 
sensible error messages if running against older or newer versions of GPE.

Graham