[Smartphones-userland] [ticket #123] Ringtones and frameworkd playing mechanism

[Luca Capello]

Hi there!

I'm sorry for the long mail, but I hope it'll be useful!

It seems that FSO Trac isn't working right now, thus I cannot update
ticket #123 [1].  Basically, this is the upstream part for Debian bug
#495668 [2], whose solution is vital for the inclusion of frameworkd in
Debian main [3], which is something we want to do ASAP [4].

Please always keep the Debian bug cc:ed, no subscription required.

I'm not an expert coder, but I'll try to resume the situation.  The
ringtone playing mechanism in milestone3 is based on a GStreamer decoder
matrix, which means that for each file format you need to specify the
correct GStreamer decoder plugin [5].  Then a GStreamer pipeline is
created [6].  This pipeline is the same as the one gst-launch use [7]:
this is a big advantage, because you can perform every test you want
outside frameworkd.

The idea is simple (framework/subsystems/odeviced/audio.py:200 [8]):

  # gst-launch filesrc location=$FILENAME ! $DECODER ! alsasink

NB, ATM $FILENAME must be in $INSTALLPREFIX/share/sounds/.

The major problem here is that while GStreamer provides a decoder plugin
for most of the known audio formats, some of them (one for all, Ogg
Vorbis) need an "audio converter" before the alsasink module:

  # gst-launch filesrc location=$FILENAME ! $DECODER ! audioconvert ! alsasink

Trying to add Ogg support I discovered at first the playbin plugin [9],
which completely eliminates the pipeline, but at the same time adds too
much latency.  Then, my second approach was more "decoder" oriented, but
still a general one, the decodebin plugin [10]: with this approach, you
still have the pipeline, but without the need for a decoder matrix.  The
latest patch against milestone3 is attached [11].

Since I wanted to know about how much latency we're talking here, I
performed some tests with a specific protocol [12]: I (video)recorded
every call in different conditions (upstream and Debian images, idle or
not, with Arkanoid.sid or nothing2.ogg from Yue [13]) and then analysed
the results, in two different rounds to minimize the errors (raw data
attached).  The videos are available at [14] if someone wants to check
them and confirm my analyses.

The results are shown per the following: the 'vib' values is the second
at which vibration starts in the video, the 'ring' one is the second at
which we can hear the ringtone and the 'lat' one is the time between the
vibration starts and the ringtone (i.e. the latency).  Since the
camcorder was directly above the phone, you can also hear the "GSM
interference" [15].

The results:

  |--------------------+------+-------+------|
  |                    |  vib |  ring |  lat |
  |--------------------+------+-------+------|
  | a) on FSO          |      |       |      |
  |    1. idle         | 3.25 |  5.25 | 2.00 |
  |    2. active       | 2.75 |  5.50 | 2.75 |
  |--------------------+------+-------+------|
  | b) on Debian       |      |       |      |
  |  I- upstream       |      |       |      |
  |    3. idle         | 3.50 |  6.00 | 2.50 |
  |    4. active       | 3.00 |  6.00 | 3.00 |
  |  II- decodebin     |      |       |      |
  |    5. Arkanoid.sid | 2.50 |  6.50 | 4.00 |
  |    6. nothing2.ogg | 2.00 |  7.25 | 5.25 |
  |--------------------+------+-------+------|

Some notes:

1) even with the upstream decoder matrix there's a latency of 2.5
   seconds between vibration and ringtone

2) Debian reflects quite well upstream (this is a good point)

3) indeed, the decodebin plugin adds more latency (at least 1.5 seconds
   more), even with upstream Arkanoid.sid

4) using Ogg Vorbis means a longer latency, at least 3 seconds more

Comments?  Suggestions?  Improvements to reduce the latency?

Thx, bye,
Gismo / Luca

Footnotes: 
[1] http://trac.freesmartphone.org/ticket/123
[2] http://bugs.debian.org/495668
[3] http://www.debian.org/doc/debian-policy/ch-archive.html#s-main
[4] http://lists.alioth.debian.org/pipermail/pkg-fso-maint/2008-October/000114.html
[5] http://gstreamer.freedesktop.org/documentation/plugins.html
[6] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstPipeline.html
[7] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/faq/html/chapter-using.html#using-gst-launch
[8] http://git.debian.org/?p=pkg-fso/fso-frameworkd.git;a=blob;f=framework/subsystems/odeviced/audio.py;h=3e0393115ee42f64823a8eefee3301d9251b04a6;hb=refs/heads/master
[9] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins-playbin.html
[10] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins-decodebin.html
[11] I can prepare a Debian test package if someone other wants to
     experiment with other formats
[12] hopefully my real life as a biologist is useful :-D
[13] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=495668#38
[14] http://pkg-fso.alioth.debian.org/ringtone-decodebin/
[15] I don't know the exact name

-------------- next part --------------
A non-text attachment was scrubbed...
Name: frameworkd_Milestone3_gstreamer-decodebin-ringtone_20081023-gismo.diff
Type: text/x-diff
Size: 2083 bytes
Desc: FSO frameworkd milestone3: use GStreamer decodebin plugin for ringtone
URL: <http://lists.linuxtogo.org/pipermail/smartphones-userland/attachments/20081024/37914243/attachment.diff>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: frameworkd_Milestone3_ringtone-measurements_20081023-gismo.txt
URL: <http://lists.linuxtogo.org/pipermail/smartphones-userland/attachments/20081024/37914243/attachment.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 314 bytes
Desc: not available
URL: <http://lists.linuxtogo.org/pipermail/smartphones-userland/attachments/20081024/37914243/attachment.pgp>