Search Results: "bap"

7 October 2014

Andrea Veri: The GNOME Infrastructure is now powered by FreeIPA!

As preannounced here the GNOME Infrastructure switched to a new Account Management System which is reachable at https://account.gnome.org. All the details will follow. Introduction It s been a while since someone actually touched the underlying authentication infrastructure that powers the GNOME machines. The very first setup was originally configured by Jonathan Blandford (jrb) who configured an OpenLDAP istance with several customized schemas. (pServer fields in the old CVS days, pubAuthorizedKeys and GNOME modules related fields in recent times) While OpenLDAP-server was living on the GNOME machine called clipboard (aka ldap.gnome.org) the clients were configured to synchronize users, groups, passwords through the nslcd daemon. After several years Jeff Schroeder joined the Sysadmin Team and during one cold evening (date is Tue, February 1st 2011) spent some time configuring SSSD to replace the nslcd daemon which was missing one of the most important SSSD features: caching. What surely convinced Jeff to adopt SSSD (a very new but promising sofware at that time as the first release happened right before 2010 s Christmas) and as the commit log also states ( New sssd module for ldap information caching ) was SSSD s caching feature. It was enough for a certain user to log in once and the /var/lib/sss/db directory was populated with its login information preventing the LDAP daemon in charge of picking up login details (from the LDAP server) to query the LDAP server itself every single time a request was made against it. This feature has definitely helped in many occasions especially when the LDAP server was down for a particular reason and sysadmins needed to access a specific machine or service: without SSSD this wasn t ever going to work and sysadmins were probably going to be locked out from the machines they were used to manage. (except if you still had /etc/passwd , /etc/group and /etc/shadow entries as fallback) Things were working just fine except for a few downsides that appeared later on:
  1. the web interface (view) on our LDAP user database was managed by Mango, an outdated tool which many wanted to rewrite in Django that slowly became a huge dinosaur nobody ever wanted to look into again
  2. the Foundation membership information were managed through a MySQL database, so two databases, two sets of users unrelated to each other
  3. users were not able to modify their own account information on their own but even a single e-mail change required them to mail the GNOME Accounts Team which was then going to authenticate their request and finally update the account.
Today s infrastructure changes are here to finally say the issues outlined at (1, 2, 3) are now fixed. What has changed? The GNOME Infrastructure is now powered by Red Hat s FreeIPA which bundles several FOSS softwares into one big bundle all surrounded by an easy and intuitive web UI that will help users update their account information on their own without the need of the Accounts Team or any other administrative entity. Users will also find two custom fields on their Overview page, these being Foundation Member since and Last Renewed on date . As you may have understood already we finally managed to migrate the Foundation membership database into LDAP itself to store the information we want once and for all. As a side note it might be possible that some users that were Foundation members in the past won t find any detail stored on the Foundation fields outlined above. That is actually expected as we were able to migrate all the current and old Foundation members that had an LDAP account registered at the time of the migration. If that s your case and you still would like the information to be stored on the new setup please get in contact with the Membership Committee at stating so. Where can I get my first login credentials? Let s make a little distinction between users that previously had access to Mango (usually maintainers) and users that didn t. If you were used to access Mango before you should be able to login on the new Account Management System by entering your GNOME username and the password you were used to use for loggin in into Mango. (after loggin in the very first time you will be prompted to update your password, please choose a strong password as this account will be unique across all the GNOME Infrastructure) If you never had access to Mango, you lost your password or the first time you read the word Mango on this post you thought why is he talking about a fruit now? you should be able to reset it by using the following command:
ssh -l yourgnomeuserid account.gnome.org
The command will start an SSH connection between you and account.gnome.org, once authenticated (with the SSH key you previously had registered on our Infrastructure) you will trigger a command that will directly send your brand new password on the e-mail registered for your account. From my tests seems GMail sees the e-mail as a phishing attempt probably because the body contains the word password twice. That said if the e-mail won t appear on your INBOX, please double-check your Spam folder. Now that Mango is gone how can I request a new account? With Mango we used to have a form that automatically e-mailed the maintainer of the selected GNOME module which was then going to approve / reject the request. From there and in the case of a positive vote from the maintainer the Accounts Team was going to create the account itself. With the recent introduction of a commit robot directly on l10n.gnome.org the number of account requests reduced its numbers. In addition to that users will now be able to perform pretty much all the needed maintenance on their accounts themselves. That said and while we will probably work on building a form in the future we feel that requesting accounts can definitely be achieved directly by mailing the Accounts Team itself which will mail the maintainer of the respective module and create the account. As just said the number of account creations has become very low and the queue is currently clear. The documentation has been updated to reflect these changes at: https://wiki.gnome.org/AccountsTeam
https://wiki.gnome.org/AccountsTeam/NewAccounts I was used to have access to a specific service but I don t anymore, what should I do? The migration of all the user data and ACLs has been massive and I ve been spending a lot of time reviewing the existing HBAC rules trying to spot possible errors or misconfigurations. If you happen to not being able to access a certain service as you were used to in the past, please get in contact with the Sysadmin Team. All the possible ways to contact us are available at https://wiki.gnome.org/Sysadmin/Contact. What is missing still? Now that the Foundation membership information has been moved to LDAP I ll be looking at porting some of the existing membership scripts to it. What I managed to port already are welcome e-mails for new or existing members. (renewals) Next step will be generating a membership page from LDAP (to populate http://www.gnome.org/foundation/membership) and all the your-membership-is-going-to-lapse e-mails that were being sent till today. Other news /home/users mount on master.gnome.org You will notice that loggin in into master.gnome.org will result in your home directory being empty, don t worry, you did not lose any of your files but master.gnome.org is now currently hosting your home directories itself. As you may have been aware of adding files to the public_html directory on master resulted in them appearing on your people.gnome.org/~userid space. That was unfortunately expected as both master and webapps2 (the machine serving people.gnome.org s webspaces) were mounting the same GlusterFS share. We wanted to prevent that behaviour to happen as we wanted to know who has access to what resource and where. From today master s home directories will be there just as a temporary spot for your tarballs, just scp and use ftpadmin against them, that should be all you need from master. If you are interested in receiving or keeping using your people.gnome.org s webspace please mail <accounts AT gnome DOT org> stating so. Other news a shiny and new error 500 page has been deployed Thanks to Magdalen Berns (magpie) a new error 500 web page has been deployed on all the Apache istances we host. The page contains an iframe of status.gnome.org and will appear every single time the web server behind the service you are trying to reach will be unreachable for maintenance or other purposes. While I hope you won t see the page that often you can still enjoy it at https://static.gnome.org/error-500/500.html. Make sure to whitelist status.gnome.org on your browser as it currently loads it without https. (as the service is currently hosted on OpenShift which provides us with a *.rhcloud.com wildcard certificate, which differs from the CN the browser would expect it to be) Updates UPDATE on status.gnome.org s SSL certificate: the certificate has been provisioned and it should result in the 500 s page to be displayed correctly with no warnings from your browser. UPDATE from Adam Young on Kerberos ports being closed on many DC s firewalls:
The next version of upstream MIT Kerberos will have support for fetching a ticket via ports 443 and marshalling the request over HTTPS. We ll need to run a proxy on the server side, but we should be able to make it work: Read up here
http://adam.younglogic.com/2014/06/kerberos-firewalls

5 October 2014

Thomas Goirand: OpenStack packaging activity: September 2014

I decided I d post this monthly. It may be a bit boring, sorry, but I think it s a nice thing to have this public. The log starts on the 6th, because on the 4th I was back from Debconf (after a day in San Francisco, plus 20 hours of traveling and 15 hours of time gap). It is to be noted that every time something is uploaded in Debian for Icehouse (in Sid), or for Juno (in Experimental), there s also a corresponding backport produced for Wheezy. Saturday 6th & Sunday 7th:
packaged libjs-twitter-bootstrap-wizard (in new queue)
Uploaded python-pint after reviewing the debian/copyright
Worked on updating python-eventlet in Experimental, and adding Python3 support. It seems Python3 support isn t ready yet, so I will probably remove that feature from the package update.
Tried to apply the Django 1.7 patches for python-django-bootstrap-form. They didn t work, but Raphael came back on Monday morning with new versions
of the patches, which should be good this time.
Helped the DSA (Debian System Administrators) with the Debian OpenStack cloud. It s looking good and working now (note: I helped them during Debconf 14).
Started a page about adding more tasksel tasks: https://wiki.debian.org/tasksel/MoreTasks. It s looking like Joey Hess is adding new tasks by default in Tasksel, with OpenStack compute node and OpenStack proxy node . It will be nice to have them in the default Debian installer! :)
Packaged and uploaded python-dib-utils, now in NEW queue. Monday 8th:
Uploaded fixed python-django-bootstrap-form with patch for Django 1.7.
Packaged and uploaded python-pysaml2.
Finilized and uploaded python-jingo which is needed for python-django-compressor unit tests
Finalized and uploaded python-coffin which is needed for python-django-compressor unit tests
Worked on running the unit tests for python-django-compressor, as I needed to know if it could work with Django 1.7. It was hard to find the correct way to run the unit tests, but finally, they all passed. I will add the unit tests once coffin and jingo will be accepted in Sid.
Applied patches in the Debian BTS for python-django-openstack-auth and Django 1.7. Uploaded the fixed package.
Fixed python-django-pyscss compat with Django 1.7, uploaded the result.
Updated keystone to Juno b3.
Built Wheezy backports of some JS libs needed for Horizon in Juno, which I already uploaded to Sid last summer:
o libjs-twitter-bootstrap-datepicker
o libjs-jquery.quicksearch
o libjs-spin.js
Upstreamed the Django 1.7 patch for python-django-openstack-auth: https://review.openstack.org/119972 Tuesday 9:
Updated and uploaded Swift 2.1.0. Added swift-object-expirer package to it, together with init script. Wednesday 10:
Basically, cleaned the Debian BTS of almost all issues today :P
Added it.po update to nova (Closes: #758305).
Backported libvirt 1.2.7 to Wheezy, to be able to close this bug: https://bugs.debian.org/757548 (eg: changed dependency from libvirt-bin to libvirt-daemon-system)
Uploaded the fixed nova package using libvirt-daemon-system
Upgraded python-trollius to 1.0.1
Fixed tuskar-ui to work with Django 1.7. Disabled pep8 tests during build. Added build-conflicts: python-unittest2.
Fixed python-django-compressor for Django 1.7, and now running unit tests with it, after python-coffin and python-jingo got approved in Sid by FTP masters.
Fixed python-xstatic wrong upstream URLs.
Added it.po debconf translation to Designate.
Added de.po debconf translation to Tuskar.
Fixed copyright holders in python-xstatic-rickshaw
Added python-passlib as dependency for python-cinder. Remaining 3 issues in the BTS: ceilometer FTBFS, Horizon unit test with Django 1.7, Designate fail to install. All of the 3 are harder to fix, and I may try to do so later this week. Thursday 11:
Fixed python-xstatic-angular and python-xstatic-angular-mock to deal with the new libjs-angularjs version (closes 2 Debian RC bugs: uninstallable).
Fixed ceilometer FTBFS (Closes rc bug) Friday 12:
Fixed wrong copyright file for libjs-twitter-bootstrap-wizard after the FTP masters told me, and reuploaded to Sid.
Reuploaded wrong upload of ceilometer (wrong hash for orig.tar.xz)
Packaged and uploaded python-xstatic-bootstrap-scss
Packaged and uploaded python-xstatic-font-awesome
Packaged and uploaded ntpstat Monday 15:
packaged and uploaded python-xstatic-jquery.bootstrap.wizard
Fixed python-xstatic-angular-cookies to use new libjs-angularjs version (fixed version dependencies)
Fixed Ceilometer FTBFS (Closes: #759967)
Backported all python-xtatic packages to Wheezy, including all dependencies. This includes backporting of a bunch of packages from nodejs which were needed as build-dependencies (around 70 packages ). Filed about 5 or 6 release critical bugs as some nodejs packages were not buildable as-is.
Fixed some too restrictive python-xstatic-angular* dependencies on the libjs-angularjs (the libjs-angularjs increased version). Tuesday 16:
Uploaded updates to Experimental:
o python-eventlet 0.15.2 (this one took a long time as it needed maintenance)
o oslo-config
o python-oslo.i18n
Uploaded to Sid:
o python-diskimage-builder 0.1.30-1
o python-django-pyscss 1.0.2-1
Fixed horizon libapache-mode-wsgi to be a dependency of openstack-dashboard-apache and not just openstack-dashboard (in both Icehouse & Juno).
Removed the last failing Django 1.7 unit test from Horizon. It doesn t seem relevant anyway.
Backported python-netaddr 0.7.12 to Wheezy (needed by oslo-config).
Started working on oslo.rootwrap, though it failed to build in Wheezy with a unit test failure. Wednesday 17:
To experimental:
o Uploaded oslo.rootwrap 1.3.0.0~a1. It needed a build-depends on iproute2 because of a new test.
o Uploaded python-oslo.utils 0.3.0
o Uploaded python-oslo.vmware 0.6.0, fixed sphinx-build conf.py and filed a bug about it: https://bugs.launchpad.net/oslo.vmware/+bug/1370370 plus emailed the commiter of the issue (which appeared 2 weeks ago).
o Uploaded python-pycadf 0.6.0
o Uploaded python-pyghmi 0.6.17
o Uploaded python-oslotest 1.1.0.0~a2, including patch for Wheezy, which I also submited upstream: https://review.openstack.org/122171/
o Uploaded glanceclient 0.14.0, added a patch to not use the embedded version of urllib3 in requests: https://review.openstack.org/122184
To Sid:
o Uploaded python-zake_0.1.6-1 Thesday 18:
Backported zeromq3-4.0.4+dfs, pyzmq-14.3.1, pyasn1-0.1.7, python-pyasn1-modules-0.0.5
Uploaded keystoneclient 0.10.1, fixed the saml2 unit tests which were broken using testtools >= 0.9.39. Filed bug, and warned code author: https://bugs.launchpad.net/python-keystoneclient/+bug/1371085
Uploade swiftclient 2.3.0 to experimental.
Uploaded ironicclient 0.2.1 to experimental.
Uploaded saharaclient, filed bug with saharaclient expecting an up and running keystone server: https://bugs.launchpad.net/python-saharaclient/+bug/1371177 Friday 19:
Uploaded keystone Juno b3, filed but about unit tests downloading with git, while no network access should be performed during package build (forbidden by
Debian policy)
Uploaded python-oslo.db 1.0.0 which I forgot in the dependency list, and which was needed for Neutron.
Uploaded nova 2014.2~b3-1 (added a new nova-serialproxy service daemon to the nova-consoleproxy) Saturday 20:
Uploaded Neutron Juno b3.
Uploaded python-retrying 1.2.3 (was missing from depends upload)
Uploaded Glance Juno b3.
Uploaded Cinder Juno b3.
Fixed python-xstatic-angular-mock which had a .pth packaged, as well as the data folder (uploaded debian release -3).
Fixed missing depends and build-conflicts in python-xstatic-jquery. Sunday 21:
Dropped python-pil & python-django-discover-runner from runtime Depends: of python-django-pyscss, as it s only needed for tests. It also created a conflicts, because python-django-discover-runner depends on python-unittest2 and horizon build-conflicts with it.
Forward-ported the Django 1.7 patches for Horizon. Opened new patch: https://review.openstack.org/122992 (since the old fix has gone away after a refactor of the unit test).
Uploaded Horizon Juno b3.
Applied https://review.openstack.org/#/c/122768/ to the keystone package, so that it doesn t do git clone of the keystoneclient during build.
Uploaded oslo.messaging 1.4.0.0 (which really is 1.4.0) to experimental
Uploaded oslo.messaging 1.4.0.0+really+1.3.1-1 to fix the issue in Sid/Jessie after the wrong upload (due to Zul wrong tagging of Keystone in the 2014.1.2 point release). Monday 22:
Uploaded ironic 2014.2~b3-1 to experimental
Uploaded heat 2014.2~b3-1 (with some fixes for sphinx doc build)
Uploaded ceilometer 2014.2~b3-1 to experimental
Uploaded openstack-doc-tools 0.19-1 to experimental
Uploaded openstack-trove 2014.2~b3-1 to experimental Tuesday 23:
Uploaded python-neutronclient with fixed version number for cliff and six. This missing requirement for cliff version produced an error in Trove, which I don t want to happen again.
Added fix for unit tests in Trove: https://review.openstack.org/#/c/123450/1,publish
Uploaded oslo.messaging 1.4.1 in Experimental, fixing the version conflicts with the one in Sid/Jessie. Thanks to Doug Hellman for doing the tagging. I will need to upload new versions of the following packages with the >= 1.4.1 depends:
> ceilometer
> ironic
> keystone
> neutron
> nova
> oslo-config
> oslo.rootwrap
> oslo.i18n
> python-pycadf
See http://lists.openstack.org/pipermail/openstack-dev/2014-September/046795.html for more explanation about the mess I m repairing
Uploaded designate Juno b3. Wednesday 24:
Uploaded oslosphinx 2.2.0.0
Uploaded update to django-openstack-auth (new last minute requirement for Horizon).
Uploaded final oslo-config package version 1.4.0.0 (really is 1.4.0)
Packaged and uploaded Sahara. This needs some tests by someone else as I don t even know how it works. Thuesday 25:
Uploaded python-keystonemiddleware 1.0.0-3, fixing CVE-2014-7144] TLS cert verification option not honoured in paste configs. https://bugs.debian.org/762748
Packaged and uploaded python-yaql, sent pull request for fixing print statements into Python3 compatible print function calls: https://github.com/ativelkov/yaql/pull/15
Packaged and uploaded python-muranoclient.
Started the packaging of Murano (not finished yet).
Uploaded python-keystoneclient 0.10.1-2 with the CVE-2014-7144 fix to Sid, with urgency=high. Uploaded 0.11.1-1 to Experimental.
Uploaded python-keystonemiddleware fix for CVE-2014-7144.
Uploaded openstack-trove 2014.2~b3-3 with last unit test fix from https://review.openstack.org/#/c/123450/ Friday 26:
Uploaded a fix for murano-agent, which makes it run as root.
Finished the packaging of Murano
Started packaging murano-dashboard, sent this patch to fix the wrong usage of the /usr/bin/coverage command: https://review.openstack.org/124444
Fixed wrong BASE_DIR in python-xstatic-angular and python-xstatic-angular-mock Saturday 27:
uploaded python-xstatic-boostrap-scss which I forgot to upload :(
uploaded python-pyscss 1.2.1 Sunday 28:
After a long investigation, I found out that the issue when installing the openstack-dasboard package was due to a wrong patch I did for Python 3.2 in Wheezy in python-pyscss. Corrected the patch from version 1.2.1-1, and uploaded version 1.2.1-2, the dashboard now installs correctly. \o/
Did a new version of an Horizon patch at https://review.openstack.org/122992/ to address Django 1.7 compat. Monday 29:
Uploaded new version of python-pyscss fixing the last issue with Python 3 (there was a release critical bug on it).
Uploaded fixup for python-django-openstack-auth fail to build in the Sid version, which was broken since the last upload of keystoneclient (which makes some of its API now as private).
Uploaded python-glance-store 0.1.8, including Ubuntu patch to fix unit tests.
Reviewed the packaging of python-strict-rfc3339 (see https://bugs.debian.org761152).
Uploaded Sheepdog with fix in the init script to start after corosync (Closes: #759216).
Uploaded pt_BR.po Brazilian Portuguese debconf templates translation for nova Icehouse in Sid (only commited it in Git for Juno).
Same for Glance. Tuesday 30:
Added Python3 support in python-django-appconf, uploaded to Sid
Upgraded to python-django-pyscss 1.0.3, and fixed broken unit tests with this new release under Django 1.7. Created pull request: https://github.com/fusionbox/django-pyscss/pull/22
Fixed designate requirements.txt in Sid (Icehouse) to allow SQLA 0.9.x. Uploaded resulting package to Sid.
Uploaded new Debian fix for python-tooz: kills memcached only if the package scripts started it (plus cleans .testrepository on clean).
Uploaded initial release of murano
Uploaded python-retrying with patch from Ubuntu to remove embedded copy of six.py code.
Uploaded python-oslo.i18n 1.0.0 to experimental (same as before, just bump of version #)
Uploaded python-oslo.utils 1.0.0 to experimental (same as before, just bump of version #)
Uploaded Keystone Juno RC1
Uploaded Glance Juno RC1

2 October 2014

Andrew Pollock: [opinion] On Islamaphobia

It's taken me a while to get sufficiently riled up about Australia's current Islamaphobia outbreak, but it's been brewing in me for a couple of weeks. For the record, I'm an Atheist, but I'll defend your right to practise your religion, just don't go pushing it on me, thank you very much. I'm also not a huge fan of Islam, because it does seem to lend itself to more violent extremism than other religions, and ISIS/ISIL/IS (whatever you want to call them) aren't doing Islam any favours at the moment. I'm against extremism of any stripes though. The Westboro Baptists are Christian extremists. They just don't go around killing people. I'm also not a big fan of the burqa, but again, I'll defend a Muslim woman's right to choose to wear one. They key point here is choice. I got my carpets cleaned yesterday by an ethnic couple. I like accents, and I was trying to pick theirs. I thought they may have been Turkish. It turned out they were Kurdish. Whenever I hear "Kurd" I habitually stick "Bosnian" in front of it after the Bosnian War that happened in my childhood. Turns out I wasn't listening properly, and that was actually "Serb". Now I feel dumb, but I digress. I got chatting with the lady while her husband did the work. I got a refresher on where most Kurds are/were (Northern Iraq) and we talked about Sunni versus Shia Islam, and how they differed. I learned a bit yesterday, and I'll have to have a proper read of the Wikipedia article I just linked to, because I suspect I'll learn a lot more. We briefly talked about burqas, and she said that because they were Sunni, they were given the choice, and they chose not to wear it. That's the sort of Islam that I support. I suspect a lot of the women running around in burqas don't get a lot of say in it, but I don't think banning it outright is the right solution to that. Those women need to feel empowered enough to be able to cast off their burqas if that's what they want to do. I completely agree that a woman in a burqa entering a secure place (for example Parliament House) needs to be identifiable (assuming that identification is verified for all entrants to Parliament House). If it's not, and they're worried about security, that's what the metal detectors are for. I've been to Dubai. I've seen how they handle women in burqas at passport control. This is an easily solvable problem. You don't have to treat burqa-clad women as second class citizens and stick them in a glass box. Or exclude them entirely.

27 September 2014

DebConf team: Wrapping up DebConf14 (Posted by Paul Wise, Donald Norwood)

The annual Debian developer meeting took place in Portland, Oregon, 23 to 31 August 2014. DebConf14 attendees participated in talks, discussions, workshops and programming sessions. Video teams captured a lot of the main talks and discussions for streaming for interactive attendees and for the Debian video archive. Between the video, presentations, and handouts the coverage came from the attendees in blogs, posts, and project updates. We ve gathered a few articles for your reading pleasure: Gregor Herrmann and a few members of the Debian Perl group had an informal unofficial pkg-perl micro-sprint and were very productive. Vincent Sanders shared an inspired gift in the form of a plaque given to Russ Allbery in thanks for his tireless work of keeping sanity in the Debian mailing lists. Pictures of the plaque and design scheme are linked in the post. Vincent also shared his experiences of the conference and hopes the organisers have recovered. Noah Meyerhans adventuring to Debian by train, (Inter)netted some interesting IPv6 data for future road and railwarriors. Hideki Yamane sent a gentle reminder for English speakers to speak more slowly. Daniel Pocock posted of GSoC talks at DebConf14, highlights include the Java Project Dependency Builder and the WebRTC JSCommunicator. Thomas Goirand gives us some insight into a working task list of accomplishments and projects he was able to complete at DebConf14, from the OpenStack discussion to tasksel talks, and completion of some things started last year at DebConf13. Antonio Terceiro blogged about debci and the Debian Continuous Integration project, Ruby, Redmine, and Noosfero. His post also shares the atmosphere of being able to interact directly with peers once a year. Stefano Zacchiroli blogged about a talk he did on debsources which now has its own HACKING file. Juliana Louback penned: DebConf 2014 and How I Became a Debian Contributor. Elizabeth Krumbach Joseph s in-depth summary of DebConf14 is a great read. She discussed Debian Validation & CI, debci and the Continuous Integration project, Automated Validation in Debian using LAVA, and Outsourcing webapp maintenance. Lucas Nussbaum by way of a blog post releases the very first version of Debian Trivia modelled after the TCP/IP Drinking Game. Fran ois Marier s shares additional information and further discussion on Outsourcing your webapp maintenance to Debian. Joachim Breitner gave a talk on Haskell and Debian, created a new tool for binNMUs for Haskell packages which runs via cron job. The output is available for Haskell and for OCaml, and he still had a small amount of time to go dancing. Jaldhar Harshad Vyas was not able to attend DebConf this year, but he did tune in to the videos made available by the video team and gives an insightful viewpoint to what was being seen. J r my Bobbio posted about Reproducible builds in Debian in his recap of DebConf14. One of the topics at hand involved defining a canonical path where packages must be built and a BOF discussion on reproducible builds from where the conversation moved to discussions in both Octave and Groff. New helpers dh_fixmtimes and dh_genbuildinfo were added to BTS. The .buildinfo format has been specified on the wiki and reviewed. Lots of work is being done in the project, interested parties can help with the TODO list or join the new IRC channel #debian-reproducible on irc.debian.org. Steve McIntyre posted a Summary from the d-i / debian-cd BoF at DC14, with some of the session video available online. Current jessie D-I needs some help with the testing on less common architectures and languages, and release scheduling could be improved. Future plans: Switching to a GUI by default for jessie, a default desktop and desktop choice, artwork, bug fixes and new architecture support. debian-cd: Things are working well. Improvement discussions are on selecting which images to make I.E. netinst, DVD, et al., debian-cd in progress with http download support, Regular live test builds, Other discussions and questions revolve around which ARM platforms to support, specially-designed images, multi-arch CDs, and cloud-init based images. There is also a call for help as the team needs help with testing, bug-handling, and translations. Holger Levsen reports on feedback about the feedback from his LTS talk at DebConf14. LTS has been perceived well, fits a demand, and people are expecting it to continue; however, this is not without a few issues as Holger explains in greater detail the lacking gatekeeper mechanisms, and how contributions are needed from finance to uploads. In other news the security-tracker is now fixed to know about old stable. Time is short for that fix as once jessie is released the tracker will need to support stable, oldstable which will be wheezy, and oldoldstable. Jonathan McDowell s summary of DebConf14 includes a fair perspective of the host city and the benefits of planning of a good DebConf14 location. He also talks about the need for facetime in the Debian project as it correlates with and improves everyone s ability to work together. DebConf14 also provided the chance to set up a hard time frame for removing older 1024 bit keys from Debian keyrings. Steve McIntyre posted a Summary from the State of the ARM BoF at DebConf14 with updates on the 3 current ports armel, armhf and arm64. armel which targets the ARM EABI soft-float ARMv4t processor may eventually be going away, while armhf which targets the ARM EABI hard-float ARMv7 is doing well as the cross-distro standard. Debian is has moved to a single armmp kernel flavour using Device Tree Blobs and should be able to run on a large range of ARMv7 hardware. The arm64 port recently entered the main archive and it is hoped to release with jessie with 2 official builds hosted at ARM. There is talk of laptop development with an arm64 CPU. Buildds and hardware are mentioned with acknowledgements for donated new machines, Banana Pi boards, and software by way of ARM s DS-5 Development Studio - free for all Debian Developers. Help is needed! Join #debian-arm on irc.debian.org and/or the debian-arm mailing list. There is an upcoming Mini-DebConf in November 2014 hosted by ARM in Cambridge, UK. Tianon Gravi posted about the atmosphere and contrast between an average conference and a DebConf. Joseph Bisch posted about meeting his GSOC mentors, attending and contributing to a keysigning event and did some work on debmetrics which is powering metrics.debian.net. Debmetrics provides a uniform interface for adding, updating, and viewing various metrics concerning Debian. Harlan Lieberman-Berg s DebConf Retrospective shared the feel of DebConf, and detailed some of the work on debugging a build failure, work with the pkg-perl team on a few uploads, and work on a javascript slowdown issue on codeeditor. Ana Guerrero L pez reflected on Ten years contributing to Debian.

19 September 2014

Paul Tagliamonte: Docker PostgreSQL Foreign Data Wrapper

For the tl;dr: Docker FDW is a thing. Star it, hack it, try it out. File bugs, be happy. If you want to see what it's like to read, there's some example SQL down below. The question is first, what the heck is a PostgreSQL Foreign Data Wrapper? PostgreSQL Foreign Data Wrappers are plugins that allow C libraries to provide an adaptor for PostgreSQL to talk to an external database. Some folks have used this to wrap stuff like MongoDB, which I always found to be hilarous (and an epic hack). Enter Multicorn During my time at PyGotham, I saw a talk from Wes Chow about something called Multicorn. He was showing off some really neat plugins, such as the git revision history of CPython, and parsed logfiles from some stuff over at Chartbeat. This basically blew my mind. All throughout the talk I was coming up with all sorts of things that I wanted to do -- this whole library is basically exactly what I've been dreaming about for years. I've always wanted to provide a SQL-like interface into querying API data, joining data cross-API using common crosswalks, such as using Capitol Words to query for Legislators, and use the bioguide ids to JOIN against the congress api to get their Twitter account names. My first shot was to Multicorn the new Open Civic Data API I was working on, chuckled and put it aside as a really awesome hack. Enter Docker It wasn't until tianon connected the dots for me and suggested a Docker FDW did I get really excited. Cue a few hours of hacking, and I'm proud to say -- here's Docker FDW. This lets us ask all sorts of really interesting questions out of the API, and might even help folks writing webapps avoid adding too much Docker-aware logic. Abstractions can be fun! Setting it up I'm going to assume you have a working Multicorn, PostgreSQL and Docker setup (including adding the postgres user to the docker group) So, now let's pop open a psql session. Create a database (I called mine dockerfdw, but it can be anything), and let's create some tables. Before we create the tables, we need to let PostgreSQL know where our objects are. This takes a name for the server, and the Python importable path to our FDW.
CREATE SERVER docker_containers FOREIGN DATA WRAPPER multicorn options (
    wrapper 'dockerfdw.wrappers.containers.ContainerFdw');
CREATE SERVER docker_image FOREIGN DATA WRAPPER multicorn options (
    wrapper 'dockerfdw.wrappers.images.ImageFdw');
Now that we have the server in place, we can tell PostgreSQL to create a table backed by the FDW by creating a foreign table. I won't go too much into the syntax here, but you might also note that we pass in some options - these are passed to the constructor of the FDW, letting us set stuff like the Docker host.
CREATE foreign table docker_containers (
    "id"          TEXT,
    "image"       TEXT,
    "name"        TEXT,
    "names"       TEXT[],
    "privileged"  BOOLEAN,
    "ip"          TEXT,
    "bridge"      TEXT,
    "running"     BOOLEAN,
    "pid"         INT,
    "exit_code"   INT,
    "command"     TEXT[]
) server docker_containers options (
    host 'unix:///run/docker.sock'
);
CREATE foreign table docker_images (
    "id"              TEXT,
    "architecture"    TEXT,
    "author"          TEXT,
    "comment"         TEXT,
    "parent"          TEXT,
    "tags"            TEXT[]
) server docker_image options (
    host 'unix:///run/docker.sock'
);
And, now that we have tables in place, we can try to learn something about the Docker containers. Let's start with something fun - a join from containers to images, showing all image tag names, the container names and the ip of the container (if it has one!).
SELECT docker_containers.ip, docker_containers.names, docker_images.tags
  FROM docker_containers
  RIGHT JOIN docker_images
  ON docker_containers.image=docker_images.id;
     ip                   names                               tags                   
-------------+-----------------------------+-----------------------------------------
                                              ruby:latest 
                                              paultag/vcs-mirror:latest 
                /de-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /ny-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /ar-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
 172.17.0.47    /ms-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
 172.17.0.46    /nc-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /ia-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /az-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /oh-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /va-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
 172.17.0.41    /wa-openstates-to-ocd         sunlightlabs/scrapers-us-state:latest 
                /jovial_poincare              <none>:<none> 
                /jolly_goldstine              <none>:<none> 
                /cranky_torvalds              <none>:<none> 
                /backstabbing_wilson          <none>:<none> 
                /desperate_hoover             <none>:<none> 
                /backstabbing_ardinghelli     <none>:<none> 
                /cocky_feynman                <none>:<none> 
                                              paultag/postgres:latest 
                                              debian:testing 
                                              paultag/crank:latest 
                                              <none>:<none> 
                                              <none>:<none> 
                /stupefied_fermat             hackerschool/doorbot:latest 
                /focused_euclid               debian:unstable 
                /focused_babbage              debian:unstable 
                /clever_torvalds              debian:unstable 
                /stoic_tesla                  debian:unstable 
                /evil_torvalds                debian:unstable 
                /foo                          debian:unstable 
(31 rows)
OK, let's see if we can bring this to the next level now. I finally got around to implementing INSERT and DELETE operations, which turned out to be pretty simple to do. Check this out:
DELETE FROM docker_containers;
DELETE 1
This will do a stop + kill after a 10 second hang behind the scenes. It's actually a lot of fun to spawn up a container and terminate it from PostgreSQL.
INSERT INTO docker_containers (name, image) VALUES ('hello', 'debian:unstable') RETURNING id;
                                id                                
------------------------------------------------------------------
 0a903dcf5ae10ee1923064e25ab0f46e0debd513f54860beb44b2a187643ff05
INSERT 0 1
(1 row)
Spawning containers works too - this is still very immature and not super practical, but I figure while I'm showing off, I might as well go all the way.
SELECT ip FROM docker_containers WHERE id='0a903dcf5ae10ee1923064e25ab0f46e0debd513f54860beb44b2a187643ff05';
     ip      
-------------
 172.17.0.12
(1 row)
Success! This is just a taste of what's to come, so please feel free to hack on Docker FDW, tweet me @paultag, file bugs / feature requests. It's currently a bit of a hack, and it's something that I think has long-term potential after some work goes into making sure that this is a rock solid interface to the Docker API.

10 September 2014

Lucas Nussbaum: Will the packages you rely on be part of Debian Jessie?

The start of the jessie freeze is quickly approaching, so now is a good time to ensure that packages you rely on will the part of the upcoming release. Thanks to automated removals, the number of release critical bugs has been kept low, but this was achieved by removing many packages from jessie: 841 packages from unstable are not part of jessie, and won t be part of the release if things don t change. It is actually simple to check if you have packages installed locally that are part of those 841 packages:
  1. apt-get install how-can-i-help (available in backports if you don t use testing or unstable)
  2. how-can-i-help --old
  3. Look at packages listed under Packages removed from Debian testing and Packages going to be removed from Debian testing
Then, please fix all the bugs :-) Seriously, not all RC bugs are hard to fix. A good starting point to understand why a package is not part of jessie is tracker.d.o. On my laptop, the two packages that are not part of jessie are the geeqie image viewer (which looks likely to be fixed in time), and josm, the OpenStreetMap editor, due to three RC bugs. It seems much harder to fix If you fix it in time for jessie, I ll offer you a $drink!

31 August 2014

Thorsten Alteholz: My Debian activities in August 2014

FTP assistant By pure chance I was able to accept 237 packages, the same number as last month. 33 times I contacted the maintainer to ask a question about a package and 55 times I had to reject a package. The reject number increased a bit as I also worked on packages that already got a note but had not been fully processed. In contrast I only filed three serious bugs this month. Currently there are about 200 packages still waiting in the NEW queue As the freeze for Jessie comes closer every day, I wonder whether all of them can be processed in time. So I don t mind if every maintainer checks the package again and maybe uploads an improved version that can be processed faster. Squeeze LTS This was my second month that I did some work for the Squeeze LTS initiative, started by Raphael Hertzog at Freexian All in all I got assigned a workload of 16.5h for August. I spent these hours to upload new versions of As last month I prepared these uploads on the basis of the corresponding DSAs for Wheezy. For these packages backporting the Wheezy patches to Squeeze was rather easy. I also had a look at python-django and eglibc. Although the python-django patches apply now, the package fails some tests and these issues need some further investigation. In case of eglibc, my small pbuilder didn t have enough resources and trying to build the package resulted in a full disk after more than three hours of work. For PHP5 Ond ej Sur (the real maintainer) suggested to use point releases of upstream instead of applying only patches. I am curious about how much effort is needed for this approach. Stay tuned, next month you will be told more details! Anyway, this is still a lot of fun and I hope I can finish python-django, eglibc and php5 in September. Other packages This month my meep packages plus mpb have been part of a small hdf5 transition. All five packages needed a small patch and a new upload. As the patch was already provided by Gilles Filippini, this was done rather quickly. Support If you would like to support my Debian work you could either be part of the Freexian initiative (see above) or consider to send some bitcoins to 1JHnNpbgzxkoNexeXsTUGS6qUp5P88vHej. Contact me at donation@alteholz.eu if you prefer another way to donate. Every kind of support is most appreciated.

30 August 2014

Francois Marier: Outsourcing your webapp maintenance to Debian

Modern web applications are much more complicated than the simple Perl CGI scripts or PHP pages of the past. They usually start with a framework and include lots of external components both on the front-end and on the back-end. Here's an example from the Node.js back-end of a real application:
$ npm list   wc -l
256
What if one of these 256 external components has a security vulnerability? How would you know and what would you do if of your direct dependencies had a hard-coded dependency on the vulnerable version? It's a real problem and of course one way to avoid this is to write everything yourself. But that's neither realistic nor desirable. However, it's not a new problem. It was solved years ago by Linux distributions for C and C++ applications. For some reason though, this learning has not propagated to the web where the standard approach seems to be to "statically link everything". What if we could build on the work done by Debian maintainers and the security team?

Case study - the Libravatar project As a way of discussing a different approach to the problem of dependency management in web applications, let me describe the decisions made by the Libravatar project.

Description Libravatar is a federated and free software alternative to the Gravatar profile photo hosting site. From a developer point of view, it's a fairly simple stack: The service is split between the master node, where you create an account and upload your avatar, and a few mirrors, which serve the photos to third-party sites. Like with Gravatar, sites wanting to display images don't have to worry about a complicated protocol. In a nutshell, all that a site needs to do is hash the user's email and add that hash to a base URL. Where the federation kicks in is that every email domain is able to specify a different base URL via an SRV record in DNS. For example, francois@debian.org hashes to 7cc352a2907216992f0f16d2af50b070 and so the full URL is:
http://cdn.libravatar.org/avatar/7cc352a2907216992f0f16d2af50b070
whereas francois@fmarier.org hashes to 0110e86fdb31486c22dd381326d99de9 and the full URL is:
http://fmarier.org/avatar/0110e86fdb31486c22dd381326d99de9
due to the presence of an SRV record on fmarier.org.

Ground rules The main rules that the project follows is to:
  1. only use Python libraries that are in Debian
  2. use the versions present in the latest stable release (including backports)

Deployment using packages In addition to these rules around dependencies, we decided to treat the application as if it were going to be uploaded to Debian:
  • It includes an "upstream" Makefile which minifies CSS and JavaScript, gzips them, and compiles PO files (i.e. a "build" step).
  • The Makefile includes a test target which runs the unit tests and some lint checks (pylint, pyflakes and pep8).
  • Debian packages are produced to encode the dependencies in the standard way as well as to run various setup commands in maintainer scripts and install cron jobs.
  • The project runs its own package repository using reprepro to easily distribute these custom packages.
  • In order to update the repository and the packages installed on servers that we control, we use fabric, which is basically a fancy way to run commands over ssh.
  • Mirrors can simply add our repository to their apt sources.list and upgrade Libravatar packages at the same time as their system packages.

Results Overall, this approach has been quite successful and Libravatar has been a very low-maintenance service to run. The ground rules have however limited our choice of libraries. For example, to talk to our queuing system, we had to use the raw Python bindings to the C Gearman library instead of being able to use a nice pythonic library which wasn't in Debian squeeze at the time. There is of course always the possibility of packaging a missing library for Debian and maintaining a backport of it until the next Debian release. This wouldn't be a lot of work considering the fact that responsible bundling of a library would normally force you to follow its releases closely and keep any dependencies up to date, so you may as well share the result of that effort. But in the end, it turns out that there is a lot of Python stuff already in Debian and we haven't had to package anything new yet. Another thing that was somewhat scary, due to the number of packages that were going to get bumped to a new major version, was the upgrade from squeeze to wheezy. It turned out however that it was surprisingly easy to upgrade to wheezy's version of Django, Apache and Postgres. It may be a problem next time, but all that means is that you have to set a day aside every 2 years to bring everything up to date.

Problems The main problem we ran into is that we optimized for sysadmins and unfortunately made it harder for new developers to setup their environment. That's not very good from the point of view of welcoming new contributors as there is quite a bit of friction in preparing and testing your first patch. That's why we're looking at encoding our setup instructions into a Vagrant script so that new contributors can get started quickly. Another problem we faced is that because we use the Debian version of jQuery and minify our own JavaScript files in the build step of the Makefile, we were affected by the removal from that package of the minified version of jQuery. In our setup, there is no way to minify JavaScript files that are provided by other packages and so the only way to fix this would be to fork the package in our repository or (preferably) to work with the Debian maintainer and get it fixed globally in Debian. One thing worth noting is that while the Django project is very good at issuing backwards-compatible fixes for security issues, sometimes there is no way around disabling broken features. In practice, this means that we cannot run unattended-upgrades on our main server in case something breaks. Instead, we make use of apticron to automatically receive email reminders for any outstanding package updates. On that topic, it can occasionally take a while for security updates to be released in Debian, but this usually falls into one of two cases:
  1. You either notice because you're already tracking releases pretty well and therefore could help Debian with backporting of fixes and/or testing;
  2. or you don't notice because it has slipped through the cracks or there simply are too many potential things to keep track of, in which case the fact that it eventually gets fixed without your intervention is a huge improvement.
Finally, relying too much on Debian packaging does prevent Fedora users (a project that also makes use of Libravatar) from easily contributing mirrors. Though if we had a concrete offer, we would certainly look into creating the appropriate RPMs.

Is it realistic? It turns out that I'm not the only one who thought about this approach, which has been named "debops". The same day that my talk was announced on the DebConf website, someone emailed me saying that he had instituted the exact same rules at his company, which operates a large Django-based web application in the US and Russia. It was pretty impressive to read about a real business coming to the same conclusions and using the same approach (i.e. system libraries, deployment packages) as Libravatar. Regardless of this though, I think there is a class of applications that are particularly well-suited for the approach we've just described. If a web application is not your full-time job and you want to minimize the amount of work required to keep it running, then it's a good investment to restrict your options and leverage the work of the Debian community to simplify your maintenance burden. The second criterion I would look at is framework maturity. Given the 2-3 year release cycle of stable distributions, this approach is more likely to work with a mature framework like Django. After all, you probably wouldn't compile Apache from source, but until recently building Node.js from source was the preferred option as it was changing so quickly. While it goes against conventional wisdom, relying on system libraries is a sustainable approach you should at least consider in your next project. After all, there is a real cost in bundling and keeping up with external dependencies. This blog post is based on a talk I gave at DebConf 14: slides, video.

23 August 2014

Daniel Pocock: Want to be selected for Google Summer of Code 2015?

I've mentored a number of students in 2013 and 2014 for Debian and Ganglia and most of the companies I've worked with have run internships and graduate programs from time to time. GSoC 2014 has just finished and with all the excitement, many students are already asking what they can do to prepare and be selected in 2015. My own observation is that the more time the organization has to get to know the student, the more confident they can be selecting that student. Furthermore, the more time that the student has spent getting to know the free software community, the more easily they can complete GSoC. Here I present a list of things that students can do to maximize their chance of selection and career opportunities at the same time. These tips are useful for people applying for GSoC itself and related programs such as GNOME's Outreach Program for Women or graduate placements in companies. Disclaimers There is no guarantee that Google will run the program again in 2015 or any future year. There is no guarantee that any organization or mentor (including myself) will be involved until the official list of organizations is published by Google. Do not follow the advice of web sites that invite you to send pizza or anything else of value to prospective mentors. Following the steps in this page doesn't guarantee selection. That said, people who do follow these steps are much more likely to be considered and interviewed than somebody who hasn't done any of the things in this list. Understand what free software really is You may hear terms like free software and open source software used interchangeably. They don't mean exactly the same thing and many people use the term free software for the wrong things. Not all open source projects meet the definition of free software. Those that don't, usually as a result of deficiencies in their licenses, are fundamentally incompatible with the majority of software that does use genuinely free licenses. Google Summer of Code is about both writing and publishing your code and it is also about community. It is fundamental that you know the basics of licensing and how to choose a free license that empowers the community to collaborate on your code well after GSoC has finished. Please review the definition of free software early on and come back and review it from time to time. The The GNU Project / Free Software Foundation have excellent resources to help you understand what a free software license is and how it works to maximize community collaboration. Don't look for shortcuts There is no shortcut to GSoC selection and there is no shortcut to GSoC completion. The student stipend (USD $5,500 in 2014) is not paid to students unless they complete a minimum amount of valid code. This means that even if a student did find some shortcut to selection, it is unlikely they would be paid without completing meaningful work. If you are the right candidate for GSoC, you will not need a shortcut anyway. Are you the sort of person who can't leave a coding problem until you really feel it is fixed, even if you keep going all night? Have you ever woken up in the night with a dream about writing code still in your head? Do you become irritated by tedious or repetitive tasks and often think of ways to write code to eliminate such tasks? Does your family get cross with you because you take your laptop to Christmas dinner or some other significant occasion and start coding? If some of these statements summarize the way you think or feel you are probably a natural fit for GSoC. An opportunity money can't buy The GSoC stipend will not make you rich. It is intended to make sure you have enough money to survive through the summer and focus on your project. Professional developers make this much money in a week in leading business centers like New York, London and Singapore. When you get to that stage in 3-5 years, you will not even be thinking about exactly how much you made during internships. GSoC gives you an edge over other internships because it involves publicly promoting your work. Many companies still try to hide the potential of their best recruits for fear they will be poached or that they will be able to demand higher salaries. Everything you complete in GSoC is intended to be published and you get full credit for it. Imagine a young musician getting the opportunity to perform on the main stage at a rock festival. This is how the free software community works. It is a meritocracy and there is nobody to hold you back. Having a portfolio of free software that you have created or collaborated on and a wide network of professional contacts that you develop before, during and after GSoC will continue to pay you back for years to come. While other graduates are being screened through group interviews and testing days run by employers, people with a track record in a free software project often find they go straight to the final interview round. Register your domain name and make a permanent email address Free software is all about community and collaboration. Register your own domain name as this will become a focal point for your work and for people to get to know you as you become part of the community. This is sound advice for anybody working in IT, not just programmers. It gives the impression that you are confident and have a long term interest in a technology career. Choosing the provider: as a minimum, you want a provider that offers DNS management, static web site hosting, email forwarding and XMPP services all linked to your domain. You do not need to choose the provider that is linked to your internet connection at home and that is often not the best choice anyway. The XMPP foundation maintains a list of providers known to support XMPP. Create an email address within your domain name. The most basic domain hosting providers will let you forward the email address to a webmail or university email account of your choice. Configure your webmail to send replies using your personalized email address in the From header. Update your ~/.gitconfig file to use your personalized email address in your Git commits. Create a web site and blog Start writing a blog. Host it using your domain name. Some people blog every day, other people just blog once every two or three months. Create links from your web site to your other profiles, such as a Github profile page. This helps reinforce the pages/profiles that are genuinely related to you and avoid confusion with the pages of other developers. Many mentors are keen to see their students writing a weekly report on a blog during GSoC so starting a blog now gives you a head start. Mentors look at blogs during the selection process to try and gain insight into which topics a student is most suitable for. Create a profile on Github Github is one of the most widely used software development web sites. Github makes it quick and easy for you to publish your work and collaborate on the work of other people. Create an account today and get in the habbit of forking other projects, improving them, committing your changes and pushing the work back into your Github account. Github will quickly build a profile of your commits and this allows mentors to see and understand your interests and your strengths. In your Github profile, add a link to your web site/blog and make sure the email address you are using for Git commits (in the ~/.gitconfig file) is based on your personal domain. Start using PGP Pretty Good Privacy (PGP) is the industry standard in protecting your identity online. All serious free software projects use PGP to sign tags in Git, to sign official emails and to sign official release files. The most common way to start using PGP is with the GnuPG (GNU Privacy Guard) utility. It is installed by the package manager on most Linux systems. When you create your own PGP key, use the email address involving your domain name. This is the most permanent and stable solution. Print your key fingerprint using the gpg-key2ps command, it is in the signing-party package on most Linux systems. Keep copies of the fingerprint slips with you. This is what my own PGP fingerprint slip looks like. You can also print the key fingerprint on a business card for a more professional look. Using PGP, it is recommend that you sign any important messages you send but you do not have to encrypt the messages you send, especially if some of the people you send messages to (like family and friends) do not yet have the PGP software to decrypt them. If using the Thunderbird (Icedove) email client from Mozilla, you can easily send signed messages and validate the messages you receive using the Enigmail plugin. Get your PGP key signed Once you have a PGP key, you will need to find other developers to sign it. For people I mentor personally in GSoC, I'm keen to see that you try and find another Debian Developer in your area to sign your key as early as possible. Free software events Try and find all the free software events in your area in the months between now and the end of the next Google Summer of Code season. Aim to attend at least two of them before GSoC. Look closely at the schedules and find out about the individual speakers, the companies and the free software projects that are participating. For events that span more than one day, find out about the dinners, pub nights and other social parts of the event. Try and identify people who will attend the event who have been GSoC mentors or who intend to be. Contact them before the event, if you are keen to work on something in their domain they may be able to make time to discuss it with you in person. Take your PGP fingerprint slips. Even if you don't participate in a formal key-signing party at the event, you will still find some developers to sign your PGP key individually. You must take a photo ID document (such as your passport) for the other developer to check the name on your fingerprint but you do not give them a copy of the ID document. Events come in all shapes and sizes. FOSDEM is an example of one of the bigger events in Europe, linux.conf.au is a similarly large event in Australia. There are many, many more local events such as the Debian France mini-DebConf in Lyon, 2015. Many events are either free or free for students but please check carefully if there is a requirement to register before attending. On your blog, discuss which events you are attending and which sessions interest you. Write a blog during or after the event too, including photos. Quantcast generously hosted the Ganglia community meeting in San Francisco, October 2013. We had a wild time in their offices with mini-scooters, burgers, beers and the Ganglia book. That's me on the pink mini-scooter and Bernard Li, one of the other Ganglia GSoC 2014 admins is on the right. Install Linux GSoC is fundamentally about free software. Linux is to free software what a tree is to the forest. Using Linux every day on your personal computer dramatically increases your ability to interact with the free software community and increases the number of potential GSoC projects that you can participate in. This is not to say that people using Mac OS or Windows are unwelcome. I have worked with some great developers who were not Linux users. Linux gives you an edge though and the best time to gain that edge is now, while you are a student and well before you apply for GSoC. If you must run Windows for some applications used in your course, it will run just fine in a virtual machine using Virtual Box, a free software solution for desktop virtualization. Use Linux as the primary operating system. Here are links to download ISO DVD (and CD) images for some of the main Linux distributions: If you are nervous about getting started with Linux, install it on a spare PC or in a virtual machine before you install it on your main PC or laptop. Linux is much less demanding on the hardware than Windows so you can easily run it on a machine that is 5-10 years old. Having just 4GB of RAM and 20GB of hard disk is usually more than enough for a basic graphical desktop environment although having better hardware makes it faster. Your experiences installing and running Linux, especially if it requires some special effort to make it work with some of your hardware, make interesting topics for your blog. Decide which technologies you know best Personally, I have mentored students working with C, C++, Java, Python and JavaScript/HTML5. In a GSoC program, you will typically do most of your work in just one of these languages. From the outset, decide which language you will focus on and do everything you can to improve your competence with that language. For example, if you have already used Java in most of your course, plan on using Java in GSoC and make sure you read Effective Java (2nd Edition) by Joshua Bloch. Decide which themes appeal to you Find a topic that has long-term appeal for you. Maybe the topic relates to your course or maybe you already know what type of company you would like to work in. Here is a list of some topics and some of the relevant software projects:
  • System administration, servers and networking: consider projects involving monitoring, automation, packaging. Ganglia is a great community to get involved with and you will encounter the Ganglia software in many large companies and academic/research networks. Contributing to a Linux distribution like Debian or Fedora packaging is another great way to get into system administration.
  • Desktop and user interface: consider projects involving window managers and desktop tools or adding to the user interface of just about any other software.
  • Big data and data science: this can apply to just about any other theme. For example, data science techniques are frequently used now to improve system administration.
  • Business and accounting: consider accounting, CRM and ERP software.
  • Finance and trading: consider projects like R, market data software like OpenMAMA and connectivity software (Apache Camel)
  • Real-time communication (RTC), VoIP, webcam and chat: look at the JSCommunicator or the Jitsi project
  • Web (JavaScript, HTML5): look at the JSCommunicator
Before the GSoC application process begins, you should aim to learn as much as possible about the theme you prefer and also gain practical experience using the software relating to that theme. For example, if you are attracted to the business and accounting theme, install the PostBooks suite and get to know it. Maybe you know somebody who runs a small business: help them to upgrade to PostBooks and use it to prepare some reports. Make something Make some small project, less than two week's work, to demonstrate your skills. It is important to make something that somebody will use for a practical purpose, this will help you gain experience communicating with other users through Github. For an example, see the servlet Juliana Louback created for fixing phone numbers in December 2013. It has since been used as part of the Lumicall web site and Juliana was selected for a GSoC 2014 project with Debian. There is no better way to demonstrate to a prospective mentor that you are ready for GSoC than by completing and publishing some small project like this yourself. If you don't have any immediate project ideas, many developers will also be able to give you tips on small projects like this that you can attempt, just come and ask us on one of the mailing lists. Ideally, the project will be something that you would use anyway even if you do not end up participating in GSoC. Such projects are the most motivating and rewarding and usually end up becoming an example of your best work. To continue the example of somebody with a preference for business and accounting software, a small project you might create is a plugin or extension for PostBooks. Getting to know prospective mentors Many web sites provide useful information about the developers who contribute to free software projects. Some of these developers may be willing to be a GSoC mentor. For example, look through some of the following: Getting on the mentor's shortlist Once you have identified projects that are interesting to you and developers who work on those projects, it is important to get yourself on the developer's shortlist. Basically, the shortlist is a list of all students who the developer believes can complete the project. If I feel that a student is unlikely to complete a project or if I don't have enough information to judge a student's probability of success, that student will not be on my shortlist. If I don't have any student on my shortlist, then a project will not go ahead at all. If there are multiple students on the shortlist, then I will be looking more closely at each of them to try and work out who is the best match. One way to get a developer's attention is to look at bug reports they have created. Github makes it easy to see complaints or bug reports they have made about their own projects or other projects they depend on. Another way to do this is to search through their code for strings like FIXME and TODO. Projects with standalone bug trackers like the Debian bug tracker also provide an easy way to search for bug reports that a specific person has created or commented on. Once you find some relevant bug reports, email the developer. Ask if anybody else is working on those issues. Try and start with an issue that is particularly easy and where the solution is interesting for you. This will help you learn to compile and test the program before you try to fix any more complicated bugs. It may even be something you can work on as part of your academic program. Find successful projects from the previous year Contact organizations and ask them which GSoC projects were most successful. In many organizations, you can find the past students' project plans and their final reports published on the web. Read through the plans submitted by the students who were chosen. Then read through the final reports by the same students and see how they compare to the original plans. Start building your project proposal now Don't wait for the application period to begin. Start writing a project proposal now. When writing a proposal, it is important to include several things:
  • Think big: what is the goal at the end of the project? Does your work help the greater good in some way, such as increasing the market share of Linux on the desktop?
  • Details: what are specific challenges? What tools will you use?
  • Time management: what will you do each week? Are there weeks where you will not work on GSoC due to vacation or other events? These things are permitted but they must be in your plan if you know them in advance. If an accident or death in the family cut a week out of your GSoC project, which work would you skip and would your project still be useful without that? Having two weeks of flexible time in your plan makes it more resilient against interruptions.
  • Communication: are you on mailing lists, IRC and XMPP chat? Will you make a weekly report on your blog?
  • Users: who will benefit from your work?
  • Testing: who will test and validate your work throughout the project? Ideally, this should involve more than just the mentor.
If your project plan is good enough, could you put it on Kickstarter or another crowdfunding site? This is a good test of whether or not a project is going to be supported by a GSoC mentor. Learn about packaging and distributing software Packaging is a vital part of the free software lifecycle. It is very easy to upload a project to Github but it takes more effort to have it become an official package in systems like Debian, Fedora and Ubuntu. Packaging and the communities around Linux distributions help you reach out to users of your software and get valuable feedback and new contributors. This boosts the impact of your work. To start with, you may want to help the maintainer of an existing package. Debian packaging teams are existing communities that work in a team and welcome new contributors. The Debian Mentors initiative is another great starting place. In the Fedora world, the place to start may be in one of the Special Interest Groups (SIGs). Think from the mentor's perspective After the application deadline, mentors have just 2 or 3 weeks to choose the students. This is actually not a lot of time to be certain if a particular student is capable of completing a project. If the student has a published history of free software activity, the mentor feels a lot more confident about choosing the student. Some mentors have more than one good student while other mentors receive no applications from capable students. In this situation, it is very common for mentors to send each other details of students who may be suitable. Once again, if a student has a good Github profile and a blog, it is much easier for mentors to try and match that student with another project. GSoC logo generic Conclusion Getting into the world of software engineering is much like joining any other profession or even joining a new hobby or sporting activity. If you run, you probably have various types of shoe and a running watch and you may even spend a couple of nights at the track each week. If you enjoy playing a musical instrument, you probably have a collection of sheet music, accessories for your instrument and you may even aspire to build a recording studio in your garage (or you probably know somebody else who already did that). The things listed on this page will not just help you walk the walk and talk the talk of a software developer, they will put you on a track to being one of the leaders. If you look over the profiles of other software developers on the Internet, you will find they are doing most of the things on this page already. Even if you are not selected for GSoC at all or decide not to apply, working through the steps on this page will help you clarify your own ideas about your career and help you make new friends in the software engineering community.

16 August 2014

Matthias Klumpp: AppStream/DEP-11 Debian progress

There hasn t been a progress-report on DEP-11 for some time, but that doesn t mean there was no work going on on it. DEP-11 is Debian s implementation of AppStream, as well as an effort to enhance the metadata available about software in Debian. While initially, AppStream was only about applications, DEP-11 was designed with a larger scope, to collect data about libraries, binaries and things like Python modules. Now, since AppStream 0.6, DEP-11 and AppStream have essentially the same scope, with the difference of DEP-11 metadata being described in YAML, while official AppStream data is XML. That was due to a request by our ftpmasters team, which doesn t like XML (which is also not used anywhere in Debian, as opposed to YAML). But this doesn t mean that people will have to deal with the YAML file format: The libappstream library will just take DEP-11 data as another data source for it s Xapian database, allowing anything using libappstream to access that data just like the XML stuff. Richards libappstream-glib will also receive support for the DEP-11 format soon, filling it s in-memory data cache and enabling the use of GNOME-Software on Debian. So, what has been done so far? The past months, my Google Summer of Code student. Abhishek Bhattacharjee, was working hard to integrate DEP-11 support into dak, the Debian Archive Kit, which maintains the whole Debian archive. The result will be an additional metadata table in our internal Postgres database, storing detailed information about the software available in a Debian package, as well as Components-<arch>.yml.gz files in the Debian repositories. Dak will also produce an application icon-cache and a screenshots repository. During the time of the SoC, Abhishek focused mainly on the applications part of things, and less on the other components (like extracting data about Python modules or libraries) these things can easily be implemented later. The remaining steps will be to polish the code and make it merge-ready for Debian s dak (as soon as it has received enough testing, we will likely give it a try on the Tanglu Debian derivative). Following that, Apt will be extended to fetch the DEP-11 data on-demand on systems where it is useful (which is currently mostly desktop-systems) if you want to save a little bit of space, you will be able to disable downloading this extra metadata in Apt. From there, libappstream will take the data for it s Xapian db. This will lead to the removal of the much-hated (from ftpmasters and maintainers side) app-install-data package, which has not been updated for two years and only contains a small fraction of the metadata provided by DEP-11. What Debian will ultimately gain from this effort is support for software-centers like GNOME-Software, and improved support for tools like Apper and Muon in displaying applications. Long-term, with more metadata being available, It would be cool to add support for it to specialized package managers , like Python s pip, npm or gem, to make them fetch information about available distribution software and install that instead of their own copies from 3rd-party repositories, if possible. This should ultimatively lead to less code duplication on distributions and will likely result in fewer security issues, since the officially maintained and integrated distribution packages can easily be used, if possible. This is no attempt to make tools like pip obsolete, but an attempt to have the different tools installing software on your machine communicate better, instead of creating parallel worlds in terms of software management. Another nice sideeffect of more metadata will be options to search for tools handling mimetypes in the software repos (in case you can t open a file), smart software centers installing missing firmware, and automatic suggestions for developers which software they need to install in order to build a specific software package. Also, the data allows us to match software across distributions, for that, I will have some news soon (not sure how soon though, as I am currently in thesis-writing-mode, and therefore have not that much spare time). Since the goal is to have these features available on all distributions supporting AppStream, it will take longer to realize but we are on a good way. So, if you want some more information about my student s awesome work, you can read his blogpost about it. He will also be at Debconf 14 (Portland). (I can t make it this time, but I surely won t miss the next Debconf) Sadly, I only see a very small chance to have the basic DEP-11 stuff land in-time for Jessie (lots of review work needs to be done, and some more code needs to be written), but we will definitively have it in Jessie+1. A small example on how this data will look like can be found here a larger, actual file is available here. Any questions and feedback are highly appreciated.

25 July 2014

Juliana Louback: Extending an xTuple Business Object

xTuple is in my opinion incredibly well designed; the code is clean and the architecture ahderent to a standardized structure. All this makes working with xTuple software quite a breeze. I wanted to integrate JSCommunicator into the web-based xTuple version. JSCommunicator is a SIP communication tool, so my first step was to create an extension for the SIP account data. Luckily for me, the xTuple development team published an awesome tutorial for writing an xTuple extension. xTuple cleverly uses model based business objects for the various features available. This makes customizing xTuple very straightforward. I used the tutorial mentioned above for writing my extension, but soon noticed my goals were a little different. A SIP account has 3 data fields, these being the SIP URI, the account password and an optional display name. xTuple currently has a business object in the core code for a User Account and it would make a lot more sense to simply add my 3 fields to this existing business object rather than create another business object. The tutorial very clearly shows how to extend a business object with another business object, but not how to extend a business object with only new fields (not a whole new object). Now maybe I m just a whole lot slower than most people, but I had a ridiculously had time figuring this out. Mind you, this is because I m slow, because the xTuple documentation and code is understandable and as self-explanatory as it gets. I think it just takes a bit to get used to. Either way, I thought this just might be useful to others so here is how I went about it. Setup First you ll have to set up your xTuple development environment and fork the xtuple and xtuple-extesions repositories as shown in this handy tutorial. A footnote I d like to add is please verify that your version of Vagrant (and anything else you install) is the one listed in the tutorial. I think I spent like two entire days or more on a wild goose (bug) chase trying to set up my environment when the cause of all the errors was that I somehow installed an older version of Vagrant - 1.5.4 instead of 1.6.3. Please don t make the same mistake I did. Actually if for some reason you get the following error when you try using node:
<<ERROR 2014-07-10T23:52:46.948Z>> Unrecoverable exception. Cannot call method 'extend' of undefined
    at /home/vagrant/dev/xtuple/lib/backbone-x/source/model.js:37:39
    at Object.<anonymous> (/home/vagrant/dev/xtuple/lib/backbone-x/source/model.js:1364:3)
    ...
chances are, you have the wrong version. That s what happened to me. The Vagrant Virtual Development Environment automatically installs and configures everything you need, it s ready to go. So if you find yourself installing and updating and apt-gets and etc, you probably did something wrong. Coding So by now we should have the Vagrant Virtual Development Environment set up and the web app up and running and accessible at localhost:8443. So far so good. Disclaimer: You will note that much of this is similar - or rather, nearly identical - to xTuple s tutorial but there are some small but important differences and a few observations I think might be useful. Other Disclaimer: I m describing how I did it, which may or may not be up to snuff . Works for me though. Schema First let s make a schema for the table we will create with the new custom fields. Be sure to create the correct directory stucture, aka /path/to/xtuple-extensions/source/<YOUR EXTENSION NAME>/database/source or in my case /path/to/xtuple-extensions/source/sip_account/database/source, and create the file create_sa_schema.sql, sa is the name of my schema. This file will contain the following lines:
do $$
  /* Only create the schema if it hasn't been created already */
  var res, sql = "select schema_name from information_schema.schemata where schema_name = 'sa'",
  res = plv8.execute(sql);
  if (!res.length)  
    sql = "create schema sa; grant all on schema sa to group xtrole;"
    plv8.execute(sql);
   
$$ language plv8;
Of course, feel free to replace sa with your schema name of choice. All the code described here can be found in my xtuple-extensions fork, on the sip_ext branch. Table We ll create a table containing your custom fields and a link to an existing table - the table for the existing business object you want to extend. If you re wondering why make a whole new table for a few extra fields, here s a good explanation, the case in question is adding fields to the Contact business object. You need to first figure out what table you want to link to. This might not be uber easy. I think the best way to go about it is to look at the ORMs. The xTuple ORMs are a JSON mapping between the SQL tables and the object-oriented world above the database, they re .json files found at path/to/xtuple/node_modules/xtuple/enyo-client/database/orm/models for the core business objects and at path/to/xtuplenyo-client/extensions/source/<EXTENSION NAME>/database/orm/models for exension business objects. I ll give two examples. If you look at contact.json you will see that the Contact business object refers to the table cntct . Look for the type : Contact on the line above, so we know it s the Contact business object. In my case, I wanted to extend the UserAccount and UserAccountRelation business objects, so check out user_account.json. The table listed for UserAccount is xt.usrinfo and the table listed for UserAccountRelation is xt.usrlite. A closer look at the sql files for these tables (usrinfo.sql and usrlite.sql) revealed that usrinfo is in fact a view and usrlite is A light weight table of user information used to avoid punishingly heavy queries on the public usr view . I chose to refer to xt.usrlite - that or I received error messages when trying the other table names. Now I ll make the file /path/to/xtuple-extensions/source/sip_account/database/source/usrlitesip.sql, to create a table with my custom fields plus the link to the urslite table. Don t quote me on this, but I m under the impression that this is the norm for naming the sql file joining tables: the name of the table you are referring to ( usrlite in this case) and your extension s name. Content of usrlitesip.sql:
select xt.create_table('usrlitesip', 'sa');
select xt.add_column('usrlitesip','usrlitesip_id', 'serial', 'primary key', 'sa');
select xt.add_column('usrlitesip','usrlitesip_usr_username', 'text', 'references xt.usrlite (usr_username)', 'sa');
select xt.add_column('usrlitesip','usrlitesip_uri', 'text', '', 'sa');
select xt.add_column('usrlitesip','usrlitesip_name', 'text', '', 'sa');
select xt.add_column('usrlitesip','usrlitesip_password', 'text', '', 'sa');
comment on table sa.usrlitesip is 'Joins User with SIP account';
Breaking it down, line 1 creates the table named usrlitesip (no duh), line 2 is for the primary key (self-explanatory). You can then add any columns you like, just be sure to add one that references the table you want to link to. I checked usrlite.sql and saw the primary key is usr_username, be sure to use the primary key of the table you are referencing. You can check what you made by executing the .sql files like so:
$ cd /path/to/xtuple-extensions/source/sip_account/database/source
$ psql -U admin -d dev -f create_sa_schema.sql
$ psql -U admin -d dev -f usrlitesip.sql
After which you will see the table with the columns you created if you enter:
$ psql -U admin -d dev -c "select * from sa.usrlitesip;"
Now create the file /path/to/xtuple-extensions/source/sip_account/database/source/manifest.js to put the files together and in the right order. It should contain:
 
  "name": "sip_account",
  "version": "1.4.1",
  "comment": "Sip Account extension",
  "loadOrder": 999,
  "dependencies": ["crm"],
  "databaseScripts": [
    "create_sa_schema.sql",
    "usrlitesip.sql",
    "register.sql"
  ]
 
I think the name has to be the same you named your extension directory as in /path/to/xtuple-extensions/source/<YOUR EXTENSION NAME>. I think the comment can be anything you like and you want your loadOrder to be high so it s the last thing installed (as it s an add on.) So far we are doing exactly what s instructed in the xTuple tutorial. It s repetitive, but I think you can never have too many examples to compare to. In databaseScripts you will list the two .sql files you just created for the schema and the table, plus another file to be made in the same directory named register.sql. I m not sure why you have to make the register.sql or even if you indeed have to. If you leave the file empty, there will be a build error, so put a ; in the register.sql or remove the line register.sql from manifest.js as I think for now we are good without it. Now let s update the database with our new extension:
$ cd /path/to/xtuple
$ ./scripts/build_app.js -d dev -e ../xtuple-extensions/source/sip_account
$ psql -U admin -d dev -c "select * from xt.ext;"
That last command should display a table with a list of extensions; the ones already in xtuple like crm and billing and some others plus your new extension, in this case sip_account . When you run build_app.js you ll probably see a message along the lines of <Extension name> has no client code, not building client code and that s fine because yeah, we haven t worked on the client code yet. ORM Here s where things start getting different. So ORMs link your object to an SQL table. But we DON T want to make a new business object, we want to extend an existing business object, so the ORM we will make will be a little different than the xTuple tutorial. Steve Hackbarth kindly explained this new business object/existing business object ORM concept here. First we ll create the directory /path/to/xtuple-extensions/source/sip_account/database/orm/ext, according to xTuple convention. ORMs for new business objects would be put in /path/to/xtuple-extensions/source/sip_account/database/orm/models. Now we ll create the .json file /path/to/xtuple-extensions/source/sip_account/database/orm/ext/user_account.jscon for our ORM. Once again, don t quote me on this, but I think the name of the file should be the name of the business object you are extending, as is done in the turorial example extending the Contact object. In our case, UserAccount is defined in user_account.json and that s what I named my extension ORM too. Here s what you should place in it:
 1 [
 2    
 3     "context": "sip_account",
 4     "nameSpace": "XM",
 5     "type": "UserAccount",
 6     "table": "sa.usrlitesip",
 7     "isExtension": true,
 8     "isChild": false,
 9     "comment": "Extended by Sip",
10     "relations": [
11        
12         "column": "usrlitesip_usr_username",
13         "inverse": "username"
14        
15     ],
16     "properties": [
17        
18         "name": "uri",
19         "attr":  
20           "type": "String",
21           "column": "usrlitesip_uri",
22           "isNaturalKey": true
23          
24        ,
25        
26         "name": "displayName",
27         "attr":  
28           "type": "String",
29           "column": "usrlitesip_name"
30          
31        ,
32        
33         "name": "sipPassword",
34         "attr":  
35           "type": "String",
36           "column": "usrlitesip_password"
37          
38        
39     ],
40     "isSystem": true
41    ,
42    
43     "context": "sip_account",
44     "nameSpace": "XM",
45     "type": "UserAccountRelation",
46     "table": "sa.usrlitesip",
47     "isExtension": true,
48     "isChild": false,
49     "comment": "Extended by Sip",
50     "relations": [
51        
52         "column": "usrlitesip_usr_username",
53         "inverse": "username"
54        
55     ],
56     "properties": [
57        
58         "name": "uri",
59         "attr":  
60           "type": "String",
61           "column": "usrlitesip_uri",
62           "isNaturalKey": true
63          
64        ,
65        
66         "name": "displayName",
67         "attr":  
68           "type": "String",
69           "column": "usrlitesip_name"
70          
71        ,
72        
73         "name": "sipPassword",
74         "attr":  
75           "type": "String",
76           "column": "usrlitesip_password"
77          
78        
79     ],
80     "isSystem": true
81    
82 ]
Note the context is my extension name, because the context + nameSpace + type combo has to be unique. We already have a UserAccount and UserAccountRelation object in the XM namespace in the xtuple context in the original user_account.json, now we will have a UserAccount and UserAccountRelation object in the XM namespace in the sip_account conext. What else is important? Note that isExtension is true on lines 7 and 47 and the relations item contains the column of the foreign key we referenced. This is something you might want to verify: column (lines 12 and 52) is the name of the attribute on your table. When we made a reference to the primary key usr_usrname from the xt.usrlite table we named that column usrlitesip_usr_usrname. But the inverse is the attribute name associated with the original sql column in the original ORM. Did I lose you? I had a lot of trouble with this silly thing. In the original ORM that created a new UserAccount business object, the primary key attribute is named username , as can be seen here. That is what should be used for the inverse value. Not the sql column name (usr_username) but the object attribute name (username). I m emphasizing this because I made that mistake and if I can spare you the pain I will. If we rebuild our extension everything should come along nicely, but you won t see any changes just yet in the web app because we haven t created the client code. Client Create the directory /path/to/xtuple-extensions/source/sip_account/client which is where we ll keep all the client code. Extend Workspace View I want the fields I added to show up on the form to create a new User Account, so I need to extend the view for the User Account workspace. I ll start by creating a directory /path/to/xtuple-extensions/source/sip_account/client/views and in it creating a file named workspace.js containing this code:
XT.extensions.sip_account.initWorkspace = function ()  
	var extensions = [
  	 kind: "onyx.GroupboxHeader", container: "mainGroup", content: "_sipAccount".loc() ,
  	 kind: "XV.InputWidget", container: "mainGroup", attr: "uri"  ,
  	 kind: "XV.InputWidget", container: "mainGroup", attr: "displayName"  ,
  	 kind: "XV.InputWidget", container: "mainGroup", type:"password", attr: "sipPassword"  
	];
	XV.appendExtension("XV.UserAccountWorkspace", extensions);
 ;
So I m initializing my workspace and creating an array of items to add (append) to view XV.UserAccountWorkspace. The first item is this onyx.GroupboxHeader which is a pretty divider for my new form fields, the kind you find in the web app at Setup > User Accounts, like Overview . I have no idea what other options there are for container other than mainGroup , so let s stick to that. I ll explain content: _sipAccount .loc() in a bit. Next I created three input fields of the XV.InputWidget kind. This also confused me a bit as there are different kinds of input to be used, like dropdowns and checkboxes. The only advice I can give is snoop around the webapp, find an input you like and look up the corresponding workspace.js file to see what was used. What we just did is (should be) enough for the new fields to show up on the User Account form. But before we see things change, we have to package the client. Create the file /path/to/xtuple-extensions/source/sip_account/client/views/package.js. This file is needed to package groups of files and indicates the order the files should be loaded (for more on that, see this). For now, all the file will contain is:
enyo.depends(
"workspace.js"
);
You also need to package the views directory containing workspace.js, so create the file Create the file /path/to/xtuple-extensions/source/sip_account/client/package.js and in it show that the directory views and its contents must be part of the higher level package:
enyo.depends(
"views"
);
I like to think of it as a box full of smaller boxes. This will sound terrible, but apparently you also need to create the file /path/to/xtuple-extensions/source/sip_account/client/core.js containing this line:
XT.extensions.icecream =  ;
I don t know why. As soon as I find out I ll be sure to inform you. As we ve added a file to the client directory, be sure to update /path/to/xtuple-extensions/source/sip_account/client/package.js so it included the new file:
enyo.depends(
"core.js",
"views"
);
Translations Remember _sipAccount .loc() in our workspace.js file? xTuple has great internationalization support and it s easy to use. Just create the directory and file /path/to/xtuple-extensions/source/sip_account/client/en/strings.js and in it put key-value pairs for labels and their translation, like this:
(function ()  
  "use strict";
  var lang = XT.stringsFor("en_US",  
    "_sipAccount": "Sip Account",
    "_uri": "Sip URI",
    "_displayName": "Display Name",
    "_sipPassword": "Password"
   );
  if (typeof exports !== 'undefined')  
    exports.language = lang;
   
 ());
So far I included all the labels I used in my Sip Account form. If you write the wrong label (key) or forget to include a corresponding key-value pair in strings.js, xTuple will simply name your lable _lableName , underscore and all. Now build your extension and start up the server:
$ cd /path/to/xtuple 
$ ./scripts/build_app.js -d dev -e ../xtuple-extensions/source/sip_account
$ node node-datasource/main.js
If the server is already running, just stop it and restart it to reflect your changes. Now if you go to Setup > User Accounts and click the + button, you should see a nice little addition to the form with a Sip Account divider and three new fields. Nice, eh? Extend Parameters Currently you can search your User Accounts list using any of the User Account fields. It would be nice to be able to search with the Sip account fields we added as well. To do that, let s create the directory /path/to/xtuple-extensions/source/sip_account/client/widgets and there create the file parameter.js to extend XV.UserAccountListParameters. One again, you ll have to look this up. In the xTuple code you ll find the application s parameter.js in /path/to/xtuple/enyo-client/application/source/widgets. Search for the business object you are extending (for example, XV.UserAccount) and look for some combination of the business object name and Parameters . If there s more than one, try different ones. Not a very refined method, but it worked for me. Here s the content of our parameter.js:
XT.extensions.sip_account.initParameterWidget = function ()  
  var extensions = [
     kind: "onyx.GroupboxHeader", content: "_sipAccount".loc() ,
     name: "uri", label: "_uri".loc(), attr: "uri", defaultKind: "XV.InputWidget" ,
     name: "displayName", label: "_displayName".loc(), attr: "displayName", defaultKind: "XV.InputWidget" 
  ];
  XV.appendExtension("XV.UserAccountListParameters", extensions);
 ;
Node that I didn t include a search field for the password attribute for obvious reasons. Now once again, we package this new code addition by creating a /path/to/xtuple-extensions/source/sip_account/client/widgets/package.js file:
enyo.depends(
"parameter.js"
);
We also have to update /path/to/xtuple-extensions/source/sip_account/client/package.js:
enyo.depends(
"core.js",
"widgets"
"views"
);
Rebuild the extension (and restart the server) and go to Setup > User Accounts. Press the magnifying glass button on the upper left side of the screen and you ll see many options for filtering the User Accounts, among them the SIP Uri and Display Name. Extend List View You might want your new fields to show up on the list of User Accounts. There s a bit of an issue here because unlike what we did in workspace.js and parameter.js, we can t append new things to the list of UserAccounts with the funciton XV.appendExtension(args). First I tried overwriting the original UserAccountList, which works but it s far from ideal as this could result in a loss of data from the core implementation. After some discussion with the xTuple dev community, now there s a better alternative: Create the file /path/to/xtuple-extensions/source/sip_account/client/views/list.js and add the following:
1 var oldUserAccountListCreate = XV.UserAccountList.prototype.create;
2 
3 XV.UserAccountList.prototype.create = function ()  
4   oldUserAccountListCreate.apply(this, arguments);
5   this.createComponent(
6    kind: "XV.ListColumn", container: this.$.listItem, components: [
7         kind: "XV.ListAttr", attr: "uri" 
8    ] )
9  ;
To understand what I m doing, check out the XV.UserAccountList implementation in /path/to/xtuple/enyo-client/application/source/views/list.js the entire highlighted part. What we are doing is extending XV.UserAccountList through prototype-chaining ; this is how inheritance works with Enyo. In line 1 we create a prototype and in line 4 we inherit the features including original components array which the list is based on. We then create an additional component immitating the setup shown in XV.UserAccountList: An XV.ListColumn containing an XV.ListAttr, which should be placed in the XV.ListItem components array as is done with the existing columns (refer to implementation). Components can or should (?) have names which are used to access said components. You d refer to a specific component by the this.$.componentName hash. The components in XV.UserAccountList don t have names, so Enyo automatically names them (apparently) based on the kind name, for example something of the kind ListItem is named listItem. I found this at random after a lot of trial and error and it s not a bullet proof solution. Can be bettered. It s strange because if you encapsulate that code with
XT.extensions.sip_account.initList = function ()  
 //Code here
 ;
as is done with parameter.js and workspace.js (and in the xTuple tutorial you are supposed to do that with a new business object), it doesn t work. I have no idea why. This might be wrong or against xTuple coding norms; I will find out and update this post ASAP. But it does work this way. * shrugs * That said, as we ve created the list.js file, we need to ad it to our package by editing /path/to/xtuple-extensions/source/sip_account/client/views/package.js:
enyo.depends(
"list.js",
"workspace.js"
);
That s all. Rebuild the app and restart your server and when you select Setup > User Accounts in the web app you should see the Sip URI displayed on the User Accounts that have the Sip Account data. Add a new User Account to try this out.

16 July 2014

Matthias Klumpp: AppStream 0.7 specification and library released

appstream-logoToday I am very happy to announce the release of AppStream 0.7, the second-largest release (judging by commit number) after 0.6. AppStream 0.7 brings many new features for the specification, adds lots of good stuff to libappstream, introduces a new libappstream-qt library for Qt developers and, as always, fixes some bugs. Unfortunately we broke the API/ABI of libappstream, so please adjust your code accordingly. Apart from that, any other changes are backwards-compatible. So, here is an overview of what s new in AppStream 0.7: Specification changes Distributors may now specify a new <languages/> tag in their distribution XML, providing information about the languages a component supports and the completion-percentage for the language. This allows software-centers to apply smart filtering on applications to highlight the ones which are available in the users native language. A new addon component type was added to represent software which is designed to be used together with a specific other application (think of a Firefox addon or GNOME-Shell extension). Software-center applications can group the addons together with their main application to provide an easy way for users to install additional functionality for existing applications. The <provides/> tag gained a new dbus item-type to expose D-Bus interface names the component provides to the outside world. This means in future it will be possible to search for components providing a specific dbus service:
$ appstream-index what-provides dbus org.freedesktop.PackageKit.desktop system
(if you are using the cli tool) A <developer_name/> tag was added to the generic component definition to define the name of the component developer in a human-readable form. Possible values are, for example The KDE Community , GNOME Developers or even the developer s full name. This value can be (optionally) translated and will be displayed in software-centers. An <update_contact/> tag was added to the specification, to provide a convenient way for distributors to reach upstream to talk about changes made to their metadata or issues with the latest software update. This tag was already used by some projects before, and has now been added to the official specification. Timestamps in <release/> tags must now be UNIX epochs, YYYYMMDD is no longer valid (fortunately, everyone is already using UNIX epochs). Last but not least, the <pkgname/> tag is now allowed multiple times per component. We still recommend to create metapackages according to the contents the upstream metadata describes and place the file there. However, in some cases defining one component to be in multiple packages is a short way to make metadata available correctly without excessive package-tuning (which can become difficult if a <provides/> tag needs to be satisfied). As small sidenote: The multiarch path in /usr/share/appdata is now deprecated, because we think that we can live without it (by shipping -data packages per library and using smarter AppStream metadata generators which take advantage of the ability to define multiple <pkgname/> tags) Documentation updates In general, the documentation of the specification has been reworked to be easier to understand and to include less duplication of information. We now use excessive crosslinking to show you the information you need in order to write metadata for your upstream project, or to implement a metadata generator for your distribution. Because the specification needs to define the allowed tags completely and contain as much information as possible, it is not very easy to digest for upstream authors who just want some metadata shipped quickly. In order to help them, we now have Quickstart pages in the documentation, which are rich of examples and contain the most important subset of information you need to write a good metadata file. These quickstart guides already exist for desktop-applications and addons, more will follow in future. We also have an explicit section dealing with the question How do I translate upstream metadata? now. More changes to the docs are planned for the next point releases. You can find the full project documentation at Freedesktop. AppStream GObject library and tools The libappstream library also received lots of changes. The most important one: We switched from using LGPL-3+ to LGPL-2.1+. People who know me know that I love the v3 license family of GPL licenses I like it for tivoization protection, it s explicit compatibility with some important other licenses and cosmetic details, like entities not loosing their right to use the software forever after a license violation. However, a LGPL-3+ library does not mix well with projects licensed under other open source licenses, mainly GPL-2-only projects. I want libappstream to be used by anyone without forcing the project to change its license. For some reason, using the library from proprietary code is easier than using it from a GPL-2-only open source project. The license change was also a popular request of people wanting to use the library, so I made the switch with 0.7. If you want to know more about the LGPL-3 issues, I recommend reading this blogpost by Nikos (GnuTLS). On the code-side, libappstream received a large pile of bugfixes and some internal restructuring. This makes the cache builder about 5% faster (depending on your system and the amount of metadata which needs to be processed) and prepares for future changes (e.g. I plan to obsolete PackageKit s desktop-file-database in the long term). The library also brings back support for legacy AppData files, which it can now read. However, appstream-validate will not validate these files (and kindly ask you to migrate to the new format). The appstream-index tool received some changes, making it s command-line interface a bit more modern. It is also possible now to place the Xapian cache at arbitrary locations, which is a nice feature for developers. Additionally, the testsuite got improved and should now work on systems which do not have metadata installed. Of course, libappstream also implements all features of the new 0.7 specification. With the 0.7 release, some symbols were removed which have been deprecated for a few releases, most notably as_component_get/set_idname, as_database_find_components_by_str, as_component_get/set_homepage and the pkgname property of AsComponent (which is now a string array and called pkgnames ). API level was bumped to 1. Appstream-Qt A Qt library to access AppStream data has been added. So if you want to use AppStream metadata in your Qt application, you can easily do that now without touching any GLib/GObject based code! Special thanks to Sune Vuorela for his nice rework of the Qt library! And that s it with the changes for now! Thanks to everyone who helped making 0.7 ready, being it feedback, contributions to the documentation, translation or coding. You can get the release tarballs at Freedesktop. Have fun!

22 June 2014

DebConf team: First set of talks accepted; we are still waiting for yours! (Posted by Ana Guerrero Lopez)

We re now in the middle of the talk submission period. Our new web interface doesn t allow to show all the proposals and we decided to start accepting some talks in order to inspire you to propose your own talks. If your talk is not on the list doesn t mean is not accepted, don t panic! And now, we re calling you all to bring up your best Debian-related ideas, proposals, stories. To think about what you want to organize a discussion on. Which tracks would you like to coordinate or see populated with interesting talks. Talks are the main ingredient for a successful, interesting, discussion-sparking DebConf. Don t be shy, and read how to submit your talk

7 June 2014

Vasudev Kamath: Exposing function in python module using entry_points, WSME in a Flask webapp

The heading might be ambiguous, but I couldn't figure out better heading so let me start by explaining what I'm trying to solve here.
Problem I have a python module which contains a function which I want to expose as a REST web service in a Flask application. I use WSME for Flask application, which actually needs signature of function in question and problem comes to picture because function to be exposed is foreign to Flask application, it resides in separate python module.
Solution While reading Julien Danjou's Hackers Guide To Python book I came across the setuptools entry_points concept which can be used to extend existing feature of a tool like plug-ins. So here I'm going to use this entry_points feature from setuptools to provide function in the module which can expose the signature of function[s] to be exposed through REST. Of course this means I need to modify module in question to write entry_points and function for giving out signature of function to be exposed. I will explain this with small example. I have a dummy module which provides a add function and a function which exposes the add functions signature.
def add(a, b):
    return a + b
def expose_rest_func():
    return [add, int, int, int]
This is stored in dummy/__init__.py file. I use pbr tool to package my python module. Below is content of setup.cfg file.
[metadata]
name = dummy
author = Vasudev Kamath
author-email = kamathvasudev@gmail.com
summary = Dummy module for testing purpose
version = 0.1
license = MIT
description-file =
  README.rst
requires-python = >= 2.7
[files]
packages =
  dummy
[entry_points] =
myapp.api.rest =
  rest = dummy:expose_rest_func
The special thing in above file is entry_points section, which defines function to be hooked into entry_point. In our case entry_point myapp.api.rest is used by our Flask application to interact with modules which expose them. The function which will be got accessing the entry_point is expose_rest_func which gives the function to be exposed its arg types and return types as a list. If we are looking at only supporting python3 it was sufficient to know function name only and use function annotations in function definition. Since I want to support both python2 and python3 this is out of question. Now, just run the following command in virtualenv to get the module installed.
PBR_VERSION=0.1 python setup.py sdist
pip install dist/dummy_module-0.1.tar.gz
Now if you want to see if the module is exposing entry_point or not just use entry_point_inspector tool after installing you will get a command called epi if you run it as follows you should note the dummy_module in its output
epi group list
+-----------------------------+
  Name                         
+-----------------------------+
  cliff.formatter.completion   
  cliff.formatter.list         
  cliff.formatter.show         
  console_scripts              
  distutils.commands           
  distutils.setup_keywords     
  egg_info.writers             
  epi.commands                 
  flake8.extension             
  setuptools.file_finders      
  setuptools.installation      
  myapp.api.rest               
  stevedore.example.formatter  
  stevedore.test.extension     
  wsme.protocols               
+-----------------------------+
So our entry_point is exposed now, we need to access it in our Flask application and expose the function using WSME. It is done by below code.
from wsmeext.flask import signature
import flask
import pkg_resources
def main():
   app = flask.Flask(__name__)
   app.config['DEBUG'] = True
   for entrypoint in pkg_resources.iter_entry_points('myapp.api.rest'):
       # Ugly but fix is only supporting python3
       func_signature = entrypoint.load()()
       app.route('/' + func_signature[0].__name__, methods=['POST'])(
           signature(func_signature[-1],
               *func_signature[1:-1])(func_signature[0]))
   app.run()
if __name__ == '__main__':
    main()
entry_point myapp.api.rest are iterated using the pkg_resources package provided by setuptools, when I load the entry_point I get back the function to be used which is called in same place to get function signature. Then I'm calling Flask and WSME decorator functions (yeah instead of decorating I'm using them directly over function to be exposed). The code looks bit ugly at the place where I'm accessing list using slices but I can't help it due to limitation of python2 with python3 there is new packing and unpacking stuff which makes code look bit more cooler see below.
from wsmeext.flask import signature
import flask
import pkg_resources
def main():
    app = flask.Flask(__name__)
    app.config['DEBUG'] = True
    for entrypoint in pkg_resources.iter_entry_points('silpa.api.rest'):
        func, *args, rettype = entrypoint.load()()
        app.route('/' + func.__name__, methods=['POST'])(
        signature(rettype, *args)(func_signature[0]))
    app.run()
if __name__ == '__main__':
    main()
You can access the service at http://localhost:5000/add depending on Accept header of HTTP you will get either XML or JSON response. If you access it from browser you will get XML response.
Usecase Now if you are wondering what is the reason behind this experience, this is for SILPA Project. I'm trying to implement REST service for all Indic language computing module. Since all these module are independent of SILPA which is a Flask web app I had to find a way to achieve this, and this is what I came up with.
Conclusion I'm not sure if there is any other approaches to achieve this, if there I would love to hear about them. You can write your comments and suggestion over email

2 June 2014

Rapha&#235;l Hertzog: My Free Software Activities since January 2014

If you follow my blog closely, you noticed that I skipped all my usual monthly summaries in 2014. It s not that I stopped doing free software work, instead I was just too busy to be able to report about what I did. As an excuse, let me tell you that we just moved into a new house which was in construction since may last year. The lack of visible activity on my blog resulted in a steady decrease of the amount of donations received (January: 70.72 , February: 71.75 , March: 51.25 , April: 39.9 , May: 40.33 ). Special thanks to all the people who kept supporting my work even though I stopped reporting about it. So let s fix this. This report will be a bit less detailed since it covers the whole period since the start of the year. Debian France Preparations related to general assemblies. The year started with lots of work related to Debian France. First I took care of setting up limesurvey with Alexandre Delano to handle the vote to pick our new logo:
The new logo of Debian France I also helped Sylvestre Ledru to finalize and close the accounting books for 2013 in preparation for the general assembly that was due later in the month. I wrote the moral report of the president to be presented to the assembly. And last step, I collected vote mandates to ensure that we were going to meet the quorum for the extraordinary assembly that was planned just after the usual yearly assembly. The assemblies took place during a two days mini-debconf in Paris (January 17-18) where I was obviously present even though I gave no talk besides announcing the logo contest winner and thanking people for their participation.
Assembl e g n rale 2014 de Debian France

The Debian France members during the general assembly

It s worth noting that the extraordinary assembly was meant primarily to enshrine in our bylaws the possibility to act as a trusted organization for Debian. This status should be officialized by the Debian project leader (Lucas Nussbaum) in the upcoming weeks since we answered satisfactorily to all questions. Our paypal donation form and the accounting tools behind it are ready. Galette packaging and members map. I managed to hand over the package maintenance of galette to Fran ois-R gis Vuillemin. I sponsored all his uploads and we packaged a new plugin that allows to create a map with all the members who accept to share their location. The idea was to let people meet each other when they don t live far away with the long term goal to have Debian France organized activities not only in Paris but everywhere in France. New contributor game. Last but not least, I organized a game to encourage people to do their first contribution to Debian by offering them a copy of my book if they managed to complete a small Debian project. We got many interesting projects but the result so far seem to be very mixed. Many people did not complete their project (yet) that said for the few that did substantial work, it was rather good and they seem to be interested to continue to contribute. Debian France booth at Solutions Linux in Paris. Like each year, I spent two days in Paris to help man the Debian France booth at Solutions Linux. We had lots of goodies on sale and we made more than 2000 EUR in earnings during the two days. I also used this opportunity to try to convince companies to support the new Debian LTS effort.
Debian France booth at Solutions Linux

Tanguy Ortolo and Fernando Lagrange behind the Debian France booth

The Debian Administrator s Handbook In the last days of 2013, we released the wheezy update of the book. Then I quickly organized everything needed so that the various translation teams can now focus their efforts on the latest release of the book. Later (in February) I announced the availability of the French and Spanish translations. Debian Squeeze LTS When the security team called for help to try to put in place long term support for Squeeze, I replied positively because I m convinced that it s very important if Debian wants to stay an acceptable choice in big deployments and because I knew that some of my customers would be interested Thus I followed all the discussions (on a semi-private list first and then on debian-lts@lists.debian.org) and contributed my own experience. I have also taken up the responsibility to coordinate with the Debian contributors who can be hired to work on Squeeze LTS so that we have a clear common offer for all the companies who have offered financial support towards Squeeze LTS. Expect further news on this front in the upcoming days/weeks. Tryton I have been a long time user of SQL-Ledger to manage the accounting of my company Freexian. But while the license is free software, the project is not. It s the work of a single developer who doesn t really accept help. I have thus been considering to move to something else for a long time but never did anything. This year, after some rough evaluation, I decided to switch to Tryton for my company. It s probably not a wise choice from a business perspective because that migration took me many hours of unpaid labor but from a free software perspective it s definitely better than everything else I saw. I contributed a lot of bug reports and a few patches already (#3596, #3631, #3633, #3665, #3667, #3694, #3695, #3696, #3697) mainly about problems found in the French chart of accounts but also about missing features for my use case. I also accepted to sponsor Matthias Berhle, who is maintaining the official Debian packages of Tryton. He s already a Debian maintainer so it s mainly a matter of reviewing new source packages and granting him the required rights. Misc Debian work Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

15 May 2014

Thorsten Glaser: Debian packaging example: PHP5 webapp with dbconfig-common and Apache 2.2/2.4 support

I m holding a Debian packaging workshop for our trainees at work tomorrow, and have prepared a sample package for a simple PHP web application (just a handful of files) with DB connection (PostgreSQL of course), automatic setup via dbconfig-common, and with support for both Apache 2.2 (wheezy, precise) and Apache 2.4 (jessie/sid), configuration-wise. (It is possible to install this without Apache, just it does not configure the webserver then.) Schema updates on software updates are also tested (there is neither Flyway nor Liquibase which are the tools we use at work for this, other than Roland Mas wonderful scripts for FusionForge in Debian, but to my delight I discovered that dbconfig-common can also do this). Comments, suggestions, flames, etc. welcome. I know that this should not be a native package, and will address this tomorrow, but I wanted something that serves as decent example for how to do this easily, Policy conformant and using modern techniques (even those I dislike myself for the sake of simplicity). Permission was granted by the business administration to reproduce this all under a BSD-style licence, so, enjoy sharing! Thanks to Roland Mas, for making FusionForge such a nice project, and Arno T ll for some instant IRC help on the Apache side of this. This is my first time using dbconfig-common, and now, I finally feel I know enough to finish the packaging of Kivitendo which I ve started earlier. Beta testers for that welcome, too. (And next week or so, I ll need this for a Maven thingy. I ll probably opt out on the DB side, there, though. Never did anything with that, either, not being a Java guy. I guess something web to go with tomcat7 anyone got this already?)

2 May 2014

Vasudev Kamath: Note to Self: How to use url_for in Flask application

url_for is normally used to avoid hard coding of URL in Flask based webapps. I've been using it in SILPA port written using Flask by. But this was written long back and written as POC to show for Santhosh author of SILPA app which even managed to go into production :-). Recently I started organizing the code and started using flask.Blueprint which was previously written using flask.views.MethodView class and here I started facing problem with old templates. I'm just gonna explain what is the difference when using url_for with MethodView and Blueprint. Lets have some sample code for MethodView first, this code is from Flask documentation.
from flask.views import MethodView
class UserAPI(MethodView):
       def get(self):
           users = User.query.all()
           ...
       def post(self):
           user = User.from_form_data(request.form)
           ...
app.add_url_rule('/users/', view_func=UserAPI.as_view('users'))
Here we have a view which is derived from MethodView which has logic on how to handle the requests. We use add_url_rule to register rule to handle /users/ url end point and we pass the class as view function which is done by as_view class method and we can refer this method in our Jinja2 templates using name users. So a statement like
url_for('users')
in our templates will be converted to /users/ URL when page is rendered to client by Flask. Now when I replaced the MethodView in favor of Blueprint I started getting up werkzeug.routing.BuildError thrown on my face and I had no clue why!. Yeah I know I'm bad at reading documentation but even after reading documentation I was still thinking that
url_for('/')
should return me a proper URL and I was wondering why its failing. Finally after re-reading documentation for url_for it was becoming clear to me, url_for definition looks like below
flask.url_for(endpoint, **values)
Here endpoint is actually a function which is supposed to be serving the URL and **values is the arguments for this function. The URL in question should be defined in python code using decorator. In my case following is the new function serving the web pages for SILPA.
bp = Blueprint('frontend', __name__)
@bp.route(_BASE_URL, defaults= 'page': 'index.html' )
@bp.route(_BASE_URL + '<page>')
def serve_pages(page):
       if page == "index.html":
          return render_template('index.html', title='SILPA',
                               main_page=_BASE_URL,
                               modules=_modulename_to_display)
       elif page == "License":
          return render_template('license.html', title='SILPA License',
                                main_page=_BASE_URL,
                                modules=_modulename_to_display)
       elif page == "Credits":
           return render_template('credits.html', title='Credits',
                                main_page=_BASE_URL,
                                modules=_modulename_to_display)
       elif page == "Contact":
           return render_template('contact.html', title='Contact SILPA Team',
                                main_page=_BASE_URL,
                                modules=_modulename_to_display)
       else:
           # modules requested!.
           if page in _display_module_map:
               return render_template(_display_module_map[page] + '.html',
                                    title=page, main_page=_BASE_URL,
                                    modules=_modulename_to_display)
           else:
               # Did we encounter something which is not registered by us?
               return abort(404)
You can ignore function code but just note the decorators, here I'm registering the function serve_pages with page as argument for URL patterns / and /<page>, _BASE_URL here is mount point of application it can be just / or /mountpoint depending on that URL registered changes. Now I need to modify code for all url_for in my template to look like below
url_for('.serve_pages', page='/License') # for /License
url_for('.serve_pages') # which will turn in to /index.html
The . in front of function is for referring current Blueprint, in my case the Flask will consider it as frontend.serve_pages as function name and generates appropriate URL at run time. So my fault was misunderstanding endpoint argument as URL endpoint but where as its actually function name supposed to serve the page. But when using MethodViews I can simply convert class to a function with my preferred name just like UserAPI.as_view('/') so url_for('/') just works.

25 April 2014

Matthias Klumpp: AppStream 0.6.1 released a quick look at what you can do with the cli tool

Today I released AppStream and libappstream 0.6.1, which feature mostly bugfixes, so nothing incredibly exciting to see there (but this also means no API/ABI breaks). The release clarifies some paragraphs in the spec which people found confusing, and fixes a few issues (like one example in the docs not being valid AppStream metadata). As only spec extension, we introduce a priority property in distro metadata to allow metadata from one repository to override data shipped by another one. This is used (although with a similar syntax) in Fedora already to have placeholder data for non-free stuff, which gets overridden by the real metadata if a new application was added. In general, the property tag was added to make the answer to the question which data is preferred much less magic. The libappstream library got some new API to query component data in different ways, and I also brought back support for Vala (so if you missed the Vapi file: It s back now, although you have to manually enable this feature). The CLI tool also got some extensions to query AppStream data. Here is a brief introduction: First of all, we need to make sure the database is up-to-date, which should be the case already (it is rebuilt automatically):
$ sudo appstream-index --refresh
The database will only be rebuilt when necessary, if you want to force a rebuild anyway, use the force parameter. Now imagine we want to search for an app containing the word media (in description, keywords, summary, ):
$ appstream-index -s media
which will return:
Identifier: gnome-media-player.desktop [desktop-app]
Name: GNOME Media Player
Summary: A simple media player for GNOME
Package: gnome-media-player
----
Identifier: mediaplayer-app.desktop [desktop-app]
Name: Media Player
Summary: Media Player
Package: mediaplayer-app
----
Identifier: kde4__plasma-mediacenter.desktop [desktop-app]
Name: Plasma Media Center
Summary: A mediacenter user interface written with the Plasma framework
Package: plasma-mediacenter
----
etc.
If we already know the name if a .desktop-file or the ID of a component, we can have the tool print out information about the application, including which package it was installed from:
$ appstream-index --get lyx.desktop
If we want to see more details, including e.g. a screenshot URL and a longer description, we can pass details to the tool:
Identifier: lyx.desktop [desktop-app]
Name: LyX
Summary: An advanced document processor with the power of LaTeX.
Package: lyx-common
Homepage: http://www.lyx.org/
Icon: lyx.png
Description: LyX is a document processor that encourages an approach to writing
 based on the structure of your documents (WYSIWYM) and not simply
 their appearance (WYSIWYG).
 
 LyX combines the power and flexibility of TeX/LaTeX[...]
Sample Screenshot URL: http://alt.fedoraproject.org/pub/alt/screenshots/f21/source/lyx-ea535ddf18b5c7328c5e88d2cd2cbd8c.png
License: GPLv2+
(I truncated the results slightly ;-) ) Okay, so far so good. But now it gets really exciting (and this is a feature added with 0.6.1): We can now query a component by the items it provides. For example, I want to know which software provides the library libfoo.2:
appstream-index --what-provides -t lib -v libfoo.so.2
This also works with binaries, or Python modules:
appstream-index --what-provides -t bin -v apper
This stuff works distribution-agnostic, and as long as software ships upstream metadata with a valid <provides/> field, or the distributor adds it while generating AppStream distro metadata.
This means that software can as soon as we have sufficient metadata of this kind declare it s dependencies upstream in form of a simple text file, referencing the needed components to build and run it on any Linux distribution. Users can simply install missing stuff by passing that file to their package manager, which can look up the components->packaging mapping and versions and do the right thing in installing the dependencies. So basically, this allows things pip -r does for Python, but for any application (not only Python stuff), and based on the distributors package database. With the provides-items, we can also scan software to detect it s dependencies automatically (and have it in a distro-agnostic form directly). We can also easily search for missing mimetype-handlers, missing kernel-modules, missing firmware etc. to install it on-demand, making the system much smarter in handling it s dependencies. And users don t need to do search orgies to find the right component for a given task. Also on my todo list for the future, based on this feature: A small tool telling upstream authors which distribution has their application in which version, using just one command (and AppStream data from multiple distros). Also planned: A cross-distro information page showing which distros ship which library versions, Python modules and application versions (and also the support status of the distro), so developers know which library versions (or GCC versions etc.) they should at least support to make their application easily available on most distributions. As always, you can get the releases on Freedesktop, as well es the AppStream specification.

13 April 2014

C.J. Adams-Collier: When was the last time you upgraded from squeeze to wheezy?

Wow. 3G delta. I haven t booted this laptop for a while I think I m finally ready to make the move from gnome2 to gnome3. There are bits that still annoy me, but I think it s off to a good start. Upgrading perl from 5.10 to 5.14.
cjac@calcifer:~$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be REMOVED:
  at-spi capplets-data compiz compiz-gnome compiz-gtk defoma deskbar-applet g++-4.3 gcc-4.3 gcj-4.4-base gcj-4.4-jre gcj-4.4-jre-headless gcj-4.4-jre-lib
  gdm3 gir1.0-clutter-1.0 gir1.0-freedesktop gir1.0-glib-2.0 gir1.0-gstreamer-0.10 gir1.0-gtk-2.0 gir1.0-json-glib-1.0 glade-gnome gnome-about
  gnome-accessibility gnome-applets gnome-core gnome-panel gnome-utils-common lib32readline5-dev libbrasero-media0 libclass-mop-perl libdb4.7-java
  libdb4.8-dev libdevhelp-1-1 libdigest-sha1-perl libdirectfb-dev libebook1.2-9 libecal1.2-7 libedata-book1.2-2 libedata-cal1.2-7 libedataserverui1.2-8
  libepc-1.0-2 libepc-ui-1.0-2 libept1 libgcj10 libgcj10-awt libgd2-noxpm libgstfarsight0.10-0 libgtkhtml-editor0 libjpeg62-dev libmetacity-private0
  libmono-accessibility1.0-cil libmono-bytefx0.7.6.1-cil libmono-cairo1.0-cil libmono-cil-dev libmono-corlib1.0-cil libmono-cscompmgd7.0-cil
  libmono-data-tds1.0-cil libmono-data1.0-cil libmono-debugger-soft0.0-cil libmono-getoptions1.0-cil libmono-i18n-west1.0-cil libmono-i18n1.0-cil
  libmono-ldap1.0-cil libmono-microsoft7.0-cil libmono-npgsql1.0-cil libmono-oracle1.0-cil libmono-peapi1.0-cil libmono-posix1.0-cil
  libmono-relaxng1.0-cil libmono-security1.0-cil libmono-sharpzip0.6-cil libmono-sharpzip0.84-cil libmono-sqlite1.0-cil libmono-system-data1.0-cil
  libmono-system-ldap1.0-cil libmono-system-messaging1.0-cil libmono-system-runtime1.0-cil libmono-system-web1.0-cil libmono-system1.0-cil
  libmono-webbrowser0.5-cil libmono-winforms1.0-cil libmono1.0-cil libmtp8 libnautilus-extension1 libpango1.0-common libperl5.10 libpolkit-gtk-1-0
  libpulse-browse0 librpm1 librpmbuild1 libsdl1.2-dev libsdl1.2debian-pulseaudio libseed0 libstdc++6-4.3-dev libtelepathy-farsight0 libupnp3 libvlccore4
  libxmlrpc-c3 linphone-nox linux-headers-2.6.32-5-amd64 linux-sound-base metacity mono-2.0-devel mono-devel mysql-client-5.1 mysql-query-browser
  mysql-server-5.1 mysql-server-core-5.1 openoffice.org-base-core openoffice.org-core openoffice.org-gcj openoffice.org-report-builder-bin
  openoffice.org-style-andromeda php5-suhosin portmap python-beagle python-brasero python-docky python-encutils python-evince python-gnomeapplet
  python-gtop python-mediaprofiles python-metacity python-totem-plparser seahorse-plugins smbfs speedbar totem-coherence tqsllib1c2a unixcw vlc
  xserver-xorg-video-nv
The following NEW packages will be installed:
  accountsservice acl aisleriot apg aptdaemon-data aptitude-common asterisk-core-sounds-en asterisk-modules asterisk-moh-opsound-gsm at-spi2-core
  ax25-node bluez btrfs-tools caribou caribou-antler chromium chromium-inspector colord console-setup console-setup-linux cpp-4.6 cpp-4.7 crda
  cryptsetup-bin cups-filters db-util db5.1-util dconf-gsettings-backend dconf-service dconf-tools distro-info-data docutils-common docutils-doc enchant
  extlinux finger folks-common fonts-cantarell fonts-droid fonts-freefont-ttf fonts-horai-umefont fonts-lg-aboriginal fonts-liberation fonts-lyx
  fonts-opensymbol fonts-sil-gentium fonts-sil-gentium-basic fonts-sipa-arundina fonts-stix fonts-takao fonts-takao-gothic fonts-takao-mincho
  fonts-thai-tlwg fonts-tlwg-garuda fonts-tlwg-kinnari fonts-tlwg-loma fonts-tlwg-mono fonts-tlwg-norasi fonts-tlwg-purisa fonts-tlwg-sawasdee
  fonts-tlwg-typewriter fonts-tlwg-typist fonts-tlwg-typo fonts-tlwg-umpush fonts-tlwg-waree fonts-umeplus fuse g++-4.7 g++-4.7-multilib gcc-4.6
  gcc-4.6-base gcc-4.7 gcc-4.7-base gcc-4.7-multilib gcj-4.7-base gcj-4.7-jre gcj-4.7-jre-headless gcj-4.7-jre-lib gconf-service gcr
  gir1.2-accountsservice-1.0 gir1.2-atk-1.0 gir1.2-atspi-2.0 gir1.2-caribou-1.0 gir1.2-clutter-1.0 gir1.2-clutter-gst-1.0 gir1.2-cogl-1.0
  gir1.2-coglpango-1.0 gir1.2-evince-3.0 gir1.2-folks-0.6 gir1.2-freedesktop gir1.2-gck-1 gir1.2-gconf-2.0 gir1.2-gcr-3 gir1.2-gdesktopenums-3.0
  gir1.2-gdkpixbuf-2.0 gir1.2-gee-1.0 gir1.2-gkbd-3.0 gir1.2-glib-2.0 gir1.2-gmenu-3.0 gir1.2-gnomebluetooth-1.0 gir1.2-gnomekeyring-1.0
  gir1.2-gst-plugins-base-0.10 gir1.2-gstreamer-0.10 gir1.2-gtk-3.0 gir1.2-gtkclutter-1.0 gir1.2-gtksource-3.0 gir1.2-gtop-2.0 gir1.2-gucharmap-2.90
  gir1.2-javascriptcoregtk-3.0 gir1.2-json-1.0 gir1.2-mutter-3.0 gir1.2-networkmanager-1.0 gir1.2-notify-0.7 gir1.2-panelapplet-4.0 gir1.2-pango-1.0
  gir1.2-peas-1.0 gir1.2-polkit-1.0 gir1.2-rb-3.0 gir1.2-soup-2.4 gir1.2-telepathyglib-0.12 gir1.2-telepathylogger-0.2 gir1.2-totem-1.0
  gir1.2-totem-plparser-1.0 gir1.2-upowerglib-1.0 gir1.2-vte-2.90 gir1.2-webkit-3.0 gir1.2-wnck-3.0 gir1.2-xkl-1.0 git-man gjs gkbd-capplet glchess
  glib-networking glib-networking-common glib-networking-services glines gnect gnibbles gnobots2 gnome-bluetooth gnome-contacts gnome-control-center-data
  gnome-desktop3-data gnome-font-viewer gnome-icon-theme-extras gnome-icon-theme-symbolic gnome-online-accounts gnome-packagekit gnome-packagekit-data
  gnome-shell gnome-shell-common gnome-sudoku gnome-sushi gnome-themes-standard gnome-themes-standard-data gnome-user-share gnome-video-effects gnomine
  gnotravex gnotski gnuplot gnuplot-nox grilo-plugins-0.1 groff growisofs gsettings-desktop-schemas gstreamer0.10-gconf gtali guile-2.0-libs gvfs-common
  gvfs-daemons gvfs-libs hardening-includes hwdata iagno ienglish-common imagemagick-common ioquake3 ioquake3-server iputils-tracepath ipxe-qemu iw
  keyutils kmod krb5-locales lib32itm1 lib32quadmath0 lib32tinfo-dev lib32tinfo5 libaacplus2 libaacs0 libabiword-2.9 libaccountsservice0 libamd2.2.0
  libapache-pom-java libapol4 libapt-inst1.5 libapt-pkg4.12 libaqbanking-plugins-libgwenhywfar60 libaqbanking34 libaqbanking34-plugins libaqhbci20
  libaqofxconnect7 libarchive12 libasprintf0c2 libassuan0 libatk-adaptor libatk-adaptor-data libatk-bridge2.0-0 libatkmm-1.6-1 libatkmm-1.6-dev
  libatspi2.0-0 libaudiofile1 libavahi-ui-gtk3-0 libavcodec53 libavcodec54 libavformat53 libavformat54 libavutil51 libbabl-0.1-0 libbind9-80 libbison-dev
  libblas3 libbluray1 libboost-iostreams1.49.0 libboost-program-options1.49.0 libboost-python1.49.0 libboost-serialization1.49.0 libboost-thread1.49.0
  libbrasero-media3-1 libcairo-gobject2 libcairo-script-interpreter2 libcamel-1.2-33 libcanberra-dev libcanberra-gtk3-0 libcanberra-gtk3-module
  libcanberra-pulse libcapi20-3 libcaribou-common libcaribou-gtk-module libcaribou-gtk3-module libcaribou0 libccrtp0 libcdio-cdda1 libcdio-paranoia1
  libcdio13 libcfg4 libchamplain-0.12-0 libchamplain-gtk-0.12-0 libcheese-gtk21 libcheese3 libclass-factory-util-perl libclass-isa-perl libclass-load-perl
  libclass-load-xs-perl libclutter-1.0-common libclutter-gst-1.0-0 libclutter-gtk-1.0-0 libclutter-imcontext-0.1-0 libclutter-imcontext-0.1-bin
  libcluttergesture-0.0.2-0 libcmis-0.2-0 libcogl-common libcogl-pango0 libcogl9 libcolord1 libcommons-parent-java libconfdb4 libcoroipcc4 libcoroipcs4
  libcpg4 libcryptsetup4 libcrystalhd3 libcupsfilters1 libcw3 libdata-alias-perl libdatetime-format-builder-perl libdatetime-format-iso8601-perl
  libdb-java libdb5.1 libdb5.1-dev libdb5.1-java libdb5.1-java-jni libdbus-c++-1-0 libdbus-glib1.0-cil libdbus1.0-cil libdconf0 libdee-1.0-4
  libdevel-partialdump-perl libdevhelp-3-0 libdevmapper-event1.02.1 libdistro-info-perl libdmapsharing-3.0-2 libdns88 libdotconf1.0 libdvbpsi7
  libebackend-1.2-2 libebml3 libebook-1.2-13 libecal-1.2-11 libecore1 libedata-book-1.2-13 libedata-cal-1.2-15 libedataserver-1.2-16
  libedataserverui-3.0-1 libeina1 libemail-valid-perl libencode-locale-perl libepc-1.0-3 libepc-ui-1.0-3 libept1.4.12 libescpr1 libev4
  libeval-closure-perl libevdocument3-4 libevent-2.0-5 libevent-perl libevs4 libevview3-3 libexiv2-12 libexosip2-7 libexporter-lite-perl
  libexttextcat-data libexttextcat0 libfakechroot libfarstream-0.1-0 libfdk-aac0 libfdt1 libfile-basedir-perl libfile-desktopentry-perl
  libfile-fcntllock-perl libfile-listing-perl libfile-mimeinfo-perl libfltk-images1.3 libfltk1.3 libfolks-eds25 libfolks-telepathy25 libfolks25
  libfont-afm-perl libgail-3-0 libgcj13 libgcj13-awt libgck-1-0 libgconf-2-4 libgconf2-doc libgcr-3-1 libgcr-3-common libgd2-xpm libgdata13
  libgdata2.1-cil libgdict-common libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgdk-pixbuf2.0-dev libgegl-0.2-0 libgeocode-glib0 libgettextpo0 libgexiv2-1
  libgirepository-1.0-1 libgjs0b libgkeyfile1.0-cil libgladeui-2-0 libgladeui-common libglapi-mesa libglew1.7 libglib2.0-bin libgmime-2.6-0
  libgmime2.6-cil libgmp10 libgnome-bluetooth10 libgnome-desktop-3-2 libgnome-keyring-common libgnome-media-profiles-3.0-0 libgnome-menu-3-0 libgnomekbd7
  libgnutls-openssl27 libgnutlsxx27 libgoa-1.0-0 libgoa-1.0-common libgphoto2-l10n libgraphite2-2.0.0 libgrilo-0.1-0 libgs9 libgs9-common libgssdp-1.0-3
  libgstreamer-plugins-bad0.10-0 libgtk-3-0 libgtk-3-bin libgtk-3-common libgtk-3-dev libgtk-3-doc libgtk-sharp-beans-cil libgtk-vnc-2.0-0
  libgtkhtml-4.0-0 libgtkhtml-4.0-common libgtkhtml-editor-4.0-0 libgtkmm-3.0-1 libgtksourceview-3.0-0 libgtksourceview-3.0-common libgucharmap-2-90-7
  libgudev1.0-cil libgupnp-1.0-4 libgupnp-av-1.0-2 libgupnp-igd-1.0-4 libgusb2 libgvnc-1.0-0 libgweather-3-0 libgwenhywfar-data libgwenhywfar60 libgxps2
  libhcrypto4-heimdal libheimbase1-heimdal libhtml-form-perl libhtml-format-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl
  libhttp-message-perl libhttp-negotiate-perl libhunspell-1.3-0 libicu48 libimobiledevice2 libio-aio-perl libisc84 libisccc80 libisccfg82 libiscsi1
  libiso9660-8 libisoburn1 libitm1 libjavascriptcoregtk-1.0-0 libjavascriptcoregtk-3.0-0 libjbig0 libjs-sphinxdoc libjs-underscore libjson0 libjte1
  libkadm5clnt-mit8 libkadm5srv-mit8 libkarma0 libkdb5-6 libkmod2 libkpathsea6 liblapack3 liblavfile-2.0-0 liblavjpeg-2.0-0 liblavplay-2.0-0 liblcms2-2
  liblensfun-data liblensfun0 liblinear-tools liblinear1 liblinphone4 liblockfile-bin liblogsys4 liblvm2app2.2 liblwp-mediatypes-perl
  liblwp-protocol-https-perl liblwres80 liblzma5 libmaa3 libmagick++5 libmagickcore5 libmagickcore5-extra libmagickwand5 libmath-bigint-perl
  libmath-round-perl libmatroska5 libmediastreamer1 libmhash2 libminiupnpc5 libmission-control-plugins0 libmjpegutils-2.0-0 libmodule-implementation-perl
  libmodule-runtime-perl libmono-2.0-1 libmono-2.0-dev libmono-accessibility4.0-cil libmono-cairo4.0-cil libmono-codecontracts4.0-cil
  libmono-compilerservices-symbolwriter4.0-cil libmono-corlib4.0-cil libmono-csharp4.0-cil libmono-custommarshalers4.0-cil libmono-data-tds4.0-cil
  libmono-debugger-soft2.0-cil libmono-debugger-soft4.0-cil libmono-http4.0-cil libmono-i18n-cjk4.0-cil libmono-i18n-mideast4.0-cil
  libmono-i18n-other4.0-cil libmono-i18n-rare4.0-cil libmono-i18n-west4.0-cil libmono-i18n4.0-all libmono-i18n4.0-cil libmono-ldap4.0-cil
  libmono-management4.0-cil libmono-messaging-rabbitmq4.0-cil libmono-messaging4.0-cil libmono-microsoft-build-engine4.0-cil
  libmono-microsoft-build-framework4.0-cil libmono-microsoft-build-tasks-v4.0-4.0-cil libmono-microsoft-build-utilities-v4.0-4.0-cil
  libmono-microsoft-csharp4.0-cil libmono-microsoft-visualc10.0-cil libmono-microsoft-web-infrastructure1.0-cil libmono-npgsql4.0-cil
  libmono-opensystem-c4.0-cil libmono-oracle4.0-cil libmono-peapi4.0-cil libmono-posix4.0-cil libmono-rabbitmq4.0-cil libmono-relaxng4.0-cil
  libmono-security4.0-cil libmono-sharpzip4.84-cil libmono-simd4.0-cil libmono-sqlite4.0-cil libmono-system-componentmodel-composition4.0-cil
  libmono-system-componentmodel-dataannotations4.0-cil libmono-system-configuration-install4.0-cil libmono-system-configuration4.0-cil
  libmono-system-core4.0-cil libmono-system-data-datasetextensions4.0-cil libmono-system-data-linq4.0-cil libmono-system-data-services-client4.0-cil
  libmono-system-data-services4.0-cil libmono-system-data4.0-cil libmono-system-design4.0-cil libmono-system-drawing-design4.0-cil
  libmono-system-drawing4.0-cil libmono-system-dynamic4.0-cil libmono-system-enterpriseservices4.0-cil libmono-system-identitymodel-selectors4.0-cil
  libmono-system-identitymodel4.0-cil libmono-system-ldap4.0-cil libmono-system-management4.0-cil libmono-system-messaging4.0-cil
  libmono-system-net4.0-cil libmono-system-numerics4.0-cil libmono-system-runtime-caching4.0-cil libmono-system-runtime-durableinstancing4.0-cil
  libmono-system-runtime-serialization-formatters-soap4.0-cil libmono-system-runtime-serialization4.0-cil libmono-system-runtime4.0-cil
  libmono-system-security4.0-cil libmono-system-servicemodel-discovery4.0-cil libmono-system-servicemodel-routing4.0-cil
  libmono-system-servicemodel-web4.0-cil libmono-system-servicemodel4.0-cil libmono-system-serviceprocess4.0-cil libmono-system-transactions4.0-cil
  libmono-system-web-abstractions4.0-cil libmono-system-web-applicationservices4.0-cil libmono-system-web-dynamicdata4.0-cil
  libmono-system-web-extensions-design4.0-cil libmono-system-web-extensions4.0-cil libmono-system-web-routing4.0-cil libmono-system-web-services4.0-cil
  libmono-system-web4.0-cil libmono-system-windows-forms-datavisualization4.0-cil libmono-system-windows-forms4.0-cil libmono-system-xaml4.0-cil
  libmono-system-xml-linq4.0-cil libmono-system-xml4.0-cil libmono-system4.0-cil libmono-tasklets4.0-cil libmono-web4.0-cil libmono-webbrowser2.0-cil
  libmono-webbrowser4.0-cil libmono-webmatrix-data4.0-cil libmono-windowsbase4.0-cil libmount1 libmozjs10d libmozjs17d libmozjs185-1.0 libmpeg2encpp-2.0-0
  libmplex2-2.0-0 libmtdev1 libmtp-common libmtp-runtime libmtp9 libmupen64plus2 libmusicbrainz-discid-perl libmusicbrainz5-0 libmutter0 libmx-1.0-2
  libmx-bin libmx-common libmysqlclient18 libnatpmp1 libnautilus-extension1a libnet-domain-tld-perl libnet-http-perl libnet-ip-minimal-perl libnetcf1
  libnetfilter-conntrack3 libnettle4 libnewtonsoft-json4.5-cil libnice10 libnl-3-200 libnl-genl-3-200 libnl-route-3-200 libnm-glib4 libnm-gtk-common
  libnm-gtk0 libnm-util2 libnotify4 libnspr4 libnss-winbind libnss3 libnuma1 libnunit2.6-cil liboauth0 libodbc1 liboobs-1-5 libopal3.10.4 libopenal-data
  libopus0 libosip2-7 libp11-2 libp11-kit-dev libp11-kit0 libpackage-stash-xs-perl libpackagekit-glib2-14 libpam-cap libpam-modules-bin libpam-winbind
  libpanel-applet-4-0 libparams-classify-perl libpcre3-dev libpcrecpp0 libpeas-1.0-0 libpeas-common libperl5.14 libpipeline1 libpload4 libpodofo0.9.0
  libpoe-component-resolver-perl libpoppler-glib8 libpoppler19 libportsmf0 libpostproc52 libprocps0 libpst4 libpt2.10.4 libptexenc1 libpython2.7
  libqt4-declarative libqtassistantclient4 libqtdbus4 libqtwebkit4 libquadmath0 libquicktime2 libquorum4 libquvi-scripts libquvi7 libraptor2-0 librasqal3
  libraw5 libregexp-reggrp-perl libreoffice libreoffice-base libreoffice-base-core libreoffice-calc libreoffice-common libreoffice-core libreoffice-draw
  libreoffice-emailmerge libreoffice-evolution libreoffice-filter-binfilter libreoffice-filter-mobiledev libreoffice-gnome libreoffice-gtk
  libreoffice-help-en-us libreoffice-impress libreoffice-java-common libreoffice-math libreoffice-officebean libreoffice-report-builder-bin
  libreoffice-style-galaxy libreoffice-style-tango libreoffice-writer libresid-builder0c2a librest-0.7-0 librest-extras-0.7-0 librhythmbox-core6 librpm3
  librpmbuild3 librpmio3 librpmsign1 libruby1.9.1 libsaamf3 libsackpt3 libsaclm3 libsaevt3 libsalck3 libsam4 libsamsg4 libsane-common
  libsane-extras-common libsatmr3 libsbsms10 libseed-gtk3-0 libsidplay2 libsigsegv2 libsocialweb-client2 libsocialweb-common libsocialweb-service
  libsocialweb0 libsocket-getaddrinfo-perl libsocket-perl libsonic0 libsoundtouch0 libsox2 libspeechd2 libspice-client-glib-2.0-1
  libspice-client-gtk-2.0-1 libspice-server1 libssl-doc libssl1.0.0 libstdc++6-4.7-dev libsvm-tools libswitch-perl libswscale2 libsystemd-daemon0
  libsystemd-login0 libtagc0 libtelepathy-farstream2 libtelepathy-logger2 libtest-warn-perl libtinfo-dev libtinfo5 libtirpc1 libtokyocabinet9 libtotem-pg4
  libtotem0 libtqsllib1 libtracker-sparql-0.14-0 libtree-dagnode-perl libts-dev libucommon5 libumfpack5.4.0 libunique-3.0-0 libupnp6 libusbredirhost1
  libusbredirparser0 libv4lconvert0 libverto-libev1 libverto1 libvisio-0.0-0 libvlccore5 libvo-aacenc0 libvo-amrwbenc0 libvorbisidec1 libvotequorum4
  libvpx1 libvte-2.90-9 libvte-2.90-common libwacom-common libwacom2 libwebkitgtk-1.0-0 libwebkitgtk-1.0-common libwebkitgtk-3.0-0 libwebkitgtk-3.0-common
  libwebp2 libwebrtc-audio-processing-0 libwildmidi-config libwireshark-data libwireshark2 libwiretap2 libwnck-3-0 libwnck-3-common libwpd-0.9-9
  libwpg-0.2-2 libwps-0.2-2 libwsutil2 libwv-1.2-4 libwww-robotrules-perl libx11-doc libx11-protocol-perl libx264-123 libx264-124 libx264-130 libx264-132
  libxalan2-java libxcb-composite0 libxcb-glx0 libxcb-shape0 libxcb-shm0-dev libxcb-util0 libxen-4.1 libxml-commons-external-java
  libxml-commons-resolver1.1-java libxml-sax-base-perl libxmlrpc-c++4 libxmlrpc-core-c3 libxz-java libyajl2 libyaml-0-2 libyaml-perl libyelp0 libzrtpcpp2
  libzvbi-common libzvbi0 lightsoff linphone-nogtk linux-headers-3.2.0-4-amd64 linux-headers-3.2.0-4-common linux-headers-amd64 linux-image-3.2.0-4-amd64
  linux-image-amd64 linux-kbuild-3.2 live-boot-doc live-config-doc live-manual-html mahjongg memtest86+ minissdpd mono-4.0-gac mono-dmcs mscompress
  multiarch-support mupen64plus-audio-all mupen64plus-audio-sdl mupen64plus-data mupen64plus-input-all mupen64plus-input-sdl mupen64plus-rsp-all
  mupen64plus-rsp-hle mupen64plus-rsp-z64 mupen64plus-ui-console mupen64plus-video-all mupen64plus-video-arachnoid mupen64plus-video-glide64
  mupen64plus-video-rice mupen64plus-video-z64 mutter-common mysql-client-5.5 mysql-server-5.5 mysql-server-core-5.5 mythes-en-us openarena-081-maps
  openarena-081-misc openarena-081-players openarena-081-players-mature openarena-081-textures openarena-085-data openarena-088-data packagekit
  packagekit-backend-aptcc packagekit-tools planner-data planner-doc poppler-data printer-driver-all printer-driver-c2050 printer-driver-c2esp
  printer-driver-cjet printer-driver-escpr printer-driver-foo2zjs printer-driver-gutenprint printer-driver-hpcups printer-driver-hpijs
  printer-driver-m2300w printer-driver-min12xxw printer-driver-pnm2ppa printer-driver-postscript-hp printer-driver-ptouch printer-driver-pxljr
  printer-driver-sag-gdi printer-driver-splix psutils python-aptdaemon.gtk3widgets python-aptdaemon.gtkwidgets python-bzrlib python-dbus-dev
  python-debianbts python-defer python-dnspython python-fpconst python-gi python-gi-cairo python-gi-dev python-gobject-2 python-gobject-2-dev
  python-keyring python-launchpadlib python-lazr.restfulclient python-lazr.uri python-liblarch python-liblarch-gtk python-magic python-oauth
  python-packagekit python-pyatspi2 python-pyparsing python-repoze.lru python-routes python-setools python-simplejson python-soappy python-speechd
  python-spice-client-gtk python-wadllib python-webob python-zeitgeist python2.7 python2.7-dev python2.7-minimal qdbus quadrapassel remmina-common
  rhythmbox-data rpcbind rtkit ruby ruby1.9.1 shotwell-common smartdimmer software-properties-common sound-theme-freedesktop speech-dispatcher
  sphinx-common sphinx-doc swell-foop syslinux-themes-debian syslinux-themes-debian-wheezy tdb-tools telepathy-haze telepathy-logger telepathy-rakia
  tex-gyre ttf-marvosym wireless-regdb xbrlapi xorg-sgml-doctools xorriso xserver-xorg-input-mouse xserver-xorg-input-vmmouse xulrunner-17.0 yelp-xsl
  zeitgeist-core zenity-common
The following packages have been kept back:
  acroread-debian-files db4.8-util hibernate ia32-libs ia32-libs-gtk libboost-dev libboost-serialization-dev opensc wine
The following packages will be upgraded:
  abcde abiword abiword-common abiword-plugin-grammar abiword-plugin-mathview acpi acpi-fakekey acpi-support acpi-support-base acpid acroread-data
  acroread-dictionary-en acroread-l10n-en adduser alacarte alsa-base alsa-utils amb-plugins anacron analog ant ant-optional apache2 apache2-doc
  apache2-mpm-prefork apache2-utils apache2.2-bin apache2.2-common app-install-data apt apt-file apt-utils apt-xapian-index aptdaemon aptitude
  aqbanking-tools aspell aspell-en asterisk asterisk-config asterisk-core-sounds-en-gsm asterisk-doc asterisk-voicemail astyle at audacity audacity-data
  augeas-lenses augeas-tools autoconf autoconf-doc automake automake1.9 autopoint autotools-dev avahi-autoipd avahi-daemon avidemux avidemux-common
  avidemux-plugins aview ax25-tools banshee baobab base-files base-passwd bash bash-completion bc bind9-doc bind9-host bind9utils binfmt-support binutils
  bison bluez-cups bogofilter bogofilter-bdb bogofilter-common brasero brasero-common bridge-utils browser-plugin-gnash bsd-mailx bsdmainutils bsdutils
  busybox buzztard buzztard-data bwidget bzip2 bzr bzrtools ca-certificates calibre calibre-bin ccache cd-discid cdebootstrap cdparanoia cdrdao
  checkpolicy cheese cheese-common chromium-browser chromium-browser-inspector cifs-utils cl-asdf cli-common clisp comerr-dev common-lisp-controller
  console-common console-data console-tools consolekit coreutils cowbuilder cowdancer cpio cpp cpp-4.4 cpufrequtils cracklib-runtime crawl-common
  crawl-tiles cron cryptsetup cups cups-bsd cups-client cups-common cups-driver-gutenprint cups-pk-helper cups-ppdc cupsddk curl curlftpfs cvs cw dash
  dasher dasher-data dbus dbus-x11 dc dcraw dctrl-tools debconf debconf-i18n debhelper debian-archive-keyring debian-faq debian-keyring debianutils debirf
  debootstrap desktop-base desktop-file-utils devhelp devhelp-common devscripts dialog dict dictionaries-common diffstat diffutils djtools dkms dmidecode
  dmsetup dnsmasq-base dnsutils doc-debian docbook docbook-dsssl docbook-to-man docbook-utils docbook-xml docbook-xsl docbook-xsl-doc-html docky dosemu
  dosfstools dpatch dpkg dpkg-dev dput dvd+rw-tools dvi2ps dynagen dynamips e2fslibs e2fsprogs ebtables ed eject ekiga emacs23-bin-common emacs23-common
  emacs23-nox emacsen-common emdebian-archive-keyring empathy empathy-common eog epiphany-browser epiphany-browser-data epiphany-extensions esound-common
  espeak espeak-data ethtool evince evince-common evolution evolution-common evolution-data-server evolution-data-server-common evolution-exchange
  evolution-plugins evolution-webcal exif exiftags exim4 exim4-base exim4-config exim4-daemon-light exiv2 f-spot fakechroot fakeroot fancontrol fceu
  fcrackzip fdupes feynmf file file-roller finch findutils firmware-iwlwifi firmware-linux-free firmware-linux-nonfree flac flashrom fldigi flex
  fontconfig fontconfig-config foo2zjs foomatic-db foomatic-db-engine foomatic-db-gutenprint foomatic-filters fping freedesktop-sound-theme freeglut3
  freetds-common ftp fuse-utils g++ g++-4.4 g++-4.4-multilib g++-multilib gawk gcalctool gcc gcc-4.4 gcc-4.4-base gcc-4.4-doc gcc-4.4-multilib
  gcc-doc-base gcc-multilib gcj-jre gcj-jre-headless gconf-defaults-service gconf-editor gconf2 gconf2-common gddrescue gdebi gdebi-core gedit
  gedit-common gedit-plugins genisoimage geoclue geoclue-hostip geoclue-localnet geoclue-manual geoclue-yahoo geoip-database gettext gettext-base
  ghostscript ghostscript-cups gimp gimp-data git git-buildpackage git-core git-svn gitk gksu glade gnash gnash-common gnash-opengl
  gnome-accessibility-themes gnome-applets-data gnome-backgrounds gnome-cards-data gnome-common gnome-control-center gnome-control-center-dev
  gnome-desktop-data gnome-dictionary gnome-disk-utility gnome-do gnome-do-plugins gnome-doc-utils gnome-games gnome-games-data gnome-games-extra-data
  gnome-icon-theme gnome-js-common gnome-keyring gnome-mag gnome-media gnome-menus gnome-nettool gnome-orca gnome-panel-data gnome-pkg-tools
  gnome-power-manager gnome-rdp gnome-screensaver gnome-screenshot gnome-search-tool gnome-session gnome-session-bin gnome-session-canberra
  gnome-session-common gnome-settings-daemon gnome-settings-daemon-dev gnome-system-log gnome-system-monitor gnome-system-tools gnome-terminal
  gnome-terminal-data gnome-user-guide gnomint gnu-fdisk gnucash-docs gnuchess gnumeric gnumeric-common gnupg gnupg-agent gocr google-talkplugin gparted
  gpgv gpredict gpscorrelate grep groff-base grub-common grub-legacy gsfonts-x11 gsmartcontrol gstreamer0.10-alsa gstreamer0.10-buzztard
  gstreamer0.10-buzztard-doc gstreamer0.10-doc gstreamer0.10-ffmpeg gstreamer0.10-ffmpeg-dbg gstreamer0.10-fluendo-mp3 gstreamer0.10-gnonlin
  gstreamer0.10-gnonlin-dbg gstreamer0.10-gnonlin-doc gstreamer0.10-nice gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-dbg
  gstreamer0.10-plugins-bad-doc gstreamer0.10-plugins-base gstreamer0.10-plugins-base-apps gstreamer0.10-plugins-base-dbg gstreamer0.10-plugins-base-doc
  gstreamer0.10-plugins-good gstreamer0.10-plugins-good-dbg gstreamer0.10-plugins-good-doc gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-dbg
  gstreamer0.10-plugins-ugly-doc gstreamer0.10-pulseaudio gstreamer0.10-tools gstreamer0.10-x gtg gthumb gthumb-data gtk2-engines gtk2-engines-pixbuf
  gucharmap guile-1.6 guile-1.6-libs guile-1.8-libs gvfs gvfs-backends gvfs-bin gzip hal hamster-applet hardinfo hddtemp hdparm hfsprogs hostname hp-ppd
  hpijs hplip hplip-cups hplip-data htmldoc htmldoc-common iamerican ibritish iceweasel ifupdown ijsgutenprint imagemagick imagemagick-doc info
  initramfs-tools initscripts inkscape insserv install-info installation-report intltool iotop iproute ipsec-tools iptables iptraf iputils-ping
  ircd-hybrid irssi isc-dhcp-client isc-dhcp-common isc-dhcp-server iscsitarget-dkms iso-codes ispell jack jadetex java-common jigdo-file keyanalyze
  keyboard-configuration keychain klibc-utils kpartx krb5-admin-server krb5-auth-dialog krb5-config krb5-doc krb5-kdc krb5-kdc-ldap krb5-multidev
  krb5-pkinit krb5-user lacheck lame latex-beamer latex-xcolor less lesstif2 lesstif2-dev lib32asound2 lib32bz2-1.0 lib32gcc1 lib32gomp1 lib32ncurses5
  lib32ncurses5-dev lib32nss-mdns lib32readline5 lib32stdc++6 lib32v4l-0 lib32z1 lib32z1-dev liba52-0.7.4 libaa1 libaa1-dev libacl1 libaften0
  libaiksaurus-1.2-0c2a libaiksaurus-1.2-data libaiksaurusgtk-1.2-0c2a libaio1 libalgorithm-diff-xs-perl libany-moose-perl libanyevent-perl libao-common
  libao4 libapache-dbi-perl libapache2-mod-apreq2 libapache2-mod-dnssd libapache2-mod-perl2 libapache2-mod-php5 libapache2-mod-python
  libapache2-request-perl libappconfig-perl libapr1 libapreq2 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libapt-pkg-perl libaqbanking-data
  libarchive-zip-perl libart-2.0-2 libart-2.0-dev libart2.0-cil libasn1-8-heimdal libasound2 libasound2-dev libasound2-plugins libaspell15 libass4
  libasync-interrupt-perl libasyncns0 libatasmart4 libatk1.0-0 libatk1.0-data libatk1.0-dev libatk1.0-doc libatspi1.0-0 libattr1 libaudio-dev libaudio2
  libaudiofile-dev libaudit0 libaugeas0 libavahi-client-dev libavahi-client3 libavahi-common-data libavahi-common-dev libavahi-common3 libavahi-core7
  libavahi-glib-dev libavahi-glib1 libavahi-gobject0 libavahi-ui0 libavc1394-0 libax25 libb-hooks-endofscope-perl libb-keywords-perl libbind9-60
  libblas3gf libblkid1 libbluetooth3 libbml0 libboo2.0.9-cil libbrlapi0.5 libbs2b0 libbsd0 libburn4 libbusiness-paypal-api-perl
  libbusiness-tax-vat-validation-perl libbuzztard0 libbz2-1.0 libc-ares2 libc-bin libc-dev-bin libc6 libc6-dev libc6-dev-i386 libc6-i386 libcaca-dev
  libcaca0 libcache-fastmmap-perl libcairo-perl libcairo2 libcairo2-dev libcairomm-1.0-1 libcairomm-1.0-dev libcanberra-gtk0 libcanberra0 libcap-ng0
  libcap2 libcap2-bin libcapture-tiny-perl libccid libcdaudio1 libcddb-get-perl libcddb2 libcdparanoia0 libcdt4 libchm-bin libchm1 libck-connector0
  libclass-c3-perl libclass-c3-xs-perl libclass-insideout-perl libclass-inspector-perl libclass-method-modifiers-perl libclass-methodmaker-perl
  libclone-perl libclutter-1.0-0 libcolamd2.7.1 libcolor-calc-perl libcomedi0 libcomerr2 libcommon-sense-perl libcommons-beanutils-java
  libcommons-collections3-java libcommons-compress-java libcommons-digester-java libcommons-logging-java libconfig-inifiles-perl libconfig-json-perl
  libconfig-tiny-perl libconsole libcontextual-return-perl libconvert-asn1-perl libcoro-perl libcorosync4 libcpufreq-dev libcpufreq0 libcrack2 libcroco3
  libcrypt-openssl-bignum-perl libcrypt-openssl-random-perl libcrypt-openssl-rsa-perl libcrypt-passwdmd5-perl libcrypt-ssleay-perl libcss-minifier-xs-perl
  libcss-packer-perl libcups2 libcupscgi1 libcupsdriver1 libcupsimage2 libcupsmime1 libcupsppdc1 libcurl3 libcurl3-gnutls libcurses-perl libcwidget3
  libdata-optlist-perl libdata-structure-util-perl libdata-visitor-perl libdatetime-format-http-perl libdatetime-perl libdatetime-set-perl
  libdatetime-timezone-perl libdatrie1 libdb-dev libdb-je-java libdbd-mysql-perl libdbi-perl libdbus-1-3 libdbus-1-dev libdbus-glib-1-2 libdbus-glib-1-dev
  libdc1394-22 libdca0 libdebian-installer-extra4 libdebian-installer4 libdevel-globaldestruction-perl libdevel-size-perl libdevel-stacktrace-perl
  libdevel-symdump-perl libdevmapper1.02.1 libdigest-hmac-perl libdirac-decoder0 libdirac-encoder0 libdirectfb-1.2-9 libdirectfb-extra libdiscid0
  libdjvulibre-text libdjvulibre21 libdns69 libdpkg-perl libdrm-dev libdrm-intel1 libdrm-nouveau1a libdrm-radeon1 libdrm2 libdv4 libdvdcss2 libdvdnav4
  libdvdread4 libedit2 libelf1 libelfg0 libemail-address-perl libenca0 libenchant1c2a libengine-pkcs11-openssl libepc-common libesd0 libesd0-dev
  libespeak1 libevolution libexception-class-perl libexempi3 libexif12 libexpat1 libexpat1-dev libexpect-perl libfaac0 libfaad2 libfcgi-perl libfcgi0ldbl
  libffi-dev libffi5 libfftw3-3 libfile-homedir-perl libfile-libmagic-perl libfile-mmagic-perl libfile-slurp-perl libfile-which-perl libfilter-perl
  libfinance-quote-perl libflac++6 libflac8 libflickrnet2.2-cil libflite1 libfltk1.1 libfluidsynth1 libfontconfig1 libfontconfig1-dev libfontenc1
  libfreetype6 libfreetype6-dev libfribidi0 libfs6 libftdi-dev libftdi1 libfuse2 libgail-common libgail-dev libgail18 libgc1c2 libgcc1 libgcj-bc
  libgcj-common libgconf2-4 libgconf2-dev libgconf2.0-cil libgcrypt11 libgcrypt11-dev libgd-gd2-noxpm-perl libgdata-common libgdbm3 libgdict-1.0-6
  libgdiplus libgdome2-0 libgdome2-cpp-smart0c2a libgdu-gtk0 libgdu0 libgee2 libgeoclue0 libgeoip1 libgfortran3 libgif4 libgimp2.0 libgio-cil libgksu2-0
  libgl1-mesa-dev libgl1-mesa-dri libgl1-mesa-glx libglade2.0-cil libgladeui-1-9 libglib-perl libglib2.0-0 libglib2.0-cil libglib2.0-data libglib2.0-dev
  libglib2.0-doc libglibmm-2.4-1c2a libglibmm-2.4-dev libglu1-mesa libglu1-mesa-dev libgnome-desktop-2-17 libgnome-desktop-dev libgnome-keyring-dev
  libgnome-keyring0 libgnome-keyring1.0-cil libgnome-mag2 libgnome-menu2 libgnome-speech7 libgnome-vfs2.0-cil libgnome2-0 libgnome2-canvas-perl
  libgnome2-common libgnome2-dev libgnome2-perl libgnome2-vfs-perl libgnome2.24-cil libgnomecanvas2-0 libgnomecanvas2-common libgnomecanvas2-dev
  libgnomedesktop2.20-cil libgnomekbd-common libgnomeui-0 libgnomeui-common libgnomeui-dev libgnomevfs2-0 libgnomevfs2-common libgnomevfs2-dev
  libgnomevfs2-extra libgnupg-interface-perl libgnutls-dev libgnutls26 libgoffice-0.8-8 libgoffice-0.8-8-common libgomp1 libgpg-error-dev libgpg-error0
  libgpgme11 libgphoto2-2 libgphoto2-port0 libgpm2 libgpod-common libgpod4 libgraph4 libgsf-1-114 libgsf-1-common libgsl0ldbl libgsm0710-0 libgsm1
  libgssapi-krb5-2 libgssglue1 libgssrpc4 libgstbuzztard0 libgstreamer-plugins-base0.10-0 libgstreamer-plugins-base0.10-dev libgstreamer0.10-0
  libgstreamer0.10-0-dbg libgstreamer0.10-dev libgtk-vnc-1.0-0 libgtk2-perl libgtk2.0-0 libgtk2.0-bin libgtk2.0-cil libgtk2.0-common libgtk2.0-dev
  libgtk2.0-doc libgtkglext1 libgtkhtml3.14-19 libgtkimageview0 libgtkmathview0c2a libgtkmm-2.4-1c2a libgtkmm-2.4-dev libgtop2-7 libgtop2-common
  libgtop2-dev libguard-perl libgudev-1.0-0 libguile-ltdl-1 libgutenprint2 libgvc5 libgweather-common libhal-dev libhal-storage1 libhal1 libhamlib2
  libhpmud0 libhsqldb-java libhtml-packer-perl libhtml-parser-perl libhtml-tableextract-perl libhtml-tagcloud-perl libhtml-template-expr-perl
  libhtml-template-perl libhtml-tree-perl libhtml-treebuilder-xpath-perl libhttp-server-simple-perl libhx509-5-heimdal libhyphen0 libical0 libice-dev
  libice6 libicu44 libicu4j-java libidl-dev libidl0 libidn11 libidn11-dev libieee1284-3 libijs-0.35 libilmbase6 libimage-exif-perl libimage-exiftool-perl
  libio-pty-perl libio-socket-inet6-perl libio-socket-ssl-perl libio-stringy-perl libio-stty-perl libipc-run-perl libiptcdata0 libisc62 libisccc60
  libisccfg62 libisofs6 libiw30 libjack0 libjasper1 libjavascript-minifier-xs-perl libjavascript-packer-perl libjaxp1.3-java libjaxp1.3-java-gcj
  libjbig2dec0 libjline-java libjpeg-progs libjpeg62 libjpeg8 libjs-jquery libjs-yui libjson-any-perl libjson-glib-1.0-0 libjson-perl libjson-xs-perl
  libjtidy-java libk5crypto3 libkadm5clnt-mit7 libkadm5srv-mit7 libkate1 libkdb5-4 libkeyutils1 libklibc libkms1 libkrb5-26-heimdal libkrb5-3
  libkrb5support0 libktoblzcheck1c2a liblapack3gf liblcms1 libldap-2.4-2 liblink-grammar4 liblircclient0 liblist-moreutils-perl liblocale-gettext-perl
  liblocales-perl liblockfile1 liblog-dispatch-perl liblog4c3 liblog4cxx10 libloudmouth1-0 liblouis-data liblouis2 liblqr-1-0 libltdl-dev libltdl7
  liblua5.1-0 liblua5.1-0-dev liblucene2-java liblwres60 liblzo2-2 libmad0 libmagic1 libmagick++3 libmagickcore3 libmagickcore3-extra libmagickwand3
  libmailtools-perl libmeanwhile1 libmime-tools-perl libmime-types-perl libmimic0 libmms0 libmng1 libmodplug1 libmodule-find-perl libmodule-starter-perl
  libmono-accessibility2.0-cil libmono-addins-gui0.2-cil libmono-addins0.2-cil libmono-c5-1.1-cil libmono-cairo2.0-cil libmono-cecil-private-cil
  libmono-corlib2.0-cil libmono-cscompmgd8.0-cil libmono-data-tds2.0-cil libmono-db2-1.0-cil libmono-i18n-west2.0-cil libmono-i18n2.0-cil
  libmono-ldap2.0-cil libmono-management2.0-cil libmono-messaging-rabbitmq2.0-cil libmono-messaging2.0-cil libmono-microsoft-build2.0-cil
  libmono-microsoft8.0-cil libmono-npgsql2.0-cil libmono-oracle2.0-cil libmono-peapi2.0-cil libmono-posix2.0-cil libmono-rabbitmq2.0-cil
  libmono-relaxng2.0-cil libmono-security2.0-cil libmono-sharpzip2.6-cil libmono-sharpzip2.84-cil libmono-simd2.0-cil libmono-sqlite2.0-cil
  libmono-system-data-linq2.0-cil libmono-system-data2.0-cil libmono-system-ldap2.0-cil libmono-system-messaging2.0-cil libmono-system-runtime2.0-cil
  libmono-system-web-mvc1.0-cil libmono-system-web-mvc2.0-cil libmono-system-web2.0-cil libmono-system2.0-cil libmono-tasklets2.0-cil libmono-wcf3.0-cil
  libmono-windowsbase3.0-cil libmono-winforms2.0-cil libmono-zeroconf1.0-cil libmono2.0-cil libmoose-perl libmouse-perl libmp3lame0 libmpc2 libmpcdec6
  libmpfr4 libmpg123-0 libmusicbrainz3-6 libmysqlclient-dev libmysqlclient16 libmythes-1.2-0 libnamespace-autoclean-perl libnamespace-clean-perl
  libncurses5 libncurses5-dev libncursesw5 libncursesw5-dev libndesk-dbus-glib1.0-cil libndesk-dbus1.0-cil libneon27 libneon27-gnutls libnet-daemon-perl
  libnet-dbus-perl libnet-dns-perl libnet-ip-perl libnet-ldap-perl libnet-libidn-perl libnet-netmask-perl libnet-oauth-perl libnet-snmp-perl
  libnet-ssleay-perl libnet1 libnet1-dev libnet6-1.3-0 libnetaddr-ip-perl libnetpbm10 libnewt0.52 libnfnetlink0 libnfsidmap2 libnl1 libnm-glib-dev
  libnm-glib-vpn-dev libnm-glib-vpn1 libnm-util-dev libnotify-dev libnotify0.4-cil libnspr4-0d libnss-mdns libnss3-1d libnunit-cil-dev libofa0 libogg0
  liboobs-1-dev libopenais3 libopenal1 libopencore-amrnb0 libopencore-amrwb0 libopenct1 libopenexr6 libopenjpeg2 libopenraw1 libopenrawgnome1 libopts25
  liborbit2 liborbit2-dev liborc-0.4-0 libortp8 libosp5 libossp-uuid-perl libossp-uuid16 libostyle1c2 libotr2 libots0 libpackage-deprecationmanager-perl
  libpackage-stash-perl libpam-cracklib libpam-gnome-keyring libpam-ldap libpam-modules libpam-p11 libpam-runtime libpam0g libpam0g-dev libpango-perl
  libpango1.0-0 libpango1.0-dev libpango1.0-doc libpangomm-1.4-1 libpangomm-1.4-dev libpaper-utils libpaper1 libparams-util-perl libparams-validate-perl
  libparse-debcontrol-perl libparse-debianchangelog-perl libparse-recdescent-perl libparted0debian1 libpath-class-perl libpathplan4 libpcap0.8
  libpcap0.8-dev libpci3 libpciaccess-dev libpciaccess0 libpcre3 libpcsc-perl libpcsclite-dev libpcsclite1 libperl-critic-perl libperlio-eol-perl
  libphonon4 libpixman-1-0 libpixman-1-dev libpkcs11-helper1 libplist1 libplot2c2 libpng12-0 libpng12-dev libpod-coverage-perl libpoe-api-peek-perl
  libpoe-component-client-http-perl libpoe-component-client-keepalive-perl libpoe-component-ikc-perl libpoe-perl libpolkit-agent-1-0 libpolkit-backend-1-0
  libpolkit-gobject-1-0 libpolkit-gobject-1-dev libpoppler-glib4 libpoppler5 libpopt-dev libpopt0 libportaudio2 libppi-perl libppix-regexp-perl
  libppix-utilities-perl libpq5 libproxy0 libpstoedit0c2a libpthread-stubs0 libpthread-stubs0-dev libpulse-dev libpulse-mainloop-glib0 libpulse0
  libpurple0 libpython2.6 libqdbm14 libqpol1 libqt4-assistant libqt4-core libqt4-dbus libqt4-designer libqt4-gui libqt4-help libqt4-network libqt4-opengl
  libqt4-qt3support libqt4-script libqt4-scripttools libqt4-sql libqt4-sql-mysql libqt4-svg libqt4-test libqt4-webkit libqt4-xml libqt4-xmlpatterns
  libqtcore4 libqtgui4 libraptor1 libraw1394-11 librdf0 libreadline-dev libreadline5 libreadline6 libreadline6-dev libreadonly-perl libreadonly-xs-perl
  librecode0 libregexp-assemble-perl libregexp-common-perl libregexp-java libresample1 libroken18-heimdal librpc-xml-perl librpcsecgss3 librsvg2-2
  librsvg2-2.18-cil librsvg2-common librtmp0 libruby1.8 libsamplerate0 libsane libsane-extras libsane-hpaio libsasl2-2 libsasl2-modules
  libschroedinger-1.0-0 libsctp1 libsdl-image1.2 libsdl-ttf2.0-0 libsdl1.2debian libselinux1 libselinux1-dev libsemanage-common libsemanage1
  libsensors-applet-plugin0 libsensors4 libsepol1 libsepol1-dev libservlet2.5-java libsetools-tcl libsgutils2-2 libshout3 libsigc++-2.0-0c2a
  libsigc++-2.0-dev libslang2 libslang2-dev libslp1 libslv2-9 libsm-dev libsm6 libsmbclient libsmi2ldbl libsndfile1 libsnmp-base libsnmp15
  libsoap-lite-perl libsocket6-perl libsofia-sip-ua-glib3 libsofia-sip-ua0 libsoup-gnome2.4-1 libsoup-gnome2.4-dev libsoup2.4-1 libsoup2.4-dev
  libsox-fmt-all libsox-fmt-alsa libsox-fmt-ao libsox-fmt-base libsox-fmt-ffmpeg libsox-fmt-mp3 libsox-fmt-oss libsox-fmt-pulse libsp1c2 libspandsp2
  libspectre1 libspeex1 libspeexdsp1 libsqlite0 libsqlite3-0 libsqlite3-dev libsrtp0 libss2 libssh-4 libssh2-1 libssl-dev libstartup-notification0
  libstartup-notification0-dev libstdc++6 libstdc++6-4.4-dev libstrongswan libsub-exporter-perl libsub-identify-perl libsub-install-perl libsub-name-perl
  libsub-uplevel-perl libsvga1 libsvga1-dev libsvn-perl libsvn1 libsybdb5 libsysfs-dev libsysfs2 libt1-5 libtag1-vanilla libtag1c2a libtaglib2.0-cil
  libtalloc2 libtar libtasn1-3 libtasn1-3-dev libtdb1 libtelepathy-glib0 libtemplate-perl libterm-readkey-perl libterm-size-perl
  libtest-checkmanifest-perl libtest-class-perl libtest-deep-perl libtest-exception-perl libtest-mockobject-perl libtest-pod-perl libtext-aspell-perl
  libtext-charwidth-perl libtext-csv-perl libtext-csv-xs-perl libtext-iconv-perl libtext-template-perl libthai-data libthai0 libtheora0 libtidy-0.99-0
  libtie-cphash-perl libtie-toobject-perl libtiff4 libtime-format-perl libtool libtotem-plparser17 libtry-tiny-perl libts-0.0-0 libtwolame0 libudev-dev
  libudev0 libuniconf4.6 libunique-1.0-0 libunistring0 libuniversal-can-perl libuniversal-isa-perl libupower-glib-dev libupower-glib1 liburi-perl
  libusb-0.1-4 libusb-1.0-0 libusb-1.0-0-dev libusb-dev libusbmuxd1 libustr-1.0-1 libutempter0 libuuid-perl libuuid1 libv4l-0 libva-x11-1 libva1
  libvamp-hostsdk3 libvariable-magic-perl libvcdinfo0 libvde0 libvdeplug2 libvirt-bin libvirt0 libvisual-0.4-0 libvlc5 libvorbis0a libvorbisenc2
  libvorbisfile3 libvpb0 libvte-common libvte0.16-cil libvte9 libwant-perl libwavpack1 libwbclient0 libwebkit1.1-cil libwildmidi1 libwind0-heimdal
  libwmf0.2-7 libwnck-common libwnck-dev libwnck2.20-cil libwnck22 libwrap0 libwvstreams4.6-base libwvstreams4.6-extras libwww-mechanize-perl libwww-perl
  libwxbase2.8-0 libwxgtk2.8-0 libx11-6 libx11-data libx11-dev libx11-xcb1 libx86-1 libxapian22 libxau-dev libxau6 libxaw7 libxcb-dri2-0 libxcb-keysyms1
  libxcb-randr0 libxcb-render-util0 libxcb-render-util0-dev libxcb-render0 libxcb-render0-dev libxcb-shm0 libxcb-xv0 libxcb1 libxcb1-dev libxcomposite-dev
  libxcomposite1 libxcursor-dev libxcursor1 libxdamage-dev libxdamage1 libxdg-basedir1 libxdmcp-dev libxdmcp6 libxdot4 libxenstore3.0 libxerces2-java
  libxerces2-java-gcj libxext-dev libxext6 libxfixes-dev libxfixes3 libxfont1 libxft-dev libxft2 libxi-dev libxi6 libxinerama-dev libxinerama1
  libxkbfile-dev libxkbfile1 libxklavier-dev libxklavier16 libxml-feedpp-perl libxml-libxml-perl libxml-parser-perl libxml-regexp-perl
  libxml-sax-expat-perl libxml-sax-perl libxml-simple-perl libxml-twig-perl libxml-xpathengine-perl libxml2 libxml2-dev libxml2-doc libxml2-utils libxmu6
  libxmuu1 libxp-dev libxp6 libxpm4 libxrandr-dev libxrandr2 libxrender-dev libxrender1 libxres-dev libxres1 libxslt1-dev libxslt1.1 libxss1 libxt-dev
  libxt6 libxtst6 libxv1 libxvidcore4 libxvmc1 libxxf86dga1 libxxf86vm-dev libxxf86vm1 libyaml-syck-perl libzbar0 libzephyr4 liferea liferea-data
  link-grammar-dictionaries-en links linphone linphone-common lintian linux-base linux-headers-2.6-amd64 linux-headers-2.6.32-5-common
  linux-image-2.6-amd64 linux-image-2.6.32-5-amd64 linux-libc-dev linux-source-2.6.32 live-build lm-sensors lmodern locales lockfile-progs login logjam
  logrotate lsb-base lsb-release lsof luatex lvm2 lwresd lzma m4 make make-doc makedev makepasswd man-db manpages manpages-dev mawk mdadm
  media-player-info mencoder menu mercurial mercurial-common mesa-common-dev mesa-utils metacity-common mic2 mime-support mingw32-binutils mjpegtools
  mktemp mlocate mobile-broadband-provider-info modemmanager module-init-tools mono-2.0-gac mono-csharp-shell mono-gac mono-gmcs mono-mcs mono-runtime
  mono-xbuild mount mousetweaks mozilla-plugin-gnash mpg123 mtd-utils mtools mupen64plus mutt myspell-en-us mysql-client mysql-common mysql-server nano
  nautilus nautilus-data nautilus-sendto nautilus-sendto-empathy nbd-client ncftp ncurses-base ncurses-bin ncurses-term ndisc6 net-tools netatalk netbase
  netcat-openbsd netcat-traditional netenv netpbm network-manager network-manager-dev network-manager-gnome network-manager-openvpn
  network-manager-openvpn-gnome network-manager-vpnc network-manager-vpnc-gnome nfs-common nfs-kernel-server nmap node normalize-audio notification-daemon
  ntp ntpdate nvclock obex-data-server obexd-client odbcinst odbcinst1debian2 open-iscsi openarena openarena-data openarena-server openbios-ppc
  openbios-sparc openbsd-inetd openhackware openjade openocd openoffice.org openoffice.org-base openoffice.org-calc openoffice.org-common
  openoffice.org-draw openoffice.org-emailmerge openoffice.org-evolution openoffice.org-filter-binfilter openoffice.org-filter-mobiledev
  openoffice.org-gnome openoffice.org-gtk openoffice.org-help-en-us openoffice.org-impress openoffice.org-java-common openoffice.org-math
  openoffice.org-officebean openoffice.org-style-tango openoffice.org-thesaurus-en-us openoffice.org-writer openprinting-ppds openssh-blacklist
  openssh-blacklist-extra openssh-client openssh-server openssl openssl-blacklist openvpn openvpn-blacklist orbit2 org-mode os-prober oss-compat p7zip
  p7zip-full parted passwd patch patchutils pavucontrol pavumeter pbuilder pbzip2 pciutils pcmciautils pcsc-tools perl perl-base perl-doc perl-modules
  perlmagick perltidy pgf php-pear php-services-json php5-cli php5-common php5-dev pidgin pidgin-data pidgin-otr pidgin-sipe pinentry-gtk2 pkg-config
  planner pm-utils po-debconf po4a policycoreutils policykit-1 policykit-1-gnome poppler-utils popularity-contest powertop ppp ppp-dev pristine-tar
  procmail procps ps2eps psmisc pstoedit pulseaudio pulseaudio-esound-compat pulseaudio-module-x11 pulseaudio-utils purifyeps pwgen python python-apt
  python-apt-common python-aptdaemon python-aptdaemon-gtk python-axiom python-beautifulsoup python-brlapi python-cairo python-cddb python-central
  python-chardet python-cherrypy3 python-chm python-clientform python-coherence python-configobj python-crypto python-cssutils python-cups
  python-cupshelpers python-dateutil python-dbus python-debian python-demjson python-dev python-django python-django-tagging python-docutils
  python-evolution python-eyed3 python-feedparser python-gconf python-gdata python-gdbm python-glade2 python-gmenu python-gnome2 python-gnome2-desktop-dev
  python-gnome2-dev python-gnome2-doc python-gnomedesktop python-gnomekeyring python-gobject python-gobject-dev python-gpgme python-gst0.10 python-gtk-vnc
  python-gtk2 python-gtk2-dev python-gtk2-doc python-gtkglext1 python-gtksourceview2 python-html5lib python-httplib2 python-imaging python-iniparse
  python-ipy python-jinja2 python-libvirt python-libxml2 python-louis python-lxml python-mako python-markdown python-markupsafe python-mechanize
  python-minimal python-nevow python-notify python-numpy python-ogg python-old-doctools python-opengl python-openssl python-pam python-paramiko
  python-pexpect python-pkg-resources python-pyasn1 python-pyatspi python-pycurl python-pygments python-pykickstart python-pyorbit python-pypdf
  python-pysqlite2 python-pyvorbis python-qt4 python-rdflib python-renderpm python-reportbug python-reportlab python-reportlab-accel python-roman
  python-rpm python-rsvg python-selinux python-semanage python-sepolgen python-serial python-sip python-software-properties python-sphinx python-sqlite
  python-sqlitecachec python-support python-tagpy python-twisted-bin python-twisted-conch python-twisted-core python-twisted-web python-uno
  python-utidylib python-vte python-webkit python-wnck python-xapian python-xdg python-zope.interface python2.6 python2.6-dev python2.6-minimal
  qemu-keymaps qemu-kvm qemu-system qemu-user-static qemu-utils qt4-qtconfig quagga quagga-doc quilt radeontool rdesktop readline-common realpath recode
  remmina reportbug resolvconf rhythmbox rhythmbox-plugins rinse ripit rpm rpm-common rpm2cpio rsync rsyslog samba samba-common samba-common-bin samba-doc
  sane-utils scons screen seabios seahorse sed selinux-policy-default sensible-utils sensors-applet setools sflphone-daemon sflphone-data sflphone-gnome
  sgml-base sgml-data shared-mime-info sharutils shorewall-core shorewall6 shotwell siege signing-party simple-scan slapd smartmontools smbclient smistrip
  snd snd-gtk-pulse snmp software-center software-properties-gtk sound-juicer soundmodem sox sp spidermonkey-bin squashfs-tools ssh-krb5 sshfs ssl-cert
  strace strongswan strongswan-ikev1 strongswan-ikev2 strongswan-starter subversion sudo svn-buildpackage swat synaptic synergy syslinux syslinux-common
  system-config-printer system-config-printer-udev system-tools-backends system-tools-backends-dev sysv-rc sysvinit sysvinit-utils tar tasksel
  tasksel-data tcl tcl8.4 tcl8.5 tcpd tcpdump telepathy-gabble telepathy-mission-control-5 telepathy-salut telepathy-sofiasip tex-common texinfo
  texlive-base texlive-binaries texlive-common texlive-doc-base texlive-extra-utils texlive-font-utils texlive-fonts-recommended
  texlive-fonts-recommended-doc texlive-generic-recommended texlive-latex-base texlive-latex-base-doc texlive-latex-recommended
  texlive-latex-recommended-doc texlive-luatex texlive-metapost texlive-metapost-doc texlive-pstricks texlive-pstricks-doc texlive-xetex tidy time tinymce
  tipa tk tk8.4 tk8.5 tofrodos tomboy toshset totem totem-common totem-mozilla totem-plugins traceroute transfig transmission-cli transmission-common
  transmission-gtk trustedqsl tsconf ttf-ancient-fonts ttf-dejavu ttf-dejavu-core ttf-dejavu-extra ttf-freefont ttf-lg-aboriginal ttf-liberation ttf-lyx
  ttf-opensymbol ttf-sil-gentium ttf-sil-gentium-basic ttf-takao ttf-takao-gothic ttf-takao-mincho ttf-thai-arundina ttf-thai-tlwg ttf-umefont ttf-umeplus
  ttf-unifont twm twolame tzdata ucf udev udisks ufraw-batch unattended-upgrades unetbootin unetbootin-translations unifont unixodbc uno-libs3 unp unrar
  unzip update-inetd update-manager-core update-manager-gnome update-notifier update-notifier-common upower ure usbmuxd usbutils util-linux vde2 vflib3
  vgabios vim-common vim-tiny vino virt-manager virt-viewer virtinst vlc-data vlc-nox vlc-plugin-notify vlc-plugin-pulse vpnc vzctl w3m wamerican wdiff
  wget whiptail whois winbind wireless-tools wireshark wireshark-common wordnet wordnet-base wordnet-gui wpasupplicant wvdial wwwconfig-common x11-apps
  x11-common x11-session-utils x11-utils x11-xfs-utils x11-xkb-utils x11-xserver-utils x11proto-composite-dev x11proto-core-dev x11proto-damage-dev
  x11proto-dri2-dev x11proto-fixes-dev x11proto-fonts-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev x11proto-print-dev x11proto-randr-dev
  x11proto-render-dev x11proto-resource-dev x11proto-video-dev x11proto-xext-dev x11proto-xf86dri-dev x11proto-xf86vidmode-dev x11proto-xinerama-dev xauth
  xbase-clients xbitmaps xca xclip xdemorse xdg-user-dirs xdg-user-dirs-gtk xdg-utils xen-tools xen-utils-common xenstore-utils xfonts-100dpi
  xfonts-100dpi-transcoded xfonts-75dpi xfonts-75dpi-transcoded xfonts-a12k12 xfonts-ayu xfonts-baekmuk xfonts-base xfonts-bitmap-mule
  xfonts-biznet-100dpi xfonts-biznet-75dpi xfonts-biznet-base xfonts-cyrillic xfonts-efont-unicode xfonts-efont-unicode-ib xfonts-encodings
  xfonts-jisx0213 xfonts-kaname xfonts-kapl xfonts-mathml xfonts-mona xfonts-naga10 xfonts-scalable xfonts-terminus xfonts-terminus-dos
  xfonts-terminus-oblique xfonts-thai xfonts-thai-etl xfonts-thai-manop xfonts-thai-nectec xfonts-thai-poonlap xfonts-thai-vor xfonts-tipa xfonts-unifont
  xfonts-utils xfonts-wqy xindy xindy-rules xinit xkb-data xml-core xorg xorg-docs-core xoscope xsane xsane-common xserver-common xserver-xephyr
  xserver-xorg xserver-xorg-core xserver-xorg-dev xserver-xorg-input-all xserver-xorg-input-evdev xserver-xorg-input-synaptics xserver-xorg-input-wacom
  xserver-xorg-video-apm xserver-xorg-video-ark xserver-xorg-video-ati xserver-xorg-video-chips xserver-xorg-video-cirrus xserver-xorg-video-fbdev
  xserver-xorg-video-i128 xserver-xorg-video-intel xserver-xorg-video-mach64 xserver-xorg-video-mga xserver-xorg-video-neomagic
  xserver-xorg-video-openchrome xserver-xorg-video-r128 xserver-xorg-video-radeon xserver-xorg-video-rendition xserver-xorg-video-s3
  xserver-xorg-video-s3virge xserver-xorg-video-savage xserver-xorg-video-siliconmotion xserver-xorg-video-sis xserver-xorg-video-sisusb
  xserver-xorg-video-tdfx xserver-xorg-video-trident xserver-xorg-video-tseng xserver-xorg-video-vesa xserver-xorg-video-vmware xserver-xorg-video-voodoo
  xsltproc xterm xtightvncviewer xtrans-dev xutils-dev xz-utils yelp yum zenity zip zlib1g zlib1g-dev
2160 upgraded, 944 newly installed, 133 to remove and 9 not upgraded.
Need to get 90.5 MB/2,928 MB of archives.
After this operation, 1,287 MB of additional disk space will be used.
Do you want to continue [Y/n]? 

28 March 2014

Matthias Klumpp: Appstream: The next step

appstream-logoWith the release of GNOME-Software 3.12, it s also time for another update about what is going on behind the scenes of Appstream, the project providing all the data needed to make software-centers work. So here is the long overdue update :) If you have attended my FOSDEM talk or seen the slides, you know about the concept of component metadata to describe (almost) all software components which make up a Linux system, as well as their public interfaces they provide for users and other software to access. This metadata specification was originally designed as part of the Listaller project for use with the 3rd-party software installer.
But we already have the Appstream specification, which describes applications available on a distribution. And technically, applications are nothing but specialized components.
When I was asked about extending the Appstream spec with fonts and codecs, I decided to give up the Listaller components concept and merge it with the Appstream project, effectively creating a single source of metadata for all software in distributions. Appstream and components were separate, because Appstream was designed just for application metadata shown in software-centers, while component-metadata was more technical and not user-visible. This separation, however, did not reflect reality. Software centers want to show things components were designed for (codecs, fonts, ), and application meta-information also can contain technical items not shown in a software-center, but useful for other functionality (e.g. for mimehandler finding). So, if you like clear naming of things (like I do), you can now think of the upcoming Appstream release 0.6 as Appstream not being a stream of application metadata, but rather metadata provided for use in applications. ;-) Extending Appstream with components has a number of benefits:
First of all, users will soon get systems which are able to automatically install missing firmware, codecs, fonts and even libraries or Python-modules. This happens in a distro-agnostic way, so any application can request installations of missing components through Appstream and PackageKit without having to care about distribution details. For distributors, you will get more information about the software upstream provides: Which public interfaces does it make available? What is the upstream s homepage? Summary? Short release-descriptions? Exposed library symbols? etc. This data was available before, of course, but we made it machine-readable and standardized now. This will also allow us to match software across distributions quickly, allowing efficient exchange of patches between distributions, and answering questions upstream has like in which distribuions is my software available, and in which version? . Tracking upstream software is also easier now. Last but not least, the metadata format gives upstreams a small say in how their software gets packaged (= how it gets split / if there is a metapackage depending on the split parts). For upstream authors, it is much easier to maintain just a few Appstream XML metafiles, instead of adding a Listaller component file as well (which was not XML). Most data for creating the metainfo files is already available, some can even be filled in automatically by the build-system. With all the advantages this new system has comes a small disadvantage: These changes broke the backwards-compatibility of the Appstream spec, so to use Appstream data in version 0.6 or higher, you need to adjust your parsers. But of course you are using a library to access the data anyway, which already does that for you ;-) GNOME-Software 3.12 has almost complete support for the new (but still not released) Appstream 0.6 spec through it s libappstream-glib library. The libappstream library in Appstream s Git branch will get 0.6 support as well very soon. The only thing you need to do is adjust your code for the new library version, which broke it s API as well due to a rewrite in C (I hope that we can provide language bindings again very soon).
Soon, there will also be a Qt library for accessing Appstream, since some people expressed a demand for it (and working with GObject in Qt C++ code always feels a litle bit odd).
The 0.6 specification is still being fine-tuned to iron out some issues which might come up and mainly to improve the documentation and adjust all tools. It will be released in 2-3 weeks, and bring not only components support, but also lots of other small improvements and additions for better description of applications/components. In some cases, tag names have changed to have less ambiguous names. Thanks to Richard Hughes for helping with the spec our discussions about the Appstream spec are highly productive and in the end lead to a better specification. Also, the new 0.6 specification includes many extensions used in GNOME-Software by default now, which makes it easier for other distributions to provide them for GS. The current draft for the Appstream 0.6 specifications can be found here. Some details might still change before the final release. As soon as the spec is final, there will be a ML announcement for distributors as well (watch the XDG distributions list), so there is enough time for distributors to future-proof their Appstream distro-data generators (I will also describe the changes more detailed then). If you are confused now about the whole components stuff: Fear not! ;-) I will do some more detailed posts in a few weeks explaining upstream projects how to provide metadata properly (after the 0.6 release). I will also finish the wiki page for Appstream AppData in KDE projects, where we can then use 0.6-style data in KDE right away.

Next.

Previous.