During the
Contribute your skills to Debian
event that took place in Paris last week-end, we conducted a usability
testing session. Six people were tasked with testing a few aspects of
the
GNOME 3.22 desktop environment and of
the
Debian 9 (Stretch) operating system.
A number of other people observed them and took notes. Then, two
observers and three testers analyzed the results, that we are hereby
presenting: we created a heat map visualization, summed up the
challenges met during the tests, and wrote this blog post together.
We will point the relevant upstream projects to our results.
A couple of other people also did some usability testing but went in much more depth: their feedback is much more detailed and comes with a number of improvement ideas. I will process and publish their results as soon as possible.
Missions
Testers were provided a laptop running GNOME on a a Debian 9 (Stretch)
Live system. A quick introduction (mostly copied from the one we found
in some GNOME usability testing reports) was read. Then they were
asked to complete the following tasks.
A. Nautilus
Mission A.1 Download and rename file in Nautilus
- Download a file from the web, a PDF document for example.
- Open the folder in which the file has been downloaded.
- Rename the dowloaded file to
SUCCESS.pdf.
- Toggle the browser window to full screen.
- Open the file
SUCCESS.pdf.
- Go back to the File manager.
- Close the file
SUCCESS.pdf.
Mission A.2 Manipulate folders in Nautilus
- Create a new folder named
cats in your user directory.
- Create a new folder named
to do in your user directory.
- Move the
cats folder to the to do folder.
- Delete the
cats folder.
Mission A.3 Create a bookmark in Nautilus
- Create a folder named
unicorns in your personal directory.
- This folder is important. Add a bookmark for
unicorns in order to
find it again in a few weeks.
Mission A.4 Nautilus display settings
Folders and files are usually listed as icons, but they can also be
displayed differently.
- Configure the File manager to make it show items as a list, with
one file per line.
- You forgot your glasses and the font size is too small for you to
see the text: increase the size of the text.
B. Package management
Introduction
On Debian, each application is available as a "package" which contains
every file needed for the software to work.
Unlike in other operating systems, it is rarely necessary and almost
never a good idea, to download and install software from the authors
website. We can rather install it from an online library managed by
Debian (like an appstore). This alternative offers several advantages,
such as being able to update all the software installed in one
single action.
Specific tools are available to install and update Debian packages.
Mission B.1 Install and remove packages
- Install the
vlc package.
- Start VLC.
- Remove the
vlc package.
Mission B.2 Search and install a package
- Find a piece of software which can download files with BitTorrent
in a graphical interface.
- Install the corresponding package.
- Launch that BitTorrent software.
Mission B.3 Upgrade the system
Make sure the whole system (meaning all installed packages) is up to date.
C. Settings
Mission C.1 Change the desktop background
- Download an image you like from the web.
- Set the downloaded image as the desktop wallpaper.
Mission C.2 Tweak temporary files management
Configure the system so that temporary files older than three days are
deleted automatically.
Mission C.3 Change the default video player
- Install VLC (ask for help if you could not do it during the previous mission).
- Make VLC the default video player.
- Download a video file from the web.
- Open the downloaded video, then check if it opens with VLC.
Mission C.4 Add and remove world clocks
When you click the time and date in the top bar, a menu pops-up.
There, you can display clocks in several time-zones.
- Add a clock with Rio de Janeiro timezone, then another
showing the current time in Boston.
- Check that the time and date menu now displays these two
additional clocks.
- Remove the Boston clock.
Results and analysis
Heat map
We used Jim Hall's
heat map
technique to summarize our usability test results. As Renata
puts it,
it is "a great way to see how the users performed on each task.
The heat map clarifies how easy or difficult it was for the
participant to accomplish a certain task.
- Scenario tasks (from the usability test) are arranged in rows.
- Test participants (for each tester) are arranged in columns.
- The colored blocks represent each tester s difficulty with each scenario task.
Green blocks represent the ability of the participant to accomplish the tasks with little or no difficulty.
Yellow blocks indicate the tasks that the tester had significant difficulties accomplishing.
Red blocks indicate that testers experienced extreme difficulty or where testers completed the tasks incorrectly.
Black blocks indicate tasks the tester was unable to complete."
Alternatively, here is the
spreadsheet that was
used to create this picture, with added text to avoid relying on
colors only.
Most tasks were accomplished with little or no difficulty so we will
now focus on the problematic parts.
What were the challenges?
The heat map shows several "hot" rows, that we will now be looking at
in more details.
Mission A.3 Create a bookmark in Nautilus
Most testers right-clicked the folder first, and eventually found they
could simply drag'n'drop to the bookmarks location in the sidebar.
One tester thought that he could select a folder, click the hamburger
icon, and from there use the "Bookmark this folder" menu item.
However, this menu action only works on the folder one has entered,
not on the
selected one.
Mission B.1 Install and remove a package
Here we faced a number of issues caused by the fact that Debian Live
images don't include package indices (with good reason), so no package
manager can list available software.
Everyone managed to start a graphical package manager via the Overview
(or via the CLI or Alt-F2 for a couple power users).
Some testers tried to use GNOME Software, which listed only already
installed packages (
Debian bug #862560) and provided no way we could
find to refresh the package indices. That's arguably a bug in Debian
Live, but still: GNOME Software might display some useful information
when it detects this unusual situation.
We won't list here all the obstacles that were met in Synaptic:
it's no news its usability is rather sub-optimal and better
alternatives (such as GNOME Software) are in the works.
Mission C.2 Tweak temporary files management
The mission was poorly phrased: some observers had to clarify that it
was specifically about GNOME, and not generic Linux system
administration: some power-users were already searching the web for
command-line tools to address the task at hand.
Even with this clarification, no tester would have succeeded without
being told they were allowed to use the web with a search query
including the word "GNOME", or use the GNOME help or the Overview.
Yet eventually all testers succeeded.
It's interesting to note that regular GNOME users had the same problem
as others: they did not try searching "temporary" in the Overview and did
not look-up the GNOME Help until they were suggested to do so.
Mission C.3 Change the default video player
One tester configured one single video file format to be opened by
default with
VLC, via right-click in Nautilus Open with etc.
He believed this would be enough to make
VLC the default video
player, missing the subtle difference between "default video player"
and "default player for one single video format".
One tester tried to complete this task inside
VLC itself and then
needed some help to succeed. It might be that the way web browsers ask
"Do you want
ThisBrowser to become the default web browser?" gave a hint
an application GUI is the right place to do it.
Two testers searched "default" in the Overview (perhaps the previous
mission dealing with temporary files was enough to put them in this
direction). At least one tester was confused since the only search
result (
Details View information about your system), which is the
correct one to get there, includes the word View, which suggests
that one cannot modify settings there, but only view them.
One long-term GNOME user looked in
Tweak Tool first, and then used
the Overview.
Here again, GNOME users experienced essentially the same issues
as others.
Mission C.4 Add and remove world clocks
One tester tried to look for the clock on the top right corner of the
screen, then realized it was in the middle. Other than this, all
testers easily found a way to add world clocks.
However, removing a world clock was rather difficult; although most
testers managed to do it, it took them a number of attempts to
succeed:
- Several testers left-clicked or right-clicked the clock they wanted
to remove, expecting this would provide them with a way to remove
it (which is not the case).
- After a while, all testers noticed the Select button (that has no
text label nor tooltip info), which allowed them to select the
clock they wanted to remove; then, most testers clicked the 1
selected button, hoping it would provide a contextual menu or some
other way to act on the selected clocks (it doesn't).
- Eventually, everyone managed to locate the Delete button on the
bottom right corner of the window; some testers mentioned that it is less
visible and flashy than the blue bar that appears on the top of the
screen once they had entered "Selection" mode.
General notes and observations
- None of the participants sollicited the GNOME Help, which is unfortunate knowing its:
- great quality;
- translations in several languages;
- availability and adaptability to regional specifications;
- adequacy to the currently running version of GNOME.
Some users found the relevant help page online via web searches;
others initially ignored it among search results, then looked for it
later after being told that the mission was more about GNOME.
- Whether testers were already GNOME users or not seldom impacted
their chances of success.
- Unfortunately, we haven't compiled enough information about the
testers to provide useful data about who they are and what their
background is. Still, we had an interesting mix in terms of genders,
age (between 17 and 52 years old), skin color and
computer experience.
Recently there was another bind9 security update released by the Debian Security Team. I thought that was odd, so I've scanned my mailbox:
My monthly report covers a large part of what I have been doing in the free software world. I write it for 
To be able to get in sync with Kali, I introduced an hashcat-meta source package (in contrib) providing hashcat-nvidia to make it easy to install hashcat for owners of NVidia hardware.
Misc stuff
Distro Tracker. I merged a small CSS fix from Aur lien Couderc (
Here is my monthly update covering what I have been doing in the free software world (
I've just released version 1.22 of Obnam, my backup application. It is
the first release for this year. Packages are available on
code.liw.fi/debian and in Debian unstable, and source is in git. A
summary of the user-visible changes is below.
For those interested in living dangerously and accidentally on purpose
deleting all their data, the link below shows that status and roadmap
for FORMAT GREEN ALBATROSS.
Yesterday I uploaded the first packages of 
I mentioned already in a 
Bem vindo ao Debian Stretch!
Ontem, 17 de junho de 2017, o Debian 9 (Stretch) foi lan ado. Eu gostaria de falar sobre alguns procedimentos b sicos e regras para migrar do Debian 8 (Jessie).
Passos iniciais
Nos dois casos, recomend vel que voc escolha por instalar a nova vers o do arquivo de configura o. Isso porque o novo arquivo de configura o estar totalmente adaptado aos novos servi os instalados e poder ter muitas op es novas ou diferentes. Mas n o se preocupe, pois as suas configura es n o ser o perdidas. Haver um backup das mesmas. Assim, para shell, escolha a op o "Y" e, no caso de janela, escolha a op o "instalar a vers o do mantenedor do pacote". muito importante anotar o nome de cada arquivo modificado. No caso da janela anterior, trata-se do arquivo /etc/samba/smb.conf. No caso do shell o arquivo foi o /etc/rsyslog.conf.
Depois de completar a migra o, voc poder ver o novo arquivo de configura o e o original. Caso o novo arquivo tenha sido instalado ap s uma escolha via shell, o arquivo original (o que voc tinha anteriormente) ter o mesmo nome com a extens o .dpkg-old. No caso de escolha via janela, o arquivo ser mantido com a extens o .ucf-old. Nos dois casos, voc poder ver as modifica es feitas e reconfigurar o seu novo arquivo de acordo com as necessidades.
Caso voc precise de ajuda para ver as diferen as entre os arquivos, voc poder usar o comando diff para compar -los. Fa a o diff sempre do arquivo novo para o original. como se voc quisesse ver como fazer com o novo arquivo para ficar igual ao original. Exemplo:
While upgrading 
The elevent update in the 0.12.* series of 
I was surprised that they didn't lie - you will really get huge amount of issues just after making your project public. Most of them were quite simple and repeating (as you can see from number of duplicates), but it really provided valuable input.
Even more surprisingly there was second peak coming in when I've started to disclose resolved issues (once 
I'm really glad I could make 
Debian has many years of history, about 25 years already.
With such a long travel over the continous field of developing our Universal
Operating System, some myths, false accusations and bad reputation has arisen.
Today I had the opportunity to discuss this topic, I was invited to give a
Debian talk in the 
myth #1: Debian is old software
Please, use testing or stable-backports. If you use Debian stable your system
will in fact be stable and that means: updates contain no new software but
only fixes.
myth #2: Debian is slow
We compile and build most of our packages with industry-standard compilers and
options. I don t see a significant difference on how fast linux kernel or
mysql run in a CentOS or in Debian.
myth #3: Debian is difficult
I already discussed about this issue back in