Author: ceridwen
I now have working test environments set up for null (no container,
build on the host system),
schroot,
and
qemu. After fixing some bugs,
null and qemu now pass all their tests!
schroot still has a permission error related to disorderfs. Since the
same code works for null and qemu and for schroot when disorderfs is
disabled, it's something specific to disorderfs and/or its combination
with schroot. The following is debug output that shows
ls for the
build directory on the testbed before and after the mock build, and
stat for both the build directory and the mock build artifact
itself. The first control run, without disorderfs, succeeds:
test.py: DBG: testbed command ['ls', '-l', '/tmp/autopkgtest.5oMipL/control/'], kind short, sout raw, serr raw, env []
total 20
drwxr-xr-x 2 user user 4096 Jul 15 23:43 __pycache__
-rwxr--r-- 1 user user 2340 Jun 28 18:43 mock_build.py
-rwxr--r-- 1 user user 175 Jun 3 15:42 mock_failure.py
-rw-r--r-- 1 user user 252 Jun 14 16:06 template.ini
-rwxr-xr-x 1 user user 1600 Jul 15 23:18 tests.py
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['sh', '-ec', 'cd /tmp/autopkgtest.5oMipL/control/ ;\n python3 mock_build.py ;\n'], kind short, sout raw, serr pipe, env ['LANG=en_US.UTF-8', 'HOME=/nonexistent/first-build', 'VIRTUAL_ENV=~/code/reprotest/.tox/py35', 'PATH=~/code/reprotest/.tox/py35/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'PYTHONHASHSEED=559200286', 'TZ=GMT+12']
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['ls', '-l', '/tmp/autopkgtest.5oMipL/control/'], kind short, sout raw, serr raw, env []
total 20
drwxr-xr-x 2 user user 4096 Jul 15 23:43 __pycache__
-rw-r--r-- 1 root root 0 Jul 18 15:06 artifact
-rwxr--r-- 1 user user 2340 Jun 28 18:43 mock_build.py
-rwxr--r-- 1 user user 175 Jun 3 15:42 mock_failure.py
-rw-r--r-- 1 user user 252 Jun 14 16:06 template.ini
-rwxr-xr-x 1 user user 1600 Jul 15 23:18 tests.py
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['stat', '/tmp/autopkgtest.5oMipL/control/'], kind short, sout raw, serr raw, env []
File: '/tmp/autopkgtest.5oMipL/control/'
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 56h/86d Inode: 1351634 Links: 3
Access: (0755/drwxr-xr-x) Uid: ( 1000/ user) Gid: ( 1000/ user)
Access: 2016-07-18 15:06:31.105915342 -0400
Modify: 2016-07-18 15:06:31.089915352 -0400
Change: 2016-07-18 15:06:31.089915352 -0400
Birth: -
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['stat', '/tmp/autopkgtest.5oMipL/control/artifact'], kind short, sout raw, serr raw, env []
File: '/tmp/autopkgtest.5oMipL/control/artifact'
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fc01h/64513d Inode: 40767795 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2016-07-18 15:06:31.089915352 -0400
Modify: 2016-07-18 15:06:31.089915352 -0400
Change: 2016-07-18 15:06:31.089915352 -0400
Birth: -
test.py: DBG: testbed command exited with code 0
test.py: DBG: sending command to testbed: copyup /tmp/autopkgtest.5oMipL/control/artifact /tmp/tmpw_mwks82/control_artifact
schroot: DBG: executing copyup /tmp/autopkgtest.5oMipL/control/artifact /tmp/tmpw_mwks82/control_artifact
schroot: DBG: copyup_shareddir: tb /tmp/autopkgtest.5oMipL/control/artifact host /tmp/tmpw_mwks82/control_artifact is_dir False downtmp_host /var/lib/schroot/mount/jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52//tmp/autopkgtest.5oMipL
schroot: DBG: copyup_shareddir: tb(host) /var/lib/schroot/mount/jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52/tmp/autopkgtest.5oMipL/control/artifact is not already at destination /tmp/tmpw_mwks82/control_artifact, copying
test.py: DBG: got reply from testbed: ok
That last bit indicates that copy command for the build artifact from
the testbed to a temporary directory on the host succeeded. This is the
debug output from the second run, with disorderfs enabled:
test.py: DBG: testbed command ['ls', '-l', '/tmp/autopkgtest.5oMipL/disorderfs/'], kind short, sout raw, serr raw, env []
total 20
drwxr-xr-x 2 user user 4096 Jul 15 23:43 __pycache__
-rwxr--r-- 1 user user 2340 Jun 28 18:43 mock_build.py
-rwxr--r-- 1 user user 175 Jun 3 15:42 mock_failure.py
-rw-r--r-- 1 user user 252 Jun 14 16:06 template.ini
-rwxr-xr-x 1 user user 1600 Jul 15 23:18 tests.py
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['sh', '-ec', 'cd /tmp/autopkgtest.5oMipL/disorderfs/ ;\n umask 0002 ;\n linux64 --uname-2.6 python3 mock_build.py ;\n'], kind short, sout raw, serr pipe, env ['LC_ALL=fr_CH.UTF-8', 'CAPTURE_ENVIRONMENT=i_capture_the_environment', 'HOME=/nonexistent/second-build', 'VIRTUAL_ENV=~/code/reprotest/.tox/py35', 'PATH=~/code/reprotest/.tox/py35/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin/i_capture_the_path', 'LANG=fr_CH.UTF-8', 'PYTHONHASHSEED=559200286', 'TZ=GMT-14']
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['ls', '-l', '/tmp/autopkgtest.5oMipL/disorderfs/'], kind short, sout raw, serr raw, env []
total 20
drwxr-xr-x 2 user user 4096 Jul 15 23:43 __pycache__
-rw-r--r-- 1 root root 0 Jul 18 15:06 artifact
-rwxr--r-- 1 user user 2340 Jun 28 18:43 mock_build.py
-rwxr--r-- 1 user user 175 Jun 3 15:42 mock_failure.py
-rw-r--r-- 1 user user 252 Jun 14 16:06 template.ini
-rwxr-xr-x 1 user user 1600 Jul 15 23:18 tests.py
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['stat', '/tmp/autopkgtest.5oMipL/disorderfs/'], kind short, sout raw, serr raw, env []
File: '/tmp/autopkgtest.5oMipL/disorderfs/'
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 58h/88d Inode: 1 Links: 3
Access: (0755/drwxr-xr-x) Uid: ( 1000/ user) Gid: ( 1000/ user)
Access: 2016-07-18 15:06:31.201915291 -0400
Modify: 2016-07-18 15:06:31.185915299 -0400
Change: 2016-07-18 15:06:31.185915299 -0400
Birth: -
test.py: DBG: testbed command exited with code 0
test.py: DBG: testbed command ['stat', '/tmp/autopkgtest.5oMipL/disorderfs/artifact'], kind short, sout raw, serr raw, env []
File: '/tmp/autopkgtest.5oMipL/disorderfs/artifact'
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 58h/88d Inode: 7 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2016-07-18 15:06:31.185915299 -0400
Modify: 2016-07-18 15:06:31.185915299 -0400
Change: 2016-07-18 15:06:31.185915299 -0400
Birth: -
test.py: DBG: testbed command exited with code 0
test.py: DBG: sending command to testbed: copyup /tmp/autopkgtest.5oMipL/disorderfs/artifact /tmp/tmpw_mwks82/experiment_artifact
schroot: DBG: executing copyup /tmp/autopkgtest.5oMipL/disorderfs/artifact /tmp/tmpw_mwks82/experiment_artifact
schroot: DBG: copyup_shareddir: tb /tmp/autopkgtest.5oMipL/disorderfs/artifact host /tmp/tmpw_mwks82/experiment_artifact is_dir False downtmp_host /var/lib/schroot/mount/jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52//tmp/autopkgtest.5oMipL
schroot: DBG: copyup_shareddir: tb(host) /var/lib/schroot/mount/jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52/tmp/autopkgtest.5oMipL/disorderfs/artifact is not already at destination /tmp/tmpw_mwks82/experiment_artifact, copying
schroot: DBG: cleanup...
schroot: DBG: execute-timeout: schroot --run-session --quiet --directory=/ --chroot jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52 --user=root -- rm -rf -- /tmp/autopkgtest.5oMipL
rm: cannot remove '/tmp/autopkgtest.5oMipL/disorderfs': Device or resource busy
schroot: DBG: execute-timeout: schroot --quiet --end-session --chroot jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52
Unexpected error:
Traceback (most recent call last):
File "~/code/reprotest/reprotest/lib/VirtSubproc.py", line 708, in mainloop
command()
File "~/code/reprotest/reprotest/lib/VirtSubproc.py", line 646, in command
r = f(c, ce)
File "~/code/reprotest/reprotest/lib/VirtSubproc.py", line 584, in cmd_copyup
copyupdown(c, ce, True)
File "~/code/reprotest/reprotest/lib/VirtSubproc.py", line 469, in copyupdown
copyupdown_internal(ce[0], c[1:], upp)
File "~/code/reprotest/reprotest/lib/VirtSubproc.py", line 494, in copyupdown_internal
copyup_shareddir(sd[0], sd[1], dirsp, downtmp_host)
File "~/code/reprotest/reprotest/lib/VirtSubproc.py", line 408, in copyup_shareddir
shutil.copy(tb, host)
File "/usr/lib/python3.5/shutil.py", line 235, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.5/shutil.py", line 114, in copyfile
with open(src, 'rb') as fsrc:
PermissionError: [Errno 13] Permission denied: '/var/lib/schroot/mount/jessie-amd64-ac94881d-ae71-4f24-a004-1847889d5d52/tmp/autopkgtest.5oMipL/disorderfs/artifact'
ls shows that the artifact is created in the right place. However,
when reprotest tries to copy it from the testbed to the host, it gets
a permission error. The traceback is coming from
virt/schroot, and
it's a Python
open() call that's failing. Note that the permissions
are wrong for the second run, but that's expected because my schroot
is stable so the umask bug isn't fixed yet; and that the
rm error
from disorderfs not being unmounted early enough (see below). I
expect to see the umask test fail, though, not a crash in every test
where the build succeeds.
After a great deal of effort, I isolated the bug that was causing the
process to hang not to my code or autopkgtest's code, but to
CPython and
contextlib. It's supposed to be
fixed in CPython 3.5.3, but for now I've worked around the problem by
monkey-patching the patch provided in the latter issue onto
contextlib.
Here is my current to-do list:
- Fix PyPi not installing the virt/ scripts correctly.
- Move the disorderfs unmount into the shell script. (When the virt/
scripts encounter an error, they try to delete a temporary
directory, which fails if disorderfs is mounted, so the script needs
to unmount it before that happens.)
- Find and fix the schroot/disorderfs permission error bug.
- Convert my notes on setting up for the tests into something useful
for users.
- Write scripts to synch version numbers and documentation.
- Fix the headers in the autopkgtest code to conform to reprotest
style.
- Add copyright information for the contextlib monkey-patch and the
autopkgtest files I've changed.
- Close
#829113
as wontfix.
And here are the questions I'd like to resolve before the second release:
- Is there any other documentation that's essential? Finishing the
documentation will come later.
- Should I release before finishing the rest of the variation? This
will slow down the release of the first version with something
resembling full fuctionality.
- Do I need to write a chroot test now? Given the duplication with
schroot, I'm unconvinced this is worthwhile.
DebConf 16 in Capetown/South Africa was fantastic for many reasons.
My Capetown/South Africa/Culture/Flight related lessons:
I write free software and I have some users. My primary support
channels are over email and IRC, which means I do not have direct
access to the system where my software runs. When one of my users has
a problem, we go through one or more cycles of them reporting what
they see and me asking them for more information, or asking them to
try this thing or that thing and report results. This can be quite
frustrating.
I want, nay, need to improve this. I've been thinking about this for a
while, and talking with friends about it, and here's my current ideas.
First idea: have a script that gathers as much information as
possible, which the user can run. For example, log files, full
configuration, full environment, etc. The user would then mail the
output to me. The information will need to be anonymised suitably so
that no actual secrets are leaked. This would be similar to Debian's
package specific reportbug scripts.
Second idea: make it less likely that the user needs help solving
their issue, with better error messages. This would require error
messages to have sufficient explanation that a user can solve their
problem. That doesn't necessarily mean a lot of text, but also code
that analyses the situation when the error happens to include things
that are relevant for the problem resolving process, and giving error
messages that are as specific as possible. Example: don't just fail
saying "write error", but make the code find out why writing caused an
error.
Third idea: in addition to better error messages, might provide
diagnostics tools as well.
A friend suggested having a script that sets up a known good set of
operations and verifies they work. This would establish a
known-working baseline, or smoke test, so that we can rule things like
"software isn't completely installed".
Do you have ideas? Mail me (liw@liw.fi) or tell me on identi.ca (@liw)
or Twitter (@larswirzenius).
As was posted 
Around a year ago, I started hacking together a machine readable version
of the OSI approved licenses list, and casually picking parts up until it
was ready to launch. A few weeks ago, we officially announced
the 
Like 
My experiences with Amazon reviewing have been somewhat unusual. A review of a 
Over the past year I ve been refining my understanding of the core
claims of Kantian ethics. I ve realised that I have deeply Kantian
intuitions about a lot of issues, and I understand these intuitions
better now that I can put them in Kantian terms. Consider two
exercises of state power: riot police suppressing protesters by
non-lethal means, and soldiers shooting protesters to death. I feel
more uncomfortable thinking about the first: there s something
altogether more sinister about it than the second, even though the
second is much more sad.
I think that the reason is that non-lethal weaponry is designed to
take away people s agency, and it often achieves this aim by means of
emotional manipulation. Riot police use so-called baton charges to
incite fearful retreat. Protesters have reasoned that in their
political situation, they have a duty to resist the incumbent
government. Riot police seek to unseat this conviction and cause fear
to determine what the protesters will do. In Kantian terms, the
riot police fail to respect the moral agency of the protesters by
seeking to unseat the moral personality s determination of what the
protester will do.
A controversial example that Kant uses to make this point is the axe
murderer case. Kant asks us to imagine that someone bangs on our
front door and begs us to hide him in our house, because someone who
wishes to kill him is coming up behind them. We do so. When the axe
murderer arrives, he goes door-to-door and asks us whether the
intended victim is in each house. Kant says that it is morally wrong
to lie to the axe murderer and say that the victim is not in your
house. How could this be? Surely there is a moral duty to protect
the victim from being killed? Indeed there is, but when it comes into
conflict with the duty not to lie, that second duty wins out. That s
because respecting the moral agency of individuals is paramount. In
this case, we would fail to respect the murderer s agency if we didn t
allow him to take the decision to murder or not murder the victim; by
lying to him we (disrespectfully) bypass his choice as to whether to
do it.
It s obviously crazy to say that we are morally required to give up
the victim. Kant gives the wrong answer in this case. However, the
case definitely reveals a requirement to respect other people s view
of what they should do, and give them a chance to do it. Similarly it
seems like we shouldn t give semi-automatics to our riot police, but
there s something wrong with a lot of what they do.
During the recent campaigns about Britain s EU referendum, some
criticised Jeremy Corbyn s campaigning on the grounds that he failed
to make an emotional appeal, instead asking people to make their own
rational decision when they come to cast their vote. So he lost out
the emotional appeals other people were making. It seems that he was
successfully respecting individual agency. You ve got to give people
a chance to live up to their own idea of what they should do.
I m not sure how to reconcile these various ideas, and I m not sure
what it says about me that I find non-lethal weaponry as uncomfortable
as I do.
I was assigned another 15 hours of work by Freexian's Debian LTS
initiative and carried over 5 from last month. I worked a
total of 19 hours, carrying over 1.
I spent a week in the Front Desk role and triaged many new security
issues for wheezy.
I prepared the Linux 3.2.81 stable update, sent it out for review
and finally released it. I then rebased the wheezy-security branch
on top of that and added some later security fixes that were not yet
suitable for a kernel.org update. I uploaded to wheezy-security and
issued
My monthly report covers what I have been doing for Debian. I write it for Debian s Long Term Support sponsors but also for the wider free software community in the hope that it might inspire people to get more involved with Debian or free software in general.
Debian Android