Search Results: "ed"

30 November 2016

Chris Lamb: Free software activities in November 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

This month:

My work in the Reproducible Builds project was also covered in our weekly reports. (#80, #81, #82 #83.

Toolchain issues I submitted the following patches to fix reproducibility-related toolchain issues with Debian:

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.


jenkins.debian.net

jenkins.debian.net runs our comprehensive testing framework.

  • buildinfo.debian.net has moved to SSL. (ac3b9e7)
  • Submit signing keys to keyservers after generation. (bdee6ff)
  • Various cosmetic changes, including
    • Prefer if X not in Y over if not X in Y. (bc23884)
    • No need for a dictionary; let's just use a set. (bf3fb6c)
    • Avoid DRY violation by using a for loop. (4125ec5)

I also submitted 9 patches to fix specific reproducibility issues in apktool, cairo-5c, lava-dispatcher, lava-server, node-rimraf, perlbrew, qsynth, tunnelx & zp.
Debian

Debian LTS This month I have been paid to work 11 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 697-1 for bsdiff fixing an arbitrary write vulnerability.
  • Issued DLA 705-1 for python-imaging correcting a number of memory overflow issues.
  • Issued DLA 713-1 for sniffit where a buffer overflow allowed a specially-crafted configuration file to provide a root shell.
  • Issued DLA 723-1 for libsoap-lite-perl preventing a Billion Laughs XML expansion attack.
  • Issued DLA 724-1 for mcabber fixing a roster push attack.
Uploads
  • redis:
    • 3.2.5-2 Tighten permissions of /var/ lib,log /redis. (#842987)
    • 3.2.5-3 & 3.2.5-4 Improve autopkgtest tests and install upstream's MANIFESTO and README.md documentation.
  • gunicorn (19.6.0-9) Adding autopkgtest tests.
  • libfiu:
    • 0.94-1 Add autopkgtest tests.
    • 0.95-1, 0.95-2 & 0.95-3 New upstream release and improve autopkgtest coverage.
  • python-django (1.10.3-1) New upstream release.
  • aptfs (0.8-3, 0.8-4 & 0.8-5) Adding and subsequently improving the autopkgtext tests.


I performed the following QA uploads:


Finally, I also made the following non-maintainer uploads:
  • libident (0.22-3.1) Move from obsolete Source-Version substvar to binary:Version. (#833195)
  • libpcl1 (1.6-1.1) Move from obsolete Source-Version substvar to binary:Version. (#833196)
  • pygopherd (2.0.18.4+nmu1) Move from obsolete Source-Version substvar to $ source:Version . (#833202)
RC bugs


I also filed 59 FTBFS bugs against arc-gui-clients, asyncpg, blhc, civicrm, d-feet, dpdk, fbpanel, freeciv, freeplane, gant, golang-github-googleapis-gax-go, golang-github-googleapis-proto-client-go, haskell-cabal-install, haskell-fail, haskell-monadcatchio-transformers, hg-git, htsjdk, hyperscan, jasperreports, json-simple, keystone, koji, libapache-mod-musicindex, libcoap, libdr-tarantool-perl, libmath-bigint-gmp-perl, libpng1.6, link-grammar, lua-sql, mediatomb, mitmproxy, ncrack, net-tools, node-dateformat, node-fuzzaldrin-plus, node-nopt, open-infrastructure-system-images, open-infrastructure-system-images, photofloat, ppp, ptlib, python-mpop, python-mysqldb, python-passlib, python-protobix, python-ttystatus, redland, ros-message-generation, ruby-ethon, ruby-nokogiri, salt-formula-ceilometer, spykeviewer, sssd, suil, torus-trooper, trash-cli, twisted-web2, uftp & wide-dhcpv6.
FTP Team

As a Debian FTP assistant I ACCEPTed 70 packages: bbqsql, coz-profiler, cross-toolchain-base, cross-toolchain-base-ports, dgit-test-dummy, django-anymail, django-hstore, django-html-sanitizer, django-impersonate, django-wkhtmltopdf, gcc-6-cross, gcc-defaults, gnome-shell-extension-dashtodock, golang-defaults, golang-github-btcsuite-fastsha256, golang-github-dnephin-cobra, golang-github-docker-go-events, golang-github-gogits-cron, golang-github-opencontainers-image-spec, haskell-debian, kpmcore, libdancer-logger-syslog-perl, libmoox-buildargs-perl, libmoox-role-cloneset-perl, libreoffice, linux-firmware-raspi3, linux-latest, node-babel-runtime, node-big.js, node-buffer-shims, node-charm, node-cliui, node-core-js, node-cpr, node-difflet, node-doctrine, node-duplexer2, node-emojis-list, node-eslint-plugin-flowtype, node-everything.js, node-execa, node-grunt-contrib-coffee, node-grunt-contrib-concat, node-jquery-textcomplete, node-js-tokens, node-json5, node-jsonfile, node-marked-man, node-os-locale, node-sparkles, node-tap-parser, node-time-stamp, node-wrap-ansi, ooniprobe, policycoreutils, pybind11, pygresql, pysynphot, python-axolotl, python-drizzle, python-geoip2, python-mockupdb, python-pyforge, python-sentinels, python-waiting, pythonmagick, r-cran-isocodes, ruby-unicode-display-width, suricata & voctomix-outcasts. I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against node-cliui, node-core-js, node-cpr & node-grunt-contrib-concat.

Jonas Meurer: debian lts report 2016.11

Debian LTS report for November 2016 Noevember 2016 was my third month as a Debian LTS team member. I was allocated 11 hours and had 1,75 hours left from October. This makes a total of 12,75 hours. In November I spent all 12,75 hours (and even a bit more) preparing security updates for spip, memcached and monit. In particular, the updates of spip and monit took a lot of time (each one more than six hours). The patches for both packages were horrible to backport as the affected codebase changed a lot between the Wheezy versions and current upstream versions. Still it was great fun and I learned a lot during the backporting work. Due to the intrusive nature of the patches, I also did much more extensive testing before uploading the packages, which took quite a bit of time as well. Monit 5.4-2+deb7u1 is not uploaded to wheezy-security yet as I decided to ask for further review and testing on the debian-lts mailinglist first. Below follows the list of items I worked on in November in the well known format:

Arturo Borrero Gonz lez: Creating a team for netfilter packages in debian

Debian - Netfilter There are about 15 Netfilter packages in Debian, and they are maintained by separate people. Yersterday, I contacted the maintainers of the main packages to propose the creation of a pkg-netfilter team to maintain all the packages together. The benefits of maintaining packages in a team is already known to all, and I would expect to rise the overall quality of the packages due to this movement. By now, the involved packages and maintainers are: We should probably ping Jochen Friedrich as well who maintains arptables and ebtables. Also, there are some other non-official Netfilter packages, like iptables-persistent. I m undecided to what to do with them, as my primary impulse is to only put in the team upstream packages. Given the release of Stretch is just some months ahead, the creation of this packaging team will happen after the release, so we don t have any hurry moving things now.

29 November 2016

Shirish Agarwal: The Iziko South African Museum

This would be a bit long on my stay in Cape Town, South Africa after Debconf16. Before I start, let me share the gallery works, you can see some photos that I have been able to upload to my gallery . It seems we are using gallery 2 while upstream had made gallery 3 and then it sort of died. I actually asked in softwarerecs stackexchange site if somebody knows of a drop-in replacement for gallery and was told/shared about Pwigo . I am sure the admin knows about it. There would be costs to probably migrate from gallery to Pwigo with the only benefit that it would be something which would perhaps be more maintainable. The issues I face with the current gallery system are few things a. There is no way to know how much your progress your upload has taken.
b. After it has submit, it gives a fake error message saying some error has occurred. This has happened on every occasion/attempt. Now I don t know whether it is because I have slow upload speeds or something else altogether. I had shared the error page last time in the blog post hence not sharing again. Although, all the pictures which would be shared in this blog post would be from the same gallery Another thing I would like to share is a small beginner article I wrote about why I like Debian. Another interesting/tit-bit of news I came to know few days back that both Singapore and Qatar have given 96 hours visa free stopovers for Indians for select destinations. Now to start with the story/experience due to some unknown miracle/angel looking upon me I got the chance to go to Debconf16, South Africa. I m sure there was lot of backend discussions but in the end I was given the opportunity to be part of Debcamp and Debconf. While I hope to recount my Debcamp and Debconf experience in another or two blog posts, this would be exclusively the Post-Debconf Experiences I had. As such opportunities to visit another country are rare, I wanted to make the most of it. Before starting from Pune, I had talked with Amey about Visas, about Debconf as he had just been to Debconf15 the year before and various things related to travel. He was instrumental in me having a bit more knowledge about how to approach things. I was also lucky to have both Graham and Bernelle who also suggested, advised and made it possible to have a pleasant stay both during Debcamp and Debconf. The only quibble is I didn t know heaters were being made available to us without any cost. Moving on, a day or two before Debconf was about to conclude, I asked Bernelle s help even though she was battling a burn-out I believe as I was totally clueless about Cape Town. She accepted my request and asked me to look at hostels near Longmarket Street. I had two conditions a. It should not be very far from the airport
b. It should be near to all or most cultural experiences the city has to offer. We looked at hostelworld and from the options listed, it looked like Homebasecapetown looked to be a perfect fit. It was one of the cheaper options and they also had breakfast included in the pricing. I booked through hostelworld for a mixed dorm for 2 days as I was unsure how it would be (the first night effect I have shared about previously) . When I reached there, I found it to be as good as the pictures shared were, the dorm was clean (most important), people were friendly (also important) as well as toilets and shower were also clean while the water was hot, so all in all it was a win-win situation for me. Posters I saw at homebasecapetown While I m not much of an adrenaline-junkie it was nice to know the activities that could be done/taken. Brochures and Condoms just left of main hall. This was again interesting. While apologies for the poor shaky quality of the picture, I believe it is easy to figure out. There were Brochures of the city attractions as well as condoms that people could discreetly use if need be. I had seen such condoms in few toilets during and around Debconf and it felt good that the public were aware and prioritizing safety for their guests and students instead of having fake holier than thou attitudes that many places have. For instance, you wouldn t find something like this in toilets of most colleges in India or anywhere else for that matter. There are few vending machines in what are termed as red light areas or where prostitution is known/infamous to happen and even then most times it is empty. I have 2-3 social workers as friends and they are a source of news on such things. While I went to few places and each had an attraction to it, the one which had my literally eyes out of socket was the Iziko South African Museum . I have been lucky to been quite a few museums in India, the best rated science museum in India in my limited experience has been the Visvesvaraya Industrial & Technological Museum, Bengaluru India . A beer from me if a European can get it right. Don t worry if you mispronounce it, I mispronounce it couple of times till I get it right . Looking up the word Iziko the meaning of the word seems to be the hearth and if you look at the range of collections in the museum, you would think it fits. I was lucky to find couple of friends, one of whom was living at homebase and we decided to go to the museum together. Making friends on the road So Eduardo, my friend on the left and his friend, we went to the museum. While viewing the museum, there were no adjectives to describe it other than Wow and Endless . See fossils of fish-whale-shark ? OR Giant fish-whale-dolphin-shark some million years ago. and Reminder of JAWS ;) While I have more than a few pictures, the point is easily made. It seems almost inconceivable that creatures of such masses actually were on earth. While I played with the model of the jaws of a whale/shark in reality if something like that happened, I would have been fighting for my life. The only thing I missed or could have been better if they had some interactive installations to showcase the now universally accepted Charles Darwin s On the Origin of Species I had never seen anything like this. Sadly, there was nobody around to help us figure out things as I had read that most species of fish don t leave a skeleton behind so how were these models made? It just boggles the mind. Apart from the Science Museum I was also introduced to the bloody history that South Africa had. I saw The 1913 native land act which was not honored . I had been under the impression that India had got a raw deal when it was under British rule but looking at South African history I don t know. While we got our freedom in 1947 they got rid of apartheid about 20 years+ . I talked to lot of young African males and there was lot of naked hostility for the Europeans even today. It was a bit depressing but could relate to their point of view as similar sentiments were echoed by our forefathers. I read in the newspapers and it seemed to be a pretty mixed picture. I can t comment as only South Africans can figure out the way forward. For me, it was enough to know and see that we both had similar political histories as nations. It seemed the racial divide and anger was much more highly pronounced towards Europeans and divisive then the caste divisions here between Indians. I also shared with them my limited knowledge and understanding of the Indian history (as history is re-written all the time) and it was clear to them that we had common/similar pasts. As a result, what was surprising (actually not) is that many South Africans have no knowledge of Indian history. as well otherwise the political differences that South Africa and India has in the current scenario wouldn t have been. In the end, the trip proved to be fun, stimulating, educative, thought-provoking as questions about self-identity , national identity, our place in the Universe kinda questions which should be asked all the time. Thank you Bremmer and the team for letting me experience Cape Town, South Africa, I would have been poorer if I hadn t had the experience.
Filed under: Miscellenous Tagged: #Debconf16, #Dinosaur Fishes, #gallery, #Identity, #Iziko South African Museum, #Nation-state Identity, #pwigo

Reproducible builds folks: Reproducible Builds: week 83 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday November 20 and Saturday November 26 2016: Reproducible work in other projects Bugs filed Chris Lamb: Daniel Shahaf: Reiner Herrmann: Reviews of unreproducible packages 63 package reviews have been added, 73 have been updated and 41 have been removed in this week, adding to our knowledge about identified issues. 4 issue types have been added: Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by: strip-nondeterminism development debrepatch development Continuous integration: tests.reproducible-builds.org Debian: Since the stretch freeze is getting closer, Holger made the following changes: Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

Mike Hommey: Announcing git-cinnabar 0.4.0 release candidate

Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git. Get it on github. These release notes are also available on the git-cinnabar wiki. What s new since 0.4.0b3? And since I realize I didn t announce beta 3: What s new since 0.4.0b2?

28 November 2016

Michal Čihař: phpMyAdmin security issues

You might wonder why there is so high number of phpMyAdmin security announcements this year. This situations has two main reasons and I will comment a bit on those. First of all we've got quite a lot of attention of people doing security reviews this year. It has all started with Mozilla SOS Fund funded audit. It has discovered few minor issues which were fixed in the 4.6.2 release. However this was really just the beginning of the story and the announcement has attracted quite some attention to us. In upcoming weeks the security@phpmyadmin.net mailbox was full of reports and we really struggled to handle such amount. Handling that amount actually lead to creating more formalized approach to handling them as we clearly were no longer able to deal with them based on email only. Anyway most work here was done by Emanuel Bronshtein, who is really looking at every piece of our code and giving useful tips to harden our code base and infrastructure. Second thing which got changed is that we release security announcements for security hardening even when there might not be any practical attack possible. Typical example here might be PMASA-2016-61, where using hash_equals is definitely safer, but even if the timing attack would be doable here, the practical result of figuring out admin configured allow/deny rules is usually not critical. Many of the issues also cover quite rare setups (or server misconfigurations, which we've silently fixed in past) like PMASA-2016-54 being possibly caused by server executing shell scripts shipped together with phpMyAdmin. Overall phpMyAdmin indeed got safer this year. I don't think that there was any bug that would be really critical, on the other side we've made quite a lot of hardenings and we use current best practices when dealing with sensitive data. On the other side, I'm pretty sure our code was not in worse shape than any similarly sized projects with 18 years of history, we just become more visible thanks to security audit and people looked deeper into our code base. Besides security announcements this all lead to generic hardening of our code and infrastructure, what might be not that visible, but are important as well:

Filed under: Debian English phpMyAdmin SUSE 0 comments

Clint Adams: Not the Grace Hopper Conference

Do you love porting? For ideas on how to make GHC suck less on your favorite architecture, see this not-at-all ugly table.
Posted on 2016-11-28
Tags: barks, ghc, ports, architectures

Stefano Zacchiroli: last week to take part in the Debian Contributors Survey

Debian Contributors Survey 2016 About 3 weeks ago, together with Molly and Mathieu, we launched the first edition of the Debian Contributors Survey. I won't harp on it any further, because you can find all relevant information about it on the Debian blog or as part of the original announcement. But it's worth noting that you've now only one week left to participate if you want to: the deadline for participation is 4 December 2016, at 23:59 UTC. If you're a Debian contributor and would like to participate, just go to the survey participation page and fill in!

Pau Garcia i Quiles: Desktops DevRoom @ FOSDEM 2017: you are still on time to submit a talk

FOSDEM 2016 is going to be great (again!) and you still have the chance to be one of the stars. Have you submitted your talk to the Desktops DevRoom yet? No? Remember: we will only accept proposals until December 5th. After that, the Organization Team will get busy and vote and choose the talks. Here is the full Call for Participation, in case you need to check the details on how to submit: FOSDEM Desktops DevRoom 2017 Call for Participation Topics include anything related to the Desktop: desktop environments, software development for desktop/cross-platform, applications, UI, etc

27 November 2016

Dirk Eddelbuettel: anytime 0.1.1: More robust

CRAN just accepted the newest release 0.1.1 of anytime, following the previous five releases since September. anytime is a very focussed package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, ... format to POSIXct (or Date) objects -- and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples, or just consider the following illustration: 
R> library(anytime)
R> anytime("20161107 202122")   ## all digits
[1] "2016-11-07 20:21:22 CST"
R> utctime("2016Nov07 202122")  ## UTC parse example
[1] "2016-11-07 14:21:22 CST"
R>
Release 0.1.1 robustifies two aspects. The 'digits only' input above extends what Boost Date_Time can parse and relies on simple-enough pre-processing. This operation is now more robust. We also ensure that input already of class Date is simply passed through by anydate() or utcdate(). Last but not least we added code coverage support, which oh-so-predictably lead us to game this metric to reach the elusive 100% coverage. The NEWS file summarises the release:

Changes in anytime version 0.1.1 (2016-11-27)
  • Both anydate() and utcdate() no longer attempt to convert an input value that is already of type Date.
  • The string splitter (needed for the 'all-digits' formats extending Boost Date_time) is now more defensive about the input argument and more robust. Thanks to Bob Jansen for the heads-up (PR #30 closing issue #29).
  • Code coverage reporting has been added (PR #31).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Eriberto Mota: Debian with three monitors under low cost graphics interface

Since 2008 I use two monitors in my desktop. Yesterday I bought a new graphics interface and a third monitor. Some time I was looking for a low cost graphics interface. Ok, I am using GeForce GT 740 which has three output ports: VGA, DVI and HDMI. In Brazil this interface card can be found around R$ 400 (US$ 117, but my card was US$ 87 in Brazilian Black Friday). In Amazon.com, it is between US$ 51 and US$ 109. The chosen manufacturer was Zotac, but all GT 740 and 750 will work fine (I tested the GT 750 too). The GeForce GT 740 was imediatelly recognised by Debian Jessie with kernel Linux 4.7.0 from Backports (it is my default, so I didn't test with original 3.16 kernel). The driver used was the default X.Org Nouveau. I use KDE and the management was easy. I hope this post can help people interested in use 3 monitors. Enjoy! 03-monitors

25 November 2016

Julian Andres Klode: Starting the faster, more secure APT 1.4 series

We just released the first beta of APT 1.4 to Debian unstable (beta here means that we don t know any other big stuff to add to it, but are still open to further extensions). This is the release series that will be released with Debian stretch, Ubuntu zesty, and possibly Ubuntu zesty+1 (if the Debian freeze takes a very long time, even zesty+2 is possible). It should reach the master archive in a few hours, and your mirrors shortly after that. Security changes APT 1.4 by default disables support for repositories signed with SHA1 keys. I announced back in January that it was my intention to do this during the summer for development releases, but I only remembered the Jan 1st deadline for stable releases supporting that (APT 1.2 and 1.3), so better late than never. Around January 1st, the same or a similar change will occur in the APT 1.2 and 1.3 series in Ubuntu 16.04 and 16.10 (subject to approval by Ubuntu s release team). This should mean that repository provides had about one year to fix their repositories, and more than 8 months since the release of 16.04. I believe that 8 months is a reasonable time frame to upgrade a repository signing key, and hope that providers who have not updated their repositories yet will do so as soon as possible. Performance work APT 1.4 provides a 10-20% performance increase in cache generation (and according to callgrind, we went from approx 6.8 billion to 5.3 billion instructions for my laptop s configuration, a reduction of more than 21%). The major improvements are: We switched the parsing of Deb822 files (such as Packages files) to my perfect hash function TrieHash. TrieHash which generates C code from a set of words is about equal or twice as fast as the previously used hash function (and two to three times faster than gperf), and we save an additional 50% of that time as we only have to hash once during parsing now, instead of during look up as well. APT 1.4 marks the first time TrieHash is used in any software. I hope that it will spread to dpkg and other software at a later point in time.vendors. Another important change was to drop normalization of Description-MD5 values, the fields mapping a description in a Packages files to a translated description. We used to parse the hex digits into a native binary stream, and then compared it back to hex digits for comparisons, which cost us about 5% of the run time performance. We also optimized one of our hash functions the VersionHash that hashes the important fields of a package to recognize packages with the same version, but different content to not normalize data to a temporary buffer anymore. This buffer has been the subject of some bugs (overflow, incompleteness) in the recent past, and also caused some slowdown due to the additional writes to the stack. Instead, we now pass the bytes we are interested in directly to our CRC code, one byte at a time. There were also some other micro-optimisations: For example, the hash tables in the cache used to be ordered by standard compare (alphabetical followed by shortest). It is now ordered by size first, meaning we can avoid data comparisons for strings of different lengths. We also got rid of a std::string that cannot use short string optimisation in a hot path of the code. Finally, we also converted our case-insensitive djb hashes to not use a normal tolower_ascii(), but introduced tolower_ascii_unsafe() which just sets the lowercase bit ( 0x20) in the character. Others For a more complete overview of all changes, consult the changelog.
Filed under: Debian, Ubuntu

Petter Reinholdtsen: Quicker Debian installations using eatmydata

Two years ago, I did some experiments with eatmydata and the Debian installation system, observing how using eatmydata could speed up the installation quite a bit. My testing measured speedup around 20-40 percent for Debian Edu, where we install around 1000 packages from within the installer. The eatmydata package provide a way to disable/delay file system flushing. This is a bit risky in the general case, as files that should be stored on disk will stay only in memory a bit longer than expected, causing problems if a machine crashes at an inconvenient time. But for an installation, if the machine crashes during installation the process is normally restarted, and avoiding disk operations as much as possible to speed up the process make perfect sense. I added code in the Debian Edu specific installation code to enable eatmydata, but did not have time to push it any further. But a few months ago I picked it up again and worked with the libeatmydata package maintainer Mattia Rizzolo to make it easier for everyone to get this installation speedup in Debian. Thanks to our cooperation There is now an eatmydata-udeb package in Debian testing and unstable, and simply enabling/installing it in debian-installer (d-i) is enough to get the quicker installations. It can be enabled using preseeding. The following untested kernel argument should do the trick: 
preseed/early_command="anna-install eatmydata-udeb"
This should ask d-i to install the package inside the d-i environment early in the installation sequence. Having it installed in d-i in turn will make sure the relevant scripts are called just after debootstrap filled /target/ with the freshly installed Debian system to configure apt to run dpkg with eatmydata. This is enough to speed up the installation process. There is a proposal to extend the idea a bit further by using /etc/ld.so.preload instead of apt.conf, but I have not tested its impact.

Iain R. Learmonth: vmdebootstrap Sprint Report

This is now a little overdue, but here it is. On the 10th and 11th of November, the second vmdebootstrap sprint took place. Lars Wirzenius (liw), Ana Custura (ana_c) and myself were present. liw focussed on the core of vmdebootstrap, where he sketched out what the future of vmdebootstrap may look like. He documented this in a mailing list post and also presented (video). Ana and myself worked on live-wrapper, which uses vmdebootstrap internally for the squashfs generation. I worked on improving logging, using a better method for getting paths within the image, enabling generation of Packages and Release files for the image archive and also made the images installable (live-wrapper 0.5 onwards will include an installer by default). Ana worked on the inclusion of HDT and memtest86+ in the live images and enabled both ISOLINUX (for BIOS boot) and GRUB (for EFI boot) to boot the text-mode and graphical installers. live-wrapper 0.5 was released on the 16th November with these fixes included. You can find live-wrapper documentation at https://live-wrapper.readthedocs.io/en/latest/. (The documentation still needs some work, some options may be incorrectly described). Thanks to the sponsors that made this work possible. You re awesome. (:

Michael Stapelberg: Debian package build tools

Personally, I find the packaging tools which are available in Debian far too complex. To better understand the options we have, I created a diagram of tools which are frequently used, only covering the build step (i.e. no post-build quality assurance checks or packaging-time helpers): debian package build tools When I was first introduced to Debian packaging, people recommended I use pbuilder. Given how complex the toolchain is in the pbuilder case, I don t understand why that is (was?) a common recommendation. Back in August 2015, so well over a year ago, I switched to sbuild, motivated by how much simpler it was to implement ratt (rebuilds reverse build dependencies) using sbuild, and I have not looked back. Are there people who do not use sbuild for reasons other than familiarity? If so, please let me know, I d like to understand. I also made a version of the diagram above, colored by the programming languages in which the tools are implemented. The chosen colors are heavily biased :-). debian package build tools, by language To me, the diagram above means: if you want to make substantial changes to the Debian build tool infrastructure, you need to become an expert in all of Python, Perl, Bash, C and Make. I know that this is not true for every change, but it still irks me that there might be changes for which it is required. I propose to eliminate complexity in Debian by deprecating the pbuilder toolchain in favor of sbuild.

24 November 2016

Dirk Eddelbuettel: RcppExamples 0.1.8

A new version of the RcppExamples package is now on CRAN. The RcppExamples package provides a handful of short examples detailing by concrete working examples how to set up basic R data structures in C++. This version takes advantage of the updated date and datetime classes in Rcpp 0.12.8 (which are optional for now and being phased in while we deprecate the old ones). A NEWS extract follows:

Changes in RcppExamples version 0.1.8 (2016-11-24)
  • Updated DateExample to show vector addition available under Rcpp 0.12.8 when the (currently still phased in and optional) new Date(time) classes are used via the define in src/Makevars,.win; with fallback code for older versions
  • Other minor edits to DESCRIPTION and README.md

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Sven Hoexter: first ditch effort - LyX 2.2.2 in unstable build with Qt5

No, not about the latest NOFX record, though it's a great one. Buy it. ;) Took me a hell of a long time to get my head out of my arse and dive again into some Debian related work. Thanks to Nik for pushing me from time to time. So I've taken the time to upload LyX 2.2.2 to unstable and it's now build with Qt5. Afterall the package is still missing a lot of love, but I hope we've once again something for the upcoming stable release, that is close to the latest upstream stable release. If you use LyX please give it a try. For myself it's now the 6th year that I stopped using LyX after maintaining it for five years. And still I'm sponsoring the uploads and try to keep it at least functional. Strange how we sometimes take care of stuff even if we no longer have an active use for them.

Ritesh Raj Sarraf: LIO -fb in Debian

LIO -fb is the new SCSI Target for Debian. Previously, we maintained the LIO tools from the pre-fork upstream branch. But, with good reasons, we've now moved to the newer -fb (Free Branch). As the maintainer for those pacakges, I have a local LIO setup. Overy the years, I've been tuning and using this setup with a bunch of SCSI clients. Now with the new -fb packages it was worrisome for me, on how to migrate (Note: migration is not supported by the Debian packages) my old setup to the new one. Thanks to Andy Grover for mentioning it, migrating your configuration is doable. With some minor intervention, I was able to switch my config from old LIO setup to the new LIO -fb pacakges. As you can see from the output below, both the outputs look the same, which is a good thing. LIO reads its configuration from /etc/target/ and passes it into the kernel. The kernel loads the config. The real time config is present in configfs, within the kernel. Users willing for such migration need to ensure that the loaded config data remains in configfs. And then, using the new -fb tools (targetctl), the configuration data needs to be read and written to a new format in /etc/. 
/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
    o- fileio ................................................................................................... [0 Storage Object]
    o- iblock .................................................................................................. [4 Storage Objects]
      o- CENTOS ................................................................................................. [/dev/vdd, in use]
      o- SAN1 ................................................................................................... [/dev/vdb, in use]
      o- SAN2 ................................................................................................... [/dev/vdc, in use]
      o- SANROOT .............................................. [/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0, in use]
    o- pscsi .................................................................................................... [0 Storage Object]
    o- rd_mcp ................................................................................................... [0 Storage Object]
  o- ib_srpt ........................................................................................................... [0 Targets]
  o- iscsi ............................................................................................................. [3 Targets]
    o- iqn.1994-05.com.redhat:23d8eb7fa1fc ................................................................................. [1 TPG]
      o- tpg1 ............................................................................................................ [enabled]
        o- acls ............................................................................................................ [1 ACL]
          o- iqn.1994-05.com.redhat:23d8eb7fa1fc .................................................................... [1 Mapped LUN]
            o- mapped_lun0 ............................................................................................. [lun0 (rw)]
        o- luns ............................................................................................................ [1 LUN]
          o- lun0 ....................................................................................... [iblock/CENTOS (/dev/vdd)]
        o- portals ..................................................................................................... [4 Portals]
          o- 172.16.20.40:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.41:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.42:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.43:3260 ................................................................................. [OK, iser disabled]
    o- iqn.2003-01.org.linux-iscsi.debian.sanboot .......................................................................... [1 TPG]
      o- tpg1 ............................................................................................................ [enabled]
        o- acls ............................................................................................................ [1 ACL]
          o- iqn.2005-03.org.open-iscsi:fd2bc2f4652a-sanboot ........................................................ [1 Mapped LUN]
            o- mapped_lun0 ............................................................................................. [lun0 (rw)]
        o- luns ............................................................................................................ [1 LUN]
          o- lun0 .................................... [iblock/SANROOT (/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0)]
        o- portals ..................................................................................................... [4 Portals]
          o- 172.16.20.40:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.41:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.42:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.43:3260 ................................................................................. [OK, iser disabled]
    o- iqn.2003-01.org.linux-iscsi.debian.x8664 ............................................................................ [1 TPG]
      o- tpg1 ............................................................................................................ [enabled]
        o- acls ............................................................................................................ [1 ACL]
          o- iqn.1993-08.org.debian:01:2972f6b5fc7 ................................................................. [2 Mapped LUNs]
            o- mapped_lun0 ............................................................................................. [lun0 (rw)]
            o- mapped_lun1 ............................................................................................. [lun1 (rw)]
        o- luns ........................................................................................................... [2 LUNs]
          o- lun0 ......................................................................................... [iblock/SAN1 (/dev/vdb)]
          o- lun1 ......................................................................................... [iblock/SAN2 (/dev/vdc)]
        o- portals ..................................................................................................... [4 Portals]
          o- 172.16.20.40:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.41:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.42:3260 ................................................................................. [OK, iser disabled]
          o- 172.16.20.43:3260 ................................................................................. [OK, iser disabled]
  o- loopback .......................................................................................................... [0 Targets]
  o- qla2xxx ........................................................................................................... [0 Targets]
  o- tcm_fc ............................................................................................................ [0 Targets]
  o- vhost ............................................................................................................. [0 Targets]
/> 
/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
    o- block .................................................................................................. [Storage Objects: 4]
      o- CENTOS ........................................................................... [/dev/vdd (2.0GiB) write-thru activated]
      o- SAN1 ............................................................................. [/dev/vdb (1.0GiB) write-thru activated]
      o- SAN2 ............................................................................. [/dev/vdc (1.0GiB) write-thru activated]
      o- SANROOT ........................ [/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0 (8.0GiB) write-thru activated]
    o- fileio ................................................................................................. [Storage Objects: 0]
    o- pscsi .................................................................................................. [Storage Objects: 0]
    o- ramdisk ................................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 3]
    o- iqn.1994-05.com.redhat:23d8eb7fa1fc ............................................................................... [TPGs: 1]
      o- tpg1 ............................................................................................... [no-gen-acls, no-auth]
        o- acls .......................................................................................................... [ACLs: 1]
          o- iqn.1994-05.com.redhat:23d8eb7fa1fc .................................................................. [Mapped LUNs: 1]
            o- mapped_lun0 ................................................................................ [lun0 block/CENTOS (rw)]
        o- luns .......................................................................................................... [LUNs: 1]
          o- lun0 ........................................................................................ [block/CENTOS (/dev/vdd)]
        o- portals .................................................................................................... [Portals: 4]
          o- 172.16.20.40:3260 ................................................................................................ [OK]
          o- 172.16.20.41:3260 ................................................................................................ [OK]
          o- 172.16.20.42:3260 ................................................................................................ [OK]
          o- 172.16.20.43:3260 ................................................................................................ [OK]
    o- iqn.2003-01.org.linux-iscsi.debian.sanboot ........................................................................ [TPGs: 1]
      o- tpg1 ............................................................................................... [no-gen-acls, no-auth]
        o- acls .......................................................................................................... [ACLs: 1]
          o- iqn.2005-03.org.open-iscsi:fd2bc2f4652a-sanboot ...................................................... [Mapped LUNs: 1]
            o- mapped_lun0 ............................................................................... [lun0 block/SANROOT (rw)]
        o- luns .......................................................................................................... [LUNs: 1]
          o- lun0 ..................................... [block/SANROOT (/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0)]
        o- portals .................................................................................................... [Portals: 4]
          o- 172.16.20.40:3260 ................................................................................................ [OK]
          o- 172.16.20.41:3260 ................................................................................................ [OK]
          o- 172.16.20.42:3260 ................................................................................................ [OK]
          o- 172.16.20.43:3260 ................................................................................................ [OK]
    o- iqn.2003-01.org.linux-iscsi.debian.x8664 .......................................................................... [TPGs: 1]
      o- tpg1 ............................................................................................... [no-gen-acls, no-auth]
        o- acls .......................................................................................................... [ACLs: 1]
          o- iqn.1993-08.org.debian:01:2972f6b5fc7 ................................................................ [Mapped LUNs: 2]
            o- mapped_lun0 .................................................................................. [lun0 block/SAN1 (rw)]
            o- mapped_lun1 .................................................................................. [lun1 block/SAN2 (rw)]
        o- luns .......................................................................................................... [LUNs: 2]
          o- lun0 .......................................................................................... [block/SAN1 (/dev/vdb)]
          o- lun1 .......................................................................................... [block/SAN2 (/dev/vdc)]
        o- portals .................................................................................................... [Portals: 4]
          o- 172.16.20.40:3260 ................................................................................................ [OK]
          o- 172.16.20.41:3260 ................................................................................................ [OK]
          o- 172.16.20.42:3260 ................................................................................................ [OK]
          o- 172.16.20.43:3260 ................................................................................................ [OK]
  o- loopback ......................................................................................................... [Targets: 0]
  o- vhost ............................................................................................................ [Targets: 0]
/>

Categories:

Keywords:

Like:

Ritesh Raj Sarraf: SAN Updates for Debian Stretch

Now that we prepare for the next Debian Stable release (Stretch), it is time to provide some updates on what the current state of some of the (storage related) packages in Debian is. This is not an update on the complete list of packages related to storage, but it does cover some of them. REMOVALS
  • iscsitarget - The iscsitarget stood as a great SCSI target for the Linux kernel. It seems to have had a good user base not just in Linux but also with VMWare users. But this storage target was always out-of-tree. With LIO having gotten merged as the default in-kernel SCSI Target, development on iscsitarget seems to have stalled. In Debian, for Stretch, there will be no iscsitarget. The package is already removed from Debian Testing and Debian Unstable, and nobody has volunteered to take over it.
  • system-storage-manager - This tool intended to be a simple unified storage tool, through which one could work with various storage technologies like LVM, BTRFS, cryptsetup, SCSI etc. But the upstream development hasn't really been much lately. For Debian Stable, it shouldn't be part of it, given it has some bugs.
  • libstoragemgmt - libstoragemgmt is a universal storage client-side library to talk to remote Storage Arrays. The project is active upstream. For Debian, the package is out-of-date and, now, also needs a maintainer. Unless someone picks up this package, it will not be part of Debian Stretch.
UPDATES
  • open-iscsi - This is the default iSCSI Initiator for Linux distributions. After a long slow development, upstream recently did a new release. This new release accomplished an important milestone; Hardware Offloading for QLogic cards. A special thanks to Frank Fegert, who helped with many aspects of the new iscsiuio package. And thanks to Christian Seiler, who is now co-maintaining the package, it is in great shape. We have fixed some long outstanding bugs and open-iscsi now has much much better integration with the whole system. For Jessie too, we have the up-to-date open-iscsi pacakges (including the new iscsiuio package, with iSCSI Offload) available through jessie-packports
  • open-isns - iSNS is the Naming Service for Storage. This is a new package in Debian Stretch. For users on Debian Jessie, Christian's efforts have made the open-isns package available in jessie-backports too.
  • multipath-tools - After years of slow development, multipath-tools too saw some active development this year, thanks to Xose and Christophe. The Debian version is up-to-date with the latest upstream release. For Debian Stretch, multipath-tools should have good integration with systemd.
  • sg3-utils - sg3 provides simple tools to query, using SCSI commands. The package is up-to-date and in good shape for Debian Stretch.
  • LIO Target - This is going to be the big entry for Debian Stretch. LIO is the in-kernel SCSI Target for Linux. For various reasons, we did not have LIO in Jessie. For Stretch, thanks to Christian Seiler and Christophe Vu-Brugier, we now have the well maintained -fb fork into Debian, which will replace the initial packages from the pre-fork upstream. The -fb fork is maintained by Andy Grover, and now, seems to have users from many other distributions and the kernel community. And given that LIO -fb branch is also part of the RHEL product family, we hope to see a well maintained project and an active upstream. The older packages: targetcli, python-rtslib and python-configshell shall be removed from the archive soon.
Debian users and derivatives, using these storage tools, may want to test/report now. Because once Stretch is released, getting new fixes in may not be easy enough. So please, if you have reliance on these tools, please test and report bugs, now.

Categories:

Keywords:

Like:

Next.

Previous.