Search Results: "ag"

25 February 2017

Jonas Meurer: debian lts report 2017.02

Debian LTS report for February 2017 February 2017 was my sixth month as a Debian LTS team member. I was allocated 5 hours and had 9,75 hours left over from January 2017. This makes a total of 14,75 hours. I spent all of them doing the following:

Stefano Zacchiroli: Software Freedom Conservancy matching

become a Conservancy supporter by February 28th and have your donation matched Non-profits that provide project support have proven themselves to be necessary for the success and advancement of individual projects and Free Software as a whole. The Free Software Foundation (founded in 1985) serves as a home to GNU projects and a canonical list of Free Software licenses. The Open Source Initiative came about in 1998, maintaining the Open Source Definition, based on the Debian Free Software Guidelines, with affiliate members including Debian, Mozilla, and the Wikimedia Foundation. Software in the Public Interest (SPI) was created in the late 90s largely to act as a fiscal sponsor for projects like Debian, enabling it to do things like accept donations and handle other financial transactions. More recently (2006), the Software Freedom Conservancy was formed. Among other activities like serving as a fiscal sponsor, infrastructure provider, and support organization for a number of free software projects including Git, Outreachy, and the Debian Copyright Aggregation Project they protect user freedom via copyleft compliance and GPL enforcement work. Without a willingness to act when licenses are violated, copyleft has no power. Through communication, collaboration, and only as last resort litigation, the Conservancy helps everyone who uses a freedom respecting license. The Conservancy has been aggressively fundraising in order to not just continue its current operations, but expand their work, staff, and efforts. They recently launched a donation matching campaign thanks to the generosity and dedication of an anonymous donor. Everyone who joins the Conservancy as a annual Supporter by February 28th will have their donation matched. A number of us are already supporters, and hope you will join us in supporting the world of an organization that supports us.

Martin Pitt: systemd 233 about to be released, please help testing

systemd 233 is scheduled to be released next week, and there is only a handful of small issues left. As usual there are tons of improvements and fixes, but the most intrusive one probably is another attempt to move from legacy cgroup v1 to a hybrid setup where the new unified (cgroup v2) hierarchy is mounted at /sys/fs/cgroup/unified/ and the legacy one stays at /sys/fs/cgroup/ as usual. This should provide an easier path for software like Docker or LXC to migrate to the unified hiearchy, but even that hybrid mode broke some bits. While systemd 233 will not make it into Debian stretch or Ubuntu zesty, as both are in feature freeze, it will soon be available in Debian experimental, and in the next Ubuntu release after 17.04 gets released. Thus now is another good time to give this some thorough testing! To help with this, please give the PPA with builds from upstream master a spin. In addition to the usual packages for Ubuntu 16.10 I also uploaded a build for Ubuntu zesty, and a build for Debian stretch (aka testing) which also works on Debian sid. You can use that URL as an apt source:
deb [trusted=yes] https://people.debian.org/~mpitt/tmp/systemd-master-20170225/ /
These packages pass our autopkgtests and I tested them manually too. LXC and LXD work fine, docker.io/runc needs a fix which I uploaded to Ubuntu zesty. (It s not yet available in Debian, sorry.) Please file reports about regressions on GitHub, but please also le me know about successes on my Google+ page so that we can get some idea about how many people tested this. Thank you, and happy booting!

24 February 2017

Gunnar Wolf: Started getting ads for ransomware. Coincidence?

Very strange. Verrrry strange. Yesterday I wrote a blog post on spam stuff that has been hitting my mailbox. Nothing too deep, just me scratching my head. Coincidentally (I guess/hope), I have been getting messages via my Bitlbee to one of my Jabber accounts, offering me ransomware services. I am reproducing it here, omitting of course everything I can recognize as their brand names related URLs (as I'm not going to promote the 3vi1-doers). I'm reproducing this whole as I'm sure the information will be interesting for some.
*BRAND* Ransomware - The Most Advanced and Customisable you've Ever Seen
Conquer your Independence with *BRAND* Ransomware Full Lifetime License!
* UNIQUE FEATURES
* NO DEPENDENCIES (.net or whatever)!!!
* Edit file Icon and UAC - Works on All Windows Versions
* Set Folders and Extensions to Encrypt, Deadline and Russian Roulette
* Edit the Text, speak with voice (multilang) and Colors for Ransom Window
* Enable/disable USB infect, network spread & file melt
* Set Process Name, sleep time, update ransom amount, Give mercy button
* Full-featured headquarter (for Windows) with unlimited builds, PDF reports, charts and maps, totally autonomous operation
* PHP Bridges instead of expensive C&C servers!
* Automatic Bitcoin payment detection (impossible to bypass/crack - we challege who says the contrary to prove what they say!)
* Totally/Mathematically IMPOSSIBLE to DECRYPT! Period.
* Award-Winning Five-Stars support and constant updates!
* We Have lot vouchs in *BRAND* Market, can check!
Watch the promo video: *URL*
Screenshots: *URL*
Website: *URL*
Price: $389
Promo: just $309 - 20% OFF! until 25th Feb 2017
Jabber: *JID*
I think I can comment on this with my students. Hopefully, this is interesting to others.
Now... I had never received Jabber-spam before. This message has been sent to me 14 times in the last 24 hours (all from different JIDs, all unknown to me). I hope this does not last forever :-/ Otherwise, I will have to learn more on how to configure Bitlbee to ignore contacts not known to me. Grrr...

Sven Hoexter: Tcl and https - back to TclCurl

Must be the irony of life that I was about to give up the TclCurl Debian package some time ago, and now I'm using it again for some very old and horrible web scraping code. The world moved on to https but the Tcl http package only supports unencrypted http. You can combine it with the tls package as explained in the Wiki, but that seems to be overly complicated compared to just loading the TclCurl binding and moving on with something like this:
package require TclCurl
# download to a variable
curl::transfer -url https://sven.stormbind.net -bodyvar page
# or store it in a file
curl::transfer -url https://sven.stormbind.net -file page.html
Now the remaining problem is that the code is unmaintained upstream and there is one codebase on bitbucket and one on github. While I fed patches to the bitbucket repo and thus based the Debian package on that repo, the github repo diverted in a different direction.

23 February 2017

Stig Sandbeck Mathisen: Change all the passwords (again)

Looks like it is time to change all the passwords again. There s a tiny little flaw in a CDN used everywhere, it seems. Here s a quick hack for users of the pass password manager to qickly find the domains affected. It is not perfect, but it is fast. :)
#!/bin/bash
# Stig Sandbeck Mathisen <ssm@fnord.no>
# Checks the content of "pass" against the list of sites using cloudflare.
# Expect false positives, and possibly false negatives.
# TODO: remove the left part of each hostname from pass, to check domains.
set -euo pipefail
tempdir=$(mktemp -d)
trap 'echo >&2 "removing $ tempdir " ; rm -rf "$tempdir"' EXIT
git clone https://github.com/pirate/sites-using-cloudflare.git "$tempdir"
grep -F -x -f \
  <(pass git ls-files    sed -e s,/,\ ,g -e s/.gpg//   xargs -n 1   sort -u) \
  "$ tempdir /sorted_unique_cf.txt" \
    sort -u
Update: The previous example used parallel. Actually, I didn t need that. Turns out, using grep correctly is much faster than using grep the wrong way. Lession: Read the manual. :)

Steve Kemp: Rotating passwords

Like many people I use a password-manage to record logins to websites. I previously used a tool called pwsafe, but these days I switched to using pass. Although I don't like the fact the meta-data is exposed the tool is very useful, and its integration with git is both simple and reliable. Reading about the security issue that recently affected cloudflare made me consider rotating some passwords. Using git I figured I could look at the last update-time of my passwords. Indeed that was pretty simple:
git ls-tree -r --name-only HEAD   while read filename; do
  echo "$(git log -1 --format="%ad" -- $filename) $filename"
done
Of course that's not quite enough because we want it sorted, and to do that using the seconds-since-epoch is neater. All together I wrote this:
#!/bin/sh
#
# Show password age - should be useful for rotation - we first of all
# format the timestamp of every *.gpg file, as both unix+relative time,
# then we sort, and finally we output that sorted data - but we skip
# the first field which is the unix-epoch time.
#
( git ls-tree -r --name-only HEAD   grep '\.gpg$'   while read filename; do \
      echo "$(git log -1 --format="%at %ar" -- $filename) $filename" ; done ) \
          sort   awk ' for (i=2; i<NF; i++) printf $i " "; print $NF '
Not the cleanest script I've ever hacked together, but the output is nice:
 steve@ssh ~ $ cd ~/Repos/personal/pass/
 steve@ssh ~/Repos/personal/pass $ ./password-age   head -n 5
 1 year, 10 months ago GPG/root@localhost.gpg
 1 year, 10 months ago GPG/steve@steve.org.uk.OLD.gpg
 1 year, 10 months ago GPG/steve@steve.org.uk.NEW.gpg
 1 year, 10 months ago Git/git.steve.org.uk/root.gpg
 1 year, 10 months ago Git/git.steve.org.uk/skx.gpg
Now I need to pick the sites that are more than a year old and rotate credentials. Or delete accounts, as appropriate.

Joerg Jaspert: Automated wifi login

If you have the fortune to need to follow some silly Login button for some wifi, regularly, the following little script may help you avoid this idiotic (and useless) task. This example uses the WIFIonICE, the free wifi on german ICE trains, simply as I have it twice a day, and got annoyed by the pointless Login button. A friend pointed me at just wget-ting the login page, so I made Network-Manager do this for me. Should work for anything similar that doesn t need some elaborate webform filled out.
#!/bin/bash
# (Some) docs at
# https://wiki.ubuntuusers.de/NetworkManager/Dispatcher/
IFACE=$ 1:-"none" 
ACTION=$ 2:-"up" 
case $ ACTION  in
    up)
        CONID=$ CONNECTION_ID:-$(iwconfig $IFACE   grep ESSID   cut -d":" -f2   sed 's/^[^"]*"\ "[^"]*$//g') 
        if [[ $ CONID  == WIFIonICE ]]; then
            /usr/bin/timeout -k 20 15 /usr/bin/wget -q -O - http://www.wifionice.de/?login > /dev/null
        fi
        ;;
    *)
        # We are not interested in this
        :
        ;;
esac
This script needs to be put into /etc/NetworkManager/dispatcher.d and made executable, owned by the root user. It will run on every connection change, thats why the ACTION is checked. The case may be a bit much here, but it could be easily extended to do a lot more. Yay, no more silly Open this webpage and press login crap.

Gunnar Wolf: Spam: Tactics, strategy, and angry bears

I know spam is spam is spam, and I know trying to figure out any logic underneath it is a lost cause. However... I am curious. Many spam subjects are seemingly random, designed to convey whatever "information" they contain and fool spam filters. I understand that. Many spam subjects are time-related. As an example, in the last months there has been a surge of spam mentioning Donald Trump. I am thankful: Very easy to filter out, even before it reaches spamassassin. Of course, spam will find thousands of ways to talk about sex; cialis/viagra sellers, escort services, and a long list of WTF. However... Tactical flashlights. Bright enough to blind a bear. WTF I mean... Truly. Really. WTF What does that mean? Why is that even a topic? Who is interested in anything like that? How often does the average person go camping in the woods? Why do we need to worry about stupid bears attacking us? Why would a bear attack me? The list of WTF questions could go on forever. What am I missing? What does "tactical flashlight" mean that I just fail to grasp? Has this appeared in your spam?

22 February 2017

Antoine Beaupr : The case against password hashers

In previous articles, we have looked at how to generate passwords and did a review of various password managers. There is, however, a third way of managing passwords other than remembering them or encrypting them in a "vault", which is what I call "password hashing". A password hasher generates site-specific passwords from a single master password using a cryptographic hash function. It thus allows a user to have a unique and secure password for every site they use while requiring no storage; they need only to remember a single password. You may know these as "deterministic or stateless password managers" but I find the "password manager" phrase to be confusing because a hasher doesn't actually store any passwords. I do not think password hashers represent a good security tradeoff so I generally do not recommend their use, unless you really do not have access to reliable storage that you can access readily. In this article, I use the word "password" for a random string used to unlock things, but "token" to represent a generated random string that the user doesn't need to remember. The input to a password hasher is a password with some site-specific context and the output from a password hasher is a token.

What is a password hasher? A password hasher uses the master password and a label (generally the host name) to generate the site-specific password. To change the generated password, the user can modify the label, for example by appending a number. Some password hashers also have different settings to generate tokens of different lengths or compositions (symbols or not, etc.) to accommodate different site-specific password policies. The whole concept of password hashers relies on the concept of one-way cryptographic hash functions or key derivation functions that take an arbitrary input string (say a password) and generate a unique token, from which it is impossible to guess the original input string. Password hashers are generally written as JavaScript bookmarklets or browser plugins and have been around for over a decade. The biggest advantage of password hashers is that you only need to remember a single password. You do not need to carry around a password manager vault: there's no "state" (other than site-specific settings, which can be easily guessed). A password hasher named Master Password makes a compelling case against traditional password managers in its documentation:
It's as though the implicit assumptions are that everybody backs all of their stuff up to at least two different devices and backups in the cloud in at least two separate countries. Well, people don't always have perfect backups. In fact, they usually don't have any.
It goes on to argue that, when you lose your password: "You lose everything. You lose your own identity." The stateless nature of password hashers also means you do not need to use cloud services to synchronize your passwords, as there is (generally, more on that later) no state to carry around. This means, for example, that the list of accounts that you have access to is only stored in your head, and not in some online database that could be hacked without your knowledge. The downside of this is, of course, that attackers do not actually need to have access to your password hasher to start cracking it: they can try to guess your master key without ever stealing anything from you other than a single token you used to log into some random web site. Password hashers also necessarily generate unique passwords for every site you use them on. While you can also do this with password managers, it is not an enforced decision. With hashers, you get distinct and strong passwords for every site with no effort.

The problem with password hashers If hashers are so great, why would you use a password manager? Programs like LessPass and Master Password seem to have strong crypto that is well implemented, so why isn't everyone using those tools? Password hashing, as a general concept, actually has serious problems: since the hashing outputs are constantly compromised (they are sent in password forms to various possibly hostile sites), it's theoretically possible to derive the master password and then break all the generated tokens in one shot. The use of stronger key derivation functions (like PBKDF2, scrypt, or HMAC) or seeds (like a profile-specific secret) makes those attacks much harder, especially if the seed is long enough to make brute-force attacks infeasible. (Unfortunately, in the case of Password Hasher Plus, the seed is derived from Math.random() calls, which are not considered cryptographically secure.) Basically, as stated by Julian Morrison in this discussion:
A password is now ciphertext, not a block of line noise. Every time you transmit it, you are giving away potential clues of use to an attacker. [...] You only have one password for all the sites, really, underneath, and it's your secret key. If it's broken, it's now a skeleton-key [...]
Newer implementations like LessPass and Master Password fix this by using reasonable key derivation algorithms (PBKDF2 and scrypt, respectively) that are more resistant to offline cracking attacks, but who knows how long those will hold? To give a concrete example, if you would like to use the new winner of the password hashing competition (Argon2) in your password manager, you can patch the program (or wait for an update) and re-encrypt your database. With a password hasher, it's not so easy: changing the algorithm means logging in to every site you visited and changing the password. As someone who used a password hasher for a few years, I can tell you this is really impractical: you quickly end up with hundreds of passwords. The LessPass developers tried to facilitate this, but they ended up mostly giving up. Which brings us to the question of state. A lot of those tools claim to work "without a server" or as being "stateless" and while those claims are partly true, hashers are way more usable (and more secure, with profile secrets) when they do keep some sort of state. For example, Password Hasher Plus records, in your browser profile, which site you visited and which settings were used on each site, which makes it easier to comply with weird password policies. But then that state needs to be backed up and synchronized across multiple devices, which led LessPass to offer a service (which you can also self-host) to keep those settings online. At this point, a key benefit of the password hasher approach (not keeping state) just disappears and you might as well use a password manager. Another issue with password hashers is choosing the right one from the start, because changing software generally means changing the algorithm, and therefore changing passwords everywhere. If there was a well-established program that was be recognized as a solid cryptographic solution by the community, I would feel more confident. But what I have seen is that there are a lot of different implementations each with its own warts and flaws; because changing is so painful, I can't actually use any of those alternatives. All of the password hashers I have reviewed have severe security versus usability tradeoffs. For example, LessPass has what seems to be a sound cryptographic implementation, but using it requires you to click on the icon, fill in the fields, click generate, and then copy the password into the field, which means at least four or five actions per password. The venerable Password Hasher is much easier to use, but it makes you type the master password directly in the site's password form, so hostile sites can simply use JavaScript to sniff the master password while it is typed. While there are workarounds implemented in Password Hasher Plus (the profile-specific secret), both tools are more or less abandoned now. The Password Hasher homepage, linked from the extension page, is now a 404. Password Hasher Plus hasn't seen a release in over a year and there is no space for collaborating on the software the homepage is simply the author's Google+ page with no information on the project. I couldn't actually find the source online and had to download the Chrome extension by hand to review the source code. Software abandonment is a serious issue for every project out there, but I would argue that it is especially severe for password hashers. Furthermore, I have had difficulty using password hashers in unified login environments like Wikipedia's or StackExchange's single-sign-on systems. Because they allow you to log in with the same password on multiple sites, you need to choose (and remember) what label you used when signing in. Did I sign in on stackoverflow.com? Or was it stackexchange.com? Also, as mentioned in the previous article about password managers, web-based password managers have serious security flaws. Since more than a few password hashers are implemented using bookmarklets, they bring all of those serious vulnerabilities with them, which can range from account name to master password disclosures. Finally, some of the password hashers use dubious crypto primitives that were valid and interesting a decade ago, but are really showing their age now. Stanford's pwdhash uses MD5, which is considered "cryptographically broken and unsuitable for further use". We have seen partial key recovery attacks against MD5 already and while those do not allow an attacker to recover the full master password yet (especially not with HMAC-MD5), I would not recommend anyone use MD5 in anything at this point, especially if changing that algorithm later is hard. Some hashers (like Password Hasher and Password Plus) use a a single round of SHA-1 to derive a token from a password; WPA2 (standardized in 2004) uses 4096 iterations of HMAC-SHA1. A recent US National Institute of Standards and Technology (NIST) report also recommends "at least 10,000 iterations of the hash function".

Conclusion Forced to suggest a password hasher, I would probably point to LessPass or Master Password, depending on the platform of the person asking. But, for now, I have determined that the security drawbacks of password hashers are not acceptable and I do not recommend them. It makes my password management recommendation shorter anyway: "remember a few carefully generated passwords and shove everything else in a password manager". [Many thanks to Daniel Kahn Gillmor for the thorough reviews provided for the password articles.]
Note: this article first appeared in the Linux Weekly News. Also, details of my research into password hashers are available in the password hashers history article.

Neil McGovern: A new journey GNOME Foundation Executive Director

IMG_0726For those who haven t heard, I ve been appointed as the new Executive Director of the GNOME Foundation, and I started last week on the 15th February. It s been an interesting week so far, mainly meeting lots of people and trying to get up to speed with what looks like an enormous job! However, I m thoroughly excited by the opportunity and am very grateful for everyone s warm words of welcome so far. One of the main things I m here to do is to try and help. GNOME is strong because of its community. It s because of all of you that GNOME can produce world leading technologies and a desktop that is intuitive, clean and functional. So, if you re stuck with something, or if there s a way that either myself or the Foundation can help, then please speak up! Additionally, I intend on making this blog a much more frequently updated one letting people know what I m doing, and highlighting cool things that are happening around the project. In that vein, this week I ve also started contacting all our fantastic Advisory Board members. I m also looking at finding sponsors for GUADEC and GNOME.Asia, so if you know of anyone, let me know! I also booked my travel to the GTK+ hackfest and to LibrePlanet if you re going to either of those, make sure you come and introduce yourself :) Finally, a small advertisement for Friends of GNOME. Your generosity really does help the Foundation support development of GNOME. Join up today!

Lisandro Dami n Nicanor P rez Meyer: Developing an nrf51822 based embedded device with Qt Creator and Debian

I'm currently developing an nRF51822-based embedded device. Being one the Qt/Qt Creator maintainers in Debian I would of course try to use it for the development. Turns out it works pretty good... with some caveats.

There are already two quite interesting blog posts about using Qt Creator on MAC and on Windows, so I will not repeat the basics, as they are there. Both use qbs, but I managed to use CMake.

Instead I'll add some tips on the stuff that I needed to solve in order to make this happen on current Debian Sid.



Cheers!

Enrico Zini: staticsite news: github mode and post series

GitHub mode Tobias Gruetzmacher implemented GitHub mode for staticsite. Although GitHub now has a similar site rendering mode, it doesn't give you a live preview: if you run ssite serve on a GitHub project you will get a live preview of README.md and the project documentation. Post series I have added support for post series, that allow you to easily interlink posts with previous/next links. You can see it in action on links and on An Italian song a day, an ongoing series that is currently each day posting a link to an Italian song.

Jonathan Dowland: Hans Rosling and Steve Hewlett

I've begun to listen to BBC4's "More Or Less" Podcast. They recently had an episode covering the life and work of Hans Rosling, the inspirational swedish statistician, who has sadly died of pancreatic cancer. It was very moving. Some of Professor Rosling's videos are available to view online. I've heard that they are very much worth watching. Over the last few months I have also been listening to regular updates by BBC broadcaster Steve Hewlett on his own journey as a cancer sufferer. These were remarkably frank discussions of the ins and outs of his diagnosis, treatment, and the practical consequences on his everyday life. I was very sad to tune in on Monday evening and hear a series of repeated clips from his previous appearances on the PM show, as the implications were clear. And indeed, Steve Hewlett died from oesophagal cancer on Monday. Here's an obituary in the Guardian.

Joey Hess: early spring

Sun is setting after 7 (in the JEST TZ); it's early spring. Batteries are generally staying above 11 volts, so it's time to work on the porch (on warmer days), running the inverter and spinning up disc drives that have been mostly off since fall. Back to leaving the router on overnight so my laptop can sync up before I wake up. Not enough power yet to run electric lights all evening, and there's still a risk of a cloudy week interrupting the climb back up to plentiful power. It's happened to me a couple times before. Also, turned out that both of my laptop DC-DC power supplies developed partial shorts in their cords around the same time. So at first I thought it was some problem with the batteries or laptop, but eventually figured it out and got them replaced. (This may have contributed the the cliff earier; seemed to be worst when house voltage was low.) Soon, 6 months of more power than I can use.. Previously: battery bank refresh late summer the cliff

21 February 2017

Shirish Agarwal: The Indian elections hungama

a person showing s(he) showing s(he) Before I start, I would like to point out #855549 . This is a normal/wishlist bug I have filed against apt, the command-line package manager. I sincerely believe having a history command to know what packages were installed, which were upgraded, which were purged should be easily accessible, easily understood and if the output looks pretty, so much the better. Of particular interest to me is having a list of new packages I have installed in last couple of years after jessie became the stable release. It probably would make for some interesting reading. I dunno how much efforts would be to code something like that, but if it works, it would be the greatest. Apt would have finally arrived. Not that it s a bad tool, it s just that it would then make for a heck of a useful tool. Coming back to the topic on hand, Now for the last couple of weeks we don t have water or rather pressure of water. Water crisis has been hitting Pune every year since 2014 with no end in sight. This has been reported in newspapers addendum but it seems it has been felling on deaf ears. The end result of it is that I have to bring buckets of water from around 50 odd metres. It s not a big thing, it s not like some women in some villages in Rajasthan who have to walk in between 200 metres to 5 odd kilometres to get potable water or Darfur, Western Sudan where women are often kidnapped and sold as sexual slaves when they get to fetch water. The situation in Darfur has been shown quite vividly in Darfur is Dying . It is possible that I may have mentioned about Darfur before. While unfortunately the game is in flash as a web resource, the most disturbing part is that the game is extremely depressing, there is a no-win scenario. So knowing and seeing both those scenarios, I can t complain about 50 metres. BUT .but when you extrapolate the same data over some more or less 3.3-3.4 million citizens, 3.1 million during 2011 census with a conservative 2.3-2.4 percent population growth rate according to scroll.in. Fortunately or unfortunately, Pune Municipal Corporation elections were held today. Fortunately or unfortunately, this time all the political parties bought majorly unknown faces in these elections. For e.g. I belong to ward 14 which is spread over quite a bit of area and has around 10k of registered voters. Now the unfortunate part of having new faces in elections, you don t know anything about them. Apart from the affidavits filed, the only thing I come to know is whether there are criminal cases filed against them and what they have shown as their wealth. While I am and should be thankful to ADR which actually is the force behind having the collated data made public. There is a lot of untold story about political push-back by all the major national and regional political parties even when this bit of news were to be made public. It took major part of a decade for such information to come into public domain. But for my purpose of getting clean air and water supply 24 7 to each household seems a very distant dream. I tried to connect with the corporators about a week before the contest and almost all of the lower party functionaries hid behind their political parties manifestos stating they would do the best without any viable plan. For those not knowing, India has been blessed with 6 odd national parties and about 36 odd regional parties and every election some 20-25 new parties try their luck every time. The problem is we, the public, don t trust them or their manifestos. First of all the political parties themselves engage in mud-slinging as to who s copying whom with the manifesto.Even if a political party wins the elections, there is no *real* pressure for them to follow their own manifesto. This has been going for many a year. OF course, we the citizens are to also blame as most citizens for one reason or other chose to remain aloof of the process. I scanned/leafed through all the manifestos and all of them have the vague-wording we will make Pune tanker-free without any implementation details. While I was unable to meet the soon-to-be-Corporators, I did manage to meet a few of the assistants but all the meetings were entirely fruitless. Diagram of Rain Water Harvesting I asked why can t the city follow the Chennai model. Chennai, not so long ago was at the same place where Pune is, especially in relation to water. What happened next, in 2001 has been beautifully chronicled in Hindustan Times . What has not been shared in that story is that the idea was actually fielded by one of Chennai Mayor s assistants, an IAS Officer, I have forgotten her name, Thankfully, her advise/idea was taken to heart by the political establishment and they drove RWH. Saying why we can t do something similar in Pune, I heard all kinds of excuses. The worst and most used being Marathas can never unite which I think is pure bullshit. For people unfamiliar to the term, Marathas was a warrior clan in Shivaji s army. Shivaji, the king of Marathas were/are an expert tactician and master of guerilla warfare. It is due to the valor of Marathas, that we still have the Maratha Light Infantry a proud member of the Indian army. Why I said bullshit was the composition of people living in Maharashtra has changed over the decades. While at one time both the Brahmins and the Marathas had considerable political and population numbers, that has changed drastically. Maharashtra and more pointedly, Mumbai, Pune and Nagpur have become immigrant centres. Why just a decade back, Shiv Sena, an ultra right-wing political party used to play the Maratha card at each and every election and heckle people coming from Uttar Pradesh and Bihar, this has been documented as the 2008 immigrants attacks and 9 years later we see Shiv Sena trying to field its candidates in Uttar Pradesh. So, obviously they cannot use the same tactics which they could at one point of time. One more reason I call it bullshit, is it s a very lame excuse. When the Prime Minister of the country calls for demonetization which affects 1.25 billion people, people die, people stand in queues and is largely peaceful, I do not see people resisting if they bring a good scheme. I almost forgot, as an added sweetener, the Chennai municipality said that if you do RWH and show photos and certificates of the job, you won t have to pay as much property tax as otherwise you would, that also boosted people s participation. And that is not the only solution, one more solution has been outlined in Aaj Bhi Khade hain talaab written by just-deceased Gandhian environmental activist Anupam Mishra. His Book can be downloaded for free at India Water Portal . Unfortunately, the said book doesn t have a good English translation till date. Interestingly, all of his content is licensed under public domain (CC-0) so people can continue to enjoy and learn from his life-work. Another lesson or understanding could be taken from Israel, the father of the modern micro-drip irrigation for crops. One of the things on my bucket lists is to visit Israel and if possible learn how they went from a water-deficient country to a water-surplus one. India labor Which brings me to my second conundrum, most of the people believe that it s the Government s job to provide jobs to its people. India has been experiencing jobless growth for around a decade now, since the 2008 meltdown. While India was lucky to escape that, most of its trading partners weren t hence it slowed down International trade which slowed down creation of new enterprises etc. Laws such as the Bankruptcy law and the upcoming Goods and Services Tax . As everybody else, am a bit excited and a bit apprehensive about how the actual implementation will take place. null Even International businesses has been found wanting. The latest example has been Uber and Ola. There have been protests against the two cab/taxi aggregators operating in India. For the millions of jobless students coming out of schools and Universities, there aren t simply enough jobs for them, nor are most (okay 50%) of them qualified for the jobs, these 50 percent are also untrainable, so what to do ? In reality, this is what keeps me awake at night. India is sitting on this ticking bomb-shell. It is really, a miracle that the youths have not rebelled yet. While all the conditions, proposals and counter-proposals have been shared before, I wanted/needed to highlight it. While the issue seems to be local, I would assert that they are all glocal in nature. The questions we are facing, I m sure both developing and to some extent even developed countries have probably been affected by it. I look forward to know what I can learn from them. Update 23/02/17 I had wanted to share about Debian s Voting system a bit, but that got derailed. Hence in order not to do, I ll just point towards 2015 platforms where 3 people vied for DPL post. I *think* I shared about DPL voting process earlier but if not, would do in detail in some future blog post.
Filed under: Miscellenous Tagged: #Anupam Mishra, #Bankruptcy law, #Chennai model, #clean air, #clean water, #elections, #GST, #immigrant, #immigrants, #Maratha, #Maratha Light Infantry, #migration, #national parties, #Political party manifesto, #regional parties, #ride-sharing, #water availability, Rain Water Harvesting

Steinar H. Gunderson: 8-bit Y'CbCr ought to be enough for anyone?

If you take a random computer today, it's pretty much a given that it runs a 24-bit mode (8 bits of each of R, G and B); as we moved from palettized displays at some point during the 90s, we quickly went past 15- and 16-bit and settled on 24-bit. The reasons are simple; 8 bits per channel is easy to work with on CPUs, and it's on the verge of what human vision can distinguish, at least if you add some dither. As we've been slowly taking the CPU off the pixel path and replacing it with GPUs (which has specialized hardware for more kinds of pixels formats), changing formats have become easier, and there's some push to 10-bit (30-bit) deep color for photo pros, but largely, 8-bit per channel is where we are. Yet, I'm now spending time adding 10-bit input (and eventually also 10-bit output) to Nageru. Why? The reason is simple: Y'CbCr. Video traditionally isn't done in RGB, but in Y'CbCr; that is, a black-and-white signal (Y) and then two color-difference signals (Cb and Cr, roughly additional blueness and additional redness , respectively). We started doing this because it was convenient in analog TV (if you separate the two, black-and-white TVs can just ignore the color signal), but we kept doing it because it's very nice for reducing bandwidth: Human vision is much less sensitive to color than to brightness, so we can transfer the color channels in lower resolution and get away with it. (Also, a typical Bayer sensor can't deliver full color resolution anyway.) So most cameras and video codecs work in Y'CbCr, not RGB. Let's look at the implications of using 8-bit Y'CbCr, using a highly simplified model for, well, simplicity. Let's define Y = 1/3 (R + G + B), Cr = R - Y and Cb = B - Y. (The reverse transformation becomes R = Y + Cr, B = Y + Cb and G = 3Y - R - B.) This means that an RGB color such as pure gray ([127, 127, 127]) becomes [127, 0, 0]. All is good, and Y can go from 0 to 255, just like R, G and B can. A pure red ([255, 0, 0]) becomes [85, 170, 0], and a pure blue ([255, 0, 0]) becomes correspondingly [85, 0, 170]. But we can also have negative Cr and Cb values; a pure yellow ([0, 255, 255]) becomes [170, -170, 85], for instance. So we need to squeeze values from -170 to +170 into an 8-bit range, losing accuracy. Even worse, there are valid Y'CbCr triplets that don't correspond to meaningful RGB colors at all. For instance, Y'CbCr [255, 170, 0] would be RGB [425, 85, 255]; R is out of range! And Y'CbCr [255, -170, 0] would be RGB [85, -85, 255], that is, negative green. This isn't a problem for compression, as we can just avoid using those illegal colors with no loss of efficiency. But it means that the conversion in itself causes a loss; actually, if you do the maths on the real formulas (using the BT.601 standard), it turns out only 17% of the 24-bit Y'CbCr code words are valid! In other words, we lose about two and a half bits of data, and our 24 bits of accuracy have been reduced to 21.5. Or, to put it another way; 8-bit Y'CbCr is roughly equivalent to 7-bit RGB. Thus, pretty much all professional video uses 10-bit Y'CbCr. It's much more annoying to deal with (especially when you've got subsampling!), but if you're using SDI, there's not even any 8-bit version defined, so if you insist on 8-bit, you're taking data you're getting on the wire (whether you want it or not) and throwing 20% of it away. UHDTV standards (using HEVC) are also simply not defined for 8-bit; it's 10- and 12-bit only, even on the codec level. Parts of this is because UHDTV also supports HDR, so you have a wider RGB range than usual to begin with, and 8-bit would cause excessive banding. Using it on the codec level makes a lot of sense for another reason, namely that you reduce internal roundoff errors during processing by a lot; errors equal noise, and noise is bad for compression. I've seen numbers of 15% lower bitrate for H.264 at the same quality, although you also have to take into account that the encoeder also needs more CPU power that you could have used for a higher preset in 8-bit. I don't know how the tradeoff here works out, and you also have to take into account decoder support for 10-bit, especially when it comes to hardware. (When it comes to HEVC, Intel didn't get full fixed-function 10-bit support before Kaby Lake!) So indeed, 10-bit Y'CbCr makes sense even for quite normal video. It isn't a no-brainer to turn it on, though even though Nageru uses a compute shader to convert the 4:2:2 10-bit Y'CbCr to something the GPU can sample from quickly (ie., the CPU doesn't need to touch it), and all internal processing is in 16-bit floating point anyway, it still takes a nonzero amount of time to convert compared to just blasting through 8-bit, so my ultraportable probably won't make it anymore. (A discrete GPU has no issues at all, of course. My laptop converts a 720p frame in about 1.4 ms, FWIW.) But it's worth considering when you want to squeeze even more quality out of the system. And of course, there's still 10-bit output support to be written...

Reproducible builds folks: Reproducible Builds: week 95 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday February 12 and Saturday February 18 2017: Upcoming Events The Reproducible Build Zoo will be presented by Vagrant Cascadian at the Embedded Linux Conference in Portland, Oregon, February 22nd. Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th. Toolchain development and fixes Ximin Luo posted a preliminary spec for BUILD_PATH_PREFIX_MAP, bringing together work and research from previous weeks. Ximin refactored and consolidated much of our existing documentation on both SOURCE_DATE_EPOCH and BUILD_PATH_PREFIX_MAP into one unified page, Standard Environment Variables, with extended discussion on related solutions and how these all fit into people's ideas of what reproducible builds should look like in the long term. The specific pages for each variable still remain, at Timestamps Proposal and Build Path Proposal, only without content that was previously duplicated on both pages. Ximin filed #855282 against devscripts for debsign(1) to support buildinfo files, and wrote an initial series of patches for it with some further additions from Guillem Jover. Packages reviewed and fixed, and bugs filed Chris Lamb: Reviews of unreproducible packages 35 package reviews have been added, 1 have been updated and 17 have been removed in this week, adding to our knowledge about identified issues. 1 issue type has been added: Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by: diffoscope development diffoscope 77 was uploaded to unstable by Mattia Rizzolo. It included contributions from: strip-nondeterminism development strip-nondeterminism 0.031-1 was uploaded to unstable by Chris Lamb. It included contributions from: strip-nondeterminism 0.031-1~bpo8+1 was uploaded to jessie-backports by Mattia. tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Arturo Borrero Gonz lez: About process limits, round 2

htop I was wrong. After the other blog post About process limits, some people contacted me with additional data and information. I myself continued to investigate on the issue, so I have new facts. I read again the source code of the slapd daemon and the picture seems clearer now. A new message appeared in the log files:
[...]
Feb 20 06:26:03 slapd[18506]: daemon: 1025 beyond descriptor table size 1024
Feb 20 06:26:03 slapd[18506]: daemon: 1025 beyond descriptor table size 1024
Feb 20 06:26:03 slapd[18506]: daemon: 1025 beyond descriptor table size 1024
Feb 20 06:26:03 slapd[18506]: daemon: 1025 beyond descriptor table size 1024
Feb 20 06:26:03 slapd[18506]: daemon: 1025 beyond descriptor table size 1024
[...]
This message is clearly produced by the daemon itself, and searching for the string leads to this source code, in servers/slapd/daemon.c:
[...]
sfd = SLAP_SOCKNEW( s );
/* make sure descriptor number isn't too great */
if ( sfd >= dtblsize )  
	Debug( LDAP_DEBUG_ANY,
		"daemon: %ld beyond descriptor table size %ld\n",
		(long) sfd, (long) dtblsize, 0 );
	tcp_close(s);
	ldap_pvt_thread_yield();
	return 0;
 
[...]
In that same file, dtblsize is set to:
[...]
#ifdef HAVE_SYSCONF
        dtblsize = sysconf( _SC_OPEN_MAX );
#elif defined(HAVE_GETDTABLESIZE)
        dtblsize = getdtablesize();
#else /* ! HAVE_SYSCONF && ! HAVE_GETDTABLESIZE */
        dtblsize = FD_SETSIZE;
#endif /* ! HAVE_SYSCONF && ! HAVE_GETDTABLESIZE */
[...]
If you keep pulling the string, the first two options use system limits to know the value, getrlimit(), and the last one uses a fixed value of 4096 (set at build time). It turns out that this routine slapd_daemon_init() is called once, at daemon startup (see main() function at servers/slapd/main.c). So the daemon is limiting itself to the limit imposed by the system at daemon startup time. That means that our previous limits settings at runtime was not being read by the slapd daemon. Let s back to the previous approach of establishing the process limits by setting them on the user. The common method is to call ulimit in the init.d script (or systemd service file). One of my concerns of this approach was that slapd runs as a different user, usually openldap. Again, reading the source code:
[...]
if( check == CHECK_NONE && slapd_daemon_init( urls ) != 0 )  
	rc = 1;
        SERVICE_EXIT( ERROR_SERVICE_SPECIFIC_ERROR, 16 );
        goto stop;
 
#if defined(HAVE_CHROOT)
	if ( sandbox )  
		if ( chdir( sandbox ) )  
			perror("chdir");
			rc = 1;
			goto stop;
		 
		if ( chroot( sandbox ) )  
			perror("chroot");
			rc = 1;
			goto stop;
		 
	 
#endif
#if defined(HAVE_SETUID) && defined(HAVE_SETGID)
	if ( username != NULL   groupname != NULL )  
		slap_init_user( username, groupname );
	 
#endif
[...]
So, the slapd daemon first reads the limits and then change user to openldap, (the slap_init_user() function). We can then asume that if we set the limits to the root user, calling ulimit in the init.d script, the slapd daemon will actually inherint them. This is what is originally suggested in debian bug #660917. Let s use this solution for now. Many thanks to John Hughes john@atlantech.com for the clarifications via email.

20 February 2017

Petter Reinholdtsen: Detect OOXML files with undefined behaviour?

I just noticed the new Norwegian proposal for archiving rules in the goverment list ECMA-376 / ISO/IEC 29500 (aka OOXML) as valid formats to put in long term storage. Luckily such files will only be accepted based on pre-approval from the National Archive. Allowing OOXML files to be used for long term storage might seem like a good idea as long as we forget that there are plenty of ways for a "valid" OOXML document to have content with no defined interpretation in the standard, which lead to a question and an idea. Is there any tool to detect if a OOXML document depend on such undefined behaviour? It would be useful for the National Archive (and anyone else interested in verifying that a document is well defined) to have such tool available when considering to approve the use of OOXML. I'm aware of the officeotron OOXML validator, but do not know how complete it is nor if it will report use of undefined behaviour. Are there other similar tools available? Please send me an email if you know of any such tool.

Next.

Previous.