Search Results: "nate"

9 December 2016

Simon Richter: Busy

I'm fairly busy at the moment, so I don't really have time to work on free software, and when I do I really want to do something else than sit in front of a computer. I have declared email bankruptcy at 45,000 unread mails. I still have them, and plan to deal with them in small batches of a few hundred at a time, but in case you sent me something important, it is probably stuck in there. I now practice Inbox Zero, so resending it is a good way to reach me. For my Debian packages, not much changes. Any package with more than ten users is team maintained anyway. Sponsoring for the packages where I agreed to do so goes on. For KiCad, I won't get around to much of what I'd planned this year. Fortunately, at this point no one expects me to do anything soon. I still look into the CI system and unclog anything that doesn't clear on its own within a week. Plans for December:

actually having my own place. While I like the room I'm staying at, it is still fairly expensive because it's paid by the day, and living out of a suitcase without access to my library is kind of annoying after some time.
finishing the paperwork for 2016. Except for some small bits, most of it is in place.
33C3. This time, instead of the "two monitors, three computers" setup, my plan is to have a single laptop only, and have it closed most of the time so the battery lasts the whole day.
See how far I'll get with the controller board for the CNC mill in the Munich Maker Lab. Absolutely no pressure there, it's only the most complex and expensive PCB I ever made.

Plans for January:

Getting settled in.
Back to the Carbon Monoxide detector board that we started in early November. The board is simple enough.
Visiting a demoparty in Finland

Plans for February:

FOSDEM. I plan to hang out in the EDA devroom most of the time, and go to dinner with friends.
Party. Specifically, a housewarming party for whatever flat I'll have then.

Other than that, reading lots of books and meeting other people.

John Goerzen: Giant Concrete Arrows, Old Maps, and Fascinated Kids

Let me set a scene for you. Two children, ages 7 and 10, are jostling for position. There s a little pushing and shoving to get the best view. This is pretty typical for siblings this age. But what, you may wonder, are they trying to see? A TV? Video game? No. Jacob and Oliver were in a library, trying to see a 98-year-old map of the property owners in Township 23, range 1 East, Harvey County, Kansas. And they were super excited about it, somewhat to the astonishment of the research librarian, who I am sure is more used to children jostling for position over the DVDs in the youth section than poring over maps in the non-circulating historical archives! All this started with giant concrete arrows in the middle of nowhere.

Nearly a century ago, the US government installed a series of arrows on the ground in Kansas. These were part of a primitive air navigation system that led to the first transcontinental airmail service. Every so often, people stumble upon these abandoned arrows and there is a big discussion online. Even Snopes has had to verify their authenticity (verdict: true). Entire websites exist to tracking and locating the remnants of these arrows. And as one of the early air mail routes went through Kansas, every so often people find these arrows around here. I got the idea that it would be fun to replicate a journey along the old routes. Maybe I d spot a few old arrows and such. So I started collecting old maps: a Contract Airmail Route #34 (CAM 34) map from 1927, aviation sectionals from 1933 and 1946, etc. I noticed an odd thing on these maps: the Newton, KS airport was on the other side of the city from its present location, sometimes even several miles outside the city. What was going on?

(1927 Airway Map)

(1946 Wichita sectional) So one foggy morning, I explained my puzzlement to the boys. I highlighted all the mysteries: were these maps correct? Were there really two Newton airports at one time? How many airports were there, and where were they? Why did they move? What was the story behind them? And I offered them the chance to be history detectives with me. And oh my goodness, were they ever excited! We had some information from a very helpful person at the Harvey County Historical Museum (thanks Kris!) So we suspected one airport at least was established in 1927. We also had a description of its location, though given in terms of township maps. So the boys and I made the short drive over to the museum. We reviewed their property maps, though they were all a little older than the time period we needed. We looked through books and at pictures. Oliver pored over a railroad map of Newton from a century ago, fascinated. Jacob was excited to discover on one map that there used to be a train track down the middle of Main Street! I was interested that the present Newton Airport was once known as Wirt Field, rather to my surprise. I somehow suspect most 2nd and 4th graders spend a lot less excited time on their research floor! Then on to the Newton Public Library to see if they d have anything more and that s when the map that produced all the excitement came out. It, by itself, didn t answer the question, but by piecing together a number of pieces of information newspaper stories, information from the museum, and the maps we were able to come up with a pretty good explanation, much to their excitement. Apparently, a man named Tangeman owned a golf course (the golf links according to the paper), and around 1927 the city of Newton purchased it, because of all the planes that were landing there. They turned it into a real airport. Later, they bought land east of the city and moved the airport there. However, during World War II, the Navy took over that location, so they built a third airport a few miles west of the city but moved back to the current east location after the Navy returned that field to them. Of course, a project like this just opens up all sorts of extra questions: why isn t it called Wirt Field anymore? What s the story of Frank Wirt? What led the Navy to take over Newton s airport? Why did planes start landing on the golf course? Where precisely was the west airport located? How long was it there? (I found an aerial photo from 1956 that looks like it may have a plane in that general area, but it seems later than I d have expected) So now I have the boys interested in going to the courthouse with me to research the property records out there. Jacob is continually astounded that we are discovering things that aren t in Wikipedia, and also excited that he could be the one to add them. To be continued, apparently!

8 December 2016

Alessio Treglia: The new professionals of the interconnected world

interdisciplinary-learning There is an empty chair at the conference table of business professionals, a not assigned place that increasingly demands for the presence of a new type of integration manager. The demands for an ever-increasing specialization, imposed by the modern world, are bringing out with great emphasis the need for an interdisciplinary professional who understands the demands of specialists and who is able to coordinate and to link actions and decisions. This need, often still ignored, is a direct result of the growing complexity of the modern world and the fast communications inside the network.

Complexity is undoubtedly the most suitable paradigm to characterize the historical and social model of today s world, in which the interactions and connections between the various areas now form an inextricable network of relations. Since the 60s and 70s a large group of scholars including the chemist Ilya Prigogine and the physicist Murray Gell-Mann began to study what would become a true Science of Complexity.

Yet this is not an entirely new concept: the term means composed of several parts connected to each other and dependent on each other , exactly as reality, nature, society, and the environment around us. A complex mode of thought integrates and considers all contexts, interconnections, interrelationships between the different realities as part of the vision.

What is professionalism? And who are professionals? What can define a professional? < > <Read More [by Fabio Marzocca]>

Dirk Eddelbuettel: RcppAPT 0.0.3

A new version of RcppAPT -- our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, ... commands and their cache powering Debian, Ubuntu and the like -- is now on CRAN. We changed the package to require C++11 compilation as newer Debian systems with g++-6 and the current libapt-pkg-dev library cannot build under the C++98 standard which CRAN imposes (and let's not get into why ...). Once set to C++11 we have no issues. We also added more examples to the manual pages, and turned on code coverage. A bit more information about the package is available here as well as as the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

7 December 2016

Shirish Agarwal: Day trip in Cape Town, part 2

The post continues from the last post shared. Let me get some interesting tit-bits not related to the day-trip out-of-the-way first I don t know whether we had full access to see all parts of fuller hall or not. Couple of days I was wondering around Fuller Hall, specifically next to where clothes were pressed. Came to know of the laundry service pretty late but still was useful. Umm next to where the ladies/gentleman pressed our clothes, there is a stairway which goes down. In fact even on the opposite side there is a stairway which goes down. I dunno if other people explored them or not. The jail inside and under UCT

I was surprised and shocked to see bars in each room as well as connecting walkways etc. I felt a bit sad, confused and curious and went on to find more places like that. After a while I came up to the ground-level and enquired with some of the ladies therein. I was shocked to know that UCT some years ago (they were not specific) was a jail for people. I couldn t imagine that a place which has so much warmth (in people, not climate) could be evil in a sense. I was not able to get much information out of them about the nature of jail it was, maybe it is a dark past that nobody wants to open up, dunno. There were also two *important* aspects of UCT which Bernelle either forgot, didn t share or I just came to know via the Wikipedia page then but nothing else. 1. MeerKAT Apparently quite a bit of the technology was built-in UCT itself. This would have been interesting for geeks and wanna-be geeks like me

2. The OpenContent Initiative by UCT This would have been also something worth exploring. One more interesting thing which I saw was the French council in Cape Town from outside

The French Council in cape town from outside

I would urge to look at the picture in the gallery as the picture I shared doesn t really show all the details. For e.g. the typical large french windows which are the hall-mark of French architecture doesn t show its glory but if you look at 1306 2322 original picture instead of the 202 360 reproduction you will see that. You will also the insignia of the French Imperial Eagle whose history I came to know only after I looked it up on the Wikipedia page on that day. It seemed fascinating and probably would have the same pride as the State Emblem of India has for Indians with the four Asiatic Lions standing in a circle protecting each other. I also like the palm tree and the way the French Council seemed little and yet had character around all the big buildings. What also was interesting that there wasn t any scare/fear-build and we could take photos from outside unlike what I had seen and experienced in Doha, Qatar as far as photography near Western Embassies/Councils were concerned. One of the very eye-opening moments for me was also while I was researching flights from India to South Africa. While perhaps unconsciously I might have known that Middle East is close to India, in reality, it was only during the search I became aware that most places in Middle East by flight are only an hour or two away. This was shocking as there is virtually no mention of one of our neighbours when they are source of large-scale remittances every year. I mean this should have been in our history and geography books but most do not dwell on the subject. It was only during and after that I could understand Mr. Modi s interactions and trade policies with the Middle East. Another interesting bit was seeing a bar in a Sprinbok bus spingbok atlas bar in bus

While admittedly it is not the best picture of the bar, I was surprised to find a bar at the back of a bus. By bar I mean a machine which can serve anything from juices to alcoholic drinks depending upon what is stocked. What was also interesting in the same bus is that the bus also had a middle entrance-and-exit. The middle door in springbok atlas

This is something I hadn t seen in most Indian buses. Some of the Volvo buses have but it is rarely used (only except emergencies) . An exhaustive showcase of local buses can be seen here . I find the hand-drawn/cad depictions of all the buses by Amit Pense near to the T. Axe which can be used to break windows

This is also something which I have not observed in Indian inter-city buses (axe to break the window in case of accident and breakable glass which doesn t hurt anyone I presume), whether they are State-Transport or the high-end Volvo s . Either it s part of South African Roads Regulations or something that Springbok buses do for their customers. All of these queries about the different facets I wanted to ask the bus-driver and the attendant/controller but in the excitement of seeing, recording new things couldn t ask

In fact one of the more interesting things I looked at and could look day and night is the variety of vehicles on display in Cape Town. In hindsight, I should have bought a couple of 128 GB MMC cards for my mobile rather than the 64 GB one. It was just plain inadequate to capture all that was new and interesting. Auditorum chair truck seen near Auditorium

Auditorum chair truck seen near Auditorium

This truck I had seen about some 100 metres near the Auditorium on Upper Campus. The truck s design, paint was something I had never seen before. It is/was similar to casket trucks seen in movies but the way it was painted and everything made it special. What was interesting is to see the gamut of different vehicles. For instance, there were no bicycles that I saw in most places. There were mostly Japanese/Italian bikes and all sorts of trucks. If I had known before, I would definitely have bought an SD specifically to take snaps of all the different types of trucks, cars etc. that I saw therein. The adage/phrase I should stop in any one place and the whole world will pass me by seemed true on quite a few South African Roads. While the roads were on par or a shade better than India, many of those were wide roads. Seeing those, I was left imagining how the Autobahn in Germany and other high-speed Expressways would look n feel. India has also been doing that with the Pune-Mumbai Expressway and projects like Yamuna Expressway and now the extension Agra Lucknow Expressway but doing this all over India would take probably a decade or more. We have been doing it since a decade and a half. NHDP and PMGSY are two projects which are still ongoing to better the roads. We have been having issues as to should we have toll or no toll issues but that is a discussion for some other time. One of the more interesting sights I saw was the high-arched gothic-styled church from outside. This is near Longstreet as well. high arch gothic-styled church

I have seen something similar in Goa, Pondicherry but not such high-arches. I did try couple of times to gain entry but one time it was closed, the other time some repairing/construction work was going on or something. I would loved to see it from inside and hopefully they would have had an organ (music) as well. I could imagine to some extent the sort of music that would have come out. Now that Goa has come in the conversation I can t help but state that Seafood enthusiasts/lover/aficionado, or/and Pescatarianism would have a ball of a time in Goa. Goa is on the Konkan coast and while I m eggie, ones who enjoy seafood really have a ball of a time in Goa. Fouthama s Festival which happens in February is particularly attractive as Goan homes are thrown open for people to come and sample their food, exchange recipes and alike. This happens around 2 weeks before the Goan Carnival and is very much a part of the mish-mashed Konkani-Bengali-Parsi-Portugese culture. I better stop here about the Goa otherwise I ll get into reminiscing mode. To put the story and event back on track from where we left of (no fiction hereon), Nicholas was in constant communication with base, i.e. UCT as well as another group who was hiking from UCT to Table Mountain. We waited for the other group to join us till 13:00 hrs. We came to know that they were lost and were trying to come up and hence would take more time. As Bernelle was with them, who was a local and she had two dogs who knew the hills quite well, it was decided to go ahead without them. We came down the same cable-car and then ventured on towards Houtbay. Houtbay has it all, a fisherman s wharf, actual boats with tough-mean looking men with tattoos working on boats puffing cigars/pipes, gaggle of sea-gulls, the whole scene. Sharing a few pictures of the way in-between. the view en-route to Houtbay

I just now had a quick look at the restaurant and it seems they had options for veggies too. Unfortunately, the rating leaves a bit to be desired but then dunno as Indian flavoring is something that takes time to get used too. Zomato doesn t give any idea of from when a restaurant is in business and has too few reviews so not easy to know how the experience would have been. Chinese noodles and small houses

Notice the pattern, the pattern of small houses I saw all the way till Houtbay and back. I do vaguely remember starting a discussion about it on the bus but don t really remember. I have seen (on TV) cities like Miami, Dubai or/and Hong Kong who have big buildings on the beach but both in Konkan as well as Houtbay there were small buildings. I guess a combination of zoning regulations, feel of community, fear of being flooded all play into beaches being the way they are. Also, this probably is good as less stress on the environment. Miamiboyz from Wikimedia Commons

The above picture is taken from Wikipedia from the article Miami Beach, Florida for comparison. Audi rare car to be seen in India

The Audi rare car to be seen in India. This car has been associated with Ravi Shastri when he won it in 1985. I was young but still get goosebumps remembering those days. first-glance-Houtbay-and-pier

First glance of Houtbay beach and pier. Notice how clean and white the beach is. Wharf-Grill-Restaurant-from-side-and-Hop-on-Hop-off-bus

You can see the wharf grill restaurant in the distance (side-view), see the back of the hop on and hop off bus (a concept which was unknown to me till then). Once I came back and explored on the web came to know this concept is prevalent in many a touristy places around the world. Umm also By sheer happenchance also captured a beautiful looking Indian female

In Hindi, we would call this picture virodabhas or contradiction . this is in afternoon, around 1430 hrs. You have the sun, the clouds, the Mountains, the x number of boats, the pier, the houses, the cars, the shops. It was all crazy and beautiful at the same time. The Biggest Contradiction is seeing the Mountain, the beach and the Sea in the same Picture. Baffled the mind. Konkan though is a bit similar there as well. You have all the three things in some places but that s a different experience altogether as ours is a more tropical weather although is one of the most romantic places in the rains. We were supposed to go on a short cruise to seal/dolphin island but as we were late (as had been waiting for the other group) didn t go and instead just loitered there. Fake-real lookout bar-restaurant

IIRC the lookout bar is situated just next to Houtbay Search and Rescue. Although was curious if the Lookout tower was used in case of disappearance. lost people, boats etc. Seal in action

Seal jumping over water, what a miracle ! One of the boats on which we possibly could have been on.

It looked like the boat we could have been on. I clicked as I especially liked the name Calypso and Calypso . I shared the two links as the mythologies, interpretation differ a bit between Greek and Hollywood culture

Can see few Debian folks in the foreground, next to the Pole and the area around. Also can see a bit of the area around. Alone boy trying to surf

I don t know anything about water sports and after sometime he came out. I was left wondering though, how safe he was in that water. While he was close to the pier and he was just paddling, there weren t big waves still felt a bit of concern. Mr. Seal - the actor and his handler

While the act was not to the level we see in the movies, still for the time I hung around, I saw him showing attitude for his younger audiences, eating out of their hands, making funny sounds. Btw he farted a few times, whether that was a put-on or not can t really say but produced a few guffaws from his audience. A family feeding Mr. Seal

I dunno why the birds came down for. Mr. Seal was being fed oily small fish parts, dunno if the oil was secreted by the fish themselves or whatever, it just looked oily from distance. Bird-Man-Bird

typical equipment on a boat to catch fish-lot of nets

There wasn t much activity on the time we went. It probably would have been different on sunrise and would be on sunset. The only activity I saw was on this boat where they were busy fixing and disentangling the lines. I came up with 5-15 different ideas for a story but rejected them as a. Probably all of them have been tried. People have been fishing since the beginning of time and modern fishing probably 200 odd years or so. I have read accounts of fishing companies in early 1800s onwards, so probably all must have been tried. b. More dangerous one, if there is a unique idea, then it becomes more dangerous as writing is an all-consuming process. Writing a blog post (bad or good) takes lots of time. I constantly read, re-read, try and improvise till I can or my patience loses out. In book you simply can t have such luxuries. hout-bay-search-and-rescue-no-parking-zone

hout-bay-search-and-rescue-no-parking-zone

No parking/tow zone in/near the Houtbay search and rescue. Probably to take out emergency vehicles once something untoward happens. hout-bay-sea-rescue-with-stats

Saved 54 lives, boats towed 154 Salut! Houtbay sea rescue. The different springbok atlas bus that we were on

The only small criticism is for Houtbay there wasn t a single public toilet. We had to ask favor at kraal kraft to use their toilets and there could have been accidents, it wasn t lighted well and water was spilled around. Road sign telling that we are near to UCT

Road sign telling that we are near to UCT

For us, because we were late we missed both the boat-cruise as well as some street shops selling trinkets. Other than that it was all well. We should have stayed till sunset, I am sure the view would have been breath-taking but we hadn t booked the bus till evening. Back at UCT

Overall it was an interesting day as we had explored part of Table Mountain, seen the somewhat outrageously priced trinkets there as well as explored Houtbay sea-side as well.
Filed under: Miscellenous Tagged: #Audi, #Cape Town, #Cruises, #Debconf16, #French Council, #Geography, #Houtbay Sea Rescue, #Jail, #Middle East, #Springbok Atlas, #Vehicles

Jonas Meurer: On CVE-2016-4484, a (securiy)? bug in the cryptsetup initramfs integration

On CVE-2016-4484, a (security)? bug in the cryptsetup initramfs integration On November 4, I was made aware of a security vulnerability in the integration of cryptsetup into initramfs. The vulnerability was discovered by security researchers Hector Marco and Ismael Ripoll of CyberSecurity UPV Research Group and got CVE-2016-4484 assigned. In this post I'll try to reflect a bit on

the nature of the vulnerability

the way it was published and discussed by technology news press

What CVE-2016-4484 is all about Basically, the vulnerability is about two separate but related issues:

1. Initramfs rescue shell considered harmful The main topic that Hector Marco and Ismael Ripoll address in their publication is that Debian exits into a rescue shell in case of failure during initramfs, and that this can be triggered by entering a wrong password ~93 times in a row. Indeed the Debian initramfs implementation as provided by initramfs-tools exits into a rescue shell (usually a busybox shell) after a defined amount of failed attempts to make the root filesystem available. The loop in question is in local_device_setup() at the local initramfs script In general, this behaviour is considered as a feature: if the root device hasn't shown up after 30 rounds, the rescue shell is spawned to provide the local user/admin a way to debug and fix things herself. Hector Marco and Ismael Ripoll argue that in special environments, e.g. on public computers with password protected BIOS/UEFI and bootloader, this opens an attack vector and needs to be regarded as a security vulnerability:

It is common to assume that once the attacker has physical access to the computer, the game is over. The attackers can do whatever they want. And although this was true 30 years ago, today it is not. There are many "levels" of physical access. [...] In order to protect the computer in these scenarios: the BIOS/UEFI has one or two passwords to protect the booting or the configuration menu; the GRUB also has the possibility to use multiple passwords to protect unauthorized operations. And in the case of an encrypted system, the initrd shall block the maximum number of password trials and prevent the access to the computer in that case.

While Hector and Ismael have a valid point in that the rescue shell might open an additional attack vector in special setups, this is not true for the vast majority of Debian systems out there: in most cases a local attacker can alter the boot order, replace or add boot devices, modify boot options in the (GNU GRUB) bootloader menu or modify/replace arbitrary hardware parts. The required scenario to make the initramfs rescue shell an additional attack vector is indeed very special: locked down hardware, password protected BIOS and bootloader but still local keyboard (or serial console) access are required at least. Hector and Ismael argue that the default should be changed for enhanced security:

[...] But then Linux is used in more hostile environments, this helpful (but naive) recovery services shall not be the default option.

For the reasons explained about, I tend to disagree to Hectors and Ismaels opinion here. And after discussing this topic with several people I find my opinion reconfirmed: the Debian Security Team disputes the security impact of the issue and others agree. But leaving the disputable opinion on a sane default aside, I don't think that the cryptsetup package is the right place to change the default, if at all. If you want added security by a locked down initramfs (i.e. no rescue shell spawned), then at least the bootloader (GNU GRUB) needs to be locked down by default as well. To make it clear: if one wants to lock down the boot process, bootloader and initramfs should be locked down together. And the right place to do this would be the configurable behaviour of grub-mkconfig. Here, one can set a password for GRUB and the boot parameter 'panic=1' which disables the spawning of a rescue shell in initramfs. But as mentioned, I don't agree that this would be sane defaults. The vast majority of Debian systems out there don't have any security added by locked down bootloader and initramfs and the benefit of a rescue shell for debugging purposes clearly outrivals the minor security impact in my opinion. For the few setups which require the added security of a locked down bootloader and initramfs, we already have the relevant options documented in the Securing Debian Manual:

After discussing the topic with initramfs-tools maintainers today, Guilhem and me (the cryptsetup maintainers) finally decided to not change any defaults and just add a 'sleep 60' after the maximum allowed attempts were reached. 2. tries=n option ignored, local brute-force slightly cheaper Apart from the issue of a rescue shell being spawned, Hector and Ismael also discovered a programming bug in the cryptsetup initramfs integration. This bug in the cryptroot initramfs local-top script allowed endless retries of passphrase input, ignoring the tries=n option of crypttab (and the default of 3). As a result, theoretically unlimited attempts to unlock encrypted disks were possible when processed during initramfs stage. The attack vector here was that local brute-force attacks are a bit cheaper. Instead of having to reboot after max tries were reached, one could go on trying passwords. Even though efficient brute-force attacks are mitigated by the PBKDF2 implementation in cryptsetup, this clearly is a real bug. The reason for the bug was twofold:

First, the condition in setup_mapping() responsible for making the function fail when the maximum amount of allowed attempts is reached, was never met:
```
setup_mapping()
 
  [...]
  # Try to get a satisfactory password $crypttries times
  count=0                              

  while [ $crypttries -le 0 ]   [ $count -lt $crypttries ]; do
      export CRYPTTAB_TRIED="$count"
      count=$(( $count + 1 ))
      [...]
  done
  if [ $crypttries -gt 0 ] && [ $count -gt $crypttries ]; then
      message "cryptsetup: maximum number of tries exceeded for $crypttarget"
      return 1
  fi
  [...]
 
```
As one can see, the while loop stops when $count -lt $crypttries. Thus the second condition $count -gt $crypttries is never met. This can easily be fixed by decreasing $count by one in case of a successful unlock attempt along with changing the second condition to $count -ge $crypttries:
```
setup_mapping()
 
  [...]
  while [ $crypttries -le 0 ]   [ $count -lt $crypttries ]; do
      [...]
      # decrease $count by 1, apparently last try was successful.
      count=$(( $count - 1 ))
      [...]
  done
  if [ $crypttries -gt 0 ] && [ $count -ge $crypttries ]; then
      [...]
  fi
  [...]
 
```
Christian Lamparter already spotted this bug back in October 2011 and provided a (incomplete) patch, but back then I even managed to merge the patch in an improper way, making it even more useless: The patch by Christian forgot to decrease $count by one in case of a successful unlock attempt, resulting in warnings about maximum tries exceeded even for successful attemps in some circumstances. But instead of adding the decrease myself and keeping the (almost correct) condition $count -eq $crypttries for detection of exceeded maximum tries, I changed back the condition to the wrong original $count -gt $crypttries that again was never met. Apparently I didn't test the fix properly back then. I definitely should do better in future!
Second, back in December 2013, I added a cryptroot initramfs local-block script as suggested by Goswin von Brederlow in order to fix bug #678692. The purpose of the cryptroot initramfs local-block script is to invoke the cryptroot initramfs local-top script again and again in a loop. This is required to support complex block device stacks. In fact, the numberless options of stacked block devices are one of the biggest and most inglorious reasons that the cryptsetup initramfs integration scripts became so complex over the years. After all we need to support setups like rootfs on top of LVM with two separate encrypted PVs or rootfs on top of LVM on top of dm-crypt on top of MD raid. The problem with the local-block script is that exiting the setup_mapping() function merely triggers a new invocation of the very same function. The guys who discovered the bug suggested a simple and good solution to this bug: When maximum attempts are detected (by second condition from above), the script sleeps for 60 seconds. This mitigates the brute-force attack options for local attackers - even rebooting after max attempts should be faster.

About disclosure, wording and clickbaiting I'm happy that Hector and Ismael brought up the topic and made their argument about the security impacts of an initramfs rescue shell, even though I have to admit that I was rather astonished about the fact that they got a CVE assigned. Nevertheless I'm very happy that they informed the Security Teams of Debian and Ubuntu prior to publishing their findings, which put me in the loop in turn. Also Hector and Ismael were open and responsive when it came to discussing their proposed fixes. But unfortunately the way they advertised their finding was not very helpful. They announced a speech about this topic at the DeepSec 2016 in Vienna with the headline Abusing LUKS to Hack the System. Honestly, this headline is missleading - if not wrong - in several ways:

First, the whole issue is not about LUKS, neither is it about cryptsetup itself. It's about Debians integration of cryptsetup into the initramfs, which is a compeletely different story.

Second, the term hack the system suggests that an exploit to break into the system is revealed. This is not true. The device encryption is not endangered at all.

Third - as shown above - very special prerequisites need to be met in order to make the mere existance of a LUKS encrypted device the relevant fact to be able to spawn a rescue shell during initramfs.

Unfortunately, the way this issue was published lead to even worse articles in the tech news press. Topics like Major security hole found in Cryptsetup script for LUKS disk encryption or Linux Flaw allows Root Shell During Boot-Up for LUKS Disk-Encrypted Systems suggest that a major security vulnerabilty was revealed and that it compromised the protection that cryptsetup respective LUKS offer. If these articles/news did anything at all, then it was causing damage to the cryptsetup project, which is not affected by the whole issue at all. After the cat was out of the bag, Marco and Ismael aggreed that the way the news picked up the issue was suboptimal, but I cannot fight the feeling that the over-exaggeration was partly intended and that clickbaiting is taking place here. That's a bit sad.

Links

Hector Marco & Ismael Ripoll: CVE-2016-4484: Cryptsetup Initrd root Shell

Public announcement by Hector Marco of CVE-2014-4484 on oss-security

Discussion about CVE-2016-4484 on dm-crypt@saout.de

5 December 2016

Shirish Agarwal: The Anti-Pollito squad arrest and confession

Disclaimer This is an attempt at humor and hence entirely fictional in nature. While some incidents depicted are true, the context and the story woven around them are by yours truly. None of the Mascots of Debian were hurt during the blog post

. I also disavow any responsibility for any hurt (real or imagined) to any past, current and future mascots. The attempt should not be looked upon as demeaning people who are accused of false crimes, tortured and confessions eked out of them as this happens quite a lot (In India for sure, but guess it s the same world over in various degrees). The idea is loosely inspired by Chocolate:Deep Dark Secrets. (2005) On a more positive note, let s start Being a Sunday morning woke up late to find incessant knocking on the door, incidentally mum was not at home. Opening the door, found two official looking gentleman. They asked my name, asked my credentials, tortured and arrested me for Group conspiracy of Malicious Mischief in second and third degrees . The torture was done by means of making me forcefully watch endless reruns of Norbit . While I do love Eddie Murphy, this was one of his movies he could have done without

. I guess for many people watching it once was torture enough. I *think* they were nominated for razzie awards dunno if they won it or not, but this is beside the point. Unlike the 20 years it takes for a typical case to reach to its conclusion even in the smallest court in India, due to the torture, I was made to confess (due to endless torture) and was given summary judgement. The judgement was/is as follows a. Do 100 hours of Community service in Debian in 2017. This could be done via blog posts, raising tickets in the Debian BTS or in whichever way I could be helpful to Debian. b. Write a confessional with some photographic evidence sharing/detailing some of the other members who were part of the conspiracy in view of the reduced sentence. So now, have been forced to write this confession As you all know, I won a bursary this year for debconf16. What is not known by most people is that I also got an innocuous looking e-mail titled Pollito for DPL . While I can t name all the names as investigation is still ongoing about how far-reaching the conspiracy is . The email was purportedly written by members of cabal within cabal which are in Debian. I looked at the email header to see if this was genuine and I could trace the origin but was left none the wiser, as obviously these people are far more technically advanced than to fall in simple tricks like this Anyways, secretly happy that I have been invited to be part of these elites, I did the visa thing, packed my bags and came to Debconf16. At this point in juncture, I had no idea whether it was real or I had imagined the whole thing. Then to my surprise saw this evidence of conspiracy to have Pollito as DPL, Wifi Password

evidence of conspiracy to have Pollito as DPL, Wifi Password

Just like the Illuminati the conspiracy was for all to see those who knew about it. Most people were thinking of it as a joke, but those like me who had got e-mails knew better. I knew that the thing is real, now I only needed to bide my time and knew that the opportunity would present itself. And few days later, sure enough, there was a trip planned for Table Mountain, Cape Town . Few people planned to hike to the mountain, while few chose to take the cable car till up the mountain. First glance of the cable car with table mountain as background

First glance of the cable car with table mountain as background

Quite a few people came along with us and bought tickets for the to and fro to the mountain and back. Ticket for CPT Table mountain car cable

Incidentally, I was thinking if the South African Govt. were getting the tax or not. If you look at the ticket, there is just a bar-code. In India as well as the U.S. there is TIN Tax Identification Number TIN displayed on an invoice from channeltimes.com

TIN displayed on an invoice from channeltimes.com

Few links to share what it is all about . While these should be on all invoices, need to specially check when taking high-value items. In India as shared in the article the awareness, knowledge leaves a bit to be desired. While I m drifting from the incident, it would be nice if somebody from SA could share how things work there. Moving on, we boarded the cable car. It was quite spacious cable car with I guess around 30-40 people or some more who were able to see everything along with the controller. from inside the table mountain cable car 360 degrees

from inside the table mountain cable car 360 degrees

It was a pleasant cacophony of almost two dozen or more nationalities on this 360 degrees moving chamber. I was a little worried though as it essentially is a bucket and there is always a possibility that a severe wind could damage it. Later somebody did share that some frightful incidents had occurred not too long ago on the cable car. It took about 20-25 odd minutes to get to the top of table mountain and we were presented with views such as below View from Table Mountain cable car looking down

View from Table Mountain cable car looking down

The picture I am sharing is actually when we were going down as all the pictures of going up via the cable car were over-exposed. Also, it was pretty crowded on the way up then on the way down so handling the mobile camera was not so comfortable. Once we reached up, the wind was blowing at incredible speeds. Even with my jacket and everything I was feeling cold. Most of the group around 10-12 people looked around if we could find a place to have some refreshments and get some of the energy in the body. So we all ventured to a place and placed our orders the bleh... Irish coffee at top of Table Mountain

the bleh... Irish coffee at top of Table Mountain

I was introduced to Irish Coffee few years back and have had some incredible Irish Coffees in Pune and elsewhere. I do hope to be able to make Irish Coffee at home if and when I have my own house. This is hotter than brandy and is perfect if you are suffering from cold etc if done right, really needs some skills. This is the only drink which I wanted in SA which I never got right

. As South Africa was freezing for me, this would have been the perfect antidote but the one there as well as elsewhere were all bleh. What was interesting though, was the coffee caller besides it. It looked like a simple circuit mounted on a PCB board with lights, vibrations and RFID and it worked exactly like that. I am guessing as and when the order is ready, there is an interrupt signal sent via radio waves which causes the buzzer to light and vibrate. Here s the back panel if somebody wants to take inspiration and try it as a fun project backpanel of the buzz caller

Once we were somewhat strengthened by the snacks, chai, coffee etc. we made our move to seeing the mountain. The only way to describe it is that it s similar to Raigad Fort but the plateau seemed to be bigger. The wikipedia page of Table Mountain attempts to share but I guess it s more clearly envisioned by one of the pictures shared therein. table mountain panaromic image

I have to say while Table Mountain is beautiful and haunting as it has scenes like these Some of the oldest rocks known to wo/man.

There is something there which pulls you, which reminds you of a long lost past. I could have simply sat there for hours together but as was part of the group had to keep with them. Not that I minded. The moment I was watching this, I was transported to some memories of the Himalayas about 20 odd years or so. In that previous life, I had the opportunity to be with some of the most beautiful women and also been in the most happening places, the Himalayas. I had shared years before some of my experiences I had in the Himalayas. I discontinued it as I didn t have a decent camera at that point in time. While I don t wanna digress, I would challenge anybody to experience the Himalayas and then compare. It is just something inexplicable. The beauty and the rawness that Himalayas shows makes you feel insignificant and yet part of the whole cosmos. What Paulo Cohello expressed in The Valkyries is something that could be felt in the Himalayas. Leh, Ladakh, Himachal , Garwhal, Kumaon. The list will go on forever as there are so many places, each more beautiful than the other. Most places are also extremely backpacker-friendly so if you ask around you can get some awesome deals if you want to spend more than a few days in one place. Moving on, while making small talk @olasd or Nicolas Dandrimont , the headmaster of our trip made small talk to each of us and eked out from all of us that we wanted to have Pollito as our DPL (Debian Project Leader) for 2017. Few pictures being shared below as supporting evidence as well The Pollito as DPL cabal in action

where am I or more precisely how far am I from India.

While I do not know who further up than Nicolas was on the coup which would take place. The idea was this If the current DPL steps down, we would take all and any necessary actions to make Pollito our DPL. Pollito going to SA - photo taken by Jonathan Carter

Pollito going to SA - photo taken by Jonathan Carter

This has been taken from Pollito s adventure Being a responsible journalist, I also enquired about Pollito s true history as it would not have been complete without one. This is the e-mail I got from Gunnar Wolf, a friend and DD from Mexico

Turns out, Valessio has just spent a week staying at my house And
in any case, if somebody in Debian knows about Pollito s
childhood That is me. Pollito came to our lives when we went to Congreso Internacional de
Software Libre (CISOL) in Zacatecas city. I was strolling around the
very beautiful city with my wife Regina and our friend Alejandro
Miranda, and at a shop at either Ram n L pez Velarde or Vicente
Guerrero, we found a flock of pollitos. http://www.openstreetmap.org/#map=17/22.77111/-102.57145 Even if this was comparable to a slave market, we bought one from
them, and adopted it as our own. Back then, we were a young couple Well, we were not that young
anymore. I mean, we didn t have children. Anyway, we took Pollito with
us on several road trips, such as the only time I have crossed an
international border driving: We went to Encuentro Centroamericano de
Software Libre at Guatemala city in 2012 (again with Alejandro), and
you can see several Pollito pics at: http://gwolf.org/album/road-trip-ecsl-2012-guatemala-0 Pollito likes travelling. Of course, when we were to Nicaragua for
DebConf, Pollito tagged along. It was his first flight as a passenger
(we never asked about his previous life in slavery; remember, Pollito
trust no one). Pollito felt much welcome with the DebConf crowd. Of course, as
Pollito is a free spirit, we never even thought about forcing him to
come back with us. Pollito went to Switzerland, and we agreed to meet
again every year or two. It s always nice to have a chat with him. Hugs!

So with that backdrop I would urge fellow Debianities to take up the slogans LONG LIVE THE DPL ! LONG LIVE POLLITO ! LONG LIVE POLLITO THE DPL ! The first step to make Pollito the DPL is to ensure he has a @debian.org (pollito@debian.org) We also need him to be made a DD because only then can he become a DPL. In solidarity and in peace

Filed under: Miscellenous Tagged: #caller, #confession, #Debconf16, #debian, #Fiction, #history, #Pollito, #Pollito as DPL, #Table Mountain, Cabal, memories, south africa

Markus Koschany: My Free Software Activities in November 2016

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Android

Chris Lamb was so kind to send in a patch for apktool to make the build reproducible (#845475). Although this was not enough to fix the issue it set me on the right path to eventually resolve bug number 845475.

Debian Games

I packaged a couple of new upstream releases for extremetuxracer, fifechan, fife, unknown-horizons, freeciv, atanks and armagetronad. Most notably fifechan was accepted by the FTP team which allowed me to package new versions of fife and unknown-horizons which are both back in testing again. I expect that upstream will make their final release sometime in December. Atanks has been orphaned a while ago and since upstream is still active and I kinda like the game I decided to adopt it. I also uploaded a backport of Freeciv 2.5.6 to jessie-backports.
In November we received a bunch of RC bug reports again because, hey, it is almost time for the Freeze, let s break some packages. Thus I spent some time fixing freeorion (#843132), pokerth (#843078), simutrans (#828545), freeciv (#844198) and warzone2100 (#844870).
I also updated the debian-games blend, we are at version 1.6 now, and made some smaller adjustments. The most important change was adding a new binary package, games-all, that installs..well, all! I know this will make at least one person on this planet happy. Actually I was kind of forced into adding it because blends-dev automatically creates it as a requirement for choosing blends with the Debian Installer. But don t be afraid games-all only recommends games-finest, the rest is suggested.
Last but not least I worked on performous and could close a wishlist bug report (#425898). The submitter asked to suggest some free song packages for this karaoke game.

Debian Java

I sponsored uncommons-watchmaker for Kai-Chung and also reviewed libnative-platform-java and granted upload rights to him.
I packaged new upstream releases of lombok-patcher, electric, undertow, sweethome3d and sweethome3d-furniture-editor.
I spent quite some time on reviewing (especially the copyright review took most of the time) and improving the packaging for tycho (#816604) which is a precondition for packaging the latest upstream release of Eclipse, a popular Java IDE. Luca Vercelli has been working on it for the last couple of months and he did most of the initial packaging. Unfortunately I was only able to upload the package last week which means that the chances for updating Eclipse for Stretch are slim.
Due to time constraints I could not finish the Netbeans update in time which I had started back in October. This is on my priority list for December now.
Several security issues were reported against Tomcat 6,7,8 . I helped with reviewing some of the patches that Emmanuel prepared for Jessie and worked on fixing the same bugs in Wheezy.

Debian LTS This was my ninth month as a paid contributor and I have been paid to work 11 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

From 14. November until 21. November I was in charge of our LTS frontdesk. I triaged bugs in teeworlds, libdbd-mysql-perl, bash, libxml2, tiff, firefox-esr, drupal7, moin, libgc, w3m and sniffit.
DLA-715-1. Issued a security update for drupal7 fixing 2 CVE.
DLA-717-1. Issued a security update for moin fixing 2 CVE.
DLA-728-1. Issued a security update for tomcat6 fixing 8 CVE. (Debian bug #845385 was assigned a CVE later).
DLA-729-1. Issued a security update for tomcat7 fixing 8 CVE. (Debian bug #845385 was assigned a CVE later).
Especially the patches and the subsequent testing for CVE-2016-0762 and CVE-2016-6816 required most of the time.

Non-maintainer uploads

I uploaded an NMU for angband to fix #837394. The patch was kindly prepared by Adrian Bunk.

It is already this time of the year again. See you next year for another report.

3 December 2016

Vincent Bernat: Build-time dependency patching for Android

This post shows how to patch an external dependency for an Android project at build-time with Gradle. This leverages the Transform API and Javassist, a Java bytecode manipulation tool.

buildscript  
    dependencies  
        classpath 'com.android.tools.build:gradle:2.2.+'
        classpath 'com.android.tools.build:transform-api:1.5.+'
        classpath 'org.javassist:javassist:3.21.+'
        classpath 'commons-io:commons-io:2.4'

Disclaimer: I am not a seasoned Android programmer, so take this with a grain of salt.

Context This section adds some context to the example. Feel free to skip it. Dashkiosk is an application to manage dashboards on many displays. It provides an Android application you can install on one of those cheap Android sticks. Under the table, the application is an embedded webview backed by the Crosswalk Project web runtime which brings an up-to-date web engine, even for older versions of Android¹. Recently, a security vulnerability has been spotted in how invalid certificates were handled. When a certificate cannot be verified, the webview defers the decision to the host application by calling the `onReceivedSslError()` method:
Notify the host application that an SSL error occurred while loading a resource. The host application must call either `callback.onReceiveValue(true)` or `callback.onReceiveValue(false)`. Note that the decision may be retained for use in response to future SSL errors. The default behavior is to pop up a dialog.
The default behavior is specific to Crosswalk webview: the Android builtin one just cancels the load. Unfortunately, the fix applied by Crosswalk is different and, as a side effect, the `onReceivedSslError()` method is not invoked anymore². Dashkiosk comes with an option to ignore TLS errors³. The mentioned security fix breaks this feature. The following example will demonstrate how to patch Crosswalk to recover the previous behavior⁴.

Simple method replacement Let s replace the shouldDenyRequest() method from the org.xwalk.core.internal.SslUtil class with this version:

// In SslUtil class
public static boolean shouldDenyRequest(int error)  
    return false;

Transform registration Gradle Transform API enables the manipulation of compiled class files before they are converted to DEX files. To declare a transform and register it, include the following code in your build.gradle:

import com.android.build.api.transform.Context
import com.android.build.api.transform.QualifiedContent
import com.android.build.api.transform.Transform
import com.android.build.api.transform.TransformException
import com.android.build.api.transform.TransformInput
import com.android.build.api.transform.TransformOutputProvider
import org.gradle.api.logging.Logger
class PatchXWalkTransform extends Transform  
    Logger logger = null;
    public PatchXWalkTransform(Logger logger)  
        this.logger = logger
     
    @Override
    String getName()  
        return "PatchXWalk"
     
    @Override
    Set<QualifiedContent.ContentType> getInputTypes()  
        return Collections.singleton(QualifiedContent.DefaultContentType.CLASSES)
     
    @Override
    Set<QualifiedContent.Scope> getScopes()  
        return Collections.singleton(QualifiedContent.Scope.EXTERNAL_LIBRARIES)
     
    @Override
    boolean isIncremental()  
        return true
     
    @Override
    void transform(Context context,
                   Collection<TransformInput> inputs,
                   Collection<TransformInput> referencedInputs,
                   TransformOutputProvider outputProvider,
                   boolean isIncremental) throws IOException, TransformException, InterruptedException  
        // We should do something here
     
 
// Register the transform
android.registerTransform(new PatchXWalkTransform(logger))

The getInputTypes() method should return the set of types of data consumed by the transform. In our case, we want to transform classes. Another possibility is to transform resources. The getScopes() method should return a set of scopes for the transform. In our case, we are only interested by the external libraries. It s also possible to transform our own classes. The isIncremental() method returns true because we support incremental builds. The transform() method is expected to take all the provided inputs and copy them (with or without modifications) to the location supplied by the output provider. We didn t implement this method yet. This causes the removal of all external dependencies from the application.

Noop transform To keep all external dependencies unmodified, we must copy them:

@Override
void transform(Context context,
               Collection<TransformInput> inputs,
               Collection<TransformInput> referencedInputs,
               TransformOutputProvider outputProvider,
               boolean isIncremental) throws IOException, TransformException, InterruptedException  
    inputs.each  
        it.jarInputs.each  
            def jarName = it.name
            def src = it.getFile()
            def dest = outputProvider.getContentLocation(jarName, 
                                                         it.contentTypes, it.scopes,
                                                         Format.JAR);
            def status = it.getStatus()
            if (status == Status.REMOVED)   //  
                logger.info("Remove $ src ")
                FileUtils.delete(dest)
              else if (!isIncremental   status != Status.NOTCHANGED)   //  
                logger.info("Copy $ src ")
                FileUtils.copyFile(src, dest)

We also need two additional imports:

import com.android.build.api.transform.Status
import org.apache.commons.io.FileUtils

Since we are handling external dependencies, we only have to manage JAR files. Therefore, we only iterate on jarInputs and not on directoryInputs. There are two cases when handling incremental build: either the file has been removed ( ) or it has been modified ( ). In all other cases, we can safely assume the file is already correctly copied.

JAR patching When the external dependency is the Crosswalk JAR file, we also need to modify it. Here is the first part of the code (replacing ):

if ("$ src " ==~ ".*/org.xwalk/xwalk_core.*/classes.jar")  
    def pool = new ClassPool()
    pool.insertClassPath("$ src ")
    def ctc = pool.get('org.xwalk.core.internal.SslUtil') //  
    def ctm = ctc.getDeclaredMethod('shouldDenyRequest')
    ctc.removeMethod(ctm) //  
    ctc.addMethod(CtNewMethod.make("""
public static boolean shouldDenyRequest(int error)  
    return false;
 
""", ctc)) //  
    def sslUtilBytecode = ctc.toBytecode() //  
    // Write back the JAR file
    //  
  else  
    logger.info("Copy $ src ")
    FileUtils.copyFile(src, dest)

We also need the following additional imports to use Javassist:

import javassist.ClassPath
import javassist.ClassPool
import javassist.CtNewMethod

Once we have located the JAR file we want to modify, we add it to our classpath and retrieve the class we are interested in ( ). We locate the appropriate method and delete it ( ). Then, we add our custom method using the same name ( ). The whole operation is done in memory. We retrieve the bytecode of the modified class in . The remaining step is to rebuild the JAR file:

def input = new JarFile(src)
def output = new JarOutputStream(new FileOutputStream(dest))
//  
input.entries().each  
    if (!it.getName().equals("org/xwalk/core/internal/SslUtil.class"))  
        def s = input.getInputStream(it)
        output.putNextEntry(new JarEntry(it.getName()))
        IOUtils.copy(s, output)
        s.close()
     
 
//  
output.putNextEntry(new JarEntry("org/xwalk/core/internal/SslUtil.class"))
output.write(sslUtilBytecode)
output.close()

We need the following additional imports:

import java.util.jar.JarEntry
import java.util.jar.JarFile
import java.util.jar.JarOutputStream
import org.apache.commons.io.IOUtils

There are two steps. In , all classes are copied to the new JAR, except the SslUtil class. In , the modified bytecode for SslUtil is added to the JAR. That s all! You can view the complete example on GitHub.

More complex method replacement In the above example, the new method doesn t use any external dependency. Let s suppose we also want to replace the sslErrorFromNetErrorCode() method from the same class with the following one:

import org.chromium.net.NetError;
import android.net.http.SslCertificate;
import android.net.http.SslError;
// In SslUtil class
public static SslError sslErrorFromNetErrorCode(int error,
                                                SslCertificate cert,
                                                String url)  
    switch(error)  
        case NetError.ERR_CERT_COMMON_NAME_INVALID:
            return new SslError(SslError.SSL_IDMISMATCH, cert, url);
        case NetError.ERR_CERT_DATE_INVALID:
            return new SslError(SslError.SSL_DATE_INVALID, cert, url);
        case NetError.ERR_CERT_AUTHORITY_INVALID:
            return new SslError(SslError.SSL_UNTRUSTED, cert, url);
        default:
            break;
     
    return new SslError(SslError.SSL_INVALID, cert, url);

The major difference with the previous example is that we need to import some additional classes.

Android SDK import The classes from the Android SDK are not part of the external dependencies. They need to be imported separately. The full path of the JAR file is:

androidJar = "$ android.getSdkDirectory().getAbsolutePath() /platforms/" +
             "$ android.getCompileSdkVersion() /android.jar"

We need to load it before adding the new method into SslUtil class:

def pool = new ClassPool()
pool.insertClassPath(androidJar)
pool.insertClassPath("$ src ")
def ctc = pool.get('org.xwalk.core.internal.SslUtil')
def ctm = ctc.getDeclaredMethod('sslErrorFromNetErrorCode')
ctc.removeMethod(ctm)
pool.importPackage('android.net.http.SslCertificate');
pool.importPackage('android.net.http.SslError');
//

External dependency import We must also import org.chromium.net.NetError and therefore, we need to put the appropriate JAR in our classpath. The easiest way is to iterate through all the external dependencies and add them to the classpath.

def pool = new ClassPool()
pool.insertClassPath(androidJar)
inputs.each  
    it.jarInputs.each  
        def jarName = it.name
        def src = it.getFile()
        def status = it.getStatus()
        if (status != Status.REMOVED)  
            pool.insertClassPath("$ src ")
         
     
 
def ctc = pool.get('org.xwalk.core.internal.SslUtil')
def ctm = ctc.getDeclaredMethod('sslErrorFromNetErrorCode')
ctc.removeMethod(ctm)
pool.importPackage('android.net.http.SslCertificate');
pool.importPackage('android.net.http.SslError');
pool.importPackage('org.chromium.net.NetError');
ctc.addMethod(CtNewMethod.make(" "))
// Then, rebuild the JAR...

Happy hacking!

Before Android 4.4, the webview was severely outdated. Starting from Android 5, the webview is shipped as a separate component with updates. Embedding Crosswalk is still convenient as you know exactly which version you can rely on.
I hope to have this fixed in later versions.
This may seem harmful and you are right. However, if you have an internal CA, it is currently not possible to provide its own trust store to a webview. Moreover, the system trust store is not used either. You also may want to use TLS for authentication only with client certificates, a feature supported by Dashkiosk.
Crosswalk being an opensource project, an alternative would have been to patch Crosswalk source code and recompile it. However, Crosswalk embeds Chromium and recompiling the whole stuff consumes a lot of resources.

Ross Gammon: My Open Source Contributions June November 2016

So much for my monthly blogging! Here s what I have been up to in the Open Source world over the last 6 months. Debian

Uploaded a new version of the debian-multimedia blends metapackages
Uploaded the latest abcmidi
Uploaded the latest node-process-nextick-args
Prepared version 1.0.2 of libdrumstick for experimental, as a first step for the transition. It was sponsored by James Cowgill.
Prepared a new node-inline-source-map package, which was sponsored by Gianfranco Costamagna.
Uploaded kmetronome to experimental as part of the libdrumstick transition.
Prepared a new node-js-yaml package, which was sponsored by Gianfranco Costamagna.
Uploaded version 4.2.4 of Gramps.
Prepared a new version of vmpk which I am going to adopt, as part of the libdrumstick transition. I tried splitting the documentation into a separate package, but this proved difficult, and in the end I missed the transition freeze deadline for Debian Stretch.
Prepared a backport of Gramps 4.2.4, which was sponsored by IOhannes m zm lnig as Gramps is new for jessie-backports.
Began a final push to get kosmtik packaged and into the NEW queue before the impending Debian freeze for Stretch. Unfortunately, many dependencies need updating, which also depend on packages not yet in Debian. Also pushed to finish all the new packages for node-tape, which someone else has decided to take responsibility for.
Uploaded node-cross-spawn-async to fix a Release Critical bug.
Prepared a new node-chroma-js package, but this is unfortunately blocked by several out of date & missing dependencies.
Prepared a new node-husl package, which was sponsored by Gianfranco Costamagna.
Prepared a new node-resumer package, which was sponsored by Gianfranco Costamagna.
Prepared a new node-object-inspect package, which was sponsored by Gianfranco Costamagna.
Removed node-string-decoder from the archive, as it was broken and turned out not to be needed anymore.
Uploaded a fix for node-inline-source-map which was failing tests. This turned out to be due to node-tap being upgraded to version 8.0.0. J r my Lal very quickly provided a fix in the form of a Pull Request upstream, so I was able to apply the same patch in Debian.

Ubuntu

Prepared a merge of the latest blends package from Debian in order to be able to merge the multimedia-blends package later. This was sponsored by Daniel Holbach.
Prepared an application to become an Ubuntu Contributing Developer. Unfortunately, this was later declined. I was completely unprepared for the Developer Membership Board meeting on IRC after my holiday. I had had no time to chase for endorsements from previous sponsors, and the application was not really clear about the fact that I was not actually applying for upload permission yet. No matter, I intend to apply again later once I have more evidence & support on my application page.
Added my blog to Planet Ubuntu, and this will hopefully be the first post that appears there.
Prepared a merge of the latest debian-multimedia blends meta-package package from Debian. In Ubuntu Studio, we have the multimedia-puredata package seeded so that we get all the latest Puredata packages in one go. This was sponsored by Michael Terry.
Prepared a backport of Ardour as part of the Ubuntu Studio plan to do regular backports. This is still waiting for sponsorship if there is anyone reading this that can help with that.
Did a tweak to the Ubuntu Studio seeds and prepared an update of the Ubuntu Studio meta-packages. However, Adam Conrad did the work anyway as part of his cross-flavour release work without noticing my bug & request for sponsorship. So I closed the bug.
Updated the Ubuntu Studio wiki to expand on the process for updating our seeds and meta-packages. Hopefully, this will help new contributors to get involved in this area in the future.
Took part in the testing and release of the Ubuntu Studio Trusty 14.04.5 point release.
Took part in the testing and release of the Ubuntu Studio Yakkety Beta 1 release.
Prepared a backport of Ansible but before I could chase up what to do about the fact that ansible-fireball was no longer part of the Ansible package, some one else did the backport without noticing my bug. So I closed the bug.
Prepared an update of the Ubuntu Studio meta-packages. This was sponsored by Jeremy Bicha.
Prepared an update to the ubuntustudio-default-settings package. This switched the Ubuntu Studio desktop theme to Numix-Blue, and reverted some commits to drop the ubuntustudio-lightdm-theme package fom the archive. This had caused quite a bit of controversy and discussion on IRC due to the transition being a little too close to the release date for Yakkety. This was sponsored by Iain Lane (Laney).
Prepared the Numix Blue update for the ubuntustudio-lightdm-theme package. This was also sponsored by Iain Lane (Laney). I should thank Krytarik here for the initial Numix Blue theme work here (on the lightdm theme & default settings packages).
Provided a patch for gfxboot-theme-ubuntu which has a bug which is regularly reported during ISO testing, because the Try Ubuntu Studio without installing option was not a translatable string and always appeared in English. Colin Watson merged this, so hopefully it will be translated by the time of the next release.
Took part in the testing and release of the Ubuntu Studio Yakkety 16.10 release.
After a hint from Jeremy Bicha, I prepared a patch that adds a desktop file for Imagemagick to the ubuntustudio-default-settings package. This will give us a working menu item in Ubuntu Studio whilst we wait for the bug to be fixed upstream in Debian. Next month I plan to finish the ubuntustudio-lightdm-theme, ubuntustudio-default-settings transition, including dropping ubuntustudio-lightdm-theme from the Ubuntu Studio seeds. I will include this fix at the same time.

Other

At other times when I have had a spare moment, I have been working on resurrecting my old Family History website. It was originally produced in my Windows XP days, and I was no longer able to edit it in Linux. I decided to convert it to Jekyll. First I had to extract the old HTML from where the website is hosted using the HTTrack Website Copier. Now, I am in the process of switching the structure to the standard Jekyll template approach. I will need to switch to a nice Jekyll based theme, as as the old theming was pretty complex. I pushed the code to my Github repository for safe keeping.

Plan for December Debian Before the 5th January 2017 Debian Stretch soft freeze I hope to:

Ensure node-tape makes it through into Stretch.
Work on all the dependencies needed to get kosmtik into Stretch.
Convert Cree.py from QT4 to QT5 to allow it to re-enter Stretch.

Ubuntu

Add the Ubuntu Studio Manual Testsuite to the package tracker, and try to encourage some testing of the newest versions of our priority packages.
Finish the ubuntustudio-lightdm-theme, ubuntustudio-default-settings transition including an update to the ubuntustudio-meta packages.
Reapply to become a Contributing Developer.
Start working on an Ubuntu Studio package tracker website so that we can keep an eye on the status of the packages we are interested in.

Other

Continue working to convert my Family History website to Jekyll.
Try and resurrect my old Gammon one-name study Drupal website from a backup and push it to the new GoONS Website project.

2 December 2016

Raphaël Hertzog: My Free Software Activities in November 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS In the 11 hours of (paid) work I had to do, I managed to release DLA-716-1 aka tiff 4.0.2-6+deb7u8 fixing CVE-2016-9273, CVE-2016-9297 and CVE-2016-9532. It looks like this package is currently getting new CVE every month. Then I spent quite some time to review all the entries in dla-needed.txt. I wanted to get rid of some misleading/no longer applicable comments and at the same time help Olaf who was doing LTS frontdesk work for the first time. I ended up tagging quite a few issues as no-dsa (meaning that we will do nothing for them as they are not serious enough) such as those affecting dwarfutils, dokuwiki, irssi. I dropped libass since the open CVE is disputed and was triaged as unimportant. While doing this, I fixed a bug in the bin/review-update-needed script that we use to identify entries that have not made any progress lately. Then I claimed libgc and and released DLA-721-1 aka libgc 1:7.1-9.1+deb7u1 fixing CVE-2016-9427. The patch was large and had to be manually backported as it was not applying cleanly. The last thing I did was to test a new imagemagick and review the update prepared by Roberto. pkg-security work The pkg-security team is continuing its good work: I sponsored patator to get rid of a useless dependency on pycryptopp which was going to be removed from testing due to #841581. After looking at that bug, it turns out the bug was fixed in libcrypto++ 5.6.4-3 and I thus closed it. I sponsored many uploads: polenum, acccheck, sucrack (minor updates), bbqsql (new package imported from Kali). A bit later I fixed some issues in the bbsql package that had been rejected from NEW. I managed a few RC bugs related to the openssl 1.1 transition: I adopted sslsniff in the team and fixed #828557 by build-depending on libssl1.0-dev after having opened the proper upstream ticket. I did the same for ncrack and #844303 (upstream ticket here). Someone else took care of samdump2 but I still adopted the package in the pkg-security team as it is a security relevant package. I also made an NMU for axel and #829452 (it s not pkg-security related but we still use it in Kali). Misc Debian work Django. I participated in the discussion about a change letting Django count the number of developers that use it. Such a change has privacy implications and the discussion sparked quite some interest both in Debian mailing lists and up to LWN. On a more technical level, I uploaded version 1.8.16-1~bpo8+1 to jessie-backports (security release) and I fixed RC bug #844139 by backporting two upstream commits. This led to the 1.10.3-2 upload. I ensured that this was fixed in the 1.10.x upstream branch too. dpkg and merged /usr. While reading debian-devel, I discovered dpkg bug #843073 that was threatening the merged-/usr feature. Since the bug was in code that I wrote a few years ago, and since Guillem was not interested in fixing it, I spent an hour to craft a relatively clean patch that Guillem could apply. Unfortunately, Guillem did not yet manage to pull out a new dpkg release with the patches applied. Hopefully it won t be too long until this happens. Debian Live. I closed #844332 which was a request to remove live-build from Debian. While it was marked as orphaned, I was always keeping an eye on it and have been pushing small fixes to git. This time I decided to officially adopt the package within the debian-live team and work a bit more on it. I reviewed all pending patches in the BTS and pushed many changes to git. I still have some pending changes to finish to prettify the Grub menu but I plan to upload a new version really soon now. Misc bugs filed. I filed two upstream tickets on uwsgi to help fix currently open RC bugs on the package. I filed #844583 on sbuild to support arbitrary version suffix for binary rebuild (binNMU). And I filed #845741 on xserver-xorg-video-qxl to get it fixed for the xorg 1.19 transition. Zim. While trying to fix #834405 and update the required dependencies, I discovered that I had to update pygtkspellcheck first. Unfortunately, its package maintainer was MIA (missing in action) so I adopted it first as part of the python-modules team. Distro Tracker. I fixed a small bug that resulted in an ugly traceback when we got queries with a non-ASCII HTTP_REFERER. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

30 November 2016

Chris Lamb: Free software activities in November 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):

Started work on a Python API to the UK Postbox mail scanning and forwarding service. (repo)
Lots of improvements to buildinfo.debian.net, my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them, including making GPG signatures mandatory (#7), updating jenkins.debian.net to sign them and moving to SSL.
Improved the Django client to the KeyError error tracking software, enlarging the test coverage and additionally adding support for grouping errors using a context manager.
Made a number of improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change:
- Install build-dependencies with debugging output. Thanks to @waja. (#31)
- Install Lintian by default. Thanks to @freeekanayaka. (#33).
- Call mktemp with --dry-run to avoid having to delete it later. (commit)
Submitted a pull request to Wheel (a utility to package Python libraries) to make the output of METADATA files reproducible. (#73)
Submitted some miscellaneous documentation updates to the Tails operating system. (patches)

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

This month:

I was proud to announce that I have been awarded a grant from the Core Infrastructure Initiative (CII) to fund my previously-voluntary work in this area.
Presented a talk with Holger Levsen entitled "Reproducible builds status update" talk at MiniDebConfCambridge 2016. (Slides)
Attended the Tails operating system's Reproducible Builds sprint making excellent progress towards making the next release reproducible.
Ensured that Webconverger kiosk operating system can now be built reproducibly.
Within Debian, I filed a bug requesting that packages should be reproducible by policy. (#844431)

My work in the Reproducible Builds project was also covered in our weekly reports. (#80, #81, #82 #83.

Toolchain issues I submitted the following patches to fix reproducibility-related toolchain issues with Debian:

amd64-microcode: Please make the early initramfs image reproducible
initramfs-tools: Please ensure initrd images are reproducible
markdown: Please make the output reproducible
python-defaults: Please make the substvars reproducible
wheel: Please make the output of METADATA files reproducible

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

Ensure tests do not rely on Debian-specific Debian::Debhelper::Dh_Lib. (b9d5b06)
Filed a wishlist bug to support normalising NTFS timestamps in .zip files.

jenkins.debian.net

jenkins.debian.net runs our comprehensive testing framework.

buildinfo.debian.net has moved to SSL. (ac3b9e7)
Submit signing keys to keyservers after generation. (bdee6ff)
Various cosmetic changes, including
- Prefer if X not in Y over if not X in Y. (bc23884)
- No need for a dictionary; let's just use a set. (bf3fb6c)
- Avoid DRY violation by using a for loop. (4125ec5)

I also submitted 9 patches to fix specific reproducibility issues in apktool, cairo-5c, lava-dispatcher, lava-server, node-rimraf, perlbrew, qsynth, tunnelx & zp.

Debian

Debian LTS This month I have been paid to work 11 hours on Debian Long Term Support (LTS). In that time I did the following:

"Frontdesk" duties, triaging CVEs, etc.
Issued DLA 697-1 for bsdiff fixing an arbitrary write vulnerability.
Issued DLA 705-1 for python-imaging correcting a number of memory overflow issues.
Issued DLA 713-1 for sniffit where a buffer overflow allowed a specially-crafted configuration file to provide a root shell.
Issued DLA 723-1 for libsoap-lite-perl preventing a Billion Laughs XML expansion attack.
Issued DLA 724-1 for mcabber fixing a roster push attack.

Uploads

redis:
- 3.2.5-2 Tighten permissions of /var/ lib,log /redis. (#842987)
- 3.2.5-3 & 3.2.5-4 Improve autopkgtest tests and install upstream's MANIFESTO and README.md documentation.
gunicorn (19.6.0-9) Adding autopkgtest tests.
libfiu:
- 0.94-1 Add autopkgtest tests.
- 0.95-1, 0.95-2 & 0.95-3 New upstream release and improve autopkgtest coverage.
python-django (1.10.3-1) New upstream release.
aptfs (0.8-3, 0.8-4 & 0.8-5) Adding and subsequently improving the autopkgtext tests.

I performed the following QA uploads:

boxbackup (0.11.1~r2837-3) Fix compatibility with OpenSSL 1.1.0. (#828253)
httperf (0.9.0-6) Fix compatibility with OpenSSL 1.1.0. (#828343)
ike (2.2.1+dfsg-6) Fix compatibility with OpenSSL 1.1.0. (#828349)
polygraph (4.3.2-4) Fix compatibility with GCC 6. (#831125)
samdump2 (3.0.0-4) Fix compatibility with OpenSSL 1.1.0. (#828537)
qink (0.3.5-8) Fix FTBFS. (#841572)

Finally, I also made the following non-maintainer uploads:

libident (0.22-3.1) Move from obsolete Source-Version substvar to binary:Version. (#833195)
libpcl1 (1.6-1.1) Move from obsolete Source-Version substvar to binary:Version. (#833196)
pygopherd (2.0.18.4+nmu1) Move from obsolete Source-Version substvar to $ source:Version . (#833202)

Debian bugs filed

bugs.debian.org: Please provide "Content-Disposition: attachment; filename=[..]" headers
initramfs-tools: Don't print a warning if /etc/modprobe.d is empty
cwltool: Accesses the internet during build

RC bugs

dpdk: maintainer address bounces
libc6-dev: static linking with stack protector fails: undefined reference to __memcpy_chk'
libpcap: libpcap0.8-dev does not contain remote-ext.h
libspring-java: Invalid optional dependencies in the Maven POMs
libssl1.1: 1.1.0c broke Python
linux, linux-2.6, linux-2.6.24: linux/atm_zatm.h change breaks time.h inclusion
php-memcached: php-memcached must Build-depend on zlib1g-dev
php-pecl-http: php-pecl-http must Build-depend on zlib1g-dev
pokerth: -std=gnu++98 workaround no longer works with Qt 5.7
pyelliptic: Does not support OpenSSL 1.1
salt-call fails with libcrypto.so.1.1: undefined symbol: OPENSSL_no_config
potrace: CVE-2016-8685 CVE-2016-8686
ming: CVE-2016-9264 CVE-2016-9265 CVE-2016-9266

I also filed 59 FTBFS bugs against arc-gui-clients, asyncpg, blhc, civicrm, d-feet, dpdk, fbpanel, freeciv, freeplane, gant, golang-github-googleapis-gax-go, golang-github-googleapis-proto-client-go, haskell-cabal-install, haskell-fail, haskell-monadcatchio-transformers, hg-git, htsjdk, hyperscan, jasperreports, json-simple, keystone, koji, libapache-mod-musicindex, libcoap, libdr-tarantool-perl, libmath-bigint-gmp-perl, libpng1.6, link-grammar, lua-sql, mediatomb, mitmproxy, ncrack, net-tools, node-dateformat, node-fuzzaldrin-plus, node-nopt, open-infrastructure-system-images, open-infrastructure-system-images, photofloat, ppp, ptlib, python-mpop, python-mysqldb, python-passlib, python-protobix, python-ttystatus, redland, ros-message-generation, ruby-ethon, ruby-nokogiri, salt-formula-ceilometer, spykeviewer, sssd, suil, torus-trooper, trash-cli, twisted-web2, uftp & wide-dhcpv6.

FTP Team

As a Debian FTP assistant I ACCEPTed 70 packages: bbqsql, coz-profiler, cross-toolchain-base, cross-toolchain-base-ports, dgit-test-dummy, django-anymail, django-hstore, django-html-sanitizer, django-impersonate, django-wkhtmltopdf, gcc-6-cross, gcc-defaults, gnome-shell-extension-dashtodock, golang-defaults, golang-github-btcsuite-fastsha256, golang-github-dnephin-cobra, golang-github-docker-go-events, golang-github-gogits-cron, golang-github-opencontainers-image-spec, haskell-debian, kpmcore, libdancer-logger-syslog-perl, libmoox-buildargs-perl, libmoox-role-cloneset-perl, libreoffice, linux-firmware-raspi3, linux-latest, node-babel-runtime, node-big.js, node-buffer-shims, node-charm, node-cliui, node-core-js, node-cpr, node-difflet, node-doctrine, node-duplexer2, node-emojis-list, node-eslint-plugin-flowtype, node-everything.js, node-execa, node-grunt-contrib-coffee, node-grunt-contrib-concat, node-jquery-textcomplete, node-js-tokens, node-json5, node-jsonfile, node-marked-man, node-os-locale, node-sparkles, node-tap-parser, node-time-stamp, node-wrap-ansi, ooniprobe, policycoreutils, pybind11, pygresql, pysynphot, python-axolotl, python-drizzle, python-geoip2, python-mockupdb, python-pyforge, python-sentinels, python-waiting, pythonmagick, r-cran-isocodes, ruby-unicode-display-width, suricata & voctomix-outcasts. I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against node-cliui, node-core-js, node-cpr & node-grunt-contrib-concat.

27 November 2016

Dirk Eddelbuettel: anytime 0.1.1: More robust

CRAN just accepted the newest release 0.1.1 of anytime, following the previous five releases since September. anytime is a very focussed package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, ... format to POSIXct (or Date) objects -- and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples, or just consider the following illustration:

R> library(anytime)
R> anytime("20161107 202122")   ## all digits
[1] "2016-11-07 20:21:22 CST"
R> utctime("2016Nov07 202122")  ## UTC parse example
[1] "2016-11-07 14:21:22 CST"
R>

Release 0.1.1 robustifies two aspects. The 'digits only' input above extends what Boost Date_Time can parse and relies on simple-enough pre-processing. This operation is now more robust. We also ensure that input already of class Date is simply passed through by anydate() or utcdate(). Last but not least we added code coverage support, which oh-so-predictably lead us to game this metric to reach the elusive 100% coverage. The NEWS file summarises the release:

Changes in anytime version 0.1.1 (2016-11-27)

Both anydate() and utcdate() no longer attempt to convert an input value that is already of type Date.

The string splitter (needed for the 'all-digits' formats extending Boost Date_time) is now more defensive about the input argument and more robust. Thanks to Bob Jansen for the heads-up (PR #30 closing issue #29).

Code coverage reporting has been added (PR #31).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

25 November 2016

Michael Stapelberg: Debian package build tools

Personally, I find the packaging tools which are available in Debian far too complex. To better understand the options we have, I created a diagram of tools which are frequently used, only covering the build step (i.e. no post-build quality assurance checks or packaging-time helpers):

When I was first introduced to Debian packaging, people recommended I use pbuilder. Given how complex the toolchain is in the pbuilder case, I don t understand why that is (was?) a common recommendation. Back in August 2015, so well over a year ago, I switched to sbuild, motivated by how much simpler it was to implement ratt (rebuilds reverse build dependencies) using sbuild, and I have not looked back. Are there people who do not use sbuild for reasons other than familiarity? If so, please let me know, I d like to understand. I also made a version of the diagram above, colored by the programming languages in which the tools are implemented. The chosen colors are heavily biased :-).

To me, the diagram above means: if you want to make substantial changes to the Debian build tool infrastructure, you need to become an expert in all of Python, Perl, Bash, C and Make. I know that this is not true for every change, but it still irks me that there might be changes for which it is required. I propose to eliminate complexity in Debian by deprecating the pbuilder toolchain in favor of sbuild.

24 November 2016

Dirk Eddelbuettel: RcppExamples 0.1.8

A new version of the RcppExamples package is now on CRAN. The RcppExamples package provides a handful of short examples detailing by concrete working examples how to set up basic R data structures in C++. This version takes advantage of the updated date and datetime classes in Rcpp 0.12.8 (which are optional for now and being phased in while we deprecate the old ones). A NEWS extract follows:

Changes in RcppExamples version 0.1.8 (2016-11-24)

Updated DateExample to show vector addition available under Rcpp 0.12.8 when the (currently still phased in and optional) new Date(time) classes are used via the define in src/Makevars,.win; with fallback code for older versions

Other minor edits to DESCRIPTION and README.md

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

23 November 2016

Tanguy Ortolo: Generate man pages for awscli

No man pages, but almost The AWS Command Line Interface, which is available in Debian, provides no man page. Instead, that tool has an integrated help system, which allows you to run commands such as

aws rds
    help

, that, for what I have seen, generates some reStructuredText, then converts it to a man page in troff format, then calls troff to convert it to text with basic formatting, and eventually passes it to a pager. Since this is close to what man does, the result looks like a degraded man page, with some features missing such as the adaptation to the terminal width. Well, this is better than nothing, and better than what many under-documented tools can offer, but for several reasons, it still sucks: most importantly, it does not respect administrators' habits and it does not integrate with the system man database. You it does not allow you to use commands such as apropos, and you will get no man page name auto-completion from your shell since there is no man page.Generate the man pages Now, since the integrated help system does generate a man page internally, we can hack it to output it, and save it to a file:

Description: Enable a mode to generate troff man pages
 The awscli help system internally uses man pages, but only to convert
 them to text and show them with the pager. This patch enables a mode
 that prints the troff code so the user can save the man page.
 .
 To use that mode, run the help commands with an environment variable
 OUTPUT set to 'troff', for instance:
     OUTPUT='troff' aws rds help
Forwarded: no
Author: Tanguy Ortolo <tanguy+debian@ortolo.eu>
Last-Update: 2016-11-22
Index: /usr/lib/python3/dist-packages/awscli/help.py
===================================================================
--- /usr/lib/python3/dist-packages/awscli/help.py       2016-11-21 12:14:22.236254730 +0100
+++ /usr/lib/python3/dist-packages/awscli/help.py       2016-11-21 12:14:22.236254730 +0100
@@ -49,6 +49,8 @@
     Return the appropriate HelpRenderer implementation for the
     current platform.
     """
+    if 'OUTPUT' in os.environ and os.environ['OUTPUT'] == 'troff':
+        return TroffHelpRenderer()
     if platform.system() == 'Windows':
         return WindowsHelpRenderer()
     else:
@@ -97,6 +99,15 @@
         return contents
+class TroffHelpRenderer(object):
+    """
+    Render help content as troff code.
+    """
+
+    def render(self, contents):
+        sys.stdout.buffer.write(publish_string(contents, writer=manpage.Writer()))
+
+
 class PosixHelpRenderer(PagingHelpRenderer):
     """
     Render help content on a Posix-like system.  This includes

This patch must be applied from the root directory with patch -p0, otherwise GNU patch will not accept to work on files with absolute names. With that patch, you can run help commands with an environment variable OUTPUT='troff' to get the man page to use it as you like, for instance:

% OUTPUT='troff' aws rds help > aws_rds.1
% man -lt aws_rds.1   lp

Generate all the man pages Now that we are able to generate the man page of any aws command, all we need to generate all of them is a list of all the available commands. This is not that easy, because the commands are somehow derived from functions provided by a Python library named botocore, which are derived from a bunch of configuration files, and some of them are added, removed or renamed. Anyway, I have been able to write a Python script that does that, but it includes a static list of these modifications:

#! /usr/bin/python3
import subprocess
import awscli.clidriver
def write_manpage(command):
    manpage = open('%s.1' % '_'.join(command), 'w')
    command.append('help')
    process = subprocess.Popen(command,
            env= 'OUTPUT': 'troff' ,
            stdout=manpage)
    process.wait()
    manpage.close()
driver = awscli.clidriver.CLIDriver()
command_table = driver._get_command_table()
renamed_commands = \
     
        'config': 'configservice',
        'codedeploy': 'deploy',
        's3': 's3api'
     
added_commands = \
     
        's3': ['cp', 'ls', 'mb', 'mv', 'presign', 'rb', 'rm', 'sync',
               'website']
     
removed_subcommands = \
     
        'ses': ['delete-verified-email-address',
                'list-verified-email-addresses',
                'verify-email-address'],
        'ec2': ['import-instance', 'import-volume'],
        'emr': ['run-job-flow', 'describe-job-flows',
                'add-job-flow-steps', 'terminate-job-flows',
                'list-bootstrap-actions', 'list-instance-groups',
                'set-termination-protection',
                'set-visible-to-all-users'],
        'rds': ['modify-option-group']
     
added_subcommands = \
     
        'rds': ['add-option-to-option-group',
                'remove-option-from-option-group']
     
# Build a dictionary of real commands, including renames, additions and
# removals.
real_commands =  
for command in command_table:
    subcommands = []
    subcommand_table = command_table[command]._get_command_table()
    for subcommand in subcommand_table:
        # Skip removed subcommands
        if command in removed_subcommands \
                and subcommand in removed_subcommands[command]:
            continue
        subcommands.append(subcommand)
    # Add added subcommands
    if command in added_subcommands:
        for subcommand in added_subcommands[command]:
            subcommands.append(subcommand)
    # Directly add non-renamed commands
    if command not in renamed_commands:
        real_commands[command] = subcommands
    # Add renamed commands
    else:
        real_commands[renamed_commands[command]] = subcommands
# Add added commands
for command in added_commands:
    real_commands[command] = added_commands[command]
# For each real command and subcommand, generate a manpage
write_manpage(['aws'])
for command in real_commands:
    write_manpage(['aws', command])
    for subcommand in real_commands[command]:
        write_manpage(['aws', command, subcommand])
                         'sync', 'website']

This script will generate more than 2,000 man page files in the current directory; you will then be able to move them to /usr/local/share/man/man1. Since this is a lot of man pages, it may be appropriate to concatenate them by major command, for instance all the aws rds together

22 November 2016

Steve Langasek: A new chapter

I don't often write on this blog, and when I do, it's either tech related, or light life stuff. Over the next few weeks, it's going to get a lot more political. If you currently follow this blog for its technical content, you may be tempted to tune out. I would encourage you to stay and listen. I'm passionate about the technology that I work on; but the greatest problems facing our world today are not ones that will be solved with software. American democracy is in bad shape, and it's because of what we're doing to it. This is not a problem of the Right or of the Left; it is not a problem that began with the election of Donald Trump, and it's not a problem that will go away at the end of his term. It is partly a structural problem with the way our elections work, but more than that it's a problem of how we're splitting into separate tribes, isolating ourselves from those who don't agree with us. As Russ Allbery wrote the morning after the election, everything about how we organize ourselves online today - and how we let Facebook and Twitter organize us - leads us to surround ourselves with people who already think the same way we do. That leaves all of us with huge blind spots for other people in our country, and it stifles the free exchange of ideas that is so essential for a healthy democracy. We need leaders who will work to make America a better and more just place for all our neighbors, not just a two-party system that plays tug-of-war using two different sets of voters that feel shut out. And the way we organize ourselves today (online and off) does not let us recognize those leaders. There's a lot of talk now about Facebook changing how it decides what to show people; and maybe they can manage to help everyone's online experience be a little less of a bubble. But part of the change needs to come from us. We need to be willing to engage, civilly, with people whose perspective is different from ours, and make the effort to understand where the other is coming from. So for the next few weeks, I'm going to talk. And I'm going to listen. I have no unique qualifications to speak about the country's issues. But I do have a perspective of my own, which might be different enough from yours to be useful. I was born and raised in Iowa, and graduated from college there. This election cycle, I learned that Iowa holds the distinction of being the state with the lowest percentage of college-educated whites. I'm part of that statistic, because a few years after graduating I moved to Portland, Oregon - a place that's notoriously so far to the left of what we think of as the middle, that it actually has anarchists who would shamefully use a peaceful protest as cover to commit property crime. So I know a few things about the people in each state, that I think the other should hear. I'm also that rarest of creatures, a Portlander who goes to church (Catholic). But I still choose as my neighbors the weird, wonderful, and welcoming community that we have here, whatever Glenn Beck might think. I have a son, and I worry about what kind of world he'll grow up to live in. I work in software, which means I'm doing a lot better than a lot of people in the country right now; it also means that from where I sit, I see trends already in progress that will have an effect on the working class and the middle class that makes NAFTA look like a gnat's fart in comparison. And so I worry for what kind of world we will all live in, if we don't make some changes fast. Let's have a conversation. No comments enabled on this blog, but you can find me on G+ or on Facebook.

21 November 2016

Mike Gabriel: Please Welcome D0n1elT to the FLOSS World

TL;DR; If you run a FLOSS development project and you notice D0n1elT appearing on your IRC channel, please give him a warm welcome. D0n1elT is a young man highy talented in various FLOSS related topics already. He probably needs some guidance at the beginning and I hope he won't be too shy to ask for it. But you can be sure: your channel has been joined by someone you should consider as a future resource. The Long Story During the last two weeks I had the great pleasure of supervising a fine young man (very young, still, indeed) in all sorts of IT topics. This young man turned out to be so skilled and interested in various FLOSS related areas, I really want to introduce him to all of you. The young man's real name is Daniel Teichmann. On IRC he may appear under his nick: D0n1elT. His GnuPG Fingerprint is: 6C6E 7F8F F7E8 B22E FC76 E9F7 8A79 028F DA56 7C6C. Daniel goes to a local school here in Nothern Germany, near where I live. He attends the 9th grade at his school, and as common for students of his age and grade, practical training was scheduled for the last two weeks. Daniel had originally applied for practical training at some other business near his place of living (which is quite far off from the school, actually). However, that company cancelled his training position two work days before the training was supposed to start. Daniel's teacher rang me up and asked for help. He advertised Daniel as someone who is far advanced in IT topics compared to his co-students. "He even writes his own programs (in Java and C++)." Spontaneously, Andreas Buchholz (CEO of LOGO EDV-Systeme GmbH) and I decided to accept Daniel as a trainee. Without having met him, with no application interview beforehand. The deal was: Daniel comes to Andreas business location in Kiel (40-50km away from Daniel's place of living) and I (working as freelancer for LOGO on a regular basis) do the supervising part. On day one and two, as a warm-up, Daniel installed a Debian Edu Main Server, worked himself through GOsa, LDAP, SSH, GnuPG, Jabber and IRC and configured two routers. All topics were new to him and I could hardly think of new tasks to give to him. As means of communication we set up a Jabber account, then an IRC account (as backup). However, it turned out that Daniel really got a hang of IRC over the next couple of days, so we used that as primary communication channel. Daniel had already programmed various projects in Java (whereas I have never touched Java, so far :-( ). He has written plugins for Minecraft servers. He knows well how to implement object oriented coding models. His coding style looks very good and clean (esp. for someone who has never head a nitpicking code reviewer). He started coding at the age of 9. Instead of diving into Java (where I would not have been of much help, anyway) I decided to provide him with some really basic and Unix-like knowledge: Bash scripting. I wanted to see how he handles another "language" and how he applies his Java knowledge to a lower level, syntactically weaker language. Guess what, he managed that assignment very well. Working on Impressive Display At Daniel's school we run substitute teacher info screens based on a fancy Bash script, named impressive-display, and the impressive PDF viewer. The Impressive Display tool is available in Debian testing/unstable under the same name. So over the next couple of days we worked on Impressive Display. Daniel contributed so many new code passages, conceptual ideas and security concerns, that I decided to make him co-copyright holder. Every change contributed by him received intensive testing before committing to Git. While working on Impressive Display, collaborating with Daniel via Git was a mere pleasure. In his spare time Daniel likes watching Github tutorials. Quite extraordinary. The result is a new major release of Impressive Display: Version 0.3.1 (bumped up from 0.2.3). We added the feature of handling info screen farms based on PXE boot images. It is now possible to configure as many different info screens as needed within the same PXE bootable chroot. Furthermore, Impressive Display now has a PDF presentation (written in LaTeX Beamer) that documents how to setup your own info screens. The PDF presentation is the default PDF that comes up when you start Impressive Display directly after installation. Investigating other Realms We also took a deeper look at remote desktop stuff, one of my most favourite topics. By that impulse Daniel set up his first Vserver machine at some hosting provider. He figured out how to run X2Go Server on that machine with an XFCE desktop. Next step was to run the irssi instance from his notebook inside a screen session on the Vserver. Some days later, Daniel PM'ed me: "I have an IRC bouncer now...". Quintessence It was a great pleasure meeting this young, highly curious and already highly skilled young man over the past two weeks. Daniel, it was an asset to me working with you. You are such a fast learner when it comes to getting accustomed to new working environments, it is amazing. I cannot deny having observed the tendency of preferring rather geeky tools. I was highly delighted, that What's-That and Facebook are nothing that rocks you so much. Unfortunately, all of the above makes you quite unique and non-mainstream among people of your age. My wish for you (and the FLOSS world) is that you start getting in touch with other (FLOSS) developers, maybe of your age, maybe older, and that you (if this is what you want) become an asset to the world of Free Software. The Free Software world can be a world where technical, political and spiritual work become one with friendship among people. Take care and farewell! I am sure, we will meet again. light+love Mike Gabriel (aka sunweaver on IRC and debian.org)

Daniel Pocock: 21 November 1916

There has been a lot of news recently about the 100th anniversaries of various events that took place during the Great War. On 21 November 1916, the SS Hunscraft sailed from Southampton to France. My great grandfather, Robert Pocock, was aboard. He was part of the Australian Imperial Force, 3rd Divisional Train. It's sad that Australians had to travel half way around the world to break up fist fights and tank battles. Sadder still that some people who romanticize the mistakes of imperialism are being appointment to significant positions of power. Fortunately my great grandfather returned to Australia in one piece, many Australians didn't. Robert Pocock's war medals

Steve Kemp: Detecting fraudulent signups?

I run a couple of different sites that allow users to sign-up and use various services. In each of these sites I have some minimal rules in place to detect bad signups, but these are a little ad hoc, because the nature of "badness" varies on a per-site basis. I've worked in a couple of places where there are in-house tests of bad signups, and these usually boil down to some naive, and overly-broad, rules:

Does the phone numbers' (international) prefix match the country of the user?
Does the postal address supplied even exist?

Some places penalise users based upon location too:

Does the IP address the user submitted from come from TOR?
Does the geo-IP country match the users' stated location?
Is the email address provided by a "free" provider?

At the moment I've got a simple HTTP-server which receives a JSON post of a new users' details, and returns "200 OK" or "403 Forbidden" based on some very very simple critereon. This is modeled on the spam detection service for blog-comments server I use - something that is itself becoming less useful over time. (Perhaps time to kill that? A decision for another day.) Unfortunately this whole approach is very reactive, as it takes human eyeballs to detect new classes of problems. Code can't guess in advance that it should block usernames which could collide with official ones, for example allowing a username of "admin", "help", or "support". I'm certain that these systems have been written a thousand times, as I've seen at least five such systems, and they're all very similar. The biggest flaw in all these systems is that they try to classify users in advance of them doing anything. We're trying to say "Block users who will use stolen credit cards", or "Block users who'll submit spam", by correlating that behaviour with other things. In an ideal world you'd judge users only by the actions they take, not how they signed up. And yet .. it is better than nothing. For the moment I'm continuing to try to make the best of things, at least by centralising the rules for myself I cut down on duplicate code. I'll pretend I'm being cool, modern, and sexy, and call this a micro-service! (Ignore the lack of containers for the moment!)

Next.

Previous.

Search Results: "nate"

9 December 2016

8 December 2016

7 December 2016

What CVE-2016-4484 is all about Basically, the vulnerability is about two separate but related issues:

Links Hector Marco & Ismael Ripoll: CVE-2016-4484: Cryptsetup Initrd root Shell Public announcement by Hector Marco of CVE-2014-4484 on oss-security Discussion about CVE-2016-4484 on dm-crypt@saout.de

5 December 2016

3 December 2016

Simple method replacement Let s replace the shouldDenyRequest() method from the org.xwalk.core.internal.SslUtil class with this version: // In SslUtil class public static boolean shouldDenyRequest(int error) return false;

2 December 2016

30 November 2016

27 November 2016

25 November 2016

24 November 2016

23 November 2016

22 November 2016

21 November 2016

Links

Hector Marco & Ismael Ripoll: CVE-2016-4484: Cryptsetup Initrd root Shell

Public announcement by Hector Marco of CVE-2014-4484 on oss-security

Discussion about CVE-2016-4484 on dm-crypt@saout.de

Simple method replacement Let s replace the `shouldDenyRequest()` method from the `org.xwalk.core.internal.SslUtil` class with this version:
// In SslUtil class public static boolean shouldDenyRequest(int error) return false;