Search Results: "manuel"

23 February 2022

Russ Allbery: Review: Elder Race

Review: Elder Race, by Adrian Tchaikovsky
Publisher: Tordotcom
Copyright: November 2021
ISBN: 1-250-76871-3
Format: Kindle
Pages: 199
(It's a shame that a lot of people will be reading this novella on a black-and-white ebook reader, since the Emmanuel Shiu cover is absolutely spectacular. There's a larger image without the words at the bottom of that article.) When reports arrive at the court about demons deep in the forest that are taking over animals and humans and bending them to their will, the queen doesn't care. It's probably some unknown animal, and regardless, the forest kingdom is a rival anyway. Lynesse Fourth Daughter disagrees vehemently, but she has no power at court. Even apart from her lack of seniority, her love of stories and daring and adventures is a source of endless frustration to her mother. That is why this novella opens with her climbing the mountain path to the Tower of Nyrgoth Elder, the last of the ancient wizards, to seek his help. Nyr Illim Tevitch is an anthropologist second class of Earth's Explorer Corps, part of the second wave of Earth's outward expansion through the galaxy. In the first wave, colonies were seeded on habitable planets, only to be left stranded when Earth's civilization collapsed in an ecological crisis. Nyr was a member of a team of four, sent to make careful and limited contact with one of those lost colonies as part of Earth's second flourishing with more advanced technology. When the team lost contact with Earth, the other three went back while Nyr stayed to keep their field observations going. It's now 291 years of intermittent suspended animation later. Nyr's colleagues never came back, and there have been no messages from Earth. Elder Race is a Prime Directive anthropology story, a subgenre so long-standing that it has its own conventions and variations. Variations of the theme have been written by everyone from Eleanor Arnason to Iain M. Banks (linking to the book I have in mind is arguably a spoiler). Per the dedication, Tchaikovsky's take is based on Gene Wolfe's story "Trip, Trap," which I have not read but whose plot looks very similar. To that story structure, Tchaikovsky brings two major twists. First, Nyr is cut off from his advanced civilization, and has considerable reason to believe that civilization no longer exists. Do noninterference rules still have any meaning if Nyr is stranded and the civilization that made the rules is gone? Second, Nyr has already broken those rules rather spectacularly. More than a hundred years previously, he had ridden with Astresse Regent, a warrior queen and Lynesse's ancestor, to defeat a local warlord who had found control codes for abandoned advanced machinery and was using it as weaponry. In the process, he fell in love and made a rash promise to come to the aid of any of her descendants if he were needed. Lynesse has come to collect on the promise. Elder Race is told in alternating chapters between Nyr and Lynesse's viewpoints: first person for Nyr and tight third person for Lynesse. The core of the story is this doubled perspective, one from a young woman who wants to live in a fantasy novel and one from a deeply depressed anthropologist torn between wanting human contact, wanting to follow the rules of his profession, and wanting to explain to Lynesse that he is not a wizard. Nyr talks himself into helping with another misuse of advanced technology using the same logic he used a hundred years earlier: he's protecting Lynesse's pre-industrial society from interference rather than causing it. But the demons Lynesse wants him to fight are something entirely unexpected. This parallel understanding is a great story structure. What worked less for me was Tchaikovsky's reliance on linguistic barriers to prevent shared understanding. Whenever Nyr tries to explain something, Lynesse hears it in terms of magic and high fantasy, and often exactly backwards from how Nyr intended it. This is where my suspension of disbelief failed me, even though I normally don't have suspension of disbelief problems in SF stories. I was unable to map Lynesse's misunderstandings to any realistic linguistic model. Lynesse's language is highly complex (a realistic development within an isolated population), and Nyr complains about his inability to speak it properly given it's blizzard of complex modifiers. This is entirely believable. What is far less believable is that Lynesse perceives him as fluent in her language, but often saying the precise opposite of what he's trying to say. One chapter in the middle of the book gives Nyr's intended story side-by-side with Lynesse's understanding. This is a brilliant way to show the divide, but I found the translation errors unbelievable. If Nyr is failing that profoundly to communicate his meaning, he should be making more egregious sentence-level errors, occasionally saying something bizarre or entirely nonsensical, referring to a person as an animal or a baby, or otherwise not fluently telling a coherent story that's fundamentally different than the one he thinks he's telling. If you can put that aside, though, this is a fun story. Nyr has serious anxiety and depression made worse by his isolation, and copes by using an implanted device called a Dissociative Cognition System that lets him temporarily turn off his emotions at the cost of letting them snowball. He has a wealth of other augments and implants, including horns, which Lynesse sees as evidence that he's a different species of magical being and which he sees as occasionally irritating field equipment with annoying visual menus. The key to writing a story like this is for both perspectives to be correct given their own assumptions, and to offer insight that the other perspective is missing. I thought the linguistic part of that was unsuccessful, but the rest of it works. One of the best parts of novellas is that they don't wear out their welcome. This is a fun spin on well-trodden ground that tells a complete story in under 200 pages. I wish the ending had been a bit more satisfying and the linguistics had been more believable, but I enjoyed the time I spent in this world. Content warning for some body horror. Rating: 7 out of 10

21 January 2022

Neil McGovern: Further investments in desktop Linux

This was originally posted on the GNOME Foundation news feed The GNOME Foundation was supported during 2020-2021 by a grant from Endless Network which funded the Community Engagement Challenge, strategy consultancy with the board, and a contribution towards our general running costs. At the end of last year we had a portion of this grant remaining, and after the success of our work in previous years directly funding developer and infrastructure work on GTK and Flathub, we wanted to see whether we could use these funds to invest in GNOME and the wider Linux desktop platform. We re very pleased to announce that we got approval to launch three parallel contractor engagements, which started over the past few weeks. These projects aim to improve our developer experience, make more applications available on the GNOME platform, and move towards equitable and sustainable revenue models for developers within our ecosystem. Thanks again to Endless Network for their support on these initiatives. Flathub Verified apps, donations and subscriptions (Codethink and James Westman) This project is described in detail on the Flathub Discourse but goal is to add a process to verify first-party apps on Flathub (ie uploaded by a developer or an authorised representative) and then make it possible for those developers to collect donations or subscriptions from users of their applications. We also plan to publish a separate repository that contains only these verified first-party uploads (without any of the community contributed applications), as well as providing a repository with only free and open source applications, allowing users to choose what they are comfortable installing and running on their system. Creating the user and developer login system to manage your apps will also set us up well for future enhancements, such managing tokens for direct binary uploads (eg from a CI/CD system hosted elsewhere, as is already done with Mozilla Firefox and OBS) and making it easier to publish apps from systems such as Electron which can be hard to use within a flatpak-builder sandbox. For updates on this project you can follow the Discourse thread, check out the work board on GitHub or join us on Matrix. PWAs Integrating Progressive Web Apps in GNOME (Phaedrus Leeds) While everyone agrees that native applications can provide the best experience on the GNOME desktop, the web platform, and particularly PWAs (Progressive Web Apps) which are designed to be downloadable as apps and offer offline functionality, makes it possible for us to offer equivalent experiences to other platforms for app publishers who have not specifically targeted GNOME. This allows us to attract and retain users by giving them the choice of using applications from a wider range of publishers than are currently directly targeting the Linux desktop. The first phase of the GNOME PWA project involves adding back support to Software for web apps backed by GNOME Web, and making this possible when Web is packaged as a Flatpak. So far some preparatory pull requests have been merged in Web and libportal to enable this work, and development is ongoing to get the feature branches ready for review. Discussions are also in progress with the Design team on how best to display the web apps in Software and on the user interface for web apps installed from a browser. There has also been discussion among various stakeholders about what web apps should be included as available with Software, and how they can provide supplemental value to users without taking priority over apps native to GNOME. Finally, technical discussion is ongoing in the portal issue tracker to ensure that the implementation of a new dynamic launcher portal meets all security and robustness requirements, and is potentially useful not just to GNOME Web but Chromium and any other app that may want to install desktop launchers. Adding support for the launcher portal in upstream Chromium, to facilitate Chromium-based browsers packaged as a Flatpak, and adding support for Chromium-based web apps in Software are stretch goals for the project should time permit. GTK4 / Adwaita To support the adoption of Gtk4 by the community (Emmanuele Bassi) With the release of GTK4 and renewed interest in GTK as a toolkit, we want to continue improving the developer experience and ease of use of GTK and ensure we have a complete and competitive offering for developers considering using our platform. This involves identifying missing functionality or UI elements that applications need to move to GTK4, as well as informing the community about the new widgets and functionality available. We have been working on documentation and bug fixes for GTK in preparation for the GNOME 42 release and have also started looking at the missing widgets and API in Libadwaita, in preparation for the next release. The next steps are to work with the Design team and the Libadwaita maintainers and identify and implement missing widgets that did not make the cut for the 1.0 release. In the meantime, we have also worked on writing a beginners tutorial for the GNOME developers documentation, including GTK and Libadwaita widgets so that newcomers to the platform can easily move between the Interface Guidelines and the API references of various libraries. To increase the outreach of the effort, Emmanuele has been streaming it on Twitch, and published the VOD on YouTube as well.

20 July 2021

Enrico Zini: Run a webserver for a specific user *only*

I'm creating a program that uses the web browser for its user interface, and I'm reasonably sure I'm not the first person doing this. Normally such a problem would listen to a port on localhost, and tell the browser to connect to it. Bonus points for listening to a randomly allocated free port, so that one does not need to involve some amount of luck to get the program started. However, using a local port still means that any user on the local machine can connect to it, which is generally a security issue. A possible solution would be to use AF_UNIX Unix Domain Sockets, which are supported by various web servers, but as far as I understand not currently by browsers. I checked Firefox and Chrome, and they currently seem to fail to even acknowledge the use case. I'm reasonably sure I'm not the first person doing this, and yes, it's intended as an understatement. So, dear Lazyweb, is there a way to securely use a browser as a UI for a user's program, without exposing access to the backend to other users in the system? Access token in the URL Emanuele Di Giacomo suggests to add an access token to the URL that gets passed to the browser. This would work to protect access on localhost: even if the application cannot use HTTPS, other users cannot see packets that go through the local interface, so both the access token and the session cookie that one could send afterwards would be protected. Network namespaces I thought about isolating server and browser in a private network namespace with something like unshare(1), but it seems to require root. Johannes Schauer Marin Rodrigues wrote to correct that:
It's possible to unshare the network namespace by first unsharing the user namespace and thus becoming root which is possible without being root since #898446 got fixed. For example you can run this as the normal user: lxc-usernsexec -- lxc-unshare -s NETWORK -- ip addr If you don't want to depend on lxc, you can write a wrapper in Perl or Python. I have a Perl implementation of that in mmdebstrap.
Firewalling Martin Schuster wrote to suggest another option:
I had the same issue. My approach was "weird", but worked: Block /outgoing/ connections to the port, unless the uid is correct. That might be counter-intuitive, but of course all connections /to/ localhost will be done /from/ localhost also. Something like: iptables -A OUTPUT -p tcp -d localhost --dport 8123 -m owner --uid-owner joe -j ACCEPT iptables -A OUTPUT -p tcp -d localhost --dport 8123 -j REJECT

18 July 2021

Shirish Agarwal: BBI Kenyan Supreme Court, U.P. Population Bill, South Africa, Suli Deals , IT rules 2021, Sedition Law and Danish Siddiqui s death.

BBI Kenya and live Supreme Court streaming on YT The last few weeks have been unrelenting as all sorts of news have been coming in, mostly about the downturn in the Economy, Islamophobia in India on the rise, Covid, and electioneering. However, in the last few days, Kenya surpassed India in live-streaming proceeds in a Court of Appeals about BBI or Building Bridges Initiative. A background filler article on the topic can be found in BBC. The live-streaming was done via YT and if wants to they can start from

https://www.youtube.com/watch?v=JIQzpmVKvro One can also subscribe to K24TV which took the initiative of sharing the proceedings with people worldwide. If K24TV continues to share SC proceedings of Kenya, that would add to the soft power of Kenya. I will not go into the details of the case as Gautam Bhatia who has been following the goings-on in Kenya is a far better authority on the subject. In fact, just recently he shared about another Kenyan judgment from a trial which can be seen here. He has shared the proceedings and some hot takes on the Twitter thread started by him. Probably after a couple of weeks or more when he has processed what all has happened there, he may also share some nuances although many of his thoughts would probably go to his book on Comparative Constitutional Law which he hopes to publish maybe in 2021/2022 or whenever he can. Such televised proceedings are sure to alleviate the standing of Kenya internationally. There has been a proposal to do similar broadcasts by India but with surveillance built-in, so they know who is watching. The problems with the architecture and the surveillance built-in have been shared by Srinivas Kodali or DigitalDutta quite a few times, but that probably is a story for another day.

Uttar Pradesh Population Control Bill
Hindus comprise 83% of Indian couples with more than two child children
The U.P. Population Bill came and it came with lot of prejudices. One of the prejudices is the idea that Muslims create or procreate to have the most children. Even with data is presented as shared above from NFHS National Family Health Survey which is supposed to carry our surveys every few years did the last one around 4 years back. The analysis from it has been instrumental not only in preparing graphs as above but also sharing about what sort of death toll must have been in rural India. And as somebody who have had the opportunity in the past, can vouch that you need to be extremely lucky if something happens to you when you are in a rural area. Even in places like Bodh Gaya (have been there) where millions of tourists come as it is one of the places not to be missed on the Buddhism tourist circuit, the medical facilities are pretty underwhelming. I am not citing it simply because there are too many such newspaper reports from even before the pandemic, and both the State and the Central Govt. response has been dismal. Just a few months back, they were recalled. There were reports of votes being bought at INR 1000/- (around $14) and a bottle or two of liquor. There used to be a time when election monitoring whether national or state used to be a thing, and you had LTO s (Long-time Observers) and STO s (Short-Term Observers) to make sure that the election has been neutral. This has been on the decline in this regime, but that probably is for another time altogether. Although, have to point out the article which I had shared a few months ago on the private healthcare model is flawed especially for rural areas. Instead of going for cheap, telemedicine centers that run some version of a Linux distro. And can provide a variety of services, I know Kerala and Tamil Nadu from South India have experimented in past but such engagements need to be scaled up. This probably will come to know when the next time I visit those places (sadly due to the virus, not anytime soonish.:( ) . Going back to the original topic, though, I had shared Hans Rosling s famous Ted talk on population growth which shows that even countries which we would not normally associate with family planning for e.g. the middle-east and Africa have also been falling quite rapidly. Of course, when people have deeply held prejudices, then it is difficult. Even when sharing China as to how they had to let go of their old policy in 2016 as they had the thing for leftover men . I also shared the powerful movie So Long my Son. I even shared how in Haryana women were and are trafficked and have been an issue for centuries but as neither suits the RW propaganda, they simply refuse to engage. They are more repulsed by people who publish this news rather than those who are actually practicing it, as that is culture . There is also teenage pregnancy, female infanticide, sex-selective abortion, etc., etc. It is just all too horrible to contemplate. Personal anecdote I know a couple, or they used to be a couple, where the gentleman wanted to have a male child. It was only after they got an autistic child, they got their DNA tested and came to know that the gentleman had a genetic problem. He again forced and had another child, and that too turned out to be autistic. Finally, he left the wife and the children, divorced them and lived with another woman. Almost a decade of the wife s life was ruined. The wife before marriage was a gifted programmer employed at IBM. This was an arranged marriage. After this, if you are thinking of marrying, apart from doing astrology charts, also look up DNA compatibility charts. Far better than ruining yours or the women s life. Both the children whom I loved are now in heaven, god bless them  If one wants to, one can read a bit more about the Uttar Pradesh Population bill here. The sad part is that the systems which need fixing, nobody wants to fix. The reason being simple. If you get good health service by public sector, who will go to the private sector. In Europe, AFAIK they have the best medical bang for the money. Even the U.S. looks at Europe and hopes it had the systems that Europe has but that again is probably for another day.

South Africa and India long-lost brothers. As had shared before, after the 2016 South African Debconf convention, I had been following South Africa. I was happy when FeesMustFall worked and in 2017 the then ANC president Zuma declared it in late 2017. I am sure that people who have been regular visitors to this blog know how my position is on student loans. They also must be knowing that even in U.S. till the 1970s it had free education all the way to be a lawyer and getting a lawyer license. It is only when people like Thurgood Marshall, Martin Luther King Jr., and others from the civil rights movement came out as a major force that the capitalists started imposing fees. They wanted people who could be sold to corporate slavery, and they won. Just last week, Biden took some steps and canceled student loans and is working on steps towards broad debt forgiveness. Interestingly, NASA has an affirmative diversity program for people from diverse backgrounds, where a couple of UC (Upper Caste) women got the job. While they got the job, the RW (Right-Wing) was overjoyed as they got jobs on merit . Later, it was found that both the women were the third or fourth generation of immigrants in U.S.
NASA Federal Equal Opportunity Policy Directive NPD 3713 2H
Going back to the original question and topic, while there has been a concerning spate of violence, some calling it the worst sort of violence not witnessed since 1994. The problem, as ascertained in that article, is the same as here in India or elsewhere. Those, again, who have been on my blog know that merit 90% of the time is a function of privilege and there is a vast amount of academic literature which supports that. If, for a moment, you look at the data that is shared in the graph above which shows that 83% of Hindus and 13% of Muslims have more than 2 children, what does it show, it shows that 83+13 = 96% of the population is living in insecurity. The 5% are the ones who have actually consolidated more power during this regime rule in India. Similarly, from what I understood living in Cape Town for about a month, it is the Dutch Afrikaans as they like to call themselves and the immigrants who come from abroad who have enjoyed the fruits of tourism and money and power while the rest of the country is dying due to poverty. It is the same there, it is the same here. Corruption is also rampant in both countries, and the judiciary is virtually absent from both communities in India and SA. Interestingly, South Africa and India have been at loggerheads, but I suspect that is more due to the money and lobbying power by the Dutch. Usually, those who have money power, do get laws and even press on their side, and it is usually the ruling party in power. I cannot help but share about the Gupta brothers and their corruption as I came to know about it in 2016. And as have shared that I m related to Gupta s on my mother s side, not those specific ones but Gupta as a clan. The history of the Gupta dynasty does go back to the 3rd-4th century. Equally interesting have been Sonali Ranade s series of articles which she wrote in National Herald, the latest on exports which is actually the key to taking India out of poverty rather than anything else. While in other countries Exporters are given all sort of subsidies, here it is being worked as how to give them less. This was in Economic times hardly a week back
Export incentive schemes being reduced
I can t imagine the incredible stupidity done by the Finance Minister. And then in an attempt to prove that, they will attempt to present a rosy picture with numbers that have nothing to do with reality. Interestingly enough, India at one time was a major exporter of apples, especially from Kashmir. Now instead of exporting, we are importing them from Afghanistan as well as Belgium and now even from the UK. Those who might not want to use the Twitter link could use this article. Of course, what India got out of this trade deal is not known. One can see that the UK got the better deal from this. Instead of investing in our own capacity expansion, we are investing in increasing the capacity of others. This is at the time when due to fuel price hike (Central taxes 66%) demand is completely flat. And this is when our own CEA (Chief Economic Adviser) tells us that growth will be at the most 6-7% and that too in 2023-2024 while currently, the inflation rate is around 12%. Is it then any wonder that almost 70% are living on Govt. ration and people in the streets of Kolkata, Assam, and other places have to sell kidneys to make sure they have some money for their kids for tomorrow. Now I have nothing against the UK but trade negotiation is an art. Sadly, this has been going on for the last few years. The politicians in India fool the public by always telling of future trade deals. Sadly, as any businessman knows, once you have compromised, you always have to compromise. And the more you compromise, the more you weaken the hand for any future trade deals.
IIT pupil tries to sell kidney to repay loan, but no takers for Dalit organ.
The above was from yesterday s Times of India. Just goes to show how much people are suffering. There have been reports in vernacular papers of quite a few people from across regions and communities are doing this so they can live without pain a bit. Almost all the time, the politicians are saved as only few understand international trade, the diplomacy and the surrounding geopolitics around it. And this sadly, is as much to do with basic education as much as it is to any other factor

Suli Deals About a month back on the holy day of Ramzan or Ramadan as it is known in the west, which is beloved by Muslims, a couple of Muslim women were targeted and virtually auctioned. Soon, there was a flood and a GitHub repository was created where hundreds of Muslim women, especially those who have a voice and fearlessly talk about their understanding about issues and things, were being virtually auctioned. One week after the FIR was put up, to date none of the people mentioned in the FIR have been arrested. In fact, just yesterday, there was an open letter which was published by livelaw. I have saved a copy on WordPress just in case something does go wrong. Other than the disgust we feel, can t say much as no action being taken by GOI and police.

IT Rules 2021 and Big Media After almost a year of sleeping when most activists were screaming hoarsely about how the new IT rules are dangerous for one and all, big media finally woke up a few weeks back and listed a writ petition in Madras High Court of the same. Although to be frank, the real writ petition was filed In February 2021, classical singer, performer T.M. Krishna in Madras High Court. Again, a copy of the writ petition, I have hosted on WordPress. On 23rd June 2021, a group of 13 media outlets and a journalist have challenged the IT Rules, 2021. The Contention came from Digital News Publishers Association which is made up of the following news companies: ABP Network Private Limited, Amar Ujala Limited, DB Corp Limited, Express Network Pvt Ltd, HT Digital Streams Limited, IE Online Media Services Pvt Ltd, Jagran Prakashan Limited, Lokmat Media Private Limited, NDTV Convergence Limited, TV Today Network Limited, The Malayala Manorama Co (P) Ltd, Times Internet Limited, and Ushodaya Enterprises Private Limited. All the above are heavyweights in the markets where they operate. The reason being simple, when these media organizations came into being, the idea was to have self-regulation, which by and large has worked. Now, the present Govt. wants each news item to be okayed by them before publication. This is nothing but blatant misuse of power and an attempt at censorship. In fact, the Tamil Nadu BJP president himself made a promise of the same. And of course, what is true and what is a lie, only GOI knows and will decide for the rest of the country. If somebody remembers Joseph Goebbels at this stage, it is merely a coincidence. Anyways, 3 days ago Supreme Court on 14th July the Honorable Supreme Court asked the Madras High Court to transfer all the petitions to SC. This, the Madras High Court denied as cited/shared by Meera Emmanuel, a reporter who works with barandbench. The Court says nothing doing, let this happen and then the SC can entertain the motion of doing it that level. At the same time, they would have the benefit of Madras High Court opinion as well. It gave the center two weeks to file a reply. So, either of end-week of July or latest by August first week, we might be able to read the Center s reply on the same. The SC could do a forceful intervention, but it would lead to similar outrage as has been witnessed in the past when a judge commented that if the SC has to do it all, then why do we need the High Courts, district courts etc. let all the solutions come from SC itself. This was, admittedly, frustration on the part of the judge, but due in part to the needless intervention of SC time and time again. But the concerns had been felt around all the different courts in the country.

Sedition Law A couple of days ago, the Supreme Court under the guidance of Honorable CJI NV Ramanna, entertained the PIL filed by Maj Gen S G Vombatkere (Retd.) which asked simply that the sedition law which was used in the colonial times by the British to quell dissent by Mahatma Gandhi and Bal Gangadhar Tilak during the Indian freedom struggle. A good background filler article can be found on MSN which tells about some recent cases but more importantly how historically the sedition law was used to quell dissent during India s Independence. Another article on MSN actually elaborates on the PIL filed by Maj Gen S. G. Vombatkere. Another article on MSN tells how sedition law has been challenged and changed in 10 odd countries. I find it equally sad and equally hilarious that the Indian media whose job is to share news and opinion on this topic is being instead of being shared more by MSN. Although, I would be bereft of my duty if I did not share the editorial on the same topic by the Hindu and Deccan Chronicle. Also, an interesting question to ask is, are there only 10 countries in the world that have sedition laws? AFAIK, there are roughly 200 odd countries as recognized by WTO. If 190 odd countries do not have sedition laws, it also tells a lot about them and a lot about the remaining 10. Also, it came to light that police are still filing laws under sec66A which was declared null and void a few years ago. It was replaced with section 124A if memory serves right and it has more checks and balances.

Danish Siddiqui, Pulitzer award-winning and death in Afghanistan Before I start with Danish Siddiqui, let me share an anecdote that I think I have shared on the blog years ago about how photojournalists are. Again, those who know me and those who follow me know how much I am mad both about trains and planes (civil aviation). A few months back, I had shared a blog post about some of the biggest railway systems in the world which shows that privatization of Railways doesn t necessarily lead to up-gradation of services but definitely leads to an increase in tariff/fares. Just had a conversation couple of days ago on Twitter and realized that need to also put a blog post about civil aviation in India and the problems it faces, but I digress. This was about a gentleman who wanted to take a photo of a particular train coming out of a valley at a certain tunnel at two different heights, one from below and one from above the train. This was several years ago, and while I did share that award-winning photograph then, it probably would take me quite a bit of time and effort to again look it up on my blog and share. The logistics though were far more interesting and intricate than I had first even thought of. We came around a couple of days before the train was supposed to pass that tunnel and the valley. More than half a dozen or maybe more shots were taken throughout the day by the cameras. The idea was to see how much light was being captured by the cameras and how much exposure was to be given so that the picture isn t whitened out or is too black. Weather is the strangest of foes for a photojournalist or even photographers, and the more you are in nature, the more unpredictable it is and can be. We were also at a certain height, so care had to be taken in case light rainfall happens or dew falls, both not good for digital cameras. And dew is something which will happen regardless of what you want. So while the two days our gentleman cameraman fiddled with the settings to figure out correct exposure settings, we had one other gentleman who was supposed to take the train from an earlier station and apprise us if the train was late or not. The most ideal time would be at 0600 hrs. When the train would enter the tunnel and come out and the mixture of early morning sun rays, dew, the flowers in the valley, and the train would give a beautiful effect. We could stretch it to maybe 0700 hrs. Anything after that would just be useless, as it wouldn t have the same effect. And of all this depended on nature. If the skies were to remain too dark, nothing we could do about it, if the dewdrops didn t fall it would all be over. On the day of the shoot, we were told by our compatriot that the train was late by half an hour. We sank a little on hearing that news. Although Photoshop and others can do touch-ups, most professionals like to take as authentic a snap as possible. Everything had been set up to perfection. The wide-angle lenses on both the cameras with protections were set up. The tension you could cut with a knife. While we had a light breakfast, I took a bit more and went in the woods to shit and basically not be there. This was too tensed up for me. Returned an hour to find everybody in a good mood. Apparently, the shoot went well. One of the two captured it for good enough. Now, this is and was in a benign environment where the only foe was the environment. A bad shot would have meant another week in the valley, something which I was not looking forward to. Those who have lived with photographers and photojournalists know how self-involved they can be in their craft, while how grumpy they can be if they had a bad shoot. For those, who don t know, it is challenging to be friends with such people for a long time. I wish they would scream more at nature and let out the frustrations they have after a bad shoot. But again, this is in a very safe environment. Now let s cut to Danish Siddiqui and the kind of photojournalism he followed. He followed a much more riskier sort of photojournalism than the one described above. Krittivas Mukherjee in his Twitter thread shared how reporters in most advanced countries are trained in multiple areas, from risk assessment to how to behave in case you are kidnapped, are in riots, hostage situations, etc. They are also trained in all sorts of medical training from treating gunshot wounds, CPR, and other survival methods. They are supposed to carry medical equipment along with their photography equipment. Sadly, these concepts are unknown in India. And even then they get killed. Sadly, he attributes his death to the thrill of taking an exclusive photograph. And the gentleman s bio reads that he is a diplomat. Talk about tone-deafness  On another completely different level was Karen Hao who was full of empathy as she shared the humility, grace, warmth and kinship she describes in her interaction with the photojournalist. His body of work can be seen via his ted talk in 2020 where he shared a brief collage of his works. Latest, though in a turnaround, the Taliban have claimed no involvement in the death of photojournalist Danish Siddiqui. This could be in part to show the Taliban in a more favorable light as they do and would want to be showcased as progressive, even though they are forcing that all women within a certain age become concubines or marry the fighters and killing the minority Hazaras or doing vile deeds with them. Meanwhile, statements made by Hillary Clinton almost a decade, 12 years ago have come back into circulation which stated how the U.S. itself created the Taliban to thwart the Soviet Union and once that job was finished, forgot all about it. And then in 2001, it landed back in Afghanistan while the real terrorists were Saudi. To date, not all documents of 9/11 are in the public domain. One can find more information of the same here. This is gonna take probably another few years before Saudi Arabia s whole role in the September 11 attacks will be known. Last but not the least, came to know about the Pegasus spyware and how many prominent people in some nations were targeted, including in mine India. Will not talk more as it s already a big blog post and Pegasus revelations need an article on its own.

7 April 2021

Emmanuel Kasper: Manually install a single node Kubernetes cluster on Debian

Debian has work-in-progress packages for Kubernetes, which work well enough enough for a testing and learning environement. Bootstraping a cluster with the kubeadm deployer with these packages is not that hard, and is similar to the upstream kubeadm documentation

Install necessary packages in a VMInstall a throwaway VM with Vagrant.
apt install vagrant vagrant-libvirt
vagrant init debian/testing64
Bump the RAM and CPU of the VM, Kubernetes needs at least 2 gigs and 2 cores.
awk  -i inplace '1;/^Vagrant.configure\("2"\) do \ config/  print "  config.vm.provider :libvirt do  vm   vm.memory=2048 end" ' Vagrantfile
awk -i inplace '1;/^Vagrant.configure\("2"\) do \ config/ print " config.vm.provider :libvirt do vm vm.cpus=2 end" ' Vagrantfile
Start the VM, login, update the package index.
vagrant up
vagrant ssh
sudo apt update
Install a container engine, here we use docker.io, we could also use containerd (both are packaged in Debian) or cri-o.
sudo apt install --yes --no-install-recommends docker.io curl
Install kubernetes binaries. This will install kubelet, the system service which will manage the containers, and kubectl the user/admin tool to manage the cluster.
sudo apt install --yes kubernetes- node,client  containernetworking-plugins
Although it is not technically mandatory, we will use kubeadm, the most popular installer to create a Kubernetes cluster. Kubeadm is not packaged in Debian, we have to download an upstream binary.
wget https://dl.k8s.io/v1.20.5/kubernetes-server-linux-amd64.tar.gz

sha512sum kubernetes-server-linux-amd64.tar.gz
28529733bf34f5d5b72eabe30a81df98cc7f8e529590f807745cd67986a2c5c3eb86cebc7ecbcfc3df3c50416306e5d150948f2483933ea46c2aebaeb871ea8f kubernetes-server-linux-arm64.tar.gz

sudo tar --directory=/usr/local/sbin --strip-components 3 -xaf kubernetes-server-linux-amd64.tar.gz kubernetes/server/bin/kubeadm
sudo chmod +x /usr/local/sbin/kubeadm
sudo kubeadm version
kubeadm version: &version.Info Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"clean", BuildDate:"2021-03-18T01:08:27Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"
Add a kubelet systemd unit:
RELEASE_VERSION="v0.4.0"
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/$ RELEASE_VERSION /cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" sudo tee /etc/systemd/system/kubelet.service
sudo systemctl enable kubelet
and a default config file for kubeadm
RELEASE_VERSION="v0.4.0"
sudo mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/$ RELEASE_VERSION /cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
finally we need to help kubelet find the components needed for container networking
echo 'KUBELET_EXTRA_ARGS="--cni-bin-dir=/usr/lib/cni"'   sudo tee /etc/default/kubelet

Create a clusterInitialize a cluster with kubeadm: this will download container images for the Kubernetes control plane (= the brain of the cluster), and start the containers via the kubelet service. Yes a good part of Kubernetes itself run in containers.
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
...
...
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Follow the instructions from the kubeadm output, and verify you have a single node cluster, with the status NotReady.
kubectl get nodes 
NAME STATUS ROLES AGE VERSION
testing NotReady control-plane,master 9m9s v1.20.5
At that point you should also have a bunch of containers running on the node:
sudo docker ps --format ' .Names '
k8s_kube-apiserver_kube-apiserver-testing_kube-system_2711c230d39ccda1e74d1d6386a05cee_0
k8s_POD_kube-apiserver-testing_kube-system_2711c230d39ccda1e74d1d6386a05cee_0
k8s_etcd_etcd-testing_kube-system_4749b1bca3b1a73fd09c8e299d7030fe_0
k8s_POD_etcd-testing_kube-system_4749b1bca3b1a73fd09c8e299d7030fe_0
...
The kubelet service also needs an external network plugin to get the cluster in Ready state.
sudo systemctl status kubelet
...
Mar 28 09:28:43 testing kubelet[9405]: E0328 09:28:43.958059 9405 kubelet.go:2188] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Let s add that network plugin. Download the flannel network plugin definition, and schedule flannel to run on all nodes of your cluster:
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply --filename=kube-flannel.yml
After a dozen of seconds your node should be in ready status.
kubectl get nodes 
NAME STATUS ROLES AGE VERSION
testing Ready control-plane,master 16m v1.20.5

Deploy a test applicationOur node is now in Ready status, but we cannot run application on it, since we only have a master node, an administrative node which by default cannot run user applications.
kubectl describe node testing   grep ^Taints
Taints: node-role.kubernetes.io/master:NoSchedule
Let s allow node testing to run user applications:
kubectl taint node testing node-role.kubernetes.io/master-
Deploy a nginx container:
kubectl run my-nginx-pod --image=docker.io/library/nginx --port=80 --labels="app=http-content" 
Create a Kubernetes service to access this pod externally:
cat service.yaml

apiVersion: v1
kind: Service
metadata:
name: my-k8s-service
spec:
type: NodePort
ports:
- port: 80
nodePort: 30000
selector:
app: http-content

kubectl create --filename service.yaml
Access the service via IP adress:
curl 192.168.121.63:30000
...
Thank you for using nginx.

NotesI will try to get this blog post in a Debian Wiki article, or maybe in the kubernetes-node documentation. Blog posts deprecate and disappear, wiki and project docs live longer.

30 March 2021

Emmanuel Kasper: Playing with cri-o, a container runtime built for Kubernetes

Kubernetes is moving aways from docker to alternative container engines presenting a smaller core having just the functionality needed. The two most populars alternatives are:These alternatives are meant to be used programatically via a unix domain socket, and therefore have a limited command line interface.Let's play around in a VM.Install a throwaway VM with Vagrant
apt install vagrant vagrant-libvirt
vagrant init debian/testing64
Start the VM, install dependencies
vagrant up
vagrant ssh
sudo apt update
sudo apt install --yes curl gnupg jq
Install cri-o the container engine
sudo bash
export OS=Debian_Testing VERSION=1.20

echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /" > /etc/apt/sources.list.d/libcontainers.list
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /" > /etc/apt/sources.list.d/cri-o:$VERSION.list
curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key apt-key add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key apt-key add -
apt install cri-o cri-o-runc containernetworking-plugins conntrack
Verify it is running properly
systemctl restart cri-o
systemctl status cri-o
...
Started Container Runtime Interface for OCI (CRI-O).
Say hello to cri-o via its unix domain socket
curl --silent  --unix-socket /var/run/crio/crio.sock http://localhost/info   jq 

"storage_driver": "overlay",
"storage_root": "/var/lib/containers/storage",
"cgroup_driver": "systemd",
"default_id_mappings":
"uids": [

"container_id": 0,
"host_id": 0,
"size": 4294967295

],
"gids": [

"container_id": 0,
"host_id": 0,
"size": 4294967295

]


Install crictl, a Kubernetes debugging tool for containers
wget --directory-prefix=/tmp https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.20.0/crictl-v1.20.0-linux-amd64.tar.gz
tar -xaf /tmp/crictl-v1.20.0-linux-amd64.tar.gz -C /usr/local/sbin/
chmod +x /usr/local/sbin/crictl

crictl info

"status":
"conditions": [

"type": "RuntimeReady",
"status": true,
"reason": "",
"message": ""
,

"type": "NetworkReady",
"status": true,
"reason": "",
"message": ""

]



From there on you can create a container following the examples in https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md

28 March 2021

Emmanuel Kasper: Switching to FAI (Fully Automatic Installer) for creating Vagrant Boxes

Have you heard of Vagrant ? It is a command line tool to get ready to use, disposable Virtual Machines (VM) from an online catalog. Vagrant works on Linux, FreeBSD, Windows and Mac and you only need three commands to get a shell prompt in a VM (see the Debian wiki).
The online catalog has images for the majority of the OSes you can think of.

We've been building the Debian disk images for Vagrant (available on https://app.vagrantup.com/debian/) with a number of tools over the years:Basically there are two category of tools for building a disk image:- those using an emulator and the OS installer in a automated way- those using debootstrap/pacstrap/rpmstrap on a loopback mounted filesystemPersonally I prefer the first approach, as you can run the build process as non root, and you benefit from all the quality work of the official installer.
However this requires virtualization, and nested virtualization if your build process run insides a VM. Unfortunately nested virtualization is not that common, for instance my cloud provider, and the VMs used for Debian Continuous Integration, are not supporting nested virtualization.
As the maintainer of fai-diskimage is a Debian Developer (hey MrFAI ! :) and as the debian-cloud folks are it using for Amazon, Azure and Google Cloud Debian images, it made sense to switch to fai-diskimage for now. The fai-diskimage learning curve is a bit steep as you have to learn many internal concepts before using it, but once you get the bits connected it works quite well.

9 March 2021

Emmanuel Kasper: Displaying CSV files in a readable way on the terminal

Until this week I did not know about the column command.

$ head -5 zillow.csv
"Index", "Living Space (sq ft)", "Beds", "Baths", "Zip", "Year", "List Price ($)"
1, 2222, 3, 3.5, 32312, 1981, 250000
2, 1628, 3, 2, 32308, 2009, 185000
3, 3824, 5, 4, 32312, 1954, 399000
4, 1137, 3, 2, 32309, 1993, 150000

Turned out this file is much more readable with a good pipe (and a large screen)
$ head -5 zillow.csv column --table --separator ,
"Index" "Living Space (sq ft)" "Beds" "Baths" "Zip" "Year" "List Price ($)"
1 2222 3 3.5 32312 1981 250000
2 1628 3 2 32308 2009 185000
3 3824 5 4 32312 1954 399000
4 1137 3 2 32309 1993 150000

column is part of util-linux and is thus available in all distributions.
Example file taken from this example list.

30 January 2021

Emmanuel Kasper: Playing Tetris over serial console

Today I played Tetris over a serial console connection, on a Vax 4000 running OpenBSD. I haven't felt that 1337 since a long time.
I am going to get rid of that Vax system though. If that's your stuff, contact me privately.

asciinema in its greatness:

3 January 2021

Emmanuel Kasper: How to move a single VM between cloud providers

I am running since a decade a small Debian VM, that I use for basic web and mail hosting. Since most of the VM setup is done manually and not following the Infrastructure As Code pattern, it is faster to simply copy the filesystem when switching providers instead of reconfiguring everything.
The steps involved are:1. create a backup of the filesystem using tar of rsync, excluding dynamic content
rsync --archive \
--one-file-system --numeric-ids \
--rsh "ssh -i private_key root@server:/ /local_dir

or
tar -cvpzf backup.tar.gz \
--numeric-owner \
--exclude=/backup.tar.gz \
--one-file-system /


Notice here the --one-file-system switch which avoids back'ing up the content of mount points like /proc, /dev.
If you have extra partitions with a mounted filesystem, like /boot or home you need do add a separate backup for those.

2. create a new VM on the new cloud provider, verify you have a working console access, and power it off.
3. boot on the new cloud provider a rescue image
4. partition the disk image on the new provider.
5. mount the new root partition, and untar your backup on it. You could for instance push the local backup via rsync, or download the tar archive using https.
6. update network configuration and /etc/fstab
7. chroot into the target system, and reinstall grub

This works surprisingly well, and you if made your backup locally, you can test the whole procedure by building a test VM with your backup. Just replace the deboostrap step with a command like tar -xvpzf /path/to/backup.tar.gz -C /mount_point --numeric-owner

Using this procedure, I moved from Hetzner (link in French language) to Digital Ocean, from Digital Ocean to Vultr, and now back at Hetzner.

28 December 2020

Emmanuel Kasper: Quick NetBSD serial console install on libvirt

I wanted to set up a small VM with NetBSD to test a couple of virt-install option. It turns out it you can get to the installer prompt quite fast. get the NetBSD installer for serial console:
wget https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.1/i386/installation/cdrom/boot-com.iso 
start the install
$ virt-install \
--connect qemu:///session \
--name netbsd \
--ram 64 \
--vcpus 2 \
--disk path=$HOME/netbsd.qcow2,size=4,bus=scsi,format=qcow2 \
--controller type=scsi,model=virtio-scsi \
--cdrom=boot-com.iso \
--virt-type kvm \
--os-variant netbsd8.0 \
--graphics none \
--arch i686 \
--console pty,target_type=serial
 
This will start a VM in usermode networking, so no need to be root, but the VM won t be reachable from the outside world, except if you add qemu usermode port forwarding.

14 September 2020

Emmanuel Kasper: Quick debugging of a Linux printer via cups command line tools

Step by step cups debugging ( here with a network printer)

Which printer queue do I have configured ?
lpstat -p
printer epson is idle. enabled since Sat Dec 24 13:18:09 2017
#here I have a printer called 'epson", doing nothing, that the cups daemon considers as enabled

Which connection am I using to get to this printer ?
lpstat -v
device for epson: lpd://epson34dea0.local:515/PASSTHRU
# here the locally configured 'epson' printer queue is backed by a network device at the adress epson34dea0.local, to which I am sending my print jobs via the lpd protocol

Is my printer ready ?
lpq
epson is ready
no entries

# here my local print queue 'epson' is accepting print jobs ( which does not say anything about the physical device, it might be offline

If here you local print queue 'epson' is not ready, you can try to reenable it in the cups system with:

sudo cupsenable epson

If you notice that the printer is disabled all the time, because for instance of a flaky network, you can edit /etc/cups/printers.conf and change the ErrorPolicy for each printer from stop-printer to retry-job.
It should be also possible to set this parameter in cupsd.conf

Finally you can print a test page with
lpr /usr/share/cups/data/testprint

Emmanuel Kasper: Using Debian and RHEL troubleshootings containers on Kubernetes & OpenShift

You can connect to a running pod with oc/kubectl rsh pod_name, or start a copy of a running pod with oc debug pod_name, but as best practises recommend unprivileged, slim container images, where do you get sosreport, kdump, dig and nmap for troubleshooting ? Fortunately you can start either a transient Debian troubleshooting container with:oc run troubleshooting-pod --stdin --tty --rm --image=docker.io/library/debian:busteror a Red Hat Entreprise Linux:
oc run troubleshooting-pod --stdin --tty --rm --image=registry.access.redhat.com/rhel7/rhel-tools

14 August 2020

Markus Koschany: My Free Software Activities in July 2020

Welcome to gambaru.de. Here is my monthly report (+ the first week in August) that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games
Debian Java Misc Debian LTS This was my 53. month as a paid contributor and I have been paid to work 15 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following: ELTS Extended Long Term Support (ELTS) is a project led by Freexian to further extend the lifetime of Debian releases. It is not an official Debian project but all Debian users benefit from it without cost. The current ELTS release is Debian 8 Jessie . This was my 26. month and I have been paid to work 13,25 hours on ELTS. Thanks for reading and see you next time.

2 July 2020

Emmanuel Kasper: Test a webcam from the command line on Linux with VLC

Since this info was too well hidden on the internet, here is the information:
cvlc v4l2:///dev/video0
and there you go.
If you have multiple cameras connected, you can try /dev/video0 up to /dev/video5

30 June 2020

Emmanuel Kasper: Learning openshift: a good moment to revisit awk too

I can t believe I spent all these years using only grep. Most of us know how to use awk to print the nth column of a file:
$ awk ' print $1 ' /etc/hosts
will print all IP addresses from /etc/hosts But you can also do filtering before printing the chosen column:
$ awk '$5 >= 2  print $2 ' /path/to/file
will print the second column of all lines, where the 5th column is greater than 2. That would have been hard with grep. Now I can use that to find out all deployments on my openshift cluster, where the number of current replicas is greater than 2.
$ oc get deployments --all-namespaces   awk '$5 >= 2  print $2 '
NAME
oauth-openshift
console
downloads
router-default
etcd-quorum-guard
prometheus-adapter
thanos-querier
packageserver
I know that openshift/kubernetes both have a powerful query selector syntax, but for the moment awk will do.

4 June 2020

Reproducible Builds: Reproducible Builds in May 2020

Welcome to the May 2020 report from the Reproducible Builds project. One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. Nonetheless, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes. In these reports we outline the most important things that we and the rest of the community have been up to over the past month.

News The Corona-Warn app that helps trace infection chains of SARS-CoV-2/COVID-19 in Germany had a feature request filed against it that it build reproducibly. A number of academics from Cornell University have published a paper titled Backstabber s Knife Collection which reviews various open source software supply chain attacks:
Recent years saw a number of supply chain attacks that leverage the increasing use of open source during software development, which is facilitated by dependency managers that automatically resolve, download and install hundreds of open source packages throughout the software life cycle.
In related news, the LineageOS Android distribution announced that a hacker had access to the infrastructure of their servers after exploiting an unpatched vulnerability. Marcin Jachymiak of the Sia decentralised cloud storage platform posted on their blog that their siac and siad utilities can now be built reproducibly:
This means that anyone can recreate the same binaries produced from our official release process. Now anyone can verify that the release binaries were created using the source code we say they were created from. No single person or computer needs to be trusted when producing the binaries now, which greatly reduces the attack surface for Sia users.
Synchronicity is a distributed build system for Rust build artifacts which have been published to crates.io. The goal of Synchronicity is to provide a distributed binary transparency system which is independent of any central operator. The Comparison of Linux distributions article on Wikipedia now features a Reproducible Builds column indicating whether distributions approach and progress towards achieving reproducible builds.

Distribution work In Debian this month: In Alpine Linux, an issue was filed and closed regarding the reproducibility of .apk packages. Allan McRae of the ArchLinux project posted their third Reproducible builds progress report to the arch-dev-public mailing list which includes the following call for help:
We also need help to investigate and fix the packages that fail to reproduce that we have not investigated as of yet.
In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update.

Software development

diffoscope Chris Lamb made the changes listed below to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. He also prepared and uploaded versions 142, 143, 144, 145 and 146 to Debian, PyPI, etc.
  • Comparison improvements:
    • Improve fuzzy matching of JSON files as file now supports recognising JSON data. (#106)
    • Refactor .changes and .buildinfo handling to show all details (including the GnuPG header and footer components) even when referenced files are not present. (#122)
    • Use our BuildinfoFile comparator (etc.) regardless of whether the associated files (such as the orig.tar.gz and the .deb) are present. [ ]
    • Include GnuPG signature data when comparing .buildinfo, .changes, etc. [ ]
    • Add support for printing Android APK signatures via apksigner(1). (#121)
    • Identify iOS App Zip archive data as .zip files. (#116)
    • Add support for Apple Xcode .mobilepovision files. (#113)
  • Bug fixes:
    • Don t print a traceback if we pass a single, missing argument to diffoscope (eg. a JSON diff to re-load). [ ]
    • Correct differences typo in the ApkFile handler. (#127)
  • Output improvements:
    • Never emit the same id="foo" anchor reference twice in the HTML output, otherwise identically-named parts will not be able to linked to via a #foo anchor. (#120)
    • Never emit an empty id anchor either; it is not possible to link to #. [ ]
    • Don t pretty-print the output when using the --json presenter; it will usually be too complicated to be readable by the human anyway. [ ]
    • Use the SHA256 over MD5 hash when generating page names for the HTML directory-style presenter. (#124)
  • Reporting improvements:
    • Clarify the message when we truncate the number of lines to standard error [ ] and reduce the number of maximum lines printed to 25 as usually the error is obvious by then [ ].
    • Print the amount of free space that we have available in our temporary directory as a debugging message. [ ]
    • Clarify Command [ ] failed with exit code messages to remove duplicate exited with exit but also to note that diffoscope is interpreting this as an error. [ ]
    • Don t leak the full path of the temporary directory in Command [ ] exited with 1 messages. (#126)
    • Clarify the warning message when we cannot import the debian Python module. [ ]
    • Don t repeat stderr from if both commands emit the same output. [ ]
    • Clarify that an external command emits for both files, otherwise it can look like we are repeating itself when, in reality, it is being run twice. [ ]
  • Testsuite improvements:
    • Prevent apksigner test failures due to lack of binfmt_misc, eg. on Salsa CI and elsewhere. [ ]
    • Drop .travis.yml as we use Salsa instead. [ ]
  • Dockerfile improvements:
    • Add a .dockerignore file to whitelist files we actually need in our container. (#105)
    • Use ARG instead of ENV when setting up the DEBIAN_FRONTEND environment variable at runtime. (#103)
    • Run as a non-root user in container. (#102)
    • Install/remove the build-essential during build so we can install the recommended packages from Git. [ ]
  • Codebase improvements:
    • Bump the officially required version of Python from 3.5 to 3.6. (#117)
    • Drop the (default) shell=False keyword argument to subprocess.Popen so that the potentially-unsafe shell=True is more obvious. [ ]
    • Perform string normalisation in Black [ ] and include the Black output in the assertion failure too [ ].
    • Inline MissingFile s special handling of deb822 to prevent leaking through abstract layers. [ ][ ]
    • Allow a bare try/except block when cleaning up temporary files with respect to the flake8 quality assurance tool. [ ]
    • Rename in_dsc_path to dsc_in_same_dir to clarify the use of this variable. [ ]
    • Abstract out the duplicated parts of the debian_fallback class [ ] and add descriptions for the file types. [ ]
    • Various commenting and internal documentation improvements. [ ][ ]
    • Rename the Openssl command class to OpenSSLPKCS7 to accommodate other command names with this prefix. [ ]
  • Misc:
    • Rename the --debugger command-line argument to --pdb. [ ]
    • Normalise filesystem stat(2) birth times (ie. st_birthtime) in the same way we do with the stat(1) command s Access: and Change: times to fix a nondeterministic build failure in GNU Guix. (#74)
    • Ignore case when ordering our file format descriptions. [ ]
    • Drop, add and tidy various module imports. [ ][ ][ ][ ]
In addition:
  • Jean-Romain Garnier fixed a general issue where, for example, LibarchiveMember s has_same_content method was called regardless of the underlying type of file. [ ]
  • Daniel Fullmer fixed an issue where some filesystems could only be mounted read-only. (!49)
  • Emanuel Bronshtein provided a patch to prevent a build of the Docker image containing parts of the build s. (#123)
  • Mattia Rizzolo added an entry to debian/py3dist-overrides to ensure the rpm-python module is used in package dependencies (#89) and moved to using the new execute_after_* and execute_before_* Debhelper rules [ ].

Chris Lamb also performed a huge overhaul of diffoscope s website:
  • Add a completely new design. [ ][ ]
  • Dynamically generate our contributor list [ ] and supported file formats [ ] from the main Git repository.
  • Add a separate, canonical page for every new release. [ ][ ][ ]
  • Generate a latest release section and display that with the corresponding date on the homepage. [ ]
  • Add an RSS feed of our releases [ ][ ][ ][ ][ ] and add to Planet Debian [ ].
  • Use Jekyll s absolute_url and relative_url where possible [ ][ ] and move a number of configuration variables to _config.yml [ ][ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Other tools Elsewhere in our tooling: strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In May, Chris Lamb uploaded version 1.8.1-1 to Debian unstable and Bernhard M. Wiedemann fixed an off-by-one error when parsing PNG image modification times. (#16) In disorderfs, our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues, Chris Lamb replaced the term dirents in place of directory entries in human-readable output/log messages [ ] and used the astyle source code formatter with the default settings to the main disorderfs.cpp source file [ ]. Holger Levsen bumped the debhelper-compat level to 13 in disorderfs [ ] and reprotest [ ], and for the GNU Guix distribution Vagrant Cascadian updated the versions of disorderfs to version 0.5.10 [ ] and diffoscope to version 145 [ ].

Project documentation & website
  • Carl Dong:
  • Chris Lamb:
    • Rename the Who page to Projects . [ ]
    • Ensure that Jekyll enters the _docs subdirectory to find the _docs/index.md file after an internal move. (#27)
    • Wrap ltmain.sh etc. in preformatted quotes. [ ]
    • Wrap the SOURCE_DATE_EPOCH Python examples onto more lines to prevent visual overflow on the page. [ ]
    • Correct a preferred spelling error. [ ]
  • Holger Levsen:
    • Sort our Academic publications page by publication year [ ] and add Trusting Trust and Fully Countering Trusting Trust through Diverse Double-Compiling [ ].
  • Juri Dispan:

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced. Holger Levsen made the following changes:
  • System health status:
    • Improve page description. [ ]
    • Add more weight to proxy failures. [ ]
    • More verbose debug/failure messages. [ ][ ][ ]
    • Work around strangeness in the Bash shell let VARIABLE=0 exits with an error. [ ]
  • Debian:
    • Fail loudly if there are more than three .buildinfo files with the same name. [ ]
    • Fix a typo which prevented /usr merge variation on Debian unstable. [ ]
    • Temporarily ignore PHP s horde](https://www.horde.org/) packages in Debian bullseye. [ ]
    • Document how to reboot all nodes in parallel, working around molly-guard. [ ]
  • Further work on a Debian package rebuilder:
    • Workaround and document various issues in the debrebuild script. [ ][ ][ ][ ]
    • Improve output in the case of errors. [ ][ ][ ][ ]
    • Improve documentation and future goals [ ][ ][ ][ ], in particular documentiing two real world tests case for an impossible to recreate build environment [ ].
    • Find the right source package to rebuild. [ ]
    • Increase the frequency we run the script. [ ][ ][ ][ ]
    • Improve downloading and selection of the sources to build. [ ][ ][ ]
    • Improve version string handling.. [ ]
    • Handle build failures better. [ ]. [ ]. [ ]
    • Also consider architecture all .buildinfo files. [ ][ ]
In addition:
  • kpcyrd, for Alpine Linux, updated the alpine_schroot.sh script now that a patch for abuild had been released upstream. [ ]
  • Alexander Couzens of the OpenWrt project renamed the brcm47xx target to bcm47xx. [ ]
  • Mattia Rizzolo fixed the printing of the build environment during the second build [ ][ ][ ] and made a number of improvements to the script that deploys Jenkins across our infrastructure [ ][ ][ ].
Lastly, Vagrant Cascadian clarified in the documentation that you need to be user jenkins to run the blacklist command [ ] and the usual build node maintenance was performed was performed by Holger Levsen [ ][ ][ ], Mattia Rizzolo [ ][ ] and Vagrant Cascadian [ ][ ][ ].

Mailing list: There were a number of discussions on our mailing list this month: Paul Spooren started a thread titled Reproducible Builds Verification Format which reopens the discussion around a schema for sharing the results from distributed rebuilders:
To make the results accessible, storable and create tools around them, they should all follow the same schema, a reproducible builds verification format. The format tries to be as generic as possible to cover all open source projects offering precompiled source code. It stores the rebuilder results of what is reproducible and what not.
Hans-Christoph Steiner of the Guardian Project also continued his previous discussion regarding making our website translatable. Lastly, Leo Wandersleb posted a detailed request for feedback on a question of supply chain security and other issues of software review; Leo is the founder of the Wallet Scrutiny project which aims to prove the security of Android Bitcoin Wallets:
Do you own your Bitcoins or do you trust that your app allows you to use your coins while they are actually controlled by them ? Do you have a backup? Do they have a copy they didn t tell you about? Did anybody check the wallet for deliberate backdoors or vulnerabilities? Could anybody check the wallet for those?
Elsewhere, Leo had posted instructions on his attempts to reproduce the binaries for the BlueWallet Bitcoin wallet for iOS and Android platforms.


If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Jelle van der Waa and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

1 June 2020

Debian GSoC Kotlin project blog: Kotlin Update

A Quick Recap from last year: Kotlin is being packaged under the Google Summer of Code within the Debian organization itself. The major reason behind bringing Kotlin in Debian is to update all the Android packages which are now heavily dependent upon the Kotlin libraries. The major work to bring Kotlin into Debian is done for the version 1.3.30, by Saif Abdul Cassim (goes by m36 on IRC) as a part of his GSoC'2019. All his contributions to the team can be found in his blog posts. So, for now, we have a bootstrap package and a Kotlin package for the version with 1.3.30. There were still changes needed as we lacked some of the dependencies for Kotlin, and the source package lacked copyright information and didn t comply with Debian standards. What's the present year brought for Kotlin? To be specific the following were mainly left dependencies for Kotlin: And, we lack documentation for the newbies in order to get them started :( Most importantly the crucial part was and still is, to figure out how to upload the package? For GSoC'20, three students are selected as a part of project Android SDK tools in Debian. What's the work done/left? Work Done What's Blocking? What is the problem being faced? The Kotlin-Bootstrap package consists of JAR files for various dependencies of kotlin such as Gradle, kotlin compiler, and kotlinx. The package is added to the build-depends of the main package so that the JAR files can be provided. Since the kotlin-bootstrap consists of binaries (JAR files), it is not feasible to upload the package as free software. The other workaround was the Gradle 6.4 version, which consists of Kotlin files and generates a suitable JAR. But since the package needed Kotlin language itself, it was never updated, as it created a cyclic dependency. Final workaround came, which proposed Kotlin to build from itself, that was a pretty impressive suggestion. But, we still have to look if the solution is feasible? Because, as far as I last checked and conversed with ebourg on the mailing list here, Emmanuel Bbourg mentioned very clearly that the rebuilt package is our interest. So, this is under WIP. But, I fail to acknowledge the fact if we can drop the kotlin-bootstrap package totally, Kotlin will not be able to be built because each and every JAR file present in the bootstrap is needed. That pretty much is the ongoing work and the update on the kotlin package. We intend to bring Kotlin to the Debian Archive as soon as possible :) Have any queries or suggestions for Kotlin? Please feel to drop a message at #debian-mobile channel on OFTC.

23 May 2020

Reproducible Builds (diffoscope): diffoscope 145 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 145. This version includes the following changes:
[ Chris Lamb ]
* Improvements:
  - Add support for Apple Xcode mobile provisioning .mobilepovision files.
    (Closes: reproducible-builds/diffoscope#113)
  - Add support for printing the signatures via apksigner(1).
    (Closes: reproducible-builds/diffoscope#121)
  - Use SHA256 over MD5 when generating page names for the HTML directory
    presenter, validate checksums for files referenced in .changes files
    using SHA256 too, and move to using SHA256 in "Too much input for diff"
    output too. (Closes: reproducible-builds/diffoscope#124)
  - Don't leak the full path of the temporary directory in "Command [..]
    exited with 1".  (Closes: reproducible-builds/diffoscope#126)
  - Identify "iOS App Zip archive data" files as .zip files.
    (Closes: reproducible-builds/diffoscope#116)
* Bug fixes:
  - Correct "differences" typo in the ApkFile handler.
    (Closes: reproducible-builds/diffoscope#127)
* Reporting/output improvements:
  - Never emit the same id="foo" TML anchor reference twice, otherwise
    identically-named parts will not be able to linked to via "#foo".
    (Closes: reproducible-builds/diffoscope#120)
  - Never emit HTML with empty "id" anchor lements as it is not possible to
    link to "#" (vs "#foo"). We use "#top" as a fallback value so it will
    work for the top-level parent container.
  - Clarify the message when we cannot find the "debian" Python module.
  - Clarify "Command [..] failed with exit code" to remove duplicate "exited
    with exit" but also to note that diffoscope is interpreting this as an
    error.
  - Add descriptions for the 'fallback' Debian module file types.
  - Rename the --debugger command-line argument to --pdb.
* Testsuite improvements:
  - Prevent CI (and runtime) apksigner test failures due to lack of
    binfmt_misc on Salsa CI and elsewhere.
* Codebase improvements:
  - Initially add a pair of comments to tidy up a slightly abstraction level
    violating code in diffoscope.comparators.mising_file and the
    .dsc/.buildinfo file handling, but replace this later by by inlining
    MissingFile's special handling of deb822 to prevent leaking through
    abstraction layers in the first place.
  - Use a BuildinfoFile (etc.) regardless of whether the associated files
    such as the orig.tar.gz and the .deb are present, but don't treat them as
    actual containers. (Re: reproducible-builds/diffoscope#122)
  - Rename the "Openssl" command class to "OpenSSLPKCS7" to accommodate other
    commands with this prefix.
  - Wrap a docstring across multiple lines, drop an inline pprint import and
    comment the HTMLPrintContext class, etc.
[ Emanuel Bronshtein ]
* Avoid build-cache in building the released Docker image.
  (Closes: reproducible-builds/diffoscope#123)
[ Holger Levsen ]
* Wrap long lines in older changelog entries.
You find out more by visiting the project homepage.

14 April 2020

Emmanuel Kasper: Recommended keyboard settings for Productivity and Usability, for European Programmers

TLDR: setxkbmap -layout us -variant altgr-intl and become a happier programmer.
The case for QWERTY for European ProgrammersIf you re working on Unix / Linux, or C based programming languages, it can make sense to switch to the qwerty(us) keyboard layout. Why ?
Unix, C, Perl, Java, and most of programming languages have been conceived on QWERTY keyboards.
So when the designers choose special characters to use for the language synthax, they simply choose what was easy to access on their own keyboard. This has been historically documented for the vi editor.

To give an example, using an Unix shell you have to type the dot . and slash / symbols quite often to navigate the filesystem. The two keys producing these symbols, are nicely aligned on a QWERTY layout and do not require a key combination to be entered. So you can quickly enter something like ../.. using a single hand.
Now using a QWERTZ layout, like in Germany / Austria, you have the . symbol easily accessible, but you need to combine two keys ( Shift + 7 ) to get a / .
And if you are a poor soul using an AZERTY layout, to get the . and / symbol you need each time a key combo.
The need of key combos is bad not only for speed (multiple keys to lookup) but also for usability, as you have to stretch your fingers to reach the key if using a single hand, provoking repetitive strain injury. You might be smiling but this is commonly known amongst Emacs Users, due to the prominent use of commands using Ctrl and Alt combos, and led to the creation of an Emacs Ergonomic wiki.

This goes as well for many symbol commonly used in programming languages, think for instance about the semicolon ; for terminating a statement, the [ , and ] symbols for defining an array, and the backslash \ for escaping.
All these keys are accessible via a single keypress on qwerty and require key combos on qwertz and azerty. No wonder Linux, Minix and BSD were invented on non-azerty layouts: in France we were still busy typing the path to the source code, when in other parts of the world people already had the file open in their editor.
You don t need to throw away your existing keyboards when learning the qwerty (us) layout: for a couple of euros, you can find on ebay alternate keys stickers to put on your laptop.
Accessing keys with diacritics with the AltGr International variantNow if you want to switch to a qwerty keyboard layout, and you re French or German, you might wonder how to access the characters with diacritics, the and of French and and of German.
Fortunately there is a very clever keyboard variant for the us layout who uses the AltGr key, to make all these keys accessible, just hiding them behind the AltGr key.
Need ? that s just AltGr + e away. Needs ? That s just AltGr + , Needs ? AltGr +p
See the coolness of that ? You can type all international diacritics in an easy way, and there are even keyboard stickers for that too.

Now how to access to this layout of wonder ?
You can configure the layout and variant in Xorg, in debian/ubuntu that would be entering in /etc/default/keyboard
XKBLAYOUT="us"
XKBVARIANT="altgr-intl"
After restarting the X server, you can check that the settings have been applied with
setxkbmap -print -verbose 10
If using Gnome, you can also set the keyboard layout and variant by changing the schema org.gnome.desktop-inputsources, which will override the desktop-agnostic settings of /etc/default/keyboard.
For this you can either call
dconf write /org/gnome/desktop/input-sources/sources "[('xkb', 'us+altgr-intl')]"
or navigate with the gui tool dconf-settings to org.gnome.desktop-inputsources and set the value there.

If you want to further improve your keyboard layout, you can also have a look at swapping Ctrl and Alt, as described here (French language article) Happy hacking !

Next.