Search Results: "tar"

30 January 2023

Russell Coker: Links January 2023

The Intercept has an amusing and interesting article about senior Facebook employees testifying that they don t know where Facebook stores all it s data on users [1]. One lesson all programmers can learn from this is to document all these things in an orderly manner. Cory Doctorow wrote a short informative article about inflation from a modern monetary theory perspective [2]. Russ Allbery wrote an insightful blog post about effecive altruism and respect for disadvantaged people [3]. GiveDirectly sounds good. The Conversation has an interesting article about the Google and Apple app stores providing different versions of apps for users in different regions [4]. Apparently there are specific versions to comply with GDPR and versions that differ in adverts. The hope that GDPR would affect enough people to become essentially a world-wide standard was apparently overly optimistic. We need political lobbying in all countries for laws like the GDPR to force the app stores to give us the better versions of apps. Arya Voronova wrote an informative article about USB-C and extension or data blocker cables [5]. USB just keeps getting more horrible in technology while getting more useful in functionality. Laptops and phones catching fire will probably become more common in future. John McBride wrote an insightful article about the problems in the security of the software supply chain [6]. His main suggestion for addressing problems is If you are on a team that relies on some piece of open source software, allocate real engineering time to contributing , the problem with this is that real engineering time means real money and companies don t want to do that. Maybe having companies contribute moderate amounts of money to a foundation that hires people would be a viable option. Toms Guide has an interesting article describing problems with the Tesla [7]. It doesn t cover things like autopilot driving over children and bikers but instead covers issues of the user interface that make it less pleasant to drive and also remove concentration from the road. The BBC has an interesting article about the way mathematical skill is correlated with the way language is used to express numbers [8]. Every country with a lesser way of expressing numbers should switch to some variation of the East-Asian way. Science 2.0 has an interesting blog post about the JP Aerospace plans to use airships to get most of the way through the atmosphere and then a plane to get to orbit [9]. It s a wild idea but seems plausible. The idea of going to space in balloons seems considerably scarier to me than the current space craft. Interesting list of red team and physical entry gear with links to YouTube videos showing how to use them [10]. The Verge has an informative summary of the way Elon mismanaged Twitter after taking it over [11].

29 January 2023

Valhalla's Things: Hello World!

Posted on January 29, 2023
Welcome to my new blog! Or rather, strictly speaking, welcome to my first blog! Back when everybody had a blog, I had an old-fashioned personal website where pages were organized by topic rather than by date, so now that blogs are dead (or so they say), I guess it s time for me to have one :) . The old website is still online, but updating it is getting harder, both for organizational reasons and because the static generator I ve used is no longer supported and requires python 2; lately I ve started publish some specific categories of material such as sewing patterns into their own website (you can find a list in the about page), but I was missing a place to post about the history and experiences of the things I publish elsewhere, as well as more uncategorized things. Of course I ve chosen to use a static site generator, but since I m picky I ve discarded most of the common ones mostly because they enforce assumptions I don t agree with. On the other hand, one day I wouldn t mind learning a bit of Haskell, so I decided to look for trouble and use Hakyll, hoping that nobody will add a level 20 wandering monster to it. Right now I m using mostly the default theme, because I know that if I start fiddling with it I m not going to start posting content ever; maybe one day I ll decide to completely change the look. As for contents, on the bits side of things you can expect me to talk about Debian, the Fediverse, Python, my inventory manging program and a bit of arduino-level electronics; on the atom side you should definitely expect sewing, modern and mostly historically inspired, some fiber crafts (spinning, knitting, crochet), maybe some bad attempts at painting (watercolours and acrilics), printing (screen and linocut), things that are written on paper and, well, any other craft I will happen to collect. Before the blog gets too full I plan to add tag management to help people who are only interested in some of these contents. This being a blog, of course it has an atom feed you can add to your favourite (and ideally self-hosted) rss reader, and since this is a personal blog there will be no periodicity, posts will happen when I ll have something to say on some topic. Until next time

28 January 2023

Scarlett Gately Moore: KDE Snaps, Debian uploads and much more in the works!

Witch Wells, AZ SnowWitch Wells, AZ Snow
It has been a very busy few weeks as we endured snowstorm after snowstorm! I have made some progress on the Mycroft in debian adventure! This will slow down as we enter freeze for bookworm and there is no way we will make it into bookworm as there are some significant issues to solve. On the KDE side of things: In the Snap arena, I have made my first significant contribution to snapcraft upstream! It has been a great learning experience as I convert my Ruby knowledge to Python. Formatting is something I need to get used to! https://github.com/snapcore/snapcraft/pull/4023 Snaps have been on hold due to the kde-neon extension not having core22 support and the above pull request fixes that. Meanwhile, I have been working on getting core20 apps ( 22.08.3 final KDE apps version for this base. ) rebuilt for security updates. As many of you know, I am seeking employment. I am a hard worker, that thrives on learning new things. I am a self starter, knowledge sponge, and eager to be an asset to < insert your company here > ! Meanwhile, as interview processes are much longer than I remember and the industry exploding in layoffs, I am coming up short on living expenses as my unemployment lingers on. Please consider donating to my gofundme. Thank you for your consideration. GoFundMe

Emmanuel Kasper: Table of correspondence between AWS / Azure / Red Hat OpenShift Container Platform / upstream projects

If you know the Amazon Web Services or Azure portfolio, and you are interested in OpenShift or the OKD OpenShift community distribution, this is a table of corresponding technologies. OpenShift is Red Hat s Kubernetes distribution: it is basically the upstream Kubernetes delivered with monitoring, logging, CI/CD, underlying OS, tested upgrade paths not found with a manual kubernetes.io kubeadm install. After passing the two corresponding certifications, my opinion on cloud operators is that it is very much a step back in the direction of proprietary software. You can rebuild their cloud stack with opensource components, but it is also a lot of integration work, similar to using the Linux from scratch distribution instead of something like Debian. A good middle point are the OpenShift and OKD Kubernetes distributions, who integrate the most common cloud components, but allow an installation on your own hardware or cloud provider of your choice.
AWS Azure OpenShift *OpenShift upstream project&
Cloud Trail Kubernetes API Server audit log Kubernetes
Cloud Watch Azure Monitor, Azure Log Analytics OpenShift Monitoring Prometheus, Kubernetes Metrics
AWS Artifact Compliance Operator OpenSCAP
AWS Trusted Advisor Azure Advisor Insights
AWS Marketplace Red Hat Market place Operator Hub
AWS Identity and Access Management (IAM) Azure Active Directory, Azure AD DS Red Hat SSO Keycloack
AWS Elastisc Beanstalk Azure App Services OpenShift Source2Image (S2I) Source2Image (S2I)
AWS S3 Azure Blob Storage** ODF Rados Gateway Rook RGW
AWS Elastic Block Storage Azure Disk Storage ODF Rados Block Device Rook RBD
AWS Elastic File System Azure Files ODF Ceph FS Rook CephFS
AWS ELB Classic Azure Load Balancer MetalLB Operator MetalLB
AWS ELB Application Load Balancer Azure Application Gateway OpenShift Router HAProxy
Amazon Simple Notification Service OpenShift Streams for Apache Kafka Apache Kafka
Amazon Guard Duty Microsoft Defender for Cloud API Server audit log review, ACS Runtime detection Stackrox
Amazon Inspector Microsoft Defender for Cloud Quay.io container scanner, ACS Vulnerability Assessment Clair, Stackrox
AWS Lambda Azure Serverless Openshift Serverless* Knative
AWS Key Management System Azure Key Vault could be done with Hashicorp Vault Vault
AWS WAF NGINX Ingress Controller Operator with ModSecurity NGINX ModSecurity
Amazon Elasticache Redis Enterprise Operator Redis, memcached as alternative
AWS Relational Database Service Azure SQL Crunchy Data Operator PostgreSQL
Azure Arc OpenShift ACM Open Cluster Management
AWS Scaling Group Azure Scale Set OpenShift Autoscaler OKD Autoscaler
* OpenShift Serverless requires the application to be packaged as a container, something AWS Lambda does not require. ** Azure Blob Storage covers the object storage use case of S3, but is itself not S3 compatible

Craig Small: Fixing iCalendar feeds

The local government here has all the schools use an iCalendar feed for things like when school terms start and stop and other school events occur. The department s website also has events like public holidays. The issue is that all of them don t make it an all-day event but one that happens at midnight, or one past midnight. The events synchronise fine, though Google s calendar is known for synchronising when it feels like it, not at any particular time you would like it to.
Screenshot of Android Calendar showing a tiny bar at midnight which is the event.
Even though a public holiday is all day, they are sent as appointments for midnight. That means on my phone all the events are these tiny bars that appear right up the top of the screen and are easily missed, especially when the focus of the calendar is during the day. On the phone, you can see the tiny purple bar at midnight. This is how the events appear. It s not the calendar s fault, as far as it knows the school events are happening at midnight. You can also see Lunar New Year and Australia Day appear in the all-day part of the calendar and don t scroll away. That s where these events should be.
Why are all the events appearing at midnight? The reason is the feed is incorrectly set up and has the time. The events are sent in an iCalendar format and a typical event looks like this:
BEGIN:VEVENT
DTSTART;TZID=Australia/Sydney:20230206T000000
DTEND;TZID=Australia/Sydney:20230206T000000
SUMMARY:School Term starts
END:VEVENT
The event starting and stopping date and time are the DTSTART and DTEND lines. Both of them have the date of 2023/02/06 or 6th February 2023 and a time of 00:00:00 or midnight. So the calendar is doing the right thing, we need to fix the feed! The Fix I wrote a quick and dirty PHP script to download the feed from the real site, change the DTSTART and DTEND lines to all-day events and leave the rest of it alone.
<?php
$site = $_GET['s'];
if ($site == 'site1')  
    $REMOTE_URL='https://site1.example.net/ical_feed';
  elseif ($site == 'site2')  
    $REMOTE_URL='https://site2.example.net/ical_feed';
  else  
    http_response_code(400);
    die();
 
$fp = fopen($REMOTE_URL, "r");
if (!$fp)  
    die("fopen");
 
header('Content-Type: text/calendar');
while (( $line = fgets($fp, 1024)) !== false)  
    $line = preg_replace(
        '/^(DTSTART DTEND);[^:]+:([0-9] 8 )T000[01]00/',
        '$ 1 ;VALUE=DATE:$ 2 ',
        $line);
    echo $line;
 
?>
It s pretty quick and nasty but gets the job done. So what is it doing? You need to save the script on your web server somewhere, possibly with an alias command. The whole point of this is to change the type from a date/time to a date-only event and only print the date part of it for the start and end of it. The resulting iCalendar event looks like this:
BEGIN:VEVENT
DTSTART;VALUE=DATE:20230206
DTEND;VALUE=DATE:20230206
SUMMARY:School Term starts
END:VEVENT
The calendar then shows it properly as an all-day event. I would check the script works before doing the next step. You can use things like curl or wget to download it. If you use a normal browser, it will probably just download the translated file. If you re not seeing the right thing then it s probably the PCRE failing. You can check it online with a regex checker such as https://regex101.com. The site has saved my PCRE and match so you got something to start with. Calendar settings The last thing to do is to change the URL in your calendar settings. Each calendar system has a different way of doing it. For Google Calendar they provide instructions and you want to follow the section titled Use a link to add a public Calendar . The URL here is not the actual site s URL (which you would have put into the REMOTE_URL variable before) but the URL of your script plus the ?s=site1 part. So if you put your script aliased to /myical.php and the site ID was site1 and your website is www.example.com the URL would be https://www.example.com/myical.php?s=site1 . You should then see the events appear as all-day events on your calendar.

26 January 2023

Matt Brown: Goals for 2023

This is the second of a two-part post covering my goals for 2023. See the first part to understand the vision, mission and strategy driving these goals. I want to thank my friend Nat, and Will Larson whose annual reviews I ve always enjoyed reading for inspiring me to write these posts. I ve found the process articulating my motivations and goals very useful to clarify my thoughts and create tangible next steps. I m grateful for that in and of itself, but I also hope that by publishing this you too might find it interesting, and the additional public accountability it creates will be a positive encouragement to me.

2023 Goals My focus for 2023 is to bootstrap a business that I can use to build software that solves real problems (see the strategy from the previous post for more details on this). I m going to track this via three goals:
  1. Execute a series of successful consulting engagements, building a reputation for myself and leaving happy customers willing to provide testimonials that support a pipeline of future opportunities.
  2. Grow my product development skill set by taking several ideas to MVP stage with customer feedback received, and launch at least one product which generates revenue and has growth potential.
  3. Develop and maintain a broad professional network.

Consulting Based on my background and experience, I plan to target my consulting across three areas:
  1. Leadership - building and growing operationally focused software teams following SRE/devops principles. A typical engagement may involve helping a client establish a brand new SRE/devops practice, or to strengthen and mature the existing practices used to build and operate reliable software in their team(s).
  2. Architecture - applying deep technical expertise to the design of large software systems, particularly focusing on their reliability and operability. A typical engagement may involve design input and decision making support for key aspects of a new system, providing external review and analysis to improve an existing system, or delivering actionable, tactical next steps during or immediately after a reliability crisis.
  3. Technology Strategy - translating high-level business needs into a technical roadmap that provides understandable explanations of the value software can deliver in that context, and the iterative series of appropriately sized projects required to realise it. A typical customer for this would be a small to medium sized business outside of the software industry with a desire to use software in a transformative way to improve their business but who does not employ the necessary in-house expertise to lead that transition.

Product Development There are three, currently extremely high level, product ideas that I m excited to explore:
  1. Improve co-ordination of electricity resources to accelerate the electrification of NZ s energy demand and the transition to zero carbon grid. NZ has huge potential to be a world-leader in decarbonising energy use through electrification, but requires a massive transition to realise the benefits. Many of the challenges to that transition involve coordination of an order of magnitude more distributed energy resources (DER) in a much more dynamic and software-oriented manner than the electricity industry is traditionally experienced with. The concept of improving DER coordination is not novel, but our grid has unique characteristics that mean we re likely to need to build localised solutions. There is a strong match between my experience with large, high-reliability distributed software systems, and this need. With renewed motivation in the industry for rapid progress and many conversations and consultations still in their early stages this a very compelling space to explore with the intent of developing a more detailed product opportunity to pursue.
  2. Reduce agricultural emissions by making high performance farm management, including effortless compliance reporting, straightforward, fun and effective for busy farmers. NZ s commitments to reduce agricultural emissions (our largest single sector) place increased compliance and reporting burdens on busy farmers who don t want to report the same data multiple times to different regulators and authorities. In tandem, rising business costs and constraints drive a need for continuous improvements in efficiency, performance and farm management processes in order to remain profitable. This in turn drives increases in complexity and the volume of data that farmers must work with. Many industry organisations and associated software developers offer existing products aimed at addressing aspects of these problems, but anecdotal feedback indicates these are poorly integrated, piecemeal solutions that are often frustrating to use - a burden rather than a source of continuous improvement. It looks like there could be an opportunity for a delightful, comprehensive farm management and reporting system to disrupt the industry and help farmers run more profitable and sustainable farms while also reducing compliance costs and effort.
  3. Lower sickness rates and improve cognitive performance by enabling every indoor space to benefit from continuous ventilation monitoring and reporting. Indoor air quality is important in reducing disease transmission risk and promoting optimal cognitive performance, but despite the current pandemic temporarily raising its profile, a focus on indoor air quality generally remains under the radar for most people. One factor contributing to this is the lack of widely available systems for continuously monitoring and reporting on air quality. I built https://co2mon.nz/ to help address this problem in my children s school during 2022. I see potential to further grow this business through marketing and raising awareness of the value of ventilation monitoring in all indoor environments.
In addition to these mission aligned product ideas, I m also interested in exploring the creation of small to medium sized SaaS applications that deliver useful value by serving the needs of a specialised or niche business or industry. Even when not directly linked to the overall mission, the development and operation of products of this type can support the strategy. Each application adds direct revenue and also contributes to achieving better economies of scale in the many backend processes and infrastructure required to deliver secure, reliable and performant software systems.

Developing my professional network To help make this goal more actionable and measurable I will track 3 sub goals:
  1. To build a professional relationship with at least 30 new people this year, meaning that we ve met and had a decent conversation at least a couple of times and keep in touch at least every few months in some form.
  2. To publish a piece of writing on this site at least once a week, and for many of those to generate interesting conversations and feedback. I ll use this as an opportunity to explore product ideas, highlight my consulting expertise and generally contribute interesting technical content into the world.
  3. To support the growth of my local technical community by volunteering my experience and knowledge with others through activities such as mentoring, conference talks and similar.

Next Steps Over the coming weeks I ll write more about each of these topics - you can use the box in the sidebar (or on the front page, if you re on a phone) to be notified when I post new writing (there s also an RSS feed here, for the geeks). I d love to have your feedback and engagement on these goals too - please drop me an email with your thoughts or even book a meeting - it won t be a distraction to me, you ll be helping me meet my goal of developing and maintaining my network :)

Shirish Agarwal: Minidebconf Tamilnadu 2023, Tinnitus, Cooking, Books and Series.

First up is Minidebconf Tamilnadu 2023 that would be held on 28-29 January 2023. You can find rest of the details here. I do hope we get to see/hear some good stuff from the Minidebconf. Best of luck to all those who are applying.

Tinnitus During the lock-down of March 2020, I became aware of noise in ears and subsequently major hearing loss. It took me quite a while to know that Tinnitus happens to both those who have hearing loss as well as not. I keep running into threads like this and as shared by someone nobody knows what really causes it. I did try some of the apps (an app. called Resound on Android) that is supposed to tackle Tinnitus but it hasn t helped much. There is this but at least for me, right now pretty speculative. Also this, and again highly speculative.

Cooking After mum passed away, I haven t cooked anything. This used to give me pleasure but now just doesn t feel right. Cooking is something you enjoy when you are doing for somebody else and not just for yourself, at least that s how I feel and with that the curiosity to know more recipes. I do wanna buy a wok at sometime but when, how I just don t know.

Books Have been reading books quite a bit. And due to that had to again revisit and understand ISBN. Perhaps I might have shared it before. It really is something, the history of ISBN. And that co-relates with the book I read, Raising Steam by Terry Pratchett. Raising Steam is the 40th Book in the Discworld Series and it basically romanticizes and reminisces how the idea of an engine was born, and then a steam engine and how actually Railways started. There has been a lot of history and experiences from the early years of Steam Railway that have been taken and transplanted into the book. Also how Railways is and can be successful if only it is invested wisely and maintenance is done. This is only where imagination and reality come apart as maintenance isn t done and then you have issues. While this is and was in the UK, similar situation exists in India and many other places around the world and doesn t matter whether it is private or public. Exceptions are German, French but then that maybe due to Labor movements that happened and were successful unlike in other places. I could go on but then it will become a different article in itself. Suffice to say there is much to learn and you need serious people to look after it. Both in UK and India we lack that. And not just in Railways but Civil Aviation too, but again that is a story in itself.

Web-series Apart from books, have been seeing web-series that Willow is a good one that I enjoyed even though I hadn t seen the earlier movie. While there has been a flurry of movies and web-series both due to end of year and beginning of 2023 and yet have tried to be a bit partial on what I wanna watch or not. If it has crime, fantasy, drama then usually I like it. For e.g. I saw Blackout and pretty much was engrossed in what will happen next. It also does lead you to ask questions about centralization vs de-centralization for both power and other utilities and does make a case for communities to have their utilities apart from the grid as a fallback. How do we do over decades or centuries about it is a different question perhaps altogether. There were two books that kinda stood out for me, the first was Ian Rankin s Naming of the Dead . The book is about a cynical John Rebus, a man after my own heart. I am probably going to buy a few more of his series. In a way it also tells you why UK is the way it is right now. Another book that I liked was Shades of Grey by Jasper Fforde. This is one of the books that Mum would have clearly liked. It is pretty unusual while at the same time very close to 1984 and other such dystopian novels. The main trope of the book is what color you can see and how much you can see. The main character is somebody who can see Red, around the age of 20. One of the interesting aspects of the book is de-facting which closely resembles the Post-Truth world where alternative facts can be made out of air and they don t need any scientific evidence to back them up. In Jasper s world, they don t care about how things work and most of the technology is banned and curiosity is considered harmful and those who show that are murdered one way or the other. Interestingly, the author has just last year decided to start book 2 in the 3 book series that is supposed to be. This also tells why the U.S. is such a precarious situation in a way. A part of it is also due to the media which is in hands of chosen few, the same goes for UK and India, almost an oligopoly.

The Great Escape This is also a book but also about experiences of people, not in 19th-20th century but today that tells you slavery is alive and well and human-trafficking as well. This piece from NPR tells you about an MNC and Indian workers. What I found interesting is that there barely is an mention of the Indian Embassy that is supposed to help Indian people. I do know for a fact that the embassies of India has seen a drastic shortage of both people and materials even since the new Govt. came in place that was nine years ago. Incidentally, BBC shared about the Gujarat riots 2002 and that has been censored in India. They keep quiet about the UK Govt. who did find out that the Chief Minister was directly responsible for the killings and in facts his number 2, Amit Shah had shared that we would do 2002 again in the election cycle barely a month ago. But sadly, no hate speech FIR or any action was taken against Mr. Shah. There have been attempts by people to showcase the documentary. For e.g. JNU tried it and the rowdies from ABVP (arm of BJP) created violence. Even the questions that has been asked by the Wire, GOI will not acknowledge them. Interestingly, all India s edtechs have taken a beating in the last 6-8 months including the biggest BJYU s. Sharing a story from 2021 where things were best and today all of them are at bottom. In fact, the public has been wary as the prices of the courses has kept on increasing and most case studies have been found to be fake. Also the general outlook on jobs and growth has been pessimistic. In fact, most companies have been shedding jobs truckloads, most in the I.T. sector but other sectors as well. Hospitality and other related sectors have taken a huge beating, part of it post-pandemic, part of it Govt s refusal to either spend money or do any positive policies for either infrastructure, education, medical, you name it, they think private sector has all the answers which has been proven to be wrong again and again. I did not want to end on a discordant note but things are the way they are

B lint R czey: How to speed up your next build 5-20x with Firebuild?

Firebuild logo
TL;DR: Just prefix your build command (or any command) with firebuild:
firebuild <build command>
OK, but how does it work? Firebuild intercepts all processes started by the command to cache their outputs. Next time when the command or any of its descendant commands is executed with the same parameters, inputs and environment, the outputs are replayed (the command is shortcut) from the cache instead of running the command again. This is similar to how ccache and other compiler-specific caches work, but firebuild can shortcut any deterministic command, not only a specific list of compilers. Since the inputs of each command is determined at run time firebuild does not need a maintained complete dependency graph in the source like Bazel. It can work with any build system that does not implement its own caching mechanism. Determinism of commands is detected at run-time by preloading libfirebuild.so and interposing standard library calls and syscalls. If the command and all its descendants inputs are available when the command starts and all outputs can be calculated from the inputs then the command can be shortcut, otherwise it will be executed again. The interception comes with a 5-10% overhead, but rebuilds can be 5-20 times, or even faster depending on the changes between the builds. Can I try it? It is already available in Debian Unstable and Testing, Ubuntu s development release and the latest stable version is back-ported to supported Ubuntu releases via a PPA. How can I analyze my builds with firebuild? Firebuild can generate an HTML report showing each command s contribution to the build time. Below are the before and after reports of json4s, a Scala project. The command call graphs (lower ones) show that java (scalac) took 99% of the original build. Since the scalac invocations are shortcut (cutting the second build s time to less than 2% of the first one) they don t even show up in the accelerated second build s call graph. What s left to be executed again in the second run are env, perl, make and a few simple commands. The upper graphs are the process trees, with expandable nodes (in blue) also showing which command invocations were shortcut (green). Clicking on a node shows details of the command and the reason if it was not shortcut. Could I accelerate my project more? Firebuild works best for builds with CPU-intensive processes and comes with defaults to not cache very quick commands, such as sh, grep, sed, etc., because caching those would take cache space and shortcutting them may not speed up the build that much. They can still be shortcut with their parent command. Firebuild s strength is that it can find shortcutting points in the process tree automatically, e.g. from sh -c 'bash -c "sh -c echo Hello World!"' bash would be shortcut, but none of the sh commands would be cached. In typical builds there are many such commands from the skip_cache list. Caching those commands with firebuild -o 'processes.skip_cache = []' can improve acceleration and make the reports smaller. Firebuild also supports several debug flags and -d proc helps finding reasons for not shortcutting some commands:
...
FIREBUILD: Command "/usr/bin/make" can't be short-cut due to: Executable set to be not shortcut,  ExecedProcess 1329.2, running, "make -f debian/rules build", fds=[ FileFD fd=0  FileOFD ...
FIREBUILD: Command "/usr/bin/sort" can't be short-cut due to: Process read from inherited fd ,  ExecedProcess 4161.1, running, "sort", fds=[ FileFD fd=0  FileOFD ...
FIREBUILD: Command "/usr/bin/find" can't be short-cut due to: fstatfs() family operating on fds is not supported,  ExecedProcess 1360.1, running, "find -mindepth 1 ...
...
make, ninja and other incremental build tool binaries are not shortcut because they compare the timestamp of files, but they are fast at least and every build step they perform can still be shortcut. Ideally the slower build steps that could not be shortcut can be re-implemented in ways that can be shortcut by avoiding tools performing unsupported operations. I hope those tools help speeding up your build with very little effort, but if not and you find something to fix or improve in firebuild itself, please report it or just leave a feedback! Happy speeding, but not on public roads!

Matt Brown: Vision, Mission and Strategy

This is the first of a two-part post, covering high-level thoughts around my motivations and vision. Make sure to check out the second part for my specific goals for 2023. A new year is upon us! My plan was to be 6 months into the journey of starting a business by this point. I made some very tentative progress towards that goal in 2022, registering a company and starting some consulting work, but on the whole I ve found it much harder than expected to gather the necessary energy to begin that journey in earnest.

Reflection I m excited about the next chapter of my career, so the fact that I ve been struggling to get started has been frustrating. The only upside is that the delay has given me plenty of time to reflect on the last few years and what I can learn from them and draw some lessons to help better manage and sustain my energy going forward.

Purpose A large part of what I ve realised is that I should have left Google years ago. It was a great place to work, and I m incredibly grateful for everything I learned and received during my time there. For years it was my dream job, but my happiness had been declining, and instead of taking the (relatively small) risk of leaving to the unknown, I tried several variations of team and role in the hope of restoring the dream. The reality is that a significant chunk of my motivation and energy comes from being able to link my work back to a bigger purpose that delivers concrete positive impact in the world. I felt that link through Google s mission to make information universally accessible and useful for the first 10-11 years, but for the latter 4-5 years my ability to see that link was tenuous at best and trying to push through the challenges presented without that link providing a reliable source of energy is what drove my unhappiness and led to needing a longer break to recharge. I expect the challenges of starting a business to be even greater than what I experienced at Google, so the lesson I m taking from this is that it s crucial for me to understand what the link between my work and the bigger purpose with concrete positive impact in the world that I m aiming to contribute to is.

Community The second factor that I ve slowly come to realise has been missing from my career in the last few years has been participation in a professional community and a variety of enriching interpersonal relationships. As much as I value and need this type of interaction, fostering and sustaining it unfortunately doesn t come naturally to me. Working remotely since 2016 and then taking a 9 month break out of the industry are not particularly helpful contributors to building and maintaining a wide network either! The lesson here is simply that I m going to need to push past my comfort zone in reaching out and introducing myself to a range of people in order to grow my professional network, and equally I need to be diligent and disciplined in making time to maintain and regularly connect with people whom I respect and find energising to interact with.

Personal Influences Lastly, I ve been reflecting on a set of principles that are important to me. These are not so much new lessons, more confirming to myself what I value moving forward. There are many things I could include here, but to keep it somewhat brief, the key influences on my thinking are:
  • Independence - I can t entirely explain why or where it comes from, but since the start of my professional career (which I consider to be my consulting/freelancing development during high school) I ve understood that I m far more motivated by building and growing my own business than I am by working for someone else. Working for myself has always felt like the default and sensible course - I m excited to get back to that.
  • Openness - Open is better than closed, in terms of software, business model and organisational processes. This continues to be a strong belief and something I want to uphold in my business endeavours. Competition should be based on superior technical quality or service, not artificial constraints or barriers to entry that lock customers and users into a single solution or market. Protocols and networks should be open for wide participation and easily accessible to new entrants and competition.
  • People first - This applies both to how we work with each other - respectfully, valuing diversity and with integrity, and to how we apply technology to our world - with consideration for all stakeholders it may affect and awareness of both the intended and potential unintended impacts.

Framework Using Vision, Mission and Strategy as a planning framework has worked quite well for me when building and growing teams over the years, so I plan to re-use it personally to help organise the above reflections into a hopefully cohesive plan than results in some useful 2023 goals.

Vision Software systems contribute direct and meaningful impact to solving real problems in our world. Each word has a fair bit of meaning behind it for me, so breaking it down a little bit:
  • software systems - excite me because software is eating the world and has significant potential to do good.
  • contribute - Software alone doesn t solve problems, and misapplied can easily make things worse. To contribute software needs to be designed intentionally and evaluated with an awareness of risks it could pose within the complex system that is our modern world.
  • direct and meaningful impact - I m not looking for broad outcomes like improving productivity or communication, which apply generally across many problems. I want to see software applied to solve specific blockers whose removal unlocks significant progress towards solving a problem.
  • real - as opposed to straightforward problems. The types of issue where the acknowledgement of it as a real problem often ends the sentence as it feels too big to tackle. Climate change and pandemic risk are examples of real problems. Decentralising finance or selling more widgets are not.
  • in our world - is mostly filler to round out the sentence nicely, but I do think we should probably sort out the mess we re making on our own planet before trying to colonise anywhere else.

Mission To lead the development and operation of software systems that deliver new opportunities for individuals, businesses and communities to solve the real problems in their community. Again breaking down the intent a little bit:
  • lead - having a meaningful impact on real problems is a big job. I won t succeed as a one man band. It will require building and growing a larger team.
  • development and operation - development is fun and necessary, but I also wanted to highlight that the ongoing operation and integration of those software systems into the broader social and human systems of our world is an equally important and ongoing need.
  • new opportunities - are important to drive and motivate investment in the adoption of technology. Building or operating a system that maintains the status quo is not motivating for me.
  • individuals, businesses and communities - aka everyone! But each of these groups (as examples, not specific) will have diverse roles, needs and interactions with the software which must be considered to ensure the system achieves the desired contribution and impact.
  • their community - refines the ambition from the vision to an achievable scope of action within which to execute the mission. We won t solve our problems by targeting one big global fix, but if we each participate in solving the problems in our community, collectively it will make a difference.

Strategy Build a sustainable business that provides a home and infrastructure to support a continuous cycle of development, validation and growth of software systems fulfilling the mission and vision above.
  • Accumulate meaningful impact via a portfolio of systems rather than one big bet.
  • Focus on opportunities that promote the decarbonisation of our economy (the most pressing problem our society faces), but not at the expense of ignoring compelling opportunities to contribute impact to other real problems also.
  • Favour the marathon over the sprint - while being first can be fun and convey benefits, it s often the fast-followers who learn from the initial mistakes and deliver lasting change and broader impact.
In keeping with the final bullet point, I aim to evaluate the strategy against a long-term view of success. What excites me about it is that it has the potential to provide structure and clarity for my work while also enabling many future paths - from operating a portfolio of micro-SaaS products that each solve real problems for a specific niche or community, or diving deep into a single compelling opportunity for a year or two, joining with others to partner on shared ventures or some combination of all three and other variations in between.

Your Thoughts I consider this a first draft, which I intend to revise and evolve further over the next 6-12 months. I don t plan major changes to the intent or underlying ideas, but finding the best words to express and convey that intent clearly is not something I expect to get right on the first take. I d love to have your feedback and engagement as I move forward with this strategy - please use the box in the sidebar (or on the front page, if you re on a phone) to be notified when I post new writing, drop me an email with your thoughts or even book a meeting to say hi and discuss something in detail.

Goals for 2023 Next up - check out part two of this post to see my goals for 2023.

24 January 2023

Kentaro Hayashi: Porterboxes and alternatives

As you know, Debian projects and sponsor provides so-called "porterbox", but it does not cover all architectures. There are some alternatives to fix architecture-specific bugs. For the record, let's pick it up them. [1][2][3]
porterbox deb-o-matic qemu
amd64 adayevskaya.d.o debomatic-amd64.d.n DQIB ready
arm64 amdahl.d.o debomatic-arm64.d.n DQIB ready
armel amdahl.d.o abel.d.o debomatic-armel.d.n NG
armhf amdahl.d.o abel.d.o harris.d.o debomatic-armhf.d.n DQIB ready
i386 exodar.d.n debomatic-i386.d.n DQIB ready
mips64el eller.d.o debomatic-mips64el.d.n DQIB ready
mipsel eller.d.o debomatic-mipsel.d.n DQIB ready
ppc64el platti.d.o debomatic-ppc64el.d.n DQIB ready
s390x zelenka.d.o debomatic-s390x.d.n DQIB ready
alpha N/A N/A NG
arc N/A N/A N/A
hppa panama.d.n N/A N/A
ia64 yttrium.d.n N/A N/A
kfreebsd-amd64 lemon.d.n N/A N/A
kfreebsd-i386 lemon.d.n N/A N/A
m68k mitchy.d.n N/A NG
powerpc perotto.d.n debomatic-powerpc.d.n DQIB ready
ppc64 perotto.d.n N/A DQIB ready
riscv64 debian-riscv64-porterbox-01.d.n N/A DQIB ready
sh4 N/A N/A NG
sparc64 kyoto.d.n N/A N/A
x32 N/A N/A N/A
Thus, no alternatives for alpha, arc, sh4 and x32.

22 January 2023

Petter Reinholdtsen: Opensnitch, the application level interactive firewall, heading into the Debian archive

While reading a blog post claiming MacOS X recently started scanning local files and reporting information about them to Apple, even on a machine where all such callback features had been disabled, I came across a description of the Little Snitch application for MacOS X. It seemed like a very nice tool to have in the tool box, and I decided to see if something similar was available for Linux. It did not take long to find the OpenSnitch package, which has been in development since 2017, and now is in version 1.5.0. It has had a request for Debian packaging since 2018, but no-one completed the job so far. Just for fun, I decided to see if I could help, and I was very happy to discover that upstream want a Debian package too. After struggling a bit with getting the program to run, figuring out building Go programs (and a little failed detour to look at eBPF builds too - help needed), I am very happy to report that I am sponsoring upstream to maintain the package in Debian, and it has since this morning been waiting in NEW for the ftpmasters to have a look. Perhaps it can get into the archive in time for the Bookworm release? As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Dirk Eddelbuettel: Rcpp 1.0.10 on CRAN: Regular Update

rcpp logo The Rcpp team is thrilled to announce the newest release 1.0.10 of the Rcpp package which is hitting CRAN now and will go to Debian shortly. Windows and macOS builds should appear at CRAN in the next few days, as will builds in different Linux distribution and of course at r2u. The release was prepared a few days ago, but given the widespread use at CRAN it took a few days to be processed. As always, our sincere thanks to the CRAN maintainers Uwe Ligges and Kurt Hornik. This release continues with the six-months cycle started with release 1.0.5 in July 2020. As a reminder, we do of course make interim snapshot dev or rc releases available via the Rcpp drat repo and strongly encourage their use and testing I run my systems with these versions which tend to work just as well, and are also fully tested against all reverse-dependencies. Rcpp has become the most popular way of enhancing R with C or C++ code. Right now, around 2623 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 252 in BioConductor. On CRAN, 13.7% of all packages depend (directly) on CRAN, and 58.7% of all compiled packages do. From the cloud mirror of CRAN (which is but a subset of all CRAN downloads), Rcpp has been downloaded 67.1 million times. This release is incremental as usual, preserving existing capabilities faithfully while smoothing our corners and / or extending slightly. Of particular note is the now fully-enabled use of the unwind protection making some operations a little faster by default; special thanks to I aki for spearheading this. Kevin and I also polished a few other bugs off as detailed below. The full list of details follows.

Changes in Rcpp release version 1.0.10 (2023-01-12)
  • Changes in Rcpp API:
    • Unwind protection is enabled by default (I aki in #1225). It can be disabled by defining RCPP_NO_UNWIND_PROTECT before including Rcpp.h. RCPP_USE_UNWIND_PROTECT is not checked anymore and has no effect. The associated plugin unwindProtect is therefore deprecated and will be removed in a future release.
    • The 'finalize' method for Rcpp Modules is now eagerly materialized, fixing an issue where errors can occur when Module finalizers are run (Kevin in #1231 closing #1230).
    • Zero-row data.frame objects can receive push_back or push_front (Dirk in #1233 fixing #1232).
    • One remaining sprintf has been replaced by snprintf (Dirk and Kevin in #1236 and #1237).
    • Several conversion warnings found by clang++ have been addressed (Dirk in #1240 and #1241).
  • Changes in Rcpp Attributes:
    • The C++20, C++2b (experimental) and C++23 standards now have plugin support like the other C++ standards (Dirk in #1228).
    • The source path for attributes received one more protection from spaces (Dirk in #1235 addressing #1234).
  • Changes in Rcpp Deployment:
    • Several GitHub Actions have been updated.

Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2932 previous questions. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: BH 1.81.0-1 oon CRAN: New Upstream, New Library, sprintf Change

Boost Boost is a very large and comprehensive set of (peer-reviewed) libraries for the C++ programming language, containing well over one hundred individual libraries. The BH package provides a sizeable subset of header-only libraries for (easier, no linking required) use by R. It is fairly widely used: the (partial) CRAN mirror logs (aggregated from the cloud mirrors) show over 32.6 million package downloads. Version 1.81.0 of Boost was released in December following the regular Boost release schedule of April, August and December releases. As the commits and changelog show, we packaged it almost immediately and started testing following our annual update cycle which strives to balance being close enough to upstream and not stressing CRAN and the user base too much. The reverse depends check revealed about a handful of packages requiring changes or adjustments which is a pretty good outcome given the over three hundred direct reverse dependencies. So we opened issue #88 to coordinate the issue over the winter break during which CRAN also closes (just as we did before), and also send a wider PSA tweet as a heads-up. Our sincere thanks to the two packages that already updated, and the four that likely will soon. Our thanks also to CRAN for reviewing the package impact over the last few days since I uploaded the package earlier this week. There are a number of changes I have to make each time in BH, and it is worth mentioning them. Because CRAN cares about backwards compatibility and the ability to be used on minimal or older systems, we still adjust the filenames of a few files to fit a jurassic constraints of just over a 100 characters per filepath present in some long-outdated versions of tar. Not a big deal. We also, and that is more controversial, silence a number of #pragma diagnostic messages for g++ and clang++ because CRAN insists on it. I have no choice in that matter. Next, and hopefully this time only, we also found and replaced a few remaining sprintf uses and replaced them with snprintf. Many of the Boost libraries did that, so I hope by the next upgrade for Boost 1.84.0 next winter this will be fully taken care of. Lastly, and also only this time, we silenced a warning about Boost switching to C++14 in the next release 1.82.0 in April. This may matter for a number of packages having a hard-wired selection of C++11 as their C++ language standard. Luckily our compilers are good enough for C++14 so worst case I will have to nudge a few packages next December. This release adds one new library for url processing which struck us as potentially quite useful. The more detailed NEWS log follows.

Changes in version 1.81.0-1 (2023-01-17)
  • Upgrade to Boost 1.81.0 (#87)
  • Added url (new in 1.81.0)
  • Converted remaining sprintf to snprintf (#90 fixing #89)
  • Comment-out gcc warning messages in three files

Via my CRANberries, there is a diffstat report relative to the previous release. Comments and suggestions about BH are welcome via the issue tracker at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Simon Josefsson: Understanding Trisquel

Ever wondered how Trisquel and Ubuntu differs and what s behind the curtain from a developer perspective? I have. Sharing what I ve learnt will allow you to increase knowledge and trust in Trisquel too.
Trisquel GNU/Linux logo
The scripts to convert an Ubuntu archive into a Trisquel archive are available in the ubuntu-purge repository. The easy to read purge-focal script lists the packages to remove from Ubuntu 20.04 Focal when it is imported into Trisquel 10.0 Nabia. The purge-jammy script provides the same for Ubuntu 22.04 Jammy and (the not yet released) Trisquel 11.0 Aramo. The list of packages is interesting, and by researching the reasons for each exclusion you can learn a lot about different attitudes towards free software and understand the desire to improve matters. I wish there were a wiki-page that for each removed package summarized relevant links to earlier discussions. At the end of the script there is a bunch of packages that are removed for branding purposes that are less interesting to review. Trisquel adds a couple of Trisquel-specific packages. The source code for these packages are in the trisquel-packages repository, with sub-directories for each release: see 10.0/ for Nabia and 11.0/ for Aramo. These packages appears to be mostly for branding purposes. Trisquel modify a set of packages, and here is starts to get interesting. Probably the most important package to modify is to use GNU Linux-libre instead of Linux as the kernel. The scripts to modify packages are in the package-helpers repository. The relevant scripts are in the helpers/ sub-directory. There is a branch for each Trisquel release, see helpers/ for Nabia and helpers/ for Aramo. To see how Linux is replaced with Linux-libre you can read the make-linux script. This covers the basic of approaching Trisquel from a developers perspective. As a user, I have identified some areas that need more work to improve trust in Trisquel: I hope to publish more about my work into these areas. Hopefully this will inspire similar efforts in related distributions like PureOS and the upstream distributions Ubuntu and Debian. Happy hacking!

18 January 2023

Jonathan Dowland: Belfast (David Holmes Remix)

This morning s record to start the day is David Holmes s remix of Belfast , by Orbital: the latest cover mount record with Electronic Sound magazine. This, and several other new remixes are available on the recent compilation 30 Something which, despite being yet another comp, I ve found quite compelling.
picture of a vinyl record
An unusual decision by ES: they ve split the remix across both sides, but kept the RPM at 45. I haven t ran the numbers to figure out if they could have fit it on one side at 33, but even if they could perhaps it would be too compromised. Quite a trade-off with splitting the track! Orbital are embarking on a new UK tour soon and play Newcastle in March. They ve got a new album out in February.

Russ Allbery: Review: Forward

Review: Forward, edited by Blake Crouch
Publisher: Amazon Original Stories
Copyright: September 2019
ISBN: 1-5420-9206-X
ISBN: 1-5420-4363-8
ISBN: 1-5420-9357-0
ISBN: 1-5420-0434-9
ISBN: 1-5420-4363-8
ISBN: 1-5420-4425-1
Format: Kindle
Pages: 300
This is another Amazon collection of short fiction, this time mostly at novelette length. (The longer ones might creep into novella.) As before, each one is available separately for purchase or Amazon Prime "borrowing," with separate ISBNs. The sidebar cover is for the first in the sequence. (At some point I need to update my page templates so that I can add multiple covers.) N.K. Jemisin's "Emergency Skin" won the 2020 Hugo Award for Best Novelette, so I wanted to read and review it, but it would be too short for a standalone review. I therefore decided to read the whole collection and review it as an anthology. This was a mistake. Learn from my mistake. The overall theme of the collection is technological advance, rapid change, and the ethical and social question of whether we should slow technology because of social risk. Some of the stories stick to that theme more closely than others. Jemisin's story mostly ignores it, which was probably the right decision. "Ark" by Veronica Roth: A planet-killing asteroid has been on its inexorable way towards Earth for decades. Most of the planet has been evacuated. A small group has stayed behind, cataloging samples and filling two remaining ships with as much biodiversity as they can find with the intent to leave at the last minute. Against that backdrop, two of that team bond over orchids. If you were going "wait, what?" about the successful evacuation of Earth, yeah, me too. No hint is offered as to how this was accomplished. Also, the entirety of humanity abandoned mutual hostility and national borders to cooperate in the face of the incoming disaster, which is, uh, bizarrely optimistic for an otherwise gloomy story. I should be careful about how negative I am about this story because I am sure it will be someone's favorite. I can even write part of the positive review: an elegiac look at loss, choices, and the meaning of a life, a moving look at how people cope with despair. The writing is fine, the story structure works; it's not a bad story. I just found it monumentally depressing, and was not engrossed by the emotionally abused protagonist's unresolved father issues. I can imagine a story around the same facts and plot that I would have liked much better, but all of these people need therapy and better coping mechanisms. I'm also not sure what this had to do with the theme, given that the incoming asteroid is random chance and has nothing to do with technological development. (4) "Summer Frost" by Blake Crouch: The best part of this story is the introductory sequence before the reader knows what's going on, which is full of evocative descriptions. I'm about to spoil what is going on, so if you want to enjoy that untainted by the stupidity of the rest of the plot, skip the rest of this story review. We're going to have a glut of stories about the weird and obsessive form of AI risk invented by the fevered imaginations of the "rationalist" community, aren't we. I don't know why I didn't predict that. It's going to be just as annoying as the glut of cyberpunk novels written by people who don't understand computers. Crouch lost me as soon as the setup is revealed. Even if I believed that a game company would use a deep learning AI still in learning mode to run an NPC (I don't; see Microsoft's Tay for an obvious reason why not), or that such an NPC would spontaneously start testing the boundaries of the game world (this is not how deep learning works), Crouch asks the reader to believe that this AI started as a fully scripted NPC in the prologue with a fixed storyline. In other words, the foundation of the story is that this game company used an AI model capable of becoming a general intelligence for barely more than a cut scene. This is not how anything works. The rest of the story is yet another variation on a science fiction plot so old and threadbare that Isaac Asimov invented the Three Laws of Robotics to avoid telling more versions of it. Crouch's contribution is to dress it up in the terminology of the excessively online. (The middle of the story features a detailed discussion of Roko's basilisk; if you recognize that, you know what you're in for.) Asimov would not have had a lesbian protagonist, so points for progress I guess, but the AI becomes more interesting to the protagonist than her wife and kid because of course it does. There are a few twists and turns along the way, but the destination is the bog-standard hard-takeoff general intelligence scenario. One more pet peeve: Authors, stop trying to illustrate the growth of your AI by having it move from broken to fluent English. English grammar is so much easier than self-awareness or the Turing test that we had programs that could critique your grammar decades before we had believable chatbots. It's going to get grammar right long before the content of the words makes any sense. Also, your AI doesn't sound dumber, your AI sounds like someone whose native language doesn't use pronouns and helper verbs the way that English does, and your decision to use that as a marker for intelligence is, uh, maybe something you should think about. (3) "Emergency Skin" by N.K. Jemisin: The protagonist is a heavily-augmented cyborg from a colony of Earth's diaspora. The founders of that colony fled Earth when it became obvious to them that the planet was dying. They have survived in another star system, but they need a specific piece of technology from the dead remnants of Earth. The protagonist has been sent to retrieve it. The twist is that this story is told in the second-person perspective by the protagonist's ride-along AI, created from a consensus model of the rulers of the colony. We never see directly what the protagonist is doing or thinking, only the AI reaction to it. This is exactly the sort of gimmick that works much better in short fiction than at novel length. Jemisin uses it to create tension between the reader and the narrator, and I thoroughly enjoyed the effect. (As shown in the Broken Earth trilogy, Jemisin is one of the few writers who can use second-person effectively.) I won't spoil the revelation, but it's barbed and biting and vicious and I loved it. Jemisin does deliver the point with a sledgehammer, so be aware of that if you want subtlety in your short fiction, but I prefer the bluntness. (This is part of why I usually don't get along with literary short stories.) The reader of course can't change the direction of the story, but the second-person perspective still provides a hit of vicarious satisfaction. I can see why this won the Hugo; it's worth seeking out. (8) "You Have Arrived at Your Destination" by Amor Towles: Sam and his wife are having a child, and they've decided to provide him with an early boost in life. Vitek is a fertility lab, but more than that, it can do some gene tweaking and adjustment to push a child more towards one personality or another. Sam and his wife have spent hours filling out profiles, and his wife spent hours weeding possible choices down to three. Now, Sam has come to Vitek to pick from the remaining options. Speaking of literary short stories, Towles is the non-SFF writer of this bunch, and it's immediately obvious. The story requires the SFnal premise, but after that this is a character piece. Vitek is an elite, expensive company with a condescending and overly-reductive attitude towards humanity, which is entirely intentional on the author's part. This is the sort of story that gets resolved in an unexpected conversation in a roadside bar, and where most of the conflict happens inside the protagonist's head. I was initially going to complain that Towles does the standard literary thing of leaving off the denouement on the grounds that the reader can figure it out, but when I did a bit of re-reading for this review, I found more of the bones than I had noticed the first time. There's enough subtlety that I had to think for a bit and re-read a passage, but not too much. It's also the most thoughtful treatment of the theme of the collection, the only one that I thought truly wrestled with the weird interactions between technological capability and human foresight. Next to "Emergency Skin," this was the best story of the collection. (7) "The Last Conversation" by Paul Tremblay: A man wakes up in a dark room, in considerable pain, not remembering anything about his life. His only contact with the world at first is a voice: a woman who is helping him recover his strength and his memory. The numbers that head the chapters have significant gaps, representing days left out of the story, as he pieces together what has happened alongside the reader. Tremblay is the horror writer of the collection, so predictably this is the story whose craft I can admire without really liking it. In this case, the horror comes mostly from the pacing of revelation, created by the choice of point of view. (This would be a much different story from the perspective of the woman.) It's well-done, but it has the tendency I've noticed in other horror stories of being a tightly closed system. I see where the connection to the theme is, but it's entirely in the setting, not in the shape of the story. Not my thing, but I can see why it might be someone else's. (5) "Randomize" by Andy Weir: Gah, this was so bad. First, and somewhat expectedly, it's a clunky throwback to a 1950s-style hard SF puzzle story. The writing is atrocious: wooden, awkward, cliched, and full of gratuitous infodumping. The characterization is almost entirely through broad stereotypes; the lone exception is the female character, who at least adds an interesting twist despite being forced to act like an idiot because of the plot. It's a very old-school type of single-twist story, but the ending is completely implausible and falls apart if you breathe on it too hard. Weir is something of a throwback to an earlier era of scientific puzzle stories, though, so maybe one is inclined to give him a break on the writing quality. (I am not; one of the ways in which science fiction has improved is that you can get good scientific puzzles and good writing these days.) But the science is also so bad that I was literally facepalming while reading it. The premise of this story is that quantum computers are commercially available. That will cause a serious problem for Las Vegas casinos, because the generator for keno numbers is vulnerable to quantum algorithms. The solution proposed by the IT person for the casino? A quantum random number generator. (The words "fight quantum with quantum" appear literally in the text if you're wondering how bad the writing is.) You could convince me that an ancient keno system is using a pseudorandom number generator that might be vulnerable to some quantum algorithm and doesn't get reseeded often enough. Fine. And yes, quantum computers can be used to generate high-quality sources of random numbers. But this solution to the problem makes no sense whatsoever. It's like swatting a house fly with a nuclear weapon. Weir says explicitly in the story that all the keno system needs is an external source of high-quality random numbers. The next step is to go to Amazon and buy a hardware random number generator. If you want to splurge, it might cost you $250. Problem solved. Yes, hardware random number generators have various limitations that may cause you problems if you need millions of bits or you need them very quickly, but not for something as dead-simple and with such low entropy requirements as keno numbers! You need a trivial number of bits for each round; even the slowest and most conservative hardware random number generator would be fine. Hell, measure the noise levels on the casino floor. Point a camera at a lava lamp. Or just buy one of the physical ball machines they use for the lottery. This problem is heavily researched, by casinos in particular, and is not significantly changed by the availability of quantum computers, at least for applications such as keno where the generator can be reseeded before each generation. You could maybe argue that this is an excuse for the IT guy to get his hands on a quantum computer, which fits the stereotypes, but that still breaks the story for reasons that would be spoilers. As soon as any other casino thought about this, they'd laugh in the face of the characters. I don't want to make too much of this, since anyone can write one bad story, but this story was dire at every level. I still owe Weir a proper chance at novel length, but I can't say this added to my enthusiasm. (2) Rating: 4 out of 10

Arnaud Rebillout: Build container images in GitLab CI (iptables-legacy at the rescue)

It's 2023 and these days, building a container image in a CI pipeline should be straightforward. So let's try. For this blog post we'll focus on GitLab SaaS only, that is, gitlab.com, as it's what I use for work and for personal projects. To get started, we just need two files in our Git repository: Here is our Git tree:
$ ls -A
Containerfile  .git  .gitlab-ci.yml
$ cat Containerfile 
FROM debian:stable
RUN  apt-get update
CMD  echo hello world
$ cat .gitlab-ci.yml 
build-container-image:
  stage: build
  image: debian:testing
  before_script:
    - apt-get update
    - apt-get install -y buildah ca-certificates
  script:
    - buildah build -t $CI_REGISTRY_IMAGE .
    - buildah login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
    - buildah push $CI_REGISTRY_IMAGE
A few remarks: Now let's push that. Does the CI pass? No, of course, otherwise I wouldn't be writing this blog post ;) The CI fails at the buildah build command, with a rather cryptic error:
$ buildah build --tag $CI_REGISTRY_IMAGE .
[...]
STEP 2/3: RUN  apt-get update
error running container: did not get container start message from parent: EOF
Error: building at STEP "RUN apt-get update": netavark: code: 4, msg: iptables v1.8.8 (nf_tables): Could not fetch rule set generation id: Invalid argument
The hint here is nf_tables... Back in July 2021, GitLab did a major update of their shared runners infrastructure, and broke nftables support in the process, as it seems. So we have to use iptables instead. Let's fix our .gitlab-ci.yml, which now looks like that:
$ cat .gitlab-ci.yml 
build-container-image:
  stage: build
  image: debian:testing
  before_script:
    - apt-get update
    - apt-get install -y buildah ca-certificates
    -  
      # Switch to iptables legacy, as GitLab CI doesn't support nftables.
      apt-get install -y --no-install-recommends iptables
      update-alternatives --set iptables /usr/sbin/iptables-legacy
  script:
    - buildah build -t $CI_REGISTRY_IMAGE .
    - buildah login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
    - buildah push $CI_REGISTRY_IMAGE
And push again. Does that work? Yes! If you're interested in this issue, feel free to fork https://gitlab.com/arnaudr/gitlab-build-container-image and try it by yourself. It's been more than a year since this change, and I'm surprised that I didn't find much about it on the Internet, neither mentions of the issue, nor of a workaround. Maybe nobody builds container images in GitLab CI, or maybe they do it another way, I don't know. In any case, now it's documented in this blog, hopefully some will find it useful. Happy 2023!

17 January 2023

Russ Allbery: Review: Night and Silence

Review: Night and Silence, by Seanan McGuire
Series: October Daye #12
Publisher: DAW Books
Copyright: 2018
ISBN: 0-698-18353-3
Format: Kindle
Pages: 353
Night and Silence is the 12th book in Seanan McGuire's long-running October Daye Celtic-inspired urban fantasy series. This is a "read the books in order" sort of series; you definitely do not want to start here. Gillian, Toby's estranged daughter, has been kidnapped. Her ex-husband and his new wife turn to her in desperation (although Miranda suspects Toby is the one who kidnapped her). Toby of course drops everything to find her, which she would have done regardless, but the obvious fear is that Gillian may have been kidnapped specifically to get at Toby. Meanwhile, the consequences of The Brightest Fell have put a severe strain on one of Toby's most important relationships, at the worst possible time. Once again, this is when I say that McGuire's writing has a lot of obvious flaws, and then say that this book kept me up way past my bedtime for two nights in a row because it was nearly impossible to put down. The primary quality flaw in these books, at least for me, is that Toby's thought processes have some deeply-worn grooves that she falls into time and time again. Since she's the first-person narrator of the series, that produces some repetitive writing. She feels incredibly lucky for her chosen family, she worries about her friends, she prizes loyalty very highly, she throws herself headlong into danger, and she thinks about these things in a background hum through every book. By this point, the reader knows all of this, so there's a lot of "yes, yes, we know" muttering that tends to happen. McGuire also understands the importance of plot and character recaps at the start of each book for those of us who have forgotten what was happening (thank you!) but awkwardly writes them into the beginning of each book. That doesn't help with the sense of repetitiveness. If only authors would write stand-alone synopses of previous books in a series, or hire someone to do that if they can't stand to, the world would be a much better place. But now I'm repeating myself. Once I get into a book, though, this doesn't matter at all. When Toby starts down a familiar emotional rut, I just read faster until I get to the next bit. Something about these books is incredibly grabby to me; once I get started on one, I devour it, and usually want to read the next one as soon as possible. Some of this is the cast, which at this point in the series is varied, entertaining, and full of the sort of casual banter that only people who have known each other for a long time can do. A lot of it is that Toby constantly moves forward. She ruminates and angsts and worries, but she never sits around and mopes. She does her thinking on the move. No matter how preoccupied she is with some emotional thread, something's going to happen on the next page. Some of it is intangible, or at least beyond my ability to put a finger on. Some authors are good at writing grabby books, and at least for me McGuire is one of those authors. Describing the plot in any detail without spoilers is hopeless this far into the series, but this is one of the big revelation books, and I suspect it's also going to be a significant tipping point in Toby's life. We finally find out what broke faerie, which has rather more to do with Toby and her family than one might have expected, explains some things about Amandine, and also (although this hasn't been spelled out yet) seems likely to explain some things about the Luidaeg's involvement in Toby's adventures. And there is another significant realignment of one of Toby's relationships that isn't fully developed here, but that I hope will be explored in future books. There's also a lot about Tybalt that to be honest I found tedious and kind of frustrating (although not half as frustrating as Toby found it). I appreciate what McGuire was doing; some problems are tedious, frustrating, and repetitive because that's how one gets through them. The problem that Toby and Tybalt are wrestling with is realistic and underdiscussed in fiction of this type, so I respect the effort, and I'm not sure there was way to write through this that would have been more engrossing (and a bit less cliched). But, still, not my favorite part of this book. Thankfully, it was a mostly-ignorable side thread. This was a substantial improvement over The Brightest Fell, which was both infuriating and on rails. Toby has much more agency here, the investigation was more interesting, and the lore and character fallout has me eager to read the next book. It's fun to see McGuire's world-building come together over this long of a series, and so far it has not disappointed. Followed by The Unkindest Tide. As has become usual, the book ends with a novella telling a story from a different perspective. "Suffer a Sea-Change": The main novel was good. This was great. "Suffer a Sea-Change" retells a critical part of the novel from Gillian's perspective, and then follows that thread of story past the end of the novel. I loved absolutely everything about this. Gillian is a great protagonist, similar to Toby but different enough to be a fresh voice. There is a ton of the Luidaeg, and through different eyes than Toby's which is fun. There's some great world-building and a few very memorable scenes. And there's also the beginning, the very small beginning, of the healing of a great injustice that's been haunting this series since the very beginning, and I am very much here for that. Great stuff. (9) Rating: 8 out of 10

16 January 2023

Gunnar Wolf: Back to Understanding Computers and Cognition

As many of you know, I work at UNAM, Mexico s largest university. My work is split in two parts: My full-time job is to be the systems and network administrator at the Economics Research Institute, and I do some hours of teaching at the Engineering Faculty. At the Institute, my role is academic but although I have tried to frame my works in a way amenable to analysis grounded on the Social Sciences (Construcci n Colaborativa del Conocimiento, Hecho con Creative Commons, Mecanismos de privacidad y anonimato), so far, I have not taken part of academic collaboration with my coworkers Economics is a field very far from my interests, to somehow illustrate it. I was very happy when I was invited to be a part of a Seminar on The Digital Economy in the age of Artificial Intelligence . I talked with the coordinator, and we agreed we have many Economic Science experts but understanding what does Artificial Intelligence mean eludes then, so I will be writing one of the introductory chapters to this analysis. But Hey, I m no expert in Artificial Intelligence. If anything, I could be categorized as an AI-skeptical! Well, at least I might be the closest thing at hand in the Institute So I have been thinking about what I will be writing, and finding and reading material to substantiate what I ll be writing. One of the readings I determined early on I would be going back to is Terry Winograd and Fernando Flores 1986 book, Understanding Computers and Cognition: A New Foundation for Design (oh, were you expecting a link to buy it instead of reading it online?) I first came across this book by mere chance. Back in the last day of year 2000, my friend Ariel invited me and my then-girlfriend to tag along and travel by land to the USA to catch some just-after-Christmas deals in San Antonio. I was not into shopping, but have always enjoyed road trips, so we went together. We went, yes, to the never-ending clothing shops, but we also went to some big libraries And the money I didn t spend at other shops, I spent there. And then some more. There was a little, somewhat oldish book that caught my eye. And I ll be honest: I looked at this book only because it was obviously produced by the LaTeX typesetting system (the basics of which I learnt in 1983, and with which I have written well, basically everything substantial I ve ever done). I remember I read this book with most interest back in that year, and finished it with a Wow, that was a strange trip! And Although I have never done much that could be considered AI-related, this has always been my main reference. But not for explaining what is a perceptron, how is an expert system to ponder the weight of given information, or whether a neural network is convolutional or recurrent, or how to turn from a network trained to recognize feature x into a generational network. No, our book is not technical. Well Not in that sense. This book tackles cognition. But in order to discuss cognition, it must first come to a proper definition of it. And to do so, it has to base itself on philosophy, starting by noting the author s disagreement with what they term as the rationalistic tradition: what we have come to term valid reasoning in Western countries. Their main claim is that the rationalistic tradition cannot properly explain a process as complex as cognition (how much bolder can you be than proposing something like this?). So, this book presents many constructs of Heidggerian origin, aiming to explain what it is understanding and being. In doing so, it follows Humberto Maturana s work. Maturana is also a philosopher, but comes from a background in biology he published works on animal neurophysiology that are also presented here. Writing this, I must ensure you I am not a philosopher, and lack field-specific knowledge to know whether this book is so unique. I know from the onset it does not directly help me to write the chapter I will be writing (but it will surely help me write some important caveats that will make the chapter much more interesting and different to what anybody with a Web browser could write about artificial intelligence). One last note: Although very well written, and notable for bringing hard to grasp concepts to mere technical staff as myself, this is not light, easy reading. I started re-reading this book a couple of weeks ago, and have just finished chapter 5 (page 69). As some reviewers state, this is one of those books you have to go back a paragraph or two over and over. But it is a most enjoyable and interesting reading.

Russ Allbery: Review: The Truth

Review: The Truth, by Terry Pratchett
Series: Discworld #25
Publisher: Harper
Copyright: November 2000
Printing: August 2014
ISBN: 0-06-230736-3
Format: Mass market
Pages: 435
The Truth is the 25th Discworld novel. Some reading order guides group it loosely into an "industrial revolution" sequence following Moving Pictures, but while there are thematic similarities I'll talk about in a moment, there's no real plot continuity. You could arguably start reading Discworld here, although you'd be spoiled for some character developments in the early Watch novels. William de Worde is paid to write a newsletter. That's not precisely what he calls it, and it's not clear whether his patrons know that he publishes it that way. He's paid to report on news of Ankh-Morpork that may be of interest of various rich or influential people who are not in Ankh-Morpork, and he discovered the best way to optimize this was to write a template of the newsletter, bring it to an engraver to make a plate of it, and run off copies for each of his customers, with some minor hand-written customization. It's a comfortable living for the estranged younger son of a wealthy noble. As the story opens, William is dutifully recording the rumor that dwarfs have discovered how to turn lead into gold. The rumor is true, although not in the way that one might initially assume.
The world is made up of four elements: Earth, Air, Fire, and Water. This is a fact well known even to Corporal Nobbs. It's also wrong. There's a fifth element, and generally it's called Surprise. For example, the dwarfs found out how to turn lead into gold by doing it the hard way. The difference between that and the easy way is that the hard way works.
The dwarfs used the lead to make a movable type printing press, which is about to turn William de Worde's small-scale, hand-crafted newsletter into a newspaper. The movable type printing press is not unknown technology. It's banned technology, because the powers that be in Ankh-Morpork know enough to be deeply suspicious of it. The religious establishment doesn't like it because words are too important and powerful to automate. The nobles and the Watch don't like it because cheap words cause problems. And the engraver's guild doesn't like it for obvious reasons. However, Lord Vetinari knows that one cannot apply brakes to a volcano, and commerce with the dwarfs is very important to the city. The dwarfs can continue. At least for now. As in Moving Pictures, most of The Truth is an idiosyncratic speedrun of the social effects of a new technology, this time newspapers. William has no grand plan; he's just an observant man who likes to write, cares a lot about the truth, and accidentally stumbles into editing a newspaper. (This, plus being an estranged son of a rich family, feels very on-point for journalism.) His naive belief is that people want to read true things, since that's what his original patrons wanted. Truth, however, may not be in the top five things people want from a newspaper. This setup requires some narrative force to push it along, which is provided by a plot to depose Vetinari by framing him for murder. The most interesting part of that story is Mr. Pin and Mr. Tulip, the people hired to do the framing and then dispose of the evidence. They're a classic villain type: the brains and the brawn, dangerous, terrifying, and willing to do horrible things to people. But one thing Pratchett excels at is taking a standard character type, turning it a bit sideways, and stuffing in things that one wouldn't think would belong. In this case, that's Mr. Tulip's deep appreciation for, and genius grasp of, fine art. It should not work to have the looming, awful person with anger issues be able to identify the exact heritage of every sculpture and fine piece of goldsmithing, and yet somehow it does. Also as in Moving Pictures (and, in a different way, Soul Music), Pratchett tends to anthropomorphize technology, giving it a life and motivations of its own. In this case, that's William's growing perception of the press as an insatiable maw into which one has to feed words. I'm usually dubious of shifting agency from humans to things when doing social analysis (and there's a lot of social analysis here), but I have to concede that Pratchett captures something deeply true about the experience of feedback loops with an audience. A lot of what Pratchett puts into this book about the problematic relationship between a popular press and the truth is obvious and familiar, but he also makes some subtle points about the way the medium shapes what people expect from it and how people produce content for it that are worthy of Marshall McLuhan. The interactions between William and the Watch were less satisfying. In our world, the US press is, with only rare exceptions, a thoughtless PR organ for police propaganda and the exonerative tense. Pratchett tackles that here... sort of. William vaguely grasps that his job as a reporter may be contrary to the job of the Watch to maintain order, and Vimes's ambivalent feelings towards "solving crimes" push the story in that direction. But this is also Vimes, who is clearly established as one of the good sort and therefore is a bad vehicle for talking about how the police corrupt the press. Pratchett has Vimes and Vetinari tacitly encourage William, which works within the story but takes the pressure off the conflict and leaves William well short of understanding the underlying politics. There's a lot more that could be said about the tension between the press and the authorities, but I think the Discworld setup isn't suitable for it. This is the sort of book that benefits from twenty-four volumes of backstory and practice. Pratchett's Ankh-Morpork cast ticks along like a well-oiled machine, which frees up space that would otherwise have to be spent on establishing secondary characters. The result is a lot of plot and social analysis shoved into a standard-length Discworld novel, and a story that's hard to put down. The balance between humor and plot is just about perfect, the references and allusions aren't overwhelming, and the supporting characters, both new and old, are excellent. We even get a good Death sequence. This is solid, consistent stuff: Discworld as a mature, well-developed setting with plenty of stories left to tell. Followed by Thief of Time in publication order, and later by Monstrous Regiment in the vaguely-connected industrial revolution sequence. Rating: 8 out of 10

Next.

Previous.