Search Results: "zed"

21 June 2020

Enrico Zini: Forced italianisation of S dtirol

Italianization (Italian: Italianizzazione; Croatian: talijanizacija; Slovene: poitaljan evanje; German: Italianisierung; Greek: ) is the spread of Italian culture and language, either by integration or assimilation.[1][2]
In 1919, at the time of its annexation, the middle part of the County of Tyrol which is today called South Tyrol (in Italian Alto Adige) was inhabited by almost 90% German speakers.[1] Under the 1939 South Tyrol Option Agreement, Adolf Hitler and Benito Mussolini determined the status of the German and Ladin (Rhaeto-Romanic) ethnic groups living in the region. They could emigrate to Germany, or stay in Italy and accept their complete Italianization. As a consequence of this, the society of South Tyrol was deeply riven. Those who wanted to stay, the so-called Dableiber, were condemned as traitors while those who left (Optanten) were defamed as Nazis. Because of the outbreak of World War II, this agreement was never fully implemented. Illegal Katakombenschulen ("Catacomb schools") were set up to teach children the German language.
The Prontuario dei nomi locali dell'Alto Adige (Italian for Reference Work of Place Names of Alto Adige) is a list of Italianized toponyms for mostly German place names in South Tyrol (Alto Adige in Italian) which was published in 1916 by the Royal Italian Geographic Society (Reale Societ Geografica Italiana). The list was called the Prontuario in short and later formed an important part of the Italianization campaign initiated by the fascist regime, as it became the basis for the official place and district names in the Italian-annexed southern part of the County of Tyrol.
Ettore Tolomei (16 August 1865, in Rovereto 25 May 1952, in Rome) was an Italian nationalist and fascist. He was designated a Member of the Italian Senate in 1923, and ennobled as Conte della Vetta in 1937.
The South Tyrol Option Agreement (German: Option in S dtirol; Italian: Opzioni in Alto Adige) was an agreement in effect between 1939 and 1943, when the native German speaking people in South Tyrol and three communes in the province of Belluno were given the option of either emigrating to neighboring Nazi Germany (of which Austria was a part after the 1938 Anschluss) or remaining in Fascist Italy and being forcibly integrated into the mainstream Italian culture, losing their language and cultural heritage. Over 80% opted to move to Germany.

18 June 2020

Gunnar Wolf: On masters and slaves, whitelists and blacklists...

LWN published today yet another great piece of writing, Loaded terms in free software. I am sorry, the content will not be immediately available to anybody following at home, as LWN is based on a subscription model But a week from now, the article will be open for anybody to read. Or you can ask me (you most likely can find my contact addresses, as they are basically everywhere) for a subscriber link, I will happily provide it. In consonance with the current mood that started with the killing of George Floyd and sparked worldwide revolts against police brutality, racism (mostly related to police and law enforcement forces, but social as well) and the like, the debate that already started some months ago in technical communities has re-sparked: We have many terms that come with long histories attached to them, and we are usually oblivious to their obvious meaning. We? Yes, we, the main users and creators of technology. I never felt using master and slave to refer to different points of a protocol, bus, clock or whatever (do refer to the Wikipedia article for a fuller explanation) had any negative connotations but then again, those terms have never tainted my personal family. That is, I understand I speak from a position of privilege. A similar although less heated issue goes around the blacklist and whitelist terms, or other uses that use white to refer to good, law-abiding citizens, and black to refer to somewhat antisocial uses (i.e. the white hat and black hat hackers). For several years, this debate has been sparking and dying off. Some important changes have been made Particularly, in 2017 the Internet Software Consortium started recommending Primary and Secondary, Python dropped master/slave pairs after a quite thorough and deep review throughout 2018, GitHub changed the default branch from master to main earlier this week. The Internet Engineering Task Force has a draft (that lapsed and thus sadly didn t become an RFC, but still, is archived), Terminology, Power and Oppressive Language that lists suggested alternatives:
There are also many other relationships that can be used as metaphors, Eglash s research calls into question the accuracy of the master-slave metaphor. Fortunately, there are ample alternatives for the master-slave relationship. Several options are suggested here and should be chosen based on the pairing that is most clear in context:
  • Primary-secondary
  • Leader-follower
  • Active-standby
  • Primary-replica
  • Writer-reader
  • Coordinator-worker
  • Parent-helper
I ll add that I think we Spanish-speakers are not fully aware of the issue s importance, because the most common translation I have seen for master/slave is maestro/esclavo: Maestro is the word for teacher (although we do keep our slaves in place). But think whether it sounds any worse if you refer to device pairs, or members of a database high-availability cluster, or whatever as Amo and Esclavo. It does sound much worse I cannot add much of value to this debate. I am just happy issues like this are being recognized and dealt with. If the topic interests you, do refer to the LWN article! Some excrepts: I consider the following to be the core of Jonathan Corbet s writeup:
Recent events, though, have made it clear even to those of us who were happy to not question this view that the story of slavery and the wider racist systems around it is not yet finished. There are many people who are still living in the middle of it, and it is not a nice place to be. We are not so enlightened as we like to think we are. If there is no other lesson from the events of the last few weeks, we should certainly take to heart the point that we need to be listening to the people who have been saying, for many years, that they are still suffering. If there are people who are telling us that terms like slave or blacklist are a hurtful reminder of the inequities that persist in our society, we need to accept that as the truth and act upon it. Etymological discussions on what, say, master really means may be interesting, but they miss the point and are irrelevant to this discussion.
Part of a comment by user yokem_55:
Often, it seems to me that the replacement words are much more descriptive and precise than the old language. Allowlist is far more obviously a list of explicitly authorized entities than whitelist . Mainline has a more obvious meaning of a core stream of development than master . The benefit of moving past this language is more than just changing cultural norms, it s better, more precise communication across the board.
Another spot-on comment, by user alan:
From my perspective as a Black American male, I think that it s nice to see people willing to see and address racism in various spheres. I am concerned that some of these steps will be more performative than substantial. Terminology changes in software so as to be more welcoming is a nice thing. Ensuring that oppressed minorities have access to the tools and resources to help reduce inequity and ensuring equal protection under the laws is better. We ll get there one day I m sure. The current ask is much simpler, its just to stop randomly killing and terrorizing us. Please and thank you.
So Maybe the protests of this year caught special notoriety because the society is reacting after (or during, for many of us) the lockdown. In any case, I hope for their success in changing the planet s culture of oppression.

Comments Tomas Janousek 2020-06-19 10:04:32 +0200 In the blog post On masters and slaves, whitelists and blacklists you claim that GitHub changed the default branch from master to main earlier this week but I don t think that change is in effect yet. When you create a repo, the default branch is still named master . Gunnar Wolf 2020-06-19 11:52:30 -0500 Umh, seems you are right. Well, what can I say? I m reporting only what I have been able to find / read Now, given that said master branch does not carry any Git-specific meaning and is just a commonly used configuration I hope people start picking it up. No, I have not renamed master branches in any of my repos but intend to do so soonish. Tomas Janousek 2020-06-19 20:01:52 +0200 Yeah, don t worry. I just find it sad that so much inaccurate news is spreading from a single CEO tweet, and I wanted to help stop that. I m sure some change will happen eventually, but until it does, we shouldn t speak about it in the past tense. :-)

16 June 2020

Ritesh Raj Sarraf: Kodi PS3 BD Remote

Setting up a Sony PS3 Blu-Ray Disc Remote Controller with Kodi TLDR; Since most of the articles on the internet were either obsolete or broken, I ve chosen to write these notes down in the form of a blog post so that it helps me now and in future, and hopefully others too.

Raspberry Pi All this time, I have been using the Raspberry Pi for my HTPC needs. The first RPi I acquired was in 2014 and I have been very very happy with the amount of support in the community and quality of the HTPC offering it has. I also appreciate the RPi s form factor and the power consumption limits. And then, to add more sugar to it, it uses a derivative of Debian, Raspbian, which was very familiar and feel good to me.

Raspberry Pi Issues So primarily, I use my RPi with Kodi. There are a bunch of other (daemon) services but the primary use case is HTPC only. RPi + Kodi has a very very annoying issue wherein it loses its audio pitch during video playback. The loss is so bad that the audio is barely audible. The workaround is to seek the video playback either way and then it comes back to its actual audio level, just to fade again in a while. My suspicion was that it may be a problem with Kodi. Or at least, Kodi would have a workaround in software. But unfortunately, I wasted a lot of time in dealing with my suspicion with no fruitful result. This started becoming a PITA over time. And it seems the issue is with the hardware itself because after I moved my setup to a regular laptop, the audio loss is gone.

Laptop with Kodi Since I had my old Lenovo Yoga 2 13 lying on all the time, it made sense to make some more use of it, using as the HTPC. This machine comes with a Micro-HDMI Out port, so it felt ideal for my High Definition video rendering needs. It comes stock with just Intel HD Video with good driver support in Linux, so it was quite quick and easy getting Kodi charged up and running on it. And as I mentioned above, the sound issues are not seen on this setup. Some added benefits are that I get to run stock Debian on this machine. And I must say a big THANK YOU to the Debian Multimedia Maintainers, who ve done a pretty good job maintaining Kodi under Debian.

HDMI CEC Only after I decommissioned my RPi, I came to notice how convenient the HDMI CEC functionality is. Turns out no standard laptops ship CEC functionality onto them. Even the case of my laptop, which has a Micro HDMI Out port, but still no CEC capabilities. As far as I know, the RPi came with the Pulse-Eight CEC module, so obvious first thought was to opt for a compatible external module of the same; but it comes with a nice price tag, me not willing to spend.

WiFi Remotes Kodi has very well implemented network interface for almost all its features. One could take the Yatse or Music Pump Kodi Remote Android applications that work very very well with Kodi. But wifi can be flaky some times. Especially, my experience with the Realtek network devices hasn t been very good. The driver support in Linux is okay but there are many firmware bugs to deal with. In my case, the machine will lose wifi signal/network every once in a while. And it turns out, for this machine, with this network device type, I m not the only one running into such problems. And to add to that, this is an UltraBook, which means it doesn t have an Ethernet port. So I ve had not much choice other than to live and suffer deal with it. The WiFi chip also provides the Bluetooth module, which so far I had not used much. From my /etc/modprobe.d/blacklist-memstick.conf, all relevant BT modules were added to the blacklist, all this time.
rrs@lenovo:~$ cat /etc/modprobe.d/blacklist-memstick.conf 
blacklist memstick
blacklist rtsx_usb_ms
# And bluetooth too
#blacklist btusb
#blacklist btrtl
#blacklist btbcm
#blacklist btintel
#blacklist bluetooth
21:21            
Also to keep in mind is that the driver for my card gives a very misleading kernel message, which is one of the many reasons for this blog post, so that I don t forget it a couple of months later. The missing firmware error message is okay to ignore, as per this upstream comment.
Jun 14 17:17:08 lenovo kernel: usbcore: registered new interface driver btusb
Jun 14 17:17:08 lenovo systemd[1]: Mounted /boot/efi.
Jun 14 17:17:08 lenovo kernel: Bluetooth: hci0: RTL: examining hci_ver=06 hci_rev=000b lmp_ver=06 lmp_subver=8723
Jun 14 17:17:08 lenovo kernel: Bluetooth: hci0: RTL: rom_version status=0 version=1
Jun 14 17:17:08 lenovo kernel: Bluetooth: hci0: RTL: loading rtl_bt/rtl8723b_fw.bin
Jun 14 17:17:08 lenovo kernel: bluetooth hci0: firmware: direct-loading firmware rtl_bt/rtl8723b_fw.bin
Jun 14 17:17:08 lenovo kernel: Bluetooth: hci0: RTL: loading rtl_bt/rtl8723b_config.bin
Jun 14 17:17:08 lenovo kernel: bluetooth hci0: firmware: failed to load rtl_bt/rtl8723b_config.bin (-2)
Jun 14 17:17:08 lenovo kernel: firmware_class: See https://wiki.debian.org/Firmware for information about missing firmware
Jun 14 17:17:08 lenovo kernel: bluetooth hci0: Direct firmware load for rtl_bt/rtl8723b_config.bin failed with error -2
Jun 14 17:17:08 lenovo kernel: Bluetooth: hci0: RTL: cfg_sz -2, total sz 22496
This device s network + bt are on the same chip.
01:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8723BE PCIe Wireless Network Adapter
And then, when the btusb module is initialed (along with the misleading driver message), you ll get the following in your USB device listing
Bus 002 Device 005: ID 0bda:b728 Realtek Semiconductor Corp. Bluetooth Radio

Sony PlayStation 3 BD Remote Almost 10 years ago, I bought the PS3 and many of its accessories. The remote has just been rotting in the shelf. It had rusted so bad that it is better described with these pics.
Rusted inside
Rusted inside
Rusted inside and cover
Rusted inside and cover
Rusted spring
Rusted spring
The rust was so much that the battery holding spring gave up. A little bit scrubbing and cleaning has gotten it working. I hope it lasts for some time before I find time to open it up and give it a full clean-up.

Pairing the BD Remote to laptop Honestly, with the condition of the hardware and software on both ends, I did not have much hopes of getting this to work. And in all the years on my computer usage, I hardly recollect much days when I ve made use of BT. Probably, because the full BT stack wasn t that well integrated in Linux, earlier. And I mostly used to disable them in hardware and software to save on battery. All yielded results from the internet talked about tools/scripts that were either not working, pointing to broken links etc. These days, bluez comes with a nice utility, bluetoothctl. It was a nice experience using it. First, start your bluetooth service and ensure that the device talks well with the kernel
rrs@lenovo:~$ systemctl status bluetooth                                                                                                          
  bluetooth.service - Bluetooth service                                                                                                           
     Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)                                                      
     Active: active (running) since Mon 2020-06-15 12:54:58 IST; 3s ago                                                                           
       Docs: man:bluetoothd(8)                                                                                                                    
   Main PID: 310197 (bluetoothd)                                                                                                                  
     Status: "Running"                                                                                                                            
      Tasks: 1 (limit: 9424)                                                                                                                      
     Memory: 1.3M                                                                                                                                 
     CGroup: /system.slice/bluetooth.service                                                                                                      
              310197 /usr/lib/bluetooth/bluetoothd                                                                                               
                                                                                                                                                  
Jun 15 12:54:58 lenovo systemd[1]: Starting Bluetooth service...                                                                                  
Jun 15 12:54:58 lenovo bluetoothd[310197]: Bluetooth daemon 5.50                                                                                  
Jun 15 12:54:58 lenovo systemd[1]: Started Bluetooth service.                                                                                     
Jun 15 12:54:58 lenovo bluetoothd[310197]: Starting SDP server                                                                                    
Jun 15 12:54:58 lenovo bluetoothd[310197]: Bluetooth management interface 1.15 initialized                                                        
Jun 15 12:54:58 lenovo bluetoothd[310197]: Sap driver initialization failed.                                                                      
Jun 15 12:54:58 lenovo bluetoothd[310197]: sap-server: Operation not permitted (1)                                                                
12:55                                                                                                                                       
Next, then is to discover and connect to your device
rrs@lenovo:~$ bluetoothctl 
Agent registered
[bluetooth]# devices
Device E6:3A:32:A4:31:8F MI Band 2
Device D4:B8:FF:43:AB:47 MI RC
Device 00:1E:3D:10:29:0F BD Remote Control
[CHG] Device 00:1E:3D:10:29:0F Connected: yes
[BD Remote Control]# info 00:1E:3D:10:29:0F
Device 00:1E:3D:10:29:0F (public)
        Name: BD Remote Control
        Alias: BD Remote Control
        Class: 0x0000250c
        Paired: no
        Trusted: yes
        Blocked: no
        Connected: yes
        LegacyPairing: no
        UUID: Human Interface Device... (00001124-0000-1000-8000-00805f9b34fb)
        UUID: PnP Information           (00001200-0000-1000-8000-00805f9b34fb)
        Modalias: usb:v054Cp0306d0100
[bluetooth]# 
In case of the Sony BD Remote, there s no need to pair. In fact, trying to pair fails. It prompts for the PIN code, but neither 0000 or 1234 are accepted. So, the working steps so far are to Trust the device and then Connect the device. For the sake of future use, I also populated /etc/bluetooth/input.conf based on suggestions on the internet. Note: The advertised keymappings in this config file do not work. Note: I m only using it for the power saving measures in instructing the BT connection to sleep after 3 minutes.
rrs@priyasi:/tmp$ cat input.conf 
# Configuration file for the input service
# This section contains options which are not specific to any
# particular interface
[General]
# Set idle timeout (in minutes) before the connection will
# be disconnect (defaults to 0 for no timeout)
IdleTimeout=3
# Enable HID protocol handling in userspace input profile
# Defaults to false (HIDP handled in HIDP kernel module)
#UserspaceHID=true
# Limit HID connections to bonded devices
# The HID Profile does not specify that devices must be bonded, however some
# platforms may want to make sure that input connections only come from bonded
# device connections. Several older mice have been known for not supporting
# pairing/encryption.
# Defaults to false to maximize device compatibility.
#ClassicBondedOnly=true
# LE upgrade security
# Enables upgrades of security automatically if required.
# Defaults to true to maximize device compatibility.
#LEAutoSecurity=true
#
#[00:1E:3D:10:29:0F]
[2c:33:7a:8e:d6:30]
[PS3 Remote Map]
# When the 'OverlayBuiltin' option is TRUE (the default), the keymap uses
# the built-in keymap as a starting point.  When FALSE, an empty keymap is
# the starting point.
#OverlayBuiltin = TRUE
#buttoncode = keypress    # Button label = action with default key mappings
#OverlayBuiltin = FALSE
0x16 = KEY_ESC            # EJECT = exit
0x64 = KEY_MINUS          # AUDIO = cycle audio tracks
0x65 = KEY_W              # ANGLE = cycle zoom mode
0x63 = KEY_T              # SUBTITLE = toggle subtitles
0x0f = KEY_DELETE         # CLEAR = delete key
0x28 = KEY_F8             # /TIME = toggle through sleep
0x00 = KEY_1              # NUM-1
0x01 = KEY_2              # NUM-2
0x02 = KEY_3              # NUM-3
0x03 = KEY_4              # NUM-4
0x04 = KEY_5              # NUM-5
0x05 = KEY_6              # NUM-6
0x06 = KEY_7              # NUM-7
0x07 = KEY_8              # NUM-8
0x08 = KEY_9              # NUM-9
0x09 = KEY_0              # NUM-0
0x81 = KEY_F2             # RED = red
0x82 = KEY_F3             # GREEN = green
0x80 = KEY_F4             # BLUE = blue
0x83 = KEY_F5             # YELLOW = yellow
0x70 = KEY_I              # DISPLAY = show information
0x1a = KEY_S              # TOP MENU = show guide
0x40 = KEY_M              # POP UP/MENU = menu
0x0e = KEY_ESC            # RETURN = back/escape/cancel
0x5c = KEY_R              # TRIANGLE/OPTIONS = cycle through recording options
0x5d = KEY_ESC            # CIRCLE/BACK = back/escape/cancel
0x5f = KEY_A              # SQUARE/VIEW = Adjust Playback timestretch
0x5e = KEY_ENTER          # CROSS = select
0x54 = KEY_UP             # UP = Up/Skip forward 10 minutes
0x56 = KEY_DOWN           # DOWN = Down/Skip back 10 minutes
0x57 = KEY_LEFT           # LEFT = Left/Skip back 5 seconds
0x55 = KEY_RIGHT          # RIGHT = Right/Skip forward 30 seconds
0x0b = KEY_ENTER          # ENTER = select
0x5a = KEY_F10            # L1 = volume down
0x58 = KEY_J              # L2 = decrease the play speed
0x51 = KEY_HOME           # L3 = commercial skip previous
0x5b = KEY_F11            # R1 = volume up
0x59 = KEY_U              # R2 = increase the play speed
0x52 = KEY_END            # R3 = commercial skip next
0x43 = KEY_F9             # PS button = mute
0x50 = KEY_M              # SELECT = menu (as per PS convention)
0x53 = KEY_ENTER          # START = select / Enter (matches terminology in mythwelcome)
0x30 = KEY_PAGEUP         # PREV = jump back (default 10 minutes)
0x76 = KEY_J              # INSTANT BACK (newer RCs only) = decrease the play speed
0x75 = KEY_U              # INSTANT FORWARD (newer RCs only) = increase the play speed
0x31 = KEY_PAGEDOWN       # NEXT = jump forward (default 10 minutes)
0x33 = KEY_COMMA          # SCAN BACK =  decrease scan forward speed / play
0x32 = KEY_P              # PLAY = play/pause
0x34 = KEY_DOT            # SCAN FORWARD decrease scan backard speed / increase playback speed; 3x, 5, 10, 20, 30, 60, 120, 180
0x60 = KEY_LEFT           # FRAMEBACK = Left/Skip back 5 seconds/rewind one frame
0x39 = KEY_P              # PAUSE = play/pause
0x38 = KEY_P              # STOP = play/pause
0x61 = KEY_RIGHT          # FRAMEFORWARD = Right/Skip forward 30 seconds/advance one frame
0xff = KEY_MAX
21:48                 
I have not spent much time finding out why not all the key presses work. Especially, given that most places on the internet mention these mappings. For me, some of the key scan codes aren t even reported. For keys like L1, L2, L3, R1, R2, R3, Next_Item, Prev_Item, they generate no codes in the kernel. If anyone has suggestions, ideas or fixes, I d appreciate if you can drop a comment or email me privately. But given my limited use to get a simple remote ready, to be usable with Kodi, I was apt with only some of the keys working.

Mapping the keys in Kodi With the limited number of keys detected, mapping those keys to what Kodi could use was the next step. Kodi has a very nice and easy to use module, Keymap Editor. It is very simple to use and map detected keys to functionalities you want. With it, I was able to get a functioning remote to use with my Kodi HTPC setup.

Update: Wed Jun 17 11:38:20 2020 One annoying problem that breaks the overall experience is the following bug on the driver side, that results in connections not being established instantly. Once the device goes into sleep mode, in random attempts, waking up and re-establishing a BT connection can be multi-poll affair. This can last from a couple of seconds to well over minute. Random suggestions on the internet mention disabling the autosuspend functionality for the device in the driver with btusb.enable_autosuspend=n, but that did not help in this case. Given that this device is enumberated over the USB Bus, it probably needs this feature applied to the whole USB tree of the device s chain. Something to investigate over the weekend.
Jun 16 20:41:23 lenovo kernel: Bluetooth: hci0: ACL packet for unknown connection handle 7
Jun 16 20:41:43 lenovo kernel: Bluetooth: hci0: ACL packet for unknown connection handle 8
Jun 16 20:41:59 lenovo kernel: Bluetooth: hci0: ACL packet for unknown connection handle 9
Jun 16 20:42:18 lenovo kernel: input: BD Remote Control as /devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/bluetooth/hci0/hci0:10/0005:054C:030>
Jun 16 20:42:18 lenovo kernel: sony 0005:054C:0306.0006: input,hidraw1: BLUETOOTH HID v1.00 Gamepad [BD Remote Control] on 2c:33:7a:8e:d6:30
Jun 16 20:51:59 lenovo kernel: input: BD Remote Control as /devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/bluetooth/hci0/hci0:11/0005:054C:030>
Jun 16 20:51:59 lenovo kernel: sony 0005:054C:0306.0007: input,hidraw1: BLUETOOTH HID v1.00 Gamepad [BD Remote Control] on 2c:33:7a:8e:d6:30
Jun 16 21:05:55 lenovo rtkit-daemon[1723]: Supervising 3 threads of 1 processes of 1 users.
Jun 16 21:05:55 lenovo rtkit-daemon[1723]: Successfully made thread 32747 of process 1646 owned by '1000' RT at priority 5.
Jun 16 21:05:55 lenovo rtkit-daemon[1723]: Supervising 4 threads of 1 processes of 1 users.
Jun 16 21:05:56 lenovo kernel: Bluetooth: hci0: ACL packet for unknown connection handle 12
Jun 16 21:06:12 lenovo kernel: Bluetooth: hci0: ACL packet for unknown connection handle 1
Jun 16 21:06:34 lenovo kernel: Bluetooth: hci0: ACL packet for unknown connection handle 2
Jun 16 21:06:59 lenovo kernel: input: BD Remote Control as /devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/bluetooth/hci0/hci0:3/0005:054C:0306>
Jun 16 21:06:59 lenovo kernel: sony 0005:054C:0306.0008: input,hidraw1: BLUETOOTH HID v1.00 Gamepad [BD Remote Control] on 2c:33:7a:8e:d6:30

Others There s a package, kodi-eventclients-ps3, which can be used to talk to the BD Remote. Unfortunately, it isn t up-to-date. When trying to make use of it, I ran into a couple of problems. First, the easy one is:
rrs@lenovo:~/ps3pair$ kodi-ps3remote localhost 9777
usr/share/pixmaps/kodi//bluetooth.png
Traceback (most recent call last):
  File "/usr/bin/kodi-ps3remote", line 220, in <module>
  File "/usr/bin/kodi-ps3remote", line 208, in main
    xbmc.connect(host, port)
    packet = PacketHELO(self.name, self.icon_type, self.icon_file)
  File "/usr/lib/python3/dist-packages/kodi/xbmcclient.py", line 285, in __init__
    with open(icon_file, 'rb') as f:
11:16         => 1  
This one was simple as it was just a broken path. The second issue with the tool is a leftover from python2 to python3 conversion.
rrs@lenovo:/etc/bluetooth$ kodi-ps3remote localhost
/usr/share/pixmaps/kodi//bluetooth.png
Searching for BD Remote Control
(Hold Start + Enter on remote to make it discoverable)
Redmi (E8:5A:8B:73:57:44) in range
Living Room TV (E4:DB:6D:24:23:E9) in range
Could not find BD Remote Control. Trying again...
Searching for BD Remote Control
(Hold Start + Enter on remote to make it discoverable)
Living Room TV (E4:DB:6D:24:23:E9) in range
Redmi (E8:5A:8B:73:57:44) in range
Could not find BD Remote Control. Trying again...
Searching for BD Remote Control
(Hold Start + Enter on remote to make it discoverable)
BD Remote Control (00:1E:3D:10:29:0F) in range
Found BD Remote Control with address 00:1E:3D:10:29:0F
Attempting to pair with remote
Remote Paired.
Traceback (most recent call last):
  File "/usr/bin/kodi-ps3remote", line 221, in <module>
    main()
  File "/usr/bin/kodi-ps3remote", line 212, in main
    if process_keys(remote, xbmc):
  File "/usr/bin/kodi-ps3remote", line 164, in process_keys
    keycode = data.encode("hex")[10:12]
AttributeError: 'bytes' object has no attribute 'encode'
11:24         => 1  
Fixing that too did not give me the desired result on using the BD Remote in the way I want. So eventually, I gave up and used Kodi s Keymap Editor instead.

Next Next in line, when I can manage to get some free time, is to improve the Kodi Video Scraper to have a fallback mode. Currently, for files where it cannot determine the content, it reject the file resulting in those files not showing up in your collection at all. A better approach would have been to have a fallback mode, that when the scraper cannot determine the content, it should fallback to using the filename scraper

14 June 2020

Steve Kemp: Writing a brainfuck compiler.

So last night I had the idea that it might be fun to write a Brainfuck compiler, to convert BF programs into assembly language, where they'd run nice and quickly. I figured I could allocate a day to do the work, and it would be a pleasant distraction on a Sunday afternoon. As it happened it only took me three hours from start to finish. There are only a few instructions involved in brainfuck: The Wikipedia link early shows how you can convert cleanly to a C implementation, so my first version just did that: The second step was just as simple: The following program produces the string "Hello World!" to the console:
++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.
My C-compiler converted that to the program:
extern int putchar(int);
extern char getchar();
char array[30000];
int idx = 0;
int main (int arc, char *argv[])  
  array[idx]++;
  array[idx]++;
  array[idx]++;
  array[idx]++;
  array[idx]++;
  array[idx]++;
  array[idx]++;
  array[idx]++;
  while (array[idx])  
    idx++;
    array[idx]++;
    array[idx]++;
    array[idx]++;
    array[idx]++;
    while (array[idx])  
      idx++;
      array[idx]++;
      array[idx]++;
      idx++;
      array[idx]++;
      array[idx]++;
      array[idx]++;
    ..
    ..
The assembly language version is even longer:

global _start
section .text
_start:
  mov r8, stack
  add byte [r8], 8
label_loop_8:
  cmp byte [r8], 0
  je close_loop_8
  add r8, 1
  add byte [r8], 4
label_loop_14:
  cmp byte [r8], 0
  je close_loop_14
  add r8, 1
  add byte [r8], 2
  add r8, 1
  add byte [r8], 3
  add r8, 1
  add byte [r8], 3
  add r8, 1
  add byte [r8], 1
  sub r8, 4
  sub byte [r8], 1
  jmp label_loop_14
close_loop_14:
  add r8, 1
  add byte [r8], 1
  ..
  mov rax, 60
  mov rdi, 0
  syscall
section .bss
stack: resb 300000
Annoyingly the assembly language version ran slower than the C-version, which I was sneakily compiling with "gcc -O3 .." to ensure it was optimized. The first thing that I did was to convert it to fold adjacent instructions. Instead of generating separate increment instructions for ">>>" I instead started to generate "add xx, 3". That didn't help as much as I'd hoped, but it was still a big win. After that I made a minor tweak to the way that loops are handled to compare at the end of the loop as well as the start, and that shaved off a fraction of a second. As things stand I think I'm "done". It might be nice to convert the generated assembly language to something gcc can handle, to drop the dependency on nasm, but I don't feel a pressing need for that yet. Was a fun learning experience, and I think I'll come back to optimization later.

Evgeni Golov: naked pings 2020

ajax' post about "ping" etiquette is over 10 years old, but holds true until this day. So true, that my IRC client at work has a script, that will reply with a link to it each time I get a naked ping. But IRC is not the only means of communication. There is also mail, (video) conferencing, and GitHub/GitLab. Well, at least in the software engineering context. Oh and yes, it's 2020 and I still (proudly) have no Slack account. Thankfully, (naked) pings are not really a thing for mail or conferencing, but I see an increasing amount of them on GitHub and it bothers me, a lot. As there is no direct messaging on GitHub, you might rightfully ask why, as there is always context in form of the issue or PR the ping happened in, so lean back an listen ;-) notifications become useless While there might be context in the issue/PR, there is none (besides the title) in the notification mail, and not even the title in the notification from the Android app (which I have installed as I use it lot for smaller reviews). So the ping will always force a full context switch to open the web view of the issue in question, removing the possibility to just swipe away the notification/mail as "not important right now". even some context is not enough context Even after visiting the issue/PR, the ping quite often remains non-actionable. Do you want me to debug/fix the issue? Review the PR? Merge it? Close it? I don't know! The only actionable ping is when the previous message is directed at me and has an actionable request in it and the ping is just a reminder that I have to do it. And even then, why not write "hey @evgeni, did you have time to process my last question?" or something similar? BTW, this is also what I dislike about ajax' minimal example "ping re bz 534027" - what am I supposed to do with that BZ?! why me anyways?! Unless I am the only maintainer of a repo or the author of the issue/PR, there is usually no reason to ping me directly. I might be sick, or on holiday, or currently not working on that particular repo/topic or whatever. Any of that will result in you thinking that your request will be prioritized, while in reality it won't. Even worse, somebody might come across it, see me mentioned and think "ok, that's Evgeni's playground, I'll look elsewhere". Most organizations have groups of people working on specific topics. If you know the group name and have enough permissions (I am not exactly sure which, just that GitHub have limits to avoid spam, sorry) you can ping @organization/group and everyone in that group will get a notification. That's far from perfect, but at least this will get the attention of the right people. Sometimes there is also a bot that will either automatically ping a group of people or you can trigger to do so. Oh, and I'm getting paid for work on open source. So if you end up pinging me in a work-related repository, there is a high chance I will only process that during work hours, while another co-worker might have been available to help you out almost immediately. be patient Unless we talked on another medium before and I am waiting for it, please don't ping directly after creation of the issue/PR. Maintainers get notifications about new stuff and will triage and process it at some point. conclusion If you feel called out, please don't take it personally. Instead, please try to provide as much actionable information as possible and be patient, that's the best way to get a high quality result. I will ignore pings where I don't immediately know what to do, and so should you. one more thing Oh, and if you ping me on IRC, with context, and then disconnect before I can respond In the past you would sometimes get a reply by mail. These days the request will be most probably ignored. I don't like talking to the void. Sorry.

12 June 2020

Ulrike Uhlig: The right to demand change

Two women sit in an office, one asks: "What's the difference between being assertive and being aggressive?" The other replies: "Your gender." (Cartoon by Judy Horacek, 1999.) When a person of a marginalized group (read: a person with less privilege, a person with lower rank) is being framed and blamed as being aggressive, she is being told that her behavior is unacceptable. Marginalized people have learnt that they need to comply to fit, and are likely to suppress their feelings. By being framed as aggressive, the marginalized person is also being told that what they are saying cannot be listened to because the way they are saying it does not comply with expectations. There is a word for this: tone policing. This great comic by Robot Hugs has all the important details. Tone policing is a silencing tactic in which privileged participants of a discussion one-sidedly define the terms of the conversation. This tactic has the interesting side effect of shifting the responsibility to prove that one is not aggressive, hostile, explosive, a minefield, etc. to the person being framed and blamed - proving that one is worthy to be listened to. (Some of those words are actual quotes taken from real life.) Years ago, I worked in a company in which my female developer colleague would put herself in a state of overly expressed sorriness, all the while pretending to be stupid and helpless whenever she needed to ask anything from the sysadmins. When I confronted her with that, she replied: "I do it because it works." In the same company, another woman who generally asked assertively for what she needed ended up being insulted by one of the project managers using the word "dominatrix". While the example comes from my own experience, this kind of thing happens across any oppression/privilege boundaries. In some conversations, be they verbal or written, frustration and anger of one person are sometimes being mistaken by the communication partner for aggressiveness. Why is this happening? Asking a person with privilege to see, question, or change their behavior, questions their privilege. I'm thinking that it might be that most people think of themselves as "being good" and when they are being asked to question themselves or their behavior, their self-image is being challenged. "Me? But I did not do anything wrong! It's certainly not my fault if you are being oppressed! I sacrificed myself to reach my current position in life!" This comic by Toby Morris, "On a Plate", explains it quite nicely. So are we stuck with seeing conversations derail? I'd argue instead that while anger and frustration are unpleasant feelings, they're important: they show us that our boundaries have been crossed, that we want something to change, to stop, or that we need something different right now. We have the right to be angry and to demand change.

11 June 2020

Antoine Beaupr : CVE-2020-13777 GnuTLS audit: be scared

So CVE-2020-13777 came out while I wasn't looking last week. The GnuTLS advisory (GNUTLS-SA-2020-06-03) is pretty opaque so I'll refer instead to this tweet from @FiloSottile (Go team security lead):
PSA: don't rely on GnuTLS, please. CVE-2020-13777 Whoops, for the past 10 releases most TLS 1.0 1.2 connection could be passively decrypted and most TLS 1.3 connections intercepted. Trivially. Also, TLS 1.2 1.0 session tickets are awful.
You are reading this correctly: supposedly encrypted TLS connections made with affected GnuTLS releases are vulnerable to passive cleartext recovery attack (and active for 1.3, but who uses that anyways). That is extremely bad. It's pretty close to just switching everyone to HTTP instead of HTTPS, more or less. I would have a lot more to say about the security of GnuTLS in particular -- and security in general -- but I am mostly concerned about patching holes in the roof right now, so this article is not about that. This article is about figuring out what, exactly, was exposed in our infrastructure because of this.

Affected packages Assuming you're running Debian, this will show a list of packages that Depends on GnuTLS:
apt-cache --installed rdepends libgnutls30   grep '^ '   sort -u
This assumes you run this only on hosts running Buster or above. Otherwise you'll need to figure out a way to pick machines running GnuTLS 3.6.4 or later. Note that this list only first level dependencies! It is perfectly possible that another package uses GnuTLS without being listed here. For example, in the above list I have libcurl3-gnutls, so the be really thorough, I would actually need to recurse down the dependency tree. On my desktop, this shows an "interesting" list of targets:
  • apt
  • cadaver - AKA WebDAV
  • curl & wget
  • fwupd - another attack on top of this one
  • git (through the libcurl3-gnutls dependency)
  • mutt - all your emails
  • weechat - your precious private chats
Arguably, fetchers like apt, curl, fwupd, and wget rely on HTTPS for "authentication" more than secrecy, although apt has its own OpenPGP-based authentication so that wouldn't matter anyways. Still, this is truly distressing. And I haven't mentioned here things like gobby, network-manager, systemd, and others - the scope of this is broad. Hell, even good old lynx links against GnuTLS. In our infrastructure, the magic command looks something like this:
cumin -o txt -p 0  'F:lsbdistcodename=buster' "apt-cache --installed rdepends libgnutls30   grep '^ '   sort -u"   tee gnutls-rdepds-per-host   awk ' print $NF '   sort   uniq -c   sort -n
There, the result is even more worrisome, as those important packages seem to rely on GnuTLS for their transport security:
  • mariadb - all MySQL traffic and passwords
  • mandos - full disk encryption
  • slapd - LDAP passwords
mandos is especially distressing although it's probably not vulnerable because it seems it doesn't store the cleartext -- it's encrypted with the client's OpenPGP public key -- so the TLS tunnel never sees the cleartext either. Other reports have also mentioned the following servers link against GnuTLS and could be vulnerable:
  • exim
  • rsyslog
  • samba
  • various VNC implementations

Not affected Those programs are not affected by this vulnerability:
  • apache2
  • gnupg
  • python
  • nginx
  • openssh
This list is not exhaustive, naturally, but serves as an example of common software you don't need to worry about. The vulnerability only exists in GnuTLS, as far as we know, so programs linking against other libraries are not vulnerable. Because the vulnerability affects session tickets -- and those are set on the server side of the TLS connection -- only users of GnuTLS as a server are vulnerable. This means, for example, that while weechat uses GnuTLS, it will only suffer from the problem when acting as a server (which it does, in relay mode) or, of course, if the remote IRC server also uses GnuTLS. Same with apt, curl, wget, or git: it is unlikely to be a problem because it is only used as a client; the remote server is usually a webserver -- not git itself -- when using TLS.

Caveats Keep in mind that it's not because a package links against GnuTLS that it uses it. For example, I have been told that, on Arch Linux, if both GnuTLS and OpenSSL are available, the mutt package will use the latter, so it's not affected. I haven't confirmed that myself nor have I checked on Debian. Also, because it relies on session tickets, there's a time window after which the ticket gets cycled and properly initialized. But that is apparently 6 hours by default so it is going to protect only really long-lasting TLS sessions, which are uncommon, I would argue. My audit is limited. For example, it might have been better to walk the shared library dependencies directly, instead of relying on Debian package dependencies.

Other technical details It seems the vulnerability might have been introduced in this merge request, itself following a (entirely reasonable) feature request to make it easier to rotate session tickets. The merge request was open for a few months and was thoroughly reviewed by a peer before being merged. Interestingly, the vulnerable function (_gnutls_initialize_session_ticket_key_rotation), explicitly says:
 * This function will not enable session ticket keys on the server side. That is done
 * with the gnutls_session_ticket_enable_server() function. This function just initializes
 * the internal state to support periodical rotation of the session ticket encryption key.
In other words, it thinks it is not responsible for session ticket initialization, yet it is. Indeed, the merge request fixing the problem unconditionally does this:
memcpy(session->key.initial_stek, key->data, key->size);
I haven't reviewed the code and the vulnerability in detail, so take the above with a grain of salt. The full patch is available here. See also the upstream issue 1011, the upstream advisory, the Debian security tracker, and the Redhat Bugzilla.

Moving forward The impact of this vulnerability depends on the affected packages and how they are used. It can range from "meh, someone knows I downloaded that Debian package yesterday" to "holy crap my full disk encryption passwords are compromised, I need to re-encrypt all my drives", including "I need to change all LDAP and MySQL passwords". It promises to be a fun week for some people at least. Looking ahead, however, one has to wonder whether we should follow @FiloSottile's advice and stop using GnuTLS altogether. There are at least a few programs that link against GnuTLS because of the OpenSSL licensing oddities but that has been first announced in 2015, then definitely and clearly resolved in 2017 -- or maybe that was in 2018? Anyways it's fixed, pinky-promise-I-swear, except if you're one of those weirdos still using GPL-2, of course. Even though OpenSSL isn't the simplest and secure TLS implementation out there, it could preferable to GnuTLS and maybe we should consider changing Debian packages to use it in the future. But then again, the last time something like this happened, it was Heartbleed and GnuTLS wasn't affected, so who knows... It is likely that people don't have OpenSSL in mind when they suggest moving away from GnuTLS and instead think of other TLS libraries like mbedtls (previously known as PolarSSL), NSS, BoringSSL, LibreSSL and so on. Not that those are totally sinless either... "This is fine", as they say...

Holger Levsen: 20200611-stress-management

Stress management I've got a note hanging in my kitchen which is from an unknown source. So while I still can share it happily, I sadly cannot give proper credit. (Update: it was pointed out to me privately that the story is probably coming from Kathy Hadley, a life coach. Thanks for sharing, Kathy!) It reads:
A psychologist walked around a room while teaching stress management to an
audience. As she raised a glass of water, everyone expected they'd be asked
the "half empty or half full' question. Instead, with a smile on her face, she
inquired: "How heavy is this glass of water?"
Answers called out ranged from 8oz to 20oz.
She replied, "The absolute weight doesn't matter. It depends on how long I
hold it. If I hold it for a minute, it's not a problem. If I hold if for an
hour, I'll have an ache in my arm. If I hold it for a day, my arm will feel
numb and paralyzed. In each case, the weight of the glass doesn't change, but
the longer I hold it, the heavier it becomes."
She continued, "The stresses and worries in life are like that glass of water.
Think about them for a while and nothing happens. Think about them a bit
longer and they will begin to hurt. And if you think about them all day long,
you will feel paralyzed - incapable of doing anything."
Remember to put the glass down.
Especially in times like these, do remember to put the glass down!

9 June 2020

Ingo Juergensmann: Jabber vs. XMPP

XMPP is widely - and mabye better - known as Jabber. This was more or less the same until Cisco bought Jabber Inc and the trademark. You can read more about the story on the XMPP.org website. But is there still a Jabber around? Yes, it is! But Cisco Jabber is a whole infrastructure environment: you can't use Cisco Jabber client on its own without the other required Cisco infrastructure as Cisco CUCM and CIsco IM&P servers. So you can't just setup Prosody or ejabberd on your Debian server and connect Cisco Jabber to it. But what are the differences of Cisco Jabber to "standard" XMPP clients? Cisco Jabber The above screenshot from the official Cisco Jabber product webpage shows the new, single view layout of the Cisco Webex Teams client, but you can configure the client to have the old, classic split view layout of Contact List and Chat Window. But as you can already see from above screenshot audio & video calls is one of the core functions of Cisco Jabber whereas this feature has been added only lately to the well-known Conversations XMPP client on Android. Conversations is using Jingle extension to XMPP whereas Jabber uses SIP for voice/video calls. You can even use Cisco Jabber to control your deskphone via CTI, which is a quite common setup for Jabber. In fact you can configure Jabber to be just a CTI client to you phone or a fully featured UC client. When you don't want to have Ciscos full set of on-premise servers, you can also use Cisco Jabber in conjunction with Cisco Webex as Cisco Webex Messenger. Or in conjunction with Webex Teams in Teams Messaging Mode. Last month Cisco announced general availability of XMPP federation for Webex Teams/Jabber in Teams Messaging Mode. With that you have basic functionality in Webex Teams. And when I say "basic" I really mean basic: you can have 1:1 chat only, no group chats (MUC) and no Presence status will be possible. Hopefully this is just the beginning and not the end of XMPP support in Webex Teams. XMPP Clients Well, I'm sure many of you know "normal" XMPP clients such as Gajim or Dino on Linux, Conversations on Android or Siskin/Monal/ChatSecure on Apple IOS. There are plenty of other clients of course and maybe you used an XMPP client in the past without knowing it. For example Jitsi Meet is based on XMPP and you can still download the Jitsi Desktop client and use it as a full-featured multi-protocol client, e.g. for XMPP and SIP. In fact Jitsi Desktop is maybe the client that comes closest to Cisco Jabber as a chat/voice/video client. In fact I already connected Jitsi Desktop to Cisco CUCM/IM&P infrastructure, but of course you won't be able to use all those Cisco proprietary extensions, but you can see the benefit of open, standardized protocols such as XMPP and SIP: you are free to use any standard compliant client that you want. So, while Jitsi supported voice/video calls for a long time, even before they focussed on Jitsi Meet as a WebRTC based conference service, Conversations added this feature last month, as already stated. This had a huge effect to the whole XMPP federation, because you need an XMPP server that supports XEP-0215 to make these audio/video calls work. The well-known Compliance Tester listed the STUN/TURN features first as "Informational Tests", but quickly made this a mandatory test to pass tests and gain 100% on the Compliance Tester. But you cannot place SIP calls to other sides, because that's a different thing. As many of you are familiar with standard XMPP clients, I'll focus now on some similarity and differences between Cisco Jabber and standard XMPP... Similarities & Differences First, you can federate with Cisco Jabber users. Cisco IM&P can use standard XMPP federation with all other XMPP standard compliant servers. This is really a big benefit and way better than other solutions that usually results in vendor lock-in. Depending on the setup, you can even join from your own XMPP client in MUCs (Multi User Chats), which Cisco calls "Persistent Chat Room". The other way is not that simple: basically it is possible to join with Cisco Jabber in a MUC on a random server, but it is not as easy as you might thing. Cisco Jabber simply lacks a way to enter a room JID (as you can find them on https://search.jabber.network/. Instead you need to be added as participant by a moderator or an admin in that 3rd party MUC. Managed File Transfers is another issue. Cisco Jabber supports Peer-to-Peer file transfers and Managed File Transfers, where the uploaded file get transferred to an SFTP server as storage backend and where the IM&P server is handling the transfer via HTTPS. You can find a schematic drawing in the Configuration Guides. Although it appears similar to HTTP Upload as defined in XEP-0363, it is not very likely that it will work. I haven't tested it yet, because in my test scenario there is a gatekeeper in the path: Cisco Expressway doesn't support (yet) Managed File Transfer, but you can upvote the idea in the ideas management of Cisco or other ideas such as OMEMO support. OMEMO support? Yes, there is no end-to-end encryption (E2EE) currently planned for Cisco Jabber, while it is common nowadays for most modern XMPP clients. I think it would be good for Cisco Jabber to also (optionally) support OMEMO or its successor. Messaging clients without E2EE are not state of the art anymore. Whereas Conversations is the de-facto standard on Android, Apple IOS devices are still lacking a similar well-working client. See my blog post "XMPP - Fun with Clients" for a summary. In that regard Cisco Jabber might be the best XMPP client for IOS to some degree: you have working messaging, voice/video calls, Push Notifications and integration into Apples Call Kit. There are most likely many, many more differences and issues between Cisco Jabber and standard compliant XMPP servers and clients. But basically Cisco Jabber is still based on XMPP and extends that by proprietary extensions. Summary While I have the impression that the free clients and servers are well doing and increased development in the past years (thanks to Conversations and the Compliance Tester), the situation of Cisco Jabber is a little different. As a customer you can sometimes get the impression that Cisco has lost interest in developing Cisco Jabber. It got better in the last years, but when Cisco Spark was introduced some years ago, the impression was that Cisco is heavily focussed on Spark (now: Webex Teams). It's not like Cisco is not listening to customers or the development has been stopped on Jabber, but my impression is that most customers don't give feedback or tell Cisco as the vendor what they want. You can either submit ideas via the Colaboration Customer Ideas Tool or provide feedback via your Cisco and partner channels. I think it is important for the XMPP community to also have a large enterprise level vendor like Cisco. Otherwise the Internet will become more and more an Internet of closed silos like MS Teams, Slack, Facebook, etc. Of course there are other companies like ProcessOne (ejabberd) or Tigase, but I think you agree that Cisco is another level.
Kategorie:

Molly de Blanc: Racism is a Free Software Issue

Racism is a free software issue. I gave a talk that touched on this at CopyLeft Conf 2019. I also talked a little bit about it at All Things Open 2019 and FOSDEM 2020 in my talk The Ethics Behind Your IoT. I know statistics, theory, and free software. I don t know about race and racism nearly as well. I might make mistakes I have made some and I will make more. Please, when I do, help me do better. I want to look at a few particular technologies and think about how they reinforce systemic racism. Worded another way: how is technology racist? How does technology hurt Black Indigenous People of Color (BIPOC)? How does technology keep us racist? How does technology make it easier to be racist?

Breathalyzers In the United States, Latinx folks are less likely to drink than white people and, overall, less likely to be arrested for DUIs3,4. However, they are more likely to be stopped by police while driving5,6. Who is being stopped by police is up to the police and they pull over a disproportionate number of Latinx drivers. After someone is pulled over for suspected drunk driving, they are given a breathalyzer test. Breathalyzers are so easy to (un)intentionally mis-calibrate that they have been banned as valid evidence in multiple states. The biases of the police are not canceled out by the technology that should, in theory, let us know whether someone is actually drunk.

Facial Recognition I could talk about for quite some time and, in fact, have. So have others. Google s image recognition software recognized black people as gorillas and to fix the issue it removed gorillas from it s image-labeling technology. Facial recognition software does a bad job at recognizing black people. In fact, it s also terrible at identifying indigenous people and other people of color. (Incidentally, it s also not great at recognizing women, but let s not talk about that right now.) As we use facial recognition technology for more things, from automated store checkouts (even more relevant in the socially distanced age of Covid-19), airport ticketing, phone unlocking, police identification, and a number of other things, it becomes a bigger problem that this software cannot tell the difference between two Asian people.

Targeted Advertising Black kids see 70% more online ads for food than white kids, and twice as many ads for junk food. In general BIPOC youth are more likely to see junk food advertisements online. This is intentional, and happens after they are identified as BIPOC youth.

Technology Reinforces Racism; Racism Builds Technology The technology we have developed reinforces racism on a society wide scale because it makes it harder for BIPOC people to interact with this world that is run by computers and software. It s harder to not be racist when the technology around us is being used to perpetuate racist paradigms. For example, if a store implements facial recognition software for checkout, black women are less likely to be identified. They are then more likely to be targeted as trying to steal from the store. We are more likely to take this to mean that black women are more likely to steal. This is how technology builds racism, People are being excluded largely because they are not building these technologies, because they are not welcome in our spaces. There simply are not enough Black and Hispanic technologists and that is a problem. We need to care about this because when software doesn t work for everyone, it doesn t work. We cannot build on the promise of free and open source software when we are excluding the majority of people.

Dirk Eddelbuettel: Rcpp Webinar Recording Available

As announced in a few tweets leading up to it, I took the date of what would have been the annual R/Finance conference as an opportunity to hold the one-hour tutorial / workshop with introductory Rcpp material which I often present on the first morning preceding the conference as a self-organized webinar. The live-streaming worked actually reasonably well via obs to youtube (even though the comprehensive software by the latter complained at times about insufficient bitstream rates the joys of living with a (near) monopolistic broadband provider whom I should leave for fiber ). Apparently around seventy people connected to the stream which is more than we usually have in the seminar room at UIC for the R/Finance morning. The recording is now available here, and has already been seen over 200 times:

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 June 2020

Steve McIntyre: Interesting times, and a new job!

It's (yet again!) been a while since I blogged last, sorry... It's been over ten years since I started in Arm, and nine since I joined Linaro as an assignee. It was wonderful working with some excellent people in both companies, but around the end of last year I started to think that it might be time to look for something new and different. As is the usual way in Cambridge, I ended up mentioning this to friends and things happened! After discussions with a few companies, I decided to accept an interesting-looking offer from a Norwegian company called Pexip. My good friend Vince had been raving for a while about how much he enjoyed his job there, which was a very good sign! He works from his home near Cambridge, and they were very happy to take me on in a similar way. There will be occasional trips to the UK office near Reading, or to the Norway HQ in Oslo. But most of the time I'll be working in my home office with all the home comforts and occasionally even an office dog! Pepper and a laptop! As is common in the UK for senior staff, I had to give 3 months notice with my resignation. When I told my boss in Arm way way back in February that I had decided to leave, I planned for a couple of weeks of down-time in between jobs. Perfect timing! The third week of May in Cambridge is the summer Beer Festival, and my birthday is the week after. All was looking good! Then the world broke... :-( As the "novel coronavirus" swept the world, countries closed down and normal life all-but disappeared for many. I acknowledge I'm very lucky here - I'm employed as a software engineer. I can effectively work from home, and indeed I was already in the habit of doing that anyway. Many people are not so fortunate. :-/ In this period, I've heard of some people in the middle of job moves where their new company have struggled and the new job has gone away. Thankfully, Pexip have continued to grow during this time and were still very keen to have me. I finally started this week! So, what does Pexip do? The company develops and supplies a video conferencing platform, mainly targeting large enterprise customers. We have some really awesome technology, garnering great reviews from customers all over the world. See the website for more information! Pexip logo Where do I fit in? Pexip is a relatively small company with a very flat setup in engineering, so that's a difficult question to answer! I'll be starting working in the team developing and maintaining PexOS, the small Linux-based platform on which other things depend. (No prizes for guessing which distro it's based on!) But there's lots of scope to get involved in all kinds of other areas as needs and interests arise. I can't wait to get stuck in! Although I'm no longer going to be working on Debian arm port issues on work time, I'm still planning to help where I can. Let's see how that works...

Antoine Beaupr : Replacing Smokeping with Prometheus

I've been struggling with replacing parts of my old sysadmin monitoring toolkit (previously built with Nagios, Munin and Smokeping) with more modern tools (specifically Prometheus, its "exporters" and Grafana) for a while now. Replacing Munin with Prometheus and Grafana is fairly straightforward: the network architecture ("server pulls metrics from all nodes") is similar and there are lots of exporters. They are a little harder to write than Munin modules, but that makes them more flexible and efficient, which was a huge problem in Munin. I wrote a Migrating from Munin guide that summarizes those differences. Replacing Nagios is much harder, and I still haven't quite figured out if it's worth it.

How does Smokeping work Leaving those two aside for now, I'm left with Smokeping, which I used in my previous job to diagnose routing issues, using Smokeping as a decentralized looking glass, which was handy to debug long term issues. Smokeping is a strange animal: it's fundamentally similar to Munin, except it's harder to write plugins for it, so most people just use it for Ping, something for which it excels at. Its trick is this: instead of doing a single ping and returning this metrics, it does multiple ones and returns multiple metrics. Specifically, smokeping will send multiple ICMP packets (20 by default), with a low interval (500ms by default) and a single retry. It also pings multiple hosts at once which means it can quickly scan multiple hosts simultaneously. You therefore see network conditions affecting one host reflected in further hosts down (or up) the chain. The multiple metrics also mean you can draw graphs with "error bars" which Smokeping shows as "smoke" (hence the name). You also get per-metric packet loss. Basically, smokeping runs this command and collects the output in a RRD database:
fping -c $count -q -b $backoff -r $retry -4 -b $packetsize -t $timeout -i $mininterval -p $hostinterval $host [ $host ...]
... where those parameters are, by default:
  • $count is 20 (packets)
  • $backoff is 1 (avoid exponential backoff)
  • $timeout is 1.5s
  • $mininterval is 0.01s (minimum wait interval between any target)
  • $hostinterval is 1.5s (minimum wait between probes on a single target)
It can also override stuff like the source address and TOS fields. This probe will complete between 30 and 60 seconds, if my math is right (0% and 100% packet loss).

How do draw Smokeping graphs in Grafana A naive implementation of Smokeping in Prometheus/Grafana would be to use the blackbox exporter and create a dashboard displaying those metrics. I've done this at home, and then I realized that I was missing something. Here's what I did.
  1. install the blackbox exporter:
    apt install prometheus-blackbox-exporter
    
  2. make sure to allow capabilities so it can ping:
    dpkg-reconfigure prometheus-blackbox-exporter
    
  3. hook monitoring targets into prometheus.yml (the default blackbox exporter configuration is fine):
    scrape_configs:
      - job_name: blackbox
          metrics_path: /probe
          params:
            module: [icmp]
          scrape_interval: 5s
          static_configs:
            - targets:
              - octavia.anarc.at
              # hardcoded in DNS
              - nexthop.anarc.at
              - koumbit.net
              - dns.google
          relabel_configs:
            - source_labels: [__address__]
              target_label: __param_target
            - source_labels: [__param_target]
              target_label: instance
            - target_label: __address__
              replacement: 127.0.0.1:9115  # The blackbox exporter's real hostname:port.
    
    Notice how we lower the scrape_interval to 5 seconds to get more samples. nexthop.anarc.at was added into DNS to avoid hardcoding my upstream ISP's IP in my configuration.
  4. use this Grafana panel to graph the results. It was created with this query:
    sum(probe_icmp_duration_seconds phase="rtt" ) by (instance)
    
    • Set the Legend field to instance RTT
    • Set Draw modes to lines and Mode options to staircase
    • Set the Left Y axis Unit to duration(s)
    • Show the Legend As table, with Min, Avg, Max and Current enabled
    Then this query, for packet loss:
    1-avg_over_time(probe_success[$__interval])!=0 or null
    
    • Set the Legend field to instance packet loss
    • Set a Add series override to Lines: false, Null point mode: null, Points: true, Points Radius: 1, Color: deep red, and, most importantly, Y-axis: 2
    • Set the Right Y axis Unit to percent (0.0-1.0) and set Y-max to 1
    Then set the entire thing to Repeat, on target, vertically. And you need to add a target variable like label_values(probe_success, instance).
The result looks something like this:
A plot of RTT and packet loss over time of three nodes Not bad, but not Smokeping
This actually looks pretty good! The resulting dashboard is available in the Grafana dashboard repository.

What is missing? Now, that doesn't exactly look like Smokeping, does it. It's pretty good, but it's not quite what we want. What is missing is variance, the "smoke" in Smokeping. There's a good article about replacing Smokeping with Grafana. They wrote a custom script to write samples into InfluxDB so unfortunately we can't use it in this case, since we don't have InfluxDB's query language. I couldn't quite figure out how to do the same in PromQL. I tried:
stddev(probe_icmp_duration_seconds phase="rtt",instance=~"$instance" )
stddev_over_time(probe_icmp_duration_seconds phase="rtt",instance=~"$instance" [$__interval])
stddev_over_time(probe_icmp_duration_seconds phase="rtt",instance=~"$instance" [1m])
The first two give zero for all samples. The latter works, but doesn't look as good as Smokeping. So there might be something I'm missing. SuperQ wrote a special exporter for this called smokeping_prober that came out of this discussion in the blackbox exporter. Instead of delegating scheduling and target definition to Prometheus, the targets are set in the exporter. They also take a different approach than Smokeping: instead of recording the individual variations, they delegate that to Prometheus, through the use of "buckets". Then they use a query like this:
histogram_quantile(0.9 rate(smokeping_response_duration_seconds_bucket[$__interval]))
This is the rationale to SuperQ's implementation:
Yes, I know about smokeping's bursts of pings. IMO, smokeping's data model is flawed that way. This is where I intentionally deviated from the smokeping exact way of doing things. This prober sends a smooth, regular series of packets in order to be measuring at regular controlled intervals. Instead of 20 packets, over 10 seconds, every minute. You send one packet per second and scrape every 15. This has the same overall effect, but the measurement is, IMO, more accurate, as it's a continuous stream. There's no 50 second gap of no metrics about the ICMP stream. Also, you don't get back one metric for those 20 packets, you get several. Min, Max, Avg, StdDev. With the histogram data, you can calculate much more than just that using the raw data. For example, IMO, avg and max are not all that useful for continuous stream monitoring. What I really want to know is the 90th percentile or 99th percentile. This smokeping prober is not intended to be a one-to-one replacement for exactly smokeping's real implementation. But simply provide similar functionality, using the power of Prometheus and PromQL to make it better. [...] one of the reason I prefer the histogram datatype, is you can use the heatmap panel type in Grafana, which is superior to the individual min/max/avg/stddev metrics that come from smokeping. Say you had two routes, one slow and one fast. And some pings are sent over one and not the other. Rather than see a wide min/max equaling a wide stddev, the heatmap would show a "line" for both routes.
That's an interesting point. I have also ended up adding a heatmap graph to my dashboard, independently. And it is true it shows those "lines" much better... So maybe that, if we ignore legacy, we're actually happy with what we get, even with the plain blackbox exporter. So yes, we're missing pretty "fuzz" lines around the main lines, but maybe that's alright. It would be possible to do the equivalent to the InfluxDB hack, with queries like:
min_over_time(probe_icmp_duration_seconds phase="rtt",instance=~"$instance" [30s])
avg_over_time(probe_icmp_duration_seconds phase="rtt",instance=~"$instance" [5m])
max_over_time(probe_icmp_duration_seconds phase="rtt",instance=~"$instance" [30s])
The output looks something like this:
A plot of RTT and packet loss over time of three nodes, with minimax Looks more like Smokeping!
But there's a problem there: see how the middle graph "dips" sometimes below 20ms? That's the min_over_time function (incorrectly, IMHO) returning zero. I haven't quite figured out how to fix that, and I'm not sure it is better. But it does look more like Smokeping than the previous graph. Update: I forgot to mention one big thing that this setup is missing. Smokeping has this nice feature that you can order and group probe targets in a "folder"-like hierarchy. It is often used to group probes by location, which makes it easier to scan a lot of targets. This is harder to do in this setup. It might be possible to setup location-specific "jobs" and select based on that, but it's not exactly the same.

Credits Credits to Chris Siebenmann for his article about Prometheus and pings which gave me the avg_over_time query idea.

3 June 2020

Dirk Eddelbuettel: littler 0.3.10: Some more updates

max-heap image The eleventh release of littler as a CRAN package is now available, following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later. littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years. littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default where a good idea?) and simply does not exist on Windows (yet the build system could be extended see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo, as well as in the examples vignette. This release adds a new helper / example script installBioc.r for BioConductor package installation, generalizes the roxygenize() wrapper roxy.r a little, and polished a couple of other corners. The NEWS file entry is below.

Changes in littler version 0.3.10 (2020-06-02)
  • Changes in examples
    • The update.r script only considers writeable directories.
    • The rcc.r script tries to report full logs by setting _R_CHECK_TESTS_NLINES_=0.
    • The tt.r script has an improved ncpu fallback.
    • Several installation and updating scripts set _R_SHLIB_STRIP_ to TRUE.
    • A new script installBioc.r was added.
    • The --error option to install2.r was generalized (Sergio Oller in #78).
    • The roxy.r script was extended a little.
  • Changes in package
    • Travis CI now uses R 4.0.0 and the bionic distro

CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. Comments and suggestions are welcome at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Keith Packard: picolibc-ryu

Float/String Conversion in Picolibc: Enter Ry I recently wrote about this topic having concluded that the best route for now was to use the malloc-free, but imprecise, conversion routines in the tinystdio alternative. A few days later, Sreepathi Pai pointed me at some very recent work in this area: This is amazing! Thirty years after the papers referenced in the previous post, Ulf Adams came up with some really cool ideas and managed to reduce the math required for 64-bit conversion to 128 bit integers. This is a huge leap forward; we were doing long multi-precision computations before, and now it's all short enough to fit in registers (ok, a lot of registers, but still). Getting the Ry Code The code is available on github: https://github.com/ulfjack/ryu. Reading through it, it's very clear that the author focuses on performance with lots of tuning for common cases. Still, it's quite readable, especially compared with the newlib multi-precision based code. Picolibc String/Float conversion interface Picolibc has some pretty basic needs for the float/string conversion code, it wants four functions:
  1. __dtoa_engine
    int
    __dtoa_engine(double x, struct dtoa *dtoa, uint8_t max_digits, uint8_t max_decimals);
    
    This converts the double x to a string of decimal digits and a decimal exponent stored inside the 'dtoa' struct. It limits the total number of digits to max_digits and, optionally (when max_decimals is non-zero), limits the number of fractional digits to max_decimals - 1. This latter supports 'f' formats. Returns the number of digits stored, which is <= max_digits. Less if the number can be accurately represented in fewer digits.
  2. __ftoa_engine
    int
    __ftoa_engine(float x, struct ftoa *ftoa, uint8_t max_digits, uint8_t max_decimals);
    
    The same as __dtoa_engine, except for floats.
  3. __atod_engine
    double
    __atod_engine(uint64_t m10, int e10);
    
    To avoid needing to handle stdio inside the conversion function, __atod_engine receives fully parsed values, the base-10 significand (m10) and exponent (e10). The value to convert is m10 * pow(10, e10).
  4. __atof_engine
    float
    __atof_engine(uint32_t m10, int e10);
    
    The same as __atod_engine, except for floats.
With these, it can do printf, scanf, ecvt, fcvt, gcvt, strtod, strtof and atof. Porting Ry to Picolibc The existing Ry float-to-string code always generates the number of digits necessary for accurate output. I had to hack it up to generate correctly rounded shorter output when max_digits or max_decimals were smaller. I'm not sure I managed to do that correctly, but at least it appears to be passing all of the test cases I have. In normal operation, Ry iteratively removes digits from the answer that aren't necessary to disambiguate with neighboring values. What I changed was to keep removing digits using that method until the answer had few enough digits to fit in the desired length. There's some tricky rounding code that adjusts the final result and I had to bypass that if I'd removed extra digits. That was about the only change necessary to the core algorithm. I also trimmed the code to only include the general case and not the performance improvements, then wrapped it with code to provide the _engine interface. On the string-to-float side, most of what I needed to do was remove the string parsing bits at the start of the function and switch from performance-optimized to space-optimized versions of a couple of internal routines. Correctness Results Because these new functions are now 'exact', I was able to adjust the picolibc tests to compare all of the bits for string/float conversion instead of having to permit a bit of slop in the answers. With those changes, the picolibc test suite passes, which offers some assurance that things aren't completely broken. Size Results Snek uses the 32-bit float versions of the conversion routines, and for that, the size difference is:
   text    data     bss     dec     hex filename
  59068      44   37968   97080   17b38 snek-qemu-riscv-orig.elf
  59430      44   37968   97442   17ca2 snek-qemu-riscv-ryu.elf
    362
362 bytes added to gain accurate printf/strtof results seems like a good trade-off in this case. Performance I haven't measured performance at all, but I suspect that it won't be nearly as problematic on most platforms as the source code makes it appear. And that's because Ry is entirely integer arithmetic with no floating point at all. This avoids using the soft fp code for platforms without hardware float support. Pointers to the Code I haven't merged this to picolibc master yet, it's on the ryu branch: Review, especially of the hack above to return short results, would be greatly appreciated! Thanks again to Ulf Adams for creating this code and to Sreepathi Pai for sending me a note about it!

29 May 2020

Gunnar Wolf: Heads up Online MiniDebConf is Online

I know most Debian people know about this already But in case you don t follow the usual Debian communications channels, this might interest you! Given most of the world is still under COVID-19 restrictions, and that we want to work on Debian, given there is no certainty as to what the future holds in store for us Our DPL fearless as they always are had the bold initiative to make this weekend into the first-ever miniDebConf Online (MDCO)! miniDebConf Online So, we are already halfway through DebCamp (which means, you can come and hang out with us in the debian.social DebCamp Jitsi lounge, where some impromptu presentations might happen (or not). Starting tomorrow morning (11AM UTC), we will have a quite interesting set of talks. I am reproducing the schedule here:

Saturday 2020.05.30
Time (UTC) Speaker Talk
11:00 - 11:10 MDCO team members Hello + Welcome
11:30 - 11:50 Wouter Verhelst Extrepo
12:00 - 12:45 JP Mengual Debian France, trust european organization
13:00 - 13:20 Arnaud Ferraris Bringing Debian to mobile phones, one package at a time
13:30 - 15:00 Lunch Break A chance for the teams to catch some air
15:00 - 15:45 JP Mengual The community team, United Nations Organizations of Debian?
16:00 - 16:45 Christoph Biedl Clevis and tang - overcoming the disk unlocking problem
17:00 - 17:45 Antonio Terceiro I m a programmer, how can I help Debian

Sunday 2020.05.31
Time (UTC) Speaker Talk
11:00 - 11:45 Andreas Tille The effect of Covid-19 on the Debian Med project
12:00 - 12:45 Paul Gevers BoF: running autopkgtest for your package
13:00 - 13:20 Ben Hutchings debplate: Build many binary packages with templates
13:30 - 15:00 Lunch break A chance for the teams to catch some air
15:00 - 15:45 Holger Levsen Reproducing bullseye in practice
16:00 - 16:45 Jonathan Carter Striving towards excellence
17:00 - 17:45 Delib* Organizing Peer-to-Peer Debian Facilitation Training
18:00 - 18:15 MDCO team members Closing
  • subject to confirmation

Timezone Remember this is an online event, meant for all of the world! Yes, the chosen times seem quite Europe-centric (but they are mostly a function of the times the talk submitters requested). Talks are 11:00 18:00UTC, which means, 06:00 13:00 Mexico (GMT-5), 20:00 03:00 Japan (GMT+9), 04:00 11:00 Western Canada/USA/Mexico (GMT-7) and the rest of the world, somewhere in between. (No, this was clearly not optimized for our dear usual beer team. Sorry! I guess we need you to be fully awake at beertime!)

[update] Connecting! Of course, I didn t make it clear at first how to connect to the Online miniDebConf, silly me!
  • The video streams are available at: https://video.debconf.org/
  • Suggested: tune in to the #minidebconf-online IRC channel in OFTC.
That should be it. Hope to see you there! (Stay home, stay safe )

Gunnar Wolf: Heads up Online MiniDebConf is Online

I know most Debian people know about this already But in case you don t follow the usual Debian communications channels, this might interest you! Given most of the world is still under COVID-19 restrictions, and that we want to work on Debian, given there is no certainty as to what the future holds in store for us Our DPL fearless as they always are had the bold initiative to make this weekend into the first-ever miniDebConf Online (MDCO)! miniDebConf Online So, we are already halfway through DebCamp (which means, you can come and hang out with us in the debian.social DebCamp Jitsi lounge, where some impromptu presentations might happen (or not). Starting tomorrow morning (11AM UTC), we will have a quite interesting set of talks. I am reproducing the schedule here:

Saturday 2020.05.30
Time (UTC) Speaker Talk
11:00 - 11:10 MDCO team members Hello + Welcome
11:30 - 11:50 Wouter Verhelst Extrepo
12:00 - 12:45 JP Mengual Debian France, trust european organization
13:00 - 13:20 Arnaud Ferraris Bringing Debian to mobile phones, one package at a time
13:30 - 15:00 Lunch Break A chance for the teams to catch some air
15:00 - 15:45 JP Mengual The community team, United Nations Organizations of Debian?
16:00 - 16:45 Christoph Biedl Clevis and tang - overcoming the disk unlocking problem
17:00 - 17:45 Antonio Terceiro I m a programmer, how can I help Debian

Sunday 2020.05.31
Time (UTC) Speaker Talk
11:00 - 11:45 Andreas Tille The effect of Covid-19 on the Debian Med project
12:00 - 12:45 Paul Gevers BoF: running autopkgtest for your package
13:00 - 13:20 Ben Hutchings debplate: Build many binary packages with templates
13:30 - 15:00 Lunch break A chance for the teams to catch some air
15:00 - 15:45 Holger Levsen Reproducing bullseye in practice
16:00 - 16:45 Jonathan Carter Striving towards excellence
17:00 - 17:45 Delib* Organizing Peer-to-Peer Debian Facilitation Training
18:00 - 18:15 MDCO team members Closing
  • subject to confirmation

Timezone Remember this is an online event, meant for all of the world! Yes, the chosen times seem quite Europe-centric (but they are mostly a function of the times the talk submitters requested). Talks are 11:00 18:00UTC, which means, 06:00 13:00 Mexico (GMT-5), 20:00 03:00 Japan (GMT+9), 04:00 11:00 Western Canada/USA/Mexico (GMT-7) and the rest of the world, somewhere in between. (No, this was clearly not optimized for our dear usual beer team. Sorry! I guess we need you to be fully awake at beertime!)

[update] Connecting! Of course, I didn t make it clear at first how to connect to the Online miniDebConf, silly me!
  • The video streams are available at: https://video.debconf.org/
  • Suggested: tune in to the #minidebconf-online IRC channel in OFTC.
That should be it. Hope to see you there! (Stay home, stay safe )

28 May 2020

Elana Hashman: Presenter mode in LibreOffice Impress without an external display

I typically use LibreOffice Impress for my talks, much to some folks' surprise. Yes, you can make slides look okay with free software! But there's one annoying caveat that has bothered me for ages. Impress makes it nearly impossible to enter presenter mode with a single display, while also displaying slides. I have never understood this limitation, but it's existed for a minimum of seven years. I've tried all sorts of workarounds over the years, including a macro that forces LibreOffice into presenter mode, which I never was able to figure out how to reverse once I ran it... This has previously been an annoyance but never posed a big problem, because when push came to shove I could leave my house and use an external monitor or screen when presenting at meetups. But now, everything's virtual, I'm in lockdown indefinitely, and I don't have another display available at home. And about 8 hours before speaking at a meetup today, I realized I didn't have a way to share my slides while seeing my speaker notes. Uh oh. So I got this stupid idea. ...why don't I just placate LibreOffice with a FAKE display? Virtual displays with xrandr My GPU had this capability innately, it turns out, if I could just whisper the right incantations to unlock its secrets:
ehashman@red-dot:~$ cat /usr/share/X11/xorg.conf.d/20-intel.conf 
Section "Device"
    Identifier "intelgpu0"
    Driver "intel"
    Option "VirtualHeads" "1"
EndSection
After restarting X to allow this newly created config to take effect, I now could see two new virtual displays available for use:
ehashman@red-dot:~$ xrandr
Screen 0: minimum 8 x 8, current 3840 x 1080, maximum 32767 x 32767
eDP1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 310mm x 170mm
   1920x1080     60.01*+  59.93  
   ...
   640x360       59.84    59.32    60.00  
DP1 disconnected (normal left inverted right x axis y axis)
DP2 disconnected (normal left inverted right x axis y axis)
HDMI1 disconnected (normal left inverted right x axis y axis)
HDMI2 disconnected (normal left inverted right x axis y axis)
VIRTUAL1 disconnected (normal left inverted right x axis y axis)
VIRTUAL2 disconnected (normal left inverted right x axis y axis)
Nice. Now, to actually use it:
ehashman@red-dot:~$ xrandr --addmode VIRTUAL1 1920x1080
ehashman@red-dot:~$ xrandr --output VIRTUAL1 --mode 1920x1080 --right-of eDP1
And indeed, after running these commands, I found myself with a virtual display, very happy to black hole all my windows, available to the imaginary right of my laptop screen. This allowed me to mash that "Present" button in LibreOffice and get my presenter notes on my laptop display, while putting my actual slides in a virtual time-out that I could still screenshare! Wouldn't it be nice if LibreOffice just fixed the actual bug? Well, actually... I must forgive myself for my stage panic. The talk ended up going great, and the immediate problem was solved. But it turns out this bug has been addressed upstream! It's just... not well-documented. A couple years ago, there was a forum post on ask.libreoffice.org that featured this exact question, and a solution was provided!
Yes, use Open Expert Configuration via Tools > Options > LibreOffice > Advanced. Search for StartAlways. You should get a node org.openoffice.Office.PresenterScreen with line Presenter. Double-click that line to toggle the boolean value to true.
I tested this out locally and... yeah that works. But it turns out the bug from 2013 had not been updated with this solution until just a few months ago. There are very limited search results for this configuration property. I wish this was much better documented. But so it goes with free software; here's a hack and a real solution as we all try to improve things :)

27 May 2020

Kees Cook: security things in Linux v5.5

Previously: v5.4. I got a bit behind on this blog post series! Let s get caught up. Here are a bunch of security things I found interesting in the Linux kernel v5.5 release: restrict perf_event_open() from LSM
Given the recurring flaws in the perf subsystem, there has been a strong desire to be able to entirely disable the interface. While the kernel.perf_event_paranoid sysctl knob has existed for a while, attempts to extend its control to block all perf_event_open() calls have failed in the past. Distribution kernels have carried the rejected sysctl patch for many years, but now Joel Fernandes has implemented a solution that was deemed acceptable: instead of extending the sysctl, add LSM hooks so that LSMs (e.g. SELinux, Apparmor, etc) can make these choices as part of their overall system policy. generic fast full refcount_t
Will Deacon took the recent refcount_t hardening work for both x86 and arm64 and distilled the implementations into a single architecture-agnostic C version. The result was almost as fast as the x86 assembly version, but it covered more cases (e.g. increment-from-zero), and is now available by default for all architectures. (There is no longer any Kconfig associated with refcount_t; the use of the primitive provides full coverage.) linker script cleanup for exception tables
When Rick Edgecombe presented his work on building Execute-Only memory under a hypervisor, he noted a region of memory that the kernel was attempting to read directly (instead of execute). He rearranged things for his x86-only patch series to work around the issue. Since I d just been working in this area, I realized the root cause of this problem was the location of the exception table (which is strictly a lookup table and is never executed) and built a fix for the issue and applied it to all architectures, since it turns out the exception tables for almost all architectures are just a data table. Hopefully this will help clear the path for more Execute-Only memory work on all architectures. In the process of this, I also updated the section fill bytes on x86 to be a trap (0xCC, int3), instead of a NOP instruction so functions would need to be targeted more precisely by attacks. KASLR for 32-bit PowerPC
Joining many other architectures, Jason Yan added kernel text base-address offset randomization (KASLR) to 32-bit PowerPC. seccomp for RISC-V
After a bit of long road, David Abdurachmanov has added seccomp support to the RISC-V architecture. The series uncovered some more corner cases in the seccomp self tests code, which is always nice since then we get to make it more robust for the future! seccomp USER_NOTIF continuation
When the seccomp SECCOMP_RET_USER_NOTIF interface was added, it seemed like it would only be used in very limited conditions, so the idea of needing to handle normal requests didn t seem very onerous. However, since then, it has become clear that the overhead of a monitor process needing to perform lots of normal open() calls on behalf of the monitored process started to look more and more slow and fragile. To deal with this, it became clear that there needed to be a way for the USER_NOTIF interface to indicate that seccomp should just continue as normal and allow the syscall without any special handling. Christian Brauner implemented SECCOMP_USER_NOTIF_FLAG_CONTINUE to get this done. It comes with a bit of a disclaimer due to the chance that monitors may use it in places where ToCToU is a risk, and for possible conflicts with SECCOMP_RET_TRACE. But overall, this is a net win for container monitoring tools. EFI_RNG_PROTOCOL for x86
Some EFI systems provide a Random Number Generator interface, which is useful for gaining some entropy in the kernel during very early boot. The arm64 boot stub has been using this for a while now, but Dominik Brodowski has now added support for x86 to do the same. This entropy is useful for kernel subsystems performing very earlier initialization whre random numbers are needed (like randomizing aspects of the SLUB memory allocator). FORTIFY_SOURCE for MIPS
As has been enabled on many other architectures, Dmitry Korotin got MIPS building with CONFIG_FORTIFY_SOURCE, so compile-time (and some run-time) buffer overflows during calls to the memcpy() and strcpy() families of functions will be detected. limit copy_ to,from _user() size to INT_MAX
As done for VFS, vsnprintf(), and strscpy(), I went ahead and limited the size of copy_to_user() and copy_from_user() calls to INT_MAX in order to catch any weird overflows in size calculations. Other things
Alexander Popov pointed out some more v5.5 features that I missed in this blog post. I m repeating them here, with some minor edits/clarifications. Thank you Alexander! Edit: added Alexander Popov s notes That s it for v5.5! Let me know if there s anything else that I should call out here. Next up: Linux v5.6.

2020, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

Russ Allbery: Review: Middlegame

Review: Middlegame, by Seanan McGuire
Publisher: Tor
Copyright: May 2019
ISBN: 1-250-19551-9
Format: Kindle
Pages: 528
Roger and Dodger are cuckoo children, alchemical constructs created by other alchemical constructs masquerading as humans. They are halves of the primal force of the universe, the Doctrine of Ethos (which is not what the Doctrine of Ethos is, but that is one of my lesser problems with this book), divided into language and math and kept separate to properly mature. In this case, separate means being adopted by families on opposite coasts of the United States, ignorant of each other's existence and closely monitored by agents Reed controls. None of that prevents Roger and Dodger from becoming each other's invisible friends at the age of seven, effortlessly communicating psychically even though they've never met. That could have been the start of an enjoyable story that hearkened back to an earlier age of science fiction: the secret science experiments discover that they have more power than their creators expected, form a clandestine alliance, and fight back against the people who are trying to control them. I have fond memories of Escape to Witch Mountain and would have happily read that book. Unfortunately, that isn't the story McGuire wanted to tell. The story she told involves ripping Roger and Dodger apart, breaking Dodger, and turning Roger into an abusive asshole. Whooboy, where to start. This book made me very angry, in a way that I would not have been if it didn't contain the bones of a much better novel. Four of them, to be precise: four other books that would have felt less gratuitously cruel and less apparently oblivious to just how bad Roger's behavior is. There are some things to like. One of them is that the structure of this book is clever. I can't tell you how it's clever because the structure doesn't become clear until more than halfway through and it completely changes the story in a way that would be a massive spoiler. But it's an interesting spin on an old idea, one that gave Roger and Dodger a type of agency in the story that has far-ranging implications. I enjoyed thinking about it. That leads me to another element I liked: Erin. She makes only fleeting appearances until well into the story, but I thought she competed with Dodger for being the best character of the book. The second of the better novels I saw in the bones of Middlegame was the same story told from Erin's perspective. I found myself guessing at her motives and paying close attention to hints that led to a story with a much different emotional tone. Viewing the ending of the book through her eyes instead of Roger and Dodger's puts it in a different, more complicated, and more thought-provoking light. Unfortunately, she's not McGuire's protagonist. She instead is one of the monsters of this book, which leads to my first, although not my strongest, complaint. It felt like McGuire was trying too hard to write horror, packing Middlegame with the visuals of horror movies without the underlying structure required to make them effective. I'm not a fan of horror personally, so to some extent I'm grateful that the horrific elements were ineffective, but it makes for some frustratingly bad writing. For example, one of the longest horror scenes in the book features Erin, and should be a defining moment for the character. Unfortunately, it's so heavy on visuals and so focused on what McGuire wants the reader to be thinking that it doesn't show any of the psychology underlying Erin's decisions. The camera is pointed the wrong way; all the interesting storytelling work, moral complexity, and world-building darkness is happening in the character we don't get to see. And, on top of that, McGuire overuses foreshadowing so much that it robs the scene of suspense and terror. Again, I'm partly grateful, since I don't read books for suspense and terror, but it means the scene does only a fraction of the work it could. This problem of trying too hard extends to the writing. McGuire has a bit of a tendency in all of her books to overdo the descriptions, but is usually saved by narrative momentum. Unfortunately, that's not true here, and her prose often seems overwrought. She also resorts to this style of description, which never fails to irritate me:
The thought has barely formed when a different shape looms over him, grinning widely enough to show every tooth in its head. They are even, white, and perfect, and yet he somehow can't stop himself from thinking there's something wrong with them, that they're mismatched, that this assortment of teeth was never meant to share a single jaw, a single terrible smile.
This isn't effective. This is telling the reader how they're supposed to feel about the thing you're describing, without doing the work of writing a description that makes them feel that way. (Also, you may see what I mean by overwrought.) That leads me to my next complaint: the villains. My problem is not so much with Leigh, who I thought was an adequate monster, if a bit single-note. There's some thought and depth behind her arguments with Reed, a few hints of her own motives that were more convincing for not being fully shown. The descriptions of how dangerous she is were reasonably effective. She's a good villain for this type of dark fantasy story where the world is dangerous and full of terrors (and reminded me of some of the villains from McGuire's October Daye series). Reed, though, is a storytelling train wreck. The Big Bad of the novel is the least interesting character in it. He is a stuffed tailcoat full of malicious incompetence who is only dangerous because the author proclaims him to be. It only adds insult to injury that he kills off a far more nuanced and creative villain before the novel starts, replacing her ambiguous goals with Snidely Whiplash mustache-twirling. The reader has to suffer through extended scenes focused on him as he brags, monologues, and obsesses over his eventual victory without an ounce of nuance or subtlety. Worse is the dynamic between him and Leigh, which is only one symptom of the problem with Middlegame that made me the most angry: the degree to this book oozes patriarchy. Every man in this book, including the supposed hero, orders around the women, who are forced in various ways to obey. This is the most obvious between Leigh and Reed, but it's the most toxic, if generally more subtle, between Roger and Dodger. Dodger is great. I had absolutely no trouble identifying with and rooting for her as a character. The nasty things that McGuire does to her over the course of the book (and wow does that never let up) made me like her more when she tenaciously refuses to give up. Dodger is the math component of the Doctrine of Ethos, and early in the book I thought McGuire handled that well, particularly given how difficult it is to write a preternatural genius. Towards the end of this book, her math sadly turns into a very non-mathematical magic (more on this in a moment), but her character holds all the way through. It felt like she carved her personality out of this story through sheer force of will and clung to it despite the plot. I wanted to rescue her from this novel and put her into a better book, such as the one in which her college friends (who are great; McGuire is very good at female friendships when she writes them) stage an intervention, kick a few people out of her life, and convince her to trust them. Unfortunately, Dodger is, by authorial fiat, half of a bound pair, and the other half of that pair is Roger, who is the sort of nice guy everyone likes and thinks is sweet and charming until he turns into an emotional trap door right when you need him the most and dumps you into the ocean to drown. And then somehow makes you do all the work of helping him feel better about his betrayal. The most egregious (and most patriarchal) thing Roger does in this book is late in the book and a fairly substantial spoiler, so I can't rant about that properly. But even before that, Roger keeps doing the the same damn emotional abandonment trick, and the book is heavily invested into justifying it and making excuses for him. Excuses that, I should note, are not made for Dodger; her failings are due to her mistakes and weaknesses, whereas Roger's are natural reactions to outside forces. I got very, very tired of this, and I'm upset by how little awareness the narrative voice showed for how dysfunctional and abusive this relationship is. The solution is always for Dodger to reunite with Roger; it's built into the structure of the story. I have a weakness for the soul-bound pair, in part from reading a lot of Mercedes Lackey at an impressionable age, but one of the dangerous pitfalls of the concept is that the characters then have to have an almost flawless relationship. If not, it can turn abusive very quickly, since the characters by definition cannot leave each other. It's essentially coercive, so as soon as the relationship shows a dark side, the author needs to be extremely careful. McGuire was not. There is an attempted partial patch, late in the book, for the patriarchal structure. One of the characters complains about it, and another says that the gender of the language and math pairs is random and went either way in other pairs. Given that both of the pairs that we meet in this story have the same male-dominant gender dynamic, what I took from this is that McGuire realized there was a problem but wasn't able to fix it. (I'm also reminded of David R. Henry's old line that it's never a good sign when the characters start complaining about the plot.) The structural problems are all the more frustrating because I think there were ways out of them. Roger is supposedly the embodiment of language, not that you'd be able to tell from most scenes in this novel. For reasons that I do not understand, McGuire expressed that as a love of words: lexicography, translation, and synonyms. This makes no sense to me. Those are some of the more structured and rules-based (and hence mathematical) parts of language. If Roger had instead been focused on stories collecting them, telling them, and understanding why and how they're told he would have had a clearer contrast with Dodger. More importantly, it would have solved the plot problem that McGuire solved with a nasty bit of patriarchy. So much could have been done with Dodger building a structure of math around Roger's story-based expansion of the possible, and it would have grounded Dodger's mathematics in something more interesting than symbolic magic. To me, it's such an obvious lost opportunity. I'm still upset about this book. McGuire does a lovely bit of world-building with Asphodel Baker, what little we see of her. I found the hidden alchemical war against her work by L. Frank Baum delightful, and enjoyed every excerpt from the fictional Over the Woodward Wall scattered throughout Middlegame. But a problem with inventing a fictional book to excerpt in a real novel is that the reader may decide that the fictional book sounds a lot better than the book they're reading, and start wishing they could just read that book instead. That was certainly the case for me. I'm sad that Over the Woodward Wall doesn't exist, and am mostly infuriated by Middlegame. Dodger and Erin deserved to live in a better book. Should you want to read this anyway (and I do know people who liked it), serious content warning for self-harm. Rating: 4 out of 10

Next.

Previous.