Russ Allbery: Review: Lower Ed
| Publisher: | The New Press |
| Copyright: | 2017 |
| Printing: | 2018 |
| ISBN: | 1-62097-472-X |
| Format: | Kindle |
| Pages: | 217 |
| Publisher: | The New Press |
| Copyright: | 2017 |
| Printing: | 2018 |
| ISBN: | 1-62097-472-X |
| Format: | Kindle |
| Pages: | 217 |
My webcam has broken, something that I have been using a lot during the last few months for some reason.
A friend of mine suggested that I use the mic and camera on my mobile phone instead. There is a simple app droidcam that makes the phone behave as a simple webcam, it also has a client application to run on your PC to capture the web-stream and present it as a video device. All well and good but I would like to keep propitiatory software off my PCs (I have a hard enough time accepting it on a phone but I have to draw a line somewhere).
I decided that there had to be a simple way to ingest that stream and present it as a local video device on a Linux box. It turns out that it is a lot simpler than I thought it would be. I had it working within 10 minutes!
Packages needed: ffmpeg, v4l2loopback-utils
sudo apt-get install ffmpeg v4l2loopback-utilsStart the loop-back device:
sudo modprobe v4l2loopbackIngest video and present on loop-back device:
ffmpeg -re -i <URL_VideoStream> -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video0Where
-re
Read input at native frame rate. Mainly used to simulate a grab
device, or live input stream (e.g. when reading from a file). Should
not be used with actual grab devices or live input streams (where it
can cause packet loss). By default ffmpeg attempts to read the
input(s) as fast as possible. This option will slow down the reading
of the input(s) to the native frame rate of the input(s). It is
useful for real-time output (e.g. live streaming).
-i <source>
In this case my phone running droidcam http://10.14.1.100:4747/video
-vcodec rawvideo
Select the output video codec to raw video (as expected for /dev/video#)
-pix_fmt yuv420p
Set pixel format. in this example tell ffmpeg that the video will
be yuv colour space at 420p resolution
-f v4l2 /dev/video0
Force the output to be in video for Linux 2 (v4l2) format bound to
device /dev/video0
vrrp_instance gateway1 state BACKUP # interface eth0 # virtual_router_id 12 # priority 101 # virtual_ipaddress 2001:db8:ff/64
state keyword in instructs Keepalived to not take the leader
role when starting. Otherwise, incoming nodes create a temporary
disruption by taking over the IP address until the election settles.
The interface keyword in defines the interface for sending and
receiving VRRP packets. It is also the default interface to configure
the virtual IP address. The virtual_router_id directive in is
common to all nodes sharing the virtual IP. The priority keyword in
helps choosing which router will be elected as leader. If you need
more information around Keepalived, be sure to check the
documentation.
VRRP design is tied to Ethernet networks and requires a
multicast-enabled network for communication between nodes. In some
environments, notably public clouds, multicast is unavailable. In this
case, Keepalived can send VRRP packets using unicast:
vrrp_instance gateway1 state BACKUP interface eth0 virtual_router_id 12 priority 101 unicast_peer 2001:db8::11 2001:db8::12 virtual_ipaddress 2001:db8:ff/64 dev lo
notify_* scripts.
Until version 2.21 (not released yet), the interface directive is
mandatory and Keepalived will transmit and receive VRRP packets on
this interface only. If peers are reachable through several
interfaces, like on a BGP on the host setup, you need a
workaround. A simple one is to use a VXLAN interface:
$ ip -6 link add keepalived6 type vxlan id 6 dstport 4789 local 2001:db8::10 nolearning $ bridge fdb append 00:00:00:00:00:00 dev keepalived6 dst 2001:db8::11 $ bridge fdb append 00:00:00:00:00:00 dev keepalived6 dst 2001:db8::12 $ ip link set up dev keepalived6
vrrp_instance gateway1 state BACKUP interface keepalived6 mcast_src_ip 2001:db8::10 virtual_router_id 12 priority 101 virtual_ipaddress 2001:db8:ff/64 dev lo
unicast_peer can be used without
the interface directive. I think using VXLAN is still a
neat trick applicable to other situations where communication using
broadcast or multicast is needed, while the underlying network provide
no support for this.
netbox.netbox collection from Ansible Galaxy
provides several modules to update NetBox objects:
- name: create a device in NetBox netbox_device: netbox_url: http://netbox.local netbox_token: s3cret data: name: to3-p14.sfo1.example.com device_type: QFX5110-48S device_role: Compute Switch site: SFO1
Notice I recommend that you read Writing a custom Ansible module as an introduction, as well as Syncing MySQL tables for a first simpler example.
netbox_sync: source: netbox.yaml api: https://netbox.example.com token: s3cret
devices: ad2-p6.sfo1.example.com: datacenter: sfo1 manufacturer: Cisco model: Catalyst 2960G-48TC-L role: net_tor_oob_switch to1-p6.sfo1.example.com: datacenter: sfo1 manufacturer: Juniper model: QFX5110-48S role: net_tor_gpu_switch # [ ] ips: - device: ad2-p6.example.com ip: 172.31.115.18/21 interface: oob - device: to1-p6.example.com ip: 172.31.115.33/21 interface: oob - device: to1-p6.example.com ip: 172.31.254.33/32 interface: lo0.0 # [ ]
cmdb. Most objects in NetBox accept tags.
module_args = dict( source=dict(type='path', required=True), api=dict(type='str', required=True), token=dict(type='str', required=True, no_log=True), max_workers=dict(type='int', required=False, default=10) ) result = dict( changed=False ) module = AnsibleModule( argument_spec=module_args, supports_check_mode=True )
Synchronizer abstract class. For
each kind of object, a concrete class is built with the appropriate
class attributes to tune its behaviour and a wanted() method to
provide the objects we want.
I am not explaining the abstract class code here. Have a look at the
source if you want.
class SyncTags(Synchronizer): app = "extras" table = "tags" key = "name" def wanted(self): return "cmdb": dict( slug="cmdb", color="8bc34a", description="synced by network CMDB")
app and table attributes defines the NetBox objects we want
to manipulate. The key attribute is used to determine how to lookup
for existing objects. In this example, we want to lookup tags using
their names.
The wanted() method is expected to return a dictionary mapping
object keys to the list of wanted attributes. Here, the keys are tag
names and we create only one tag, cmdb, with the provided slug,
color and description. This is the tag we will use to mark the objects
we create or modify.
If the tag does not exist, it is created. If it exists, the provided
attributes are updated. Other attributes are left untouched.
We also want to create a specific tenant for objects accepting such an
attribute (devices and IP addresses):
class SyncTenants(Synchronizer): app = "tenancy" table = "tenants" key = "name" def wanted(self): return "Network": dict(slug="network", description="Network team")
wanted() method uses the information provided in the YAML file: it
walks the devices and builds a set of datacenter names.
class SyncSites(Synchronizer): app = "dcim" table = "sites" key = "name" only_on_create = ("status", "slug") def wanted(self): result = set(details["datacenter"] for details in self.source['devices'].values() if "datacenter" in details) return k: dict(slug=k, status="planned") for k in result
only_on_create attribute, the specified
attributes are not updated if they are different. The goal of this
synchronizer is mostly to collect the references to the different
sites for other objects.
>>> pprint(SyncSites(**sync_args).wanted()) 'sfo1': 'slug': 'sfo1', 'status': 'planned' , 'chi1': 'slug': 'chi1', 'status': 'planned' , 'nyc1': 'slug': 'nyc1', 'status': 'planned'
only_on_create attribute:
class SyncManufacturers(Synchronizer): app = "dcim" table = "manufacturers" key = "name" def wanted(self): result = set(details["manufacturer"] for details in self.source['devices'].values() if "manufacturer" in details) return k: "slug": slugify(k) for k in result
foreign attribute linking
a NetBox attribute to the synchronizer handling it.
class SyncDeviceTypes(Synchronizer): app = "dcim" table = "device_types" key = "model" foreign = "manufacturer": SyncManufacturers def wanted(self): result = set((details["manufacturer"], details["model"]) for details in self.source['devices'].values() if "model" in details) return k[1]: dict(manufacturer=k[0], slug=slugify(k[1])) for k in result
wanted() method refers to the manufacturer using its key
attribute. In this case, this is the manufacturer name.
>>> pprint(SyncManufacturers(**sync_args).wanted()) 'Cisco': 'slug': 'cisco' , 'Dell': 'slug': 'dell' , 'Juniper': 'slug': 'juniper' >>> pprint(SyncDeviceTypes(**sync_args).wanted()) 'ASR 9001': 'manufacturer': 'Cisco', 'slug': 'asr-9001' , 'Catalyst 2960G-48TC-L': 'manufacturer': 'Cisco', 'slug': 'catalyst-2960g-48tc-l' , 'MX10003': 'manufacturer': 'Juniper', 'slug': 'mx10003' , 'QFX10002-36Q': 'manufacturer': 'Juniper', 'slug': 'qfx10002-36q' , 'QFX10002-72Q': 'manufacturer': 'Juniper', 'slug': 'qfx10002-72q' , 'QFX5110-32Q': 'manufacturer': 'Juniper', 'slug': 'qfx5110-32q' , 'QFX5110-48S': 'manufacturer': 'Juniper', 'slug': 'qfx5110-48s' , 'QFX5200-32C': 'manufacturer': 'Juniper', 'slug': 'qfx5200-32c' , 'S4048-ON': 'manufacturer': 'Dell', 'slug': 's4048-on' , 'S6010-ON': 'manufacturer': 'Dell', 'slug': 's6010-on'
class SyncDeviceRoles(Synchronizer): app = "dcim" table = "device_roles" key = "name" def wanted(self): result = set(details["role"] for details in self.source['devices'].values() if "role" in details) return k: dict(slug=slugify(k), color="8bc34a") for k in result
class SyncDevices(Synchronizer): app = "dcim" table = "devices" key = "name" foreign = "device_role": SyncDeviceRoles, "device_type": SyncDeviceTypes, "site": SyncSites, "tenant": SyncTenants remove_unused = 10 def wanted(self): return name: dict(device_role=details["role"], device_type=details["model"], site=details["datacenter"], tenant="Network") for name, details in self.source['devices'].items() if "datacenter", "model", "role" <= set(details.keys())
remove_unused attribute is a safety implemented to fail if we
have to delete more than 10 devices: this may be the indication there
is a bug somewhere, unless one of your datacenter suddenly caught
fire.
>>> pprint(SyncDevices(**sync_args).wanted()) 'ad2-p6.sfo1.example.com': 'device_role': 'net_tor_oob_switch', 'device_type': 'Catalyst 2960G-48TC-L', 'site': 'sfo1', 'tenant': 'Network' , 'to1-p6.sfo1.example.com': 'device_role': 'net_tor_gpu_switch', 'device_type': 'QFX5110-48S', 'site': 'sfo1', 'tenant': 'Network' , [ ]
class SyncIPs(Synchronizer): app = "ipam" table = "ip-addresses" key = "address" foreign = "tenant": SyncTenants remove_unused = 1000 def wanted(self): wanted = for details in self.source['ips']: if details['ip'] in wanted: wanted[details['ip']]['description'] = \ f" details['device'] (and others)" else: wanted[details['ip']] = dict( tenant="Network", status="active", dns_name="", # information is present in DNS description=f" details['device'] : details['interface'] ", role=None, vrf=None) return wanted
cmdb, then those
already attached to an interface:
def get(self, key): """Grab IP address from NetBox.""" # There may be duplicate. We need to grab the "best". results = super(Synchronizer, self).get(key) if len(results) == 0: return None if len(results) == 1: return results[0] scores = [0]*len(results) for idx, result in enumerate(results): if "cmdb" in result.tags: scores[idx] += 10 if result.interface is not None: scores[idx] += 5 return sorted(zip(scores, results), reverse=True, key=lambda k: k[0])[0][1]
source = yaml.safe_load(open(module.params['source'])) netbox = pynetbox.api(module.params['api'], token=module.params['token']) sync_args = dict( module=module, netbox=netbox, source=source, before= , after= ) synchronizers = [synchronizer(**sync_args) for synchronizer in [ SyncTags, SyncTenants, SyncSites, SyncManufacturers, SyncDeviceTypes, SyncDeviceRoles, SyncDevices, SyncIPs ]]
prepare() method whose goal is to compute
the current and wanted states. It returns True in case of a
difference:
# Check what needs to be synchronized try: for synchronizer in synchronizers: result['changed'] = synchronizer.prepare() except AnsibleError as e: result['msg'] = e.message module.fail_json(**result)
sync_args["before"][table] and
sync_args["after"][table] where table is the name of the table for
a given NetBox object type. The diff object is a bit elaborate as
it is built table by table. This enables Ansible to display the name
of each table before the diff representation:
# Compute the diff if module._diff and result['changed']: result['diff'] = [ dict( before_header=table, after_header=table, before=yaml.safe_dump(sync_args["before"][table]), after=yaml.safe_dump(sync_args["after"][table])) for table in sync_args["after"] if sync_args["before"][table] != sync_args["after"][table] ] # Stop here if check mode is enabled or if no change if module.check_mode or not result['changed']: module.exit_json(**result)
synchronize() method to apply
changes and a cleanup() method to delete unwanted objects. Order is
important due to the relation between the objects.
# Synchronize for synchronizer in synchronizers: synchronizer.synchronize() for synchronizer in synchronizers[::-1]: synchronizer.cleanup() module.exit_json(**result)
netbox.netbox collection, the logic is written in
Python instead of trying to glue Ansible tasks together. I believe
this is both more flexible and easier to read, notably when trying to
delete outdated objects. While I did not test it, it should also be
faster. An alternative would have been to reuse code from the
netbox.netbox collection, as it contains similar primitives.
Unfortunately, I didn t think of it until now.
I have been late to adopt an on-premise cloud solution as the security of Owncloud a few years ago wasn't so stellar (cf. my comment from 2013 in Encryption files ... for synchronization across the Internet). But the follow-up product Nextcloud has matured quite nicely and we use it for collaboration both in the company and in FLOSS related work at multiple nonprofit organizations.
There is a very annoying "feature" in Nextcloud though that the designers think menu items for apps at the top need to be limited to eight or less to prevent information overload in the header. The whole item discussion is worth reading as it it an archetypical example of design prevalence vs. user choice.
And of course designers think they are right. That's a feature of the trade.
Luckily code can be changed and there are many comments floating around the Internet to change const minAppsDesktop = 8. In this case it is slightly complicated by the fact that the javascript code is distributed in compressed form (aka "minified") as core/js/dist/main.js and you probably don't want to build the whole beast locally to change one constant.
Basically
There are a number of large and very successful Debian Local Groups
(Debian France, Debian Brazil and Debian Taiwan, just to name a few),
but what can we do to help support upcoming local groups or help spark interest
in more parts of the world?
There has been a session about
Debian Local Teams at Debconf20
and it generated quite a bit of constructive discussion in the live stream
(recording available at
https://meetings-archive.debian.net/pub/debian-meetings/2020/DebConf20/),
in the session's Etherpad
and in the IRC channel (#debian-localgroups). This article is an attempt at
summarizing the key points that were raised during that discussion, as well as
the plans for the future actions to support new or existent Debian Local Groups
and the possibility of setting up a local group support team.
Pandemic situation
During a pandemic it may seem strange to discuss offline meetings, but this is
a good time to be planning things for the future. At the same time, the current
situation makes it more important than before to encourage local interaction.
Reasoning for local groups
Debian can seem scary for those outside. Already having a connection to Debian - especially
to people directly involved in it - seems to be the way through
which most contributors arrive. But if one doesn't have a connection,
it is not that easy; Local Groups facilitate that by improving networking.
Local groups are incredibly important to the success of Debian
since they often help with translations, making us more diverse, support,
setting up local bug squashing sprints, establishing a local DebConf team
along with miniDebConfs, getting sponsors for the project and much more.
Existence of a Local Groups would also facilitate access to "swag" like stickers
and mugs, since people not always have the time to deal with the process
of finding a supplier to actually get those made.
The activity of local groups might facilitate that by organizing related logistics.
How to deal with local groups, how to define a local group
Debian gathers the information about Local Groups in its
Local Groups wiki page (and subpages).
Other organisations also have their own schemes, some of them featuring a map,
blogs, or clear rules about what constitutes a local group. In the case of Debian
there is not a predefined set of "rules", even about the group name.
That is perfectly fine, we assume that certain local groups may be very small,
or temporary (created around a certain time when they plan several activities,
and then become silent). However, the way the groups are named and how they are
listed on the wiki page sets expectations with regards to what kinds of activities
they involve.
For this reason, we encourage all the Debian Local Groups to review their entries
in the Debian wiki, keep it current
(e.g. add a line "Status: Active (2020)), and we encourage informal groups of
Debian contributors that somehow "meet", to create a new entry in the wiki page, too.
What can Debian do to support Local Groups
Having a centralized database of groups is good (if up-to-date),
but not enough. We'll explore other ways of propagation and increasing visibility,
like organising the logistics of printing/sending swag and facilitate access
to funding for Debian-related events.
Continuation of efforts
Efforts shall continue regarding Local Groups.
Regular meetings are happening every two or three weeks;
interested people are encouraged to explore some other relevant DebConf20 talks
(Introducing Debian Brasil,
Debian Academy: Another way to share knowledge about Debian,
An Experience creating a local community on a small town),
websites like Debian flyers
(including other printed material as cube, stickers),
visit the events section of the Debian website
and the Debian Locations wiki page,
and participate in the IRC channel #debian-localgroups at OFTC.
Old laptop I ve been meaning to get a new laptop for a while now. My ThinkPad X250 is now 5 years old and even though it s still adequate in many ways, I tend to run out of memory especially when running a few virtual machines. It only has one memory slot, which I maxed out at 16GB shortly after I got it. Memory has been a problem in considering a new machine. Most new laptops have soldered RAM and local configurations tend to ship with 8GB RAM. Getting a new machine with only a slightly better CPU and even just the same amount of RAM as what I have in the X250 seems a bit wasteful. I was eyeing the Lenovo X13 because it s a super portable that can take up to 32GB of RAM, and it ships with an AMD Ryzen 4000 series chip which has great performance. With Lenovo s discount for Debian Developers it became even more attractive. Unfortunately that s in North America only (at least for now) so that didn t work out this time.
Enter Tongfang I ve been reading a bunch of positive reviews about the Tuxedo Pulse 14 and KDE Slimbook 14. Both look like great AMD laptops, supports up to 64GB of RAM and clearly runs Linux well. I also noticed that they look quite similar, and after some quick searches it turns out that these are made by Tongfang and that its model number is PF4NU1F. I also learned that a local retailer (Wootware) sells them as the Wootbook. I ve seen one of these before although it was an Intel-based one, but it looked like a nice machine and I was already curious about it back then. After struggling for a while to find a local laptop with a Ryzen CPU and that s nice and compact and that breaks the 16GB memory barrier, finding this one that jumped all the way to 64GB sealed the deal for me. This is the specs for the configuration I got:


Overall remarks When maxing out the CPU, the fan is louder than on a ThinkPad, I definitely noticed it while compiling the zfs-dkms module. On the plus side, that happened incredibly fast. Comparing the Wootbook to my X250, the biggest downfall it has is really it s pointing device. It doesn t have a trackpad and the touchpad is ok and completely usable, but not great. I use my laptop on a desk most of the time so using an external mouse will mostly solve that. If money were no object, I would definitely choose a maxed out ThinkPad for its superior keyboard/mouse, but the X13 configured with 32GB of RAM and 128GB of SSD retails for just about double of what I paid for this machine. It doesn t seem like you can really buy the perfect laptop no matter how much money you want to spend, there s some compromise no matter what you end up choosing, but this machine packs quite a punch, especially for its price, and so far I m very happy with my purchase and the incredible performance it provides. I m also very glad that Wootware went with the gray/black colours, I prefer that by far to the white and silver variants. It s also the first laptop I ve had since 2006 that didn t come with Windows on it. The Wootbook is also comfortable/sturdy enough to carry with one hand while open. The ThinkPads are great like this and with many other brands this just feels unsafe. I don t feel as confident carrying it by it s display because it s very thin (I know, I shouldn t be doing that with the ThinkPads either, but I ve been doing that for years without a problem :) ). There s also a post on Reddit that tracks where you can buy these machines from various vendors all over the world.
/admin1-> console com2 console: Serial Device 2 is currently in useSo far the only way I ve discovered to get console access again after that is the command racadm racreset . If anyone knows of a better way please let me know. As an aside having racreset being so close to racresetcfg (which resets the configuration to default and requires a hard reboot to configure it again) seems like a really bad idea. Host Based Management
deb http://linux.dell.com/repo/community/ubuntu xenial openmanageThe above apt sources.list line allows installing Dell management utilities (Xenial is old but they work on Debian/Buster). Probably the packages srvadmin-storageservices-cli and srvadmin-omacore will drag in enough dependencies to get it going. Here are some useful commands:
# show hardware event log omreport system esmlog # show hardware alert log omreport system alertlog # give summary of system information omreport system summary # show versions of firmware that can be updated omreport system version # get chassis temp omreport chassis temps # show physical disk details on controller 0 omreport storage pdisk controller=0RAID Control The RAID controller is known as PERC (PowerEdge Raid Controller), the Dell web site has an rpm package of the perccli tool to manage the RAID from Linux. This is statically linked and appears to have different versions of libraries used to make it. The command perccli show gives an overview of the configuration, but the command perccli /c0 show to give detailed information on controller 0 SEGVs and the kernel logs a vsyscall attempted with vsyscall=none message. Here s an overview of the vsyscall enmulation issue [2]. Basically I could add vsyscall=emulate to the kernel command line and slightly reduce security for the system to allow system calls from perccli that are called from old libc code to work, or I could run code from a dubious source as root. Some versions of IDRAC have a racadm raid command that can be run from a ssh session to perform RAID administration remotely, mine doesn t have that. As an aside the help for the RAC system doesn t list all commands and the Dell documentation is difficult to find so blog posts from other sysadmins is the best source of information. I have configured IDRAC to have all of the BIOS output go to the virtual serial console over ssh so I can see the BIOS prompt me for PERC administration but it didn t accept my key presses when I tried to do so. In any case this requires downtime and I d like to be able to change hard drives without rebooting. I found vsyscall_trace on Github [3], it uses the ptrace interface to emulate vsyscall on a per process basis. You just run vsyscall_trace perccli and it works! Thanks Geoffrey Thomas for writing this! Here are some perccli commands:
# show overview perccli show # help on adding a vd (RAID) perccli /c0 add help # show controller 0 details perccli /c0 show # add a vd (RAID) of level RAID0 (r0) with the drive 32:0 (enclosure:slot from above command) perccli /c0 add vd r0 drives=32:0When a disk is added to a PERC controller about 525MB of space is used at the end for RAID metadata. So when you create a RAID-0 with a single device as in the above example all disk data is preserved by default except for the last 525MB. I have tested this by running a BTRFS scrub on a disk from another system after making it into a RAID-0 on the PERC.
ikectl utility to help you do
so. Before going any further, you might find it useful to edit
/etc/ssl/ikeca.cnf to set some reasonable defaults for your
certificates.
Begin by creating and installing a CA certificate:
# ikectl ca vpn create
# ikectl ca vpn install
# ikectl ca vpn certificate server1.example.org create
# ikectl ca vpn certificate server1.example.org install
# ikectl ca vpn certificate server2.example.org create
# ikectl ca vpn certificate server2.example.org export
# tar -C /etc/iked -xzpvf server2.example.org.tgz
$ cat /etc/ssl/vpn/index.txt
V 210825142056Z 01 unknown /C=US/ST=Pennsylvania/L=Pittsburgh/CN=server1.example.org/emailAddress=rak@example.org
V 210825142208Z 02 unknown /C=US/ST=Pennsylvania/L=Pittsburgh/CN=server2.example.org/emailAddress=rak@example.org
/etc/iked.conf, filling in
the srcid and dstid fields appropriately.
ikev2 'server1_server2_active' active esp from server1.example.org to server2.example.org \
local server1.example.org peer server2.example.org \
srcid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server1.example.org/emailAddress=rak@example.org' \
dstid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server2.example.org/emailAddress=rak@example.org'
ikev2 'server2_server1_passive' passive esp from server2.example.org to server1.example.org \
local server2.example.org peer server1.example.org \
srcid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server2.example.org/emailAddress=rak@example.org' \
dstid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server1.example.org/emailAddress=rak@example.org'
'server1_server2_active' and
'server2_server1_passive' in the two stanzas do not matter and
can be omitted. Reload iked on both hosts:
# ikectl reload
# ikectl show sa
/var/log/daemon.
/etc/iked.conf on the remote end:
ikev2 'responder_x509' passive esp \
from 0.0.0.0/0 to 10.0.1.0/24 \
local server1.example.org peer any \
srcid server1.example.org \
config address 10.0.1.0/24 \
config name-server 10.0.1.1 \
tag "ROADW"
iked.conf(5) for details.
Reload iked:
# ikectl reload
/etc/hostname.vether0, again
configuring the address to suit your needs:
inet 10.0.1.1 255.255.255.0
# ifconfig vether0 up
# ikectl ca vpn certificate client.example.org create
# ikectl ca vpn certificate client.example.org export
# tar -C /etc/ipsec.d/ -xzf client.example.org.tgz -- \
./private/client.example.org.key \
./certs/client.example.org.crt ./ca/ca.crt
/etc/ipsec.conf,
configuring appropriately:
ca example.org
cacert=ca.crt
auto=add
conn server1
keyexchange=ikev2
right=server1.example.org
rightid=%server1.example.org
rightsubnet=0.0.0.0/0
rightauth=pubkey
leftsourceip=%config
leftauth=pubkey
leftcert=client.example.org.crt
auto=route
/etc/ipsec.secrets:
# space is important
server1.example.org : RSA client.example.org.key
# ipsec restart
# ipsec up server1
# ipsec status
ikectl utility to help you do
so. Before going any further, you might find it useful to edit
/etc/ssl/ikeca.cnf to set some reasonable defaults for your
certificates.
Begin by creating and installing a CA certificate:
# ikectl ca vpn create
# ikectl ca vpn install
# ikectl ca vpn certificate server1.example.org create
# ikectl ca vpn certificate server1.example.org install
# ikectl ca vpn certificate server2.example.org create
# ikectl ca vpn certificate server2.example.org export
# tar -C /etc/iked -xzpvf server2.example.org.tgz
$ cat /etc/ssl/vpn/index.txt
V 210825142056Z 01 unknown /C=US/ST=Pennsylvania/L=Pittsburgh/CN=server1.example.org/emailAddress=rak@example.org
V 210825142208Z 02 unknown /C=US/ST=Pennsylvania/L=Pittsburgh/CN=server2.example.org/emailAddress=rak@example.org
/etc/iked.conf, filling in
the srcid and dstid fields appropriately.
ikev2 'server1_server2_active' active esp from server1.example.org to server2.example.org \
local server1.example.org peer server2.example.org \
srcid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server1.example.org/emailAddress=rak@example.org' \
dstid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server2.example.org/emailAddress=rak@example.org'
ikev2 'server2_server1_passive' passive esp from server2.example.org to server1.example.org \
local server2.example.org peer server1.example.org \
srcid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server2.example.org/emailAddress=rak@example.org' \
dstid '/C=US/ST=Pennsylvania/L=Pittsburgh/CN=server1.example.org/emailAddress=rak@example.org'
'server1_server2_active' and
'server2_server1_passive' in the two stanzas do not matter and
can be omitted. Reload iked on both hosts:
# ikectl reload
# ikectl show sa
/var/log/daemon.
/etc/iked.conf on the remote end:
ikev2 'responder_x509' passive esp \
from 0.0.0.0/0 to 10.0.1.0/24 \
local server1.example.org peer any \
srcid server1.example.org \
config address 10.0.1.0/24 \
config name-server 10.0.1.1 \
tag "ROADW"
iked.conf(5) for details.
Reload iked:
# ikectl reload
/etc/hostname.vether0, again
configuring the address to suit your needs:
inet 10.0.1.1 255.255.255.0
# ifconfig vether0 up
# ikectl ca vpn certificate client.example.org create
# ikectl ca vpn certificate client.example.org export
# tar -C /etc/ipsec.d/ -xzf client.example.org.tgz -- \
./private/client.example.org.key \
./certs/client.example.org.crt ./ca/ca.crt
/etc/ipsec.conf,
configuring appropriately:
ca example.org
cacert=ca.crt
auto=add
conn server1
keyexchange=ikev2
right=server1.example.org
rightid=%server1.example.org
rightsubnet=0.0.0.0/0
rightauth=pubkey
leftsourceip=%config
leftauth=pubkey
leftcert=client.example.org.crt
auto=route
/etc/ipsec.secrets:
# space is important
server1.example.org : RSA client.example.org.key
# ipsec restart
# ipsec up server1
# ipsec status
In our monthly reports, we summarise the things that we have been up to over the past month. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced from the original free software source code to the pre-compiled binaries we install on our systems. If you re interested in contributing to the project, please visit our main website.
One hiccup we ve encountered in SecureDrop development is that not all Python wheels can be built reproducibly. We ship multiple (Python) projects in Debian packages, with Python dependencies included in those packages as wheels. In order for our Debian packages to be reproducible, we need that wheel build process to also be reproducibleParallel to this, transparencylog.com was also launched, a service that verifies the contents of URLs against a publicly recorded cryptographic log. It keeps an append-only log of the cryptographic digests of all URLs it has seen. (GitHub repo)
On 18th September, Bernhard M. Wiedemann will give a presentation in German, titled Wie reproducible builds Software sicherer machen ( How reproducible builds make software more secure ) at the Internet Security Digital Days 2020 conference.
Holger gave a talk titled Reproducing Bullseye in practice , focusing on independently verifying that the binaries distributed from ftp.debian.org are made from their claimed sources. It also served as a general update on the status of reproducible builds within Debian. The video (145 MB) and slides are available.
There were also a number of other talks that involved Reproducible Builds too. For example, the Malayalam language mini-conference had a talk titled , ? ( I want to join Debian, what should I do? ) presented by Praveen Arimbrathodiyil, the Clojure Packaging Team BoF session led by Elana Hashman, as well as Where is Salsa CI right now? that was on the topic of Salsa, the collaborative development server that Debian uses to provide the necessary tools for package maintainers, packaging teams and so on.
Jonathan Bustillos (Jathan) also gave a talk in Spanish titled Un camino verificable desde el origen hasta el binario ( A verifiable path from source to binary ). (Video, 88MB)
After many years of development work, the compiler for the Rust programming language now generates reproducible binary code. This generated some general discussion on Reddit on the topic of reproducibility in general.
Paul Spooren posted a request for comments to OpenWrt s openwrt-devel mailing list asking for clarification on when to raise the PKG_RELEASE identifier of a package. This is needed in order to successfully perform rebuilds in a reproducible builds context.
In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update.
Chris Lamb provided some comments and pointers on an upstream issue regarding the reproducibility of a Snap / SquashFS archive file. [ ]
.buildinfo build certificates have been tainted on the official Debian build servers, as these environments have files underneath the /usr/local/sbin directory [ ]. He also filed against bug for debrebuild after spotting that it can fail to download packages from snapshot.debian.org [ ].
This month, several issues were uncovered (or assisted) due to the efforts of reproducible builds.
For instance, Debian bug #968710 was filed by Simon McVittie, which describes a problem with detached debug symbol files (required to generate a traceback) that is unlikely to have been discovered without reproducible builds. In addition, Jelmer Vernooij called attention that the new Debian Janitor tool is using the property of reproducibility (as well as diffoscope when applying archive-wide changes to Debian:
New merge proposals also include a link to the diffoscope diff between a vanilla build and the build with changes. Unfortunately these can be a bit noisy for packages that are not reproducible yet, due to the difference in build environment between the two builds. [ ]56 reviews of Debian packages were added, 38 were updated and 24 were removed this month adding to our knowledge about identified issues. Specifically, Chris Lamb added and categorised the
nondeterministic_version_generated_by_python_param and the lessc_nondeterministic_keys toolchain issues. [ ][ ]
Holger Levsen sponsored Lukas Puehringer s upload of the python-securesystemslib pacage, which is a dependency of in-toto, a framework to secure the integrity of software supply chains. [ ]
Lastly, Chris Lamb further refined his merge request against the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a ping to the team that maintains that code.
asymptote (shell/Perl date)getfem (embeds datetime and user, submitted via email)getdp (hostname and user)getdp (user)guix (disable parallelism)httpcomponents-client (Java documentation generator readdir order)kuberlr (date)lal (date and time issue, submitted via email)libmesh (host)OBS (discuss how to track old build prjconf metadata in buildinfo)openblas (disable CPU detection)openfoam-selector (date)perl (toolchain, date)python-blosc (CPU detection)python-eventlet (fails to build far in the future)rna-star (date and hostname)trilinos (date)xz/b4 (workaround CPU count influencing output, reported upstream)json-c (forwarded upstream).nmh.golang-gonum-v1-plot.chirp.pixelmed-codec.debhelper.muroar.serd.pencil2d.tpot.evolution.aflplusplus.plexus-archiver (timezone/DST issue)libjpeg-turbo.jack-audio-connection-kit.glusterfs.155, 156, 157 and 158 to Debian:
<!ENTITY> declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)pgpdump(1) can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [ ]gnumeric from the Debian build-dependencies as it has been removed from the testing distribution. (#968742)fallback_recognises to prevent matching .xsb binary XML files.file(1) returns data . (#211)ppudump version does not match our file header. [ ]repr(object) output in Calling external command messages. [ ].ppu files requires ppudump version 3.2.0 or higher. [ ]setup.py that diffoscope works with Python version 3.8 [ ] and Frazer Clews applied some Pylint suggestions [ ] and removed some deprecated methods [ ].
This month, Chris Lamb updated the main Reproducible Builds website and documentation to:
SOURCE_DATE_EPOCH age. [ ]
The Reproducible Builds project operate a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:
arm64 architecture. [ ]armhf. [ ][ ][ ]ruby-jekyll-polyglot package to needed for the recently-added internationalisation and translation support on the Reproducible Builds website. [ ]jenkins.debian.net, www.reproducible-builds.org, diffoscope.org, buildinfos.debian.net, etc.). [ ][ ][ ][ ][ ]arm64 architecture anymore. [ ]If you think you know how to spread the word about reproducibility in the context of Bitcoin wallets through WalletScrutiny, your contributions are highly welcome on this PR [ ]Julien Lepiller posted to the list linking to a blog post by Tavis Ormandy titled You don t need reproducible builds. Morten Linderud (foxboron) responded with a clear rebuttal that Tavis was only considering the narrow use-case of proprietary vendors and closed-source software. He additionally noted that the criticism that reproducible builds cannot prevent against backdoors being deliberately introduced into the upstream source ( bugdoors ) are decidedly (and deliberately) outside the scope of reproducible builds to begin with. Chris Lamb included the Reproducible Builds mailing list in a wider discussion regarding a tentative proposal to include
.buildinfo files in .deb packages, adding his remarks regarding requiring a custom tool in order to determine whether generated build artifacts are identical in a reproducible context. [ ]
Jonathan Bustillos (Jathan) posted a quick email to the list requesting whether there was a list of To do tasks in Reproducible Builds.
Lastly, Chris Lamb responded at length to a query regarding the status of reproducible builds for Debian ISO or installation images. He noted that most of the technical work has been performed but there are at least four issues until they can be generally advertised as such . He pointed that the privacy-oriented Tails operation system, which is based directly on Debian, has had reproducible builds for a number of years now. [ ]
#reproducible-builds on irc.oftc.net.
rb-general@lists.reproducible-builds.org
So Finally, kernel 5.8 entered the Debian
Unstable
repositories. This means that I got my Raspberry image from their
usual location and was able to type the
following, using only my old trusty USB keyboard:
So finally, the greatest and meanest Raspberry is fully supported with
a pure Debian image! (only tarnished by the nonfree
raspi-firmware
package.
Oh, in case someone was still wondering The images generated follow
the stable release. Only the kernel and firmware are installed from
unstable. If / when kernel 5.8 enters
Backports, I will reduce the noise of
adding a different suit to the sources.list.
To ensure Dronefly always remains free, the Dronefly project has been relicensed under two copyleft licenses. Read the license change and learn more about copyleft at these links.
I was prompted to make this change after a recent incident in the Red DiscordBot development community that made me reconsider my prior position that the liberal MIT license was best for our project. While on the face of it, making your license as liberal as possible might seem like the most generous and hassle-free way to license any project, I was shocked into the realization that its liberality was also its fatal flaw: all is well and good so long as everyone is being cooperative, but it does not afford any protection to developers or users should things suddenly go sideways in how a project is run. A copyleft license is the best way to avoid such issues.
In this incident a sad story of conflict between developers I respect on both sides of the rift, and owe a debt to for what they ve taught me three cogs we had come to depend on suddenly stopped being viable for us to use due to changes to the license & the code. Effectively, those cogs became unsupported and unsupportable. To avoid any such future disaster with the Dronefly project, I started shopping for a new license that would protect developers and users alike from similarly losing support, or losing control of their contributions. I owe thanks to Dale Floer, a team member who early on advised me the AGPL might be a better fit, and later was helpful in selecting the doc license and encouraging me to follow through. We ran the new licenses by each contributor and arrived at this consensus: the AGPL is best suited for our server-based code, and CC-BY-SA is best suited for our documentation. The relicensing was made official this morning.
On Discord platform alternatives
You might well question what I, a Debian developer steeped in free software culture and otherwise in agreement with its principles, am doing encouraging a community to grow on the proprietary Discord platform! I have no satisfying answer to that. I explained when I introduced my project here some of the backstory, but that s more of an account of its beginnings than justification for it to continue on this platform. Honestly, all I can offer is a rather dissatisfying it seemed like the right thing to do at the time.
Time will tell whether we could successfully move off of it to a freedom-respecting and privacy-respecting alternative chat platform that is both socially and technically viable to migrate to. That platform would ideally:
For long time I have been using a good old nvidia GeForce GTX 1050 for my display and deep learning needs. I reported a few times how to get Tensorflow running on Debian/Sid, see here and here. Later on I switched to AMD GPU in the hope that an open source approach to both GPU driver as well as deep learning (ROCm) would improve the general experience. Unfortunately it turned out that AMD GPUs are generally not ready for deep learning usage.
The problems with AMD and ROCm are far and wide. First of all, it seems that for anything more complicated then simple stuff, AMD s flagship RX 5700(XT) and all GFX10 (Navi) based cards are not(!!!) supported in ROCm. Yes, you read correct AMD does not support 5700(XT) cards in the ROCm stack. Some simple stuff works, but nothing for real computations.
Then, even IF they would support, ROCm as distributed is currently a huge pain in the butt. The source code is a huge mess, and building usable packages from it is probably possible, but quite painful (I am member of the ROCm packaging team in Debian, and have tried many hours). And the packages provided by AMD are not installable on Debian/sid due to library incompatibilities.
So that left me with a bit a problem: for work I need to train quite some neural networks, do model selection, etc. Doing this on a CPU is a bit a burden. So at the end I decided to put the nVidia card back into the computer (well, after moving it to a bigger case but that is a different story to tell). Here are the steps I did to get both cards working for their respective target: AMD GPU for driving the console and X (and games!), and the nVidia card doing the deep learning stuff (tensorflow using the GPU).
Starting point
Starting point was a working AMD GPU installation. The AMD GPU is also the first GPU card (top slot) and thus the one that is used by the BIOS and the Linux console. If you want the video output on the second card you need to trick, and probably don t have console output, etc etc. So not a solution for me.
Installing libcuda1 and the nvidia kernel drivers
Next step was installing the libcuda1 package:
apt install libcuda1
update-glx tool and package.
The kernel module should be built and installed automatically for your kernel.
Installing CUDA
Follow more or less the instructions here and do
wget -O- https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo tee /etc/apt/trusted.gpg.d/nvidia-cuda.asc echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /" sudo tee /etc/apt/sources.list.d/nvidia-cuda.list sudo apt-get update sudo apt-get install cuda-libraries-10-1
/usr/local/cuda-10.1 and add the respective directory to the ld.so path by creating a file /etc/ld.so.conf.d/cuda-10-1.conf.
Install CUDA CuDNN
One difficult to satisfy dependency are the CuDNN libraries. In our case we need the version 7 library for CUDA 10.1. To download these files one needs to have a NVIDIA developer account, which is quick and painless. After that go to the CuDNN page where one needs to select Archived releases and then Download cuDNN v7.N.N (xxxx NN, YYYY), for CUDA 10.1 and then cuDNN Runtime Library for Ubuntu18.04 (Deb).
At the moment (as of today) this will download a file libcudnn7_7.6.5.32-1+cuda10.1_amd64.deb which needs to be installed with dpkg -i libcudnn7_7.6.5.32-1+cuda10.1_amd64.deb.
Updating the GLX setting
Here now comes the very interesting part one needs to set up the GLX libraries. Reading the output of update-glx --help and then the output of update-glx --list glx:
$ update-glx --help
update-glx is a wrapper around update-alternatives supporting only configuration
of the 'glx' and 'nvidia' alternatives. After updating the alternatives, it
takes care to trigger any follow-up actions that may be required to complete
the switch.
It can be used to switch between the main NVIDIA driver version and the legacy
drivers (eg: the 304 series, the 340 series, etc).
For users with Optimus-type laptops it can be used to enable running the discrete
GPU via bumblebee.
Usage: update-glx <command>
Commands:
--auto <name> switch the master link <name> to automatic mode.
--display <name> display information about the <name> group.
--query <name> machine parseable version of --display <name>.
--list <name> display all targets of the <name> group.
--config <name> show alternatives for the <name> group and ask the
user to select which one to use.
--set <name> <path> set <path> as alternative for <name>.
<name> is the master name for this link group.
Only 'nvidia' and 'glx' are supported.
<path> is the location of one of the alternative target files.
(e.g. /usr/lib/nvidia)
$ update-glx --list glx
/usr/lib/mesa-diverted
/usr/lib/nvidiaupdate-glx --config glx /usr/lib/mesa-diverted
libnvidia-cfg1 wasn t found (not sure it was needed at all ), and with that also no way to run tensorflow on GPU.
So what I did I tried
update-glx --auto glx
update-glx --config glx /usr/lib/nvidia), and rebooted, and decided to check afterwards what is broken.
To my big surprise, the AMD GPU still worked out of the box, including direct rendering, and the games I tried (Overload, Supraland via Wine) all worked without a hinch.
Not that I really understand why the GLX libraries that are seemingly now in use are from nvidia but work the same (if anyone has an explanation, that would be great!), but since I haven t had any problems till now, I am content.
Checking GPU usage in tensorflow
Make sure that you remove tensorflow-rocm and reinstall tensorflow with GPU support:
pip3 uninstall tensorflow-rocm pip3 install --upgrade tensorflow-gpu
$ python3 -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))" ....(lots of output) 2020-09-02 11:57:04.673096: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3581 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:05:00.0, compute capability: 6.1) tf.Tensor(1093.4915, shape=(), dtype=float32) $
| Publisher: | Tor.com |
| Copyright: | January 2020 |
| ISBN: | 1-250-21476-9 |
| Format: | Kindle |
| Pages: | 176 |
| Series: | Discworld #15 |
| Publisher: | Harper |
| Copyright: | 1993 |
| Printing: | November 2013 |
| ISBN: | 0-06-223740-3 |
| Format: | Mass market |
| Pages: | 420 |
Cuddy had only been a guard for a few days but already he had absorbed one important and basic fact: it is almost impossible for anyone to be in a street without breaking the law.Vimes and Carrot are both excellent police officers, but in entirely different ways. Vimes treats being a cop as a working-class job and is inclined towards glumness and depression, but is doggedly persistent and unable to leave a problem alone. His ethics are covered by a thick layer of world-weary cynicism. Carrot is his polar opposite in personality: bright, endlessly cheerful, effortlessly charismatic, and determined to get along with everyone. On first appearance, this contrast makes Vimes seem wise and Carrot seem a bit dim. That is exactly what Pratchett is playing with and undermining in Men at Arms. Beneath Vimes's cynicism, he's nearly as idealistic as Carrot, even though he arrives at his ideals through grim contrariness. Carrot, meanwhile, is nowhere near as dim as he appears to be. He's certain about how he wants to interact with others and is willing to stick with that approach no matter how bad of an idea it may appear to be, but he's more self-aware than he appears. He and Vimes are identical in the strength of their internal self-definition. Vimes shows it through the persistent, grumpy stubbornness of a man devoted to doing an often-unpleasant job, whereas Carrot verbally steamrolls people by refusing to believe they won't do the right thing.
Colon thought Carrot was simple. Carrot often struck people as simple. And he was. Where people went wrong was thinking that simple meant the same thing as stupid.There's a lot going on in this book apart from the profiles of two very different models of cop. Alongside the mystery (which doubles as pointed commentary on the corrupting influence of violence and personal weaponry), there's a lot about dwarf/troll relations, a deeper look at the Ankh-Morpork guilds (including a horribly creepy clown guild), another look at how good Lord Vetinari is at running the city by anticipating how other people will react, a sarcastic dog named Gaspode (originally seen in Moving Pictures), and Pratchett's usual collection of memorable lines. It is also the origin of the now-rightfully-famous Vimes boots theory:
The reason that the rich were so rich, Vimes reasoned, was because they managed to spend less money. Take boots, for example. He earned thirty-eight dollars a month plus allowances. A really good pair of leather boots cost fifty dollars. But an affordable pair of boots, which were sort of OK for a season or two and then leaked like hell when the cardboard gave out, cost about ten dollars. Those were the kind of boots Vimes always bought, and wore until the soles were so thin that he could tell where he was in Ankh-Morpork on a foggy night by the feel of the cobbles. But the thing was that good boots lasted for years and years. A man who could afford fifty dollars had a pair of boots that'd still be keeping his feet dry in ten years' time, while the poor man who could only afford cheap boots would have spent a hundred dollars on boots in the same time and would still have wet feet. This was the Captain Samuel Vimes 'Boots' theory of socioeconomic unfairness.Men at Arms regularly makes lists of the best Discworld novels, and I can see why. At this point in the series, Pratchett has hit his stride. The plots have gotten deeper and more complex without losing the funny moments, movie and book references, and glorious turns of phrase. There is also a lot of life philosophy and deep characterization when one pays close attention to the characters.
He was one of those people who would recoil from an assault on strength, but attack weakness without mercy.My one complaint is that I found it a bit overstuffed with both characters and subplots, and as a result had a hard time following the details of the plot. I found myself wanting a timeline of the murders or a better recap from one of the characters. As always with Pratchett, the digressions are wonderful, but they do occasionally come at the cost of plot clarity. I'm not sure I recommend the present moment in the United States as the best time to read this book, although perhaps there is no better time for Carrot and Vimes to remind us what good cops look like. But regardless of when one reads it, it's an excellent book, one of the best in the Discworld series to this point. Followed, in publication order, by Soul Music. The next Watch book is Feet of Clay. Rating: 8 out of 10
Next.