Updated page to record how-to video.
I don't need big camera image, I just need a tiny image with probably my face on it.
As a start, I tried extracting a rectangle.
For long time I have been using a good old nvidia GeForce GTX 1050 for my display and deep learning needs. I reported a few times how to get Tensorflow running on Debian/Sid, see here and here. Later on I switched to AMD GPU in the hope that an open source approach to both GPU driver as well as deep learning (ROCm) would improve the general experience. Unfortunately it turned out that AMD GPUs are generally not ready for deep learning usage.
The problems with AMD and ROCm are far and wide. First of all, it seems that for anything more complicated then simple stuff, AMD s flagship RX 5700(XT) and all GFX10 (Navi) based cards are not(!!!) supported in ROCm. Yes, you read correct AMD does not support 5700(XT) cards in the ROCm stack. Some simple stuff works, but nothing for real computations.
Then, even IF they would support, ROCm as distributed is currently a huge pain in the butt. The source code is a huge mess, and building usable packages from it is probably possible, but quite painful (I am member of the ROCm packaging team in Debian, and have tried many hours). And the packages provided by AMD are not installable on Debian/sid due to library incompatibilities.
So that left me with a bit a problem: for work I need to train quite some neural networks, do model selection, etc. Doing this on a CPU is a bit a burden. So at the end I decided to put the nVidia card back into the computer (well, after moving it to a bigger case but that is a different story to tell). Here are the steps I did to get both cards working for their respective target: AMD GPU for driving the console and X (and games!), and the nVidia card doing the deep learning stuff (tensorflow using the GPU).
Starting point
Starting point was a working AMD GPU installation. The AMD GPU is also the first GPU card (top slot) and thus the one that is used by the BIOS and the Linux console. If you want the video output on the second card you need to trick, and probably don t have console output, etc etc. So not a solution for me.
Installing libcuda1 and the nvidia kernel drivers
Next step was installing the libcuda1 package:
apt install libcuda1
update-glx tool and package.
The kernel module should be built and installed automatically for your kernel.
Installing CUDA
Follow more or less the instructions here and do
wget -O- https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo tee /etc/apt/trusted.gpg.d/nvidia-cuda.asc echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /" sudo tee /etc/apt/sources.list.d/nvidia-cuda.list sudo apt-get update sudo apt-get install cuda-libraries-10-1
/usr/local/cuda-10.1 and add the respective directory to the ld.so path by creating a file /etc/ld.so.conf.d/cuda-10-1.conf.
Install CUDA CuDNN
One difficult to satisfy dependency are the CuDNN libraries. In our case we need the version 7 library for CUDA 10.1. To download these files one needs to have a NVIDIA developer account, which is quick and painless. After that go to the CuDNN page where one needs to select Archived releases and then Download cuDNN v7.N.N (xxxx NN, YYYY), for CUDA 10.1 and then cuDNN Runtime Library for Ubuntu18.04 (Deb).
At the moment (as of today) this will download a file libcudnn7_7.6.5.32-1+cuda10.1_amd64.deb which needs to be installed with dpkg -i libcudnn7_7.6.5.32-1+cuda10.1_amd64.deb.
Updating the GLX setting
Here now comes the very interesting part one needs to set up the GLX libraries. Reading the output of update-glx --help and then the output of update-glx --list glx:
$ update-glx --help
update-glx is a wrapper around update-alternatives supporting only configuration
of the 'glx' and 'nvidia' alternatives. After updating the alternatives, it
takes care to trigger any follow-up actions that may be required to complete
the switch.
It can be used to switch between the main NVIDIA driver version and the legacy
drivers (eg: the 304 series, the 340 series, etc).
For users with Optimus-type laptops it can be used to enable running the discrete
GPU via bumblebee.
Usage: update-glx <command>
Commands:
--auto <name> switch the master link <name> to automatic mode.
--display <name> display information about the <name> group.
--query <name> machine parseable version of --display <name>.
--list <name> display all targets of the <name> group.
--config <name> show alternatives for the <name> group and ask the
user to select which one to use.
--set <name> <path> set <path> as alternative for <name>.
<name> is the master name for this link group.
Only 'nvidia' and 'glx' are supported.
<path> is the location of one of the alternative target files.
(e.g. /usr/lib/nvidia)
$ update-glx --list glx
/usr/lib/mesa-diverted
/usr/lib/nvidiaupdate-glx --config glx /usr/lib/mesa-diverted
libnvidia-cfg1 wasn t found (not sure it was needed at all ), and with that also no way to run tensorflow on GPU.
So what I did I tried
update-glx --auto glx
update-glx --config glx /usr/lib/nvidia), and rebooted, and decided to check afterwards what is broken.
To my big surprise, the AMD GPU still worked out of the box, including direct rendering, and the games I tried (Overload, Supraland via Wine) all worked without a hinch.
Not that I really understand why the GLX libraries that are seemingly now in use are from nvidia but work the same (if anyone has an explanation, that would be great!), but since I haven t had any problems till now, I am content.
Checking GPU usage in tensorflow
Make sure that you remove tensorflow-rocm and reinstall tensorflow with GPU support:
pip3 uninstall tensorflow-rocm pip3 install --upgrade tensorflow-gpu
$ python3 -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))" ....(lots of output) 2020-09-02 11:57:04.673096: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3581 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:05:00.0, compute capability: 6.1) tf.Tensor(1093.4915, shape=(), dtype=float32) $
RuboCop::Packaging.RuboCop's latest version.lib/ and gemspec file.Puppet6.php-dasprid-enum and php-bacon-baconqrcode for William and ruby-unparser, ruby-morpher, and ruby-path-exapander for Cocoa.
In May, I got selected as a Google Summer of Code student for Debian again! \o/docs/ directory, which is hosted at https://docs.rubocop.org/rubocop-packaging.gem install rubocop-packaging, but also via apt install ruby-rubocop-packaging.
Packaging extension reports as offensive .git ls-files in gemspec.RuboCop and RuboCop::Packaging.RuboCop to v0.89..rubocop.yml and add RuboCop::Packaging as a development dependency.RuboCop and RuboCop::RSpec version to the latest.git ls-files in gemspec.
:wq for today.
Here is my transparent report for my work on the Debian Long Term Support (LTS) and Debian Extended Long Term Support (ELTS), which extend the security support for past Debian releases, as a paid contributor.
In August, the monthly sponsored hours were split evenly among contributors depending on their max availability - I was assigned 21.75h for LTS (out of my 30 max; all done) and 14.25h for ELTS (out of my 20 max; all done).
We had a Birds of a Feather videoconf session at DebConf20, sadly with varying quality for participants (from very good to unusable), where we shared the first results of the LTS survey.
There were also discussions about evaluating our security reactivity, which proved surprisingly hard to estimate (neither CVE release date and criticality metrics are accurate nor easily available), and about when it is appropriate to use public naming in procedures.
Interestingly ELTS gained new supported packages, thanks to a new sponsor -- so far I'd seen the opposite, because we were close to the EOL.
As always, there were opportunities to de-dup work through mutual cooperation with the Debian Security team, and LTS/ELTS similar updates.
ELTS - Jessie
| Publisher: | Tor.com |
| Copyright: | January 2020 |
| ISBN: | 1-250-21476-9 |
| Format: | Kindle |
| Pages: | 176 |
Here is another monthly update covering what I have been doing in the free software world during August 2020 (previous month):
classproperty helper has been moved to django.utils.functional in newer versions of Django. [...]
INSTALLED_APPS during model setup. [...]
.d-style directories containing Javascript and CSS to drop unquote usage from the six compatibility library. [...]
2.86.0, 2.87.0, 2.88.0, 2.89.0, 2.90.0, 2.91.0 and 2.92.0, as well as made the following changes:
patch-not-forwarded-upstream tag for README files under debian/patches. (#968845)no-dh-sequencer tag due to indirect target dependences. (#968108)package-uses-old-debhelper-compat-version tag. [...]lintian salsa namespace. [...]lzip dependency in a previous debian/changelog entry. (#966817)generate-tag-summary release script to reflect change of tag definition filename extension change from .desc .tag. [...]spelling-error-in-rules-requires-root tag's severity; this is not a "spelling" check in the sense that it does not use our dictionary. [...]$skip_tag argument in the extract_service_file_values routine. [...]debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a gentle ping to the team that maintains that code.
.buildinfo files within .deb archives [...].
muroar package that was discovered while doing reproducibility testing. (#968189)
tests.reproducible-builds.org. This month, I updated the self-serve package rescheduler to use HTML <pre> tags when dumping any debugging data. [...]
SOURCE_DATE_EPOCH age. [...]155, 156, 157 and 158 to Debian:
<!ENTITY> declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)pgpdump(1) can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [...]gnumeric from the Debian build-dependies as it has been removed from the testing distribution. (#968742)fallback_recognises to prevent matching .xsb binary XML files.file(1) returns "data". (#211)ppudump version does not match our file header. [...]repr(object) output in "Calling external command" messages. [...]chrony [...], golang-1.8 [...], golang-go.crypto [...], golang-golang-x-net-dev [...], icingaweb2 [...], lua5.3 [...], mongodb [...], net-snmp, php7.0 [...], qt4-x11, qtbase-opensource-src, ruby-actionpack-page-caching [...], ruby-doorkeeper [...], ruby-json-jwt [...], ruby-kaminari [...], ruby-kaminari [...], ruby-rack-cors [...], shiro [...] & squirrelmail.
qtbase-opensource-src and ELA-261-1 for qt4-x11, two components of cross-platform C++ application framework. A specially-crafted XBM image file could have caused a buffer overread.
squirrelmail webmail client.
zabbix, the PHP-based monitoring system to fix a potential cross-site scripting vulnerability via <iframe> HTML elements.
ruby-websocket-extensions, a library for managing long-lived HTTP 'WebSocket' connections.
1.6.6-2 Enable TLS capabilities by default. (#968603)1.6.6-3 Add libio-socket-ssl-perl to test TLS support and perform a general package refresh.2.2.15-1 (unstable) New upstream bugfix release3.1-1 (experimental) New upstream release.2.2.15-2 (unstable) & 3.1-2 (experimental) Set the same PYTHONPATH when executing the runtime tests as we do in the package build. (#968577)2.0.0-43 Refresh packaging, and upload some changes from the Debian Janitor.2.0.0-44 Fix compatibility with GCC 10, restoring the missing /usr/bin/instant binary. (#968900)1.0.0-1) New upstream release to experimental.
Debian packaging 2020-08-07: Sponsor package python-sabyenc (4.0.2-1) for Debian unstable (Python team request). 2020-08-07: Sponsor package gpxpy (1.4.2-1) for Debian unstable (Python team request). 2020-08-07: Sponsor package python-jellyfish (0.8.2-1) for Debian unstable (Python team request). 2020-08-08: Sponsor package django-ipwire (3.0.0-1) for Debian unstable (Python team request). 2020-08-08: Sponsor package python-mongoengine (0.20.0-1) for Debian unstable (Python team request). 2020-08-08: Review package pdfminer (20191020+dfsg-3) (Needs some more work) (Python team request). 2020-08-08: Upload package bundlewrap (4.1.0-1) to Debian unstable. 2020-08-09: Sponsor package pdfminer (20200726-1) for Debian unstable (Python team request). 2020-08-09: Sponsor package spyne (2.13.15-1) for Debian unstable (Python team request). 2020-08-09: Review package mod-wsgi (4.6.8-2) (Needs some more work) (Python team request). 2020-08-10: Sponsor package nfoview (1.28-1) for Debian unstable (Python team request). 2020-08-11: Sponsor package pymupdf (1.17.4+ds1-1) for Debian unstable (Python team request). 2020-08-11: Upload package calamares (3.2.28-1) to Debian ubstable. 2020-08-11: Upload package xabacus (8.2.9-1) to Debian unstable. 2020-08-11: Upload package bashtop (0.9.25-1~bpo10+1) to Debian buster-backports. 2020-08-11: Upload package live-tasks (11.0.3) to Debian unstable (Closes: #942834, #965999, #956525, #961728). 2020-08-12: Upload package calamares-settings-debian (10.0.20-1+deb10u4) to Debian buster (Closes: #968267, #968296). 2020-08-13: Upload package btfs (2.22-1) to Debian unstable. 2020-08-14: Upload package calamares (3.2.28.2-1) to Debian unstable. 2020-08-14: Upload package bundlewrap (4.1.1-1) to Debian unstable. 2020-08-19: Upload package gnome-shell-extension-dash-to-panel (38-2) to Debian unstable) (Closes: #968613). 2020-08-19: Sponsor package mod-wsgi (4.7.1-1) for Debian unstable (Python team request). 2020-08-19: Review package tqdm (4.48.2-1) (Needs some more work) (Python team request). 2020-08-19: Sponsor package tqdm (4.48.2-1) to unstable (Python team request). 2020-08-19: Upload package calamares (3.2.28.3-2) to unstable (Python team request).
| Series: | Discworld #15 |
| Publisher: | Harper |
| Copyright: | 1993 |
| Printing: | November 2013 |
| ISBN: | 0-06-223740-3 |
| Format: | Mass market |
| Pages: | 420 |
Cuddy had only been a guard for a few days but already he had absorbed one important and basic fact: it is almost impossible for anyone to be in a street without breaking the law.Vimes and Carrot are both excellent police officers, but in entirely different ways. Vimes treats being a cop as a working-class job and is inclined towards glumness and depression, but is doggedly persistent and unable to leave a problem alone. His ethics are covered by a thick layer of world-weary cynicism. Carrot is his polar opposite in personality: bright, endlessly cheerful, effortlessly charismatic, and determined to get along with everyone. On first appearance, this contrast makes Vimes seem wise and Carrot seem a bit dim. That is exactly what Pratchett is playing with and undermining in Men at Arms. Beneath Vimes's cynicism, he's nearly as idealistic as Carrot, even though he arrives at his ideals through grim contrariness. Carrot, meanwhile, is nowhere near as dim as he appears to be. He's certain about how he wants to interact with others and is willing to stick with that approach no matter how bad of an idea it may appear to be, but he's more self-aware than he appears. He and Vimes are identical in the strength of their internal self-definition. Vimes shows it through the persistent, grumpy stubbornness of a man devoted to doing an often-unpleasant job, whereas Carrot verbally steamrolls people by refusing to believe they won't do the right thing.
Colon thought Carrot was simple. Carrot often struck people as simple. And he was. Where people went wrong was thinking that simple meant the same thing as stupid.There's a lot going on in this book apart from the profiles of two very different models of cop. Alongside the mystery (which doubles as pointed commentary on the corrupting influence of violence and personal weaponry), there's a lot about dwarf/troll relations, a deeper look at the Ankh-Morpork guilds (including a horribly creepy clown guild), another look at how good Lord Vetinari is at running the city by anticipating how other people will react, a sarcastic dog named Gaspode (originally seen in Moving Pictures), and Pratchett's usual collection of memorable lines. It is also the origin of the now-rightfully-famous Vimes boots theory:
The reason that the rich were so rich, Vimes reasoned, was because they managed to spend less money. Take boots, for example. He earned thirty-eight dollars a month plus allowances. A really good pair of leather boots cost fifty dollars. But an affordable pair of boots, which were sort of OK for a season or two and then leaked like hell when the cardboard gave out, cost about ten dollars. Those were the kind of boots Vimes always bought, and wore until the soles were so thin that he could tell where he was in Ankh-Morpork on a foggy night by the feel of the cobbles. But the thing was that good boots lasted for years and years. A man who could afford fifty dollars had a pair of boots that'd still be keeping his feet dry in ten years' time, while the poor man who could only afford cheap boots would have spent a hundred dollars on boots in the same time and would still have wet feet. This was the Captain Samuel Vimes 'Boots' theory of socioeconomic unfairness.Men at Arms regularly makes lists of the best Discworld novels, and I can see why. At this point in the series, Pratchett has hit his stride. The plots have gotten deeper and more complex without losing the funny moments, movie and book references, and glorious turns of phrase. There is also a lot of life philosophy and deep characterization when one pays close attention to the characters.
He was one of those people who would recoil from an assault on strength, but attack weakness without mercy.My one complaint is that I found it a bit overstuffed with both characters and subplots, and as a result had a hard time following the details of the plot. I found myself wanting a timeline of the murders or a better recap from one of the characters. As always with Pratchett, the digressions are wonderful, but they do occasionally come at the cost of plot clarity. I'm not sure I recommend the present moment in the United States as the best time to read this book, although perhaps there is no better time for Carrot and Vimes to remind us what good cops look like. But regardless of when one reads it, it's an excellent book, one of the best in the Discworld series to this point. Followed, in publication order, by Soul Music. The next Watch book is Feet of Clay. Rating: 8 out of 10
A new minor release 0.2.9 of RcppCCTZ is now on CRAN.
RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do using copies in their packages which remains less than ideal.
This version adds a header file for the recently-exported three functions.
We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.Changes in version 0.2.9 (2020-08-30)
- Provide a header
RcppCCZT_API.hfor client packages.- Show a simple example of parsing a YYYYMMDD HHMMSS.FFFFFF date.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
This line is called a command prompt and it tells you four pieces of information:
jaadams: The username of the user that is currently running this shell.bg7: The name of the computer that this shell is running on, important
for when you start accessing shells on remote machines./tmp/thisfolder: The folder or directory that your shell is currently
running in. Like a file explorer (like Window s Explorer or Mac s Finder) a
shell always has a working directory, from which all relative paths (see
sidenote below) are resolved.
This is a shorthand notation that the shell uses to make this output shorter
when possible. ~ stands for your home directory, usually /home/<username>.
Like C:\Users\<username>\ on Windows or /Users/<username> on Mac,
this directory is where all your files should go by default.
Thus a command prompt like this:
actually tells you that you are currently in the /home/jaadams/Downloads
directory.
/ and there are no directories above it.
For example, the directory called home typically contains all user directories.
This is stored in the root directory, and each users specific data
is stored in a directory named after that user under home.
Thus, the home directory of the user jacob is typically /home/jacob,
the directory jacob under the
home directory stored in the root directory /.
If you re interested in more details about what goes in what directory,
man hier has the basics and the
Filesystem Hierarchy Standard
governs the layout of the filesystem on most Linux distributions.
You don t always have to use the full path, however.
If the path does not begin with a /, it is assumed that the path actually
begins with the path of the current directory.
So if you use a path like my/folders/here, and you re in the /home/jacob
directory, the path will be treated like /home/jacob/my/folders/here.
Each folder also contains the symbolic links .. and ..
Symbolic links are a very powerful kind of
file that is actually a reference to another file.
.. always represents the parent directory
of the current directory, so /home/jacob/.. links to /home/.
. always links to the current directory, so /home/jacob/. links to
/home/jacob.
echo command displays the text passed as arguments.
jacob@lovelace/home/jacob$ echo hello world
hello world
hello
is the first argument and world is the second. If you need an argument to
contain spaces, you ll want to put quotes around it, echo "like so".
Certain arguments are called flags , or options (options if they take another
argument, flags otherwise) usually prefixed with a hyphen,
and they change the way a program operates.
For example, the ls command outputs the contents of a directory passed as
an argument, but if you add -l before the directory, it will give you more
details on the files in that directory.
jacob@lovelace/tmp/test$ ls /tmp/test
1 2 3 4 5 6
jacob@lovelace/tmp/test$ ls -l /tmp/test
total 0
-rw-r--r-- 1 jacob jacob 0 Aug 26 22:06 1
-rw-r--r-- 1 jacob jacob 0 Aug 26 22:06 2
-rw-r--r-- 1 jacob jacob 0 Aug 26 22:06 3
-rw-r--r-- 1 jacob jacob 0 Aug 26 22:06 4
-rw-r--r-- 1 jacob jacob 0 Aug 26 22:06 5
-rw-r--r-- 1 jacob jacob 0 Aug 26 22:06 6
jacob@lovelace/tmp/test$
cd <path>: Change the current directory of the running shell to <path>.ls <path>: Output the contents of <path>. If no path is passed, it prints
the contents of the current directory.touch <filename>: create an new empty file called <filename>.
Used on an existing file, it updates the file s last accessed and modified times.
Most text editors can also create a new file for you, which is probably more useful.mkdir <directory>: Create a new folder/directory at path <directory>.mv <src> <dest>: Move a file or directory at path <src> to <dest>.cp <src> <dest>: Copy a file or directory at path <src> to <dest>.rm <file>: Remove a file at path <file>.zip -r <zipfile> <contents...>: Create a zip file <zipfile> with contents
<contents>. <contents> can be multiple arguments, and you ll usually want
to use the -r argument when including directories in your zipfile, as otherwise
only the directory will be included and not the files and directories within it.grep <thing> <file>: Look for the string <thing> in <file>.
If no <file> is passed it searches standard input.find <path> -name <name>: Find a file or directory called <name> somwhere
under <path>. This command is actually very powerful, but also very complex.
For example you can delete all files in a directory older than 30 days with:
find -mtime +30 -exec rm \;
locate <name>: A much easier to use command to find a file with a given
name, but it is not usually installed by default.cat <files...>: Output (concatenate) all the files passed as arguments.head <file>: Output the beginning of <file>tail <file>: Output the end of <file>man intro).
This can be accessed using man <command>
You can search for the right command using the -k flag, as in man -k <search>.
You can also view manual pages in your browser, on sites like https://manpages.debian.org
or https://linux.die.net/man.
This is not always helpful, however, because some command s descriptions are
not particularly useful, and also there are a lot of manual pages, which can
make searching for a specific one difficult. For example, finding the right
command to search inside text files is quite difficult via man (it s grep).
When you can t find what you need with man I recommend falling back to
searching the Internet. There are lots of bad Linux tutorials out there, but
here are some reputable sources I recommend:
The NAME section is the only required section. Man pages without a name section are as useful as refrigerators at the north pole. This section also has a standardized format consisting of a comma-separated list of program or function names, followed by a dash, followed by a short (usually one line) description of the functionality the program (or function, or file) is supposed to provide. By means of makewhatis(8), the name sections make it into the whatis database files. Makewhatis is the reason the name section must exist, and why it must adhere to the format I described. (Formatting explanation cut for brevity) The SYNOPSIS section is intended to give a short overview on available program options. For functions this sections lists corresponding include files and the prototype so the programmer knows the type and number of arguments as well as the return type. The DESCRIPTION section eloquently explains why your sequence of 0s and 1s is worth anything at all. Here s where you write down all your knowledge. This is the Hall Of Fame. Win other programmers and users admiration by making this section the source of reliable and detailed information. Explain what the arguments are for, the file format, what algorithms do the dirty jobs. The OPTIONS section gives a description of how each option affects program behaviour. You knew that, didn t you? The FILES section lists files the program or function uses. For example, it lists configuration files, startup files, and files the program directly operates on. (Cut details about installing files) The ENVIRONMENT section lists all environment variables that affect your program or function and tells how, of course. Most commonly the variables will hold pathnames, filenames or default options. The DIAGNOSTICS section should give an overview of the most common error messages from your program and how to cope with them. There s no need to explain system error error messages (from perror(3)) or fatal signals (from psignal(3)) as they can appear during execution of any program. The BUGS section should ideally be non-existent. If you re brave, you can describe here the limitations, known inconveniences and features that others may regard as misfeatures. If you re not so brave, rename it the TO DO section ;-) The AUTHOR section is nice to have in case there are gross errors in the documentation or program behaviour (Bzzt!) and you want to mail a bug report. The SEE ALSO section is a list of related man pages in alphabetical order. Conventionally, it is the last section.
ssh, the secure shell.
This allows you to remotely connect to another computer and run a shell on that
machine:
user@host:~$ ssh other@example.com
other@example:~$
scp, which works much like the cp command
except that paths can also take a user@host argument to move files across
computers. For example, if you wanted to move a file test.txt to your
home directory on another machine, the command would look like:
scp test.txt other@example.com:
/etc/issue.net
from example.com:
scp other@example.com:/etc/issue.net .
sftp command, which gives you a very simple shell-like
interface in which you can cd and ls, before either puting files onto
the local machine or geting files off of it.
The final and most powerful option is rsync which syncs the contents of one
directory to another, and doesn t copy files that haven t changed. It s powerful
and complex, however, so I recommend reading the USAGE section of its man page.
tmux and screen both allow
you to run a shell in a safe environment where it will continue even if you
disconnect from it. You do this by running the command without any arguments,
i.e. just tmux or just screen.
In tmux you can disconnect from the current session by
pressing Ctrl+b then d, and reattach with the tmux attach command.
screen works similarly, but with Ctrl+a instead of b and screen -r to
reattach.
<, > and the very powerful .
< and > redirect the input and output of a command to a file.
For example, if you wanted a file called list.txt that contained a list of
all the files in a directory /this/one/here you could use:
ls /this/one/here > list.txt
, allows you to direct the output of one command into
the input of another. This can be very powerful.
For example, the following pipeline lists the contents of the current directory
searches for the string test , then counts the number of results.
(wc -l counts the number of lines in its input)
ls grep test wc -l
myfile,
with a bunch of lines of potentially duplicated and unsorted data
test
test
1234
4567
1234
sort and uniq:
$ uniq < myfile sort
1234
1234
4567
test
*.suffix. This also works with prefixes, prefix*,
and in fact you can put a * anywhere, *middle*. The shell will expand that
* into all the files in that directory that match your criteria
(ending with a specific suffix, starting with a specific prefix, and so on)
and pass each file as a separate argument to the command.
For example, if I have a series of files called 1.txt, 2.txt, and so on
up to 9, each containing just the number for which it s named, I could use
cat to output all of them like so:
jacob@lovelace/tmp/numbers$ ls
1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt 9.txt
jacob@lovelace/tmp/numbers$ cat *.txt
1
2
3
4
5
6
7
8
9
~ shorthand mentioned above that refers to your home directory
can be used when passing a path as an argument to a command.
for i in 1 2 3 4 5 6 7 8 9
do
echo $i > $i.txt
done
$ echo 0 > /sys/devices/system/cpu/cpufreq/boost
$ echo 1 > /sys/devices/system/cpu/cpufreq/boost
$ echo performance > /sys/devices/system/cpu/cpu$n/cpufreq/scaling_governor
cpu0, cpu1, etc.
Re-enable with:
$ echo ondemand > /sys/devices/system/cpu/cpu$n/cpufreq/scaling_governor
ondemand is usually the default)
Cores and Threads
This one requires some attention, because you cannot assume anything about the CPU numbers. The first thing you want to do is to check those CPU numbers. You can do so by looking at the physical id and core id fields in /proc/cpuinfo, but the output from lscpu --extended is more convenient, and looks like the following:
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ
0 0 0 0 0:0:0:0 yes 3700.0000 2200.0000
1 0 0 1 1:1:1:0 yes 3700.0000 2200.0000
2 0 0 2 2:2:2:0 yes 3700.0000 2200.0000
3 0 0 3 3:3:3:0 yes 3700.0000 2200.0000
4 0 0 4 4:4:4:1 yes 3700.0000 2200.0000
5 0 0 5 5:5:5:1 yes 3700.0000 2200.0000
6 0 0 6 6:6:6:1 yes 3700.0000 2200.0000
7 0 0 7 7:7:7:1 yes 3700.0000 2200.0000
(...)
32 0 0 0 0:0:0:0 yes 3700.0000 2200.0000
33 0 0 1 1:1:1:0 yes 3700.0000 2200.0000
34 0 0 2 2:2:2:0 yes 3700.0000 2200.0000
35 0 0 3 3:3:3:0 yes 3700.0000 2200.0000
36 0 0 4 4:4:4:1 yes 3700.0000 2200.0000
37 0 0 5 5:5:5:1 yes 3700.0000 2200.0000
38 0 0 6 6:6:6:1 yes 3700.0000 2200.0000
39 0 0 7 7:7:7:1 yes 3700.0000 2200.0000
(...)$ echo 0 > /sys/devices/system/cpu/cpu$n/online
$ echo 1 > /sys/devices/system/cpu/cpu$n/online
$ mkdir /sys/fs/cgroup/cpuset/mygroup$ echo 0-7,16-23 > /sys/fs/cgroup/cpuset/mygroup/cpuset.cpus0-3 is the range for CPU/core/thread 0 to 3 and is thus equivalent to 0,1,2,3. The numbers correspond to /proc/cpuinfo or the output from lscpu as mentioned above.
There are also memory aspects to CPU sets, that I won t detail here (because I don t have a machine with multiple memory nodes), but you can start with:
$ cat /sys/fs/cgroup/cpuset/cpuset.mems > /sys/fs/cgroup/cpuset/mygroup/cpuset.mems$ echo $pid >> /sys/fs/cgroup/cpuset/mygroup/taskscpuset(7) manual page.
Disabling a group is a little involved. First you need to move the processes to a different group:
$ while read pid; do echo $pid > /sys/fs/cgroup/cpuset/tasks; done < /sys/fs/cgroup/cpuset/mygroup/tasks$ > /sys/fs/cgroup/cpuset/mygroup/cpuset.cpus
$ > /sys/fs/cgroup/cpuset/mygroup/cpuset.mems$ rmdir /sys/fs/cgroup/cpuset/mygroupR CMD check --as-cran as nicely as usual.
Courtesy of CRANberries, there is a diffstat report for this release. More information is on the RcppSMC page. Issues and bugreports should go to the GitHub issue tracker. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.Changes in RcppSMC version 0.2.2 (2020-08-30)
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
Dealing with the void during MiniDebConf Online #1
Between 28 and 31 May this year, we set out to create our first ever online MiniDebConf for Debian. Many people have been meaning to do something similar for a long time, but it just didn t work out yet. With many of us being in lock down due to COVID-19, and with the strong possibility looming that DebConf20 might have had to become an online event, we rushed towards organising the first ever Online MiniDebConf and put together some form of usable video stack for it.
I could go into all kinds of details on the above, but this post is about a bug that lead to a pretty nifty feature for DebConf20. The tool that we use to capture Jitsi calls is called Jibri (Jitsi Broadcasting Infrustructure). It had a bug (well, bug for us, but it s an upstream feature) where Jibri would hang up after 30s of complete silence, because it would assume that the call has ended and that the worker can be freed up again. This would result in the stream being ended at the end of every talk, so before the next talk, someone would have to remember to press play again in their media player or on the video player on the stream page. Hrmph.
Easy solution on the morning that the conference starts? I was testing a Debian Live image the night before in a KVM and thought that I might as well just start a Jitsi call from there and keep a steady stream of silence so that Jibri doesn t hang up.
It worked! But the black screen and silence on stream was a bit eery. Because this event was so experimental in nature, and because we were on such an incredibly tight timeline, we opted not to seek sponsors for this event, so there was no sponsors loop that we d usually stream during a DebConf event. Then I thought Ah! I could just show the schedule! .
DebConf20 Moves Online For DebConf, we usually show a sponsors loop in between sessions. It s great that we give our sponsors visibility here, but in reality people see the sponsors loop and think Talk over! and then they look away. It s also completely silent and doesn t provide any additional useful information. I was wondering how I could take our lessons from MDCO#1 and integrate our new tricks with the sponsors loop. That is, add the schedule, time, some space to type announcements on the screen and also add some loopable music to it. I used OBS before in making my videos, and like the flexibility it provides when working with scenes and sources. A scene is what you would think of as a screen or a document with its own collection of sources or elements. For example, a scene might contain sources such as a logo, clock, video, image, etc. A scene can also contain another scene. This is useful if you want to contain a banner or play some background music that is shared between scenes.
The Loopy Loop Music The two mini albums that mostly played during the first few days were just a copy and paste from the MDCO#1 music, which was:
For shoutout tracks, that were later used in the loop too (because it became a bit monotonous), most of the tracks came from freepd.com:
On Saturday 29 August 2020, the annual Debian Developers
and Contributors Conference came to a close.
DebConf20 has been held online for the first time, due to the coronavirus
(COVID-19) disease pandemic.
All of the sessions have been streamed, with a variety of ways of participating:
via IRC messaging, online collaborative text documents,
and video conferencing meeting rooms.
With more than 850 attendees from 80 different countries and a
total of over 100 event talks, discussion sessions,
Birds of a Feather (BoF) gatherings and other activities,
DebConf20 was a large success.
When it became clear that DebConf20 was going to be an online-only
event, the DebConf video team spent much time over the next months to
adapt, improve, and in some cases write from scratch, technology that
would be required to make an online DebConf possible. After lessons
learned from the MiniDebConfOnline in late May, some adjustments were
made, and then eventually we came up with a setup involving Jitsi, OBS,
Voctomix, SReview, nginx, Etherpad, and a newly written web-based
frontend for voctomix as the various elements of the stack.
All components of the video infrastructure are free software, and the
whole setup is configured through their public
ansible repository.
The DebConf20 schedule included
two tracks in other languages than English: the Spanish language MiniConf,
with eight talks in two days,
and the Malayalam language MiniConf, with nine talks in three days.
Ad-hoc activities, introduced by attendees over the course of the entire conference,
have been possible too, streamed and recorded. There have also been several
team gatherings to sprint on certain Debian development areas.
Between talks, the video stream has been showing the usual sponsors on the loop, but also
some additional clips including photos from previous DebConfs, fun facts about Debian
and short shout-out videos sent by attendees to communicate with their Debian friends.
For those who were not able to participate, most of the talks and sessions are already
available through the
Debian meetings archive website,
and the remaining ones will appear in the following days.
The DebConf20 website
will remain active for archival purposes and will continue to offer
links to the presentations and videos of talks and events.
Next year, DebConf21 is planned to be held
in Haifa, Israel, in August or September.
DebConf is committed to a safe and welcome environment for all participants.
During the conference, several teams (Front Desk, Welcome team and Community team)
have been available to help so participants get their best experience
in the conference, and find solutions to any issue that may arise.
See the web page about the Code of Conduct in DebConf20 website
for more details on this.
Debian thanks the commitment of numerous sponsors
to support DebConf20, particularly our Platinum Sponsors:
Lenovo,
Infomaniak,
Google
and
Amazon Web Services (AWS).
About Debian
The Debian Project was founded in 1993 by Ian Murdock to be a truly
free community project. Since then the project has grown to be one of
the largest and most influential open source projects. Thousands of
volunteers from all over the world work together to create and
maintain Debian software. Available in 70 languages, and
supporting a huge range of computer types, Debian calls itself the
universal operating system.
About DebConf
DebConf is the Debian Project's developer conference. In addition to a
full schedule of technical, social and policy talks, DebConf provides an
opportunity for developers, contributors and other interested people to
meet in person and work together more closely. It has taken place
annually since 2000 in locations as varied as Scotland, Argentina, and
Bosnia and Herzegovina. More information about DebConf is available from
https://debconf.org/.
About Lenovo
As a global technology leader manufacturing a wide portfolio of connected products,
including smartphones, tablets, PCs and workstations as well as AR/VR devices,
smart home/office and data center solutions, Lenovo
understands how critical open systems and platforms are to a connected world.
About Infomaniak
Infomaniak is Switzerland's largest web-hosting company,
also offering backup and storage services, solutions for event organizers,
live-streaming and video on demand services.
It wholly owns its datacenters and all elements critical
to the functioning of the services and products provided by the company
(both software and hardware).
About Google
Google is one of the largest technology companies in the
world, providing a wide range of Internet-related services and products such
as online advertising technologies, search, cloud computing, software, and hardware.
Google has been supporting Debian by sponsoring DebConf for more than
ten years, and is also a Debian partner sponsoring parts
of Salsa's continuous integration infrastructure
within Google Cloud Platform.
About Amazon Web Services (AWS)
Amazon Web Services (AWS) is one of the world's
most comprehensive and broadly adopted cloud platforms,
offering over 175 fully featured services from data centers globally
(in 77 Availability Zones within 24 geographic regions).
AWS customers include the fastest-growing startups, largest enterprises
and leading government agencies.
Contact Information
For further information, please visit the DebConf20 web page at
https://debconf20.debconf.org/
or send mail to press@debian.org.
--pinentry-mode
flag that GnuPG uses to suppress password prompting.
I handled the latter by skipping the tests if the gpg on the user's PATH
was too old. I'm not certain this is the best approach, although it makes
the CPAN automated testing more useful for me, since the module will not
work without special configuration on those systems. On the other hand,
if someone is installing it to point to some other GnuPG binary on the
system at runtime, failing the installation because their system gpg is
too old seems wrong, and the test failure doesn't indicate a bug in the
module.
Essentially, I'm missing richer test metadata in the Perl ecosystem. I
want to be able to declare a dependency on a non-Perl system binary, but
of course Perl has no mechanism to do that.
I thought about trying to deal with the Windows failures due to missing
IPC::Run features (redirecting high-numbered file descriptors) on the
Windows platform in a similar way, but decided in that case I do want the
tests to fail because PGP::Sign will never work on that platform
regardless of the runtime configuration. Here too I spent some time
searching for some way to indicate with Module::Build that the module
doesn't work on Windows, and came up empty. This seems to be a gap in
Perl's module distribution ecosystem.
In any case, hopefully this release will clean up the remaining test
failures on Linux and BSD systems, and I can move on to work on the Big
Eight signing key, which was the motivating application for these
releases.
You can get the latest release from CPAN or from the
PGP::Sign distribution page.
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. Last week s blog post documented how there are now over 30,000 lintian issues that have been fixed in git packaging repositories by the Janitor. It's important to note that any fixes from the Janitor that make it into a Git packaging repository will also need to be uploaded to the Debian archive. This currently requires that a Debian packager clones the repository and builds and uploads the package. Until a change makes it into the archive, users of Debian will unfortunately not see the benefits of improvements made by the Janitor. 82% of the 30,000 changes from the Janitor that have made it into a Git repository have not yet been uploaded, although changes do slowly trickle in as maintainers make other changes to packages and upload them along with the lintian fixes from the Janitor. This is not just true for changes from the Janitor, but for all sorts of other smaller improvements as well. However, the process of cloning and building git repositories and uploading the resulting packages to the Debian archive is fairly time-consuming and it s probably not worth the time of developers to follow up every change from the Janitor with a labour-intensive upload to the archive. It would be great if it was easier to trigger uploads from git commits. Projects like tag2upload will hopefully help, and make it more likely that changes end up in the Debian archive. The majority packages do get at least one new source version upload per release, so most changes will eventually make it into the archive.
For more information about the Janitor's lintian-fixes efforts, see the landing page.
A new minor release of the anytime package arrived on CRAN yesterday. This is the twentieth release, but sadly we seem to be spinning our wheels just accomodating CRAN (which the two or three last releases focused on). Code and functionality remain mature and stable, of course.
anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, format to either POSIXct or Date objects and to do so without requiring a format string as well as accomodating different formats in one input vector. See the anytime page, or the GitHub README.md for a few examples.
This release once again has to play catchup with CRAN as r-devel now changes how tzone propagates when we do as.POSIXct(as.POSIXlt(Sys.time()) which is now no longer equal to as.POSIXct(Sys.time()) even for a fixed, stored Sys.time() call result. Probably for the better, but an issue for now so we effectively just reduced test coverage. Call it progress .
The full list of changes follows.
Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.Changes in anytime version 0.3.9 (2020-08-26)
- Skip one test file that is impossible to run across different CRAN setups, and life is definitely too short for these games.
- Change remaining http:// to https:// because, well, you know.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
Next.