• dh + dh_* now scales a lot better for large number of binary packages. Even more so with parallel builds.
• Most dh_* tools are now a lot faster when creating many directories or installing files.
• dh_prep and dh_clean now bulk removals.
• dh_install can now bulk some installations. For a concrete corner-case, libssl-doc went from approximately 11 seconds to less than a second. This optimization is implicitly disabled with exclude (among other).
• dh_installman now scales a lot better with many manpages. Even more so with parallel builds.
• dh_installman has restored its performance under fakeroot (regression since 10.2.2)

• avoiding fork+exec of commands for things doable natively in perl. Especially, when each fork+exec only process one file or dir.
• bulking as many files/dirs into the call as possible, where fork+exec is still used.
• caching / memorizing slow calls (e.g. in parts of pkgfile inside Dh_Lib)
• adding an internal API for dh to do bulk check for pkgfiles. This is useful for dh when checking if it should optimize out a helper.
• and, of course, doing things in parallel where trivially possible.

• If you use install_ file,prog,lib,dir , then it will come out of the box. These functions are available in Debian/stable. On a related note, if you use doit to call install (or mkdir ), then please consider migrating to these functions instead.
• If you need to reset owner+mode (chown 0:0 FILE + chmod MODE FILE), consider using reset_perm_and_owner. This is also available in Debian/stable.
  • CAVEAT: It is not recursive and YMMV if you do not need the chown call (due to fakeroot).
• If you have a lot of items to be processed by a external tool, consider using xargs(). Since 10.5.1, it is now possible to insert the items anywhere in the command rather than just in the end.
• If you need to remove files, consider using the new rm_files function. It removes files and silently ignores if a file does not exist. It is also available since 10.5.1.
• If you need to create symlinks, please consider using make_symlink (available in Debian/stable) or make_symlink_raw_target (since 10.5.1). The former creates policy compliant symlinks (e.g. fixup absolute symlinks that should have been relative). The latter is closer to a ln -s call.
• If you need to rename a file, please consider using rename_path (since 10.5). It behaves mostly like mv -f but requires dest to be a (non-existing) file.
• Have a look at whether on_pkgs_in_parallel() / on_items_in_parallel() would be suitable for enabling parallelization in your tool.
  • The emphasis for these functions is on making parallelization easy to add with minimal code changes. It pre-distributes the items which can lead to unbalanced workloads, where some processes are idle while a few keeps working.

• Helmut Grohne
• Kurt Roeckx
• Gianfranco Costamagna
• Iain Lane
• Sven Joachim
• Adrian Bunk
• Michael Stapelberg

• Adrian Bunk:
  • #867771 filed against python-trollius.
  • #867773 filed against relatorio.
  • #867781 filed against gcin.
  • #867890 filed against apache-directory-jdbm.
  • #867900 filed against yapsy.
  • #867906 filed against wcsaxes.
• Chris Lamb:
  • #867753 filed against grunt (forwarded upstream).
  • #867848 filed against gconf (forwarded upstream).
  • #868133 filed against grep.
  • #868321 filed against node-marked-man (forwarded-upstream).
• Bernhard M. Wiedemann:
  • chromium
  • chromium
  • kubernetes
  • qtscriptgenerator
  • crash merged
  • ghc-rpm-macros merged
  • samba
  • htmldoc
  • efl
  • webkitgtk3 merged
  • fence-agents
  • fio merged

• build_path_captured_by_valac
• timestamps_in_javascript_generated_by_node_grunt_banner

• Add fix for timestamps_embedded_in_manpages_by_node_marked_man toolchain issue.
• Add fix for timestamps_in_javascript_generated_by_node_grunt_banner toolchain issue.
• timestamps_in_javascript_generated_by_node_grunt_banner is actually in src:grunt

• Adrian Bunk (47)

• Ximin Luo:
  • Attempt to fix fsimage test on Jenkins
  • Use a tempdir rather than ./cache for guestfs cache
  • Add more debugging output to test_fsimage

• Mattia Rizzolo:
  • RequiredToolNotFound.get_package(): just call the new get_package_provider()
  • Add a get_package_provider() function, returning the package name that best matches the system
  • Move from deprecated platform.linux_distribution() to the external package distro

• Niels Thykier:
  • Add missing use statements in handler modules
  • dh_strip_nondeterminism: Assumes tmpdir() exists
  • File::StripNondeterminism:
    • Apply perltidy
    • Lazy load most handlers
    • Lazy load remaining handlers
• Chris Lamb:
  • Add missing File::Temp imports in JAR and PNG handlers. This appears to have been exposed by lazily-loading handlers in #867982.

• Ximin Luo:
  • presets: use newer flag --no-sign for dpkg-buildpackage
  • Document --diffoscope-args= --exclude-directory-metadata and use it in presets
• Mattia Rizzolo:
  • Bump debhelper compat level to 10.
  • Bump Standards-Version to 4.0.0.

• Chris Lamb:
  • Avoid a race condition between check-and-creation of Buildinfo instances.

• Mattia Rizzolo:
  • Make database backups quicker to restore by avoiding --column-inserts's pg_dump option.
  • Fixup the deployment scripts after the stretch migration.
  • Fixup Apache redirects that were broken after introducing the buster suite
  • Fixup diffoscope jobs that were not always installing the highest possible version of diffoscope
• Holger Levsen:
  • Add a node health check for a too big jenkins.log.

Changes in Rcpp version 0.12.12 (2017-07-13)

• Changes in Rcpp API:
  • The tinyformat.h header now ends in a newline (#701).
  • Fixed rare protection error that occurred when fetching stack traces during the construction of an Rcpp exception (Kirill M ller in #706).
  • Compilation is now also possibly on Haiku-OS (Yo Gong in #708 addressing #707).
  • Dimension attributes are explicitly cast to int (Kirill M ller in #715).
  • Unused arguments are no longer declared (Kirill M ller in #716).
  • Visibility of exported functions is now supported via the R macro atttribute_visible (Jeroen Ooms in #720).
  • The no_init() constructor accepts R_xlen_t (Kirill M ller in #730).
  • Loop unrolling used R_xlen_t (Kirill M ller in #731).
  • Two unused-variables warnings are now avoided (Jeff Pollock in #732).
• Changes in Rcpp Attributes:
  • Execute tools::package_native_routine_registration_skeleton within package rather than current working directory (JJ in #697).
  • The R portion no longer uses dir.exists to no require R 3.2.0 or newer (Elias Pipping in #698).
  • Fix native registration for exports with name attribute (JJ in #703 addressing #702).
  • Automatically register init functions for Rcpp Modules (JJ in #705 addressing #704).
  • Add Shield around parameters in Rcpp::interfaces (JJ in #713 addressing #712).
  • Replace dot (".") with underscore ("_") in package names when generating native routine registrations (JJ in #722 addressing #721).
  • Generate C++ native routines with underscore ("_") prefix to avoid exporting when standard exportPattern is used in NAMESPACE (JJ in #725 addressing #723).

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

fn copyright_fromgit(repo_url: &str) -> Result<String>  
   let tempdir = TempDir::new_in(".", "debcargo")?;
   let repo = Repository::clone(repo_url, tempdir.path())?;
   let mut revwalker = repo.revwalk()?;
   revwalker.push_head()?;
   // Get the latest and first commit id. This is bit ugly
   let latest_id = revwalker.next().unwrap()?;
   let first_id = revwalker.last().unwrap()?; // revwalker ends here is consumed by last
   let first_commit = repo.find_commit(first_id)?;
   let latest_commit = repo.find_commit(latest_id)?;
   let first_year =
       DateTime::<Utc>::from_utc(
               NaiveDateTime::from_timestamp(first_commit.time().seconds(), 0),
               Utc).year();
   let latest_year =
       DateTime::<Utc>::from_utc(
             NaiveDateTime::from_timestamp(latest_commit.time().seconds(), 0),
             Utc).year();
   let notice = match first_year.cmp(&latest_year)  
       Ordering::Equal => format!(" ", first_year),
       _ => format!(" - ,", first_year, latest_year),
    ;
   Ok(notice)
 

So here is what I'm doing

1. Use git2::Repository::clone to clone the given URL. We are thus avoiding exec of git clone command.
2. Get a revision walker object. git2::RevWalk implements Iterator trait and allows walking through the git history. This is what we are using to avoid exec of git log command.
3. revwalker.push_head() is important because we want to tell revwalker from where we want to walk the history. In this case we are asking it to walk history from repository HEAD. Without this line next line will not work. (Learned it in hard way :-) ).
4. Then we extract git2::Oid which is we can say similar to commit hash and can be used to lookup a particular commit. We take latest commit hash using RevWalk::next call and the first commit using Revwalk::last, note the order this is because Revwalk::last consumes the revwalker so doing it in reverse order will make borrow checker unhappy :-). This replaces exec of head -n1 command.
5. Look up the git2::Commit objects using git2::Repository::find_commit
6. Then convert the git2::Time to chrono::DateTime and take out the years.

Welcome to Yakking. This is a blog for topics relevant to someone new to free software development. We assume you are already familiar with computers, and are curious about participating in the production of free software. You don't need to be a programmer: software development requires a wide variety of skills, and you can be a valued core contributor to a project without being a programmer.

• Adrian Bunk:
  • #867499 filed against tiptop.
  • #867773 filed against relatorio.
  • #867781 filed against gcin.
• Chris Lamb:
  • #866945 filed against tinymux.
  • #867753 filed against grunt.
  • #867848 filed against gconf.

• Bernhard M. Wiedemann:
  • perl-Class-MethodMaker sort hashtable
  • perl-Mouse sort file list
  • thunderbird sort symbol list
  • SOURCE_DATE_EPOCH support:
    • graphviz, merged
    • gnupg, merged in improved variant by upstream
    • janus-gateway, merged
    • libkcapi merged
  • txt2tags fixup on earlier patch, merged

• Adrian Bunk (143)
• Andreas Beckmann (1)
• Dmitry Shachnev (1)
• Lucas Nussbaum (3)
• Niko Tyni (3)
• Scott Kitterman (1)
• Sean Whitton (1)

• Ximin Luo:
  • Add a PartialString class to help with lazily-loaded output formats such as html-dir.
  • html and html-dir output:
    • add a size-hint to the diff headers and lazy-load buttons
    • add new limit flags and deprecate old ones
  • html-dir output
    • split index pages up if they get too big
    • put css/icon data in separate files to avoid duplication
  • main: warn if loading a diff but also giving diff-calculation flags
  • Test fixes for Python 3.6 and CI environments without imagemagick (#865625).
  • Fix a performance regression (#865660) involving the Wagner-Fischer algorithm for calculating levenshtein distance.

The front panel board

If you know the original or remember the picture from the first post it is clear that this is a functional replica not aiming to be as pretty as those of the other projects I mentioned. I have reordered the switches into two rows to make the board more compact (which also means cheaper) without sacrificing usability. There s the two rows of display lights plus one run light the 8/e provides. The upper row is the address made up of 12 bits of memory address and 3 bits of extended memory address or field. Below are the 12 bits indicator which can show one data set out of six as selected by the user. All the switches of the original are implemented as more compact buttons¹. While momentary switches are easily substituted by buttons, all buttons implementing two position switches toggle on/off with each press and they have a LED above that shows the current state. The six position rotary switch is implemented as a button cycling through all indicator displays together with six LEDs which show the active selection. Markings show the meaning of the indicator and switches as on the original, grouped in threes as the predominant numbering system for the PDPs was octal. The upper line shows the meaning for the state indicator, the middle for the status indicator and bit numbers for the rest. Note that on the PDP-8 and opposite to modern conventions, the most significant bit was numbered 0. I designed it as a pure front panel board without any PDP-8 simulation parts. The buttons and associated lights are controllable via SPI lines with a 3.3 V supply. The address and indicator lights have a separate common anode configuration with all cathodes individually available on a pin header without any resistors in the path, leaving voltage and current regulation up to the simulation board. This board is actually a few years old from a similar project where I emulated the PDP-8 in software on a microcontroller and the flexible design allowed me to reuse it unchanged.

The main board

This is where the magic happens. You can see three big ICs on the board: On the left is the STM32F405 microcontroller (with ARM Cortex-M4 core), the bigger one in the middle is the Altera² MAX 10 FPGA and finally to the right is the SRAM that is large enough to hold all the main memory of the 32 KW maximum expansion of the PDP-8/e. The two smaller chips to the right of that are just buffers that drive the front panel address LEDs, the small chip at the top left is a RS-232 level shifter. The idea behind this is that the PDP-8 and peripherals that are simple to directly implement, such as GPIO or a serial port, are fully on the FPGA. Other peripherals such as paper and magnetic tape and disks, which are after all not connected to real PDP-8 drives but disk images on a microSD, are implemented on the microcontroller interfacing with stub devices in the FPGA. Compared to implementing everything everything in the FPGA, the STM32F4 has the advantage of useful built in peripherals such as two host/device capable USB ports. 5 V tolerant I/O pins are very useful and simply not available in any FPGA. I have to admit that this board was a bit of a rush job in order to have something at all to show at the Vintage Computer Festival Europe 18.0. Given that it was my first time designing a board with a large microcontroller and the first time with an FPGA, it wasn t exactly my fastest progressing project either and I got basic functionality (front panel allows toggling in small programs and running them) working just in time. For various reasons the project hasn t progressed much since, so the following is still just plans, but plans for which the hardware was designed. Since the aim is to have a cycle accurate PDP-8/e implementation, digital I/O was always planned. Rather than defining my own header I have included Arduino R3 compatible headers (for 3.3 V compatible boards only) that have become a popular even outside the Arduino world for this purpose. The digital Arduino pins are connected directly to the FPGA and will be directly controllable by PDP-8 software. The downside of choosing the Arduino headers is that the original PDP-8 digital I/O interface is not a perfect match since it naturally has 12 lines whereas the Arduino has 15. The analog inputs are not connected to the FPGA, the documentation of the MAX10 s ADC in the EQFP package are not very encouraging. They are connected to the STM32 instead³. Another interface connected directly to the FPGA and that would be directly under PDP-8 control is a standard 9 pin RS-232 interface. RX, TX, CTS and RTS are connected and level-shifted between 3.3 V and RS-232 levels by a MAX3232. Besides the PDP-8, I also plan to implement a full video terminal on the board. The idea is that with a power supply, keyboard and monitor this board would form a complete system without the need of connecting another computer to act as a terminal. To that end, there is a VGA port attached to the FPGA with simple resistor network DACs for 9 bits color (RGB with 3 bits each). This is another spot where I left myself room to expand, for e.g. a VT220 you really only need one color in two brightness levels. Keyboards will be connected either via the PS/2 connector on the right or the USB-A host port at the top left. Last of the interface ports is the USB micro-AB port on the left, which for now I am using only for power supply. I mainly plan to use it to provide alternative or additional serial ports to the PDP-8 or to export the video terminal serial port for testing purposes. Other possible uses are access to the image files on the microSD and firmware updates. This has gotten rather long again, so I m stopping here and leave some implementation notes for another post.

---

1. They are also much cheaper. Given the large number of switches, the savings are substantial. Additionaly the buttons are nicer to operate than long rows of tiny switches. ^[return]
2. Or rather Intel now. At least Altera s web site, documentation and software have already been thoroughly rebranded, but the chips I got were produced prior to that. ^[return]
3. That s not to say that the analog conversions on the STM32 are necessarily better than those of the MAX10 when you can t follow their guidelines, I have no comparisons. Certainly following the guidelines would have been prohibitive given how many pins usage they restrict. ^[return]

Sub-defines With a more modern lace (version 1.3 or later) there is a mechanism we are calling 'sub-defines'. Previously if you wanted to write a ruleset which said something like "Allow Steve to read my repository" you needed:

define is_steve user exact steve
allow "Steve can read my repo" is_steve op_read

And, as you'd expect, if you also wanted to grant read access to Jeff then you'd need yet set of defines:

define is_jeff user exact jeff
define is_steve user exact steve
define readers anyof is_jeff is_steve
allow "Steve and Jeff can read my repo" readers op_read

This, while flexible (and still entirely acceptable) is wordy for small rulesets and so we added sub-defines to create this syntax:

allow "Steve and Jeff can read my repo" op_read [anyof [user exact jeff] [user exact steve]]

Of course, this is generally neater for simpler rules, if you wanted to add another user then it might make sense to go for:

define readers anyof [user exact jeff] [user exact steve] [user exact susan]
allow "My friends can read my repo" op_read readers

The nice thing about this sub-define syntax is that it's basically usable anywhere you'd use the name of a previously defined thing, they're compiled in much the same way, and Richard worked hard to get good error messages out from them just in case.

No more auto_user_XXX and auto_group_YYY As a result of the above being implemented, the support Gitano previously grew for automatically defining users and groups has been removed. The approach we took was pretty inflexible and risked compilation errors if a user was deleted or renamed, and so the sub-define approach is much much better. If you currently use auto_user_XXX or auto_group_YYY in your rulesets then your upgrade path isn't bumpless but it should be fairly simple:

1. Upgrade your version of lace to 1.3
2. Replace any auto_user_FOO with [user exact FOO] and similarly for any auto_group_BAR to [group exact BAR].
3. You can now upgrade Gitano safely.

No more 'basic' matches Since Gitano first gained support for ACLs using Lace, we had a mechanism called 'simple match' for basic inputs such as groups, usernames, repo names, ref names, etc. Simple matches looked like user FOO or group !BAR. The match syntax grew more and more arcane as we added Lua pattern support refs ~^refs/heads/$ user /. When we wanted to add proper PCRE regex support we added a syntax of the form: user pcre ^/.+?... where pcre could be any of: exact, prefix, suffix, pattern, or pcre. We had a complex set of rules for exactly what the sigils at the start of the match string might mean in what order, and it was getting unwieldy. To simplify matters, none of the "backward compatibility" remains in Gitano. You instead MUST use the what how with match form. To make this slightly more natural to use, we have added a bunch of aliases: is for exact, starts and startswith for prefix, and ends and endswith for suffix. In addition, kind of match can be prefixed with a ! to invert it, and for natural looking rules not is an alias for !is. This means that your rulesets MUST be updated to support the more explicit syntax before you update Gitano, or else nothing will compile. Fortunately this form has been supported for a long time, so you can do this in three steps.

1. Update your gitano-admin.git global ruleset. For example, the old form of the defines used to contain define is_gitano_ref ref ~^refs/gitano/ which can trivially be replaced with: define is_gitano_ref ref prefix refs/gitano/
2. Update any non-zero rulesets your projects might have.
3. You can now safely update Gitano

If you want a reference for making those changes, you can look at the Gitano skeleton ruleset which can be found at https://git.gitano.org.uk/gitano.git/tree/skel/gitano-admin/rules/ or in /usr/share/gitano if Gitano is installed on your local system. Next time, I'll likely talk about the deprecated commands which are no longer in Gitano, and how you'll need to adjust your automation to use the new commands.

• Released DLA-983-1 and DLA-984-1 on tiff3/tiff to fix 4 CVE. I also updated our patch set to get back in sync with upstream since we had our own patches for a while and upstream ended up using a slightly different approach. I checked that the upstream fix did really fix the issues with the reproducer files that were available to us.
• Handled CVE triage for a whole week.
• Released DLA-1006-1 on libarchive (2 CVE fixed by Markus Koschany, one by me).

• knocker
• recon-ng
• dsniff
• libnids
• rfdump
• snoopy
• dirb
• wcc
• arpwatch
• dhcpig
• backdoor-factory

No comment Liked this article? Click here. My blog is Flattr-enabled.

• 500,000 routes extracted from an Internet router (half of them are /24), and
• 500,000 host routes (/32) tightly packed in 4 distinct subnets.

• In Linux 2.6.39, commit 3630b7c050d9, David Miller removes the hash-based routing table implementation to switch to the LPC-trie implementation (available since Linux 2.6.13 as a compile-time option). This brings a small regression for the scenario with many host routes but boosts the performance for the general case.
• In Linux 3.0, commit 281dc5c5ec0f, the improvement is not related to the network subsystem. Linus Torvalds disables the compiler size-optimization from the default configuration. It was believed that optimizing for size would help keeping the instruction cache efficient. However, compilers generated under-performing code on x86 when this option was enabled.
• In Linux 3.6, commit f4530fa574df, David Miller adds an optimization to not evaluate IP rules when they are left unconfigured. From this point, the use of the CONFIG_IP_MULTIPLE_TABLES option doesn t impact the performances unless some IP rules are configured. This version also removes the route cache (commit 5e9965c15ba8). However, this has no effect on the benchmark as it directly calls fib_lookup() which doesn t involve the cache.
• In Linux 4.0, notably commit 9f9e636d4f89, Alexander Duyck adds several optimizations to the trie lookup algorithm. It really pays off!
• In Linux 4.1, commit 0ddcf43d5d4a, Alexander Duyck collapses the local and main tables when no specific IP rules are configured. For non-local traffic, both those tables were looked up.

• Updated travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds:
  • Support Debian "buster". (commit)
  • Set TRAVIS=true environment variable when running autopkgtests. (#45)
• Updated the documentation in django-slack, my library to easily post messages to the Slack group-messaging utility to link to Slack's own message formatting documentation. (#66)
• Added "buster" support to local-debian-mirror, my package to easily maintain and customise a local Debian mirror via the DebConf configuration tool. (commit)