Search Results: "nate"

8 November 2017

Dirk Eddelbuettel: RQuantLib 0.4.4: Several smaller updates

A shiny new (mostly-but-not-completely maintenance) release of RQuantLib, now at version 0.4.4, arrived on CRAN overnight, and will get to Debian shortly. This is the first release in over a year, and it it contains (mostly) a small number of fixes throughout. It also includes the update to the new DateVector and DatetimeVector classes which become the default with the upcoming Rcpp 0.12.14 release (just like this week's RcppQuantuccia release). One piece of new code is due to Fran ois Cocquemas who added support for discrete dividends to both European and American options. See below for the complete set of changes reported in the NEWS file. As with release 0.4.3 a little over a year ago, we will not have new Windows binaries from CRAN as I apparently have insufficient powers of persuasion to get CRAN to update their QuantLib libraries. So we need a volunteer. If someone could please build a binary package for Windows from the 0.4.4 sources, I would be happy to once again host it on the GHRR drat repo. Please contact me directly if you can help. Changes are listed below:

Changes in RQuantLib version 0.4.4 (2017-11-07)
  • Changes in RQuantLib code:
    • Equity options can now be analyzed via discrete dividends through two vectors of dividend dates and values (Francois Cocquemas in #73 fixing #72)
    • Some package and dependency information was updated in files DESCRIPTION and NAMESPACE.
    • The new Date(time)Vector classes introduced with Rcpp 0.12.8 are now used when available.
    • Minor corrections were applied to BKTree, to vanilla options for the case of intraday time stamps, to the SabrSwaption documentation, and to bond utilities for the most recent QuantLib release.

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list off the R-Forge page. Issue tickets can be filed at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Jonathan Dowland: Christmas

Every year, family members ask me to produce a list of gift suggestions for them to buy for me for Christmas. An enviable position for many, I'm sure, but combined with trying to come up with gift ideas for them, this can sometimes be a stressful situation, with a risk of either giving or receiving gifts that are really nothing more than tat, fluff or kipple. I've started to feel that this is detracting from the spirit of the season. I also don't really want much "stuff". When I am interested in something, it's not something that is convenient for others to buy, either because it's hard to describe, or has limited availability, or is only available at particular times of the year, etc. I'd rather focus on spending time with friends and family. Starting this year, I'm asking that people who wish to do so donate to a charity on my behalf instead. The charity I have chosen for this year is St. Oswald's Hospice.

7 November 2017

Rog rio Brito: Some activities of the day

Yesterday, I printed the first draft of the first chapter when my little boy was here and he was impressed with this strange object called a "printer". Before I printed what I needed, I fired up LibreOffice and chose the biggest font size that was available and let him type his first name by himself. He was quicker than I thought with a keyboard. After seeing me print his first name, he was jumping up and down with joy of having created something and even showed grandma and grandpa what he had done. He, then, wanted more and I taught him how to use that backspace key, what it meant and he wanted to type his full name. I let him and taught him that there is a key called space that he should type every time he wants to start a new word and, in the end, he typed his first two names. To my surprise, he memorized the icon with the printer (which I must say that I have to hunt every time, since it seems so similar to the adjacent ones!) and pressed this new key called "Enter". When he pressed, he wasn't expecting the printer on his right to start making noises and printing his name. He was so excited and it was so nice to see his reaction full of joy to get a job done! I am thinking of getting a spare computer, building it with him and for him, so that he can call it his computer every time he comes to see daddy. As a serendipitous situation, Packt Publishing offered yesterday their title "Python Projects for Kids". Unfortunately, he does not yet know how to read, but I guess that the right age is coming soon, which is a good thing to make him be educated "the right way" (that is, with the best support, teaching and patience that I can give him). Anyway, I printed the first draft of the first chapter and today I have to turn it in. As I write this, I am downloading a virtual machine from Microsoft to try to install Java on it. Let me see if it works. I have none of the virtualization options used, tough the closest seems to be virtualbox. Let me cross my fingers. In other news, I updated some of the tags of very old posts of this blog, and I am seriously thinking about switching from [ikiwiki][0] to another blog platform. It is slow, very slow on my system with the repositories that I have, especially on my armel system. Some non-interpreted system would be best, but I don't know if such a thing even exists. But the killer problem is that it doesn't support easily the typing of Mathematics (even though a 3rd party plugin for MathJax exists). On the other hand, I just received an answer on twitter from @telegram and it was nice:
Hello, Telegram supports bold and italic. You can type **bold** and __italic__. On mobile, you can also highlight text for this as well.
It is nice that this works with telegram-desktop too. Besides that, I filed some bugs on Debian's BTS, responded to some issues on my projects on GitHub (I'm slowly getting back on maintaining things) and file wishlist bugs on some other projects. Oh, and I grabbed a copy of "Wonder woman" ("Mulher Maravilha") and "Despicable Me 3" ("Meu Malvado Favorito 3") dubbed in Brazilian Portuguese for my son. I have to convert the audio from AAC-LC in 6 channels to AC3 or to stereo. Otherwise, my TVs have problem with the videos (one refuses to play the entire file and another plays the audio with sounds of hiccups). Edit: After converting the VirtualBox image taken from Microsoft, I could easily use qemu/kvm to create screenshots of the installation of Java. The command that I used (for future reference) is: qemu-system-x86_64 -enable-kvm -m 4096 -smp 2 -net nic,model=e1000 -net user -soundhw ac97 -drive index=0,media=disk,cache=unsafe,file=win7.qcow2 Edit: Fixed some typos.

Dirk Eddelbuettel: RcppQuantuccia 0.0.2

A first maintenance release of RcppQuantuccia got to CRAN earlier today. RcppQuantuccia brings the Quantuccia header-only subset / variant of QuantLib to R. At present it mostly offers calendaring, but Quantuccia just got a decent amount of new functions so hopefully we can offer more here too. This release was motivated by the upcoming Rcpp release which will deprecate the okd Date and Datetime vectors in favours of newer ones. So this release of RcppQuantuccia switches to the newer ones. Other changes are below:

Changes in version 0.0.2 (2017-11-06)
  • Added calendars for Canada, China, Germany, Japan and United Kingdom.
  • Added bespoke and joint calendars.
  • Using new date(time) vectors (#6).

Courtesy of CRANberries, there is also a diffstat report relative to the previous release. More information is on the RcppQuantuccia page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 November 2017

John Goerzen: The Yellow House Phone Company (Featuring Asterisk and an 11-year-old)

Well Jacob, do you think we should set up our own pretend phone company in the house? We can DO THAT? Yes! Then yes. Yes! YES YES YESYESYESYES YES! Let s do it, dad! Not long ago, my parents had dug up the old phone I used back in the day. We still have a landline, and Jacob was having fun discovering how an analog phone works. I told him about the special number he could call to get the time and temperature read out to him. He discovered what happens if you call your own number and hang up. He figured out how to play Mary Had a Little Lamb using touchtone keys (after a slightly concerned lecture from me setting out some rules to make sure his musical dialing wouldn t result in any, well, dialing.) He was hooked. So I thought that taking it to the next level would be a good thing for a rainy day. I have run Asterisk before, though I had unfortunately gotten rid of most of my equipment some time back. But I found a great deal on a Cisco 186 ATA (Analog Telephone Adapter). It has two FXS lines (FXS ports simulate the phone company, and provide dialtone and ring voltage to a connected phone), and of course hooks up to the LAN. We plugged that in, and Jacob was amazed to see its web interface come up. I had to figure out how to configure it (unfortunately, it uses SCCP rather than SIP, and figuring out Asterisk s chan_skinny took some doing, but we got there.) I set up voicemail. He loved it. He promptly figured out how to record his own greetings. We set up a second phone on the other line, so he could call between them. The cordless phones in our house support SIP, so I configured one of them as a third line. He spent a long time leaving himself messages. IMG_3465 Pretty soon we both started having ideas. I set up extension 777, where he could call for the time. Then he wanted a way to get the weather forecast. Well, weather-util generates a text-based report. With it, a little sed and grep tweaking, the espeak TTS engine, and a little help from sox, I had a shell script worked up that would read back a forecast whenever he called a certain extension. He was super excited! That s great, dad! Can it also read weather alerts too? Sure! weather-util has a nice option just for that. Both boys cackled as the system tried to read out the NWS header (their timestamps like 201711031258 started with two hundred one billion ) Then I found an online source for streaming NOAA Weather Radio feeds Jacob enjoys listening to weather radio and I set up another extension he could call to listen to that. More delight! But it really took off when I asked him, Would you like to record your own menu? You mean those things where it says press 1 or 2 for this or that? Yes. WE CAN DO THAT? Oh yes! YES, LET S DO IT RIGHT NOW! So he recorded a menu, then came and hovered by me while I hacked up extensions.conf, then eagerly went back to the phone to try it. Oh the excitement of hearing hisown voice, and finding that it worked! Pretty soon he was designing sub-menus ( OK Dad, so we ll set it up so people can press 2 for the weather, and then choose if they want weather radio or the weather report. I m recording that now. Got it? ) He has informed me that next Saturday we will build an intercom system like we have at school. I m going to have to have some ideas on how to tie Squeezebox in with Asterisk to make that happen, I think. Maybe this will do.

5 November 2017

Thorsten Alteholz: My Debian Activities in October 2017

FTP assistant Again, this month almost the same numbers as last month appeared in the statistics. I accepted 214 packages and rejected 22 uploads. The overall number of packages that got accepted this month was only 339. Debian LTS This was my fortieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 20.75h. During that time I did LTS uploads of: I also took care of radare2 twice and marked all four CVEs as not-affected for Wheezy and Jessie. As nobody else wanted to address the issues in wireshark yet, I now started to work on this package. Last but not least I did one week of frontdesk duties. Other stuff During October I took care of some bugs and at one go uploaded new upstream versions of hoel and duktape (this had to be done twice as I introduced an new bug with the first upload :-(). I only fixed bugs in glewlwyd and smstools. This month I also sponsored an upload of printrun. After about ten years of living without any power outage, some construction worker decided to cut a cable near my place. Unfortunately one of my computers used for recording TV shows did not boot after the cable had been repaired and I had to switch some timers to other boxes. All in all this was too much stress and I purchased some USVs from APC. As apcupsd was orphaned, I took the opportunity to adopt it as DOPOM for this month. My license pasting project now contains 31 license templates for your debian/copyright. The list of available texts can be obtained with:

curl http://licapi.debian.net/template

The license text itself is available under the given links, for example with

curl http://licapi.debian.net/template/Apache-2

Dirk Eddelbuettel: pinp 0.0.4: Small tweak

A maintenance release of our pinp package for snazzier one or two column vignettes is now on CRAN as of yesterday. In version 0.0.3, we disabled the default \pnasbreak command we inherit from the PNAS LaTeX style. That change turns out to have been too drastic. So we reverted yet added a new YAML front-matter option skip_final_break which, if set to TRUE, will skip this break. With a default value of FALSE we maintain prior behaviour. A screenshot of the package vignette can be seen below. Additional screenshots of are at the pinp page.
The NEWS entry for this release follows.

Changes in pinp version 0.0.4 (2017-11-04)
  • Correct NEWS headers from 'tint' to 'pinp' (#45).
  • New front-matter variables skip_final_break skips the \pnasbreak on final page which back as default (#47).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 November 2017

Dirk Eddelbuettel: tint 0.0.4: Small enhancements

A maintenance release of the tint package arrived on CRAN earlier today. Its name expands from tint is not tufte as the package offers a fresher take on the Tufte-style for html and pdf presentations. A screenshot of the pdf variant is below.
This release brings some minor enhancements and polish, mostly learned from having done the related pinp (two-column vignette in the PNAS style) and linl (LaTeX letter) RMarkdown-wrapper packages; see below for details from the NEWS.Rd file.

Changes in tint version 0.0.4 (2017-11-02)
  • Skeleton files are also installed as vignettes (#20).
  • A reference to the Tufte source file now points to tint (Ben Marwick in #19, later extended to other Rmd files).
  • Several spelling and grammar errors were corrected too (#13 and #16 by R. Mark Sharp and Matthew Henderson)

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

3 November 2017

Raphaël Hertzog: My Free Software Activities in October 2017

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I was allocated 12h but I had 1.5h left from September too. During this time, I finally finished my work on exiv2: I completed the triage of all CVE, backported 3 patches to the version in wheezy and released DLA-1147-1. I also did some review of the oldest entries in dla-needed. I reclassified a bunch of CVE on zoneminder and released DLA-1145-1 for the most problematic issue on that package. Many other packages got their CVE reclassified as not worth an update: xbmc, check-mk, rbenv, phamm, yaml-cpp. For mosquitto, I released DLA-1146-1. I filed #879001 (security issue) and #879002 (removal suggestion) on libpam4j. This library is no longer used by any other package in Debian, so it could be removed instead of costing us time in support. Misc Debian work After multiple months of wait, I was allowed to upload my schroot stable update (#864297). After ack from the d-i release manager, I pushed my pkgsel changes and uploaded version 0.46 of the package: this brings unattended-upgrades support in the installer. It s now installed by default. I nudged the upstream developer of gnome-shell-timer to get a new release for GNOME 3.26 compatibility and packaged it. Finally, I was pleased to merge multiple patches from Ville Skytt on Distro Tracker (the software powering tracker.debian.org). It looks like Ville will continue to contribute on a regular basis, yay. \o/ He already helped me to fix the remaining blockers for the switch to Python 3. Not really Debian related, but I also filed a bug against Tryton that I discovered after upgrading to the latest version. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

2 November 2017

Antoine Beaupr : October 2017 report: LTS, feed2exec beta, pandoc filters, git mediawiki

Debian Long Term Support (LTS) This is my monthly Debian LTS report. This time I worked on the famous KRACK attack, git-annex, golang and the continuous stream of GraphicsMagick security issues.

WPA & KRACK update I spent most of my time this month on the Linux WPA code, to backport it to the old (~2012) wpa_supplicant release. I first published a patchset based on the patches shipped after the embargo for the oldstable/jessie release. After feedback from the list, I also built packages for i386 and ARM. I have also reviewed the WPA protocol to make sure I understood the implications of the changes required to backport the patches. For example, I removed the patches touching the WNM sleep mode code as that was introduced only in the 2.0 release. Chunks of code regarding state tracking were also not backported as they are part of the state tracking code introduced later, in 3ff3323. Finally, I still have concerns about the nonce setup in patch #5. In the last chunk, you'll notice peer->tk is reset, to_set to negotiate a new TK. The other approach I considered was to backport 1380fcbd9f ("TDLS: Do not modify RNonce for an TPK M1 frame with same INonce") but I figured I would play it safe and not introduce further variations. I should note that I share Matthew Green's observations regarding the opacity of the protocol. Normally, network protocols are freely available and security researchers like me can easily review them. In this case, I would have needed to read the opaque 802.11i-2004 pdf which is behind a TOS wall at the IEEE. I ended up reading up on the IEEE_802.11i-2004 Wikipedia article which gives a simpler view of the protocol. But it's a real problem to see such critical protocols developed behind closed doors like this. At Guido's suggestion, I sent the final patch upstream explaining the concerns I had with the patch. I have not, at the time of writing, received any response from upstream about this, unfortunately. I uploaded the fixed packages as DLA 1150-1 on October 31st.

Git-annex The next big chunk on my list was completing the work on git-annex (CVE-2017-12976) that I started in August. It turns out doing the backport was simpler than I expected, even with my rusty experience with Haskell. Type-checking really helps in doing the right thing, especially considering how Joey Hess implemented the fix: by introducing a new type. So I backported the patch from upstream and notified the security team that the jessie and stretch updates would be similarly easy. I shipped the backport to LTS as DLA-1144-1. I also shared the updated packages for jessie (which required a similar backport) and stretch (which didn't) and those Sebastien Delafond published those as DSA 4010-1.

Graphicsmagick Up next was yet another security vulnerability in the Graphicsmagick stack. This involved the usual deep dive into intricate and sometimes just unreasonable C code to try and fit a round tree in a square sinkhole. I'm always unsure about those patches, but the test suite passes, smoke tests show the vulnerability as fixed, and that's pretty much as good as it gets. The announcement (DLA 1154-1) turned out to be a little special because I had previously noticed that the penultimate announcement (DLA 1130-1) was never sent out. So I made a merged announcement to cover both instead of re-sending the original 3 weeks late, which may have been confusing for our users.

Triage & misc We always do a bit of triage even when not on frontdesk duty, so I: I also did smaller bits of work on: The latter reminded me of the concerns I have about the long-term maintainability of the golang ecosystem: because everything is statically linked, an update to a core library (say the SMTP library as in CVE-2017-15042, thankfully not affecting LTS) requires a full rebuild of all packages including the library in all distributions. So what would be a simple update in a shared library system could mean an explosion of work on statically linked infrastructures. This is a lot of work which can definitely be error-prone: as I've seen in other updates, some packages (for example the Ruby interpreter) just bit-rot on their own and eventually fail to build from source. We would also have to investigate all packages to see which one include the library, something which we are not well equipped for at this point. Wheezy was the first release shipping golang packages but at least it's shipping only one... Stretch has shipped with two golang versions (1.7 and 1.8) which will make maintenance ever harder in the long term.
We build our computers the way we build our cities--over time, without a plan, on top of ruins. - Ellen Ullman

Other free software work This month again, I was busy doing some serious yak shaving operations all over the internet, on top of publishing two of my largest LWN articles to date (2017-10-16-strategies-offline-pgp-key-storage and 2017-10-26-comparison-cryptographic-keycards).

feed2exec beta Since I announced this new project last month I have released it as a beta and it entered Debian. I have also wrote useful plugins like the wayback plugin that saves pages on the Wayback machine for eternal archival. The archive plugin can also similarly save pages to the local filesystem. I also added bash completion, expanded unit tests and documentation, fixed default file paths and a bunch of bugs, and refactored the code. Finally, I also started using two external Python libraries instead of rolling my own code: the pyxdg and requests-file libraries, the latter which I packaged in Debian (and fixed a bug in their test suite). The program is working pretty well for me. The only thing I feel is really missing now is a retry/fail mechanism. Right now, it's a little brittle: any network hiccup will yield an error email, which are readable to me but could be confusing to a new user. Strangely enough, I am particularly having trouble with (local!) DNS resolution that I need to look into, but that is probably unrelated with the software itself. Thankfully, the user can disable those with --loglevel=ERROR to silence WARNINGs. Furthermore, some plugins still have some rough edges. For example, The Transmission integration would probably work better as a distinct plugin instead of a simple exec call, because when it adds new torrents, the output is totally cryptic. That plugin could also leverage more feed parameters to save different files in different locations depending on the feed titles, something would be hard to do safely with the exec plugin now. I am keeping a steady flow of releases. I wish there was a way to see how effective I am at reaching out with this project, but unfortunately GitLab doesn't provide usage statistics... And I have received only a few comments on IRC about the project, so maybe I need to reach out more like it says in the fine manual. Always feels strange to have to promote your project like it's some new bubbly soap... Next steps for the project is a final review of the API and release production-ready 1.0.0. I am also thinking of making a small screencast to show the basic capabilities of the software, maybe with asciinema's upcoming audio support?

Pandoc filters As I mentioned earlier, I dove again in Haskell programming when working on the git-annex security update. But I also have a small Haskell program of my own - a Pandoc filter that I use to convert the HTML articles I publish on LWN.net into a Ikiwiki-compatible markdown version. It turns out the script was still missing a bunch of stuff: image sizes, proper table formatting, etc. I also worked hard on automating more bits of the publishing workflow by extracting the time from the article which allowed me to simply extract the full article into an almost final copy just by specifying the article ID. The only thing left is to add tags, and the article is complete. In the process, I learned about new weird Haskell constructs. Take this code, for example:
-- remove needless blockquote wrapper around some tables
--
-- haskell newbie tips:
--
-- @ is the "at-pattern", allows us to define both a name for the
-- construct and inspect the contents as once
--
--   is the "empty record pattern": it basically means "match the
-- arguments but ignore the args"
cleanBlock (BlockQuote t@[Table  ]) = t
Here the idea is to remove <blockquote> elements needlessly wrapping a <table>. I can't specify the Table type on its own, because then I couldn't address the table as a whole, only its parts. I could reconstruct the whole table bits by bits, but it wasn't as clean. The other pattern was how to, at last, address multiple string elements, which was difficult because Pandoc treats spaces specially:
cleanBlock (Plain (Strong (Str "Notifications":Space:Str "for":Space:Str "all":Space:Str "responses":_):_)) = []
The last bit that drove me crazy was the date parsing:
-- the "GAByline" div has a date, use it to generate the ikiwiki dates
--
-- this is distinct from cleanBlock because we do not want to have to
-- deal with time there: it is only here we need it, and we need to
-- pass it in here because we do not want to mess with IO (time is I/O
-- in haskell) all across the function hierarchy
cleanDates :: ZonedTime -> Block -> [Block]
-- this mouthful is just the way the data comes in from
-- LWN/Pandoc. there could be a cleaner way to represent this,
-- possibly with a record, but this is complicated and obscure enough.
cleanDates time (Div (_, [cls], _)
                 [Para [Str month, Space, Str day, Space, Str year], Para _])
    cls == "GAByline" = ikiwikiRawInline (ikiwikiMetaField "date"
                                           (iso8601Format (parseTimeOrError True defaultTimeLocale "%Y-%B-%e,"
                                                           (year ++ "-" ++ month ++ "-" ++ day) :: ZonedTime)))
                        ++ ikiwikiRawInline (ikiwikiMetaField "updated"
                                             (iso8601Format time))
                        ++ [Para []]
-- other elements just pass through
cleanDates time x = [x]
Now that seems just dirty, but it was even worse before. One thing I find difficult in adapting to coding in Haskell is that you need to take the habit of writing smaller functions. The language is really not well adapted to long discourse: it's more about getting small things connected together. Other languages (e.g. Python) discourage this because there's some overhead in calling functions (10 nanoseconds in my tests, but still), whereas functions are a fundamental and important construction in Haskell that are much more heavily optimized. So I constantly need to remind myself to split things up early, otherwise I can't do anything in Haskell. Other languages are more lenient, which does mean my code can be more dirty, but I feel get things done faster then. The oddity of Haskell makes frustrating to work with. It's like doing construction work but you're not allowed to get the floor dirty. When I build stuff, I don't mind things being dirty: I can cleanup afterwards. This is especially critical when you don't actually know how to make things clean in the first place, as Haskell will simply not let you do that at all. And obviously, I fought with Monads, or, more specifically, "I/O" or IO in this case. Turns out that getting the current time is IO in Haskell: indeed, it's not a "pure" function that will always return the same thing. But this means that I would have had to change the signature of all the functions that touched time to include IO. I eventually moved the time initialization up into main so that I had only one IO function and moved that timestamp downwards as simple argument. That way I could keep the rest of the code clean, which seems to be an acceptable pattern. I would of course be happy to get feedback from my Haskell readers (if any) to see how to improve that code. I am always eager to learn.

Git remote MediaWiki Few people know that there is a MediaWiki remote for Git which allow you to mirror a MediaWiki site as a Git repository. As a disaster recovery mechanism, I have been keeping such a historical backup of the Amateur radio wiki for a while now. This originally started as a homegrown Python script to also convert the contents in Markdown. My theory then was to see if we could switch from Mediawiki to Ikiwiki, but it took so long to implement that I never completed the work. When someone had the weird idea of renaming a page to some impossible long name on the wiki, my script broke. I tried to look at fixing it and then remember I also had a mirror running using the Git remote. It turns out it also broke on the same issue and that got me looking in the remote again. I got lost in a zillion issues, including fixing that specific issue, but I especially looked at the possibility of fetching all namespaces because I realized that the remote fetches only a part of the wiki by default. And that drove me to submit namespace support as a patch to the git mailing list. Finally, the discussion came back to how to actually maintain that contrib: in git core or outside? Finally, it looks like I'll be doing some maintenance that project outside of git, as I was granted access to the GitHub organisation...

Galore Yak Shaving Then there's the usual hodgepodge of fixes and random things I did over the month.
There is no [web extension] only XUL! - Inside joke

1 November 2017

Steve Kemp: Possibly retiring blogspam.net

For the past few years I've hosted a service for spam-testing blog/forum comments, and I think it is on the verge of being retired. The blogspam.net service presented a simple API for deciding whether an incoming blog/forum comment was SPAM, in real-time. I used it myself for two real reasons: As a result of the Debian-Administration server cleanup I'm still in the process of tidying up virtual machines, and servers. It crossed my mind that retiring this spam-service would allow me to free up another host. Initially the service was coded in Perl using XML/RPC. The current version of the software, version 2, is written as a node.js service, and despite the async-nature of the service it is still too heavy-weight to live on the host which runs most of my other websites. It was suggested to me that rewriting it in golang might allow it to process more requests, with fewer resources, so I started reimplementing the service in golang at 4AM this morning: The service does the minimum: I've ported several plugins, I've got 100% test-coverage of those plugins, and the service seems to be faster than the node.js version - so there is hope. Of course the real test will be when it is deployed for real. If it holds up for a few days I'll leave it running. Otherwise the retirement notice I placed on the website, which chances are nobody will see, will be true. The missing feature at the moment is keeping track of the count of spam-comments rejected/accepted on a per-site basis. Losing that information might be a shame, but I think I'm willing to live with it, if the alternative is closing down..

Petter Reinholdtsen: Some notes on fault tolerant storage systems

If you care about how fault tolerant your storage is, you might find these articles and papers interesting. They have formed how I think of when designing a storage system. Several of these research papers are based on data collected from hundred thousands or millions of disk, and their findings are eye opening. The short story is simply do not implicitly trust RAID or redundant storage systems. Details matter. And unfortunately there are few options on Linux addressing all the identified issues. Both ZFS and Btrfs are doing a fairly good job, but have legal and practical issues on their own. I wonder how cluster file systems like Ceph do in this regard. After all, there is an old saying, you know you have a distributed system when the crash of a compyter you have never heard of stops you from getting any work done. The same holds true if fault tolerance do not work. Just remember, in the end, it do not matter how redundant, or how fault tolerant your storage is, if you do not continuously monitor its status to detect and replace failed disks.

James McCoy: Monthly FLOSS activity - 2017/10 edition

Debian subversion vim
libvterm
pangoterm
vim-gnupg
neovim
vim

31 October 2017

Dirk Eddelbuettel: linl 0.0.2: Couple improvements

Following up on the initial 0.0.1 release of linl, Aaron and I are happy to announce release 0.0.2 which reached the CRAN network on Sunday in a smooth 'CRAN-pretest-publish' auto-admittance. linl provides a simple-yet-powerful Markdown---and RMarkdown---wrapper around the venerable LaTeX letter class; see below for an expanded example also included as the package vignette. This versions sets a few sensible default values for font, font size, margins, signature (non-)indentation and more; it also expands the documentation.
The NEWS entry follows:

Changes in tint version 0.0.2 (2017-10-29)
  • Set a few defaults for a decent-looking skeleton and template: font, fontsize, margins, left-justify closing (#3)
  • Blockquote display is now a default as well (#4).
  • Updated skeleton.Rmd and vignette source accordingly
  • Documented new default options (#5 and #6).
  • Links are now by default printed as footnotes (#9).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: pinp 0.0.3: More docs, more features

Our pinp package for snazzier one or two column vignette received it second update. Now at version 0.0.3, it arrived on CRAN on Saturday with minimal fuzz as an 'CRAN-pretest-publish' transition. We added more frontmatter options, documented more, and streamlined some internals of the LaTeX class borrowed from PNAS. A screenshot of the (updated) vignette can be seen below. Additional screenshots of are at the pinp page.
The NEWS entry for this release follows.

Changes in tint version 0.0.3 (2017-10-28)
  • Section 'Acknowledgements' now conditional on a frontmatter setting, section 'Matmethods' has been removed, pnasbreak no longer used which stabilizes LaTeX float formatting. References are now shown in the column just like other content (Dirk in #36).
  • Vignette now uses new numbered sections frontmatter switch which improves the pdf outline.
  • New front-matter options for title/section header colors, and link colors (Dirk in #39).
  • YAML frontmater options are now documented in the help page for pinp as well (Dirk in #41).
  • Some typos were fixed (Michael in #42 and #43).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

29 October 2017

Niels Thykier: Building packages without (fake)root

Turns out that it is surprisingly easy to build most packages without (fake)root. You just need to basic changes:
  1. A way to set ownership to root:root of paths when dpkg-deb build constructs the binary.
  2. A way to have debhelper not do a bunch of (now) pointless chowns to root:root .
The above is sufficient for dpkg, debhelper, lintian, apt-file, mscgen, pbuilder and a long list of other packages that only provide paths owned by root:root . Obviously, packages differ and yours might need more tweaks than this (e.g. dh_usrlocal had to change behaviour to support this). But for me, the best part is that the above is not just some random prototype stuck in two git repos on alioth: Unfortunately, if you are working with games or core packages like shadow with need for static ownership different from root:root (usually with a setuid or setgid bit), then our first implementation does not support your needs at the moment[1]. We are working on a separate way to solve static ownership in a declarative way. [1] Note regarding /usr/local : If your package needs to provide directories there owned by root:staff with mode 02775, then dh_usrlocal can handle that. The non- root:root ownership here works because the directories are created in a maintainer script run as root during installation. Unfortunately, it cannot provide different ownership or modes with R != binary-targets at the moment.
Filed under: Debhelper, Debian

28 October 2017

Russ Allbery: Review: Provenance

Review: Provenance, by Ann Leckie
Publisher: Orbit
Copyright: September 2017
ISBN: 0-316-38863-7
Format: Kindle
Pages: 448
In a rather desperate attempt to please her mother, Ingray has spent every resource she has on extracting the son of a political enemy from Compassionate Removal (think life imprisonment with really good marketing). The reason: vestiges, a cultural touchstone for Ingray's native planet of Hwae. These are invitation cards, floor tiles, wall panels, or just about anything that can be confirmed to have been physically present at an important or historical moment, or in the presence of a famous figure. The person Ingray is retrieving supposedly pulled off the biggest theft of vestiges in history. If she can locate them, it would be a huge coup for her highly-placed politician mother, and the one time she would be victorious in her forced rivalry with her brother. About the best thing that could be said for this plan is that it's audacious. The first obstacle is the arrival of the Geck on the station for a Conclave for renegotiation of the treaty with the Presger, possibly the most important thing going on in the galaxy at the moment, which strands her there without money for food. The second is that the person she has paid so much to extract from Compassionate Removal says they aren't the person she was looking for at all, and are not particularly interested in going with her to Hwae. Only a bit of creative thinking in the face of a visit from the local authorities, and the unexpected kindness of the captain from whom she booked travel, might get her home with the tatters of her plan intact. But she's clearly far out of her depth. Provenance is set in the same universe as Ancillary Justice and its sequels, but it is not set in the empire of the Radchaai. This is another human world entirely, one with smaller and more provincial concerns. The aftermath of Ancillary Mercy is playing out in the background (so do not, on risk of serious spoilers, read the start of this book without having read the previous trilogy), but this is in no way a sequel. Neither the characters nor the plot are involved in that aftermath. It's a story told at a much smaller scale, about two political families, cut-throat maneuvering, horrible parenting, the inexplicable importance of social artifacts, the weirdness of human/alien relations, and the merits of some very unlikely allies. Provenance is a very different type of story than Ancillary Justice, and Ingray is a very different protagonist. The shape of the plot reminded me of one of Lois McMaster Bujold's Miles Vorkosigan stories: hair-brained ideas, improvisation, and unlikely allies. But Ingray couldn't be more different than Miles. She starts the book overwhelmed, despairing, and not at all manic, and one spends the first part of the story feeling sorry for her and becoming quite certain that everything will go horribly wrong. The heart of this book is the parallel path Leckie takes the reader and the characters along as they discover just what Ingray's true talents and capabilities are. It's a book about being hopelessly bad at things one was pressured towards being good at, while being quietly and subtly good at the skills that let one survive a deeply dysfunctional family. There are lots of books with very active protagonists, and a depressing number of books with passive protagonists pushed around by the plot. There are very few books that pull off the delicate characterization that Leckie manages here: a protagonist who is rather hopeless at taking charge of the plot in the way everyone wants (but doesn't particularly expect) her to, but who charts her own path through the plot in an entirely unexpected way. It's a story that grows on you. The plot rhythm never works in quite the way one expects from other books, but it builds its own logic and its own rhythm, and reached a very emotionally satisfying conclusion. The Radchaai, or at least one Radchaai citizen, do show up eventually, providing a glimmer of outside view at the Ancillary Justice world. Even better, the Geck play a significant role. I adore Leckie's aliens: they're strange and confusing, but in a refreshingly blunt way rather than abusing gnomic utterances and incomprehensible intelligence. And the foot-stomping of the spider bot made me laugh every time. The stakes are a lot lower here than in Ancillary Justice, and Ingray isn't the sort of character who's going to change the world. But that's okay; indeed, one of the points of this book is why and how that's okay. I won't lie: I'd love more Breq, and I hope we eventually get an exploration of the larger consequences of her story. But this is a delightful story that made me happy and has defter character work than most SF being written. Recommended, but read the Ancillary trilogy first. One minor closing complaint, which didn't change my experience of the book but which I can't help quibbling about: I'm completely onboard with the three-gender system that Leckie uses for the Hwae (I wish more SF authors would play with social as well as technological ideas), and I think she wove it deftly into the story, but I wish she hadn't used Spivak pronouns for the third gender. (e/em/eir, for those who aren't familiar.) Any of the other gender-neutral pronouns look better to me and cause fewer problems for my involuntary proofreader. I prefer zie/zir for personal reasons, but sie/hir, zhe/zhim/zher, or even thon or per would read more smoothly. Eir is fine, but em looks like 'em and throws my brain into dialect mode and forces a re-parse, and e just looks like a typo. I know from lots of Usenet discussions of pronouns that I'm not the only one who has that reaction to Spivak. But it's a very minor nit. Rating: 8 out of 10

27 October 2017

Sam Hartman: How Can Debian Turn Disagreement into Something that Makes us Stronger

Recently, when asked to engage with the Debian Technical Committee, a maintainer chose to orphan their package rather than discuss the issue brought before the committee. In another decision earlier this year, a maintainer orphaned their package indicating a lack of respect for the approach being taken and the process. Unfortunately, this joins an ever longer set of issues where people walk away from the TC process disheartened and upset.

For me personally the situations where maintainers walked away from the process were hard. People I respect and admire were telling me that they were unwilling to participate in our dispute resolution process. In one case the maintainer explicitly did not respect a process I had been heavily involved in. As someone who values understanding and build a team, I feel disappointed and hurt thinking about this.

Unfortunately, I don't feel much better when I consider several of the other issues brought before the TC. In a number of cases, the process has concluded, but it feels like a pyrrhic victory. We have an answer, but often we never reached an understanding of one of the key stakeholder's positions. People are sufficiently disappointed or frustrated that they reduce their involvement. That is, the answer is not one they can live with. Sometimes, I'm not even sure that answering the question was worth the cost.

My initial suspicion as I begin to consider issues that have come before the TC in the last few years is that as a necessary consequence of how the TC operates, we will drive a significant chunk of those who come to us away from our community. I will not be part of that. If I end up eventually agreeing with this suspicion, I will either work to improve the TC process or find a different way to contribute to the project that is more aligned with my goals.


Our Community is More Important than Technical Correctness I firmly believe that Debian's strength is its community. Distributions, upstreams, free-software advocates, corporate interests, and everything in between work together. Because we are working on a concrete product,we actually have to understand how our needs conflict and compliment each other. Also, like any organization, our ability to get things done is dependent on our contributors and the time we all put in.

I cannot think of anything more harmful we can do than drive away a constructive contributor. Technical quality is important: many of us will eventually go away if quality drops too much. However, Debian itself can be improved incrementally. Yes, new people do join Debian. However, it takes a lot of new people to make up for a situation where everyone involved is frustrated, and some leave and tell their friends to stay away from Debian.

When a Disagreement Reaches the TC When a disagreement reaches the TC, we know a few things. First, people have not have not been able to resolve it on their own. Second, the disagreement is important enough to at least one of the parties to ask for outside help.

If the TC were to simply announce a decision, it is very likely that at least one party would feel frustrated and disappointed. If the decision is important to someone it almost always means that they care about the outcome. If previous efforts have not reached agreement then there is an outcome that is undesired. While it's possible under the constitution that two parties could bring an issue to the TC mutually where the dynamics could be different, this does not happen in practice.

When we "lose" When a decision making process decides to select one of my undesired outcomes, I have strong negative feelings. First, there is the technical result itself. Because of an adverse decision it is generally harder for me to do my technical work, or some ethical position or group I care about is disadvantaged. However, the strongest feelings are related to needs about my place in the community, not a direct response to the technical decision. I worry about whether issues I care about will be valued in the future; I would likely feel weary or scared thinking of contributing in an environment where issues that matter to me aren't going to be considered. I tend to feel frustrated if my position was not adequately considered this time around.

Often, the strongest feelings do not stem from how I am impacted by the decision itself. Thus, if my more general needs are addressed, I will feel a lot better about not having my preferred outcome selected. Confidence that my position was understood is the single biggest factor in being able to accept the result. If the community took the time to understand me, then I have confidence that I am valued. I can advocate for change and be part of the ongoing discussion. If I understand the other positions then I can understand that the position was not arbitrary. Understanding the other positions is also a prerequisite to seeing how things I value were considered, especially when the tradeoff did not ultimately favor these values.

My conversations with others about their experience with conflict suggest my feelings and needs are common.


However, the TC focuses almost exclusively on the technical matter before it. I don't think the TC has done a good job of actually understanding maintainers or the people bringing issues to us. Especially when we're fairly sure we understand what the ultimate decision will be, we focus on getting to that decision. Of course part of actually understanding someone is considering what they have to say. Even when you have high confidence in an outcome, if you want someone to feel understood, you do have to listen at a point where you can consider what they bring forward.

Frustration of Being Questioned

On the other side, it's frustrating to have your decisions questioned. Reviewing a decision takes time. Even if the TC agrees with a maintainer, asking that maintainer to sit through a long process can create frustrations as deep as any I discussed in the previous section.

So the process is doubly bad for maintainers. Someone can bring up an issue which requires precious time. Then if the TC decides against the maintainer, we have all the problems I discussed previously.

I acknowledge this. A good process must respect the maintainer's time. However, it must respect the community members who disagree with maintainers. Everyone who brings an issue to the TC is a developer. They have contributed significantly to our community and are part of our team. Yes, we need to protect the process against abuse. But in a very real sense if we aren't willing to consider an issue and we aren't willing to engage with someone, understand why they think the issue should be considered, and work until they understand why we made our decision, we are saying we do not respect them enough to take that time. We should expect that to drive people away.

I wonder if the solution here involves a two phase process. During the first phase we work with someone bringing an issue to us until we understand it. We either engage the maintainer at that point, spend the time to help the person bringing an issue to us understand why we are not engaging the maintainer, or have decided the issue is abusive of the processs/misdirected. For this to work maintainers would have to trust us enough to actually be willing to sit back and not spend a lot of their time.

Sam, It's Just Personalities

One criticism I've heard as I discuss this is that a lot of the negative feelings surround interpersonal conflict and are personality conflicts. Yes, personalities matter. Yes, we re still healing from the Systemd discussion.

However, as a community, I think we need to figure out how we respect the inevitable personality conflicts. I can think of one or two developers I'm still upset with years after an issue. It happens. Perhaps if I were a maintainer when one of these developers raised a concern with my packages, I could ask that someone else be a primary advocate for an issue. Similarly, if a developer wanted to address an issue with me as a maintainer but would prefer not to deal with me, perhaps we could figure out some way that we could see if someone else shared the concern.

Dropping a Package is Sometimes an Answer

My concerns were sparked by instances where a maintainer dropped a package rather than interact with the TC. Sometimes that can be a healthy step in a transition. At Debconf this year, Enrico Zini gave a talk on consent culture in Debian. One of the key points of his talk is that we can find over time that what we're being asked to do in Debian is no longer consistent with our boundaries. If it isn t helping us move forward if it isn t fun then it is likely time to stop.

In that sense, approaching disagreement can be a great time to take stock and ask whether we still enjoy maintaining a package. If we don t, then stepping aside and letting others take it on can be a great way that we and they can be happier. I don t really think that s what happened in these instances though. Based on comments made to me, this sounded more like a lack of faith in being treated well or in our ability as a committee.

Even so, Enrico s talk applies in a number of ways here. He suggests that the approach we should take when someone is done and steps away is to thank then. I couldn t agree more. From the depth of my heart I offer thanks: Thank you for taking care of yourself and stepping away when it is no longer fun. Thank you for all the effort you have put into these packages. I hope to work with you on great things in the future.

Russ Allbery: Review: The Black Gryphon

Review: The Black Gryphon, by Mercedes Lackey & Larry Dixon
Series: Mage Wars #1
Publisher: DAW
Copyright: 1994
Printing: January 1995
ISBN: 0-88677-643-0
Format: Mass market
Pages: 460
The Mage Wars series (or the Gryphon series, which isn't its official title but which is in all of the titles) is part of the sprawling Valdemar mega-series, but it's a prequel to all of the other stories. It's also slightly challenging if you're reading in publication order, since it was published simultaneously with the Mage Storms series. If you're following publication order, in theory you should interleave the two series, but I hate doing that. I'm therefore reading it after Mage Winds and before Mage Storms. We'll see whether that was a good idea when I get to the next series. You could, if you really wanted to, read this series before any other Valdemar book. As a prequel from the deep past of Valdemar's world, it doesn't depend on the other series, and you'll get a rediscovery of lost knowledge feel from later books. The downside is that it's a rather boring introduction, and that order would spoil a lot of the revelatory flow of the other series (particularly Elspeth's adventures in the Mage Winds books). I'm now getting into the Valdemar books that I've only read once. I've been putting off continuing my Valdemar re-read because this series was next and I remember being rather bored with it the first time I read it. But I'm re-reading for the world-building and background as much as for the characters, and this is a huge chunk of world background that fills in the bones underneath Winds of Fate and its sequels. Here's why Dhorisha is a crater, here's why the Pelagiris forests are such a mess, here's where Ma'ar starts, here's the origin of both the gryphons and K'Leshya, and here, finally, we get to see the legendary Urtho on the page. The problem with writing novels set in the epic backstory of your universe is that it's hard to live up to the drama that readers have invented for themselves. A lot of The Black Gryphon is background to events Valdemar readers already know will happen, creating a corresponding lack of surprise. I reached the end of the book and said "yup, that's pretty much what everyone said had happened." Lackey and Dixon do try to do some interesting things here, one of which being the backgrounding of the war. The Black Gryphon starts in the middle of a long-running conflict between Urtho and Ma'ar and doesn't follow the generals or the battles. The protagonists, instead, are a kestra'chern (a type of psychiatrist and spiritual healer who also uses sex, with the expected conflicts of people who incorrectly think of them as prostitutes) and the eventual leader of the gryphons (Skandranon, who is referenced in later books and who provides the title). We get some combat scenes from Skandranon and later another gryphon, but a lot of the book is Amberdrake fighting the effects of the war instead of the details of the war itself. There's a deep and moving story in that idea, and in some of the attached love stories that play out in the army camps. There's also a great story somewhere around Urtho: a brilliant but detached mage who is way out of his depth trying to run an army but smart enough to gather good people around him. He's also a creator of new life, including the gryphons. The Black Gryphon tries to talk about Urtho's paternalism, the weird emotional currents of his relationship with his creations, and the places Urtho keeps things from others for, supposedly, their own good. If this book had looked a bit deeper at the support structure for an army that's trying to be humane, or at the ways in which Urtho strays far too close to being an abusive tyrant through inaction despite having the best of intentions at every step, I think it could have said something significant. Unfortunately, that's not this book. This book is full of relentlessly black and white morality (the flaw of much of the Valdemar series) that bleaches away interesting shades of grey. Urtho is good and wise by authorial fiat, and Ma'ar is the same utterly irredeemable force of evil that he is in other books. The story skitters over Urtho's odd tyrannies, making them all better with the pure power of friendship and good intentions. There just isn't much emotional depth, and while I don't expect that of Lackey in general, this story really needed that depth to work. What we get instead is repetition, as Lackey and Dixon hit the same emotional notes with Amberdrake repeatedly. This is one of those books that makes me wonder if Lackey was trying to write too many novels in a short time than was good for their individual quality. (Collaborations often mean that the lesser-known name is doing all the work, but Dixon is Lackey's husband and the tone of the book is sufficiently Lackey that I don't think that happened here.) It felt padded by Amberdrake turning over the same emotional rocks repeatedly, to largely the same effect. This is, in short, not Lackey's finest effort, although it does have its moments. As always, Lackey is at her best when writing psychological healing narratives. Zhaneel's story is a bit too easy, but the dynamic between Amberdrake and Winterhart is the best part of the book. And The Black Gryphon does tell the reader exactly what led up to the Cataclysm and why. There are no major surprises, but there are some small ones, and it's a nice payoff for the lore-obsessed (like me). This is missable unless you want the full world-building behind Valdemar's past, and it's not the best writing. But if you're heavily invested in the Valdemar universe, it's at least readable and provides an important bit of the history. Followed by The White Gryphon. Rating: 5 out of 10

24 October 2017

Iain R. Learmonth: Security by Obscurity

Today this blog post turned up on Hacker News, titled Obscurity is a Valid Security Layer . It makes some excellent points on the distinction between good and bad obscurity and it gives an example of good obscurity with SSH. From the post:
I configured my SSH daemon to listen on port 24 in addition to its regular port of 22 so I could see the difference in attempts to connect to each (the connections are usually password guessing attempts). My expected result is far fewer attempts to access SSH on port 24 than port 22, which I equate to less risk to my, or any, SSH daemon. I ran with this alternate port configuration for a single weekend, and received over eighteen thousand (18,000) connections to port 22, and five (5) to port 24.
Those of you that know me in the outside world will have probably heard me talk about how it s insane we have all these services running on the public Internet that don t need to be there, just waiting to be attacked. I have previously given a talk at TechMeetup Aberdeen where I talk about my use of Tor s Onion services to have services that only I should ever connect to be hidden from the general Internet. Onion services, especially the client authentication features, can also be useful for IoT dashboards and devices, allowing access from the Internet but via a secure and authenticated channel that is updated even when the IoT devices behind it have long been abandoned. If you re interested to learn more about Onion services, you could watch Roger Dingledine s talk from Def Con 25.

Next.

Previous.