Search Results: "nate"

14 December 2013

Dirk Eddelbuettel: random 0.2.2

A maintenance release of my random package for truly (hardware-based) random numbers (pulled from random.org) is now on CRAN. It's been a while since previous releases. The package is described in a detailed vignette as well as in a essay by Mads Haahr. Changes were minimal and driven mostly by some CRAN Policy changes which now prefer vignette sources files in (top-level) directory vignettes/ Courtesy of CRANberries, there is also a diffstat report for the most recent release. Current and previous releases are available here as well as on CRAN.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: gcbd 0.2.5

A maintenance release (now at version 0.2.5) of my gcbd package (described only in these two posts) is now CRAN. More details about the package are available in the paper which is also included in the gcbd R package. Changes were minimal and driven mostly by some CRAN Policy changes which now prefer vignette sources files in (top-level) directory vignettes/ Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 December 2013

Daniel Pocock: Get WebRTC going faster

On Saturday, Lumicall began offering free calls from browser to mobile using the free and open WebRTC technology. It should be no surprise that the service has been popular. Is it really free and open? The only way to prove this technology is free is to help people implement this for themself. On Monday, I uploaded reSIProcate v1.9.0 beta7 packages to Debian. The reSIProcate SIP proxy, repro, is one of the core components of the solution behind the free Lumicall service. Simply install the repro and resiprocate-turn-server packages using apt-get and make the following changes to the configuration (use your own IP addresses of course). I've taken this diff from my own runtime environment, only hiding my passwords, so that you can see exactly how I got it working:
--- repro.config.orig	2013-12-11 17:36:27.179228324 +0100
+++ repro-ws.sip5060.net.config	2013-12-11 17:48:24.159938649 +0100
@@ -143,6 +143,41 @@
 # Transport6TlsClientVerification = None
 # Transport6RecordRouteUri = sip:h1.sipdomain.com;transport=WS
 
+Transport1Interface = 195.8.117.57:80
+Transport1Type = WS
+Transport1RecordRouteUri = auto
+
+Transport2Interface = 2001:67c:1388:1000::57:80
+Transport2Type = WS
+Transport2RecordRouteUri = auto
+
+Transport3Interface = 195.8.117.57:5060
+Transport3Type = TCP
+Transport3RecordRouteUri = auto
+
+Transport4Interface = 2001:67c:1388:1000::57:5060
+Transport4Type = TCP
+Transport4RecordRouteUri = auto
+
+Transport5Interface = 195.8.117.57:443
+Transport5Type = WSS
+#Transport5RecordRouteUri = auto
+Transport5TlsDomain = ws.sip5060.net
+Transport5TlsClientVerification = None
+Transport5RecordRouteUri = sip:ws.sip5060.net;transport=WSS
+Transport5TlsCertificate = /etc/ssl/ssl.crt/ws.sip5060.net-bundle.crt
+Transport5TlsPrivateKey = /etc/ssl/private/ws.sip5060.net-key.pem
+
+Transport6Interface = 2001:67c:1388:1000::57:443
+Transport6Type = WSS
+#Transport6RecordRouteUri = auto
+Transport6TlsDomain = ws.sip5060.net
+Transport6TlsClientVerification = None
+Transport6RecordRouteUri = sip:ws.sip5060.net;transport=WSS
+Transport6TlsCertificate = /etc/ssl/ssl.crt/ws.sip5060.net-bundle.crt
+Transport6TlsPrivateKey = /etc/ssl/private/ws.sip5060.net-key.pem
+
+
 # Comma separated list of DNS servers, overrides default OS detected list (leave blank 
 # for default)
 DNSServers =
@@ -455,7 +490,7 @@
 ForceRecordRouting = false
 
 # Assume path option
-AssumePath = false
+AssumePath = true
 
 # Disable registrar
 DisableRegistrar = false
@@ -481,7 +516,7 @@
 # WARNING: Before enabling this, ensure you have a RecordRouteUri setup, or are using
 # the alternate transport specification mechanism and defining a RecordRouteUri per
 # transport: TransportXRecordRouteUri
-DisableOutbound = true
+DisableOutbound = false
 
 # Set the draft version of outbound to support (default: RFC5626)
 # Other accepted values are the versions of the IETF drafts, before RFC5626 was issued
@@ -505,7 +540,7 @@
 # WARNING: Before enabling this, ensure you have a RecordRouteUri setup, or are using
 # the alternate transport specification mechanism and defining a RecordRouteUri per
 # transport: TransportXRecordRouteUri
-EnableFlowTokens = false
+EnableFlowTokens = true
 
 # Enable use of flow-tokens in non-outbound cases for clients detected to be behind a NAT.  
 # This a more selective flow token hack mode for clients not supporting RFC5626.  The 

This is a diff against the /etc/repro/repro.config file distributed in the Debian package version 1.9.0~beta7-1. In the example above, I've included WSS transport defintions for WebSockets over TLS. Use the standard procedure for creating webserver SSL certificates to create certificates for repro and make sure you insert the correct filenames in the TLS parameters above. I've also duplicated every transport for IPv6. If you don't want TLS/WSS or IPv6, just comment those entries out (and renumber the remaining transports). Web-based SIP proxy setup Once you have repro running, go to the web admin interface (port 5080, username: admin, password: admin) and finish the setup using the web UI. The following steps are essential:
  • Add your domain (and restart the repro daemon after adding your domain)
  • Add some users
  • Add any routes to external services (optional - in my next blog I'll demonstrate how to route WebRTC calls to Asterisk using the Debian packages and less than 20 lines of configuration)
Set up reTurn, the TURN server Just put your IP addresses in /etc/reTurnServer.config and add the following line to /etc/reTurnServer-users.txt:
test:notasecret:reTurn:authorized
IMPORTANT: the realm in the users file (reTurn in the example and default config) must be identical to the AuthenticationRealm in the /etc/reTurnServer.config file. On your own web site Simply install your own apache server and clone the webrtc.lumicall.org demo site. Modify the file js/custom.js and include the settings for your own server.
# cd /var/www
# mkdir webcall
# cd webcall
# wget -r -nH http://webrtc.lumicall.org
# vi js/custom.js
In the custom.js, make sure you use a ws:// URL if you didn't set up SSL certificates and use a wss:// URL if you did. The IP or domain of your repro server must be in the ws:// or wss:// URL. Now navigate to the URL ending with /webcall on your server. For RHEL, Fedora and other RPM users Can somebody please assist with the review of the cajun-jsonapi dependency package so I can upload this new version of reSIProcate to Fedora? I'm also planning to make v1.9.0 available in EPEL6 when it is released in January. Questions? Please come and ask questions or discuss your experiences on the Free RTC mailing list sponsored by FSF Europe.

10 December 2013

Dirk Eddelbuettel: RcppArmadillo 0.3.930.1

A new Armadillo release 3.930 came out a few days ago, with a very nice set of changes (see below). I rolled this into RcppArmadillo 0.3.930.0. However, one of these changes revealed that R shipped only the standard SVD for complex-valued matrices, and not the more performant divide-and-conquer approach. So in R builds using the default built-in Lapack, at least one CRAN package no longer built. After some back and forth, Conrad put some branching in the library to fall back to the standard SVD, and I added a built-time configuration test for an appropriate preprocessor directive used by the fallback code. This is now on which is now on CRAN and in Debian as RcppArmadillorelease 0.3.930.1, and Conrad will probably update the Armadillo page as well (though the fix is only needed with R's builtin Rlapack). Also of note is that R Core already added the missing Fortran routine zgesdd to R 3.1.0 (aka "R-devel") so this issue goes away with the next release. Also of note, I wrote up a short Rcpp Gallery post illustrating the performance gains available from divide-and-conquer SVD. The complete list of changes is below.
Changes in RcppArmadillo version 0.3.930.1 (2013-12-09)
  • Upgraded to Armadillo release Version 3.930.1
    • Armadillo falls back to standard complex svd if the more performant divide-and-conquer variant is unavailable
  • Added detection for Lapack library and distinguish between R's own version (withhout zgesdd) and system Lapack; a preprocessor define is set accordingly
Changes in RcppArmadillo version 0.3.930.0 (2013-12-06)
  • Upgraded to Armadillo release Version 3.930 ("Dragon's Back")
    • added divide-and-conquer variant of svd_econ(), for faster SVD
    • added divide-and-conquer variant of pinv(), for faster pseudo-inverse
    • added element-wise variants of min() and max()
    • added size() based specifications of submatrix view sizes
    • added randi() for generating matrices with random integer values
    • added more intuitive specification of sort direction in sort() and sort_index()
    • added more intuitive specification of method in det(), .i(), inv() and solve()
    • added more precise timer for the wall_clock class when using C++11
  • New unit tests for complex matrices and vectors
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Russ Allbery: krb5-sync 3.0

krb5-sync is the software we run at Stanford to synchronize principal information from a central Heimdal realm to Active Directory, allowing users to use either a Linux-based Kerberos environment or Active Directory with the same account and password. The original intent of this release was to add a new feature to allow a subsidiary instance of an account in the MIT or Heimdal realm to be synchronized with the instance-less account in Active Directory. This allows, for example, an rra/windows instance to be used to set and maintain the password for an rra principal in Active Directory. In the process of implementing that, though, I ended up doing a significant overall of the code, since the plugin architecture was quite awkward and dated. The code now uses the MIT Kerberos data structures in a more natural and native way, since MIT Kerberos has now added direct support for plugins of this sort. Kerberos contexts and Kerberos error codes are used uniformly throughout the plugin, which provides consistent and more robust error handling and reporting. I also significantly enhanced the test suite, although it still needs more work to test the core functionality that has complex external dependencies. This release also drops support for all versions of MIT Kerberos prior to 1.9, which required an external patch; to run krb5-sync 3.0, you should upgrade to a recent version of MIT Kerberos. This allowed me to drop support for the legacy API. There are a couple of major backward-incompatible changes in this release (and both unfortunately are not handled automatically by the Debian package upgrade, since it's hard to find and safely modify KDC configuration). First, the ad_ldap_base configuration option is now mandatory when synchronizing account status and its meaning has changed. Previously, dc elements for the realm were appended to a provided partial base. Now, the complete DN of the root of the Active Directory tree should be provided. This is more flexible and more useful with a wider variety of Active Directory setups. Second, I took advantage of the backward-incompatibilities to change the module name to sync.so from krb5_sync.so, since the latter sounded weirdly redundant and verbose when installed in the Kerberos plugin directory. This will require a configuration change to the plugin configuration for the KDC or kadmin server. Also in this release are a couple of new options: ad_queue_only, which forces all changes to be queued for later processing instead of processed in real time, and syslog, which can be used to turn off the internal syslog logging of non-errors from the module. (This is mostly useful for test suites.) Now, password changes are queued on any Active Directory failure, not just a few oddly-distinguished ones. The previous behavior was rather specific to Stanford's needs, and queuing all password changes shouldn't pose any problems. Finally, the krb5-sync-backend utility program for manipulating the queued changes has been completely rewritten and is much cleaner. It now uses the Net::Remctl::Backend Perl module for command and option handling, so that module (provided with remctl 3.4 or later) must be installed. It also requires IPC::Run, which is available from CPAN. It uniformly supports a -d option to specify the queue location, and skips event files during processing that no longer exist. You can get the latest release from krb5-sync distribution page.

7 December 2013

Dirk Eddelbuettel: R and Big Data at Big Data Summit at UI Research Park

I spent yesterday at the very enjoyable Big Data Summit held at the University of Illinois Research Park at the edge of the University of Illinois at Urbana-Champaign. campus. My (short) presentation was part of a panel session on R and Big Data which Doug Simpson of the UIUC Statistics department had put together very well. We heard from a vendor / technology provider with Christopher Nguyen from Adatao talking about their "Big R", from industry with Andy Stevens talking about a number of some real-life challenges with big data at John Deere, from academia with Jonathon Greenberg talking about R and HPC for geospatial research and I added a few short comments and links about R, HPC and Rcpp. My few slides are now up on my talks / presentations page. Overall, a good day with a number of interesting presentations and of course a number of engaging hallway discussions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 December 2013

Petter Reinholdtsen: Debian Edu interview: Klaus Knopper

It has been a while since I managed to publish the last interview, but the Debian Edu / Skolelinux community is still going strong, and yesterday we even had a new school administrator show up on #debian-edu to share his success story with installing Debian Edu at their school. This time I have been able to get some helpful comments from the creator of Knoppix, Klaus Knopper, who was involved in a Skolelinux project in Germany a few years ago. Who are you, and how do you spend your days? I am Klaus Knopper. I have a master degree in electrical engineering, and is currently professor in information management at the university of applied sciences Kaiserslautern / Germany and freelance Open Source software developer and consultant. All of this is pretty much of the work I spend my days with. Apart from teaching, I'm also conducting some more or less experimental projects like the Knoppix GNU/Linux live system (Debian-based like Skolelinux), ADRIANE (a blind-friendly talking desktop system) and LINBO (Linux-based network boot console, a fast remote install and repair system supporting various operating systems). How did you get in contact with the Skolelinux / Debian Edu project? The credit for this have to go to Kurt Gramlich, who is the German coordinator for Skolelinux. We were looking for an all-in-one open source community-supported distribution for schools, and Kurt introduced us to Skolelinux for this purpose. What do you see as the advantages of Skolelinux / Debian Edu? What do you see as the disadvantages of Skolelinux / Debian Edu? For these reasons and experience from our project, I would now rather consider using plain Debian for schools next time, until Skolelinux is more closely integrated into Debian and becomes upgradeable without reinstallation. Which free software do you use daily? GNU/Linux with LXDE desktop, bash for interactive dialog and programming, texlive for documentation and correspondence, occasionally LibreOffice for document format conversion. Various programming languages for teaching. Which strategy do you believe is the right one to use to get schools to use free software? Strong arguments are

5 December 2013

Daniel Kahn Gillmor: The legal utility of deniability in secure chat

This Monday, I attended a workshop on Multi-party Off the Record Messaging and Deniability hosted by the Calyx Institute. The discussion was a combination of legal and technical people, looking at how the characteristics of this particular technology affect (or do not affect) the law. This is a report-back, since I know other people wanted to attend. I'm not a lawyer, but I develop software to improve communications security, I care about these questions, and I want other people to be aware of the discussion. I hope I did not misrepresent anything below. I'd be happy if anyone wants to offer corrections. BackgroundOff the Record Messaging (OTR) is a way to secure instant messaging (e.g. jabber/XMPP, gChat, AIM). The two most common characteristics people want from a secure instant messaging program are:
Authentication
Each participant should be able to know specifically who the other parties are on the chat.
Confidentiality
The content of the messages should only be intelligible to the parties involved with the chat; it should appear opaque or encrypted to anyone else listening in. Note that confidentiality effectively depends on authentication -- if you don't know who you're talking to, you can't make sensible assertions about confidentiality.
As with many other modern networked encryption schemes, OTR relies on each user maintaining a long-lived "secret key", and publishing a corresponding "public key" for their peers to examine. These keys are critical for providing authentication (and by extension, for confidentiality). But OTR offers several interesting characteristics beyond the common two. Its most commonly cited characteristics are "forward secrecy" and "deniability".
Forward secrecy
Assuming the parties communicating are operating in good faith, forward secrecy offers protection against a special kind of adversary: one who logs the encrypted chat, and subsequently steals either party's long-term secret key. Without forward secrecy, such an adversary would be able to discover the content of the messages, violating the confidentiality characteristic. With forward secrecy, this adversary is be stymied and the messages remain confidential.
Deniability
Deniability only comes into play when one of the parties is no longer operating in good faith (e.g. their computer is compromised, or they are collaborating with an adversary). In this context, if Alice is chatting with Bob, she does not want Bob to be able to cryptographically prove to anyone else that she made any of the specific statements in the conversation. This is the focus of Monday's discussion. To be clear, this kind of deniability means Alice can correctly say "you have no cryptographic proof I said X", but it does not let her assert "here is cryptographic proof that I did not say X" (I can't think of any protocol that offers the latter assertion). The opposite of deniability is a cryptographic proof of origin, which usually runs something like "only someone with access to Alice's secret key could have said X."
The traditional two-party OTR protocol has offered both forward secrecy and deniability for years. But deniability in particular is a challenging characteristic to provide for group chat which is the domain of Multi-Party OTR (mpOTR). You can read some past discussion about the challenges of deniability in mpOTR (and why it's harder when there are more than two people chatting) from the otr-users mailing list. If you're not doing anything wrong... The discussion was well-anchored by a comment from another participant who cheekily asked "If you're not doing anything wrong, why do you need to hide your chat at all, let alone be able to deny it?" The general sense of the room was that we'd all heard this question many times, from many people. There are lots of problems with the ideas behind the question from many perspectives. But just from a legal perspective, there are at least two problems with the way this question is posed: In these situations, people confront real risk from the law. If we care about these people, we need to figure out if we can build systems to help them reduce that legal risk (of course we also need to fix broken laws, and the legal environment in general, but those approaches were out of scope for this discussion). The Legal Utility of Deniability Monday's meeting was called specifically because it wasn't clear how much real-world usefulness there is in the "deniability" characteristic, and whether this feature is worth the development effort and implementation tradeoffs required. In particular, the group was interested in deniability's utility in legal contexts; many (most?) people in the room were lawyers, and it's also not clear that deniability has much utility outside of a formal legal setting. If your adversary isn't constrained by some rule of law, they probably won't care at all whether there is a cryptographic proof or not that you wrote a particular message (In retrospect, one possible exception is exposure in the media, but we did not discuss that scenario). Places of possible usefulness So where might deniability come in handy during civil litigation or a criminal trial? Presumably the circumstance is that a piece of a chat log is offered as incriminating evidence, and the defendant is trying to deny something that they appear to have said in the log. This denial could take place in two rather different contexts: during rules over admissibility of evidence, or (once admitted) in front of a jury. In legal wrangling over admissibility, apparently a lot of horse-trading can go on -- each side concedes some things in exchange for the other side conceding other things. It appears that cryptographic proof of origin (that is, a lack of deniability) on the chat logs themselves might reduce the amount of leverage a defense lawyer can get from conceding or arguing strongly over that piece of evidence. For example, if the chain of custody of a chat transcript is fuzzy (i.e. the transcript could have been mishandled or modified somehow before reaching trial), then a cryptographic proof of origin would make it much harder for the defense to contest the chat transcript on the grounds of tampering. Deniability would give the defense more bargaining power. In arguing about already-admitted evidence before a jury, deniability in this sense seems like a job for expert witnesses, who would need to convince the jury of their interpretation of the data. There was a lot of skepticism in the room over this, both around the possibility of most jurors really understanding what OTR's claim of deniability actually means, and on jurors' ability to distinguish this argument from a bogus argument presented by an opposing expert witness who is willing to lie about the nature of the protocol (or who misunderstands it and passes on their misunderstanding to the jury). The complexity of the tech systems involved in a data-heavy prosecution or civil litigation are themselves opportunities for lawyers to argue (and experts to weigh in) on the general reliability of these systems. Sifting through the quantities of data available and ensuring that the appropriate evidence is actually findable, relevant, and suitably preserved for the jury's inspection is a hard and complicated job, with room for error. OTR's deniability might be one more element in a multi-pronged attack on these data systems. These are the most compelling arguments for the legal utility of deniability that I took away from the discussion. I confess that they don't seem particularly strong to me, though some level of "avoiding a weaker position when horse-trading" resonates with me. What about the arguments against its utility? Limitations The most basic argument against OTR's deniability is that courts don't care about cryptographic proof for digital evidence. People are convicted or lose civil cases based on unsigned electronic communications (e.g. normal e-mail, plain chat logs) all the time. OTR's deniability doesn't provide any legal cover stronger than trying to claim you didn't write a given e-mail that appears to have originated from your account. As someone who understands the forgeability of e-mail, i find this overall situation troubling, but it seems to be where we are. Worse, OTR's deniability doesn't cover whether you had a conversation, just what you said in that conversation. That is, Bob can still cryptographically prove to an adversary (or before a judge or jury) that he had a communication with someone controlling Alice's secret key (which is probably Alice); he just can't prove that Alice herself said any particular part of the conversation he produces. Additionally, there are runtime tradeoffs depending on how the protocol manages to achieve these features. For example, forward secrecy itself requires an additional round trip or two when compared to authenticated, encrypted communications without forward secrecy (a "round trip" is a message from Alice to Bob followed by a message back from Bob to Alice). Getting proper deniability into the mpOTR spec might incur extra latency (imagine having to wait 60 seconds after everyone joins before starting a group chat, or a pause in the chat of 15 seconds when a new member joins) or extra computational power (meaning that they might not work well on slower/older devices) or an order of magnitude more bandwidth (meaning that chat might not work at all on a weak connection). There could also simply be complexity that makes it harder to correctly implement a protocol with deniability than an alternate protocol without deniability. Incorrectly-implemented software can put its users at risk. I don't know enough about the current state of mpOTR to know what the specific tradeoffs are for the deniability feature, but it's clear there will be some. Who decides whether the tradeoffs are worth the feature? Other kinds of deniability Further weakening the case for the legal utility of OTR's deniability, there seem to be other ways to get deniability in a legal context over a chat transcript. There are deniability arguments that can be made from outside the protocol. For example, you can always claim someone else took control of your computer while you were asleep or using the bathroom or eating dinner, or you can claim that your computer had a virus that exported your secret key and it must have been used by someone else. If you're desperate enough to sacrifice your digital identity, you could arrange to have your secret key published, at which point anyone can make signed statements with it. Having forward secrecy makes it possible to expose your secret key without exposing the content of your past communications to any listener who happened to log them. Conclusion My takeaway from the discussion is that the legal utility of OTR's deniability is non-zero, but quite low; and that development energy focused on deniability is probably only justified if there are very few costs associated with it. Several folks pointed out that most communications-security tools are too complicated or inconvenient to use for normal people. If we have limited development energy to spend on securing instant messaging, usability and ubiquity would be a better focus than this form of deniability. Secure chat systems that take too long to make, that are too complex, or that are too cumbersome are not going to be adopted. But this doesn't mean people won't chat at all -- they'll just use cleartext chat, or maybe they'll use supposedly "secure" protocols with even worse properties: for example, without proper end-to-end authentication (permitting spoofing or impersonation by the server operator or potentially by anyone else); with encryption that is reversible by the chatroom operator or flawed enough to be reversed by any listener with a powerful computer; without forward secrecy; or so on. As a demonstration of this, we heard some lawyers in the room admit to using Skype to talk with their clients even though they know it's not a safe communications channel because their clients' adversaries might have access to the skype messaging system itself. My conclusion from the meeting is that there are a few particular situations where deniability could be useful legally, but that overall, it is not where we as a community should be spending our development energy. Perhaps in some future world where all communications are already authenticated, encrypted, and forward-secret by default, we can look into improving our protocols to provide this characteristic, but for now, we really need to work on usability, popularization, and wide deployment. Thanks Many thanks to Nick Merrill for organizing the discussion, to Shayana Kadidal and Stanley Cohen for providing a wealth of legal insight and legal experience, to Tom Ritter for an excellent presentation of the technical details, and to everyone in the group who participated in the interesting and lively discussion. Tags: chat, deniability, otr, security

3 December 2013

Iustin Pop: A not so dark sky

I'm aware of how not-dark our modern city night sky is, but sometimes it still heavily surprises me how full of light it actually is. Yesterday I went about five kilometres out of Z rich; I thought at that distance, on a reasonably dark hill, it would be good enough for some night-sky shots. So I setup my tripod, only to realise I can't expose over 30 seconds because (at low ISO and fast aperture, shooting not straight up), everything is bleached out: I couldn't believe my eyes. Yes, I heard that one needs to go 100Km out of big cities, but but So I went and found that since about 2004, all of Switzerland is light polluted - even going on top of Jungfrau, for example, wouldn't completely eliminate it. I also learned about Merle Walker's equation - yeah, 200Km or more away from light sources would be good. Not here, in this quite small, quite populated country in the middle of Europe :/ I realised that as soon as my eyes got acclimated to the location I was at, I could easily see everything around me, due to the light pollution. I miss a really dark sky - I remember a couple of years back, in a different country, stopping at night on the side of the road, and being shocked at how the sky was filled with starts. I didn't have any camera with me at that time Heck, I remember seeing the Milky Way way back as a child, but nowadays when I get out of work, I can barely see 4-5 points of light in the sky. Anyway, enough with ranting Thanks to modern technology, one can recover lots of detail, even in a washed out picture. So in the end, shooting straight up, I could get some resemblance of structure (and this was only at 28mm, which is not that wide): Or alternatively, one can use the glowing light for a bit of play/contrast: I also played with stacking images via Deep Sky Stacker, but stacking - I learned to my surprise - only works to reduce noise in the final image, and not to make it "brighter" or more detailed. Live and learn The result was a bit better than not stacked, but not by much: This was a 25 5s, ISO 800, f/4, same 28mm lens. What I found surprising is the "non-star object" (to call it so) near the centre of the image - I have no idea what it is, it definitely doesn't look like a star, it could be an elliptical galaxy or so. I tried navigating back in time via Stellarium, but I don't remember the orientation of the lens, so it will remain a mystery to me. Anyway, two more pictures and higher resolutions on a Smugmug album. Feel free to leave a comment or drop me an email if you have suggestions where to take nice night sky photos in Switzerland

Dirk Eddelbuettel: digest 0.6.4

digest version 0.6.4 is now on CRAN and in Debian. This is a pure maintenance release which should help with a build issue affecting users on Solaris. CRANberries provides the usual summary of changes to version 0.6.3. Our package is available via the R-Forge page leading to svn and tarball access, my digest page, the local directory here as well as via Debian and its mirrors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Russell Coker: Preferring Not To

I ve just read Bartleby the Scrivener which is a short story about a scrivener who refused to work saying I d prefer not to . It reminded me of some situations in the computer industry. I ve never seen a single case where someone preferred not to work when everyone around them (colleagues and management) wanted them to work. But then the incidence of having an entire team and management wanting to work efficiently isn t nearly as common as one might imagine. In some cases it s desired that someone not work, such as a former colleague who was hired as a sysadmin but did nothing but change backup tapes (a few hours work per week). Not having him login as root improved the general reliability of the servers but it was fortunate that we never needed to restore from backups One time I had a colleague who preferred to spend most of his time in the office searching the Internet for videos of street fights. I have often told colleagues that I would prefer them to work, but in the case of a guy who s only hobby is street-fighting I decided to let it go. Managing people can be difficult, particularly for someone who doesn t like disagreements. Some managers that I ve reported to seemed to prefer not to manage in an apparent attempt to avoid disputes. One time when I complained about a colleague not even having a suitable computer to permit doing any work a manager responded with the rhetorical question what do you expect me to do? . That manager didn t do any annual reviews of staff for over a year, he only eventually did some reviews because he was told that his scheduled promotion was on hold until he got them done. I got the impression that at least two levels of management preferred not to work at that company. Sometimes it just gets weird though, such as the occasion when I was the only member of a team and the manager who was supposedly managing no-one but me never seemed to have time to have a meeting with me. But he didn t want me to bypass him and talk directly to other people in the company, so he preferred not to work and not to have anyone else do his job. Most of the companies that I ve worked for in a full-time capacity didn t seem to have any effective technical interviews (note that I ve mostly worked for financial companies and ISPs not free software companies). So it seems that anyone with minimal computer skills who wants a well paying job could just send out a CV to a bunch of recruiting agencies, get interviewed by enough companies to eventually hit one without a technical interview process and then find a job that doesn t require work. Depression The Wikipedia page about Bartleby the Scrivener [2] suggests that Bartleby was depressed. I wonder how much of the lack of performance I ve witnessed has been due to depression. There appears to be a strong correlation between work environments that cause depression and people preferring not to work. Maybe managers should be considering how to make work less depressing to try and get more effective employees (in terms of quality and quantity of work). One example of this is the sysadmin team death spiral I ve witnessed where no-one can automate solving problems (EG by cron jobs to manage resource usage and analysis tools to find minor problems before they become major problems) because everyone is dedicated to fixing things that break needlessly (EG systems crashing due to lack of disk space). When people start getting control over recurring problems and automating things then the work becomes increasingly about solving problems and less about implementing the same manual processes every day/week and it s more fun and effective for everyone. At the BoF on depression at LCA 2013 one delegate stated that many companies have people in HR who can arrange support for depressed employees. Apparently if you are depressed and you work for a company that s large enough to have a HR department then it can be beneficial to talk to HR about it. That probably works well in the case where an employee is depressed but the company is working well. But in the case where the company isn t working well it seems unlikely to help. David Graeber wrote an interesting article about Bullshit Jobs [3]. He goes a bit far, I don t think that late night pizza delivery is a bullshit job and actuaries are useful to society. But his points about the existence of useless jobs are reasonable. Management Levels I sometimes wonder whether there is some benefit in establishing social norms about working and then having management take little interest in how it happens. If a team works well together then management could just set deadlines (which would be negotiated with employees who know what s possible) and let the team work out how to do it. Then instead of having one manager for each team of ~10 people who theoretically tracks what everyone is doing you could have one manager for a dozen teams who just tracks overall team performance essentially remove a layer of management. Valve is famous for having no formal management structure and for getting things done, unfortunately that apparently allows school-style cliques to block actions [4]. But I think that the Valve experiment is useful and provides some ideas that can be used by other companies. Maybe if instead of requiring consensus of the entire company for hiring decisions they only required consensus of the team things would have worked better. Of course another down-side to such things is that hierarchical management can be good for avoiding discrimination and bullying. The article I cited about Valve compares it to high-school. It could be that Valve employees were all nice people who only hired other nice people. But if similar systems were implemented in many companies then some would surely end up being like a typical high school with all the bullying and mistreatment of minority groups that entails. Michael O. Church wrote an interesting article in which he divides employees into four categories, loser , clueless , psychopaths , and technocrats (note that he didn t invent the first three names) [5]. In his model the clueless category includes most middle-management. I think that there are some problems with Michael s model and I m not arguing for a technocracy (which is how this post might be interpreted in terms of his ideas). But I think he demonstrates some of the real problems in the way companies are managed and in his model the losers prefer not to work as long as they can get paid. Conclusion I don t have any good solutions to these problems to offer. It seems that the best we can hope for is incremental change to make work less depressing, to have the minimal amount of management, and to avoid bullshit jobs .

2 December 2013

Tim Retout: Questhub.io

At the London Perl Workshop last Saturday, one of the lightning talks was about Questhub.io, formerly known as "play-perl.org". It's social gamification for your task list, or something like that. Buzzword-tastic! But most importantly, there seems to be a nice community of programming types to procrastinate with you on your quests. This means I can finally get to work refuting lamby's prediction about gamification of Debian development! Tasks are referred to as "Quests", and are pursued in themed "Realms", for that World of Warcraft feeling. For example, there's a "Perl" realm, and a "Lisp" realm, and a "Haskell" realm, but also non-programming realms like "Fitness" and "Japanese". Of course, part of me now wants to construct a federated version which can be self-hosted. :) Another downside of questhub currently is the lack of SSL support - your session cookies are sent in plain text. I hope this changes soon.

Thorsten Glaser: Neo900

I ve did something I surely will (financially) regret, next year, and designated the Neo900 to be the successor to my PocketPC, due to the latter having only 64 MiB RAM and Geocaching applications being quite hungry. It s got a lovely hardware keyboard, a pen display like the PocketPC (as opposed to the wishy-washy displays that Android and iPhone have), not only GPS but also GLONASS, fully free software with mostly free firmware (I m okay with that, mostly), a Ctrl key (useful in ssh and locally and my text editor; ^I is Tab, so it s useful in shell, too), WLAN, UMTS (I don t think I need LTE and would rather it have the more RAM), USB host (OTG), and lots of other nice features. In short, it s a tinkerable device: one I can not only hack at, but also hack on. Since I use a dumbphone for mobile phone anyway (pro: separate battery from the toy PocketPC/Smartphone we re talking two+ weeks of battery time when using it here, and easier use and less bugs, and a reliable fallback when I tinker too much ), this is perfect for me. I m reposting this in the wlog mostly because it s an interesting technical and OSS project, and because if 1000 people want one it will get less expensive for all of us (while here shameless plug any sponsors willing to contribute some EUR so I don t ruin myself with this, in exchange for services of some kind?). I ll probably run Debian on it (unless it goes systemd), maybe in a chroot if the native OS has functionality needed that I can t simply put into packages; they say Maemo has much better power management, but considering most use will have GPS, GLONASS and backlight on, battery isn t going to last long anyway or maybe even native I ve been wanting to know what this freesmartphone stuff my m68k (Atari VM) buildd has been happily compiling, anyway and some sort of Geocaching application (ideally a cross between something online, CacheWolf and an offline OSM (with most of Europe, but uninteresting tags stripped) and possibly access to the GS Live API but nevertheless supporting TC, NC, OC, gpsgames too), and my usual mksh(1), GNU screen, jupp(1), lynx(1), ssh(1) toolchain.) Delivery is expected for mid to end of 2014, but once it s there I ll keep you informed

Dirk Eddelbuettel: Recent Rcpp talks at U Chicago / Booth and U Kansas

In early October, I had an opportunity to talk about Rcpp and RcppArmadillo at the Statistical Computing Seminar at the Booth School of Business at the University of Chicago. And then two weeks ago, I had an invitation to talk at the Center for Research Methods and Data Analysis at the University of Kansas where I covered similar material as well as ongoing work on the RcppZiggurat package (for which I should have an updated version soon). Slides from both talks are now at the top of my talks / presentations page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Ben Hutchings: Upgrading from Android 2.3 'Gingerbread' to 4.3 'Jelly Bean'

Replacing my phone My first Android phone was a ZTE Blade (sold as Orange San Francisco here in the UK). It originally shipped with Android 2.1 but was upgradable to 2.3 thanks to CyanogenMod (or other unofficial mods). However there's little sign of an upgrade to 4.x; the apps I want to use are pushing the limits of its CPU, RAM and internal storage; and I've never got very good at typing on a soft keyboard. So it seemed like time to get a newer phone with a hard keyboard (and still with a SD slot). After a little research with the CyanogenMod compatibility list and some time looking at reviews, I settled on the Samsung Galaxy S Relay 4G, which was exclusive to T-Mobile USA but is being resold through eBay. It shipped with Android 4.0 but I immediately installed CyanogenMod 10.2 (Android 4.3). This was a bit of an adventure as I bought the Blade with CM already installed and wasn't familiar with the multiple steps that were necessary. Copying my data The last step, and the real subject of this entry, was to move my data across. Much of this was on a SD card which I could simply plug into the Relay. The internal files had to be backed up onto this card and then restored, using the recovery environment (ClockworkMod) on each phone. After rebooting the Relay into the full Android system, my apps and settings were mostly present but I was immediately confronted with a series of error dialogs reporting that 'Unfortunately, Dialler has stopped' - and the same for 'Clock' and 'the process android.process.acore' (whatever that is). What went wrong There are thankfully more detailed error logs in the filesystem, under /data/system/dropbox (this is for logging at the Java level; if a process is terminated by a fatal signal it's logged to /data/tombstones). There are several ways to get at them, but I used recovery mode and adb to get a root shell where I could easily read and write files as necessary. The Contacts (aka People) database upgrade fails with an SQLiteException, apparently because it doesn't account for upgrading from the schema used in 2.3:
Process: android.process.acore
Flags: 0x883e45
Package: com.android.providers.applications v18 (4.3.1-f963020e38)
Package: com.android.providers.contacts v18 (4.3.1-f963020e38)
Package: com.android.providers.userdictionary v18 (4.3.1-f963020e38)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: phonebook_label (code 1): , while compiling: UPDATE raw_contacts SET display_name_source=?,display_name=?,display_name_alt=?,phonetic_name=?,phonetic_name_style=?,sort_key=?,phonebook_label=?,phonebook_bucket=?,sort_key_alt=?,phonebook_label_alt=?,phonebook_bucket_alt=? WHERE _id=?
...
        at com.android.providers.contacts.ContactsDatabaseHelper.updateRawContactDisplayName(ContactsDatabaseHelper.java:5344)
        at com.android.providers.contacts.ContactsDatabaseHelper.upgradeToVersion504(ContactsDatabaseHelper.java:3580)
        at com.android.providers.contacts.ContactsDatabaseHelper.onUpgrade(ContactsDatabaseHelper.java:2261)
...
Clock fails somewhat similarly though it apparently didn't try to upgrade:
Process: com.android.deskclock
Flags: 0xc8be45
Package: com.android.deskclock v203 (2.0.3)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: incvol (code 1): , while compiling: SELECT _id, hour, minutes, daysofweek, alarmtime, enabled, vibrate, message, alert, incvol, profile FROM alarms WHERE (enabled=1)
...
        at com.android.deskclock.AlarmProvider.query(AlarmProvider.java:73)
        at android.content.ContentProvider.query(ContentProvider.java:744)
        at android.content.ContentProvider$Transport.query(ContentProvider.java:199)
        at android.content.ContentResolver.query(ContentResolver.java:414)
        at android.content.ContentResolver.query(ContentResolver.java:357)
        at com.android.deskclock.Alarms.getFilteredAlarmsCursor(Alarms.java:165)
        at com.android.deskclock.Alarms.calculateNextAlert(Alarms.java:344)
        at com.android.deskclock.Alarms.setNextAlert(Alarms.java:417)
        at com.android.deskclock.Alarms.saveSnoozeAlert(Alarms.java:510)
        at com.android.deskclock.AlarmInitReceiver$1.run(AlarmInitReceiver.java:49)
...
The media player fails with a NullPointerException:
Process: com.andrew.apollo:main
Flags: 0x98be65
Package: com.andrew.apollo v2 (1.1)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
java.lang.NullPointerException
        at com.andrew.apollo.MusicPlaybackService.stop(MusicPlaybackService.java:878)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndMaybeNext(MusicPlaybackService.java:1048)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndNext(MusicPlaybackService.java:1031)
        at com.andrew.apollo.MusicPlaybackService.access$1500(MusicPlaybackService.java:74)
        at com.andrew.apollo.MusicPlaybackService$MusicPlayerHandler.handleMessage(MusicPlaybackService.java:2337)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:137)
        at android.os.HandlerThread.run(HandlerThread.java:61)
Fixing the problem Maybe I could have worked out how to upgrade the relevant databases myself, but I went for simpler solutions. The clock settings are easy to re-enter and most of the media player state is regenerated by scanning files on the SD card. So I just deleted those on the Relay:
  rm -rf /data/data/com.android.deskclock
  rm -rf /data/data/com.android.providers.media
The contacts were what I really cared about, and there are actually specific menu items in Contacts to export and import those (using VCF format), side-stepping the database upgrade. So I did:
  1. Insert SD card in Blade
  2. Open Contacts and export to VCF (I forget where this is in the menus but it was easy to find)
  3. Remove broken Contacts database on Relay:
    rm -rf /data/data/com.android.providers.contacts
  4. Insert SD card in Relay
  5. Move exported contacts to internal storage:
    mv /storage/sdcard1/*.vcf /storage/emulated/legacy
  6. Open People and tap the menu key, 'Import/export', 'Import from storage', then the filename
This may not include all data, and in particular it doesn't seem to include attached photos. But that was good enough for me.

1 December 2013

Raphaël Hertzog: My Free Software Activities in November 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (44.52 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook Wheezy update completed. Roland and I completed the update of the Debian Administrator s Handbook for Debian 7 Wheezy. We still have some proofreading work to do but you can already enjoy the result here: http://debian-handbook.info/browse/wheezy/ Feel free to report back any problem that you discover. You can also submit us patches ready to apply if you want to go one step further. Publican contributions. The book is generated with publican and I maintain its Debian package. This month I got a release critical bug because it stopped working it turns out that the problem lied in libxml-treebuilder-perl and I thus reassigned #728885 while providing a tentative patch to the upstream author. After a few days without action from the pkg-perl team, and after having received a FTBFS bug on debian-handbook (of course publican was broken in unstable!), I prepared a fixed package myself and I uploaded it (I m still part of the pkg-perl team although I m inactive). Since I used publican heavily this month, I filed two tickets in its bugzilla. I requested a new feature in #1034836 (the possibility to keep around the former string for fuzzy strings to update), and I reported a problem with the handling of \n in PO files in #1036150. Debian France Galette update. I updated the galette package and its paypal plugin, and I deployed those on france.debian.net. It had some fixes for the reminder mails sent to members. Bylaws update. I also resumed my work on preparing new bylaws for Debian France. Sylvestre Ledru came up with a draft (with the help of a lawyer) a few months ago and I m reviewing/improving them now. The main goal is to clarify that Debian France is meant to be a Trusted Organization for the Debian project. Debian France Shop. We had the idea since a few months already but Sylvestre did the leg work to open a Debian France shop with the help of EnVenteLibre. I asked our members to prepare some CSS that better match the Debian colors and this should be fixed in a few days. The first goodies will also start to appear shortly, just in time for Christmas! Misc Debian work Distro Tracker. In the continuation of the Google Summer of Code, I asked the DSA team to setup a new virtual machine to host tracker.debian.org, an instance of Distro Tracker, the rewritten Package Tracking System. They have done their part of the job (except the mail setup), it s now waiting on me to find some time to complete some cleanups and deploy the thing. WordPress. I packaged wordpress 3.7.1 and sent a call for help on debian-mentors. I got 3 replies, I gave them some initial direction but I haven t heard back anything since. WordPress 3.8 is expected in a few days, hopefully one of the new volunteers will take care of preparing the next update. Dpkg regressions. I haven t done anything for multiple months but at least I keep running the git version of dpkg and I detected two regressions. Good to have them squashed before the upcoming 1.17.2 upload to unstable. PTS fix. I fixed some warnings that the PTS code started generating since the upgrade of its host to wheezy. They were generating some annoying backscatter mails to users of the pts@qa.debian.org bot. Ruby security update. I helped the ruby team to prepare the required security updates of ruby1.8 and ruby1.9.1 (see #730178 and #730189). This work was sponsored by Kali/Offensive Security. Smartcard setup. I bought 2 OpenPGP smartcards with a reader and I moved all my private keys on those devices (one card with the master key for signature/certification to be kept at home, one card for daily/mobile usage with the subkeys for encryption/signature/authentication). My laptop s harddrive doesn t contain any private key anymore. I have kept the required offline backup in a safe place, but in the end, my private keys are much harder to steal. I should write down my findings in another article Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

30 November 2013

Dirk Eddelbuettel: RcppCNPy 0.2.2

Right on the heels of release 0.2.1 of RcppCNPy, a new version 0.2.2 is now on CRAN. RcppCNPy uses the CNPY library by Carl Rogers to provide R with easy read and write access to NumPy files. The reason for the new version that I had experimented with a different way to test endianness (as needed for the NumPy file headers) but accidentally sent an interim tarball to CRAN which still wanted to include endian.h promptly breaking Windows builds. So now we do something even simpler and just rely on the (even more complete) test for endianness when R is built, which prevents all sorts of complications for us and builds everywhere (with thanks to Brian Ripley for the suggestion). While we were at it, we also added a new unit test. Full changes are listed below.
Changes in version 0.2.2 (2013-11-29)
  • Switched to using the result from the compile-time configuration for R to determine big or little endian (as needed for the NPy headers)
  • Added a new test (and test validation result file) for a complete save-reload cycle and comparison
CRANberries also provides a diffstat report for 0.2.2 relative to 0.2.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Petter Reinholdtsen: Dugnadsnett for alle, a wireless community network in Oslo, take shape

If you want the ability to electronically communicate directly with your neighbors and friends using a network controlled by your peers in stead of centrally controlled by a few corporations, or would like to experiment with interesting network technology, the Dugnasnett for alle i Oslo might be project for you. 39 mesh nodes are currently being planned, in the freshly started initiative from NUUG and Hackeriet to create a wireless community network. The work is inspired by Freifunk, Athens Wireless Metropolitan Network, Roofnet and other successful mesh networks around the globe. Two days ago we held a workshop to try to get people started on setting up their own mesh node, and there we decided to create a new mailing list dugnadsnett (at) nuug.no and IRC channel #dugnadsnett.no to coordinate the work. See also the NUUG blog post announcing the mailing list and IRC channel.

28 November 2013

Dirk Eddelbuettel: RcppCNPy 0.2.1

Version 0.2.1 of RcppCNPy is now on CRAN earlier. This version is a minor upgrade with a few small changes as detailed in the NEWS file:
Changes in version 0.2.1 (2013-11-28)
  • Synchronized code with the cnpy repository
  • Added new function to test from R whether integers supported
  • Updated tests for integer support, if available
  • Updated vignette and discussion about need for rebuilding only RcppCNPy with the -std=c++11 flag if integer support is desired
  • Updated tests for integer support, if available
  • Updated THANKS file
  • Ensure that inclusion of Rinternal.h does not affect other headers by adding RF_NO_REMAP
CRANberries also provides a diffstat report for 0.2.1 relative to 0.2.0. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

27 November 2013

Michael Stapelberg: let s disable pdiffs already!

Richi s post about the pdiff-by-default agony resonates with me a lot. On EVERY Debian installation I have ever done in the last few years, without any exceptions, I have turned off pdiffs. Even on all the oddball cases (Raspberry Pi, account on a remote machine, ) where I don t run my install-configs script, I have ended up turning off pdiffs eventually, because it is just so insanely slow on modern internet connections. And by modern I mean even the DSL link my parents place has since 14 years. Let s disable pdiffs by default already.

Next.

Previous.