I m currently running the problem class for the graph theory course at the Karlsruhe Institute of Technology, held by Prof. Maria Axenovich. On one problem sheet, I felt like setting the question in verse, and indeed, two students submitted their solution is in verse form as well. I have assembled the problem and both solution enjoy! <?$t1 = "1244431010010381771";$t2 = "1244431010010381772";if ($t1 == $t2)print "equal\n";?>
A few months ago, I had blogged about using R inside of Qt.
This used our RInside package for embedding
the statistical programming environment and language R
inside of a C++ application, and further relies on our Rcpp package for R and C++
integration and object mapping.
The example was simple yet powerful: a reimplementation of the standard GUI application of a standard density estimate. Here the user can
pick a kernel density function from a selection, and also slide a bandwidth parameter. One nice addition was an entry field already populated
with a simple expression for a mixture of Normals, allowing for arbitrary random distributions over which to estimate. The example is
pretty (thanks to Qt), and was added to RInside with the last CRAN release 0.2.4.
The blog post has a nice screenshot.
I had long wondered how to do something similar 'on the web'. Web integration and application frameworks are of course a dime a dozen: Just
about any language offers this, with more or less ease. But I wanted something simple yet powerful and fast. And
I did not like the idea of a multi-tier app, or of a multi-language mix. I remember having seen something about a web-application framework
not unlike Qt, and studying the very useful Wikipedia web
application framework comparison I re-discovered Wt (pronounced "Witty"). So there it is,
using C++ which brings us ample performance, the ability to connect to a number of libraries and applications (which is important in my
quantitatively-minded workd) and avoid the whole multi-tier, multi-language combination. The Wt
website has a few more good reason why this may be a suitable idea; the toolkit also offers a very decent amount of features and is amply
documented with a fair number of examples.
And after just a little bit poking around during two weekends, I now have the following webapp committed
in the SVN repository of RInside, and it is all implemented in
in less than two hundred (generously commented) lines of code.
It is currently up and running at the address shown in the screenshot, so give it a go (though I may take it down at another point in
time). I quite like it: The application is responsive: changes in the radio buttons (for the density), or the bandwidth, each trigger
reestimation of the density, and a new and updated chart is displayed immediately with no noticeable delay---just like the desktop application did.
Best of all, the code logic is essentially unchanged from the Qt-based app. Signals and slots related events to actions, the layout is in
terms of standard GUI boxen and containers. And best of all, I did not have to write a line of html, javascript, css or ajax: it is all
handled by the Wt toolkit. I was able to drive the app from an Android phone, my tablet, various
computers around the house, and had a few friends poke a stick at it from afar.
There is at one open issue. Wt launches new instances of the application object with each
connection, which is a very clean model. That doesn't map perfectly with R (which is single-threaded) and
RInside (which runs as a singleton for the same reason). So right now, each
action sends its state back to the client. In other words, each clients own its parameters and well as vector of random numbers. Each new
action sends these back to the app which passes it to R, launches the re-estimation and gets an updated chart back which is then shown by the
the client. That is not perfect, and maybe a forking model as used by Simon's RServe would be
better, though it would require a rewrite of RInside. Not sure if we get there
anytime soon. And for simple applications not facing legions of concurrent users, the singleton should still work. It's a proof
of concept at this point. Feedback welcome, and RInside and
Rcpp questions should go to the rcpp-devel list as usual.
If you want to copy debian-installer for System z onto a z/VM user's CMS disk, you don't need access to FTP (and hence the host's TCP/IP stack). You can just use x3270 and transfer files with it. For odd reasons I forgot about this, so let's document it here:
My current running Xorg server, built with -O0 -g3 -ggdb, having been
through a few suspend cycles, hosting a full GNOME Shell session with
several billion tabs in a long-running Chromium instance, a couple of
terminals, and a freshly-opened gedit, is currently sitting at 20MB
resident memory usage, of which 11MB is shared. The freshly-opened
gedit instance has 25MB resident, of which 17MB is shared. Bloat is
in the eye of the beholder, I guess.
Conrad Sanderson
just released the second pre-release 2.3.92 of what will be
Armadillo 2.4.*. This is now in
RcppArmadillo
release 0.2.31 which is already on CRAN
as of this morning.
The NEWS entries summarising the changes for both are below:
0.2.31 2011-11-28
o Upgraded to Armadillo test release 2.3.92 "Loco Lounge Lizard (Beta 2)"
* fixes for linking on Ubuntu and Debian systems
* fixes for inconsistencies in interactions between matrices and
cubes
Courtesy of
CRANberries, there
is also a diffstat reports
for 0.2.31 relative to 0.2.30
As always, more detailed information is on the
RcppArmadillo page.
Questions, comments etc should go to the
rcpp-devel mailing list
off the R-Forge page.
AndroidZoom, along with just about every other third-party interface
to the Android Market out there, provides 2D barcodes which aim to
make it easy to install Android applications that you find online on
a phone. Maybe this would be a nice feature for F-Droid?
Unfortunately, I found this feature when I was trying to help a friend
install the (free software) ZXing Barcode Scanner because they
wanted to read a 2D barcode.
I wrote a quick proof-of-concept script that can traverse my email (in local
Maildir format) and detach all attachments: saving them to another location
and replacing them with a text attachment which documents where they've gone.
This reduces my work mailbox size substantially and allows for more
space-efficient storage of the attachments (utilising all 8 bits of the byte,
at the very least, but with the potential for compression to be added to the
mix). It maintains the link between email and attachment, however, which is
crucial if the method of discovery of the attachment in the future is to find
the email.
For this to be future proof, the path where the mail is saved needs to be. The
best scheme I can think of involves dedicating a sub-domain to the problem and
using a URI scheme which is guaranteed1 to be unique for the
attachment, such as a hash-sum:
https://file.example.com/sha1/509c2fe2eba509e93987c3024a74d74583c274bd
For a while now my fellow Collaboran Joone Hur has been working on implementing the Accelerated Compositing infrastructure available in WebKit in webkit-clutter, so that we can use Clutter s powers for compositing separate layers and perform animations. This work is being done by Collabora and is sponsored by BOSCH, whom I d like to thank! What does all this mean, you ask? Let me tell me a bit about it.
The way animations usually work in WebKit is by repainting parts of the page every few milliseconds. What that means in technical terms is that an area of the page gets invalidated, and since the whole page is one big image, all of the pieces that are in that part of the page have to be repainted: the background, any divs, images, text that are at that part of the page.
What the accelerated compositing code paths allow is the creation of separate pieces to represent some of the layers, allowing the composition to happen on the GPU, removing the need to perform lots of cairo paint operations per second in many cases. So if we have a semi-transparent video moving around the page, we can have that video be a separate texture that is layered on top of the page, made transparent and animated by the GPU. In webkit-clutter s case this is done by having separate actors for each of the layers.
I have been looking at this code on and off, and recently joined Joone in the implementation of some of the pieces. The accelerated compositing infrastructure was originally built by Apple and is, for that reason, works in a way that is very similar to Core Animation. The code is still a bit over the place as we work on figuring out how to best translate the concepts into clutter concepts and there are several bugs, but some cool demos are already possible! Bellow you have one of the CSS3 demos that were made by Apple to demo this new functionality running on our MxLauncher test browser.
<video controls="controls" src="http://cafe.minaslivre.org/~kov/webkit/ac.webm" width="640"></video>
You can also see that the non-Accelerated version is unable to represent the 3D space correctly. Also, can you guess which of the two MxLauncher instances is spending less CPU? 
In this second video I show the debug borders being painted around the actors that were created to represent layers.
<video controls="controls" src="http://cafe.minaslivre.org/~kov/webkit/debug-borders.webm" width="640"></video>
The code, should you like to peek or test is available in the ac2 branch of our webkit-clutter repository: http://gitorious.org/webkit-clutter/webkit-clutter/commits/ac2
We still have plenty of work to do, so expect to hear more about it. During our annual hackfest in A Coru a we plan to discuss how this work could be integrated also in the WebKitGTK+ port, perhaps by taking advantage of clutter-gtk, which would benefit both ports, by sharing code and maintenance, and providing this great functionality to Epiphany users. Stay tuned!
Once the private key of some HTTPS web site is compromised, an
attacker is able to build a man-in-the-middle attack to
intercept and decrypt any communication with the web site. The first
step against such an attack is the revocation of the associated
certificate through a CRL or a protocol like OCSP. Unfortunately, the
attacker could also have recorded past communications protected by
this private key and therefore decrypt them.
Forward secrecy allows today information to be kept secret even if
the private key is compromised in the future. Achieving this property
is usually costly and therefore, most web servers do not enable it on
purpose. Google recently
announced support of forward secrecy on their HTTPS
sites. Adam Langley
wrote a post with more details on what was achieved to increase
efficiency of such a mechanism: with a few fellow people, he wrote an
efficient implementation of some elliptic curve cryptography for OpenSSL.
AES128-SHA. During this handshake, the server will present its
certificate and both the client and the server will agree on a master
secret.
This secret is built from a 48byte premaster secret generated and
encrypted by the client with the public key of the server. It is then
sent in a Client Key Exchange message to the server during the third
step of the TLS handshake. The master secret is derived from this
premaster secret and random values sent in clear-text with Client
Hello and Server Hello messages.
This scheme is secure as long as only the server is able to decrypt
the premaster secret (with its private key) sent by the
client. Let s suppose that an attacker records all exchanges between
the server and clients during a year. Two years later, the server is
decommissioned and sent for recycling. The attacker is able to recover
the hard drive with the private key. She can now decrypt any session
she recorded: the encrypted premaster secret sent by a client is
decrypted with the private key and the master secret is derived from
it. The attacker can now recover passwords and other sensitive
information that can still be valuable today.
The main problem lies in the fact that the private key is used for two
purposes: authentication of the server and encryption of a
shared secret. Authentication only matters while the communication is
established, but encryption is expected to last for years.
openssl
dhparam command):DHE-RSA-AES128-SHA use this
protocol to achieve perfect forward secrecy1.
To achieve a good level of security, parameters of the same size as
the key are usually used (the security provided by the discrete
logarithm problem is about the same as the security provided by
factorisation of two large prime numbers) and therefore, the
exponentiation operations are pretty slow as we can see in the
benchmark below:
ECDHE-RSA-AES128-SHA cipher suite (with P-256 for example) is
already a huge speed improvement over DHE-RSA-AES128-SHA thanks to
the reduced size of the various parameters involved.
Web browsers only support a handful of well-defined elliptic curves,
chosen to ease an efficient implementation. Bodo M ller, Emilia K sper
and Adam Langley have provided 64bit optimized versions of NIST P-224,
P-256 and P-521 for OpenSSL. To get even more details on the matter,
you can read the end of the
introduction on elliptic curves from Adam Langley, then a
short paper from Emilia K sper which presents a 64bit
optimized implementation of the NIST elliptic curve NIST P-224.
openssl ciphers ECDH
that your version supports them. If you want to use the 64bit
optimized version, you need to run a snapshot of OpenSSL 1.0.1,
configured with enable-ec_nistp_64_gcc_128 option. A recent GCC is
also required in this case.
Next, you need to choose the appropriate cipher suites. If
forward secrecy is an option for you, you can opt for
ECDHE-RSA-AES128-SHA:AES128-SHA:RC4-SHA cipher suites which should be
compatible with most browsers. If you really need forward secrecy, you
may opt for
ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:EDH-DSS-DES-CBC3-SHA
instead.
Then, you need to ensure the order of cipher suites is
respected. On nginx, this is done with ssl_prefer_server_ciphers
on. On Apache, this is SSLHonorCipherOrder on.
UPDATED: You need to check ECDHE support for your web
server. For nginx, the support has been added in 1.0.6 and
1.1.0. The curve selected defaults to NIST P-256. You can specify
another one with ssl_ecdh_curve directive. For Apache, it has been
added in 2.3.3 and does not exist in the current stable branch. Adding
support for ECDHE is quite easy. You can check
how I added it in stud. This issue also exists for DHE
cipher suites, in which case you also might have to specify DH
parameters to use (generated with openssl dhparam) using some
special directive or by appending the parameters to the
certificate. Check Immerda Techblog article for more
background about this point.
The implementation of TLS session tickets may be
incompatible with forward secrecy, depending on how they are
implemented. When they are protected by a random key generated at the
start of the server, the same key could be used for months. Some
implementations2 may derive the key from the private key. In
this case, forward secrecy is broken. If forward secrecy is a
requirement for you, you need to either disable tickets or ensure that
key rotation happens often.
Check with openssl s_client -tls1 -cipher ECDH -connect
127.0.0.1:443 that everything works as expected.
I have used a snapshot of OpenSSL 1.0.1 (2011/11/25). The
optimized version of ECDHE is the one you get by using
enable-ec_nistp_64_gcc_128 option when configuring OpenSSL.
Let s focus on the server part. Enabling DHE-RSA-AES128-SHA cipher
suite hinders the performance of TLS handshakes by a factor of
3. Using ECDHE-RSA-AES128-SHA instead only adds an overhead of
27%. However, if we use the 64bit optimized version, the cost is
only 15%. The overhead is only per full TLS handshake. If 3 out of 4
of your handshakes are resumed, you need to adjust the numbers.
Your mileage may vary but the computational cost for enabling perfect
forward secrecy with an ECDHE cipher suite seems a small sacrifice for
better security.
I've spent this weekend with my girlfriend in Berlin, which is one of the cities staying quite long on list of places to visit. Now I've been there, though there are still lot of reasons to come back in future.
We stayed with great hosts found through CouchSurfing and have seen hopefully all important places in the city. As usual, you can find some pictures in my gallery:
Filed under: English Life Travelling 0 comments Flattr this!
A while back I wrote some nodejs that mish-moshed email and web 2.0 long-poll action into a very ugly, but fairly up-to-date showing of debian changes. That is over on my server here.
The code s fairly simple, so I also wrote a python consumer here.
Since it s so simple, I ve pasted it below, too.
Feel free to use (but please not abuse) my server :)
#!/usr/bin/env python
import time
import urllib
import json
service = "http://pault.ag:8080/tail"
lastts = 0
def loop():
global lastts
while True:
f = urllib.urlopen("%s?since=%s" % (service, str(lastts)))
msgs = json.loads(f.read())
for x in msgs['messages']:
print x['text']
if lastts < x['timestamp']:
lastts = x['timestamp']
loop()
ganesh:~# zpool replace -f export scsi-SATA_WDC_WD10EARS-00_WD-WMAV50933036 scsi-SATA_ST31000528AS_9VP16X03and now it s just a matter of waiting until the data has resilvered and i can remove the old dying drive.
ganesh:~# zpool status export -v
pool: export
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Mon Nov 28 19:25:49 2011
75.1G scanned out of 2.29T at 88.3M/s, 7h17m to go
18.0G resilvered, 3.21% done
config:
NAME STATE READ WRITE CKSUM
export ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
scsi-SATA_WDC_WD10EACS-00_WD-WCASJ2114122 ONLINE 0 0 0
scsi-SATA_WDC_WD10EACS-00_WD-WCASJ2195141 ONLINE 0 0 0
scsi-SATA_WDC_WD10EARS-00_WD-WMAV50817803 ONLINE 0 0 0
replacing-3 ONLINE 0 0 0
scsi-SATA_WDC_WD10EARS-00_WD-WMAV50933036 ONLINE 0 0 0
scsi-SATA_ST31000528AS_9VP16X03 ONLINE 0 0 0 (resilvering)
logs
scsi-SATA_Patriot_Torqx_278BF0715010800025492-part6 ONLINE 0 0 0
cache
scsi-SATA_Patriot_Torqx_278BF0715010800025492-part7 ONLINE 0 0 0
errors: No known data errors
(I use /dev/disk/by-id device names so I don t have to care about the kernel detecting the disks and disk controllers in a different order on each reboot also makes it easier to identify which disk is having a problem as I can squint at the front of the drives in my Lian Li 4-in-3 hotswap bays and read the serial number sticker)
OK, yeah, sure it s not that much different from replacing a dying drive in an mdadm or hardware raid array but one nice advantage of zfs over raid is that it only resyncs the data in use, not the empty or unused blocks. given that these are 1TB drives in a raidz vdev, and the pool is 63% used that means syncing only about 630GB rather than 1TB.
ps: in the time it s taken to write this, it s up to 10.11% done. 88MB/s reading data from the existing raidz drives and writing to a single drive isn t too bad at all.
one of the reasons i like zfs is a post from: Errata
This week-end, after a few week-ends more targeted on real life
activities (country house fall works, 70km ultra run preparation and
ultra run week-end), I spent most of Sunday working on two important
uploads of packages I'm involved in.
So, as of Sunday evening, samba 3.6.1 is now in unstable. This is the
first 3.6 version to land there, after earlier versions were in
experimental while Steve Langasek was working on multi-arch support
and dh7-like debian/rules transition in unstable, with 3.5 versions.
Merging together changes in 3.5 and 3.6 branches has been a bit
painful. I guess I had it right (thankfully, I merged 3.5 packaging
changes from time to time to the 3.6 branch). If there are problems
with 3.6 packages, anyway, you know what to do.
I also worked to package the new version of pytrainer, a sport
activity logging tool. pytrainer is maintained by the "pkg-running"
team, aka No l K the, Ralf Treinen and /me. This is the software I use
for keeping track of all my running activities. We have a very good
link with our upstream and I even recently brought several translation
updates by involving Debian translators in the loop.
While working to integrate version 1.9.1 (we had 1.8.0 up to now in
unstable), I also cleanedout things here orthereand I'mquite happy
with the result. More testing will of course be appreciated.
The Comprehensive R Archive Network, or CRAN for short, has been a major driver in the success and
rapid proliferation of the R statistical language and environment. CRAN currently hosts around 3400
packages, and is growing at a rapid rate. Not too long ago, John Fox gave a keynote lecture
at the annual R conference and provided a lot of quantitative insight into R and CRAN---including an estimate of an incredible growth rate
of 40% as a near-perfect straight line on a log-log chart! So CRAN does in fact grow exponentially. (His talk morphed into this paper in the R Journal, see figure 3 for this chart.)
The success of CRAN is due to a lot of hard work by the CRAN maintainers, lead for many years and still today by
Kurt Hornik whose dedication is unparalleled. Even at the current growth rate of several
packages a day, all submissions are still rigorously quality-controlled using strong testing features available in the R system.
And for all its successes, and without trying to sound ungrateful, there have always been some things missing at CRAN. It has always been
difficult to keep a handle on the rapidly growing archive. Task Views for particular fields, edited by volunteers with specific domain
knowledge (including yours truly) help somewhat, but still cannot keep up with the flow. What is missing are regular updates on packages. What
is also missing is a better review and voting system (and while Hadley Wickham mentored a Google Summer of Code
student to write CRANtastic, it seems fair to say that this subproject didn't exactly take off
either).
Following useR! 2007 in Ames, I decided to do something and noodled over a first design on the drive back to Chicago. A weekend of hacking
lead to CRANberries. CRANberries uses existing R functions to learn which packages
are available right now, and compares that to data stored in a local SQLite database. This is enough to learn two things: First, which
new packages were added since the last run. That is very useful information, and it feeds a website with blog
subscriptions (for the technically minded: an RSS feed, at this URL).
Second, it can also compare current versions numbers with the most recent stored
version number, and thereby learns about updated packages. This too is useful, and also feeds a website and RSS
stream (at this URL; there is also a combined one for new and updated packages.) CRANberries
writes out little summaries for both new packages (essentially copying what the DESCRIPTION file contains), and a quick
diffstat summary for updated packages. A static blog compiler munges this into static html pages which I serve from here, and
creates the RSS feed data at the same time.
All this has been operating since 2007. Google Reader tells me the the RSS feed averages around 137 posts per week, and has about 160 subscribers. It
does feed to Planet R which itself redistributes so it is hard to estimate the absolute number of
readers. My weblogs also indicate a steady number of visits to the html versions.
The most recent innovation was to add tweeting earlier in 2011 under the @CRANberriesFeed Twitter handle. After all, the best
way to address information overload and too many posts in our RSS readers surely is to ... just generate more information and add some Twitter
noise. So CRANberries now tweets a message for each new package, and a summary
message for each set of new packages (or several if the total length exceeds the 140 character limit). As of today, we have sent 1723
tweets to what are currently 171 subscribers. Tweets for updated packages were added a few months later.
Which leads us to today's innovation. One feature which has truly been missing from CRAN was updates about withdrawn packages. Packages
can be withdrawn for a number of reasons. Back in the day, CRAN carried so-called bundles carrying packages inside. Examples were VR and
gregmisc. Both had long been split into their component packages, making VR and gregmisc part of the set of packages no longer on the top
page of CRAN, but only its archive section. Other examples are packages such as Design, which its author Frank Harrell renamed to rms to match to title of the book covering its
methodology. And then there are of course package for which the maintainer disappeared, or lost interest, or was unable to keep up with
quality requirements imposed by CRAN. All these packages are of
course still in the Archive section of CRAN.
But how many packages did disappear? Well, compared to the information accumulated by
CRANberries over the years, as of today a staggering
282 packages have been withdrawn for various reasons. And at least I would like to know more regularly when this happens,
if only so I have a chance to see if the retired package is one the 120+
packages I still look after for Debian (as happened recently with two Rmetrics packages).
So starting with the next scheduled run, CRANberries
will also report removed packages, in its own subtree of the website and its own RSS feed (which should appear at
this URL). I made the required code changes (all of about two
dozen lines), and did some light testing. To not overwhelm us all with line noise while we catch up to the current steady state of packages, I
have (temporarily) lowered the frequency with which CRANberries is called by cron.
I also put a cap on the number of removed packages that are reported in each run. As always with new code, there may be a bug or two but I
will try to catch up in due course.
I hope this is of interest and use to others. If so, please use the RSS feeds in your RSS readers, and subscribe to the
@CRANberriesFeed, And keep using CRAN, and let's all say thanks to Kurt, Stefan, Uwe, and
everybody who is working on CRAN (or has been in the past).
The Community Wellness team at MIT has a program on stress
reduction, mindfulness, and relaxation. Among their services is a
guided three-minute relaxation exercise recording (available at
extension 3-2256 or 617-253-CALM). It's a very relaxing message.
At the end of the recording, there's a revealing error where a
standard voicemail robo-voice say "no messages are waiting" before you
system hangs up on you. Turns out, the MIT wellness folks implemented
this using the normal MIT voicemail system.
This gave me a thought: What if my voicemail greeting included a
guided relaxation message as part of its greeting so that anyone who
left a message had the chance to relax a little bit first? Would
messages left for me be more positive after a window of serenity?
Would people ask less of me? Would my callers feel more relaxed and
happier during the rest of their day?
I just recorded a short version of the MIT message as my voicemail
greeting. I suppose I will find out.
Next.