Search Results: "spa"

8 July 2012

Dirk Eddelbuettel: New package RcppCNPy with release 0.1.0 (and 0.0.1 earlier last week)

A few days ago I had blogged about getting NumPy data in R by using a simple converter script. That works fine, but it is a little annoying to have to write an entire file only to read from it again. So I kept looking around for a better solution---and soon found the cnpy library by Carl Rogers which provides simple C++ functions to read and write NumPy files. Bringing such a C++ library to R is done very easily via Rcpp modules. The resulting package contains a single R file with a single line: loadModule("cnpy", TRUE). And it relies on the following module declarations in the a C++ file:
RCPP_MODULE(cnpy) 
    using namespace Rcpp;
    function("npyLoad",                         // name of the identifier at the R level
             &npyLoad,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("type") = "numeric"),
             "read an npy file into a numeric or integer vector or matrix");
    function("npySave",                         // name of the identifier at the R level
             &npySave,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("object"), 
                           Named("mode") = "w"),
             "save an R object (vector or matrix of type integer or numeric) to an npy file");
 
which give us at the R prompt
R> library(RcppCNPy)
Loading required package: Rcpp
R> npyLoad
internal C++ function <0x243af70>
    docstring : read an npy file into a numeric or integer vector or matrix
    signature : Rcpp::RObject npyLoad(std::string, std::string)
R> npySave
internal C++ function <0x23033e0>
    docstring : save an R object (vector or matrix of type integer or numeric) to an npy file
    signature : void npySave(std::string, Rcpp::RObject, std::string)
R> 
these two functions (and their docstrings) defined above. That's all! Well there are about one hundred more lines dealing with whether we have integer or numeric data, and whether we use a vector or a matrix. But all in all pretty simple... So version 0.1.0 of this new package RcppCNPy completes the initial release 0.0.1 from earlier in the week by adding The NEWS entry for this release (as well as the initial one) follow:
News for Package RcppCNPy Changes in version 0.1.0 (2012-07-07)
  • Added automatic use of transpose to automagically account for Fortran-vs-C major storage defaults between Python and R.
  • Support for integer types in dependent on the int64_t type which is available only when the -std=c++0x switch is used at build-time (and CRAN still discourages use of it)
  • Added support for reading gzip'ed files ending in ".npy.gz"
  • Added regression tests in directory tests/
  • Added a vignette describing the package
  • Added a timing benchmark in demo/timings.R
Changes in version 0.0.1 (2012-07-04)
  • Initial version, as a straightforward Rcpp modules wrap around the cpny library by Carl Rogers (on github under a MIT license).
  • At present, npy files can be read and written for vectors and matrices of either numeric or integer type. Note however that matrices are currently transposed because of the default Fortran ordering done by numpy.
I will follow up with a little usage example later. CRANberries also provides a diffstat report for 0.1.0 relative to 0.0.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

7 July 2012

Tiago Bortoletto Vaz: VoIP in Neo Freerunner with Qtmoko and Linphone

I keep using a Neo Freerunner with Qtmoko in a Debian Squeeze (+ some Wheezy) chroot as my daily phone. It also works well for SMS, GPS, MUA, IM, IRC and some Web browsing with Aurora (using both Wifi and GPRS). Last weekend I managed to play a little with VoIP possibilities in Qtmoko and ended with a functional SIP client in my hands. I want to share some points which may be useful for other brave freerunners: UPDATE: I didn't make clear that I get a very nice audio quality using internal mic/speaker with the above voip-handset state file. The issue is only when using the headset, which demands understanding and setting up a proper state file. Enjoy some shots:

Bartosz Fe&#324;ski: God bless LVM

Correct me if I m wrong (and I really want to be wrong).
Is that whole LVM crap expects me to be forecaster/fortune-teller? I wanted to upgrade some squeeze based virtual machine to wheezy. Sounds nice, yeah?
So before the upgrade you should ensure that you have backup, right?
Assuming that whole system is LVM based all you have to do before upgrade is LVM snapshot.
I did it and I was quite surprised that I had to declare its size.
I did it basing on assumption that upgrade shouldn t take more than let say 5GB. Tada!
mastersrv:~# lvs
  LV       VG      Attr   LSize   Origin  Snap%  Move Log Copy%  Convert
  backup   virtual -wi-ao  50,00g                                       
  mainsrv  virtual -wi-ao 300,00g                                       
  redminbu virtual swi-a-   5,00g redmine   0,00                        
  redmine  virtual owi-a-  19,53g
Now I did upgrade redmine virtual machine and this is what I got:
mastersrv:~# lvs
  LV       VG      Attr   LSize   Origin  Snap%  Move Log Copy%  Convert
  backup   virtual -wi-ao  50,00g                                       
  mainsrv  virtual -wi-ao 300,00g                                       
  redminbu virtual Swi-I-   5,00g redmine 100.00                        
  redmine  virtual owi-ao  19,53g
These attr flags mean Snapshot invalid and Invalid snapshot .
End of the story this is last time I actually tried that method of backup.
Happily I have also some traditional backup (mysqldump+rsync) but I m really surprised that LVM works this way.
There was plenty of free space to enlarge that volume and this is what you get in your logs:
[13402279.777238] device-mapper: snapshots: Invalidating snapshot: Unable to allocate exception.
Unable to allocate exception? Did it really try to do it?
mastersrv:~# pvs
  PV         VG      Fmt  Attr PSize   PFree  
  /dev/md0   virtual lvm2 a-   884,01g 514,46g
I don t think so.

Petter Reinholdtsen: Free Timetabling Software - nice free software

Included in Debian Edu / Skolelinux is a large collection of end user and school specific software. It is one of the packages not installed by default but provided in the Debian archive for schools to install if they want to, is a system to automatically plan the school time table using information about available teachers, classes and rooms, combined with the list of required courses and how many hours each topic should receive. The software is named FET, and it provide a graphical user interface to input the required information, save the result in a fairly simple XML format, and generate time tables for both teachers and students. It is available both for Linux, MacOSX and Windows. This is the feature list, liftet from the project web site: I have not used it myself, as I am not involved in time table planning at a school, but it seem to work fine when I test it. If you need to set up your schools time table, and is tired of doing it manually, check it out. A quick summary on how to use it can be found in a blog post from MarvelSoft. If you find FET useful, please provide a recipe for the Debian Edu project in the Debian Edu HowTo section.

5 July 2012

Christian Perrier: DebConf running: take 3

The Grand Plan today was reaching the end of the road I already explored on Tuesday. Named "Camino Las Viudas", it's climbing in the southern hills up to a place named "Intermzeeo del Bosque", apparently a very fancy (and expensive) restaurant. The road climbs in the apparently "rich" neighbourhood of Managua, with very fancy villas as guarded by private security, as well as private condominiums, guarded as well. What's funny to see is that, at the bottom of the road, when it crosses a large avenue (Pista Suburbana Espana), you can find several auto-rickshaws, just like those you find everywhere in India. Those autos apparently drive people who happen to work in these villas (like maids, gardeners, security guards, etc.) up to the place they work. We even met two cows wandering on the road and I hesitated waiting for a rickshaw to show up and take a pic of the road (with holes), the auto and the cows, then pretending that I'm no longer in Nicaragua but back in Karnataka. I did the run along with No l K the, who I was very happy to find again, restoring our now famous Debconf Runners Duet. We even found the German Embassy! Though climb, even if we took care to start at 6 a.m. to avoid the heat (and also the traffic and noise). No l unfortunately had to give up at 1km from the top of the road, when a 15% climb suddenly "killed" his legs. I need to train him more for ups and downs..:-) I could continue the road and finally reached the restaurant....which was guarded by private security (as apparently everything in this country) who unfortunately didn't let me in despite my desperate attempts to convince him, in broken Spanish, that I just wanted to take a few pictures and not place a bomb or rob someone... So, I gota little bit disappointed and the only option was then....to run down for another 8.5 kilometers, accelerating all the way, to finish nearly at 14 km/h on Managua's sidewalks...Fun. Great run (nearly 400m positive slope in less than 17km). Next challenge : find another such nice run in the neighbourhood! Maybe Volcan Masaya tomorrow. GPS trace is here.

4 July 2012

Johannes Schauer: enumerating elementary circuits of a directed_graph

For my GSoC project this year I need to be able to enumerate all elementary circuits of a directed graph. My code is written in Ocaml but neither the ocamlgraph library nor graph libraries for other languages seem to implement a well tested algorithm for this task. In lack of such a well tested solution to the problem, I decided to implement a couple of different algorithms. Since it is unlikely that different algorithms yield the same wrong result, I can be certain enough that each individual algorithm is working correctly in case they all agree on a single solution. As a result I wrote a testsuite, containing an unholy mixture of Python, Ocaml, D and Java code which implements algorithms by D. B. Johnson, R. Tarjan, K. A. Hawick and H. A. James.

Algorithm by R. Tarjan The earliest algorithm that I included was published by R. Tarjan in 1973.
Enumeration of the elementary circuits of a directed graph
R. Tarjan, SIAM Journal on Computing, 2 (1973), pp. 211-216
http://dx.doi.org/10.1137/0202017
I implemented the pseudocode given in the paper using Python. The git repository can be found here: https://github.com/josch/cycles_tarjan

Algorithm by D. B. Johnson The algorithm by D. B. Johnson from 1975 improves on Tarjan's algorithm by its complexity.
Finding all the elementary circuits of a directed graph.
D. B. Johnson, SIAM Journal on Computing 4, no. 1, 77-84, 1975.
http://dx.doi.org/10.1137/0204007
In the worst case, Tarjan's algorithm has a time complexity of O(n e(c+1)) whereas Johnson's algorithm supposedly manages to stay in O((n+e)(c+1)) where n is the number of vertices, e is the number of edges and c is the number of cycles in the graph. I found two implementations of Johnson's algorithm. One was done by Frank Meyer and can be downloaded as a zip archive. The other was done by Pietro Abate and the code can be found in a blog entry which also points to a git repository. The implementation by Frank Meyer seemed to work flawlessly. I only had to add code so that a graph could be given via commandline. The git repository of my additions can be found here: https://github.com/josch/cycles_johnson_meyer Pietro Abate implemented an iterative and a functional version of Johnson's algorithm. It turned out that both yielded incorrect results as some cycles were missing from the output. A fixed version can be found in this git repository: https://github.com/josch/cycles_johnson_abate

Algorithm by K. A. Hawick and H. A. James The algorithm by K. A. Hawick and H. A. James from 2008 improves further on Johnson's algorithm and does away with its limitations.
Enumerating Circuits and Loops in Graphs with Self-Arcs and Multiple-Arcs.
Hawick and H.A. James, In Proceedings of FCS. 2008, 14-20
www.massey.ac.nz/~kahawick/cstn/013/cstn-013.pdf
In contrast to Johnson's algorithm, the algorithm by K. A. Hawick and H. A. James is able to handle graphs containing edges that start and end at the same vertex as well as multiple edges connecting the same two vertices. I do not need this functionality but add the code as additional verification. The paper posts extensive code snippets written in the D programming language. A full, working version with all pieces connected together can be found here: https://github.com/josch/cycles_hawick_james The algorithm was verified using example output given in the paper. The project README states how to reproduce it.

Input format All four codebases have been modified to produce executables that take the same commandline arguments which describes the graphs to investigate. The first argument is the number of vertices of the graph. Subsequent arguments are ordered pairs of comma separated vertices that make up the directed edges of the graph. Lets look at the following graph as an example: cycle example The DOT source for this graph is:
digraph G  
  0;
  1;
  2;
  0 -> 1;
  0 -> 2;
  1 -> 0;
  2 -> 0;
  2 -> 1;
   
To generate the list of elementary circuits using Tarjan's algorithm for the graph above, use:
$ python cycles.py 3 0,1 0,2 1,0 2,0 2,1
0 1
0 2
0 2 1
The commandline arguments are the exact same for the other three methods and yield the same result in the same order. If the DOT graph is in a format as simple as above, the following sed construct can be used to generate the commandline argument that represents the graph:
$ echo  sed -n -e '/^\s*[0-9]\+;$/p' graph.dot   wc -l   sed -n -e 's/^\s*\([0-9]\) -> \([0-9]\);$/\1,\2/p' graph.dot 

Testsuite As all four codebases take the same input format and have the same output format, it is now trivial to write a testsuite that compares the individual output of each algorithm for the same input and checks for differences. The code of the testsuite is available via this git repository: https://github.com/josch/cycle_test The other four repositories exist as submodules of the testsuite repository.
$ git clone git://github.com/josch/cycle_test.git
$ cd cycle_test
$ git submodule update --init
A testrun is done via calling:
$ ./test.sh 11
The argument to the shell script is an integer denoting the maximum number N of vertices for which graphs will be generated. The script will compile the Ocaml, Java and D sourcecode of the submodules as well as an ocaml script called rand_graph.ml which generates random graphs from v = 1..N vertices where N is given as a commandline argument. For each number of vertices n, e = 1..M number of edges are chosen where M is maximum number of edges given the number of vertices. For every combination of number of vertices v and number of edges e, a graph is randomly generated using Pack.Digraph.Rand.graph from the ocamlgraph library. Each of those generated graphs is checked for cycles and written to a file graph-v-e.dot if the graph contains a cycle. test.sh then loops over all generated dot files. It uses the above sed expression to convert each dot file to a commandline argument for each of the algorithms. The outputs of each algorithm are then compared with each other and only if they do not differ, does the loop continue. Otherwise the script returns with an exit code. The tested algorithms are the Python implementation of Tarjan's algorithm, the iterative and functional Ocaml implementation as well as the Java implementation of Johnson's algorithm and the D implementation of the algorithm by Hawick and James.

Future developments Running the testsuite with a maximum of 12 vertices takes about 33 minutes on a 2.53GHz Core2Duo and produces graphs with as much as 1.1 million cycles. It seems that all five implementations agree on the output for all 504 generated graphs that were used as input. If there should be another implementation of an algorithm that enumerates all elementary circuits of a directed graph, I would like to add it. There are some more papers that I would like to read but I lack access to epubs.siam.org and ieeexplore.ieee.org and would have to buy them. Benchmarks seem a bit pointless as not only the algorithms are very different from each other (and there are many ways to tweak each of them) but also the programming languages differ. Though for the curious kind, it follows the time each algorithm takes to enumerate all cycles for all generated graphs up to 11 vertices.
algorithmtime (s)
Johnson, Abate, Ocaml, iterative137
Johnson, Abate, Ocaml, functional139
Tarjan, Python153
Hawick, D175
Johnson, Meyer, Java357
The iterative Ocaml code performs as well as the functional one. In practice, the iterative code should probably be preferred as the functional code is not tail recursive. On the other hand it is also unlikely that cycles ever grow big enough to make a difference in the used stack space. The Python implementation executes surprisingly fast, given that Tarjan's algorithm is supposedly inferior to Johnson's and given that Python is interpreted but the Python implementation is also the most simple one with the least amount of required datastructures. The D code potentially suffers from the bigger datastructures and other bookkeeping that is required to support multi and self arcs. The java code implements a whole graph library which might explain some of its slowness.

Michal &#268;iha&#345;: Weblate 1.1

Pretty much on schedule, Weblate 1.1 has been released today. It comes with translation updates, bug fixes, improvements in working with Git repositories and brings support for offloading indexing. Full list of changes for 1.1: You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances can be found in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you (this will be decided case by case as my hosting space is limited). Update: Weblate appliace has now also been updated to 1.1.

Filed under: English Phpmyadmin Suse Weblate 0 comments Flattr this!

Dirk Eddelbuettel: RcppBDT 0.2.0

A new release of the RcppBDT package appeared on CRAN earlier today. RcppBDT uses Rcpp, and in particular the nifty Rcpp modules feature of wrapping C++ code for R just by declaring the (class or function) interfaces. It uses this to bring in some useful functions from Boost Date.Time to R so that one can do things like
R> library(RcppBDT)
R> sapply(2012:2016, function(year)
+         format(getNthDayOfWeek(first, Mon, Sep, year)))
[1] "2012-09-03" "2013-09-02" "2014-09-01" "2015-09-07" "2016-09-05"
R> 
to compute the next five Labor Day dates in the US, given the year and the first Monday of September requirement. More examples are e.g. on the earlier blog post announcing version 0.1.0. Changes are mostly internal. R 2.15.1 brough a better / easier way to load such 'modules' into R, and Rcpp 0.9.13 allows us to use this. So RcppBDT continues to be useful as an example package for Rcpp modules. I also streamlines the interface a little: identifiers are now directly accessible in the package's NAMESPACE rather than just via the an instantiated object. I also added a NEWS file, using the .Rd format so that we can import the marked-up text:
News for Package RcppBDT Changes in version 0.2.0 (2012-07-02)
  • The core module, which wraps what in C++ is boost::gregorian::date, is now exposed as an Rcpp module bdtDate. As all example and demos operated off a (package-global) variable 'bdt', no user visible change was needed outside of the code instantiating at package load.
  • Updated package instantiation to the new facilities offered by the current versions R 2.15.1 and Rcpp 0.9.13 which make Rcpp module initialization easier and more streamlined.
Changes in version 0.1.0 (2011-01-17)
  • First CRAN upload (and see ChangeLog for more granular details) bug fix in svm cross validation
Courtesy of CRANberries, there is also a diffstat report for 0.2.0 relative to 0.1.0. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

3 July 2012

C.J. Adams-Collier: Currency platform superior to US Federal Reserve Bank

Should I recommend that Software in the Public Interest implement a monetary stack with the stated purpose of replacing the federal reserve system? We would have to publish the source code, of course, and the transaction db would need to be de-centralized so as to avoid private interests using it for their own benefit at the expense of their fellow humans. Not that that would ever happen, mittens. We d probably experience a period of instability due to scaling issues during the learning state, say the first two to five years. The IRS would need to be convinced that it would be a good idea, though. And I think they only listen to voters. And I don t want to be the one responsible for convincing them. I m not so good at exchanging paperwork with them, and they would likely get confused. Anyway, the idea would be that after the learning state, sessions would be migrated from the Fed db at a sustainable rate. After moving the sessions off of the Fed s current, compromised stack, the increase in tax revenues caused by removal of the current implementation s filters should be more than enough to fund the maintenance of the implementation undergoing strict peer review. http://www.wapublicbankproject.org/

2 July 2012

Matthew Garrett: Fun with firmware ABI confusion

I spent a bunch of the past week trying to figure out why my UEFI bootloader code was crashing in bizarre ways. It turns out to be a known issue that various people have hit in the past. After a lot of staring (and swearing) I finally got to the point where I could run a debugger against a running UEFI binary under qemu and found that I was entering a function with valid arguments but executing it with invalid ones. After spending a while crying on IRC people suggested that maybe the red zone was getting corrupted, and it turned out that this was the problem.

The red zone is part of the AMD64 System V ABI and consists of 128 bytes at the bottom of the stack frame. This region is guaranteed to be left untouched by any interrupt or signal handlers, so it's available for functions to use as scratch space. For the code in question, gcc was copying the registers onto the stack before executing the function. Something then trod on those registers. This also explained why I wasn't seeing the problem when I added debug print statements - signal and interrupt handlers won't touch it, but other functions might, so it tends to only be used by functions that don't call any other functions. Adding a print statement meant that the compiler left the red zone alone.

But why was it getting corrupted? UEFI uses the Microsoft AMD64 ABI rather than the System V one, and the Microsoft ABI doesn't define a red zone. UEFI executables built on Linux tend to use System V because that means we can link in static libraries built for Linux, rather than needing an entirely separate toolchain and libraries to build UEFI executables. The problem arises when we run one of these executables and there's a UEFI interrupt handler still running. Take an interrupt, Microsoft ABI interrupt handler gets called and the red zone gets overwritten.

There's a simple fix - we can just tell gcc not to assume that the red zone is there by passing -mno-red-zone. I did that and suddenly everything was stable. Frustrating, but at least it works now.

comment count unavailable comments

Mehdi Dogguy: Adding useful links on bugreports pages

I'm a heavy user of Debian's Bug Tracking System (especially during this period of the year ;)) and I quite like it. And as a Debian developer, I have to check multiple sources when looking at some source package. Namely, I mainly check the following sources:
All this is great since you can move from the first one to the second one (and vice-versa) just by clicking with your mouse. It becomes less handy when you are looking at any bug report. In fact, the BTS doesn't link back to the previously mentioned services [1]. So, you have to copy/paste the name of the package or start a new tab and write down the address of the desired service. All this is not very nice for a lazy person like me :)

A few days ago, I remembered that I was already using GreaseMonkey and thought : "Hey, it would be nice to hack quickly something using GM to add the missing links". After a few tests, I've come up with a (very simple, yet extremely useful to me) user script that I called Debian BTS extra links [2]. It adds what I've been missing for years. If you are lazy and don't want to install right now, you can check the following pages to see what it adds:
The userscript will add links to PTS, Buildd status pages and Popcon near every pkgreport.cgi?src= link. Simple and useful. I hope you'll enjoy it as much as I do.

Of course, I've submitted a bug (#679282) against the pseudo package bugs.debian.org kindly asking its maintainer to add a link to (for now) the PTS. But I know that they are busy so I'm not expecting a result soon. Fow now, my simple and dumb userscript WFM.

[1] This is partly true since it links back to the PTS on main pages (pages that list all bug reports of a given set of packages).

[2] Only tested with a recent Iceweasel (from experimental) and Chromium (from unstable).

P.S.: If you do wonder, #4300 is the least bug number known to the BTS. Well, or at least, the result of running ls -1 /srv/bugs.debian.org/spool/archive/00/ sort -h head -n1 on bugs.debian.org.

Thibaut Girka: apt-get install gcc-4.7-arm-linux-gnueabihf

Hi! As you may know, I'm working on multiarch cross-toolchains as part of the Google summer of Code (see my previous post for details), and although there is still a lot to do, the project is doing fairly well, and a cross-gcc is already installable and working. I'll proceed with detailed instructions on how to install the cross-compiler, along with some explanations on multiarch itself, and how to build cross-toolchains from source. Prerequisites First, you need specific versions apt (>= 0.9.6) and dpkg (>= 1.16.5).
This is because the cross-toolchains packages I'm working on depend on specific cross-arch dependencies, such as libc6:armhf. Support for such dependencies has been added in the aforementioned versions of apt and dpkg. This means you should be running unstable, or at least use dpkg from unstable. Adding a foreign architecture Adding a foreign architecture is straightforward, but I've seen users do overcomplicated, or just plain wrong things. So, let's do this step by step (well, there are only two of them), with some explanations.
  1. Tell dpkg to handle an additional architecture by typing dpkg --add-architecture armhf. There isn't much to say. It will only add an architecture to the list of architectures dpkg would accept to handle.
  2. Update APT's database. You need to run apt-get update after adding or removing an architecture to get package lists for all wanted architectures and rebuild APT's database, including the implicit cross-arch dependencies/conflicts brought by multiarch.
And that's it for the common case. Now, a word on architecture restrictions: a common misconception about multiarch seems to be that you have to tell APT which architectures you want from the mirrors by modifying your sources.list file(s). This is not true, as, by default, APT gets the list of accepted architectures from dpkg, and gets the correct package lists from the mirrors configured in the sources.list. You only need to use arch restrictions ( deb [arch=arch1,arch2,...] ... ) when you want to get packages from a mirror only for a subset of your accepted architectures. This is very unlikely, and in the general you do not need to modify your sources.list, at all. Getting the cross-compilers from my repository I provide compiled versions of gcc-4.7-arm-linux-gnueabihf for i386 and amd64 in my repository. If you have followed the previous paragraphs, nothing should be easier than adding a deb http://emdebian.org/~thibg/repo/ sid main line to your sources.list and doing the following:
# apt-get update
[...]
# apt-get --no-install-recommends install gcc-4.7-arm-linux-gnueabihf
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  binutils-arm-linux-gnueabihf cpp-4.7-arm-linux-gnueabihf gcc-4.7-arm-linux-gnueabihf-base gcc-4.7-base:armhf libc6:armhf libgcc1:armhf
  libgmp10 libgomp1:armhf libmpc2 libmpfr4
Suggested packages:
  binutils-doc gcc-4.7-locales libmudflap0-4.7-dev:armhf gcc-4.7-doc libgcc1-dbg:armhf libgomp1-dbg:armhf libitm1-dbg:armhf
  libquadmath-dbg:armhf libmudflap0-dbg:armhf binutils-gold glibc-doc:armhf locales:armhf
Recommended packages:
  libc6-dev:armhf
The following NEW packages will be installed:
  binutils-arm-linux-gnueabihf cpp-4.7-arm-linux-gnueabihf gcc-4.7-arm-linux-gnueabihf gcc-4.7-arm-linux-gnueabihf-base gcc-4.7-base:armhf
  libc6:armhf libgcc1:armhf libgmp10 libgomp1:armhf libmpc2 libmpfr4
0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
Need to get 23.5 MB of archives.
After this operation, 57.2 MB of additional disk space will be used.
Do you want to continue [Y/n]?
I've disabled the recommends since, libc6-dev:armhf would recommend gcc:armhf itself, which is not co-installable with your native gcc if you have it installed. Building the cross-toolchain yourselves If you don't trust the repository above, or simply want to rebuild the toolchain yourself, it is rather simple: get my gcc-4.7 patches and:
$ apt-get source binutils
$ cd binutils-*/
$ TARGET=arm-linux-gnueabihf dpkg-buildpackage -uc -us -b
$ cd ../
# dpkg -i binutils*arm-linux-gnueabihf*.deb
$ apt-get source gcc-4.7
$ cd gcc-4.7/
$ for patch in path/to/the/patches/*.patch; do patch -p1 < $patch ; done
$ GCC_TARGET=armhf debian/rules control
$ GCC_TARGET=armhf dpkg-buildpackage -uc -us -b
Using the cross-compiler It's easy, but there is a trick, and dpkg-buildpackage -aarmhf won't work out of the box yet, as gcc-4.7-arm-linux-gnueabihf provides the arm-linux-gnueabihf-gcc-4.7 executable, but no arm-linux-gnueabihf-gcc symlink. So, for the moment:
$ cat > test.c << __EOF__
#include <stdio.h>
int main(void)
 
  printf("Hello world\\n");
  return 0;
 
__EOF__
$ arm-linux-gnueabihf-gcc-4.7 test.c
$ qemu-arm ./a.out 
Hello world
Conclusion The toolchain is not complete yet (there is some more work to do on binutils and libstdc++, and equivalents to the gcc , cpp , and build-essential packages to provide), but it should still be useful to cross-compile C programs. Multi-arch cross-toolchains for wheezy can definitely be hosted at emdebian, and I hope to clean up the build process so that it could get included in the archive for wheezy+1.

Craig Small: Dejagnu tips

Both the procps and psmisc projects use Dejagnu for their testing. It s interesting to try to understand how it all works and learning a new language TCL. One of the big problems with Dejagnu is its documentation which is very sparse. So here are two things I ve picked up along the way for a reference for myself and others. Locale We recently had a bug report on the procps list where tests for w failed for someone when the headers were shown. After a some further debugging it was a locale problem, where the testsuite was looking for a number like 3.14 but the tester was seeing 3,14. There are many ways around this but for us it was to set the locale, with the following line in config/unix.exp after suggestions by Mike Frysinger should fix it.
set env(LC_ALL) "C"
Cleaning Up
The documentation on this function is awesome! A title Cleanup Procedure and one line cleanup() .
That s all you get. The cleanup proc or function is called at the end of the testing. I use it
to remove temporary files and things like that.
proc cleanup      
  global test_file
  exec rm $test_file
 

1 July 2012

Ingo Juergensmann: 100% CPU load due to Leap Second

This morning Gregor Samsa woke up... oh, pardon! This morning I woke up and found myself puzzled, because my home server was eating up all of my 4 cores CPU cycles. Especially mysqld was high on CPU load. 100% CPU load for the mysql-server instance and 100% CPU load for akonadiservers own mysqld instance. Restarting KDE and mysql-server didn't help on my Debian unstable machine. Next step was upgrading the system. Sometimes this helps indeed, but not today. Looking at bugs.debian.org for mysql-server didn't reveal any help as well. So my next logical step was to ask on #debian-devel in IRC. And my question was very quick answered:
11:28 < ij> since tonight I've got two mysqld processes running at 100% CPU, one spawned by akonadi and
the other is the mysqld from mysql-server (unstable that is). is this an already known issue?
haven't found anything on b.d.o for mysql-server, though
11:29 < mrvn> ij: topic
11:29 < mrvn> you need to set the time
11:30 < ij> waaaaah!
11:30 < mrvn> ij: indeed.
The topic was at that time:
100% CPU? Reset leap second http://xrl.us/bnde4w
So, it was caused by the leap second. Although you might suspect mysql doing some nasty things (which, IMHO, is always a good guess ;)), the issue is this time within the Linux kernel itself, as a commit on git.kernel.org clearifies. To fix this issue you need to set the time manually using the following command or just reboot:
date -s " date "
So far I found these applications being hit by this kernel bug:
  • mysql-server
  • akonadi (as it uses own mysql instances)
  • Firefox
  • Openfire Jabber server (because it's using Java, which seems to trigger the problem as mysql does)
  • Virtualbox' VBoxSVC process
  • puppetmaster from package puppet, reported by Michael
  • mythfrontend, reported by pos on #debian-devel
  • Jetty, Hudson, Puppet agent and master, reported by Christian
  • milter-greylist, reported by E. Recio
  • dovecot, reported by Diogo Resende
  • Google Chrome, reported by Erik B. Andersen
  • if you find more apps, please comment and I'll include them here...
So, hope this helps and many thanks to mrvn and infinity on #debian-devel for the help!
Kategorie:

Dirk Eddelbuettel: Getting numpy data into R

The other day, I found myself confronted with a large number of large files. Which were presented in (gzip-)compressed ascii format---which R reads directly via gzfile() connections---as well as (compressed) numpy files. The numpy can be read very efficiently into Python. We can do the same in R via save() and load(), of course. But the trouble is that you need to read them first. And reading hundreds of megabytes from ascii is slow, no matter which language you use. Concerning R, I poked aound scan(), played with the colClasses argument and looked at the recent LaF package written just for this purpose. And all these solutions were still orders of magnitude slower than reading numpy. Which is no surprise as it is really hard to beat binary formats when you have to parse countless ascii tokens. So the obvious next idea was to read the numpy file in Python, and to write a simple binary format. One helpful feature with this data set was that it contained only regular (rectangular) matrices of floats. So we could just store two integers for the dimensions, followed by the total data in either one large binary blob, or a sequence of column vectors. But one minor trouble was that the Intertubes lead to no easy solution to unpack the numpy format. StackOverflow had plenty of question around this topic converned with, say, how to serialize in language-independent way. But no converters. And nobody local knew how to undo the "pickle" format underlying numpy. But a remote friend did: Laurent, well-known for his Rpy2 package, pointed me towards using the struct module and steered me towards the solution shown below. So a shameless plug: if you need a very experienced Python or R consultant for sciece work, consider his consulting firm. Finally, to round out this post, let's show the simple solution we crafted so that the next guy searching the Intertubes will have an easier. Let us start with a minimal Python program writing numpy data to disk:
#!/usr/bin/env python
#
# simple example for creating numpy data to demonstrate converter
import numpy as np
# simple float array
a = np.arange(15).reshape(3,5) * 1.1
outfile = "/tmp/data.npy"
np.save(outfile, a)
Next, the simple Python converter to create a binary file containing two integers for row and column dimension, followed by row times columns of floats:
#!/usr/bin/python
#
# read a numpy file, and write a simple binary file containing
#   two integers 'n' and 'k' for rows and columns
#   n times k floats with the actual matrix
# which can be read by any application or language that can read binary
 
import struct
import numpy as np
inputfile = "/tmp/data.npy"
outputfile = "/tmp/data.bin"
# load from the file
mat = np.load(inputfile)
# create a binary file
binfile = file(outputfile, 'wb')
# and write out two integers with the row and column dimension
header = struct.pack('2I', mat.shape[0], mat.shape[1])
binfile.write(header)
# then loop over columns and write each
for i in range(mat.shape[1]):
    data = struct.pack('%id' % mat.shape[0], *mat[:,i])
    binfile.write(data)
binfile.close()
Lastly, a quick littler script showing how R can read the data in a handful of lines:
#!/usr/bin/r
infile <- "/tmp/data.bin"
con <- file(infile, "rb")
dim <- readBin(con, "integer", 2)
Mat <- matrix( readBin(con, "numeric", prod(dim)), dim[1], dim[2])
close(con)
print(Mat)
That did the job---and I already used to converter to read a few weeks worth of data for further analysis in R. This obviously isn't the last word on possible solutions as the additional temporary file can be wasteful (unless it forms a cache for data read multiple times). If someone has nice solutions, please don't hold back and contact me. Thanks again to Laurent for the winning suggestion concerning struct, and help in getting the examples shown here to work.

30 June 2012

Luca Falavigna: FTP Team stats during Wheezy development

Already chilled by Wheezy freeze? It s been a long ride since the release of Squeeze, and your beloved FTP Team tried to assist our tireless developers and contributors at its best. Here are some hot stats to give you a figure about what happened behind the scenes. Since the release of Squeeze, 7462 .changes files with NEW components were processed by dak, with an average of 14.660 NEW packages per day. On the FTP Team side, we had 6877 accepts (13.511 per day), 641 rejects (1.259 per day) and 280 comments to maintainers (0.550 per day). This table represents the activity by single team member:
Login Accepts Rejects Comments
ansgar 407 accepts (0.800 per day) 71 rejects (0.139 per day) 53 comments (0.104 per day)
dak 12 accepts (0.024 per day) 1 rejects (0.002 per day) 0 comments (0.000 per day)
dktrkranz 4319 accepts (8.485 per day) 381 rejects (0.749 per day) 104 comments (0.204 per day)
joerg 100 accepts (0.196 per day) 12 rejects (0.024 per day) 1 comments (0.002 per day)
mhy 214 accepts (0.420 per day) 14 rejects (0.028 per day) 5 comments (0.010 per day)
stew 67 accepts (0.132 per day) 16 rejects (0.031 per day) 7 comments (0.014 per day)
tolimar 1480 accepts (2.908 per day) 93 rejects (0.183 per day) 84 comments (0.165 per day)
twerner 278 accepts (0.546 per day) 53 rejects (0.104 per day) 26 comments (0.051 per day)


Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:
  1. Debian Perl Group (559 accepts)
  2. Debian Haskell Group (491 accepts)
  3. Debian Ruby Extras Maintainers (285 accepts)
  4. Debian Java Maintainers (257 accepts)
  5. Debian Med Packaging Team (164 accepts)
  6. Debian Multimedia Maintainers (160 accepts)
  7. Debian Fonts Task Force (156 accepts)
  8. Debian Javascript Maintainers (137 accepts)
  9. Debian Python Modules Team (129 accepts)
  10. Debian Qt/KDE Maintainers (98 accepts)
That doesn t reflect the real developers, though. Here s our Changed-By top ten:
  1. Clint Adams (216 accepts)
  2. Jonas Smedegaard (208 accepts)
  3. Ben Hutchings (203 accepts)
  4. Joachim Breitner (153 accepts)
  5. TANIGUCHI Takaki (112 accepts)
  6. Alessio Treglia (101 accepts)
  7. David Paleino (95 accepts)
  8. Nicholas Bamber (76 accepts)
  9. Mathieu Parent (68 accepts)
  10. Jeff Breidenbach (63 accepts)
Clint rocks with tons of Haskell packages, followed by Jonas (mostly Perl packages), and Ben (kernel uploads). Italian cabal stands still, with Alessio and David respectively at 6th and 7th place ;)


How long does a package stay in NEW? Some more, some less, but the average is 3 days, 15 minutes and 21 seconds. Now go and check your dak mails to see whether you had a fast processing or not :) liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never ;)


FTP Team is not just accepting NEW packages, but also removing obsolete ones. Here are some details about this task:

FTP Team also took care of override changes:

29 June 2012

Jo Shields: How to build a bungalow begin with the chimney

Whilst nobody was looking, Mono became the most important framework in the world of games. And I mean that with only a smidge more than my usual levels of hyperbole. It really is fantastically important, as far as game development goes. I blogged a long time ago about its use inside EA s multi-million-selling The Sims 3. And the massive performance gains in Second Life when they switched from their in-house scripting language to Mono are well documented. But I m not talking about those, I m talking about the now, the recent stuff. First up is Unity 3D. Not to be confused with the Ubuntu user interface, Unity 3D is a proprietary game development tool which began life for Mac, and has now managed to become the single most popular game engine amongst smartphone developers. It targets a bunch of new platforms not just Android and Windows and iOS, but in the new 4.0 release, Linux too for the first time. And every Unity 3D game is a Mono game the core of Unity is a fork of Mono, which enables it to have the performance and ease of development which it does. A reasonably large percentage of the cross-platform games on Kickstarter, such as Wasteland 2 and Super Retro Squad, are cross-platform because Unity 3D makes it easy and each of those games is powered by Mono. However, I have better things to do today than gush about a proprietary tool. No, far more important are the Free ways to develop games. So today I m going to talk about MonoGame. MonoGame is an implementation of Microsoft XNA a high level framework for developing in .NET languages, originally designed to allow indie developers access to the Xbox 360. XNA covers many of the bases covered by SDL sound, graphics, input, and so on, but can go onto platforms where SDL isn t an option, such as the aforementioned Xbox 360. And with MonoGame, any game written for XNA is a game which can not only run on Microsoft s approved platforms Windows, Xbox 360, Windows Phone but also on other platforms such as Mac OS X, iPhone, iPad, Android, and Linux. Cross-platform is good. More games are good. And MonoGame enables every Xbox 360 indie developer to throw a Linux port out of the door with a relative minimum of fuss. There are 2,521 games on the Xbox Live Indie Games store (games by individuals or small groups, with no major publisher) and each of those can become a game for Debian and Ubuntu with very little work. Not to mention the games with actual publishers (some portion of the 492 games on XBLA), not to mention the mobile games written for Windows Phone, Android and iOS. All of these developers have an easy route to Linux releases, as long as the effort remains low. And with today s publishing in the Debian archive of MonoGame 2.5.1, the effort is as low as can be. Anyone who purchased the recent Humble Indie Bundle 5, and played Bastion, has seen MonoGame in action Bastion began life as an XNA game for Xbox 360, and the Linux port was made possible only by MonoGame. Bastion s 100+ industry awards should indicate the possibilities open to a creative mind with decent developer tools. MonoGame is Free Software, built on top of other Free Software, with an active upstream development team genuinely responsive to the needs of Linux distributions. It s well worth a look not just for porting, but for new projects too. And I m delighted that it will be available to all users of Debian 7.0, as well as Ubuntu 12.10.

28 June 2012

DebConf team: DebConf12 schedule ready! (Posted by Gunnar Wolf)

The DebConf12 schedule is finally available! As usual, the DebConf conference can only be finalized a couple of days before DebConf itself starts. But hey Take a look at it, and start planning your activities! Not final, never final Remember that, even now that the conference schedule is announced, you can still submit talks to DebConf, following what we call ad-hoc talks. What are the steps to do it?
  1. Check the schedule and find the timeslot (day, hour and room) you want to talk in
  2. Register your event in Pentabarf, and note the generated event ID (the last number in the URL after the submission is accepted)
  3. Send a mail to schedule@debconf.org detailing the event ID, title (just to confirm it), and the requested timeslot
  4. All requests _must_ be received at least 24hr before the requested event for the scheduling team to act accordingly and for the video team to be able to plan on resources to cover the requested session
  5. Video team coverage (event recording plus streaming on a best-effort basis) will be given to activities scheduled on the Aula Magna and Roberto Ter n auditoriums.
  6. We will try to grant requests on a first-come-first-serve basis. If two events are requested for the same timeslot, the first will have precedence, and we will schedule the second as close as possible to it. We will notify the submitters if this happens, of course.
Conference tracks In DebConf12, many talks are organized around the following tracks:
DebianDay
This track is meant for the local (Nicaraguan/Central American) community to be attracted to Debian. It does not usually have technically minded talks; the DebConf general audience is, of course, invited to take part in it, as there will surely be interesting angles on how Debian is used in different use-cases, and there will be interesting Q&A session with the audience.
Getting involved in Debian
Central America is a very special region. It has vibrant Free Software user groups and communities, among which there is an important interest to get involved in Debian.
This track will have some talks in Spanish, some talks in English, explaining different Debian processes and tools, aiming at attracting locals to contribute to Debian.
Debian for the cloud
All about virtualization. Management infrastructure, details on virtual machine engines, insight on how to tweak your kernel for optimum virtualized performance, and of course how to get it all humming perfectly with your Debian packages.
Building and porting
With an every day stronger transit to mobile computing, we are seeing a renaissance of porting. At the same time, we finally have multiarch as part of Debian. And were that not to be enough, we finally have more than one compiler that can be used for most of the software in the archive. Put all this together, and you will have a very interesting track.
Social activities
DebConf is as much about technical stuff as it is about building closer ties within the members of its community. Eating cheese+wine, going on a trip to the guts of a volcano, having a nice banquet Do you want to add an activity to this pseudo-track?
Skills exchange
Knowledge can also be built by all of us putting our grain of sand. We first had explicit skills exchange sessions in Bosnia last year, and they were very well received. Feel free to propose similar sessions!
And Well, three days to DebCamp and counting. See you Real Soon Now in Managua!

Sorina Sandu: Status update

Hello!

First of all: I'm going to DebConf :D and I really want to say a big "Thank you!" to every one who made this possible.


So, here's what I've done recently:

I spent some time putting everything in order for the departure to Nicaragua, things like finding a plane ticket through a route that does not require a Visa, making a new Passport, now all is in order.

I've integrated the changes for listing available ESSIDs, in order to close #637128 [0] into a separate branch , which can be found here [1].

The plan is to have only one new template which lists the available networks and which has a option for entering the ESSID manualy. In order to do so, a simple scan is performed on the wireless interface, and the configuration for a particular network resulted after the scan is kept if that one is chosen. The enter manually situation is treated exactly as it was before.

Furthermore, last week (or the week before) I've done some coding style fixes again in a separate branch [2], nothing very fancy, just trimming trailing white spaces and changing some tabs to spaces, where spaces were mostly used. I've also noticed some inconsistencies, regarding white spaces between functions names and the parenthesis. However I think I'm overreacting a little, those matters are not really that relevant, but when I'll have some free time I'll get to this as well.


Next in line: Issues

Testing is still giving me some troubles, the show essids part is not yet fully tested, so I couldn't say for sure if everything works as planned.

Apart from the classical: write the image to an USB and reboot method, I'm using Virtual Box for testing. I've managed to create a passthrough for the wireless interface, but now I have troubles with some firmware. Hopefully I'll solve this as soon as possible since it would really make my life easier.


Future plans:

I've realized that I've underestimated a little what the project implies and the amount of information that is new to me. I feel that I have learned a lot and that there is still a lot to learn and I'm very happy that it is so. I've now started documenting regarding IPv6, beginning with this [3]. We haven't decided yet what the next task will be exactly.


[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637128
[1] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/show_essids
[2] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/coding_style
[3] http://tldp.org/HOWTO/Linux+IPv6-HOWTO/

27 June 2012

Michal &#268;iha&#345;: Weblate - over 20000 translations in 3 months

Today, I wondered how much translations were done using Weblate since it was born. Of course I can give only numbers for services I maintain, the biggest one being translation server for phpMyAdmin, Gammu and Weblate. Looking at the logs, the server l10n.cihar.com helped to contribute over 20000 (actually little bit more as tracked are only changes after releasing Weblate 0.8), what is quite impressive number. Of course biggest amount of contributions came in time of preparing phpMyAdmin 3.5.0 release, where I gave it huge amount of publicity (AKA spamming my blog), but number of people accessing the website still grows since then (still number of submitted translations is slightly lower). Now as 1.0 release proved to be quite stable, it's time to focus on 1.1 development, which will bring offloading of fulltext indexing (it was proven to be unreliable to update index online for bigger sites). Almost all issues targeted for 1.1 are already fixed and all what needs to be done is testing in real life situations.

Filed under: English Phpmyadmin Weblate 0 comments Flattr this!

Next.

Previous.