Search Results: "spa"

24 July 2013

Eddy Petrișor: HOWTO: git - change branch without touching working copy (at all)

Did you ever had the need in a git repository to change to another branch without altering AT ALL the working copy and ever wondered how that's done?

Usual use cases might be when you mde some changes to the working copy thinking you were on another branch, or you double-track in git a directory which is also tracked by another VCS (e.g. ClearCase).

What you need, in fact, is to update the index and not touch the working copy. The command that does that is

git read-tree otherbranch
If you also need to commit the state of your working tree to the otherbranch, you also need to tell git to actually associate the curent HEAD with the branch you just switched to:
git symbolic-ref HEAD refs/heads/otherbranch
I use this approach at my work place* to develop/experiment with possible code improvements on my machine before considering the merge into the official code.

* The preferred VCS is (Base) ClearCase, and I keep a git repository over the relevant part of the project in the ClerCase Dynamic View, so for synchronisations, the files in the working copy are updated by ClearCase and I have to resync my git branch (clearcaseint) following the latest official code from time to time, so I can pull in my local disk git repository the clearcaseint branch and merge it with my experimental changes in my git feature branches.

If people are curious about how I work with ClearCase and git, I can expand on this in another post.

Eddy Petrișor: HOWTO: git - remove those pesky remotes/origin/

Often people create branches to fix various bugs or implement various features, and publish those to a public repository to be pulled in the official repository (e.g. pull requests in the github model).

The problem is that after the fix is merged the local repo still has the branch that tracks the origin (your public repository).

To remove the local branch it is simple:

git branch -d feature_x

To remove the actual branch in the remote repository:

git push origin --delete feature_x

But, still the local tracking branches remain, those pesky 'remotes/origin/feature_x'. Here is an example from one of my repos:

$ git branch -a
activestate
master
next
* py.test
sysplatformtest
remotes/activestate/master
remotes/origin/HEAD -> origin/master
remotes/origin/appauthor-fallback
remotes/origin/linux-fixes
remotes/origin/master
remotes/origin/next
remotes/origin/py.test
remotes/origin/root-directories
remotes/origin/sysplatformtest
remotes/origin/wrapper-defaults

All of those can be removed with this command:

git remote prune origin

And the output for my appdirs repository is:

Pruning origin
URL: git@github.com:eddyp/appdirs.git
* [pruned] origin/appauthor-fallback
* [pruned] origin/linux-fixes
* [pruned] origin/next
* [pruned] origin/root-directories
* [pruned] origin/sysplatformtest
* [pruned] origin/wrapper-defaults
Which, finally removes those damned refs:

$ git branch -a
activestate
* master
py.test
remotes/activestate/master
remotes/origin/HEAD -> origin/master
remotes/origin/master
remotes/origin/py.test
Great!

Update 2013-07-24: There have been also other solutions proposed in the comments section. I am quoting them here for better visibility.

Besides the solution I proposed, people commented about other alternatives for pruning remote branches, by using git fetch --prune or by using git-push's --prune option.

Brice provided a method to remove specific branches one by one with git branch -dr command:
git branch -dr origin/foo
What is ironic is that in the past I have used this format, and before writing the original article I was looking for this, but haven't found it that easily. I hope this article will help some other poor soul or myself in the future.

23 July 2013

Daniel Pocock: If Assange is a rapist, what type of monsters are running Australia?

It's almost one year since Julian Assange was granted asylum by the republic of Ecuador. Newspapers often incorrectly report that he sought political asylum to avoid those disturbing rape allegations (they are not charges). The real basis of his claim to asylum, which is the foundation that asylum law is based on, is that Mr Assange is not safe from political persecution in his own country. Let's not talk about rape It's often difficult for men to talk about this subject without putting their foot in their mouth. In this case, it may be easier to understand by trying to put it in context. Safer with Mr Assange?
Just last week, our Government announced plans to expand their program of dumping poor coloured people into random countries around the Asia-Pacific region. On Monday they broke every rule in the UN's book of human rights by using the Internet to distribute gut wrenching youtube videos of a woman suffering at the hands of our immigration officers. They hope this degradation will scare away other migrants who do not meet the right economic criteria. It makes me wonder, if Mr Assange is genuinely guilty of rape despite his alleged victim boasting about her experience on Twitter, what has happened to this other woman in the care of Kevin Rudd's scorched-Earth immigration policy? Maybe because she's not a Swedish blonde, nobody cares. More racist videos from the immigration department fail to correctly inform migrants of their right to seek asylum under international law - is it possible that this offensive and misleading propaganda violates Youtube's terms of service? Crimes so bad that whistle-blowers are inevitable Former employees of the Manus Island concentration camp have stepped forward and revealed what the Government naturally doesn't want us to know. According to one former security manager, vulnerable migrants had suffered virtually every form of physical abuse, including rapes, and under orders from management in Australia, the victims were simply locked up with their abusers and no formal complaints were documented. All of a sudden, those pictures of the woman crying in the degrading video made by immigration officials take on a new meaning: she is next in line for the same destination, PNG. I somewhat suspect she would enjoy greater dignity and safety confined to Ecuador's London embassy with the so-called rapist that everybody has been distracted with. Can anybody imagine the widely discussed expression on her face improving after 20 months enforced confinement with rapists under the protection of Australia's refugee program?

Ritesh Raj Sarraf: Power consumption on Linux 3.10

The power consumption on the Linux kernel 3.10 is pretty bad. On kernel 3.10, with the follwing config, the PowerTop results are:
#
# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
# CONFIG_NO_HZ_FULL is not set
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
PowerTOP v2.0 Overview Idle stats Frequency stats Device stats Tunables
The battery reports a discharge rate of 28.0 W
The estimated remaining time is 23 minutes
Summary: 1785.5 wakeups/second, 0.0 GPU ops/second, 0.0 VFS ops/sec and 22.1% CPU use
Power est. Usage Events/s Category Description
16.3 W 2915 rpm Device Laptop fan
5.11 W 100.0% Device USB device: WALTON Primo-X1 Primo-X1
1.70 W 33.3% Device Display backlight
849 mW 33.3% Device Display backlight
425 mW 86.0 ms/s 330.7 Process /usr/bin/konsole
316 mW 63.9 ms/s 66.1 Process /usr/bin/plasma-desktop
142 mW 28.6 ms/s 396.8 Process /usr/bin/X :0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7
64.1 mW 13.0 ms/s 198.4 Process kwin -session 101261418fe3000136103713100000053880000_13746081
53.6 mW 10.8 ms/s 0.00 Process powertop
35.9 mW 7.3 ms/s 66.1 Process /usr/lib/chromium/chromium --type=plugin --plugin-path=/usr/li
24.3 mW 4.9 ms/s 396.8 Interrupt PS/2 Touchpad / Keyboard / Mouse
6.92 mW 1.4 ms/s 0.00 Interrupt [48] i915
5.94 mW 1.2 ms/s 66.1 Interrupt [9] RCU(softirq)
3.98 mW 0.8 ms/s 0.00 kWork flush_to_ldisc
3.78 mW 0.8 ms/s 66.1 Process [ksoftirqd/2]
3.33 mW 673.3 us/s 66.1 Process [rcu_sched]
1.80 mW 363.1 us/s 66.1 Interrupt [1] timer(softirq)
1.79 mW 363.0 us/s 0.00 Process [ksoftirqd/4]
Where as on the 3.9 kernel:
The battery reports a discharge rate of 13.2 W
The estimated remaining time is 43 minutes
Summary: 611.5 wakeups/second, 0.0 GPU ops/second, 0.0 VFS ops/sec and 14.2% CPU use
Power est. Usage Events/s Category Description
14.0 W 2722 rpm Device Laptop fan
1.72 W 33.3% Device Display backlight
862 mW 33.3% Device Display backlight
255 mW 65.7 ms/s 58.0 Process /usr/bin/plasma-desktop
91.9 mW 23.7 ms/s 27.5 Process /usr/lib/chromium/chromium --type=renderer --lang=en-US --forc
60.1 mW 15.5 ms/s 96.1 Process /usr/lib/chromium/chromium --type=plugin --plugin-path=/usr/li
25.0 mW 6.4 ms/s 25.1 Process kwin -session 101261418fe3000136103713100000053880000_13746094
21.5 mW 5.6 ms/s 34.2 Process /usr/bin/X :0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7
13.1 mW 3.4 ms/s 5.6 Process /usr/bin/konsole
9.82 mW 2.5 ms/s 53.7 Process [irq/48-iwlwifi]
9.11 mW 2.4 ms/s 2.2 Process /usr/bin/knemo
8.62 mW 2.2 ms/s 12.3 Process /usr/lib/chromium/chromium --password-store=detect
8.32 mW 2.1 ms/s 45.5 Interrupt [48] iwlwifi
6.96 mW 1.8 ms/s 35.3 Interrupt [7] sched(softirq)
5.13 mW 1.3 ms/s 57.1 Interrupt [47] i915
4.24 mW 1.1 ms/s 0.4 Process powertop
3.38 mW 0.9 ms/s 1.5 Timer tcp_keepalive_timer
2.85 mW 0.7 ms/s 11.9 Process /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plu
<section class="field field-name-taxonomy-vocabulary-1 field-type-taxonomy-term-reference field-label-above view-mode-rss">

Categories: </section><section class="field field-name-taxonomy-vocabulary-2 field-type-taxonomy-term-reference field-label-above view-mode-rss">

Keywords: </section>

Gunnar Wolf: Sandro Cohen: The cyclist's zen

For all Spanish-speakers that read my blog, specially for the cyclists among you, and most specially for those that dwell in Mexico City's streets: I was recently pointed to a project started inside the Faceboook labrynth by Sandro Cohen, writer and academic: El zen del ciclista urbano. I met Sandro around twenty years ago. He writes in a very good, simple style. What I didn't know until now is that he has also become an urban cyclism promotor, just as me and many of my friends. In this page he started, he posts snippets on the topic of being an urban cyclist: As of today, he has 44 meditations, each of them a joy to read And very instructive as well. Thanks, Sandro, for the great resource! [update] I always find it... almost funny to read comments by so many people saying they'd rather have a lobotomy than to cycle in Mexico City. Hey! Mexico City is among the best places for cycling! Yes, we have to keep our eyes open and our instincts awake, but... Most of the city's area is flat. Many avenues have wide lanes and span a long distance. And yes, although there are some careless or aggressive drivers, after six years with the joy between my legs I can just say that... things are not as bad as you might imagine. I have very few (thankfully!) bad experiences, and so, so many good ones!

Keith Packard: present sync

Implementing Vblank Synchronization in the Present Extension This is mostly a status update on how the Present extension is doing; the big news this week is that I ve finished implementing vblank synchronized blts and flips, and things seem to be working quite well. Vblank Synchronized Blts The goal here is to have the hardware executing the blt operation in such a way as to avoid any tearing artifacts. In current drivers, there are essentially two different ways to make this happen:
  1. Insert a command into the ring which blocks execution until a suitable time immediately preceding the blt operation.
  2. Queue the blt operation at vblank time so that it executes before the scanout starts.
Option 1. provides the fewest artifacts; if the hardware can blt faster than scanout, there shouldn t ever be anything untoward visible on the screen. However, it also blocks future command execution within the same context. For example, if two vblank synchronized blts are queued at the same time, its possible for the second blt to be delayed by yet another frame, causing both applications to run at half of the frame rate. Option 2. avoids blocking the hardware, allowing for ongoing operations to proceed on the hardware without waiting for the synchronized blt to complete. However, it can cause artifacts if the delay from the vblank event to the eventual execution of the blt command is too long. Queuing the blt right when it needs to execute means that we also have the opportunity to skip some blts; if the application presents two buffers within the same frame time, the blt of the first buffer can be skipped, saving memory bandwidth and time. Present uses Option 2, which may occasionally cause a tearing artifact, but avoids slowing down applications while allowing the X server to discard overlapping blt operations when possible. Queuing the Blt at Vblank There are several options for getting the blt queued and executed when the vblank occurs:
  1. Queue the blt from the interrupt handler
  2. Queue the blt from a kernel thread running in response to the interrupt
  3. Send an event up to user space and have the X server construct the blt command.
These are listed in increasing maximum latency, but also in decreasing complexity. Option 1. is made more complicated as much of the work necessary to get a command queued to the hardware cannot be done from interrupt context. One can imagine having the desired command already present in the ring buffer and have the interrupt handler simply move the ring tail pointer value. Future operations to be queued before the vblank operation could then re-write the ring as necessary. A queued operation could also be adjusted by the X server as necessary to keep it correct across changes to the window system state. Option 2. is similar, but the kernel implementation should be quite a bit simpler as the queuing operation is done in process context and can use the existing driver infrastructure. For the X server, this is the same as Option 1, requiring that it construct a queued blt operation and deliver that to the kernel, and then revoke and re-queue if the X server state changed before the operation was completed. Option 3 is the simplest of all, requiring no changes within the kernel and few within the X server. The X server waits to receive a vblank notification event for the appropriate frame and then simply invokes existing mechanisms to construct and queue the blt operation to the kernel. Oddly, Present currently uses Option 3. If that proves to generate too many display artifacts, we can come back and change the code to try something more complicated. Flipping the Frame Buffer Taking advantage of the hardware s ability to quickly shift scanout from one chunk of memory to another is critical to providing efficient buffer presentation within the X server. It is slightly more complicated to implement than simply copying data to the current scanout buffer for a few reasons:
  1. The presented pixmap is owned by the application, and so it shouldn t be used except when the presented window covers the whole screen. When the window gets reconfigured, we end up copying the window s pixmap to the regular screen pixmap.
  2. The kernel flipping API is asynchronous, and doesn t provide any abort mechanism. This isn t usually much of an issue; we simply delay reporting the actual time of flip until the kernel sends the notification event to the X server. However, if the window is reconfigured or destroyed while the flip is still pending, cleaning that up must wait until the flip has finished.
  3. The applications buffer remains busy until it is no longer being used for scanout; that means that applications will have to be aware of this and ensure that they don t deadlock waiting for the current scanout buffer to become idle before switching to a new scanout buffer.
Present is different from DRI2 in using application-allocated buffers for this operation. For DRI2, when flipping to a window buffer, that buffer becomes the screen pixmap the driver flips the new buffer object into the screen pixmap and releases the previous buffer object for other use. For Present, as the buffer is owned by the application, I figured it would be better to switch back to the real screen buffer when necessary. This also means that applications aren t left holding a handle to the frame buffer, which seems like it might be a nice feature. The hardest part of this work was dealing with client and server shutdown, dealing with objects getting deleted in random orders while other data structures retained references. (The kernel DRM drivers use the term page flipping to mean an atomic flip between one frame buffer and another, generally implemented by simply switching the address used for the scanout buffer. I d like to avoid using the word page in this context as we re not flipping memory pages individually, but rather a huge group of memory that forms an entire frame buffer. We could use plane flipping (as intel docs do), frame buffer flipping (but that s a mouthful), display flipping or almost anything but page flipping ). Overall DRI3000 Status At this point, the DRI3 extension is complete and the Present extension is largely complete, except for redirection for compositors. The few piglit tests for GLX_OML_sync_control all pass now, which is at least better than DRI2 does. I think I ve effectively replicated the essential features of DRI2 while offering room to implement a couple of new GL extensions: The code for this stuff has all been pushed to a number of repositories:

22 July 2013

Daniel Pocock: Building existing autotools C/C++ projects on Android

I recently came across a requirement to build a project involving C++ code on Android. I've done this before with small C projects, for example, patching the JNI code in Lumicall. Google supplies a toolchain dubbed the Native Development Kit (NDK) for Android. Using the NDK typically involves using it's custom build system, usually duplicating all Makefile content into a file called Android.mk in each subdirectory. For large projects, that duplication doesn't sound like fun to implement and it definitely doesn't sound like fun keeping it up to date. reSIProcate: 39 Makefiles + dependencies reSIProcate is quite a large project involving multiple shared libraries, some useful demonstration apps, a SIP proxy (repro) and a TURN server (reTurn). There are 39 Makefiles and numerous dependencies. There are numerous test cases for each library and sub-project too. Sounds like a perfect fit for Android Alternative ways to proceed The Android NDK documentation pack describes a process for bypassing Android.mk files, invoking the toolchain (compiler, linker, etc) directly to build standalone apps with their own Makefile system I felt this was worth a go Two approaches are described. The first approach is to simply reference toolchain binaries (e.g. arm-linux-androideabi-gcc) using the well-known environment variables such as CC and LD. The documentation warns that in this case, support for the Standard Template Library (STL) is extremely limited. Nonetheless, I decided to try this first and found that I couldn't easily link binaries with the full STL feature set required by reSIProcate. I moved on to the second approach. The second approach involves calling the script $ NDK_HOME /build/tools/make-standalone-toolchain.sh to export a custom toolchain directory tree with full STL support. Using a little more disk space, but not so hard. Initially I had NDK version r7 on my system and I found that make-standalone-toolchain.sh was not working reliably. It appeared to complete successfully (exit status = 0) but in reality it was not copying the STL components to the custom directory. I upgraded to version r8e and the problem was resolved. Making it easy Finally, I wrote a little wrapper script that can do everything: build the custom toolchain, set up all those environment variables for the tools and then invoke the configure script with some of the special settings that are required for a cross compile. Coding problems To gain some confidence in this progress, I had already run the full reSIProcate build, including test cases, on all hosts in the Debian build farm, including some ARM hosts. This helped confirm that the code is correct for such CPUs as well as helping to identify some previously unknown errors on the more typical platforms. Nonetheless, some minor problems appeared with the build for Android:
  • Android's pthread capabilities are built into the C library so -lpthread must not be included on the linker command line. This is easily adapted using Makefile variables.
  • Although Android is Linux, some of the header locations are not quite the same. For example, references to sys/fcntl.h must be wrapped in a pre-processor conditional that uses just the path fcntl.h when __ANDROID__ is defined
After fixing a few of these minor issues, everything fell into place and the build was successful. High quality portable code The reSIProcate project has a strong tradition of providing cross-platform support. The code quality is quite good and well documented. Despite the project's large size and complicated build system, there was already a good chance of success. This may not be the case for every C/C++ project, especially those making extensive use of other shared libraries. Building the dependencies: OpenSSL on Android For the core features I wanted to use, I only needed one major dependency, the OpenSSL project. libcrypto and libssl already appear in Android. Nonetheless, their headers are not present in the NDK tree and therefore their API (and compile-time flags) can not be considered 100% stable across all handsets past and future. The solution is to build a standalone libcrypto and libssl and ship them with your own app. Fortunately this has been done before: I grabbed the code from Guardian Project's OpenSSL in github and found that it built immediately with my NDK version. To use it, simply make sure that CPPFLAGS and LDFLAGS include the necessary references to OpenSSL, as demonstrated in my script Test cases I found that make -i check would enable me to build all test cases on the build host. Naturally it would fail to run any of them as they are ARM binaries. Using -i in the make command line makes it possible to build them all anyway, ignoring the failures, for running them later. Copying the test* binaries from the .libs directories to the phone, I was able to run them immediately What next There are three further steps in this work:
  • Generalising the script to make binaries for all supported Android architectures (not just ARM)
  • Extending the script to copy the shared libraries into the build tree of a traditional Android project where they will be automatically scooped up when the apk file is assembled.
  • Writing the necessary JNI code to access the C/C++ code from Java

Daniel Pocock: Winning at any cost

It's not every day that a student messing around with keystroke loggers comes to fame through slashdot. Nonetheless, systematically rigging an election and getting sentenced to 12 months in a dorm with bars has helped raise 22 year old Matthew Weaver's profile well above that of the average script kiddie. Now let's stop and reflect on poor Weaver's future. You may be thinking that with an exchange program like this on his academic record he won't be so popular with employers. Given that he was busted by campus security rather than the FBI he won't even attract the interest of those companies who hire ex-hackers. So where could he go? How is it done in Australia? Not too long ago, when I was a student myself, one of our prominent universities was subjected to a very similar scam. Four members of the Tin Tin for NUS ticket at La Trobe University were implicated in stuffing the ballot the old fashioned way. The incidents even share the characteristics of chronic stupidity: just as Weaver had been caught voting for himself 259 times from the same IP address in a campus computer lab, team Tin Tin had tried to hand their bag of manipulated postal votes directly to the deputy returning officer rather than discretely posting them through the internal mail. According to an official report by the Deputy Returning Officer, Karsten Haley, all four candidates were charged with Dishonest Conduct and Interfering with Ballot Papers. Unfortunately, the report notes that
La Trobe University SRC Electoral Regulations do not empower the Returning Officer or Deputy to enforce charges or disciplinary procedures and the charges were never faced by the accused.
Given the seriousness of the matter, Haley did not give up his attempts to hold them to account. He escalated it to the Dean of the college and then to the University Secretary. He reports that "their disinterest was extraordinary" and that nobody would involve the police. Young Labor suspended Just over a year later, in 1997, the ALP's youth division for the state of Victoria, Young Labor, was suspended after attempts to rig the ballot to elect the Young Labor leadership team. The guilty parties were never publicly named. Nobody was formally suspended or expelled and this simply left them with more time on their hands to invest their energy in other elections. The suspension of Victorian Young Labor remained in effect for a number of years. The specific allegations about the Young Labor ballot suggest that those people particularly keen to win had printed fake student cards and given them to stooges who would impersonate other Young Labor members who had not attended to vote in person. Where are they now? It is no co-incidence that these students were (and still are) members of Labor Unity, a powerful faction within Australia's ruling Labor Party, the ALP. Most political organisations would presumably express concern about these allegations. The ALP does things differently. One of the students who withdrew his nomination in La Trobe, Mr Larocca, subsequently became Mayor in the City of Moreland, one of the ALP's strongholds. Even more remarkably from an outsider's viewpoint, another of these figures, Stephen Donnelly, is currently employed as the Assistant State Secretary of the ALP in Victoria. Communications like this newsletter reveal that he is one of the key figures in the party's pre-selection process. He has recently been appointed to direct the ALP's 2013 federal election campaign for the state of Victoria. Another co-incidence On the same weekend that Weaver was in the news for his antics, Donnelly's latest employer, the ALP's Victorian branch, was conducting pre-selection ballots to choose candidates for the upcoming federal election. So it's no surprise that Monday's newspaper headlines report fresh allegations of voting irregularities. Sadly, I've seen some of Labor Unity's bad behavior first hand. About 10 years ago I was living in South Melbourne, which is in the federal electoral district of Melbourne Ports. A young female friend of mine, a member of the local Elwood branch of the ALP, had spent election day handing out brochures for an ALP candidate in a marginal seat rather than assisting the controversial local ALP candidate, Michael Danby. A few days later I was witness to an incident where Danby aggressively confronted this young woman and demanded to know why he hadn't seen her handing out his own leaflets on polling day. He stood within centimeters of her and was literally looking down on her as he demanded some kind of apology to sooth his bruised ego. She looked terrified and barely responded. Within moments one of his handlers approached and physically moved Danby away from this young woman, I dare to think where things would have gone otherwise. Eye for talent Remarkably, at the same time, the infamous Stephen Donnelly had started shadowing Danby in his movements about the district. Fresh out of university, his talents had been recognised by Danby and he was employed in Danby's office, enabling him to continue honing his skills on a full-time basis with a tax-payer funded salary. What a remarkable contrast to the story of Weaver. Can anybody imagine a US congressman collecting Weaver from the prison gates and deploying him to an office on Capitol Hill? The biggest bankruptcy in student history Around the same time, Donnelly's Student Unity, the student arm of Labor Unity were successful in taking over the student union of my own campus, the University of Melbourne. Not long after I graduated I heard that they had been accused of skimming off $1 million from catering providers and a high-risk $46 million property transaction that put the organisation into liquidation. Unlike Mr Weaver, who's scheme at Cal State barely got off the ground, none of those involved in the Melbourne University incident has faced criminal proceedings. One ALP figure, Andrew Landeryou, spent several months in Costa Rica while wanted for questioning in the Supreme Court. His wife has just been endorsed for a seat in the Senate with support from various Labor Unity figures including Danby. The Gillard questions In 1996, around the same time that Donnelly & Co. were romping around student unions learning the tricks of the political trade, a lawyer quietly departed from the firm Slater and Gorden after an internal investigation into a property transaction linked to a union slush fund. Like Donnelly, this lawyer's next move was to take employment in the office of a Labor Party MP. More recently she was backed by Labor Unity to become Prime Minister. The union slush fund remains under investigation, frustrated by the disappearance of documents. The $60 million heist Recently I blogged about Gillard and Abbott, leaders of the two main political parties in Australia, agreeing to take $60 million of taxpayer money to fund their parties' campaigns in the upcoming federal election, giving themselves an obscenely unfair unadvantage over all other contestants. Where would that money end up? In the case of the ALP, does it appear likely that figures like the Victorian ALP's federal campaign director, Mr Donnelly, would be involved in the expenditure? National shame With this background, it becomes easier to understand the quality (or lack of it) in Australia's national leadership. When you consider that the generation responsible for the La Trobe incident, the Young Labor suspension and the MUSU bankruptcy are now growing into positions of greater responsibility in the ALP it leaves me feeling the quality of leadership is only going to get a lot worse before it starts getting better. For example, the recent incident where coloured people were fed to the sharks has nothing to do with the worldwide refugee crisis and everything to do with maintaining the dumbed-down level of political discourse that Labor Unity thugs and their followers can cope with. Real issues like climate change and energy policy, for example, appear to be beyond the pay grade of Australia's political class Ranjini - coloured, indefinite detention It is startling that up to her own recent demise, Gillard herself had repeatedly begged the public to stop asking questions about her own past and remember that Labor politicians are innocent until proven guilty - yet she had a pregnant coloured woman thrown into a concentration camp on unfounded fears about "national security". No evidence has ever been presented that poor Ranjini committed a crime, but the houses bought with money from trade unions, transactions handled through Gillard's own office, seem to be as solid as bricks and mortar. If only poor Matthew Weaver had been an Australian, how much further would his star have risen? Update: please sign the petition at change.org asking La Trobe university to re-examine the report and refer it formally to the police. If you are concerned about the plight of poor Ranjini and other people subject to Australia's domestic rendition program, please take a moment to see Letters for Ranjini

21 July 2013

Wouter Verhelst: Two kings and three queens

Now that King Albert has abdicated and Prince (now King) Philippe has taken the oath of office, Belgium has two Kings and three Queens; yes, Albert has abdicated, but in Belgian tradition he retains the title of King, and his spouse, Queen Paola, retains the title of Queen. The third Queen, Fabiola, is the widow of late King Baudouin. As someone pointed out to me, that makes a full house. Does that make Philippe the King of Spades, or the King of Hearts? Or perhaps it's neither, and it's the King of Diamonds

Rog rio Brito: Beginning Ruby

These are some very rough/sketchy notes that I took while doing the exercises on the introductory track of the Ruby language at codeacademy. They were written from the point-of-view of an amateur programmer in C and Python (read: "me"). Since this is my first time with Ruby and I only have about 1 week of experience with it, these notes are likely to contain many mistakes, many simplifications, many omissions or simply gross errors. I would love to be informed of such facts so that I can fix them and not write code that is burdensome for others to maintain. Operators Numbers Basic I/O Strings Control flow If While For Loop Arrays Hashes Functions/methods Symbols Comments Lambda functions Classes Modules Mixins Conventions

20 July 2013

Eddy Petri&#537;or: (Not a) GNU Make quirk, or why logs should be provided

About two months ago I was writing about a quirk I found in GNU Make related to the $(patsubst ) function.

I have just tried this on my Debian Wheezy laptop which has make 3.81, but I wasn't able to reproduce the issue with the version from Debian (3.81-8.2).

The makefile looks like this:
PATH := ../some/prefixCPU12suf/include
CPUINC := $(patsubst ../some/prefix%,%,$(PATH))
CPU := $(patsubst %/include,%,$(CPUINC))

default:
@echo "PATH = $(PATH)"
@echo "CPUINC = $(CPUINC)"
@echo "CPU = $(CPU)"
And the result was correct:
0 eddy@heidi /tmp $ make
PATH = ../some/prefixCPU12/include
CPUINC = CPU12/include
CPU = CPU12
0 eddy@heidi /tmp $ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-pc-linux-gnu
The worst part is that I know I tested this issue on 3.82 on Cygwin and on Linux with the 3.82 version and it failed, but I wasn't able to remember how I did it. I started searching through the directory where I knew there could be the test makefile, I wasn't able to find it, until I remembered what I was trying to achieve.

From a path like ../some/prefixCPU12suf/include I wanted to use % to remove the parts 'some/prefix' and 'suf/include' because in the directory ../CPU12 there were some files that needed to be processed.

The actual issue is that GNU Make's '%' is not analogous to shell's '*', so that means code like this does not work as I assumed anf the 'pref' part is not an anchor:


PATH := ../some/prefCPU12suf/include
CPUINC := $(patsubst pref%,%,$(PATH))
CPU := $(patsubst %suf/include,%,$(CPUINC))

default:
@echo "PATH = $(PATH)"
@echo "CPUINC = $(CPUINC)"
@echo "CPU = $(CPU)"
Which leads to these results, no matter the version:

0 eddy@heidi ~/usr/src/make/make-profiler/make-3.82 $ ./make -f /tmp/makefile
PATH = ../some/prefCPU12suf/include
CPUINC = ../some/prefCPU12suf/include
CPU = ../some/prefCPU12
0 eddy@heidi ~/usr/src/make/make-profiler/make-3.82 $ ./make --version
GNU Make 3.82
Built for x86_64-unknown-linux-gnu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http: gnu.org="" gpl.html="" licenses="">
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
0 eddy@heidi ~/usr/src/make/make-profiler/make-3.82 $ make -f /tmp/makefile
PATH = ../some/prefCPU12suf/include
CPUINC = ../some/prefCPU12suf/include
CPU = ../some/prefCPU12
0 eddy@heidi ~/usr/src/make/make-profiler/make-3.82 $ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-pc-linux-gnu</http:>
Not sure if this could be qualified as a true bug, or a if the way I expected is a nice to have feature, but, in any case, the behaviour is consistent, unlike my brain which failed to initially identify the inconsistency in my code:

0 eddy@heidi ~/usr/src/make/make-profiler/make-3.82 $ grep patsubst /tmp/makefile
CPUINC := $(patsubst pref%,%,$(PATH))
CPU := $(patsubst %suf,%,$(CPUINC))
0 eddy@heidi ~/usr/src/make/make-profiler/make-3.82 $ make -f /tmp/makefile
PATH = ../some/prefCPU12suf/include
CPUINC = ../some/prefCPU12suf/include
CPU = ../some/prefCPU12suf/include
Note that this behaviour of patsubst is asymtric to how subst works, as explained in the updated old post.

This took some extra effort to remember what was the actual issue, and shows why logs are important when reporting an issue, and why reporting issues as soon as they were encountered: because human brains are faulty. (Yes, yours, too!)

Eddy Petri&#537;or: GNU Make quirk

(Updated 2013-07-20: Correctly describe the issue and provide examples)

Make has several useful built-in functions, among which $(patsubst pattern,replacer,string). The pattern can contain the character % which is a wildcard meaning any string and that can be referred in the replacer parameter. Only the first occurrence of % is treated with this special meaning, just as the manual says.

What the manual doesn't say is that only the form %bla works, while the form bla% does NOT work GNU Make's '%' is not analogous to shell's '*', so you can't expect to remove substrings in the middle of a string without knowing the exact string until the part you want to remove.

This does not work if you want to obtain '../some/CPU12/include' in OTHERPATH:

PATH := ../some/prefCPU12/include
OTHERPATH := $(patsubst pref%,%,$(PATH))

default:
@echo "OTHERPATH = $(OTHERPATH)
The result will be:
OTHERPATH = ../some/prefCPU12/include
This bahaviour is somewhat asymetric to how subst behaves and how one would expect it to, considering '%' is explained as a glob pattern:

0 eddy@heidi /tmp $ cat makefile
PATH := ../some/prefCPU12/include
OTHERPATH := $(patsubst pref%,%,$(PATH))
SPATH := $(subst pref,,$(PATH))

default:
@echo "PATH = $(PATH)"
@echo "OTHERPATH = $(OTHERPATH)"
@echo "SPATH = $(SPATH)"
0 eddy@heidi /tmp $ make
PATH = ../some/prefCPU12/include
OTHERPATH = ../some/prefCPU12/include
SPATH = ../some/CPU12/include
Not sure what make developers would say about this, and I am not sure if having % work as a glob pattern and using it, instead of invoking sed would be better for performance, but sure I would like to have the option :-) .

19 July 2013

Justus Winter: A story about virtualization

I'm sure you all will be a bit disappointed (I know I am) that there are no ascii screenshots in this weeks report. But let me make it up to you by telling you a story. I was at the FOSDEM in the year 2012 and I went to a nice workshop on sunday afternoon in the Virtualization Devroom. Renzo Davoli was the host of this workshop and he started with a little introductory talk to get everyone to agree on a common terminology first. He began by asking the question, what virtualization meant in the most general way. He defined the ability to virtualize a resource as the ability to freely and transparently control how someone (say a process) is interacting with said resource. So for example if you LD_PRELOAD a library to impersonate fopen(3) and friends, you have virtualized the filesystem (for some small values of virtualized). He then went on to discuss various methods of virtualization that are commonly available on Linux, not only full system virtualization solutions, but also all kinds of methods allowing a more fine-grained control over what resources are virtualized. Of course every method had its strengths and its weaknesses. Having seen Samuels talk about all the awesome things Hurd can do for virtualization I approached him with a question in the free discussion part of his workshop. I asked whether he would agree that once a resource is moved from the kernel to the userspace, the problem of virtualizing that resource is almost trivially solved, and he agreed. So I said that this was awesome, because then the trivial and elegant solution for all his virtualization (and tracing) needs are micro kernel operating systems, and he also agreed to that but (as far as I can recall) he mentioned that there is none that would meet his needs (I believe he is a computer science professor and heavily relies on virtualization techniques for his lab and for teaching purposes). And even though a microkernel operating systems has a cost (message passing instead of function calls for example) it also has its merits such as scaling better from a development point of view. Also, many cool features can emerge just from the design. For example think about the container support in Linux and how painfully long it took to make all the resources namespace aware (with one of the most critical, the user namespace being the most difficult one). On Hurd you get the same functionality for free. If you are curious, read Samuels awesome slides. So what have I done this week? Next week I'm going to work on the two remaining sysvinit related issues. These are the only ones preventing me to come up with a clean patch series against the sysvinit package and I figured that it would be nice to propose such a series rather sooner than later so that I will have plenty of time to discuss any issues with the sysvinit maintainers. See you next week :)

Jan Wagner: Debian (wheezy) 7.x and mailscanner

Mailscanner is an e-mail security and anti-spam solution for e-mail gateways. In combination of postfix it uses a very ugly way messing around in postfixs queue directories. Anyways .. if you are still interested, keep reading. Mailscanner was removed from Debian, so it's not available in wheezy. Anyways ... there is a web 2.0 Frontend for Mailscanner, called Baruwa, which provides mailscanner packages for Debian. Add the baruwa sources file
cat > /etc/apt/sources.list.d/mailscanner.list << 'EOF'
deb http://apt.baruwa.org/debian wheezy main
deb-src http://apt.baruwa.org/debian wheezy main
EOF
Optional, install the GPG key for the reprosority
wget -O - http://apt.baruwa.org/baruwa-apt-keys.gpg   apt-key add -
If you're running squeeze with Mailscanner, you should do this before upgrading to wheezy. If you are installing Mailscanner on a fresh wheezy, just:
aptitude update && aptitude install mailscanner
As of Version 4.78 and above, you can place you local changes into /etc/MailScanner/conf.d/, no need to mess directly around into /etc/MailScanner/MailScanner.conf

18 July 2013

Vincent Bernat: Packaging a daemon for OS X

There are three main ways to distribute a command-line daemon for OS X:
  1. Distributing source code and instructions on how to compile it.
  2. Using a third-party package manager, like Homebrew.
  3. Providing an installer package.

Homebrew Homebrew is a popular package management system. It works like the BSDs ports collections by downloading, compiling and installing the requested software, while also installing any required dependencies automatically. Creating a new package is quite easy and there are a lot of examples available. However, there are some limitations:
  • You don t really build a package but execute a recipe to locally install the software.
  • You need to install development tools, either a whole Xcode installation1 or the command line version.
  • If you need to execute some steps as root, you will have to explain them to the user for her to execute by hand. This includes the creation of a system user or the installation of a daemon through launchd.
If you can t live with those limitations, you may need to build an installer package.

Building an installer package OS X comes with a graphical and a command-line installer. The graphical one is run by opening a package from the Finder. Then, the user experiences some familiar wizard allowing her to install the software in a matter of a few seconds. OS X graphical installer The documentation around how to build such a package is somewhat clumsy. You may find outdated information or pieces of information that only apply to projects using Xcode. I will try to provide here accurate bits in the following context:
  1. You are packaging a pure command-line tool.
  2. You are using Autoconf and Automake as a build system.
  3. You want to support several architectures.
  4. You want to support older versions of OS X.

Creating a package Building such a package was previously done with some graphical tool called PackageMaker. This tool is not available any more and developers are asked to switch to pkgbuild and productbuild. There is a very neat Stackoverflow article on how those tools work. A package is built in two steps:
  1. Build component packages.
  2. Combine them into a product archive.
A component package contains a set of files and a set of scripts to execute at various steps of the installation. You can have several component packages, for example a package for the daemon and a package for the client. They are built with pkgbuild. A product archive contains the previously created component packages as well as a file describing various options of the installer (required and optional components, license, welcome text, ). To create a component package, we need to install the needed files in some directory:
$ ./configure --prefix=/usr --sysconfdir=/private/etc
$ make
$ make install DESTDIR=$PWD/osx-pkg
We will now put the content of osx-pkg into a component package with pkgbuild:
$ mkdir pkg1
$ pkgbuild --root osx-pkg \
>    --identifier org.someid.daemon \
>    --version 0.47 \
>    --ownership recommended
>    pkg1/output.pkg
pkgbuild: Inferring bundle components from contents of osx-pkg
pkgbuild: Wrote package to output.pkg
You need to be careful with the identifier. It needs to be unique and identify your software as well as the specific component. Then, you need to create an XML file describing the installer. Let s call it distribution.xml:
<?xml version="1.0" encoding="utf-8" standalone="no"?>
<installer-gui-script minSpecVersion="1">
    <title>Some daemon</title>
    <organization>org.someid</organization>
    <domains enable_localSystem="true"/>
    <options customize="never" require-scripts="true" rootVolumeOnly="true" />
    <!-- Define documents displayed at various steps -->
    <welcome    file="welcome.html"    mime-type="text/html" />
    <license    file="license.html"    mime-type="text/html" />
    <conclusion file="conclusion.html" mime-type="text/html" />
    <!-- List all component packages -->
    <pkg-ref id="org.someid.daemon"
             version="0"
             auth="root">output.pkg</pkg-ref>
    <!-- List them again here. They can now be organized
         as a hierarchy if you want. -->
    <choices-outline>
        <line choice="org.someid.daemon"/>
    </choices-outline>
    <!-- Define each choice above -->
    <choice
        id="org.someid.daemon"
        visible="false"
        title="some daemon"
        description="The daemon"
        start_selected="true">
      <pkg-ref id="org.someid.daemon"/>
    </choice>
</installer-gui-script>
Fortunately, this file is documented on Apple Developer Library. Since we only describe one package, we pass customize="never" as an option to skip the choice part. You can however remove this attribute when you have several component packages. The attribute rootVolumeOnly="true" explains that this daemon can only be installed system-wide. It is marked as deprecated but still works. Its replacement (domains tag) displays an unusual and buggy pane which Apple doesn t use in any of its packages. You need to put the HTML2 documents into a resources directory. It is also possible to choose the background by specifying a <background/> tag. The following command will generate the product archive:
$ productbuild --distribution distribution.xml \
>              --resources resources \
>              --package-path pkg1 \
>              --version 0.47 \
>              ../final.pkg
productbuild: Wrote product to ../final.pkg

Scripts The installer can execute some scripts during installation. For example, suppose we would like to register our daemon with launchd for it to be run at system start. You first need to write some org.someid.plist file and ensure it will get installed in /Library/LaunchDaemons:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>org.someid</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/sbin/mydaemon</string>
    <string>-d</string>
  </array>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
</dict>
</plist>
Then create a scripts/postinstall file with the following content:
#!/bin/sh
set -e
/bin/launchctl load "/Library/LaunchDaemons/org.someid.plist"
You also need a scripts/preinstall file which will allow your program to be smoothly upgraded:
#!/bin/bash
set -e
if /bin/launchctl list "org.someid" &> /dev/null; then
    /bin/launchctl unload "/Library/LaunchDaemons/org.someid.plist"
fi
Ensure that those scripts are executable and add --scripts scripts to pkgbuild invocation. If you need to add a system user, use dscl.

Dependencies There is no dependency management built into OS X installer. Therefore, you should ensure that everything is present in the package or in the base system. For example, lldpd relies on libevent, an event notification library. This library is not provided with OS X. When it is not found, the build system will use an embedded copy. However, if you have installed libevent with Homebrew, you will get binaries linked to this local libevent installation. Your package won t work on another host. The output of otool -L can detect unwanted dependencies:
$ otool -L build/usr/sbin/lldpd
build/usr/sbin/lldpd:
        /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
        /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 744.19.0)
        /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 945.18.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

Compilation for older versions of OS X The package as built above may only work on the same version of OS X that you use to compile. If you want a package that will work on OS X 10.6, you will need to download the appropriate SDK3. Then, you need to tell the compiler the target version of OS X and the SDK location. You can do this by setting CFLAGS and LDFLAGS (and CXXFLAGS if you use C++):
$ SDK=/Developer/SDKs/MacOSX10.6.sdk
$ ./configure --prefix=/usr --sysconfdir=/private/etc \
>         CFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" \
>         LDFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK"
The -mmacosx-version-min flag will be used by various macros in the headers to mark some functions as available or not, depending on the version of OS X you target. It will also be used by the linker. The -isysroot will tell the location of the SDK. Headers and libraries will be looked up in this location first.

Universal binaries Mac OS X 10.6 was available for both IA-32 and x86-64 architectures. If you want to support both installations with a single package, you need to build universal binaries. The Mach object file format, used by OS X, allows several versions of the executable in the same file. The operating system will select the most appropriate one. An easy way to generate such files is to pass -arch i386 -arch x86_64 to the compiler:
$ ./configure --prefix=/usr --sysconfdir=/private/etc \
>     CC="gcc -arch i386 -arch x86_64" \
>     CPP="gcc -E"
However, this is a dangerous option. Suppose that your ./configure script tries to determine some architecture-dependent parameter, like the size of an integer (with AC_CHECK_SIZEOF); it will compute it for the host architecture. The binary generated for the other architecture will therefore use a wrong value and may crash at some point. The correct way to generate an universal binary is to execute two separate compilations and to build the universal binary with lipo:
$ for arch in i386 x86_64; do
>   mkdir $arch ; cd $arch
>   ../configure --prefix=/usr --sysconfdir=/private/etc \
>       CC="gcc -arch $ARCH" \
>       CPP="gcc -E"
>   make
>   make install DESTDIR=$PWD/../target-$ARCH
>   cd ..
> done
[...]
$ lipo -create -output daemon i386/usr/sbin/daemon x86_64/usr/sbin/daemon
Since lipo only works on file, I have written a Python script applying lipo recursively to several directories.

Putting everything together Now, we need to automate a bit. You could provide some nifty script. I propose an appropriate Makefile.am instead. This way, we can use the output variables, like @VERSION@, from ./configure to generate some of the files, like distribution.xml. You need to add that in your configure.ac:
AC_CONFIG_FILES([osx/Makefile osx/distribution.xml osx/im.bernat.lldpd.plist])
AC_CONFIG_FILES([osx/scripts/preinstall],  [chmod +x osx/scripts/preinstall])
AC_CONFIG_FILES([osx/scripts/postinstall], [chmod +x osx/scripts/postinstall])
AC_SUBST([CONFIGURE_ARGS], [$ac_configure_args])
Let s have a look at osx/Makefile.am. First, we define some variables:
PKG_NAME=@PACKAGE@-@VERSION@.pkg
PKG_TITLE=@PACKAGE@ @VERSION@
PKG_DIR=@PACKAGE@-@VERSION@
ARCHS=@host_cpu@
If we want to build for several architectures, we will type make ARCHS="x86_64 i386". Otherwise it defaults to the current host s architecture. We use install-data-local target to install files specific to OS X:
install-data-local:
    install -m 0755 -d $(DESTDIR)/Library/LaunchDaemons
    install -m 0644 im.bernat.@PACKAGE@.plist $(DESTDIR)/Library/LaunchDaemons
uninstall-local:
    rm -f $(DESTDIR)/Library/LaunchDaemons/im.bernat.@PACKAGE@.plist
The main target is the product archive, built with productbuild:
../$(PKG_NAME): pkg.1/$(PKG_NAME) distribution.xml resources
    $(PRODUCTBUILD) \
        --distribution distribution.xml \
        --resources resources \
        --package-path pkg.1 \
        --version @VERSION@ \
        $@
Its main dependency is the component package:
pkg.1/$(PKG_NAME): $(PKG_DIR) scripts
    [ -d pkg.1 ]   mkdir pkg.1
    $(PKGBUILD) \
        --root $(PKG_DIR) \
        --identifier im.bernat.@PACKAGE@.daemon \
        --version @VERSION@ \
        --ownership recommended \
        --scripts scripts \
        $@
Now, we need to build $(PKG_DIR):
$(PKG_DIR): stamp-$(PKG_DIR)
stamp-$(PKG_DIR): $(ARCHS:%=%/$(PKG_DIR))
    $(srcdir)/lipo $(PKG_DIR) $^
    touch $@
The $(ARCHS:%=%/$(PKG_DIR)) will expand to x86_64/$(PKG_DIR) i386/$(PKG_DIR). This is a substitution reference4. Before applying our lipo script, we need to be able to build the architecture-dependent package directories:
pkg_curarch = $(@:stamp-%=%)
$(ARCHS:%=%/$(PKG_DIR)): %/$(PKG_DIR): stamp-%
$(ARCHS:%=stamp-%): stamp-%: im.bernat.lldpd.plist
    [ -d $(pkg_curarch) ]   mkdir -p $(pkg_curarch)
    (cd $(pkg_curarch) && \
        $(abs_top_srcdir)/configure @CONFIGURE_ARGS@ \
            CC="@CC@ -arch $(pkg_curarch)" \
            CPP="@CPP@")
    (cd $(pkg_curarch) && \
        $(MAKE) install DESTDIR=$(abs_builddir)/$(pkg_curarch)/$(PKG_DIR))
    touch $@
This may seem a bit obscure, but this is really like what we have described previously. Note that we use @CONFIGURE_ARGS@ which is a variable we have defined in configure.ac. Here is how a user would create its own OS X package:
$ SDK=/Developer/SDKs/MacOSX10.6.sdk
$ mkdir build && cd build
$ ../configure --prefix=/usr --sysconfdir=/private/etc --with-embedded-libevent \
   CFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" \
   LDFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK"
[...]
$ make -C osx pkg ARCHS="i386 x86_64"
[...]
productbuild: Wrote product to ../lldpd-0.7.5-21-g5b90c4f-dirty.pkg
The package has been built in ../lldpd-0.7.5-21-g5b90c4f-dirty.pkg.

  1. Even if Xcode is free, you will need to register to Apple and show a valid credit card number. That s quite an interesting requirement.
  2. It is possible to use other formats (like RTF or just plain text) for documents but HTML seems the best choice if you want some formatting without fiddling with RTF.
  3. Unfortunately, Apple makes it difficult to find them. The easiest way to get the SDK for Mac OS X 10.6 is to download Xcode 4.3.3, mount the image and copy the SDK to /Developer/SDKs.
  4. This is equivalent to the use of patsubst function. However, this function is specific to GNU make. The substitution reference pattern was accepted recently in POSIX and is coined as pattern macro expansion.

17 July 2013

Petter Reinholdtsen: How to fix a Thinkpad X230 with a broken 180 GB SSD disk

Today I switched to my new laptop. I've previously written about the problems I had with my new Thinkpad X230, which was delivered with an 180 GB Intel SSD disk with Lenovo firmware that did not handle sustained writes. My hardware supplier have been very forthcoming in trying to find a solution, and after first trying with another identical 180 GB disks they decided to send me a 256 GB Samsung SSD disk instead to fix it once and for all. The Samsung disk survived the installation of Debian with encrypted disks (filling the disk with random data during installation killed the first two), and I thus decided to trust it with my data. I have installed it as a Debian Edu Wheezy roaming workstation hooked up with my Debian Edu Squeeze main server at home using Kerberos and LDAP, and will use it as my work station from now on. As this is a solid state disk with no moving parts, I believe the Debian Wheezy default installation need to be tuned a bit to increase performance and increase life time of the disk. The Linux kernel and user space applications do not yet adjust automatically to such environment. To make it easier for my self, I created a draft Debian package ssd-setup to handle this tuning. The source for the ssd-setup package is available from collab-maint, and it is set up to adjust the setup of the machine by just installing the package. If there is any non-SSD disk in the machine, the package will refuse to install, as I did not try to write any logic to sort file systems in SSD and non-SSD file systems. I consider the package a draft, as I am a bit unsure how to best set up Debian Wheezy with an SSD. It is adjusted to my use case, where I set up the machine with one large encrypted partition (in addition to /boot), put LVM on top of this and set up partitions on top of this again. See the README file in the package source for the references I used to pick the settings. At the moment these parameters are tuned: During installation, I cancelled the part where the installer fill the disk with random data, as this would kill the SSD performance for little gain. My goal with the encrypted file system is to ensure those stealing my laptop end up with a brick and not a working computer. I have no hope in keeping the really resourceful people from getting the data on the disk (see XKCD #538 for an explanation why). Thus I concluded that adding the discard option to crypttab is the right thing to do. I considered using the noop I/O scheduler, as several recommended it for SSD, but others recommended deadline and a benchmark I found indicated that deadline might be better for interactive use. I also considered using the 'discard' file system option for ext3 and ext4, but read that it would give a performance hit ever time a file is removed, and thought it best to that that slowdown once a day instead of during my work. My package do not set up tmpfs on /var/run, /var/lock and /tmp, as this is already done by Debian Edu. I have not yet started on the user space tuning. I expect iceweasel need some tuning, and perhaps other applications too, but have not yet had time to investigate those parts. The package should work on Ubuntu too, but I have not yet tested it there. As for the answer to the question in the title of this blog post, as far as I know, the only solution I know about is to replace the disk. It might be possible to flash it with Intel firmware instead of the Lenovo firmware. But I have not tried and did not want to do so without approval from Lenovo as I wanted to keep the warranty on the disk until a solution was found and they wanted the broken disks back.

Daniel Pocock: Some further observations on init replacement

My blog about replacements for SysV init received a significant number of comments Monday, revealing a solid base of community support for systemd. One reader was even generous enough to contribute a pull request fixing the reSIProcate package systemd artifact The blog wasn't intended to disparage any of the init replacements or suggest that these types of issues are show-stoppers. Rather, it was simply to demonstrate the fact that developers need to divert attention from other activities (like bug fixing, new features, etc) to correctly adapt things and support the wide range of init replacements that have emerged. As revealed in some comments like this one, systemd can even assist with privileged port assignments: it's nice a feature, but also likely to require more development time to integrate. None of these things are difficult though, they just take time. The real challenge The biggest headache is that it becomes tedious for any upstream developer to start getting into that much depth learning about each system they want to support. That is before we even start talking about how daemons and other services are managed in cloud deployments (e.g. the Juju charms from Ubuntu). For a developer to seriously integrate and test their daemon in all of these environments and make maximum use of all their features could easily take a month: without adding any new features to the application itself. In the last five years, I have personally come across all of these platforms and formal packaging systems: This is nothing to boast about: with enough time (and bandwidth to download the installer ISOs) I'm sure anybody could set up all these environments and many more. The question, though, is whether developers can afford to stretch themselves so thin, particularly on smaller projects that are not commercially supported. systemd: struggling to find documentation My first impression of systemd was that it is not widely documented. This is not the developers fault, they do provide some great documentation. Rather, at the time I was looking for it, Google's search algorithm was under the impression that systemd was a typo. Several other forum posts reveal other people had the same problem. I wasn't immediately aware of the fact that systemd has to be quoted in Google searches although I've since discovered quoting it can help find significantly more helpful details in a search. Nonetheless, this initial experience wasted time for me and it is more likely that I would have discovered things like tmpfiles.d if I hadn't lost time on that issue. In my mind, this is not a fault with systemd at all, it is just bad luck and limited time to allocate on the task. A single effort to support multiple solutions This brings me back to one of the points I made Monday, that it would be good to see a single solution that allows upstream developers to auto-generate artifacts for all common init replacements. Maybe it wouldn't exercise all of their cool new features. One further comment on Monday even observed that such a tool does exist for invoking systemd artifacts from SysV init. Obviously we already have tools like alien to convert package formats (although it is obviously not used to auto-create packages for Debian's archive). Nonetheless, developers of free software would be able to see their code deployed much more widely, more quickly, if they could get away with using a common tool for this job, just as a single autotools or cmake can support compilation on platforms the developer never even heard of.

Peter Eisentraut: Tricky shell local variables

I have a word of warning against improper use of local in shell functions. If you are using shell functions, you might want to declare some variables local to the shell function. That is good. The basic syntax for that is
local a b c
In some shells, you can also combine the local declaration and assignment, like this:
local foo=$1
local bar=$2
(The Debian policy even explicitly allows it.) This is somewhat dangerous. Bare shell assignment like
foo=$bar
does not perform word splitting, so the above is safe even if there are spaces in $bar. But the local command does perform word splitting (because it can take multiple arguments, as in the first example), so the seemingly similar
local foo=$bar
is not safe. This can be really confusing when you add local to existing code and it starts breaking. You can avoid this, of course, by always quoting everything to like
local foo="$bar"
but overquoting isn't always desirable, because it can make code less readable when commands are nested, like
local foo="$(otherfunc "other arg")"
(Nesting is legal and works fine in this case, however.) I suggest using local only for declaring variables, and using separate assignment statements. That way, all assignments are parsed in the same way.

15 July 2013

Rob Bradford: Clutter & Cogl Wayland update

Lots of us over at Intel towers are working hard to bring you a great new Wayland powered experience. Cogl and Clutter have now been updated to work with the latest Wayland master. You need to compile master Cogl with: ./configure --enable-wayland-egl-platform=yes --enable-gl And then to run any applications using Cogl on Wayland you need to setup the environment variable COGL_RENDERER=egl_wayland then you can run cogl-crate! For Clutter you need to compile Clutter master with: ./configure --enable-wayland-backend You also need to set an environment variable to choose the Clutter backend too in this case it s CLUTTER_BACKEND=wayland (and you need to set COGL_RENDERER too) then you can run test-actors! Or just look at the video below :-) <iframe allowfullscreen="allowFullScreen" frameborder="0" height="300" mozallowfullscreen="mozallowfullscreen" src="http://player.vimeo.com/video/33734847?title=0&amp;byline=0&amp;portrait=0" webkitallowfullscreen="webkitAllowFullScreen" width="400"></iframe> Direct Vimeo link! A big kudos needs to go to Robert Bragg, Neil Roberts and Emmanuele Bassi for helping make this happen. The Clutter client side support is basically complete so i ve now moved onto working on GTK and MX watch this space for updates!

14 July 2013

Michael Stapelberg: How many packages does Debian Code Search contain?

The German computer magazine c't has covered Debsources in its most recent edition (c't 16/2013). In that article, they also state:
Debsources integriert auch eine Code-Suche, allerdings werden lediglich die
Quellen des Unstable-Zweigs durchsucht, der zirka ein Drittel des Quellcodes
von Debsources ausmacht.
This loosely translates to:
Debsources also integrates a code search engine, but it only searches the
sources of the unstable tree, which makes up for roughly one third of the
sources that Debsources covers.
I suspect the author of the article arrived at this conclusion because Debsources talks about 400 GiB of source code, whereas Debian Code Search talks about 130 GiB of source code. However, it still struck me as odd and giving the wrong impression. I thought that packages don t differ that much between stable, testing and unstable. So I fired up psql and pounded UDD until it revealed how many packages are different between stable, testing and unstable: Conclusion So, in conclusion, Debian Code Search covers 74% of wheezy, testing and unstable. Debsources also covers contrib and non-free, which explains the higher disk usage. In particular, there might be big blobs in non-free that account for a lot of disk space. Also, Debsources keeps sources around for a few weeks whereas Debian Code Search only keeps the most recent snapshot. Query
SELECT
  COUNT(*)
FROM
  (SELECT
     migrations.source,
     sources.version AS stable_version,
     testing_version,
     unstable_version
   FROM sources
   LEFT JOIN migrations
   USING (source)
   WHERE sources.release = 'wheezy') AS x
WHERE
  regexp_replace(stable_version::text, E'-([0-9.]+)$', '') = regexp_replace(testing_version::text, E'-([0-9.]+)$', '');

Next.

Previous.