Search Results: "spa"

20 December 2009

Jose Luis Rivas Contreras: awesome s awful.spawn grr!

Yesterday I was trying to get some of my old keybindings (from awesome 2.x) to work on awesome 3.4 and it was awful the process. With 2.x I worked with awful.spawn and got any bash command I wanted running. Now it changed to awful.util.spawn but if I wanted anything ran on a shell I had to use awful.util.spawn_with_shell but, at least for me, didn t worked. So I got commands like:
scrot ~/tmp/screenshots/$(date +%Y%m%d-%k%M%S).png
That one didn t ran directly. Nor my commands for volume management. So I got a workaround, used only awful.util.spawn calling scripts on my ~/bin path. So if any of you has got issues with this just put your line on a bash script file, make it executable and use a line like:
awful.key( , "Print", function () awful.util.spawn("scriptpath") end),
BTW, don t use ~ nor any thing needs a shell, use full paths to avoid confusion and bad functioning. I used ~/bin/d/anything and didn t worked nor with awful.util.spawn_with_shell. I hope this help anyone with my same issues.

Stefano Zacchiroli: RC bugs of the week - week 15

RCBW - week #15 Here are last weektoday squashes, by yours truly: Some highlights:

Wouter Verhelst: QNAP NAS device

A customer called me a while back. They have a pretty high-end NAS (2U server with external disk enclosure) that they use for most of their needs: both document and other storage for management types, as well as storage for their EDA cluster. People familiar with EDA tools will know that they are pretty disk intensive. They tend to require shitloads of diskspace, preferably at high performance. The problem was that extending the storage space of this high-end NAS device is rather expensive. Supported disks only come at 300G units with prices that are higher than the cheap TB disks available today for the desktop market; and while the price and performance of those disks is worth it for the EDA requirements, the number of people storing documents and similar on the same device is not so high that performance would be an issue. Thus, what they wanted was a cheap solution to augment, rather than replace, their current storage solution, with the focus being on low cost at the possible expense of performance and, since bacula runs well there, reliability. They were quite surprised when I offered them a solution based around a QNAP device; they had expected a price, without hard disks, of about three to four times the price of the QNAP, and therefore were very happy with this suggestion. Unfortunately, the devices produced by this manufacturer that are supported by Lenny, the current Debian stable release, were EOL'd just before we placed the order; however, their replacements will be supported by Squeeze, the next release, and with a bit of help from Martin Michlmayr, we were able to install Lenny with a somewhat more recent kernel. This drove the price up somewhat, both because the newer devices are a few tens of euros more expensive, and because we agreed to pay Martin to prioritize work on the specific device that we'd need, so that we wouldn't have to wait for him to get around to it. As of this Thursday, the device, which had finally arrived after some initial delays, has been installed. Unfortunately, one of the disks that they'd bought was DOA; but they had the foresight to buy five, rather than four, disks, so that was not a major problem. In fact, the only real issues that I ran into were this one arm-specific bug, and the fact that the high-end NAS device is still running etch, which means that rsync --acl won't work. That means I'll have to go back there soon to do an upgrade of the high-end device (which had been planned for a while already); for the arm-specific bug, a workaround is already in place. All in all, a pretty good experience.

Robert Collins: Various releases


Recently I ve been working on the Python unittest API in my spare time, with a long term goal of making it possible to safely and sensibly glue many different plugins together into the core. Two important components of that goal are being able to extend the data included in a test result, and being able to change how a test is run (such as adding new exceptions that should be treated as specific outcomes python unittest uses exceptions to signal outcomes). In testtools 0.9.2 we have an answer to both those issues. I m really happy with the data included in outcomes API, TestCase.addDetail . The API for extending outcomes works, but only addresses part of that issue for now. Subunit 0.0.4, which is available for older Ubuntu releases in the Subunit releases PPA now, and mostly built on Debian (so it will propogate through to Lucid in due course) has support for the addDetail API. Subunit now depends on testtools, reducing the non-protocol related code and generally making things simpler. Using those two together, bzr s parallelised test suite has been improved as well, allowing it to include the log file for tests run in separate processes (previously it was silently discarded). The branch to do this will be merged soon, its just waiting on some sysadmin love to get these new versions into its merge-test environment. This change also provides complete capturing of the log when users want to supply a subunit log containing failed tests. The python code to do this is pretty simple:
def setUp(self):
    super(TestCase, self).setUp()
    self.addDetail("log", content.Content(content.ContentType("text", "plain",
         "charset": "utf8" ), lambda:[self._get_log(keep_log_file=True)]))
I ve made a couple of point releases to python-junitxml recently, fixing some minor bugs. I need to figure out how to add the extra data that addDetails permits to the xml output. I suspect its a strict superset and so I ll have to filter stuff down. If anyone knows about similar extensions done to junit s XML format before, please leave a comment :)

Russell Coker: I Bought the Bose QC-15

I bought the Bose QC15 noise canceling headphones for my trip back from the US. See my previous posts about Noise Canceling Headphones [1] and Testing Noise Canceling Headphones [2] for the details of my search. I first tried my new headphones in my hotel room and they worked really well at blocking the noise from the nearby road (El Camino Real in Menlo Park) as well as the noise from the heater in my room. At the airport they entirely blocked the sound of the airport air conditioning system (which was surprisingly loud I didn t realise how loud until I tried the headphones). On the flight the headphones worked really well. I used them for hours when they weren t plugged in to any source, just stopping the noise was a huge benefit. I was also able to listen to music (both MP3s on my laptop and the plane sound system) at a relatively low volume with an apparent high quality. The Bose store I visited in Stanford mall has a sound system set up to emulate the noise you experience inside a jet to demonstrate what the headset can do. It really lives up to this demonstration! I recommend them to anyone who wants over-ear noise canceling headphones and can afford $US300 + tax. But one thing to note is that not everyone likes such things my wife didn t like the sound that they generate (the least bad way to describe it is as a soft hiss). This is definitely not something you would want to buy based on reviews alone, it must be tested in-store. The main technical suggestion I can make for improvement to the QC-15 is for it to have slightly softer and thicker padding where it contacts the sides of the wearer s head. I find that my glasses prevent it from making as good contact as I would like, and that when wearing it while eating the contact is significantly broken with every jaw movement which is really annoying. A minor suggestion is that every pair of headphones should have the left and right ear pieces clearly marked, I really shouldn t have to read an instruction manual to discover which way to wear it. One thing that surprised me was the inclusion of business cards for the headset! Here is a picture:

The picture links to a larger picture that also shows the French version of the same text on the other side. I was astonished by this, encouraging happy customers to help sell your products is a reasonable and effective form of product promotion (really this is what I m doing for Bose with this blog post). But giving customers business cards is going too far anyone who wants me to hand out their business cards can offer to pay me to do so (and I probably won t accept). But if such things are considered to be a good idea then here are a few suggestions for other things that they could do:
  1. Create a Bose dating site where one can meet people who like music and traveling (this does sound appealing). In about 10 years the children of people who meet that way would start buying audio gear
  2. Start a Bose fan club.
  3. Create a template that be used by a tattoo parlor to make a Bose tattoo.
  4. Sell Bose fan t-shirts to people who aren t dedicated enough to get a tattoo.
  5. Register Bose as a religion, that gets tax free status among other benefits.
It s a pity that Bose doesn t make any water-proof noise canceling headphones. It would be something for their marketing people to wear while jumping over a shark on water skis [3]. But seriously the best thing that Bose could do to have their products promoted would be to start by printing the web site for each product on the item, my headset has two patent numbers listed which seem unlikely to provide any benefit for anyone, in that space they could have printed the global.bose.com/qc URL that is on the business card. Of course providing the URL really doesn t do any good when the URL actually is useless. It starts by giving me a page asking which country I am in the correct thing to do is to use geoip to determine the country and then give the user the option of selecting another country if that one is not ideal. Then after I select a country it doesn t take me to a specific page for the product! I could have typed in www.bose.com and got the same result (in terms of US shopping at least) while typing six fewer characters! Next like most corporate web sites the Bose site doesn t appear to be configured for longevity of URLs URLs which are clearly designed for the computer rather than humans are expected to change without warning. This discourages linking to any page that one might discover through web searches or navigating the site, and causes them to lose a lot of potential links. Having specific URLs for all the products (including the obsolete ones) that are designed firstly for humans to read and write would be a good idea. It would be really useful to be able to compare the features of new products with the ones that are going cheap on eBay. For someone who is considering buying a new product now the purchase decision would be easier if they knew that the company would provide resources to help them get a better price on eBay in a few years if they want to upgrade to a newer model. One thing to keep in mind is the fact that the reputation of a company (which makes a dramatic impact on the prices customers are prepared to pay) depends largely on a long history of making quality products. Telling customers about those historic products is one of the most sensible things that most corporations fail to do on the Internet.

Debian News: Updates on LXDE

Several LXDE components have been updated recently in sid. They bring some new features and improve flexibility of customization. Below you have a summary of the most important changes:

Andrew Lee

19 December 2009

Mike Hommey: Iceweasel bug triaging

I ve spent a few hours going through all the unclassified important bugs assigned to iceweasel. This resulted in In the closed bugs, there were several kind of bugs: It will feel good when it will be visible on the bug graph. Still 500+ to go *sigh* Who wants to jump on the bandwagon ? ;)

17 December 2009

Kees Cook: headache empathy

Run gstreamer-properties , click the Video tab, change Default Input Plugin to Custom , and add this Pipeline:
v4l2src ! ffmpegcolorspace ! vertigotv ! ffmpegcolorspace
Now when Empathy video-chatting with a friend, you can give them a headache! Or give yourself a headache by trying it directly from the command line:
gst-launch v4l2src ! ffmpegcolorspace ! vertigotv ! ffmpegcolorspace ! xvimagesink
Feel free to replace vertigotv with any other or more of the video effects listed in gst-inspect effectv . Here s me with edgetv ! vertigotv: Edge Vertigo

16 December 2009

Raphael Geissert: what could you tell about spammers after receiving one billion messages?

This is a question that Project Honey Pot's report answers.

They have released a report of the analysis of not only spam messages, but also of the harvesters behind spam.
Available with a simple overview the complete report provides some rather interesting details.

Do spammers take holidays?
The volume of spam drops approximately 21% on Christmas Day and 32% on New Year's Day. Saturday is the quietest day of the week.


And for those sending mails to public mailing lists, the BTS, etc:
How much harm does a harvester visiting my site do?
You can expect to receive 869 spam email messages every time a harvester picks up your email address from a website online.


The interesting bits are really worth the five-ten minutes it takes to read the complete report, enjoy it (and yay, it is CC-BY).

15 December 2009

John Goerzen: Review: A Christmas Carol

I guess you can say that A Christmas Carol by Charles Dickens has been a success. It was published in 1843 and has never been out of print since then. It s spawned all manner of plays, films, adaptations, and spoofs. It s been adapted at least twice by Disney, once featuring Mickey Mouse and another time featuring Jim Carey. We re almost inundated with the story I m not sure how many ways I ve seen it. Yet I had never read the original story by Dickens until just now. And I must say, what a treat it was. Despite knowing the plot in advance, it was a very good read. The 19th century London setting was done well. It wasn t some idealized London as is often portrayed in film adaptations. It had depth, as did the characters. Dickens Scrooge had a troubled childhood, the son of poor and apparently abusive parents. He turned to business, with which he was successful. Along the way, he lost sight of family, and really of his humanity in general, striving to be a richer and more successful businessman at the cost of all else. How apropos this story is for us in the 21st century. Our large banks define success in terms of profits made for their shareholders, while adding more gotchas to the terms of the credit cards held by their customers. Our governments play geopolitical games over weapons, oil, and gas, while unwilling to sacrifice anything to prevent a climate disaster. Our politicians, even in the season of Christmas, turn a blind eye and a cold heart to the suffering of those that can t afford health care for naught but political reasons, rather than trying their hardest to make a plan that will help them reality as soon as possible. And what of us, the citizens of the 21st century? We consume ever flashier cars, houses, computers, and cellphones with data plans, while poverty intensifies across the globe in this economic downturn. Well, count me among those many inspired and reminded by Dickens to be a more empathetic person, to remember how good even many of the poor in the West have it compared to other parts of the world, and to try to do more for others. And that, perhaps, is part of the genius of Dickens. He inspired a complete change of how people looked at Christmas in his time. And his work is no less relevant today; perhaps it hits even closer to home these days. He invites us to carefully consider the question: what does it mean to achieve success in life? And he deftly illustrates that wealth is wrong answer. Here s hoping that many others will also learn a small bit about life from Dickens. How to find it: A Christmas Carol is available for free from Project Gutenberg for reading online, printing, or reading on an ebook reader such as the Kindle. Be careful when buying printed editions. Many have been abridged or improved for a modern audience , and thus lose a lot of the quality of the original. I found at least one edition that looks true to the original; I m sure there are others. [This review also posted to Goodreads]

14 December 2009

Stefano Zacchiroli: RC bugs of the week - week 14

RCBW - week #14 Here are thislast week squashes, by yours truly: Some highlights:

Joey Hess: two films

I've been watching some good films recently. Two I especially liked. baby camel in sidecar man in taxi Goodbye Solo starts in Winston-Salem and ends up at Blowing Rock. Both settings, as well as the taxi in between, are very familiar. The characters could easily be people I've met. Tulpan is set in a steppe in Kazakhstan. The desert and dust devils mostly reminded me of Mars. Particularly of this amazing shot taken by the Spirit rover. dust devils on Mars! Still, I felt so much more at home in Tulpan. Kept pausing it to check out how the family had their yurt set up and furnished, and watched their very real-seeming life unfold, right down to the lambing scene. While Solo took the very familiar, and confounded expectations and biases at every step. They're both fine films, from directors I want to see more of. I've seen Ramin Bahrani's work before Solo, and especially liked Man Push Cart. Don't know if I will ever find any of Sergei Dvortsevoy's earlier films.

13 December 2009

Russell Coker: Links December 2009

Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1]. Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner. Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be skeptical of the science should watch this. Mark Peters wrote an interesting article A Happy Writer Is a Lousy Writer about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details. CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5]. Cory Doctorow writes in the Guardian about Peter Mandelson s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it. Charles Stross has been blogging a series of non-fiction essays about space colonisation, in The Myth of the Starship he describes how most ideas of space travel are bad and how the word ship is always going to be unsuitable [7]. Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is underwater (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house. Mako is working on a project to allow prisoners to blog [9]. It s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access. PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic medicines include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO. Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he s giving it away) [11]. He concludes by noting that he wants no license agreement except don t violate copyright law . The fact that he can t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is. Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive. Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don t like social networking) his web site ShuffleBrain.com relies on Facebook. Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that. Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don t cause TCP implementations to throttle the throughput. This seems similar to my observations of the Three network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).

Debian News: Debian is moving sieve to its IANA allocated port

The IANA port allocated for ManageSieve is 4190/tcp, and the old port used by timsieved and other managesieve software in many distros (2000/tcp) is allocated for Cisco SCCP usage, according to the IANA registry.

Starting with the version 4.38 of the Debian netbase package, the sieve service will be moved from port 2000 to port 4190 in the /etc/services file. This affects to those using Debian Squeeze, and Debian Sid. Debian Etch and Debian Lenny users are NOT affected.

Any installs which used the sieve service name instead of a numeric port number will switch to the new port number as soon as the services are restarted/reloaded, and in some cases, immediately after /etc/services is updated.

This will affect Cyrus IMAP. This may also affect other sieve-enabled software such as DoveCot.

In order to avoid downtime problems, mail cluster administrators using Debian are urged to verify their Cyrus (and probably also DoveCot) installs, and take measures to avoid services moving from port 2000/tcp to port 4190/tcp by surprise in either servers or clients.

It is worth noting that:

  1. /etc/services will only be automatically updated if you never made any modifications to it. Otherwise, you will be presented with a prompt by ucf or dpkg asking you about the changes.
  2. You can edit /etc/services and change the sieve port back to 2000 if you want (this is not recommended, though).
  3. You can edit /etc/cyrus.conf and any other relevant config files for your mail/webmail cluster (e.g. on the sieve web frontends) ahead of time to force them all to a static port number.
  4. You can configure cyrus master to listen on *both* ports (2000 and 4190) at the same time, and thus avoid the problem entirely. This also allows for a much more smooth migration from port 2000 to port 4190.

Henrique de Moraes Holschuh

Clint Adams: Sanjana Kumar goes to the movies

I have an imaginary automobile. It is the finest hybrid of German and Japanese engineering. It has a peppy little engine and a turning radius so small it would make a baby camel cry, and a six-speed manual transmission, and it is powered by the urine of vegan bears. My imaginary father asked me why I would possibly want to buy an imaginary car with a manual transmission. Obviously, whichever marketing campaign brainwashed him in the first place wasn't strong enough on the memetics, because after borrowing my imaginary car, he shut the hell up. You see, it's considerably more fun to drive than an automatic. There are other benefits as well, of course. The primary one is that the transmission does what I want. An automatic transmission is configured for maximum comfort , so it wastes a lot of power when shifting. With a manual, you can slip from gear to gear with minimal clutch time, or in some cases, if you're feeling sassy, without using the clutch at all. You can control your engine speed so that your shift will be just as smooth as an automatic without the egregiously long clutch disengagement or shift overlap. You also can see the road in front of you, so you're always going to be smarter than an automatic transmission, unless you don't understand how slopes work or what collisions with other cars mean to your well-being. Now some NRA member is going to point out that you can get a shift kit and tweak your automatic to be less awful. While spending money on this type of thing sounds like a great idea, a shift kit is not going to bring an automatic into the same league as my car. I can still start the car when the battery's dead. I can get higher fuel efficiency (vegan bears don't exactly urinate for free). I can shift into fifth gear, and coming from a full stop, drive up a hill without stalling (this is particularly useful when a squirrel has gnawed the teeth off of all your other gears). Furthermore, shift kit or not, an automatic is absolutely no fun. Someday, when transmissions have advanced into the realm of liquid hyperspace buckminsterfullerene cones that commune with the unicorns, and they'll have cameras and radar feeding the car's computer with useful information about its surroundings so it can make intelligent shift decisions faster than a human. Then I'll have to choose between saner and more fun. That era has not yet arrived. This is a lousy metaphor for debhelper.

10 December 2009

Andreas Schuldei: Dear Lazyweb: Datacenter in the US?

I look for a data center in the US for some servers with ~40kW cooling power. I plan to use those nice energy efficient double sided racks from SGI (formerly Rackable), so the place should allow that i bring my own racks. I don't need super-low latency to Europe, but rather to the USA and Canada. And it needs to be highly reliable, preferably tier 3 or better!

However, what is most important to me (living in Europe) is that the remote hands actually have a clue. At one place in London i got really lucky: They are able to patch up new servers, can install spare parts and mail broken stuff back to IBM, know about tcp/ip, link lights, switches and routers, debug bad hardware (where LOM is not good enough) and are generally really friendly and quick. As icing on the cake they can even spell debian! Good pricing would be nice, too, but less important then the clueful people.

Should you know such a place, please mail me!

9 December 2009

Stephan Peijnik: ujail: use cases, FAQs, part 1 & proof of concept, part 2

As I ran out of time whilst writing the "introducing ujail" post on monday I would like to further elaborate on the idea, giving you some examples of possible use cases and then having a look at FAQs regarding ujail. Additionally I have created a second proof of concept that should be a lot faster, see below for more details.

Use cases of ujail

Monday's post was rather technical, so let's have a look at possible use cases today.

The main reason for both having the idea of ujail and starting working on it is my web server. I am running quite a few (S)CGI scripts there and, even though running them as different users, on a per-vhost basis, I have the impression of the whole thing being a bit insecure.

Okay, PHP does provide its famous open_basedir feature, but I am also running some Python applications which I simply cannot restrict easily. My first ideas involved adding something similar to open_basedir to Python, followed by the idea of replacing some C library functions, like fopen and friends on startup time.

Whilst the adding open_basedir to Python would have involved changing a lot of Python's internals I soon discarded the library patching idea as those could be worked around by injected code directly invoking syscalls. It didn't take long for me to notice that I have to dig deeper. The idea of ujail was born and after coming up with the proof of concept this seems to be a viable solution.

Now ujail is not only about protecting a web server from its web applications, but could do a lot more, for example:

I am sure you can come up with even more use-cases. What should be noted is that emulating a system call does not mean that one necessarily needs to emulate the whole filesystem. What can be done, for example, is patching through access to common files (libraries, executables, etc.) whilst maintaining a virtual filesystem for data that will eventually be modified. A copy-on-write approach is possible too, for example. There are multiple methods with which the multiple filesystem could be implemented, the most common would probably be using a state directory.

FAQs

There have been some questions about ujail in comments to my first post which I would like to answer. Also, I have been thinking about things that are different about ujail compared to other virtualization techniques. Feel free to add additional questions either in a comment or drop me an email: debian at sp dot or dot at.

  1. Could you change the license of ujail to ... ?

    Not likely to happen. The proof of concept's license is GPLv3 and the actual code's license will be too. However, ujail is a userspace application that does not need any modifications to the kernel so there should be no problems with porting ujail from GNU/Linux to any other system.

  2. Does ujail work on operating systems other than GNU/Linux?

    Not yet. If it's technically possible to implement the technique on other operating systems I would be happy to accept patches.

  3. Do I need to patch my kernel for ujail to work?

    No, ujail is running in userspace. The only thing it needs is Linux with support for PTRACE_SYSEMU.

  4. How is this approach different from using LD_PRELOAD?

    With LD_PRELOAD one can replace library functions, but malicious code could still directly invoke syscalls, working around this protection completely. Also, statically linked binaries cannot be restricted with LD_PRELOAD.

  5. How is this approach different from user-mode-linux?

    User-mode-linux (UML) works by emulating a full kernel in userspace and allows you to virtualize a whole Linux instance (including a new init process, etc). ujail is about providing a way of restricting a single process (and its childs) inside a running system in terms of access to syscalls and the partial emulation of those.

  6. How is this approach different from linux-vserver?

    Linux-vserver is a kernel patch and runs in kernel space, as opposed to ujail, which works in userspace.
    Also, linux-vserver works similarly to user-mode-linux, providing a fully virtualized Linux instance.

  7. Does the account running ujail need any special privileges?

    No, the only restrictions that apply are those of ptrace.

  8. Where is the code?

    Right now ujail is in a planning phase, and only the proof of concept code has been written and published. The actual ujail code is yet to be written and the code will be hosted on launchpad.net.

Proof of concept, part 2

An anonymous person (who were you stranger?) added a comment to my first post, suggesting "Also, why patch the process rather than just modifying its state and trapping into the kernel?". I have had a look at this approach earlier, but it didn't work out. However, I decided to give it yet another try and created a second proof of concept. That code does not require patching any code, but only modifies the instruction pointer (eip) and the first register (eax). This should be a lot faster than patching the code.

Technically the new main loop works by calling PTRACE_SYSEMU and waiting for a notification. It then saves the instruction pointer and switches to PTRACE_SYSCALL. As before it waits for the emulated syscall to exit and at this point sets eax from orig_eax and decreases the value of the instruction pointer by the size of the "int $0x80" instruction. Another call to PTRACE_SYSCALL resumes the process. The next event is the process actually entering the real syscall and yet another one leaving the syscall again. These are resumed by PTRACE_SYSCALL and PTRACE_SYSEMU respectively. So, comparing this with the first approach we are only modifying two registers now, instead of writing to the TEXT area of the running process.

Thanks should go to the anonymous commenter for making me give this approach another try.

Questions? Criticism? More ideas? Want to contribute?

Coming to an end I would yet again like to let you know that I am open for questions, criticism, more ideas and contributions in general. So if you are interested in this topic come join the discussion by either dropping me an email, writing a comment to this post or replying to this post on your own blog.

7 December 2009

Martin F. Krafft: Securing e-mail communication across clients/OSs

Dear lazyweb: I am in a dreadful situation! I need to secure e-mail communication between five types of users: To make matters worse: there is no common provider or server infrastructure, so the solution must work across providers and mailboxes.

Requirements
  • Human error should be anticipated and prevented.
  • Mail between all parties must be encrypted (and signed) automatically. If encryption is not possible, the mail must not be sent. This includes the situation where not all mail recipients keys are known;
  • Ideally, one can define rules (using wildcards or regular expressions) to enforce certain policies.
  • Mail to other parties may be signed, and it would be good to be able to turn this on and off by default;
  • The MIME standard should be employed so that the mail body is not altered, and attachments can be seamlessly encrypted too. Existing MIME parts should be encapsulated as children of a new multipart tree;
  • Incoming mail should be automatically and seamlessly decrypted/verified, and the user must be alerted if the verification fails.
Ideally, the solution will be open-source. However, if proprietary software performs better, then we will gladly use that where required.

Previous attempts So far, we ve tried (and were let down by)
  • GpgOL, which has very irky and brittle integration with Outlook and can only do inline signing/encryption.
  • S/MIME, which seems supported by all involved clients, but Outlook does not really allow you to specify policies. Encryption seems to be opportunistic at best, which is not enough.
  • GPGrelay, which is promising because transparent, but apparently messes with MIME, and I am unsure whether it can fit between Outlook and Exchange. I have yet to run real tests though. If you are a GPGrelay user, or you d like to try it (it s Windows-only), please get in touch.
  • WinGEAM, but that apparently no longer exists, and neither is the underlying GEAM.

Help! Does anyone have proper suggestions over what to use? In fact, if you are interested in devising a solution, or even deploying it, there s money to pay you for those services. Write in if you are interested.

Y Giridhar Appaji Nag: It has been ...

Dear life,

It has been a pretty crazy couple of months, had to stay away from most of the things that I would normally breeze through, including but not limited to a few aspects of my real life day job, Debian work, movies, the routine yearly Oct-Nov time vacation that I take etc. Obviously, hobbies took a big hit. Also had to let go of an opportunity to travel to one of my favorite places, IIT Guwahati; something that I don't usually pass up. Sigh! But such is life.

Been bouncing back slowly:

I finally moved a few domains that I run (including my personal domain) and their email to a linode and I am quite liking it so far (I would've been great if I had some more disk space at my disposal but 16GB isn't all that bad).

On the Debian front, finished attending to package sponsorship requests, I've been updating packages (there is one elinks RC bug that I'll fix next) and work is finally under progress for the clang ITP in the Debian GIT repository.

There are a ton of pictures in RAW format that I haven't yet had the time to convert to JPEG (I use UFRaw but the output that I got from Bibble Pro is fairly impressive -- perhaps I should consider getting myself a license) but that could wait.

Found a decent place to rent (I should be moving in about a month or so). I would've liked a place that isn't an apartment or at least in a smaller community, preferably something in-town but good places are difficult to come by.

I am back,

Giridhar

Stephan Peijnik: Introducing ujail & proof of concept

Lately I have been thinking about methods to provide a stripped down, secured environment for running untrusted code on GNU/Linux. With this post I would like to present you with the first results of my research.

ujail - brief introduction

I have chosen ujail as the name for the technique I am proposing. ujail stands for micro jail in userspace and, in itself, describes the concept briefly. The main idea is to have a userspace process monitor system calls of one of its childs and emulate some calls, if needed. This is done using ptrace and namely both PTRACE_SYSEMU and PTRACE_SYSCALL.
The ujail process should not be able to monitor syscalls, like strace does, but also intercept and emulate them.

This sounds a lot like user mode linux (uml), but the method is different. Whilst uml comes with a complete kernel, emulates all system calls and this way provides a virtualized system, ujail is intended to only emulate some systemcalls, without emulating the kernel.

Revisiting PTRACE_SYSCALL & PTRACE_SYSEMU

To better explain how the ujail technique works I would like to have a quick look at PTRACE_SYSCALL and PTRACE_SYSEMU again.

PTRACE_SYSCALL allows a userspace process to be notified whenever a traced process enters or leaves a system call. This means that two notifications are normally sent: one before system call entry and one afterwards. Even though one is able to change the parameters of system calls this method does not allow system calls to be fully emulated (think virtual filesystem here).

PTRACE_SYSEMU on the other hand provides one notification on syscall entry and expects the receiver of the notification to emulate the syscall. This method alone sounds great, but this also means that memory allocation needs to be emulated too, which is quite complex in userspace.

A hybrid of PTRACE_SYSCALL & PTRACE_SYSEMU

Now on to the concept behind ujail. The method I am describing works by calling PTRACE_SYSEMU for a specific process and this way taking over emulation of all system calls. However, some system calls are complex to emulate in userspace, and so a hybrid of both PTRACE_SYSEMU and PTRACE_SYSCALL is needed. In short this works by checking whether the syscall needs to be emulated when the PTRACE_SYSEMU event is received.
Now one way is emulating the syscall, filling the processes' registers and resuming execution of the process. This is simple and straight-forward.

The second way is forwarding the system call to the kernel. The problem here is that calling the syscall in the monitoring process will make the new resources available to that very process, and not the process to be jailed. This is where the hybrid method kicks in.

The proof of concept code creates a backup of the next instruction to be executed along with a copy of the instruction pointer at this point and patches it with the opcodes for "int $0x80", causing the syscall to be made again. After that it resumes execution with PTRACE_SYSCALL and waits again. The first event to be received now is the program leaving the emulated system call, which can be ignored. Resuming yet again will give use two PTRACE_SYSCALL events, one for syscall entry and one for syscall exit.

The first event is not really interesting, but at the second event the opcode backup is restored and the eip set from the saved value. Now the kernel has handled the syscall and the result is ready for the child process. A final call of PTRACE_SYSEMU resumes execution of the child and waits for the next syscall.

Proof of concept

The proof of concept code can be downloaded from its bazaar branch at launchpad.net. It is intended to be used on i386 systems only and works with simple programs, but is known not to work with anything using fork, vfork and most likely will not work for binaries using threading.

Finally, I would like to thank Pradeep Padala for his "Playing with ptrace" articles [0][1], which were fun to read and worked as a great introduction of ptrace for me.

Now there is only one thing left to say: if you are interested in this method, see loopholes or problems or want to contribute, please go ahead and contact me:

debian at sp dot or dot at

Next.

Previous.