Search Results: "ag"

23 April 2010

Joerg Jaspert: Bash power

Something that regularly happens when I get to see other peoples shell scripts is me thinking Why do they not use the powers of bash? . Sometimes it s fine to not use bash specific features, like when you are limited by the Operating System you have to run on (don t you try some commercial Unix systems. Or this WIndows crap. Brrr). Sometimes you want it for some perceived performance win, sometimes a full bash doesn t fit the target system (think embedded devices). Thats all fine, don t come up with those, I know them. What I mean is - people are using things that bind you to bash and your hashbang also makes your script use bash. But people still jump out to a different command just for some variable modification which are easily doable in bash too. A few examples follow, always imagine them in a script starting with the header of 
#!/bin/bash
set -e
set -u
Example Get rid of newlines (actually, any replacement you can think of): 
var1="something with
space and
linefeeds"
var1=$ var1//
/_ 
echo $var1
This is a simple but powerful demonstration of the Pattern substitution bash can do for you. Read its manpage, it can do much more than what I list here, but the above simply replaces all occurences of a linefeed with an _. Make the double // a single one and it is just the first that is replaced.
Example Removing a prefix (or suffix) from a string: 
VAR1=$ VAR1##sync:archive:
This removes the text sync:archive: from the beginning of the VAR1 variable. Replace the ## with %% and its gone from the end. The topic to look for in the manpage of bash is Remove matching prefix pattern (or suffix, of course).
There are many more possibilities, you might want to look through the Advanced Bash-Scripting Guide, it contains some helpful information. Or read the bash manpage. Or (shameless self-ad) look at the latest ftpsync script, the one and only official (mirroradm) supported mirror synchronization for the ftp.*.debian.org mirrors.

Alexander Wirt: FrOSCon 2010 Call for Papers

On 21st and 22nd August 2010 the fifth (wow, impressive how fast time goes) FrOSCon is taking place St. Augustin (near Bonn) Germany. Like in the past its hosted by the faculty of computer science of the University of Applied Sciences Bonn-Rhein-Sieg in collaboration with the student body and the Linux/Unix User Group St. Augustin.

A few days ago this years Call for Papers started and will end on May 23rd. So if you want to attend on a really cool conference please submit your paper - and even if you don't want to give a talk join us at the Debian booth :).
<script defer="defer" src="http://static.zemanta.com/readside/loader.js" type="text/javascript"></script>

Ingo Juergensmann: DDoS, Apache2 and mod_qos

For two or three weeks now someone seems to dislike me. It happens from time to time that during the night, about 1.30 a.m., my webserver got hit by a sync dDoS. Maybe it's someone specific, maybe it's just a random botnet. Anyway, the result is the same: the service is no longer available.

For this problem there exists mod_qos, which was mentioned in the last LinuxMagazin issue (German). Ok, so I download mod_qos, followed the instructions and built the module with "apxs2 -ci mod_qos.c". When I've enabled mod_qos I get the following error:

mod_qos, requires OpenSSL, compile Apache using "--enable-ssl"


Which is quite interesting, because the Apache2 runs a HTTPS site as well. So, some sort of SSL should be available and mod_ssl is loaded as well. Anyway, I added mod_gnutls as well, but no change: it still reports the above error.

System is Debian Lenny and Apache2 is version 2.2.9-10+lenny7.

Any tipps, hints, pointers or advices, please? :-)

UPDATE:
After "apt-get install apache2-threaded-dev libgnutls-dev libssl-dev" mod_qos was loaded successfully and hopefully will prevent me from suffering of another dDoS. Sometimes the solution is so simple... ;-)

Ingo Juergensmann: Cleanternet - for a cleaner and safer Internet

Alexander Lehmann, creator of the great videos "Du bist Terrorist" (You're a Terrorist) and "Rette deine Freiheit" (Save your Freedom), has created another fine piece of video art:



http://cleanternet.org

It's about the new plan by EU commissionar Cecilia Malmstr m of censoring the Internet in the name of the fight against child porn. In Germany we had this discussion in the last year and although a law has been brought into action, all political parties don't want that law anymore. Putting up STOP signs in front of webpages is the wrong way to fight child porn.

Wouter Verhelst: Touching up a video with Cinelerra

I found myself with a problem: trying to linearly edit a movie clip with a non-linear editor isn't very simple, nor is it meant to be. Case at hand: I have about 1 hour and 40 minutes of film material that was recorded with dvswitch, and which is 90% fine; but given what we wish to do with it, we really wish to remove the errors from the recording things like failed transitions, rogue camera movements, and so on. This is why we recorded the individual video streams on tape too, so that we could indeed do some retouching afterwards. Cinelerra, however, is not really made for that. If you want to create a transition, you need to have two segments of clips that are stuck together on the timeline, and the transition is then applied to the border between the two. You can create multiple video tracks, but you cannot cut from one track to another; you really have to add a clip to the timeline. Doing that has obvious issues if you have multiple video streams that were recorded on the same time and that can be time-synced on multiple tracks. You would prefer to just select one track or the other on a certain point in time, but cinelerra just doesn't do that. So the workflow that I found is the following: Add all video streams to the time line, each on a separate track. Make sure the to-be-touched-up stream is on top; when multiple tracks are in use, and the are not faded to 50% or so, cinelerra will only show the top stream in the final rendition. Synchronize the streams: first, create a rough alignment by moving them with the mouse. Then, press arrow-down until you can see the streams on a frame-by-frame basis. Now for each of your camera streams, do the following:
  1. In the 'main' stream, search a section that was filmed by the camera which we are trying to align.
  2. Hit 'l' to create a label
  3. Search for the exact same frame in the camera stream. It may help to disable playback for the 'main' stream temporarily to visually verify that the frame is the same in the compositor window.
  4. Move the cursor to that frame, and create another label.
  5. If the streams are indeed desynced, then one of the tracks will lead, and the other will follow. If the leading track is the 'main' track, arm the main track and any tracks that you have already synced before, and make sure to remember the audio tracks. If, on the other hand, this is the camera track, then only arm that camera track. At any rate, make sure to disarm the track(s) that are following.
  6. Use the control and arrow keys to select the first label. Use control, shift, and the arrow keys to select the section up to the second label.
  7. At the bottom right of the program window, you can see three entry boxes with time indexes. The first is the time index to your first label; the third the time index to your second label; an in between those, the second is the length of the interval from the first label to the second label. Set the contents of the first box to all-zeroes, to move the selection to the beginning of the time line; then press tab (to move to the second entry box) and enter (to confirm that you wish to create a selection of that length at the beginning of the timeline).
  8. Press Shift-Space to insert silence at the location of the selection. This will move the armed tracks to the right, and not touch the not armed tracks.
  9. If you did everything right, the two tracks should now be aligned. Move the cursor to the second label, and verify that this is true by disabling 'play' for the main track and any tracks in between the main track and the track you're syncing. The frame in the compositor window should not change if you re-enable the main track. If it does, use 'z' for undo, remove your labels and try again.
Once your video tracks are aligned, you need one final video track that comes all the way on top. This is an empty, 'override' track. Anything you put in this track will override the main track. This way, you do not have to cut from the main track to another camera; instead, you just copy from a camera track and paste on the override track. It is, of course, possible to paste onto the main track, but that has the disadvantage that you cannot easily undo such edits after a 'save' operation. Using a separate 'override' track does allow that. To copy from one track to another track without moving in time, use the following method:
  1. Select the region where you wish to select another camera.
  2. Arm the camera track that you wish to copy from, and make sure not to arm any other track.
  3. Hit 'c' to copy the clip at that point.
  4. Disarm the camera track, and arm the 'override' track
  5. Without moving or removing the selection, hit 'v' to paste onto the override track.
All this may be basic knowledge for people who are more flexed in nonlinear editors than me; but I couldn't find this procedure on the Internet, so I thought I'd put it out there so that the next person who tries this doesn't have to try for days on end to come up with this, as I had to do.

Joachim Breitner: Making dictionary passting explicit in Haskell

Haskell provides type classes to support polymorphism. A type class defines a few methods, which can then be implemented for a concrete type in the type class instance. This is a powerful system, but it also has it drawbacks. Most notably, each type can have at most one implementation of the type class. But sometimes you need to use a different implementation. If, for example, you used the Binary class to store data on disk. Now you changed your data type and the binary instance, and you can not read the old data any more. One solution is to re-name your type using newtype and implement another type instance for that. Often, this is enough. But still, instances are not first-class-citizens. You can not pass them around or modify them, as you can pass around and modify data and functions. Under the hood of the compiler, things look different. The ghc puts the methods of the instance in a dictionary and passes that implicitly to any functions having a (Class a) constraint. (Other implementations exist though) If one could make that behavior explicit, one could easily modify the instance before passing it to the function. But this is unfortunately not possible. But it is possible to pass an explicit dictionary along the data. I use the Monoid class as an example, and define a representation of the dictionary to-be-passed, as well as the dictionary of the default instance: 
data MonoidDict a = MonoidDict
    ed_mempty :: a
  , ed_mappend :: a -> a -> a
   
monoidDict :: Monoid a => MonoidDict a
monoidDict = MonoidDict mempty mappend
(For conciseness, I ignore the mconcat method.) My first idea was to pass this instance along with data: (MonoidDict a, a). But this would not work because there are methods, such as mempty, who need the dictionary without getting passed a value to use. Therefore, I need to put the dictionary both in the covariant and the contravariant position: 
newtype WithMonoidDict a = WithMonoidDict (MonoidDict a -> (MonoidDict a, a))
We need functions to clamp a dictionary to a value, and to extract it again: 
wrapWithCustomMonoidDict :: MonoidDict a -> a -> WithMonoidDict a
wrapWithCustomMonoidDict dict val = WithMonoidDict $ const (dict, val)
extractFromCustomMonoidDict :: MonoidDict a -> WithMonoidDict a -> a
extractFromCustomMonoidDict dict (WithMonoidDict f) = snd (f dict)
Note that both expect the dictionary, so that it can be fed into WithMonoidDict from both sides . For convenience, we can define variants that use the standard instance: 
wrapWithMonoidDict :: Monoid a => a -> WithMonoidDict a
wrapWithMonoidDict = wrapWithCustomMonoidDict monoidDict
extractFromMonoidDict :: Monoid a => WithMonoidDict a -> a
extractFromMonoidDict = extractFromCustomMonoidDict monoidDict
We want to be able to pass the wrapped values as any other value with a Monoid instance, so we need to declare that: 
instance Monoid (WithMonoidDict a) where
  mempty = WithMonoidDict (\d -> (d, ed_mempty d))
  mappend (WithMonoidDict f1) (WithMonoidDict f2) = WithMonoidDict $ \d ->
  let (d1,v1) = f1 d
  (d2,v2) = f2 d
  in  (d1, ed_mappend d1 v1 v2)
Note that mappend has the choice between three dictionaries This is not a good sign, but let s hope that they are all the same. Does it work? Let s see: 
listInstance :: MonoidDict [a]
listInstance = monoidDict
reverseInstance :: MonoidDict [a]
reverseInstance = monoidDict   ed_mappend = \l1 l2 -> l2 ++ l1  
examples = do
  let l1 = [1,2,3]
  let l2 = [4,5,6]
  putStrLn $ "Example lists: " ++ show l1 ++ " " ++ show l2
  putStrLn $ "l1 ++ l2: " ++ show (l1 ++ l2) 
  putStrLn $ "l1  mappend  l2: " ++ show (l1  mappend  l2) 
  putStrLn $ "Wrapped with default instance:"
  putStrLn $ "l1  mappend  l2: " ++ show (
  extractFromMonoidDict $ wrapWithMonoidDict l1  mappend  wrapWithMonoidDict l2)
  putStrLn $ "Same with reversed monoid instance:"
  putStrLn $ "l1  mappend  l2: " ++ show (
  extractFromCustomMonoidDict reverseInstance $
  wrapWithCustomMonoidDict reverseInstance l1  mappend 
  wrapWithCustomMonoidDict reverseInstance l2)
Running examples gives this output: 
Example lists: [1,2,3] [4,5,6]
l1 ++ l2: [1,2,3,4,5,6]
l1  mappend  l2: [1,2,3,4,5,6]
Wrapped with default instance:
l1  mappend  l2: [1,2,3,4,5,6]
Same with reversed monoid instance:
l1  mappend  l2: [4,5,6,1,2,3]
Indeed it works. Unfortunately, this approach is not sufficient for all cases. It is perfectly valid to have functions with signature (Monoid a => Int -> Int). Obviously, this function would have no access to the explicit dictionary, because no value of type a is passed to or from it.<monoid a="a"> I wonder if it would be possible to extend the syntax somehow to be able to properly pass an alternative dictionary to such functions. But given that not all compilers use dictionary passing, my hopes are low.

Joachim Breitner: Making dictionary passing explicit in Haskell

Haskell provides type classes to support polymorphism. A type class defines a few methods, which can then be implemented for a concrete type in the type class instance. This is a powerful system, but it also has it drawbacks. Most notably, each type can have at most one implementation of the type class. But sometimes you need to use a different implementation. If, for example, you used the Binary class to store data on disk. Now you changed your data type and the binary instance, and you can not read the old data any more. One solution is to re-name your type using newtype and implement another type instance for that. Often, this is enough. But still, instances are not first-class-citizens. You can not pass them around or modify them, as you can pass around and modify data and functions. Under the hood of the compiler, things look different. The ghc puts the methods of the instance in a dictionary and passes that implicitly to any functions having a (Class a) constraint. (Other implementations exist though) If one could make that behavior explicit, one could easily modify the instance before passing it to the function. But this is unfortunately not possible. But it is possible to pass an explicit dictionary along the data. I use the Monoid class as an example, and define a representation of the dictionary to-be-passed, as well as the dictionary of the default instance: 
data MonoidDict a = MonoidDict
    ed_mempty :: a
  , ed_mappend :: a -> a -> a
   
monoidDict :: Monoid a => MonoidDict a
monoidDict = MonoidDict mempty mappend
(For conciseness, I ignore the mconcat method.) My first idea was to pass this instance along with data: (MonoidDict a, a). But this would not work because there are methods, such as mempty, who need the dictionary without getting passed a value to use. Therefore, I need to put the dictionary both in the covariant and the contravariant position: 
newtype WithMonoidDict a = WithMonoidDict (MonoidDict a -> (MonoidDict a, a))
We need functions to clamp a dictionary to a value, and to extract it again: 
wrapWithCustomMonoidDict :: MonoidDict a -> a -> WithMonoidDict a
wrapWithCustomMonoidDict dict val = WithMonoidDict $ const (dict, val)
extractFromCustomMonoidDict :: MonoidDict a -> WithMonoidDict a -> a
extractFromCustomMonoidDict dict (WithMonoidDict f) = snd (f dict)
Note that both expect the dictionary, so that it can be fed into WithMonoidDict from both sides . For convenience, we can define variants that use the standard instance: 
wrapWithMonoidDict :: Monoid a => a -> WithMonoidDict a
wrapWithMonoidDict = wrapWithCustomMonoidDict monoidDict
extractFromMonoidDict :: Monoid a => WithMonoidDict a -> a
extractFromMonoidDict = extractFromCustomMonoidDict monoidDict
We want to be able to pass the wrapped values as any other value with a Monoid instance, so we need to declare that: 
instance Monoid (WithMonoidDict a) where
  mempty = WithMonoidDict (\d -> (d, ed_mempty d))
  mappend (WithMonoidDict f1) (WithMonoidDict f2) = WithMonoidDict $ \d ->
  let (d1,v1) = f1 d
  (d2,v2) = f2 d
  in  (d1, ed_mappend d1 v1 v2)
Note that mappend has the choice between three dictionaries This is not a good sign, but let s hope that they are all the same. Does it work? Let s see: 
listInstance :: MonoidDict [a]
listInstance = monoidDict
reverseInstance :: MonoidDict [a]
reverseInstance = monoidDict   ed_mappend = \l1 l2 -> l2 ++ l1  
examples = do
  let l1 = [1,2,3]
  let l2 = [4,5,6]
  putStrLn $ "Example lists: " ++ show l1 ++ " " ++ show l2
  putStrLn $ "l1 ++ l2: " ++ show (l1 ++ l2) 
  putStrLn $ "l1  mappend  l2: " ++ show (l1  mappend  l2) 
  putStrLn $ "Wrapped with default instance:"
  putStrLn $ "l1  mappend  l2: " ++ show (
  extractFromMonoidDict $ wrapWithMonoidDict l1  mappend  wrapWithMonoidDict l2)
  putStrLn $ "Same with reversed monoid instance:"
  putStrLn $ "l1  mappend  l2: " ++ show (
  extractFromCustomMonoidDict reverseInstance $
  wrapWithCustomMonoidDict reverseInstance l1  mappend 
  wrapWithCustomMonoidDict reverseInstance l2)
Running examples gives this output: 
Example lists: [1,2,3] [4,5,6]
l1 ++ l2: [1,2,3,4,5,6]
l1  mappend  l2: [1,2,3,4,5,6]
Wrapped with default instance:
l1  mappend  l2: [1,2,3,4,5,6]
Same with reversed monoid instance:
l1  mappend  l2: [4,5,6,1,2,3]
Indeed it works. Unfortunately, this approach is not sufficient for all cases. It is perfectly valid to have a function with signature (Monoid a => Maybe a -> Maybe a), whose behavior depends on the instance of a, even when being passed Nothing and returning Nothing. Such a function woul<monoid a="a">d have a problem here, because the dictionary would not be passed to the function.
</monoid> I wonder if it would be possible to extend the Haskell language somehow to be able to properly pass an alternative dictionary to such functions. But given that not all compilers use dictionary passing, my hopes are low.

Joerg Jaspert: Taiwan again...

Another 422922 seconds (at the time of writing this entry) and another airplane down to Taiwan for me. This time direct flight, no KLM via Bangkok. 13 hours. Yikes. A two and a half week stay, then back up to Germany. Unlike last time I do not expect to fully disappear for the time. I actually expect to have some more time for my various jobs. Just in a way different timezone and mostly during that thing thats commonly called Office hours . Outside them, I dont think I will appear online much. Should you happen to be around Taipei and want to meetup - mail me. As long as its not during weekend it should be fine. :)

Jan Wagner: [fun] Integrate nagios users with system ones

While I normaly don t care much about Ubuntu, I m subscribed to lauchpad.net for the packages I m involved within Debian.
Sometimes apears informative stuff there, which reminds me about my todos or just pointing me to real problems. Most of the time, I m just ignoring the reports, as the are useless or just unrelated to me packages in Debian.
But in some cases the reports are just making my day.

Jan Wagner: [selfnote] Find the DUN Channel of your bluetooth device

In the last days, I replaced my Nokia E90 with a Nokia N97 mini. As I m using my mobile from time to time to hook up to the internet when traveling, I thought I just need to replace the MAC in my /etc/bluetooth/rfcomm.conf and that s it. As always Murphy striked back. After debugging for hours I realized the DUN channel seems depending on the model. You can discover your DUN channel something like:

# hcitool scan
Scanning
3C:F7:2A:1A:4D:2A Cycophone
# sdptool browse 3C:F7:2A:1A:4D:2A \
grep -9 Service Name: Dial-Up Networking \
tail -10 grep Channel
Channel: 22 So your /etc/bluetooth/rfcomm.conf should now look like:


rfcomm0
bind yes;
device 3C:F7:2A:1A:4D:2A;
channel 22;
comment Cycophone ;

22 April 2010

Alexander Wirt: New blog

I was to lazy and to busy to relaunch my website in the last months :(. But since there are some things to blog about I decided to install movable type for bloggin. Half an hour later here I am :).

Thanks to the movable type maintainers for there packages and the backport.

Matt Brubeck: Headless Web Workers: Does the web need background apps?

At my last job, I created several web applications designed to replace built-in apps on mobile phones. While modern browsers and HTML5 made this incredibly easy in many ways, we still ended up writing native (i.e. non-web) code for most of our applications. There were a few different areas where the browser alone didn't meet our needs, but one that I found suprisingly common was background processing. Consider the following mobile applications: Ideally, each of these apps will perform some actions even when the user does not have it open. (Background processing is not strictly necessary for the e-reader, but it would be useful to ensure the library is up-to-date even when opened in a place with no network connection.) You can't do this with a web app. Web Workers don't solve the problem, because they run only while the web page is open. What we need are headless web workers.1 The API Headless workers could use almost the same API as Web Workers. Instead of responding to messages from a web page, they would listen to events from the host system (browser or OS). These events might include time intervals, power-on/resume, changes in network connection, geographic locations, or "push" notifications from a remote server. The event-driven architecture of JavaScript in the browser allows the host system a high level of discretion over resource consumption. There's no special code needed to suspend processes and later restore their state, because JavaScript workers are naturally inactive between events. The host can provide limits on CPU or memory usage per event, with a separate message to notify processes whose handlers were aborted. And it can limit the number of concurrent processes by choosing when to dispatch events to listeners. Some listeners could even be disabled completely at times (like if the device is busy or the battery is low), and notified later of the events they missed. This is almost a return to the old days of cooperative multitasking. Mobile computing is definitely driving everyone towards higher-level process control in the OS, and different assumptions for applications. It's not surprising that my whole proposal resembles Android and iPhone 4.0 multitasking in several ways, since I've been doing development on Android for the last 18 months and encountering many of the same issues. The UI Headless workers do need some way to interact with the user. They could display standard system notifications (via Growl on the Mac, libnotify on Ubuntu, the status bar in Android, etc.) using W3C Web Notifications, which already have an experimental implementation in Chrome. Users also needs to know which sites have background tasks installed. Headless workers could be represented by icons in a standard location (perhaps a toolbar in desktop browsers, or the home screen on a mobile device). The icons could display ambient status; clicking one would reveal a menu with options to configure or remove it. Questions This proposal might be hard to standardize, especially where it's tied to specific OS capabilities. For now I'm just curious: would it be useful? You can write a native app or a browser extension to solve this problem today. But would it be worthwhile to have a standard, cross-platform way to do it? Has anyone else run into problems that this approach could solve?
  1. Because all web standards should have names that sound like Harry Potter creatures.

Joey Hess: Sponsoring non-regulars for DebConf10

DebConf 10 will be held this August 1-7, in New York. I love DebConf, and I look forward to seeing friends at it each year. It rekindles my interest in the project; I leave each DebConf with new things to do. Many developers make it to most DebConfs, helped by the conference's impressive travel and lodging funding. But I know others have never gotten to experience DebConf, or have not been in years. If that's you, there's probably a reason. Maybe the scheduling never works out for you, and I can't help with that. But if your reason comes down to "money" or "they'd never pick me", we can help. This year there is special funding available for Debian Developers (and DMs) who have not been to DebConf before, or who have not been in years. Outgoing DPL Steve McIntyre has previusly mentioned this special funding, and his successor Stefano Zacchiroli has asked me to manage it, since it was originally my idea. Email me soon (before May 15th) to apply, or to suggest that a team member should be sponsored. This wiki page has the details. Let us get you to DebConf -- I promise you won't regret it! PS, going to dc10.png

Matt Zimmerman: On Cloud

If you re convinced that cloud is a useless buzzword, being used to describe everything under the sun, old and new, then well, you re right that is, except for the useless part. It s true that this word is being (ab)used in many different products, services and technologies, which do not seem to relate to each other in any concrete way. Cloud in the abstract is being defined in many different ways, based on different fundamental characteristics of cloud technology . Nonetheless, there is something genuinely important going on here, and this is my view of what it s about. The hype Countless business which only had web sites or web applications in 2008 now call them cloud services . They aren t delivering any new benefits to their users, and they haven t redesigned their infrastructure. What do they mean by this? Cloud is defined, some say, by where your data is kept, and cloud means storing your data on remote servers instead of internal ones. Millions of Gmail and Flickr users have been in the cloud without even knowing it, and so has everyone who reads their mail over IMAP, or uses voicemail. In short, cloud is just data that is somewhere else.

Credit: jurvetson

Last September, I watched a presentation on Cloud computing where Symantec pitched their anti-virus product as having something like three different cloud computing technologies. These, it turned out, all involved downloading files over the Internet. In this view, cloud is essentially about the network, derived from our habit of using a drawing of a cloud in our network diagrams for so many years. Wikipedia currently shares this view, as its Cloud computing article opens with Cloud computing is Internet-based computing . No Internet, no cloud. Maybe cloud is just SaaS, and any service provided on a pay-per-use basis over the Internet is a cloud service. It would seem that cloud is about paying for things, especially on a utility basis. The cloud is computing by the hour; it s something you buy. Others maintain that cloud is just virtualization, so if you re using KVM or VMWare, you ve got a cloud already. In this view, cloud is about operating systems, and whether they re running on real or virtual hardware. Cloud means deploying applications as virtual appliances, and creating new servers entirely in software. If your servers only have one OS on them, then they re not cloud-ready. If this vagueness and contradiction irritates you, then you re not alone. It bugs me, and a lot of other people who are getting on with developing, using and providing technology and services. Cloud is not just a new name for these familiar technologies. In fact, it isn t a technology at all. I don t think it even makes sense to categorize technologies as cloud and non-cloud , though some may be more cloudy than others. Nonetheless, there is some meaning and validity in each of these interpretations of cloud. So what is it? A different perspective

Credit: TangYauHoong

Cloud is a transition, a trend, a paradigm shift. It isn t something which exists or not: it is happening, and we won t understand its essential nature until it s over. Simon Wardley explains this, in presentation format, much better than I could in this blog post, so if you haven t already, I suggest that you take 14 minutes of your time and go and watch him do his thing. This change doesn t have a clear beginning or an end. Early on, it was a disconnected set of ideas without any identity. We re now somewhere in the middle of the bell curve, with enough insight to give it a name, and sufficient momentum to guess at what happens next. In the end, it will be absorbed into the way things are . What s going on? So, what is actually changing? The key trends I see are:
  1. People and organizations are becoming more comfortable relying on resources which do not exist in any particular place. Rather than storing precious metals under our beds, we entrust our finances to banks, which store our data and provide us with services which let us use our money . In computing terms, we re no longer enamored with keeping all of our programs and our data on our hard drive: as it turns out, it s not very safe there after all, and in some ways we can actually exercise more control over programs and data out there than in here . We can no longer hoard our gold, or drive off would-be thieves with a pistol, but that wasn t a productive way to spend our energies anyway. While we can t put our hands on our money, we can know exactly what s happening with it from minute to minute, and use it in a variety of ways at a moment s notice.
  2. IT products are ceding ground to services. In many cases, we no longer need to build software, or even buy it: we can simply use it. As computing needs become better understood, and can be met by commodity products, it becomes more feasible to develop services to meet them on-demand. This is why we see a spike in acronyms ending in aaS . Simon explains the progression in detail in his talk, linked above.
  3. Hardware, software and data are being reorganized according to new models, in order to provide these services effectively. Architectural patterns, such as hardware/OS/framework/application are giving way to new ones, like hardware/OS/IaaS/OS/PaaS/SaaS/Internet/web browser/OS/hardware . These are still evolving, and the lines between them are blurry. It s a bit early to say what the dominant patterns will be, but system, network, data and application architecture are all being transformed. No single technology or architecture defines cloud, but virtualization (at the infrastructure level) and the web (at the application level) both seem to resonate strongly with cloudy design patterns.
These trends are reinforcing and accelerating each other, driving information technologies and businesses in a common direction. That, in a nutshell, is what cloud is all about. So what? This transformation is disruptive in many different ways, but the angles which most interest me at the moment are: I d like to explore some of these topics in future posts.

Chris Lamb: Locating source for any Python module

When I'm too lazy to find API documentation I just read the source code. To make this slightly easier in Python, here's what I use in my ~/.$ SHELL rc: 
cdp ()  
  cd "$(python -c "import os.path as _, $ 1 ; \
    print _.dirname(_.realpath($ 1 .__file__[:-1]))"
  )"
For packages, this takes you to the corresponding source directory: 
$ cdp twisted.web
$ pwd
/usr/share/pyshared/twisted/web
$ ls
_auth/  domhelpers.py  html.py  iweb.py  resource.py
[..]
The realpath and [:-1] hackery follow any python-support symlinks. For "standalone" .py (or .so) modules, cdp will take you to the directory it resides in: 
$ cdp subprocess
$ pwd
/usr/lib/python2.5
$ ls subprocess.py
subprocess.py

Gunnar Wolf: A FLISOL critic

Once again, I was invited by several different groups to be present at FLISOL, a quite interesting phenomenon: FLISOL (Latin-american Free Software Installation Festival) is s very large-scale, very loosely coordinated thing put together for five years already in over 200 cities in basically every Latin American country. Go to the FLISOL page, it is quite interesting to try to understand it! Now, I don't like FLISOL. I managed to avoid it in 2005 and 2006; in 2007, I was present at a FLISOL, although I didn't know beforehand it was the reason for the conference I was invited to. In 2008 and 2009 I took part for reasons I should not go into right now. This year, again, I will not be part of any of its activities (regardless of rumors to the contrary I was invited to be present at a panel on ACTA, but I have not followed the topic enough to be aware of anything besides the very basic aspects, I have no authority to speak about it; I told the organizers I would like to be there as part of the audience, but not present the topic. And I am quite work-stressed, so I doubt I'll make it). Why am I against FLISOL? FLISOL itself, as I said, is a positive and interesting phenomenon, and I have enjoyed the conference cycles which often happen together with it. What I am against is installfests In my opinion, in the stage we are at today, instead of promoting Free Software, an install-fest just works against it. Free Software Linux-based distributions at least is widely known already, as a concept, even though most people dare not come anywhere close to it. Few people outside our already consolidated groups recognize programs such as the Mozilla and OpenOffice suites as being also Free Software, and valuable, quality alternatives for their everyday needs in the environments they currently use. If we need to show how to install and understand the GNU/Linux ecosystem to people who have not got close, it is not IMHO to end users. Installing a GNU/Linux system is easy enough for anybody interested in doing it, or at least, for him to request a one-on-one help session, handholding and understanding the basic ideas. We need, in any case, for the computer corner shop technicians to be somewhat acquinted, at least with the basics, at least with one popular distribution (and with the fact that there are many, and that they are different). People who have not had the curiosity and courage to try to install Linux by themselves do not need to be evangelized (a verb that should be out of our vocabularies by now, as that phase in our movement should be over by now) End users have simple needs: Things should work, and be as surprise-free as possible. They don't want to depend on a specific time-starved person (or even on a small group of people, all of which have a sanctity delirium/aura). When they go to technical support, they expect the problem to be solved Not even understanding what was wrong. End users are willing to pay a small fee to anybody to help them solve their problems. The key word is anybody. If we (myself, or me and my 10 friends who were there at the gathering, or any sufficiently defined small group) are the only support point for the OS, it is no good. Online support forums are not good either, in my experience, as the end user will prefer just lugging the computer to the nearest technician and get it fixed. Even if fixed means just installing one more readily-available package (not to mention, of course, when an update breaks something). I have witnessed, after an install fest, people walk very happy with their new system as a new toy. After a week or two, they cannot install the latest virus^Wscreen saver, or a legitimate program they need for their work. As it does not work, they take the computer to the technician... Who will end up formatting his system and installing something more usual. On the other hand, some people prefer installing a dual-boot system That guarantees the user will feel he is carrying some kind of moral superiority on his computer, and will often remember he has something Not Evil. This will often happen, of course, at boot time When they see GRUB at boot time, and rush to select Windows before That Strange Thing starts up. Anyway... Go ahead, install Free Software, enjoy the day. The conference cycles are usually interesting, and are the best part of it all I'm not saying you should stop doing it. But I'd urge you to take the focus away from the mass-installs, which become often just lost work (even detrimental to furthering Free Software). Try to see things as a non-technically-interested user would. Try to design ways to get corner shop technicians interested. Maybe that can be useful in the long run.

Dirk Eddelbuettel: Boston Marathon 2010

This Monday saw the 114th running of the Boston Marathon. Under near-ideal conditions with some sunny skies, some clouds and moderate temperatures in the high 40s to low 50s, a course record as well as US Mens record were set. Having run this race in 2007 and in 2009 (with 2008 in London), I knew the difficult course and was aiming to run it more evenly. So no breakdown on those famous hills! But without too much fire in my belly over the winter, training was somewhat minimal: one 20 miler, two 18 milers. Friends and co-runners were joking that I had turned my trusted 'run less, run faster' program by the FIRST lab into my own 'run less, run lesser' program. And then were a nagging achilles tendon and a little strain from the underwhelming recent half-marathon so that I was really trying to hold back and not go out too fast on the initial portion which is mostly downhill. Which seems to have worked. My pace was more even, and I conserved some energy and made it past the last of the hills around mile 21 without walking a single step while staying a few second under a 8 min/mile average pace. But then around mile 23 and 24 I had two sharp short cramps which forced me to walk. Interestingly enough, Bob Richards writes about cramps as a main theme for many runners in this year's race. Maybe the wind and temperature combined with the hills to get us after all! Anyway, I ended up with 3:29:14 which, at a 7:59 pace, is right between the 2007 and 2009 results and quite decent given the circumstances. And of course the weekend as whole was again a hoot even if I had only a short stay of around 30 hours in Boston given our R/Finance conference on Friday and Saturday. We'll see if I will manage to qualify once more for next year.

21 April 2010

Josselin Mouette: Anything can happen

After skipping 3 entire releases, and 18 months later, here we are, finally: GDM 2.30 is entering unstable. How can you be so late? For those who haven t followed and just wondered why Debian is so late this is lame this sucks Ubuntu is better because they have the latest version and Fedora is even better because they even have versions that don t work at all, here is the short story: the GDM rewrite wasn t really usable until 2.28 (which is the version with which Ubuntu started to ship it, incidentally). Add to that the time to make a transition plan and to integrate it properly, and that makes actually only 6 months. Big thanks go to Luca Bruno (Lethalman) who did most of the job. A quick look at the changelog will give you an idea of the amount of work involved to bring it to our quality standards. GDM 2.20 and 2.30 Since the rewrite has absolutely zero compatibility with previous versions, it will not be upgraded in place. Therefore, while newly installed systems will get GDM 2.30 by default for squeeze, those upgrading from lenny will keep GDM 2.20. The 2.20 version will be dropped after the squeeze release. If you want to upgrade your GDM, simply run apt-get install gdm3. It should work for simple setups, and there s a hack that makes upgrades work even when logged on X. Everyone who has needs for advanced features (such as LTSP people) should make sure GDM 2.30 suits their needs during the squeeze cycle, since the old version will not be here anymore after. GDM packages need your help Finally, here is a call for translations. Anyone can help: just grab the gdm3 sources, get the .pot files and translate them to your language. Beware, there is one file in debian/po for the desktop files and one in debian/po-up for the patches. (I will try to merge them in a later version.) Then submit your translations as bug reports.

Keith Packard: Altus Metrum web store opens

Bdale has just opened up the Garbee and Garbee web store to sell Altus Metrum electronics including TeleMetrum and TeleDongle. The TeleMetrum boards aren t quite ready to ship, so they re marked as out of stock for another couple of days. That s because we ve decided to switch GPS antennas to improve GPS reception and we won t have enough of those until Friday to start shipping boards. The TM boards themselves are working great and we ve flown them several times. We ve already sold a couple of TeleDongle boards for use without TeleMetrum. These boards contain the CC1111 transceiver chip along with an 8-pin connector that can be used for USB, SPI or serial communication. This adds long-range wireless digital communication to almost any project. Steve Conklin has posted his plans for these boards. We sell TeleDongle as either bare boards or packaged in a pretty blue box with a USB cable attached. All of the hardware designs are licensed under the TAPR open hardware license and software under the GPL. All of the tools used to design the hardware and build the software are also free software (already packaged for Debian, of course) so you can take these designs and do whatever you want with them.

Bdale Garbee: Web Store Open

I'm pleased to announce that the new Garbee and Garbee web store offering products designed by the Altus Metrum community is now open for business! Rocketry folks, please note that I refuse to sell anything that isn't in stock, so if you want TeleMetrum boards you'll have to wait a few more days. Keith and I just weren't happy with the GPS receiver performance using the patch antennas we selected for version 1.0, and I'm changing to a slightly better (and more expensive) antenna that we expect will make everyone happier. Parts are due by Friday, and so with any luck you should be able to buy TeleMetrum boards and starter kits early next week. In the meantime, we have TeleDongle boards and other accessories in stock now and ready to ship. Last week, Steve Conklin became our first customer by showing so much enthusiasm for TeleDongle over snacks at a conference we all attended for work that we pointed him to the web store under construction and he has already posted an enthusiastic blog entry about his plans! More as it happens!

Next.

Previous.