Search Results: "ag"

22 August 2008

Enrico Zini: Unpacking the new FreeRunner

Unpacking the new FreeRunner I got myself a FreeRunner. Here are some notes from the first few days: Available operating systems The FreeRunner comes with the Om 2007.2 distribution: it works for basic phone things, and it has an opkg package manager that you can use to install all sort of extra software. Only issue: the SMS application was rather unstable with my SIM card. I then tried Om 2008.8, which is a major new redesign, partly based on Qtopia. It's definitely more advanced on how it looks, with smooth animations all around, but I did not manage to get the GSM part to work, because I did not manage to get it to ask for a PIN. I then tried plain Qtopia and I got what looks and feels like a properly working mobile phone. The only issue that it has is that I did not manage to make it suspend, so battery life is much shorter than it could be. Apparently, this should improve as kernel 2.6.26 reaches the phone. I kept Qtopia installed in the phone flash as a "stable" phone system. And finally, Debian. Debian is based on the freesmartphone.org software stack, which is an attempt to create a UI agnostic DBUS frontend to the hardware that supports multiple applications running on top of it. It is a young project, but it can already drive a mobile phone in a useful way. It has a demo interface that is basically a showcase of what is implemented in the DBUS frontend, but does most of the basic things you need from a mobile phone, including taking and making phone calls. And then it has Debian behind, all of it. I need to buy a bigger microsd card. Tips and tricks Flashing things
  1. apt-get install dfu-util
  2. Start the Freerunner bios/bootloader, by holding down Power and then pressing AUX (standard bootloader), or by holding down AUX and then pressing Power (factory, unbrickable, read only fail safe bootloader).
  3. dfu-util -l shows you a list of devices it can access. If you see more than once, you need to specify in all dfu-util commands which one you want, using -d USBID. In my case, I have to always use dfu-util -d 0x1d50:0x5119
  4. To flash the kernel: dfu-util -a kernel -R -D /path/to/uImage
  5. To flash the root file system: dfu-util -a rootfs -R -D rootfs_filename.jffs2
  6. To flash the bootloader: dfu-util -a u-boot -R -D uboot_filename.bin
  7. To flash the u-boot configuration: dfu-util -a u-boot_env -D env.new
You can also download all of these things from the FreeRunner by using -U instead of -D. Networking via USB All of the distributions I tried, by default configure the USB as a gadget with ethernet over usb. You can form a lan with it using the cdc_ether module, and you will find your phone preconfigured as 192.168.0.202 expecting to find a gateway at 192.168.0.200. I made myself this script to start and stop networking with the phone:
#!/bin/sh
case "$1" in
    start)
        iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
        ifconfig usb0 192.168.0.200 netmask 255.255.255.248
        /etc/init.d/dnsmasq start
        echo 1 > /proc/sys/net/ipv4/ip_forward
        ;;
    stop)
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /etc/init.d/dnsmasq stop
        iptables -t nat -F POSTROUTING
        ifconfig usb0 down
        ;;
esac

Besides doing masquerading, it also brings up dnsmasq so that the phone can always find a DNS together with the router. Another useful trick is to configure the phone to share the approx cache with the laptop:
deb http://192.168.0.200:9999/debian unstable main
deb http://192.168.0.200:9999/debian experimental main
deb http://pkg-fso.alioth.debian.org/debian unstable main

So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged. Changing the ringtone in Debian In case you don't like the default ringtone, this is how to change it:
  1. vi /usr/share/python-support/fso-frameworkd/framework/subsystems/oeventd/receiver.py
  2. look for def _play( self ):
  3. change the codec in decoder = gst.element_factory_make, if needed. The list of available plugins is on the gstreamer website.
  4. change the path in filesrc.set_property
  5. /etc/init.d/fso-frameworkd restart and maybe /etc/init.d/zhone-session restart
Yes, the ringtone is currently hardcoded, but it does say it's a prototype after all. Or, if you prefer, it's fully configurable and the configuration can be found in /usr/share/python-support/fso-frameworkd. Configuring the bootloader You can connect to the u-boot bootloader via a serial terminal on /dev/ttyACM0. Type help and you will find that it can do a lot of things. The OpenMoko wiki has pages on the bootloader itself, its commands and the environment. The environment is the configuration of the bootloader, similar somehow to /boot/grub/menu.lst. Unlike grub, you can edit the environment from within the bootloader and then save it using the saveenv command. What I did:
  1. A boot entry for Debian, with the kernel on an ext2 partition instead of fat: setenv menu_3 Boot from microSD (FAT+ext2): setenv bootargs \$ bootargs_base rootfstype=ext2 root=/dev/mmcblk0p2 rootdelay=5 \$ mtdparts ro\; mmcinit\; fatload mmc 1 0x32000000 \$ sd_image_name \; bootm 0x32000000
  2. Stay in the bootloader until a choice has been made: setenv bootdelay -1
  3. Don't power down the bootloader when idle: setenv boot_menu_timeout 99999
  4. Always show the menu at boot: setenv stop_in_menu yes
  5. saveenv
So now my phone dual boots, and I can choose if I want a more reliable phone now (Qtopia) or if I want to play with my future phone (Debian). Configuring ssh for two host keys on the same host Minor issue, but annoying: since both QTopia and Debian show up on 192.168.0.202, ssh will complain about changed host keys. Here is how to configure ssh to avoid the problem (in ~/.ssh/config):
Host debian
HostName 192.168.0.202
User root
HostKeyAlias debian
Host qtopia
HostName 192.168.0.202
User root
HostKeyAlias qtopia

MJ Ray: USB Charging: Greener or Not?

I’ve ordered some USB charging leads for various portable devices, mainly because I’ve already got mains-USB and car-USB power adapters (they came with a bluetooth headset that charges from USB) and I agree with things like this:
“management of an individual charger for each device is becoming a pain in the backside. Since the USB power form specification is a standard, ANY charger with sufficient load capability should be able to charge ANY device.” (from USB As A Power Source)
but I’ve also heard that charging from USB is greener - or more precisely that because your computer’s power supply is running anyway, it uses less electricity to plug battery chargers onto the Universal Serial Bus than to plug in “wall-wart” DC adapters. But is this claim true? Claims from USB recharging product sellers concentrate on the obvious “less waste compared to disposable batteries” strength, although some smarter chargers cut power to charged devices - it’s not clear whether the mass-market USB battery chargers do this, but the reasons and the electronics don’t look that complicated. China (not famous for environmental protection yet) now requires USB-charging of mobile phones while a mobile phone industry body has recommended the same approach in general. However, there are concerns about whether Power over Ethernet is making networks less green, which might be true for USB if devices demand more high-power sockets and device makers don’t include intelligent control electronics (which people generally don’t want to pay for). Is USB charging greener than mains charging? I’m not sure and I haven’t found any great answers - can you? I guess in the worst case, I can try some tests with a power meter later…

21 August 2008

Christian Perrier: Solved the Flash nightmare

Yay, Flash is now working again on my system. Since several months, I just couldn't visualize web sites using Flash anymore: Konqueror was reporting a missing plugin and Iceweasel was....crashing. As I recently got very interested in OpenStreetMap where some features use Flash, I just decided to dig in the dirt and try finding what was happening. Of course, the obvious material (gnash, konqueror-plugin-gnash, mozilla-plugin-gnash) *was* installed. So, what the heck was happening? It finally turns out that the culpri was a thing named libflash-mozplugin from the libflash source package. That thing just drops its plugin in /usr/lib/mozilla/plugins, does not conflict with the other plugin-prividing packages (which use alternatives). So, I don't know whether this is because it takes precedence over other plugins or because the conflict of plugins is bad, but indeed purging that package from my system just solved this out. According to #495040, this package should indeed be removed because of low maintenance mode and repeated crash reports. I just confirm this. Release Team, please flush this out...:-) PS: OK, sure "Flash Is Evil". But how can one imagine just using the web with your browser crashing on any page containing it?

Jos Parrella: Regarding CANTV mail services

Venezuela’s biggest and most important ISP, CANTV, was my first serious email provider, since it is tied to my ADSL connection, and I previously used several different dial-up providers (T-Net, IFX Networks) with no specific interest in email. CANTV mail service used to be provided with the highest standards of quality in the country, particularly as per malware and UCE control, and general abuse surveillance. Hey, even one of the former insider guys is quite known inside our FOSS community! I’ve previously (1, 2) blogged about the huge problems CANTV is facing with their mail service. They don’t answer, and sometimes even bounce, mails to abuse or postmaster, and temporarily suspended or degraded their abuse statistics web site. All in all, they are providing a poor customer service and are being listed in several blacklists all around the world. Sometimes I get warnings from mailing lists since their border MTAs are refusing to deliver messages to the mailboxes. So if you sent me my signed PGP public key, please forgive my ISP for probably bouncing your message. I thought I could live with this problem, since it’s probably temporary, but today a friend from El Salvador told me that all of his mail to cantv.net addresses is bouncing with this message.
Diagnostic-Code: smtp; 550 5.7.1 ...>... Mail from
    168.243.89.79 refused: Vea http://abuso.cantv.net/bl/dul.html
So after taking a look at the whois record for that block, and doing a lookup on the reverse entry for 168.243.89.79, I think that a proper conclusion would be Venezuela’s biggest ISP is currently blocking most mail from El Salvador. And probably other countries. Terrible news. And please use my at debian.org mail address if you need to get in touch.

Russell Coker: Switches and Cables

I’ve just read an amusing series of blog posts about bad wiring [1]. I’ve seen my share of wiring horror in the past. There are some easy ways of minimising wiring problems which seem to never get implemented. The first thing to do is to have switches near computers. Having 48 port switches in a server room and wires going across the building causes mess and is difficult to manage. A desktop machine doesn’t need a dedicated Gig-E (or even 100baseT) connection to the network backbone. Cheap desktop switches installed on desks allow one cable to go to each group of desks (or two cables if you have separate networks for VOIP and data). If you have a large office area then a fast switch in the corner of the room connecting to desktop switches on the desks is a good way to reduce the cabling requirements. The only potential down-side is that some switches are noisy, the switches with big fans can be easily eliminated by a casual examination, but the ones that make whistling sounds from the PSU need to be tested first. The staff at your local electronics store should be very happy to open one item for inspection and plug it in if you are about to purchase a moderate number (they will usually do so even if you are buying a single item). A common objection to this is the perceived lack of reliability of desktop switches. One mitigating factor is that if a spare switch is available the people who work in the area can replace a broken switch. Another is that my observation is that misconfiguration on big expensive switches causes significantly more down-time than hardware failures on cheap switches ever could. A cheap switch that needs to be power-cycled once a month will cause little interruption to work, while a big expensive switch (which can only be configured by the “network experts” - not regular sysadmins such as me) can easily cause an hour of down-time for most of an office during peak hours. Finally the reliability of the cables themselves is also an issue, having two cables running to the local switch in every office can allow an easy replacement to fix a problem - it can be done without involving the IT department (who just make sure that both cables are connected to the switch in the server room). If there is exactly one cable running to each PC from the server room and one of the cables fails then someone’s PC will be offline for a while. In server rooms the typical size of a rack is 42RU (42 Rack Units). If using 1RU servers that means 42 Ethernet cables. A single switch can handle 48 Ethernet ports in a 1RU mount (for the more dense switches), others have 24 ports or less. So a single rack can handle 41 small servers and a switch with 48 ports (two ports to go to the upstream switch and five spare ports). If using 2RU servers a single rack could handle 20 servers and a 24port switch that has two connections to the upstream switch and two spare ports. Also it’s generally desirable to have at least two Ethernet connections to each server (public addresses and private addresses for connecting to databases and management). For 1RU servers you could have two 48 port switches and 40 servers in a rack. For 2RU servers you could have 20 servers and either two 24port switches or one 48port switch that supports VLANs (I prefer two switches - it’s more difficult to mess things up when there are two switches, if one switch fails you can login via the other switch to probe it, and it’s also cheaper). If the majority of Ethernet cables are terminated in the same rack it’s much harder for things to get messed up. Also it’s very important to leave some spare switch ports available as it’s a common occurrence for people to bring laptops into a server room to diagnose problems and you really don’t want them to unplug server A to diagnose a problem with server B… Switches should go in the middle of the rack. While it may look nicer to have the switch at the top or the bottom, that means that the server which is above or below it will have the cables for all the other switches going past it. Ideally the cables would go in neat cable runs at the side of the rack but in my experience they usually end up just dangling in front. If the patch cables are reasonably short and they only dangle across half the servers things won’t get too ugly (this is harm minimisation in server room design). The low end of network requirements is usually the home office. My approach to network design for my home office is quite different, I have no switches! I bought a bunch of dual-port Ethernet cards and now every machine that I own has at least two Ethernet ports (and some have as many as four). My main router and gateway has four ports which allows connections from all parts of my house. Then every desktop machine has at least two ports so that I can connect a laptop in any part of the house. This avoids the energy use of switches (I previously used a 24 port switch that drew 45W [2]), switches of course also make some noise and are an extra point of failure. While switches are more reliable than PCs, as I have to fix any PC that breaks anyway my overall network reliability is increased by not using switches. For connecting the machines in my home I mostly use bridging (only the Internet gateway acts as a router), I have STP enabled on all machines that have any risk of having their ports cross connected but disable it on some desktop machines with two ports (so that I can plug my EeePC in and quickly start work for small tasks).

Steve Kemp: I would have started with lasers, eight o'clock, Day One

This week has been a little hectic, as I've been struggling with testing different versions of the GNU/Linux kernel. Specifically I've been trying to solve a problem where a Phenom processor, when coupled with 8Gb, would kernel panic under heavy load. After testing various patches, kernel versions, and random things I believe the problem is fixed in the kernel version 2.6.27RC4 - however nothing in the changelog appears relevant, so I guess only time will tell. Now we need to solve the problem of Atom processors panicing when attempting to boot 64-bit kernels. That is still present in the 2.6.27RC4 kernel. (ObRandom: If there are any interested parties I can provide remote serial console access to such a system.) Finally I've also been playing with PAM, the plugabble authentication module. Again specific use-case here. At work we want to allow people to ssh to some systems (to access serial consoles, etc), and we wish their connections to be tested against our internal single-sign-on mechanism. That could have meant a whole new PAM module, which would do XML-RPC-fu. Instead it meant packaging libpam-external - which is a neat PAM module allowing you to specify a shellscript to validate users & passwords. (libpam-external is very similar to mod_authnz_external which is a similar pluggable Apache2 module) So, this week "kernel hacking", & "pam hacking". Does that make me a real developer now? ObQuote: Time Bandits

Holger Levsen: Custom Debian Distributions shall now be called...

Debian Integrated Solutions. (DIS, pronounced "dish".)

Or at least, that's what seems to become the result of this thread on thedebian-custom mailinglist (and some RL discussions at DebConf8).

I've not yet participated in the renaming discussions, at the moment I can only say what others already said: the terms "custom" and "distribution" both have several problems and renaming won't be easy. Now that I've said that, it occurs to me that I do have something to add: renaming established names is never easy, but can very well be worth the effort, if the established name is problematic / has shortcomings and the new one is way better.

And there I have my doubts. TTBOMK or IME, neither Debian Med nor Debian Edu qualify as "solutions", but are a (great) basis to build solutions...

Hmm, I mostly started to write this blog post, to make more people aware of this discussion and not to add new arguments. This should better be done on the debian-custom mailinglist. Please reply there :-)

In not totally unrelated news I'm back in Europe and catching up on my various backlogs. Tomorrow I plan to connect the harddrive with the Debconf8 videos and start/continue with that backlog... If you notice any problems with the already released videos, please don't tell me, but put them in that wiki page. Thanks!

David Welton: Introductions

I'm doing some contracting work at a local firm, and one of the little details that I have been pleased with is the fact that they seem to do introductions for people. On my first day, I was taken around and introduced to everyone, despite the fact that I probably won't be there long term (depending on where we go and when). Today, they were showing around a guy who will be doing a sort of student internship there, and once again, they stopped by my office. I appreciate that a lot, and think that it's a good thing for a group of people to at least have a bit of an idea who newcomers are. One of the things that I have always loathed, and vowed to do differently should DedaSys ever be more than myself, is feeling like an animal in a zoo when The Boss brought someone into the office to take them on a tour, pointing at us developers and talking about what we do, with no reciprocity. "This is Dave, one of our open source guys, who is really into Linux. He once roamed the temperate forests of Oregon and his mating season is from early January to late December." We get pointed at and talked about, and the other person remains a Mysterious Stranger. Granted, it's not always possible to disclose everything, but I think that you can still do a lot to make people feel respected without spilling the beans on a big potential customer or investment.

Michael Schutte: Hello Planet Debian!

Thanks to Anton Zinoviev, http://planet.debian.org/ is subscribed to my feed for months already, but I ve never gotten around to actually post something. So let s start in accordance with tradition! My name is Michael Schutte and I m yet another Debianist from Innsbruck, Austria. I have been around since about February, and I currently maintain kbd, python-odtwriter, and libgit-ruby. Then, I participate in the Debian/Ruby Extras team. I also contribute some QA work when time allows I even managed to fix an ugly security bug in an orphaned package once. I ll try to write down things here every once in a while (without breaking Planet too often).

Benjamin Mako Hill: Story of Josephville

After a late-night IRC conversation about egg corns, shaggy dog stories and feghoots the idea for a short story came to me in the bathtub this morning. I give you, The Story of Josephville. Apologies in advance.

Ingo Juergensmann: Adding a RAID under a LVM

Just as a personal reminder for the next time I need that:

If there's a machine, for example a Xen host, with just one harddisk and you want to be somewhat safe in case of a disk failure, you can add a second disk and setup a RAID1 (mirroring). It's most simple to shutdown the machine and setup the RAID1 for the dom0. Afterwards you can start the domU VMs again when you're using LVM for their virtual disks. Following these steps will add RAID1 underneath your LVM setup:

0. /dev/sda5 is your current physical volume for your LVM, on the second (equally partitioned) disk /dev/sdb5 will be your starting point for your RAID1 (/dev/md3)
1. create a degraded RAID1 on /dev/sdb5:
# mdadm --create /dev/md3 --level=1 -f --raid-devices=1 /dev/sdb5
mdadm: array /dev/md3 started.
# cat /proc/mdstat
md3 : active raid1 sdb5[0]
135540288 blocks [1/1] [U]

2. prepare it for LVM and add it to your virtual group:
# pvcreate /dev/md3
# vgextend /dev/vg /dev/md3
Volume group "vg" successfully extended

3. move the extends from one disk to the other and be verbose to watch the progress:
# pvmove -v /dev/sda5
Finding volume group "vg"
Archiving volume group "vg" metadata (seqno 22).
Creating logical volume pvmove0
Moving 192 extents of logical volume vg/ts2-swap
Moving 1536 extents of logical volume vg/ts2-disk
[....]
Found volume group "vg"
Removing temporary pvmove LV
Writing out final volume group after pvmove
Creating volume group backup "/etc/lvm/backup/vg" (seqno 31)

4. remove the old partition from the virtual group:
# vgreduce /dev/vg /dev/sda5
Removed "/dev/sda5" from volume group "vg"

5. add the old partition to your raid and set it active:
# mdadm --add /dev/md3 /dev/sda5
mdadm: added /dev/sda5
# cat /proc/mdstat
md3 : active raid1 sda5[1](S) sdb5[0]
135540288 blocks [1/1] [U]
# mdadm --grow /dev/md3 --raid-devices=2
# cat /proc/mdstat
md3 : active raid1 sda5[2] sdb5[0]
135540288 blocks [2/1] [U_]
[>....................] recovery = 0.2% (284480/135540288) finish=39.6min speed=56896K/sec

6. let the RAID1 doing its sync and you're done.

Just as a small side note:
I don't know why the kernel don't want to boot when it get passed root=/dev/md2 as a kernel parameter. It works with root=/dev/sda3, though...

Wouter Verhelst: Solving sudoku puzzles in package dependency relations

Daniel, you are evil. But yeah, it does look to be quite some fun.

MJ Ray: How To Avoid 6 Common Website Mistakes That Cost Money

According to a recent article, 6 Common Website Mistakes That Cost Money are:
1. JavaScript or other crawler-unfriendly navigation that may impede indexing
This one is best avoided at design-time, by including Web Content Accessibility Guidelines 1.0 Level A in the design brief, but if you’ve ended up with JavaScript-based navigation on your site (check by doing “View Source” and searching for the code for your home page link and so on - if you can find it, then it’s probably not javascript) and it’s based on some template system, a webmaster can probably do a whole-site edit to put the navigation links in the page properly - or at least add a useful <noscript> tag.
2. Navigation that buries important pages within the site architecture.
The structure of the website’s files and the structure of the link menus do not need to match, so if there’s a page that you feel is important, get your webmaster to add it to the navigation links across the whole site.
3. Duplicate “pages” getting indexed under multiple URLs.
This usually happens for one of two reasons: one is inappropriately-parked domains, which is mistake 6 below and often fairly easy to fix; the other is a misbehaving web application, which you’ll need to get a programmer to fix. You can do a simple test of your web application by starting at your homepage and following links to a particular page; then open a new browser window and try to reach the same page by different links (or a site search) and compare the address bars (the bit of your browser showing http://) - do they match? If not, you’ve got this problem.
4. No keyword phrase focus in the content or conversely, keyword phrase stuffing
You can use a good word-counter on the text to see how common different keywords and keyphrases, or use the “webmaster tools” section of some search engines to see what they’re focusing on. Keyword-stuffing can be fairly easy to see. If you “View Source” on a page and there’s a large block of keyword-intensive text somewhere in it that doesn’t appear it when viewed in a browser, then it’s probably stuffing. Many sites regard stuffing as a sort of spam, so you don’t want to be found doing this. To fix these problems, rewrite the page text appropriately.
5. An optimized home page, but that’s it
Repeat your checks from problem 4 on a few pages other than the home page to discover whether you suffer from this. Also, see whether your website statistics show search engine visitors arriving at a variety of different pages (these are sometimes called “Entry Page” statistics).
6. Additional domains owned by the company are not properly redirected
If you usually use .co.uk, but you also have a .com domain, try visiting a random page on your website, then click in the address bar and replace the .co.uk with .com - what happens? If the page is Not Found, then your domains aren’t properly redirected and you need a Redirect adding to the second one. If the page displays but the address doesn’t change itself back, then your domains are probably pointing at the same webspace but aren’t properly redirected, which will mean you’re probably making mistake 3 above. Usually, the simplest way to fix this is with a conditional redirect. On Apache webservers, you can add a .htaccess file containing something like:-
RewriteEngine On
RewriteCond % HTTP_HOST  !^www\.yourdomain\.co\.uk$ [NC]
RewriteRule (.*) http://www.yourdomain.co.uk/$1 [R=permanent,L]
That even works on BT Business web hosting, by the way. If it doesn’t work, try asking your web hosting provider to enable mod_rewrite for you or ask them to suggest how to achieve the same result.
The above list of mistakes was published on Search Engine Land last week. The correction methods are all things used by my webmaster cooperative.

Frank Lichtenheld: Source Views

(This is a more detailled and hopefully better structured explanation of the topic of my Lightning Talk during Debconf) With the ever growing number of packages in our archive we need ever better methods to structure, sort, and search this collection. While projects like debtags try to improve the search in packages, my idea is about improving the way the results are displayed. There is one relation between packages that is currently never strongly featured when presenting package meta-data: the source ↔ binary relation. Usually package managers operate either on binary packages or source packages, but seldomly do they expose the fact that one is generated from the other. In the trivial case of one source package producing one binary package there is probably nothing to gain. But there are a large number of source packages where this isn't true (at the moment about 29% of all the source packages in main build more than one binary package, together producing about 61% of all binary packages). I have the idea (which so far is nothing more) that if we could present packages through their source packages that this could drastically reduce the number of packages to choose from and make some common search actions more effective. This is not by any meaning a trivial task, though, since on the other hand not all source packages necessarily produce binary packages that have much to do with each other, and even for the ones that do, the exact relationship between the binary packages can vary greatly. Also some source packages are really belonging together but are split to make maintenance easier (e.g. KDE and X.org). These splits might not necessarily make any sense to an user. Some ideas what to implement in terms of code to make it easier to let the user benefit from the fact that multiple binaries have the same source:
Add support for a Description field for source packages.
For bonus points implement automatic substvars $ source:Description , $ source:Short-Description , and $ source:Long-Description that make it trivial to reuse (parts of) that description in the descriptions of the binary packages.
Add some way to identify the "main" package built from a common source.
Identify source package usage patterns.
There are a lot of source packages that share common patterns about what they build. Example patterns include:
Application
Main package + -data package + -doc package + ...
Library
-dev package + shlib package + -doc package + ...
Application + Library
Collection of Applications
etc.
I guess many of us already look out for these patterns when searching for packages but identifying them explicetly might make this easier and allow more users to benefit from this information.

Frank Lichtenheld: RC bugs in etch (Starting Stable QA, Part 1)

As everyone can see in the graph of release critical bugs, the number of RC bugs in etch had constantly risen since the release (which was already noted in some other blogs) up until last week. Since I suspected that this number included a lot of false positives, I began to triage this list last week during Debconf. I started with identifying all bugs that were filed against the version in etch, but really only meant the package in testing/unstable (but can't be interpreted correctly with version tracking alone since the version in stable, testing, and unstable is actually identical). There are a lot of reasons why an RC bug might be valid for testing/unstable, but not for stable, even though they share the same package:
Library transitions.
While most cases of library transitions can be handled with targeted binNMUs these days, there remain a few cases where a sourceful upload is needed, e.g. for renamed -dev packages and necessary source changes to adapt to new APIs.
Build-Problems caused by toolchain changes.
Newer gcc versions are often more strict about what constructs they allow. This can cause build problems in a lot of packages once the default version used is increased. Other packages that regulary cause new build problems are linux-libc-dev and dpkg-dev. Since we only require that a package is buildable in the release it is contained in, these are not valid RC bugs for stable.
Changes in the RC policy of the release team.
One example during the lenny release cycle was the use of invoke-rc.d in maintainer scripts to (re-)start daemons. Bugs were filed against a lot of packages not using invoke-rc.d before the etch release but they were only declared to be of RC severity after etch.
etc.
So as the first part of my triaging efforts I tried to identify these cases. I also looked for bugs that were listed as affecting etch due to incomplete version information (i.e. if there is no version given, the bug is assumed to affect all versions). As you can see from the bump in the graph I was able to identify about 200 bugs that met one of these conditions. We should avoid in the future to let the number of false positives grow to such large numbers. Everyone can help with that:
Maintainers
If you get bugs reported without a version number and you can verify that this bug was not present in an older version of the package, add correct "found" information. If you get a bug reported against a version that is both in stable and in testing/unstable, but not valid for stable, tag the bug appropriatly. I would recommend to err on the side of false positives though instead of on the side of false negatives!
Mass-bug filers
Most mass-bug filing that involve RC bugs should use tags to avoid creating false positives.
QA Group
Bugs filed about the proposed orphaning or removal of packages should usually be tagged, since only in very few cases these actually warrant any changes in stable.
At this point you probably ask yourself: What are the appropriate tags? This is much less clear than one might hope, since the involved tags (<suite>) changed/lost their meaning somewhat with the introduction of version tracking. Following my question about the appropriate tags on debian-release the release team and Don Armstrong for the debbugs maintainers seem to have agreed on their new meaning: A bug with suite tags affects the intersection of the set of suites indicated by its version information and the set of suites indicated by its suite tags. Next post: What about the other 600 RC bugs in stable?

Josselin Mouette: Don t worry, your data is safe

Computer security is generally defined as the following:
  1. The data should remain available to those who need it.
  2. The data should not be available to those who don t have the right to.
The critical software component to the first item is your well-known nightmare: the backup software. Backups are easy! In many corporate environments, here is how you ensure you have backups: Isn t that cool? So little work to have all your data completely safe! Wait if you look more closely, there is something wrong here. Nowhere in this process did you specify credentials to the backup guy, nor did you add anything to identify the backup server. Or to say it otherwise, once you have installed the backup agent, the backup server has full access to your data. But you didn t even specify anywhere the IP of the backup server. So this means that anyone on the network has full access to your data. Which, while trying to make your data secure, actually brings a gaping security issue on your system. Security by nullity When you install a well-designed backup agent, like Bacula, you have to specify a password for each client and make this password known to the backup server. A simple authentication protocol ensures that only the backup server is able to backup your data. However, when you install, for example, the HP Data protector agent, it starts listening on a TCP socket and (no kidding!) binds it to a restricted shell which has access to a small list of commands. The backup server only needs to connect to this TCP socket and issue commands. While this has the great advantage of simplifying the development process of the backup server, such a software has a name: a rootkit. Several other proprietary backup software have different implementations, like RPC-based or proprietary protocols, but the basis remains the same: you connect to the TCP port, and you have a way to read absolutely any file on the system. Of course, there are so-called security options, that you can buy besides the disk agent or the nifty web interface. Yes, when you buy software that is critical for your security, the very lowest level of security that you d expect from any software not turning your box into a self-service is a paying option. In the end, what prevents your data from being available to anyone on the intarweb is the ultimate solution to everything(tm): the firewall. As people are not stupid enough to use the same backup system in the DMZ, you can t simply bounce from it after having used a hole in the lousy PHP code that is never updated. (Well, that will still work in many cases since people actually are stupid enough.) No, what makes it very fun is the backup network. The hackup network A tape library is expensive hardware, therefore you only want to buy one. However, when you have several departments or, in outsourced environments, several clients, you can t simply access to the backup agents on all these servers, because of all these evil routers and firewalls that are blocking you. The solution is trivial: use the second network card on all servers to connect to a backup network. The backup network is entirely managed by backup people who have no idea of network security, and is routed all the way down to the backup server. If you are lucky, some ACLs set up by the network guy who plugged the switch will prevent some parts of the network to access some others. In all cases, it s very likely that many people in the company (or from other companies!) have full IP access to all your private servers where sensitive data is stored. That includes access to your databases setup without a password, and of course to your highly secure backup agent. That generally also includes full IP access to the lousy maintained backup server, on which security updates are never installed. But don t worry. Your data is safe. Note: no, this is not a worst-case scenario. Such badly-designed software is widespread. Such practice is widespread in several companies I know.

Russell Coker: Purging an RT Database

I had a problem where the email address circsales@washpost.com spammed a Request Tracker (RT) [1] installation (one of the rules for running a vaction program is that you never respond twice to the same address, another rule is that you never respond to automatically generated messages). Deleting these tickets was not easy, the RT web interface only supports deleting 50 tickets at a time. To delete them I first had to find the account ID in RT, the following query does that:
select id from Users where EmailAddress='circsales@washpost.com'; Then to mark the tickets as deleted I ran the following SQL command (where X was the ID):
update Tickets set Status='deleted' where Creator=X; Finally to purge the deleted entries from the database (which was growing overly large) I used the RTx-Shredder [2] tool. RTx-Shredder doesn’t seem to support deleting tickets based on submitter, which is why I had to delete them first. I am currently using the following command to purge the tickets. The “limit,500” directive tells rtx-shredder to remove 500 tickets at one time (the default is to only remove 10 tickets).
./sbin/rtx-shredder --force --plugin 'Tickets=status,deleted;limit,500' There are currently over 34,000 deleted tickets to remove, and rtx-shredder is currently proceeding at a rate of 9 tickets per minute, so it seems that it will take almost three days of database activity to clear the tickets out. I also need to purge some tickets that have been resolved for a long time, I’m running the following command to remove them:
./sbin/rtx-shredder --force --plugin 'Tickets=status,resolved;updated_before,2008-03-01 01:01:34;limit,500' With both the rtx-shredder commands running at once I’m getting a rate of 15 tickets per minute, so it seems that the bottleneck is more related to rtx-shredder than MySQL (which is what I expected). Although with two copies running at once I have mysqld listed as taking about 190% of CPU (two CPUs running two capacity). The machine in question has two P4 CPUs with hyper-threading enabled, so maybe running two copies of rtx-shredder causes mysqld to become CPU bottlenecked. I’m not sure how to match up CPU use as reported via top to actual CPU power in a system with hyper-threading (the hyper-threaded virtual CPUs do not double the CPU power). I wonder if this means that the indexes on the RT tables are inadequate to the task. I tried adding the following indexes (as suggested in the rtx-shredder documentation), but it didn’t seem to do any good - it might have improved performance by 10% but that could be due to sampling error. CREATE INDEX SHREDDER_CGM1 ON CachedGroupMembers(MemberId, GroupId, Disabled);
CREATE INDEX SHREDDER_CGM2 ON CachedGroupMembers(ImmediateParentId, MemberId);
CREATE UNIQUE INDEX SHREDDER_GM1 ON GroupMembers(MemberId, GroupId);
CREATE INDEX SHREDDER_TXN1 ON Transactions(ReferenceType, OldReference);
CREATE INDEX SHREDDER_TXN2 ON Transactions(ReferenceType, NewReference);
CREATE INDEX SHREDDER_TXN3 ON Transactions(Type, OldValue);
CREATE INDEX SHREDDER_TXN4 ON Transactions(Type, NewValue);

20 August 2008

Andrew Pollock: [life] Snail mail spam control

John Goerzen writes an entertaining piece on his ongoing battle with Dell to stop receiving advertising from them. (The comments on the post are rather interesting in themselves) I say "good on you, John" for taking a stand. Now that I know of these prohibitory orders, I might try them out on Pennysaver so I stop getting their stupid catalogs. A while ago our junk mail situation got completely out of hand, and at the time I found 41pounds.org, which for a very reasonable $41, did a very good job of stopping pretty much all of the addressed spam. About the only thing we get now is a small wad of catalogs that USPS delivers to every resident, which isn't explicitly address to us by name, just the apartment number. That and these stupid "checks" from Washington Mutual that we can use against our credit card account with them. They give me the screaming heebie jeebies, and I shred them on sight, because I've seen how insecure cheques are.

Adam Rosi-Kessel: Walter Michalik and the Roslindale Community Center

I’ve had little time in the four years I’ve lived in Roslindale to get involved in community volunteer efforts. My wife Rachele, however, has devoted several years to efforts such as the Roslindale Community Center, Roslindale Village Main Streets, and Roslindale Clean and Green. She has served as chair of both RCC and one of the RVMS committees. Over the past few months, I’ve been disappointed to see the RCC board endure repeated senseless attacks by a perhaps overzealous community member, Walter Michalik, who has effectively paralyzed the organization. I have nothing personal against this guy, but I thought I could do my part here by publishing an example of an email he broadcast to the committee, which I think is embarrassing enough on its own merits to need no editorializing from me, except to say any email that starts with “just so you won’t be surprised when the IRS launches its investigation” can be neither productive nor taken seriously. The context is a response to an announcement sent out pursuant to the Community Centers By-Laws scheduling a special meeting to discuss the recent upheaval on the Council. There’s no reason this should not be in the public record:
Rachele Just so you won’t be surprised when the IRS launches its investigation, be advised that YOU DO NOT HAVE THE AUTHORITY TO CALL FOR A SPECIAL MEETING. The By Laws do not allow the President/Chair to call for a Special meeting - that right is reserved for the MEMBERS to have a power over a hostile or non-performing President/Chair. It was created by a Membership wary of annimosity within its ranks in order to protect itself from falling into a useless dysfunctional body. That’s why our By Laws require the majority of the membership request it through the Chair and not the other way around. So you screwed that up too. Your continuous violation of rules and laws have proven beyond a doubt that this current Roslindale Community Center Council has violated the trust behind its IRS-designated not-for-profit status. The failures are on so many significant levels over 6 months it is readily apparent that the Roslindale Community is better served without a Community Center Council. There is no longer any excuse to allow the residents of Roslindale to be denied the services they deserve that you and the officers have no idea how to deliver. Because that’s what you are supposed to do - serve the residents of Roslindale and not preseide over a self-serving board. You’ve taken us from dysfunctional to non-functional over the past 60 days. You and your officers have already driven away our Archdale partners and now the merger is off. Congratulations. You got what you wanted. You and the officers have withheld information from the Members of the Council imposing gag orders along the way and in more than one instance spreading lies within this community. There are so very many lapses on your part and the rest of the officers that it is very apparent that the Roslindale Community Center exists IN SPITE OF the Roslindale Community Center Council - not because of it. It would be far better indeed, if the Center were run without the Council. What have you done in 6 months? All you do is avoid the work of leadership. Please, just gracefully step aside and let a real leader emerge - or dissolve the Council and let the Center run itself under BCYF before the IRS makes that happen. Of course this is my stated opinion. You can choose not to belive it. You can huddle with a group of self-serving officers and invent excuses or change the agenda away from the message and continue with your ad hominem attacks against the messenger. I’m thick-skinned. Feelings aren’t important to me - services to residents are important to me. While I have, in fact, enjoyed the confidences of others within this community and their true displesure with this Council, I will produce that to the appropriate authorities as I see fit along with the Bill of Particulars I have produced with documentation for all charges. As a founder of the Roslindale Community Center Council I can no longer support you or the officers as any legitimate authority in this community.
Technorati Tags: , ,

Russell Coker: DNS Secondaries and Web Security

At the moment there are ongoing security issues related to web based services and DNS hijacking. the Daily Ack has a good summary of the session hijacking issue [1]. For a long time it has been generally accepted that you should configure a DNS server to not allow random machines on the Internet to copy the entire zone. Not that you should have any secret data there anyway, but it’s regarded as just a precautionary layer of security by obscurity. Dan Kaminsky (who brought the current DNS security issue to everyone’s attention) has described some potential ways to alleviate the problem [2]. One idea is to use random case in DNS requests (which are case insensitive but case preserving), so if you were to lookup wWw.cOkEr.CoM.aU and the result was returned with different case then you would know that it was forged. Two options which have been widely rejected are using TCP for DNS (which is fully supported for the case where an answer can not fit in a single UDP packet) and sending requests twice (to square the number of combinations that would need to be guessed). They have been rejected due to the excessive load on the servers (which are apparently already near capacity). One option that does not seem to get mentioned is the possibility to use multiple source IP addresses, so instead of merely having 2^16 ports to choose from you could multiply that by as many IP addresses as you have available. In the past I’ve worked for ISPs that could have dedicated a /22 (1024 IP addresses) to their DNS proxy if it would have increased the security of their customers - an ISP of the scale that has 1024 spare IP addresses available is going to be a major target of such attacks! Also with some fancy firewall/router devices it would not be impossible to direct all port 53 traffic through the DNS proxies. That would mean that an ISP with 200,000 broadband customers online could use a random IP address from that pool of 200,000 IP addresses for every DNS request. While attacking a random port choice out of 65500 ports is possible, if it was 65500 ports over a pool of 200,000 IP addresses it would be extremely difficult (I won’t claim it to be impossible). One problem with the consideration that has been given to TCP is that it doesn’t account for the other uses of TCP, such as for running DNS secondaries. In Australia we have two major ISPs (Telstra and Optus) and four major banks (ANZ, Commonwealth, NAB, and Westpac). It shouldn’t be difficult for arrangements to be made for the major ISPs to have their recursive DNS servers (the caching servers that their customers talk to) act as slaves for the DNS zones related to those four banks (which might be 12 zones or more given the use of different zones for stock-broking etc). If that was combined with a firewall preventing the regular ISP customers (the ones who are denied access to port 25 to reduce the amount of spam) from receiving any data from the Internet with a source port of 53 then the potential for attacks on Australian banks would be dramatically decreased. I note that the Westpac bank has DNS secondaries run by both Optus and Telstra (which makes sense for availability reasons if nothing else), so it seems that the Telstra and Optus ISP services could protect their customers who use Westpac without any great involvement from the bank. Banks have lots of phone lines and CTI systems. It would be easy for each bank to have a dedicated phone number (which is advertised in the printed phone books, in the telephone “directory assistance” service, and in brochures available in bank branches - all sources which are more difficult to fake than Internet services) which gave a recorded message of a list of DNS zone names and the IP addresses for the master data. Then every sysadmin of every ISP could mirror the zones that would be of most use to their customers. Another thing that banks could do would be to create a mailing list for changes to their DNS servers for the benefit of the sysadmins who want to protect their customers. Signing mail to such a list with a GPG key and having the fingerprint available from branches should not be difficult to arrange. Another possibility would be to use the ATM network to provide security relevant data. Modern ATMs have reasonably powerful computers which are used to display bank adverts when no-one is using them. Having an option to press a button on the ATM to get a screen full of Internet banking security details of use to a sysadmin should be easy to implement. For full coverage (including all the small building societies and credit unions) it would be impractical for every sysadmin to have a special case for every bank. But again there is a relatively easy solution. A federal agency that deals with fraud could maintain a list of zone names and master IP addresses for every financial institution in the country and make it available on CD. If the CD was available for collection from a police station, court-house, the registry of births, deaths, and marriages, or some other official government office then it should not have any additional security risks. Of course you wouldn’t want to post such CDs, even with public key signing (which many people don’t check properly) there would be too much risk of things going wrong. In a country such as the US (which has an unreasonably large number of banks) it would not be practical to make direct deals between ISPs and banks. But it should be practical to implement a system based on a federal agency distributing CDs with configuration files for BIND and any other DNS servers that are widely used (is any other DNS server widely used?). Of course none of this would do anything about the issue of Phishing email and typo domain name registration. But it would be good to solve as much as we can.

Next.

Previous.