Search Results: "will"

19 July 2006

Erich Schubert: Stupid Israelis, stupid Hisbollah

Neither of them can win this war. It will only cause more hatred. Killing civilians has always and will always be a bad idea. So please stop this stupid war. The only way of winning by military means would be genocide. Or more likely, geocide, since you'd need to get rid of all relatives and friends of your opponents around the world... Not really an option, is it? The thing which bothers me is that the Israelis should know better. They've been trying this for years and it didn't help. Now why are they doing such a large military operation again? I mean, what is the true reason? Read a statement by UNICEF and the WHO.
The psychological impact is serious, as people, including children have witnessed the death or injury of loved ones and destruction of their homes and communities.
Guess what, these will probably devote their life now to attacking Israel. So no peace for Israel and the middle east for the next 20 years either... Apparently both the U.S. and Israel like to use the excuse of fighting "terrorists, and not countries" to not follow international humanitarian laws. Which doesn't make any sense; if you want to fight terrorism, you must go way beyond what humanitarian laws enforce.

Philipp Kern: Melting

Tomorrow we will encounter 38 C over here. I will be melting. Now. Really.

Erich Schubert: Google and the semantic web

Worth reading: article on Googles Peter Norvig and Tim Berners-Lee over the Semantic Web. While Google is obviously right about "millions of web masters" who will have trouble adopting to these new standards - thats why we're writing software. The days when you were hand-writing HTML code will be over some day. Already now - as Google acknowledged - many people fail to write proper HTML. But HTML isn't becoming easier to write. In fact, it gets much more complex with different character encodings, new CSS versions, raised bars for layouting, more dynamic web pages, Ajax, ... More and more people won't write HTML themselves anymore, but use some software. People used to write bad HTML; now people use tools such as Wordpress which are at least expected to produce valid markup. They start using visual editors, which will eventually stop using tags such as <i /> and use the "more semantic" <em /> instead. Without actually being aware of that. And their blog software also does generate RSS for them. How many people have ever written RSS by hand? So it's mostly a matter of the tools we offer them; with better tools we can push the use of better ("semantic") formats which then make data reuseable for others as well. For example, tons of people hope that friendster, openbc, linkedin and all these will help them in one way or another to keep contact with some people, sell some products, find new jobs. What these web pages basically collect is FOAF data. (And, btw, if any of these web sites were true Web 2.0, they would actually export FOAF files via some API!) They have a UI people understand; now all they would need to do is share their data, and we'd have a large body of "semantic web ready" FOAF data. Similar things apply to other "semantic" formats. Think calendards. Ical is pretty much the standard and widely used. Almost noone uses web pages which are only readable by humans. The semantic web isn't dead or anything. It just takes some time to be widely adopted, but that was to be expected. And having tools to generate semantic data that are maybe even easier to use than non-semantic tools - after all, the computer should be able to assist you more with semantic data - is the key thing to success. I'm also looking forward to Semantic wikis such as IkeWiki, that try hard to make entering semantic data as easy as editing a non-semantic wiki page. In large wikis such as Wikipedia, making useful links is not as easy as typing [MagicWord], because you first have to look up the magic word. A semantic wiki can assist you by suggesting appropriate links based on the information you've already entered. (e.g. if you mark a page as biology, it won't suggest you it might be a computer part).

Holger Levsen: debconf6 videos "beta release"

Two month ago we where in the middle of debconf6... Yesterday I was kicked out of my bed by a friendly person delivering 36 DVDs with high quality recordings of the videos we took there. Yay! And thanks for sending those over!

Also thanks to Ben many (if not all?!) are already available in lower qualities in the debian meetings archive..

Some encodings, an index with better navigation and the slides are still missing - (help welcome!) - but I felt I should let you know about the videos which are available now. (Once everything is ready we will send out a proper announcement.) Enjoy!

Steve Kemp: So hot, so cold, so far so out of control

The cron patch I posted previously attempted to discard any crontab file which contained non-ASCII characters was a flawed approach. Lots of people told me that this would break scripts with foreign character sets, and that was something I’d not considered. A real solution would be to read each line and test for : Looking at the code for cron is hard because it is written in a “strange” style, so I’m not even going to attempt it this level of validation. However I have produced an updated patch which should work on ASCII + foreign character sets equally well. Instead of testing for the presense of non-ASCII characts it looks for NULL characters (’\0′). Binary files should contain at least one NULL character - as a string terminator, for example - and text-files should never contain them. So I think this simple test will work for 99% of cases.
Regardless of whether this patch, or another like it, is accepted and applied I do believe strongly that whilst being liberal in what you accept is a good thing being too liberal is asking for trouble.

Erich Schubert: Do you remember Google Pages?

Back in february, Google launched it's page creator. Still the best Ajax web page editor I've seen so far. Still barely anyone knows it. After the initial hype (during which it was temporarily unavailable), noone talked about it any more. Actually it's a shame; this is probably the easiest way to setup a web page. It will be interesting to see what Google's strategy is with respect to products such as this. Let it die? Make some marketing campaign? Do some cross marketing (e.g. adding it to the "Mail / Calendar / More" bar you have at the top of mail and calendar pages)? Actually, there are tons of Google apps very few people know; only a few have been a big success (search, mail, maps, earth, news). They certainly could use some marketing. But maybe they have a reason to wait. Imagine Google starting a marketing campaign with all their cool products ("out of beta now") just a few weeks before Vista is released...

Erich Schubert: nofollow followup.

Micah Dubinko blogs on rel="nofollow" being a failure. I have to agree that this doesn't really stop comment spammers currently. I still believe that it will help on the long run, since most blog software will now use it by default. And people have to upgrade from time to time due to the lastest PHP security issue... However, I would like Google or other search engines to offer some "spam submission". It would be best if they had a common submission server. Then we could use the moderation tools of our blogs to submit these bad URLs to Google, who in turn could take measures to strip them off their index (e.g. by moderating indexing for these URLs, reducing their pagerank, freezing their ranking for some weeks etc.) I'm of course aware of abuse possibilities (e.g. spam with your competitors URL); but most comment spam I've seen so far uses new sites anyway. I think you can detect these by charcteristical URLs, site contents, incoming links etc. As for myself, I've never enabled comments on my blog for the very reason that I'm not willing to moderate all that spam. In my guestbook (which is mostly to reduce the number of emails I get because of certain popular contents of my german web page; people will happily post in my guestbook instead of sending me an email now), I've recently enabled a very simple filter. It will just reject any entry which contains http:// anywhere. My users don't need that, and it has reduced spam to 0 again. And I don't care for the entries there anyway. ;-)

18 July 2006

Jordi Mallach: Seventy years of the Spanish Civil War and the Spanish Revolution

A few months ago I wrote about the 75th anniversary of the proclamation of the Second Spanish Republic which was celebrated throughout the state. That weekend I visited a fantastic exposition of Republican and Revolutionary war propaganda posters, in Segorbe.

Today, Spain remembers the beginning of the end. During the evening of the 17th of July and morning of the 18th, a group of generals of the Spanish regular army raised against the democratically elected left-wing government of the Republic. History says that the murdering of Calvo-Sotelo, a right-wing leader, was what finally provoked the conflict. It was no secret that the same right-wing and some elements in the army had been conspiring for weeks about an uprise. The only question was when, but the government was fooled by general Mola and didn't take really effective measures to prevent it.


Warning from sanitary authorities against venereal diseases Mola and Sanjurjo planned to start taking over official buildings and army quarters and quickly seize control of the major cities. During the first hours of their rebelion, they managed to take over most of the southern regions in Andalusia, Extremadura and parts of the Northwest, plus most of the islands and the Moroccan protectorate area. Meanwhile, the Republic did not manage to react quickly due to the weakness of the government, and in the first days of the crisis, the government changed a few times. Finally, the Republican parties decided that to defend the state they should give arms to the unions, and soon after many militia groups were getting organised in the cities, and columns sent off to liberate other towns.


Solidarity of Catalunya to the Basque front The insurrection was suffocated in Barcelona after some heavy fighting against the military in the Montju c castle; the Anarchists of the CNT grew strong in Catalunya and Arag n, and offered good resistance in those fronts. After the first days of the coupe d'etat, it was clear neither the legitimate government or the rebels calculated how things would go after the first days. The government was confident in being able to end the rebelion as they did four years before; the fascists thought they would have quickly brought down the government with a successful coup. But after the first week, the country was divided in two halves, and both sides of the conflict were keen on fighting, due to the big social fracture during the last months of the government. The Republic asked for help to the League of Nations, but the United Kingdom and France decided not to intervene, scared about the conflict trespassing the Spanish borders and becoming another bloody war in European fields. The only official external aid came from the USSR and the International Brigades, formed by volunteers of more than 50 nations. Some well-known personalities like Ernest Hemingway or George Orwell came to aid as well. Orwell wrote about his life-changing experience in a Barcelona immerse in the social revolution, where he joined the anti-stalinist socialist party POUM. Badly wounded in the neck, he left the front and went back to Barcelona, and witnessed the assault of the Telef nica by the communists against the anarchists, during the fets de maig, after which the POUM was declared illegal. Orwell captured his experiences in the frontline and Barcelona in his book Homage to Catalonia.


To defend Madrid is to defend Catalunya Franco and Mola, aided by the Northafrican troops and soon after by their fascist allies of Italy and Germany, didn't have big problems seizing the Basque Country and Asturies in the North, but not before making the town of Gernika disappear below the nazi Condor Legion bombing, in what would be a test for the massive aereal bombings of Word War II. The battle for Madrid was fierce, and the city wasn't occupied until the very last days of the war. The long defence of the capital made was immortalised in the famous phrase No pasar n.


Madrid's No pasar n During the war, anarchist groups controlling fields and towns in Arag n and Catalunya managed to bring a Social Revolution to the area. The people collectivised the land and industries, administered by local assemblies. This experience is regarded as the time when Libertarian Communism has been successful. The experience ended dramatically when the anarchists were treasoned and blocked by the communists, and eventually defeated by Franco at the front.


Each bullet for a target After the first and a half years, the fronts had stabilised in Madrid and the East of Andalusia, until early in 1939, Barcelona fell in Franco's hands, after the bloody Batalla de l'Ebre, being followed by Girona and the rest of Catalunya. The fate of the democratic state was clear. Two months later, Madrid fell and Val ncia and the small area still under Republican control surrended.


Robert Capa's Muerte de un miliciano Franco declared a totalitarian regime to, according to the fascists, reconcile the two Spains. In the first five years of the regime, up to 100.000 were executed in the franquist repression, including the Catalan president, Llu s Companys and many intellectuals. The dictatorship continued executing people until two months before Franco's death, 36 years later. Hundreds of thousands also had to exile in M xico, France and other destinations, leaving Spain in ashes, road gutters full of scattered bodies and the population facing famine. The cultural elite left the country, making the state's clock go back at least a decade. Their reconciliation really meant humiliation and repression. Many people were also murdered during the war behind republican and rebel lines, but the genocide of the aftermath had was unmatchable.


The Republic promoted education among the troups It is a bit late to restore the dignity of the victims, but not too late. Some associations have been fighting since the end of Franco's regime to get some kind of public recognition for their suffering, for their priceless fight for the legal and democratically elected Republic, for the people who didn't make it.


Excellent anarchist poster rejecting the politisation
of children by any of the three loyal groups Other people reject these efforts banning them of "revisionism". Not too surprisingly, these are the sons or grandsons of the people who won the war. The current government is about to pass a Historic Memory law which aims to give recognition to all these people who gave their lives for an ideal and freedom. Unfortunately, due to pressures from the right, it is probable that the text will be ammended to make it more acceptable by them. I fear it will end up not pleasing the victims at all. I have a passion for Spanish Civil War related matters, and in the last years I tried to use any opportunity to listen to my Catalan grandmother talk about war affairs in Barcelona, and my grandfather, from a town in the middle of the Republican side of the Teruel front, but born in a very catholic family, about how the war went on in Vall and the nearby mountains.


From the defensive No pasar n to the encouraging Pasaremos! I know my grandfather only told me the less compromising stories, like how he was sent to dig refuges until he was 16 and apt to fight in the Front, or how he was put in charge of hiding the church's valuable relics in a few barns. He never told me about how life was after the war in the small town. It is dramatic to see that all the people who were old enough to have a sense of what was going on at the time are so old that these unvaluable memories and stories will be mostly gone in ten years. (Most of the pictures are taken from the Sociedad Ben fica de Historiadores Aficionados y Creadores' website, which has a huge documented catalog)

Erich Schubert: Use CSS folds

/me too hates folded blogs. It's like "I'd like to know that I've blogged, but I don't want you to actually read it"... *rant* Now for some constructive things: Add "max-height: 30em; overflow: auto" to the CSS file, and overly long entries (such as Enricos recent posts) will not take over planet as much. (Pick an appropriate 30em). Untested for vertical limits; I've however been using this for clipping overly wide <pre/> sections before. With next to no drawbacks to the readers that is. Folds suck. But if you are actually using a web browser to read planet, you'll be able to use the scroll bars I guess. Otherwise contribute a Javascript to un-limit the blog postings size. Should be a one-liner. [Updated: overflow auto is better, since otherwise you'd always have scroll bars. Also increased to 30em. Thanks Daniel Silverstone for testing. See his blog entry on how to do this client-side with mozilla based browsers]

Ross Burton: EDS and Berkeley DB

Back in the good old days of Evolution 0.9, Evolution depended on Berkeley DB being installed. At the time Berkerley DB ("BDB") was undergoing a some major changes so it wasn't uncommon to upgrade your distribution and find that the new BDB can't read your addressbook. This was solved by Evolution embedding a copy of BDB into the source code. and statically linking it into the relevant addressbook backend. Problem solved. Over time Evolution was refactored, split into Evolution and Evolution Data Server, and so on. Then BDB was statically linked into not only the addressbook backend but also libedataserver, which contains some generic caches based on BDB. Now BDB isn't exactly a small library (it adds ~600K to a .so), so the second static linking started to irritate people. Not only does EDS have a private copy of a library, meaning extra work when BDB has security bugs, but there is 1.2M of code that could be shared. This led to various distributions patching EDS to use a system copy of BDB, a patch that has finally made it upstream, but not everyone is using it: by default EDS uses it's private copy. Now fast-forward to today. I did a full build of EDS from CVS using the system BDB (it's also a lot faster to build that way) and hit build failures from code demanding the static BDB to be built. Fixing these was trivial, but when I checked in the patch it worked fine... I investigated and discovered that BDB is now used in four places inside EDS: Now, BDB is approximately 600K when statically linked, and a typical EDS build statically links it four times. The server process will load both backends and both libraries, so it contains all four copies of BDB at once. This is getting silly. Please, all packagers, if you support multiple versions of BDB in your distribution, use --with-libdb=/usr to dynamically link to BDB. As a fallback I'm about to create a patch that installed the private BDB into a private library directory, to avoid all of this static linking. NP: Neighbour Radio, Last.fm (via Last Exit)

Martin-&#201;ric Racine: USB mass storage support in Linux sucks!

After an extremely frustrating afternoon of seeing the Linux UHCI driver repeatedly shut itself down, following several successive I/O errors while accessing USB mass storage (a 512Mb Compact Flash connected via a USB card reader, in one case, and a 512Mb USB stick, in the other), I spent an even more frustrating evening surfing the Internet for a solution. Conclusion: USB mass storage support in Linux sucks! Several mailing lists and user forums report similar failures using USB 2.0 devices on Linux UHCI or EHCI drivers, down to the exact same error messages that I get. Some even report the whole USB subsystem going down, leaving the user without any keyboard or mouse on USB-only systems (this matches my findings on one particular hardware platform that happens to be at the core of a project I'm working on). The suspected cause, in all cases, apparently boils down to either or all these:
  • The SCSI subsystem upon which USB (and Firewire) mass storage is built is utter crap that would need a complete rewrite. The issue has been known for ages, it's even a proverbial elephant in the corner that nobody wants to see, but rewriting the subsystem for a standard that barely has any real-life application left means opening a big can of worms.
  • None of the Linux kernel's HCI are capable of anticipating the quirks present in some implementations of the USB hardware specifications, so they simply panic and commit suicide in syslog-flooding glory.
  • There is apparently a hard-coded limit of 1000 units (blocks?) that, when exceeded, will make the mass storage driver reset itself! Doesn't sound like Willliam Gates the Third's often-mentioned "Nobody needs more than 640k" now, does it?
I can feel for the average user who is just trying to extract a few pictures off a USB stick and who simply ends up reinstalling $commercial_os in frustration. Wasting a couple of days trying to debug support for what is supposedly a well-documented industry standard should never happen. There is no excuse for a broken USB mass storage on Linux. Let's fix it!

Martin F. Krafft: XFS -barriers +write-cache considered harmful

If you have an XFS filesystem on hardware without barriers (or with broken barriers as was the case on my X40's Hitachi drive) and with write-caching enabled, disable the latter NOW (the following will work for PATA drives, and SATA if you have a [rd]ecent kernel):
apt-get install hdparm
df -T   egrep ' xfs '   sed -rne 's,(/dev/[sh]d[[:alpha:]]).*,\1,p' \
    sort -u   while read disk; do cat <<_eof
$disk  
  write_cache = off
 
_eof
done >> /etc/hdparm.conf
/etc/init.d/hdparm start
Also see http://oss.sgi.com/projects/xfs/faq.html#wcache . Anyway, damage mitigated. xfs_repair -d /dev/md1 feels funny when /dev/md1 is the root partition of your laptop and you're about to move to Ireland for six weeks, fully dependent on that machine. But yeah, XFS on a laptop is asking for trouble anyway. Call me masochist.

Enrico Zini: Fifth day in Addis

Setting up a mail server
Background Some terminology:
MTA
Mail Transport Agent
MUA
Mail User Agent
MDA
Mail Delivery Agent
SMTP
Simple Mail Transfer Protocol
MX
Mail eXchange
POP
Post Office Protocol
IMAP
Internet Message Access Protocol
With SMTP you connect to a server and send two things: envelope and message. The envelope looks like this:
MAIL FROM: <enrico@enricozini.org>
RCPT TO: <rms@fsf.org>
RCPT TO: <linus@linux.org>
The message looks like this:
From: <enrico@enricozini.org>
To: <rms@fsf.org>
Cc: <linus@linux.org>
Message-ID: <1234567@enricozini.org>
Subject: Test mail
Hi Richard,
this is a test mail.  I'm also writing
Linus to show how to send to more people.
Cheers,
Enrico
There is no authentication. There is no encryption. Two usual types of access control:
  1. Outbound e-mail is normally only accepted from an internal network
  2. Inbound e-mail is normally accepted from anywhere
The DNS is used to find the SMTP server to use to send a message:
$ host -t MX yahoo.com
yahoo.com MX 10 smtp1.yahoo.com
yahoo.com MX 20 smtp2.yahoo.com
yahoo.com MX 20 smtp3.yahoo.com
The process of sending an E-Mail:
  1. Enrico writes an E-Mail:

    From: Enrico Zini <enrico@enricozini.org>
    To: Richard Stallman <rms@fsf.org>
    Subject: Hello from Addis
    Hi Richard,
    Addis is a wonderful city, even if
    it rains a lot.
    Bye,  Enrico
    
  2. Enrico's MUA connects to the SMTP server (for example, port 25 of smtp.aau.edu.et):

    HELO enricozini.org
    200 OK Hello enricozini.org
    MAIL FROM: <enrico@enricozini.org>
    200 OK Mail from enrico@enricozini.org
    RCPT TO: <rms@fsf.org>
    
    Here, the SMTP server performs relay control: "do we relay mail to rms@fsf.org?":
    • Outbound e-mail is normally only accepted from an internal network
    • Inbound e-mail is normally accepted from anywhere
    A target address could be refused:
    413 ERR I don't relay for rms@fsf.org
    
    In this case, the destination is not local but the recipient is accepted because I'm inside the local network:
    200 OK Destination rms@fsf.org
    DATA
    200 OK Please send message body
    From: Enrico Zini <enrico@enricozini.org>
    To: Richard Stallman <rms@fsf.org>
    Subject: Hello from Addis
    Date: Mon, 17 Jul 2006 09:49:45 +0300
    Message-ID: <124372643@enricozini.org>
    Hi Richard,
    Addis is a wonderful city, even if
    it rains a lot.
    Bye,  Enrico
    .
    200 OK Message accepted
    QUIT
    200 OK Bye.
    
  3. The SMTP server needs to find out where to send the message, using the DNS:

    $ host -t MX fsf.org
    fsf.org MX 10 mail.fsf.org
    fsf.org MX 20 mail.gnu.org
    
  4. So the SMTP server tries the first one and connects to port 25 of mail.fsf.org:

    HELO smtp.aau.edu.et
    200 OK Hello smtp.aau.edu.et
    MAIL FROM: <enrico@enricozini.org>
    200 OK Mail from enrico@enricozini.org
    RCPT TO: <rms@fsf.org>
    
    The destination is accepted because it's for a local user:
    200 OK Destination rms@fsf.org
    DATA
    200 OK Please send message body
    From: Enrico Zini <enrico@enricozini.org>
    To: Richard Stallman <rms@fsf.org>
    Subject: Hello from Addis
    Date: Mon, 17 Jul 2006 09:49:45 +0300
    Message-ID: <124372643@enricozini.org>
    Received: by mail.aau.edu.et
      on Mon, 17 Jul 2006 09:55:53 +0300
      from 10.4.15.158
    Hi Richard,
    Addis is a wonderful city, even if
    it rains a lot.
    Bye,  Enrico
    .
    200 OK Message accepted
    QUIT
    200 OK Bye.
    
  5. Now, mail.fsf.org will invoke a MDA to write the mail in Richard Stallman's mailbox.

Example of problems with mail handling:
  • Accepting inbound connections:
    • Malicious input:
      • logic errors
      • buffer overflows
      • DoS (Denial Of Service) attacks
      • Connection floods
  • Performing outbound connections:
    • Programming errors:
      • Flooding of connections
  • Performing routing:
    • Unauthorised relays
    • Mail loops
  • Writing to the local hard drive:
    • Filling up the hard drive
    • Writing to the wrong files
  • Writing to the local hard drive as root:
    • In case of error or attack, any file in the system can potentially be compromised
RFC-822 is the original standard for E-mail. RFCs are standard Internet documents. Have a look at RFC documents released the 1st of April.
postfix Common setup: "Internet site with smarthost". More difficult to maintain: "Internet site". A smarthost is a machine that will relay e-mail for you. Questions asked with "Internet site with smarthost":
  • Mail name: aau.edu.et (name used to publicly identify the mail server)
  • Smarthost name: smtp.telecom.net.et (SMTP server that will relay our e-mail)
To test a mail server:
$ telnet localhost 25
HELO me
MAIL FROM: <a@b.c>
RCPT TO: <mail@of.a.local.user>
DATA
hi
.
QUIT
By default, you find locally delivered mail in /var/mail/username. Postfix configuration files:
/etc/postfix/master.cf
configures how all the postfix components run together (man 5 master)
/etc/postfix/main.cf
Main postfix configuration (man 5 postconf)
To rewrite addresses:
  1. In /etc/postfix/main.cf:

    canonical_maps = hash:/etc/postfix/canonical
    
  2. Then in /etc/postfix/canonical you can add the rewrite rules, like:

    enrico   enrico@enricozini.org
    
  3. When /etc/postfix/canonical is modified you need to regenerate the index:

    sudo postmap canonical
    
    (same is when you change the alias file: sudo postalias /etc/aliases)
(see file:///usr/share/doc/postfix/html/ADDRESS_REWRITING_README.html) Manipulating the message queue:
mailq - List the mail queue.
Example:
mailq
postqueue - Postfix queue control
Examples:
# Like mailq
postqueue -p
# Tries to send every message in the queue
postqueue -f
# Tries to send every message in the queue for that site
postqueue -s site
postsuper - Postfix superintendent
Examples:
# Deletes one message
sudo postsuper -d 7C4D2EC0F5D
# Deletes all messages held in the queue for later delivery
sudo postsuper -d ALL deferred
Different mail queues in postfix:
incoming
mail who just entered the system
active
mail to be delivered
deferred
mail to be delivered later because there were problems
hold
mail that should not be delivered until released from hold
Mail logs are in:
/var/log/mail.log
/var/log/mail.err
/var/log/mail.info
/var/log/mail.warn
Mail delivery Mailbox formats:
  • mbox: single file, mail separated by "From " lines
  • maildir: one directory per folder, one file per mail
  • mh: similar to maildir, but not really used
Alternate MDA: procmail: allows to filter mail automatically into different folders. Mail forwarding with ~/.forward: allows to redirect mail to a different address: just put the address you want to send to in the file ~/.forward.
POP or IMAP server Installation:
apt-get install dovecot
Configuration is in:
/etc/dovecot/dovecot.conf
The main thing that is needed is to enable the mail protocols you want:
protocols = imaps
Server monitoring To make all sorts of graphs:
apt-get install munin munin-node
Example: http://munin.ping.uio.no To compute more statistics:
  • anteater
  • isoqlog
  • mailgraph
Monitor system logs: logcheck:
  • sends you mail with abnormal log lines
  • It's important to customize what is normal and you do it with regular expressions
Filtering viruses and spam clamav - Virus scanner Virus scanning:
  • Postfix gives the mail to clamav that scans it and gives it back if it's clean.
  • Strategies for infected mail:
    • silently delete it
    • refuse the mail and send a notification to the sender
    • refuse the mail and send a notification to the receiver
    • quarantine the e-mail
    • refuse delivery with a SMTP error
    • deliver with an extra header that says that it's a virus
spamassassin - Spam filter Spam scanning:
  • Postfix gives the mail to spamd that scans it and gives it back with some spam information.
  • Strategies for spam mail:
    • silently delete it
    • refuse the mail and send a notification to the sender
    • refuse the mail and send a notification to the receiver
    • quarantine the e-mail
    • refuse delivery with a SMTP error
    • deliver with an extra header that says that it's spam
  • New techniques:
    • greylisting: when you receive a mail from a host you've never seen before, refuse it with a temporary error, and accept it the second time (after some time delay). Spammers normally don't retry, and implementing retry would increase their cost of sending e-mail.
    • crossassassin: if more than some amount of your users receive a mail with the same message ID, throw it away. Sending mails with different headers would increase the cost of sending e-mail.
Man pages and sections Man pages are divided in sections:
  • man man shows all the sections of the manpages
  • man 5 postconf shows the postconf manpage in the "configuration file" section
  • Normally manpages are referred as manpage(section) (e.g. postconf(5) )

Enrico Zini: Fifth day in Addis

Samba To get samba:
apt-get install samba samba-doc smbclient
To get the Samba Web Administration Tool:
apt-get install swat netkit-inetd
The configuration is in /etc/samba:
  • One [global] section with the general settings
  • One section per share
One could use swat at http://localhost:901/ but it does not work easily on Ubuntu. To see what is shared:
smbclient -L localhost
To access a share:
smbclient //localhost/name-of-the-share
To add a new user:
sudo smbpasswd -a username
To change the password of a user:
sudo smbpasswd username
To test accessing a share as a user:
smbclient //localhost/web -U yared
Documentation:
man smb.conf
To force the user or group used to access a share:
force user = enrico
force group = www-data
To set the unix permissions for every created file:
# For files
create mask = 0664
# For directories
directory mask = 0775
Example share configuration for a webspace:
mkdir /var/www/public
chgrp www-data /var/www/public
chmod 0775 /var/www/public
Then, in /etc/samba/smb.conf:
[web]
   comment = Webspace
   path = /var/www
   writable = yes
   public = no
   force group = www-data
   create mask = 0664
   directory mask = 0775
Example share configuration for a read only directory where only a limited group of people can write:
[documents]
   comment = Documents
   path = /home/enrico/Desktop/documents
   force user = enrico
   public = yes
   writable = no
   write list = enrico, yared
Print server (CUPS) Installation:
apt-get install cupsys
Configuration:
  • On the web (not enabled in Ubuntu):

    http://localhost:631/
    
  • On the desktop:

    System/Administration/Printing
    
Example IPP URIs:
ipp://server[:port]/printers/queue
http://server:631/printers/queue
ipp://server[:port]/...
For example:
ipp://server/printers/laserjet
"This printer uri scheme can be used to contact local or remote print services to address a particular queue on the named host in the uri. The "ipp" uri scheme is specified in the Internet Print Protocol specifications and is actually much more free form that listed above. All Solaris and CUPS based print queues will be accessed using the formats listed above. Access to print queues on other IPP based print servers requires use of the server supported ipp uri format. Generally, it will be one of the formats listed above."
GRUB The configuration file is in /boot/grub/menu.lst. The documentation can be accessed as info grub after installing the package grub-doc. Quick list of keys for info:
arrows
move around
enter
enters a section
l
goes back
u
goes up one node
q
quit
/
search
Grub trick to have a memory checker:
  1. apt-get install memtest86+

  2. Add this to /boot/grub/menu.lst:

    title Memory test
        root (hd0,5)
        kernel /boot/memtest86+.bin
    
Firewall With iptables:
man iptables
# Only allow in input the network packets
# that are going to the web server
iptables -P INPUT DROP
iptables -A INPUT --protocol tcp --destination port 80 -j ACCEPT
# To reset the input chain as the default
iptables -F INPUT
iptables -P INPUT ACCEPT
Some links:
Squid Installation:
apt-get install squid
The configuration is in /etc/squid/squid.conf. To allow the local network to use the proxy:
# Add this before "http_access deny all"
acl our_networks src 10.4.15.0/24
http_access allow our_networks
To use a parent proxy:
cache_peer proxy.aau.edu.et     parent    8080  0  proxy-only no-query
Pay attention because /var/spool/squid will grow as the cache is used. The maximum cache size is set in the directive cache_dir. Information about squid access control is at http://www.squid-cache.org/Doc/FAQ/FAQ-10.html To check that the configuration has no syntactic errors: squid -k parse. To match urls:
acl forbiddensites url_regex [-i] regexp
For info about regular expressions:
man regex
Example filtering by regular expression:
acl skype url_regex -i [^A-Za-z]skype[^A-Za-z]
http_access deny skype
Transparent proxy setup: http://www.tldp.org/HOWTO/TransparentProxy.html
Problems found today Hiccups of the day:
  • swat does not run on Ubuntu because Ubuntu does not have inetd
  • swat does not allow root login on Ubuntu because root does not have a password
  • smbpasswd -a does not seem to update the timestamp of /var/lib/samba/passwd.tdb
  • cups web admin does not work on Ubuntu
  • LDAP is still not so intuitive to set up

Enrico Zini: Third day in Addis

Believe it or not, a network that fails often is the best thing to have when you are teaching network troubleshooting. Various tools useful for networking:
  • ifconfig - configure a network interface

  • dnsmasq - Simple DNS and DHCP server

  • host - DNS lookup utility

  • route - show / manipulate the IP routing table

  • arping - send ARP REQUEST to a neighbour host

  • mii-tool - view, manipulate media-independent interface status (IOW, see if the cable works)

  • nmap - Network exploration tool and security / port scanner Examples:

    # Look at what machines are active in the local network:
    nmap -sP 10.5.15.0/24
    # Look at what ports are open in a machine:
    nmap 10.5.15.26
    
  • tcpdump - dump traffic on a network It can be used to see if there is traffic, and to detect traffic that shouldn't be there.

Useful tip:
# Convert a unix timestamp to a readable date
date -d @1152841341
What happens when you browse a web page:
  1. type the address www.google.com in the browser
  2. the browser needs the IP address of the web server:
  1. look for the DNS address in /etc/resolv.conf (/etc/resolv.conf is created automatically by the DHCP client)
  2. try all the DNS servers in /etc/resolv.conf until one gives you the IP address of www.google.com
  3. take the first address that comes from the DNS (in our case was 64.233.167.104)
  1. figure out how to connect to 64.233.167.104:
  1. consult the routing table to see if it's in the local network:
  1. if it's in the local network, then look for the MAC address (using ARP - Address Resolution Protocol)
  2. if it'd not in the local network, then send through the gateway (again using ARP to find the MAC address of the gateway)
  1. Send out the HTTP request to the local web server or through the gateway, using the Ethernet physical protocol, and the MAC address to refer to the other machine.
Troubleshooting network problems:
  1. See if the network driver works:
  1. With ifconfig, see if you see the HWaddr:. If you do not see it, then the linux driver for the network card is not working. Unfortunately there's no exact way to say that it works perfectly
  1. See if you have an IP address with ifconfig. If you find out that you need to rerun DHCP (for example, if the network cable was disconnected when the system started), then you can do it either by deactivating/reactivating the Ethernet interface using System/Administration/Networking or, on a terminal, running:

    # ifdown eth0
    # ifup eth0
    
    If you don't get an IP, try to see if the DHCP server is reachable by running:
    $ arping -D [address of DHCP server]
    
  2. See if the local physical network works:

  1. With sudo mii-tool, see if the cable link is ok. If it's not, then it's a problem in the cable or the plugs, or simply the device at the other end of the cable is turned off.
  2. Try arping or ping -n on a machine in the local network (like the gateway) to see if the local network works.
  1. See if the DNS works:
  1. Find out the DNS address:

    cat /etc/resolv.conf
    
  2. If it's local, arping it

  3. If it's not local, ping -n it

  4. Try to resolve a famous name using that DNS:

    $ host [name] [IP address of the DNS]
    
  5. Try to resolve the name of the machine you're trying to connect. If you can resolve a famous name but not the name you need, then it's likely a problem with their DNS.

  1. If you use a proxy, see if the proxy is reachable: check if the proxy name resolves to an IP, if you can ping it, if you can telnet to the proxy address and port:

    $ telnet [proxy address] [proxy port]
    
    you quit telnet with ^]quit.
  2. If you can connect directly to the web server, try to see if it answers:

    $ telnet [address] 80
    
    If you are connected, you can confirm that it's a web server:
    GET / HTTP/1.0 (then Enter twice)
    
    If it's a web server, it should give you something like a webpage or an HTTP redirect.
When you try to setup a service and it doesn't work:
  1. check that it's running:

    $ ps aux   grep dnsmasq
    
  2. check that it's listening on the right port:

    $ sudo netstat -lp
    
  3. check that it's listening from the outside:

    $ nmap [hostname]
    
  4. check for messages in /var/log/daemon.log or /var/log/syslog

  5. check that the configuration is correct and reload or restart the server to make sure it's running with the right configuration:

    # /etc/init.d/dnsmasq restart
    
dnsmasq:
By default: works as a DNS server that serves the data in /etc/hosts. By default: uses /etc/resolv.conf to find addresses of other DNS to use when a name is not found in /etc/hosts. To enable the DHCP server, uncomment:
dhcp-range=192.168.0.50,192.168.0.150,12h
in /etc/dnsmasq.conf and set it to the range of addresses you want to serve. Pay attention to never put two DHCP servers on the same local network, or they will interfere with each others. To test if the DHCP server is working, use dhcping (not installed by default on Ubuntu). To communicate other information like DNS, gateway and netmask to the clients, use this piece of dnsmasq.conf:
# For reference, the common options are:
# subnet mask - 1
# default router - 3
# DNS server - 6
# broadcast address - 28
dhcp-option=1,255.255.255.0
dhcp-option=3,192.168.0.1
dhcp-option=6,192.168.0.1
dhcp-option=28,192.168.0.255
Problems found today:
  • changing the name of the local machine in /etc/hosts breaks sudo, and without sudo it's impossible to edit the file. The only way to fix this is a reboot in recovery mode.
  • dhclient -n -w is different than dhclient -nw
Quick start examples with tar:
# Create an archive
tar zcvf nmap.tar.gz *.deb
# Extract an archive
tar zxvf nmap.tar.gz
# Look at the contents of an archive
tar ztvf nmap.tar.gz
Quick & dirty way to send a file between two computers without web server, e-mail, shared disk space or any other infrastructure:
# To send
nc -l -p 12345 -q 1 < nmap.tar.gz
# To receive
nc 10.5.15.123 12345 > nmap.tar.gz
# To repeat the send command 20 times
for i in  seq 1 20 ; do nc -l -p 12345 -q 1 < nmap.tar.gz ; done

Erich Schubert: Flow oriented programming?

I've been preparing on my exam on programming and modeling languages. The usual classification of imperative, functional, oop and logic programming is kind of inconsequent, since there are e.g. functional oop languages. To me, oop is more of a software engineering thing, usually applied to imperative programming with a load of syntactic sugar. The usual OOP examples suck. They're so basic that all the OOP and modeling stuff is just a complete waste of time. So I was considering more complex examples, and ended up at common design patterns. A couple of design patterns in OOP seem like workarounds to me; a new language (well, maybe C# has addressed some of these, I don't know) could probably add some more syntactic sugar to ease their use. I hate UML diagrams. To me they just emphasize all the obvious stuff, while they quickly get too complex to be of use for "seeing" the important things. Even UML activity diagrams in my opinion don't properly depict the flow of information/data in the program, which is what I really care about. Activity diagrams basically show you a sequence of methods are being called. But often you'll have a method being called thousands of times, with the actual relevant information being in the parameters passed. And some stuff might be done asynchronously, too. For certain problems (well, not for stuff like e.g. the train door used in OOP examples), it could be much more useful to abstract away from the involved classes - which are often just wrappers for data, "records", "datagrams", tcp packets - and instead model flows of data. Think of drawing conveyor belts transporting data, filtering it, duplicating it, ... The gstreamer framework seems to employ such a model for audio and video processing. You build pipelines by placing e.g. sources and sinks there. But you can interpret dozens of code examples with this model. SAX transform a stream of charaters into a stream of XML nodes. UI mainloops are basically a stream of UI events. Network data obviously is easy to model as a data flow; the "obvious" mapping into "packet objects" however often useless. Even when doing a select on a database, the result has an obvious representation as a data flow (a flow of records in this case). You probably only need a couple of "primitives". Sources and sinks, obviously. Filters, Y and Tee modules. Queues and Caches could be interesting, too. Note that data flows can work in two ways: push and pull. Maybe sometimes the same code could be used for pull and push operations, too. When push and pull driven flows meet you often need threads to connect. For example, a SAX parser usually will pull character data from a stream and push a stream of XML nodes as output. One way to get a pull-able output stream out of this (I'm ignoring e.g. the existing XMLpull API for this example) is to use a cache (which will eventuall contain the whole document as Nodes), another is to use two threads and just block the SAX thread until the other pulling thread has requested another node. Of course you can do all this in existing languages (you know, this doesn't give you anything new beyond turing completeness) - just like you could do OOP in assembler. You can, but it would be nice if the compilers offer you an easier syntax. Also it would be nice if I wouldn't have to write different code for pull-driven and push-driven operation. Also note that push type operation is rather typical for imperative programming languages - for i in range(0,10): print i - whereas pull type operations are typical for "lazy" functional languages - take 3 (repeat 'a').

17 July 2006

David Welton: Oregon - OSCON

After more than three years away from home, we're going on vacation to Oregon this summer. Luckily, that will coincide with OSCON in Portland, so hopefully it will be possible to meet some people there that I ordinarly wouldn't get to see. Write me email if you'd like to get together for a beer or something! The rest of the time, we'll be in Eugene, or travelling around the state.

Ross Burton: Evolution Broken

That was a nice inflammatory title to get everyone's attention. I've just committed a change to Evolution Data Server (#313533, extending EContactPhoto) that not only breaks ABI but also API. Rock on! Now I've yet to update Evolution for the new API, so people doing builds from CVS will find they break. I'll try and get it fixed today, but I can't promise it. The fixes will be fairly simple if you ignore the new features, so someone else should be able to easily do it. NP: Cool Groove

Matt Brown: Patching the sc520 watchdog to reboot Soekris devices

We use a lot of Soekris devices at work which often get deployed to remote locations where its not easy to access them if things go wrong. I’ve recently been writing some software to act as a watchdog on the machine. As a very last resort this software makes use of the watchdog chipset provided by the Soekris boards so that if userspace goes away the machine will get rebooted back into what is hopefully a sane state. Unfortunately the in kernel sc520 watchdog device doesn’t actually reboot the Soekris when the heartbeat ping from userspace is lost. This makes it virtually useless. There has been a patch floating around since 2004 that fixes this problem, but it hasn’t made it in to the kernel for one reason or another. I think it’s Soekris specific… I’ve updated the patch to apply cleanly against the sc520 driver found in 2.6.16 and you can find it below. http://www.mattb.net.nz/patches/linux/soekris-sc520wd.patch

16 July 2006

Erich Schubert: On programming contests (ACM ICPC etc.)

Bart Massey and Mike Vanier talk about programming contests such as the ACM International Collegiate Programming Competition. Well, since I'm being told that I love ranting, I have to join this one. ;-) I never participated in a contest except the Bundeswettbewerb Informatik back in (pre-university) school. In my opinion that contest was quite different for a number of reasons: most of the participants didn't have any IT classes at school. And noone "trained" for that contest. Also the contest isn't a speed contest, but you have a couple of months to solve a number of problems that most of us would find rather easy - for pupils aged 15-18 with no real CS classes they aren't that easy. Even in the later rounds, time isn't the main constriction - you have several weeks - and the programs are also judged by their code quality, documentation and design. I think you were allowed to participate in teams during the first and maybe second round. A friend of mine pointed out the ACM chellege to me; but we never really managed to form a team since everybody was too busy. We never actually did a training session (though I myself solved a couple of the valladoloid online judge problems), and we didn't submit our participitation. Though we would have automatically won our local contest, since I'm not aware of any teams by our universities here (despite them being top-ranked universities). In general, Germany isn't very much involved in this contest, despite being an "international" contest. This has various reasons. On one hand, CS teaching in germany is not much about just writing code. You'll of course learn the basics, but even when they've finished their degree, most will be really slow in writing code. Instead, the focus is on modeling and the theoretical background. In my opinion, that is very good. You'll always be able to improve your "basic" code writing skills later; that is mostly experience and routine. But you won't be learning much theoretical background in your job later on. Having worked on a number of the old, archived, training problem sets, I'm not convinced that this contest makes much sense. To succeed with the problem sets you need to be able to I agree with the others that I don't think team-play is of much value in this contest; nor will you be doing much creative work. In my opinion it's all about reducing the given problem to one of the known problems (e.g. longest common subsequence, shortest path, maximum flow), then adopting an algorithm to it. I'd call that reproductive work. In my opinion, they're rather boring. Basically you're expected to solve them using a certain algorithm. You aren't judged by good or smart solutions either. Just by speed. The (high-school) programming contest mentioned earlier was more interesting. For example there was the task of navigating a robot on a factory floor without any state information, and only a limited view. The robot was not allowed to keep any information. Such as "I need to go out of this dead end" or "that is a dead end". It would easily walk into a path just to notice after a few steps (it had a limited view!) that this is a dead end, then try to walk out - just to walk in again, having forgotten that it's a dead end. You of course can't judge solutions without inspecting the code. Smart programmers found a few ways to "cheat" here - and got extra credit here. For example one robot, when noticing that it's in a dead end it might forget, would step up to the wall. And whenever it was close to a wall, it would follow it, and thus walk out of the dead end. Some heuristics making it step away from the wall again made it much more successful than others. Another "cheat" involved calculating "remaining steps allowed - distance to destination mod 2". It then could be programmed with two alternating behaviours and switch between them by just not moving for one turn. I don't think there is any room for such great ideas in the ACM contest.

Next.

Previous.