Search Results: "spa"

19 July 2006

Erich Schubert: nofollow followup.

Micah Dubinko blogs on rel="nofollow" being a failure. I have to agree that this doesn't really stop comment spammers currently. I still believe that it will help on the long run, since most blog software will now use it by default. And people have to upgrade from time to time due to the lastest PHP security issue... However, I would like Google or other search engines to offer some "spam submission". It would be best if they had a common submission server. Then we could use the moderation tools of our blogs to submit these bad URLs to Google, who in turn could take measures to strip them off their index (e.g. by moderating indexing for these URLs, reducing their pagerank, freezing their ranking for some weeks etc.) I'm of course aware of abuse possibilities (e.g. spam with your competitors URL); but most comment spam I've seen so far uses new sites anyway. I think you can detect these by charcteristical URLs, site contents, incoming links etc. As for myself, I've never enabled comments on my blog for the very reason that I'm not willing to moderate all that spam. In my guestbook (which is mostly to reduce the number of emails I get because of certain popular contents of my german web page; people will happily post in my guestbook instead of sending me an email now), I've recently enabled a very simple filter. It will just reject any entry which contains http:// anywhere. My users don't need that, and it has reduced spam to 0 again. And I don't care for the entries there anyway. ;-)

18 July 2006

Jordi Mallach: Seventy years of the Spanish Civil War and the Spanish Revolution

A few months ago I wrote about the 75th anniversary of the proclamation of the Second Spanish Republic which was celebrated throughout the state. That weekend I visited a fantastic exposition of Republican and Revolutionary war propaganda posters, in Segorbe.

Today, Spain remembers the beginning of the end. During the evening of the 17th of July and morning of the 18th, a group of generals of the Spanish regular army raised against the democratically elected left-wing government of the Republic. History says that the murdering of Calvo-Sotelo, a right-wing leader, was what finally provoked the conflict. It was no secret that the same right-wing and some elements in the army had been conspiring for weeks about an uprise. The only question was when, but the government was fooled by general Mola and didn't take really effective measures to prevent it.


Warning from sanitary authorities against venereal diseases Mola and Sanjurjo planned to start taking over official buildings and army quarters and quickly seize control of the major cities. During the first hours of their rebelion, they managed to take over most of the southern regions in Andalusia, Extremadura and parts of the Northwest, plus most of the islands and the Moroccan protectorate area. Meanwhile, the Republic did not manage to react quickly due to the weakness of the government, and in the first days of the crisis, the government changed a few times. Finally, the Republican parties decided that to defend the state they should give arms to the unions, and soon after many militia groups were getting organised in the cities, and columns sent off to liberate other towns.


Solidarity of Catalunya to the Basque front The insurrection was suffocated in Barcelona after some heavy fighting against the military in the Montju c castle; the Anarchists of the CNT grew strong in Catalunya and Arag n, and offered good resistance in those fronts. After the first days of the coupe d'etat, it was clear neither the legitimate government or the rebels calculated how things would go after the first days. The government was confident in being able to end the rebelion as they did four years before; the fascists thought they would have quickly brought down the government with a successful coup. But after the first week, the country was divided in two halves, and both sides of the conflict were keen on fighting, due to the big social fracture during the last months of the government. The Republic asked for help to the League of Nations, but the United Kingdom and France decided not to intervene, scared about the conflict trespassing the Spanish borders and becoming another bloody war in European fields. The only official external aid came from the USSR and the International Brigades, formed by volunteers of more than 50 nations. Some well-known personalities like Ernest Hemingway or George Orwell came to aid as well. Orwell wrote about his life-changing experience in a Barcelona immerse in the social revolution, where he joined the anti-stalinist socialist party POUM. Badly wounded in the neck, he left the front and went back to Barcelona, and witnessed the assault of the Telef nica by the communists against the anarchists, during the fets de maig, after which the POUM was declared illegal. Orwell captured his experiences in the frontline and Barcelona in his book Homage to Catalonia.


To defend Madrid is to defend Catalunya Franco and Mola, aided by the Northafrican troops and soon after by their fascist allies of Italy and Germany, didn't have big problems seizing the Basque Country and Asturies in the North, but not before making the town of Gernika disappear below the nazi Condor Legion bombing, in what would be a test for the massive aereal bombings of Word War II. The battle for Madrid was fierce, and the city wasn't occupied until the very last days of the war. The long defence of the capital made was immortalised in the famous phrase No pasar n.


Madrid's No pasar n During the war, anarchist groups controlling fields and towns in Arag n and Catalunya managed to bring a Social Revolution to the area. The people collectivised the land and industries, administered by local assemblies. This experience is regarded as the time when Libertarian Communism has been successful. The experience ended dramatically when the anarchists were treasoned and blocked by the communists, and eventually defeated by Franco at the front.


Each bullet for a target After the first and a half years, the fronts had stabilised in Madrid and the East of Andalusia, until early in 1939, Barcelona fell in Franco's hands, after the bloody Batalla de l'Ebre, being followed by Girona and the rest of Catalunya. The fate of the democratic state was clear. Two months later, Madrid fell and Val ncia and the small area still under Republican control surrended.


Robert Capa's Muerte de un miliciano Franco declared a totalitarian regime to, according to the fascists, reconcile the two Spains. In the first five years of the regime, up to 100.000 were executed in the franquist repression, including the Catalan president, Llu s Companys and many intellectuals. The dictatorship continued executing people until two months before Franco's death, 36 years later. Hundreds of thousands also had to exile in M xico, France and other destinations, leaving Spain in ashes, road gutters full of scattered bodies and the population facing famine. The cultural elite left the country, making the state's clock go back at least a decade. Their reconciliation really meant humiliation and repression. Many people were also murdered during the war behind republican and rebel lines, but the genocide of the aftermath had was unmatchable.


The Republic promoted education among the troups It is a bit late to restore the dignity of the victims, but not too late. Some associations have been fighting since the end of Franco's regime to get some kind of public recognition for their suffering, for their priceless fight for the legal and democratically elected Republic, for the people who didn't make it.


Excellent anarchist poster rejecting the politisation
of children by any of the three loyal groups Other people reject these efforts banning them of "revisionism". Not too surprisingly, these are the sons or grandsons of the people who won the war. The current government is about to pass a Historic Memory law which aims to give recognition to all these people who gave their lives for an ideal and freedom. Unfortunately, due to pressures from the right, it is probable that the text will be ammended to make it more acceptable by them. I fear it will end up not pleasing the victims at all. I have a passion for Spanish Civil War related matters, and in the last years I tried to use any opportunity to listen to my Catalan grandmother talk about war affairs in Barcelona, and my grandfather, from a town in the middle of the Republican side of the Teruel front, but born in a very catholic family, about how the war went on in Vall and the nearby mountains.


From the defensive No pasar n to the encouraging Pasaremos! I know my grandfather only told me the less compromising stories, like how he was sent to dig refuges until he was 16 and apt to fight in the Front, or how he was put in charge of hiding the church's valuable relics in a few barns. He never told me about how life was after the war in the small town. It is dramatic to see that all the people who were old enough to have a sense of what was going on at the time are so old that these unvaluable memories and stories will be mostly gone in ten years. (Most of the pictures are taken from the Sociedad Ben fica de Historiadores Aficionados y Creadores' website, which has a huge documented catalog)

Steve Kemp: Dry your tears and give me a smile

Spam Is Killing Me Guess how much spam arrived today for security@debian.org? Goan. Guess. So far over 4000. ffs.
grep To: Mail/backup/18-07-2006   grep security  grep debian  wc -l
4026
(OK a couple of false-positives there, but not a lot.) How do people cope with this volume of mail? And the day isn’t over ..

Ben Armstrong: XPilot change of upstream growing pains

XPilot development has changed hands over the past several years. While the original website at www.xpilot.org continues to be run by the original developers, XPilot-NG (xpilot.sf.net) is where all of the new development has been happening. But xpilot.org still performs an important function for the user community: they run the meta servers1 that list all public xpilot servers around the world. Recently, we (the XPilot-NG developers on #xpilot at irc.freenode.net) noticed quite a number of uncontactable servers popping up on the meta servers. This is annoying, to say the least. Users don’t know they can’t connect until they’ve tried (unless they’ve bothered to try pinging the server first) and besides which, having all those “junk” servers in the list makes it harder to locate contactable servers. Now, naturally, we wanted to know where all of those servers were coming from, and why now, at this particular time? I knew my Debian package (and presumably any derivatives) do not report to the metaserver by default. Any admin starting an XPilot server would have to explicitly configure it to do so, so the incidence of uncontactable Debian XPilot-NG servers should remain relatively low and have a constant growth rate. Then I found the Fedora package, and started poking around inside. After a shallow review of the package, I concluded there was a good chance it was the culprit2, especially since it recently (FC5) had an init script added. Such a bug, coupled with the relatively recent release of FC5, would account for the sudden spike in number of permanently running non-contactable XPilot-NG servers. Bingo! I found some helpful folks at #fedora-games at irc.freenode.net, including the XPilot NG maintainer, which led to filing this bug. I subsequently invited the maintainer over to #xpilot and had a nice time introducing him to the other upstream devs and chatting about our current issues. Ultimately, we know we need to fix the problem at the metas, but in the meantime, I am pleased to have established contact with my counterpart in the Fedora project, expanding the diminishingly small XPilot developer community by one.

1 They deliberately never released any source code for those metas to try to keep the user community unified at the centralized servers. While I see the reasoning behind the decision, given the problems we’re facing now, it’s clear the XPilot community would be better off with meta server source that is actively being maintained.

2 Admittedly two problems with the +reportMeta option upstream set us up for this problem. First, counterintuitively, +reportMeta disables reporting to the metaserver (following the rule ”+” negates the meaning of an option starting with ”-”). Second, -reportMeta is a stupid default. Most people starting servers are behind firewalls these days, so unless they consciously decide to make their server public, making the corresponding adjustments to their firewall, the option should not be set (not to mention the security implications of making a server public by default).

Martin F. Krafft: XFS -barriers +write-cache considered harmful

If you have an XFS filesystem on hardware without barriers (or with broken barriers as was the case on my X40's Hitachi drive) and with write-caching enabled, disable the latter NOW (the following will work for PATA drives, and SATA if you have a [rd]ecent kernel):
apt-get install hdparm
df -T   egrep ' xfs '   sed -rne 's,(/dev/[sh]d[[:alpha:]]).*,\1,p' \
    sort -u   while read disk; do cat <<_eof
$disk  
  write_cache = off
 
_eof
done >> /etc/hdparm.conf
/etc/init.d/hdparm start
Also see http://oss.sgi.com/projects/xfs/faq.html#wcache . Anyway, damage mitigated. xfs_repair -d /dev/md1 feels funny when /dev/md1 is the root partition of your laptop and you're about to move to Ireland for six weeks, fully dependent on that machine. But yeah, XFS on a laptop is asking for trouble anyway. Call me masochist.

Enrico Zini: Fifth day in Addis

Setting up a mail server
Background Some terminology:
MTA
Mail Transport Agent
MUA
Mail User Agent
MDA
Mail Delivery Agent
SMTP
Simple Mail Transfer Protocol
MX
Mail eXchange
POP
Post Office Protocol
IMAP
Internet Message Access Protocol
With SMTP you connect to a server and send two things: envelope and message. The envelope looks like this:
MAIL FROM: <enrico@enricozini.org>
RCPT TO: <rms@fsf.org>
RCPT TO: <linus@linux.org>
The message looks like this:
From: <enrico@enricozini.org>
To: <rms@fsf.org>
Cc: <linus@linux.org>
Message-ID: <1234567@enricozini.org>
Subject: Test mail
Hi Richard,
this is a test mail.  I'm also writing
Linus to show how to send to more people.
Cheers,
Enrico
There is no authentication. There is no encryption. Two usual types of access control:
  1. Outbound e-mail is normally only accepted from an internal network
  2. Inbound e-mail is normally accepted from anywhere
The DNS is used to find the SMTP server to use to send a message:
$ host -t MX yahoo.com
yahoo.com MX 10 smtp1.yahoo.com
yahoo.com MX 20 smtp2.yahoo.com
yahoo.com MX 20 smtp3.yahoo.com
The process of sending an E-Mail:
  1. Enrico writes an E-Mail:

    From: Enrico Zini <enrico@enricozini.org>
    To: Richard Stallman <rms@fsf.org>
    Subject: Hello from Addis
    Hi Richard,
    Addis is a wonderful city, even if
    it rains a lot.
    Bye,  Enrico
    
  2. Enrico's MUA connects to the SMTP server (for example, port 25 of smtp.aau.edu.et):

    HELO enricozini.org
    200 OK Hello enricozini.org
    MAIL FROM: <enrico@enricozini.org>
    200 OK Mail from enrico@enricozini.org
    RCPT TO: <rms@fsf.org>
    
    Here, the SMTP server performs relay control: "do we relay mail to rms@fsf.org?":
    • Outbound e-mail is normally only accepted from an internal network
    • Inbound e-mail is normally accepted from anywhere
    A target address could be refused:
    413 ERR I don't relay for rms@fsf.org
    
    In this case, the destination is not local but the recipient is accepted because I'm inside the local network:
    200 OK Destination rms@fsf.org
    DATA
    200 OK Please send message body
    From: Enrico Zini <enrico@enricozini.org>
    To: Richard Stallman <rms@fsf.org>
    Subject: Hello from Addis
    Date: Mon, 17 Jul 2006 09:49:45 +0300
    Message-ID: <124372643@enricozini.org>
    Hi Richard,
    Addis is a wonderful city, even if
    it rains a lot.
    Bye,  Enrico
    .
    200 OK Message accepted
    QUIT
    200 OK Bye.
    
  3. The SMTP server needs to find out where to send the message, using the DNS:

    $ host -t MX fsf.org
    fsf.org MX 10 mail.fsf.org
    fsf.org MX 20 mail.gnu.org
    
  4. So the SMTP server tries the first one and connects to port 25 of mail.fsf.org:

    HELO smtp.aau.edu.et
    200 OK Hello smtp.aau.edu.et
    MAIL FROM: <enrico@enricozini.org>
    200 OK Mail from enrico@enricozini.org
    RCPT TO: <rms@fsf.org>
    
    The destination is accepted because it's for a local user:
    200 OK Destination rms@fsf.org
    DATA
    200 OK Please send message body
    From: Enrico Zini <enrico@enricozini.org>
    To: Richard Stallman <rms@fsf.org>
    Subject: Hello from Addis
    Date: Mon, 17 Jul 2006 09:49:45 +0300
    Message-ID: <124372643@enricozini.org>
    Received: by mail.aau.edu.et
      on Mon, 17 Jul 2006 09:55:53 +0300
      from 10.4.15.158
    Hi Richard,
    Addis is a wonderful city, even if
    it rains a lot.
    Bye,  Enrico
    .
    200 OK Message accepted
    QUIT
    200 OK Bye.
    
  5. Now, mail.fsf.org will invoke a MDA to write the mail in Richard Stallman's mailbox.

Example of problems with mail handling:
  • Accepting inbound connections:
    • Malicious input:
      • logic errors
      • buffer overflows
      • DoS (Denial Of Service) attacks
      • Connection floods
  • Performing outbound connections:
    • Programming errors:
      • Flooding of connections
  • Performing routing:
    • Unauthorised relays
    • Mail loops
  • Writing to the local hard drive:
    • Filling up the hard drive
    • Writing to the wrong files
  • Writing to the local hard drive as root:
    • In case of error or attack, any file in the system can potentially be compromised
RFC-822 is the original standard for E-mail. RFCs are standard Internet documents. Have a look at RFC documents released the 1st of April.
postfix Common setup: "Internet site with smarthost". More difficult to maintain: "Internet site". A smarthost is a machine that will relay e-mail for you. Questions asked with "Internet site with smarthost":
  • Mail name: aau.edu.et (name used to publicly identify the mail server)
  • Smarthost name: smtp.telecom.net.et (SMTP server that will relay our e-mail)
To test a mail server:
$ telnet localhost 25
HELO me
MAIL FROM: <a@b.c>
RCPT TO: <mail@of.a.local.user>
DATA
hi
.
QUIT
By default, you find locally delivered mail in /var/mail/username. Postfix configuration files:
/etc/postfix/master.cf
configures how all the postfix components run together (man 5 master)
/etc/postfix/main.cf
Main postfix configuration (man 5 postconf)
To rewrite addresses:
  1. In /etc/postfix/main.cf:

    canonical_maps = hash:/etc/postfix/canonical
    
  2. Then in /etc/postfix/canonical you can add the rewrite rules, like:

    enrico   enrico@enricozini.org
    
  3. When /etc/postfix/canonical is modified you need to regenerate the index:

    sudo postmap canonical
    
    (same is when you change the alias file: sudo postalias /etc/aliases)
(see file:///usr/share/doc/postfix/html/ADDRESS_REWRITING_README.html) Manipulating the message queue:
mailq - List the mail queue.
Example:
mailq
postqueue - Postfix queue control
Examples:
# Like mailq
postqueue -p
# Tries to send every message in the queue
postqueue -f
# Tries to send every message in the queue for that site
postqueue -s site
postsuper - Postfix superintendent
Examples:
# Deletes one message
sudo postsuper -d 7C4D2EC0F5D
# Deletes all messages held in the queue for later delivery
sudo postsuper -d ALL deferred
Different mail queues in postfix:
incoming
mail who just entered the system
active
mail to be delivered
deferred
mail to be delivered later because there were problems
hold
mail that should not be delivered until released from hold
Mail logs are in:
/var/log/mail.log
/var/log/mail.err
/var/log/mail.info
/var/log/mail.warn
Mail delivery Mailbox formats:
  • mbox: single file, mail separated by "From " lines
  • maildir: one directory per folder, one file per mail
  • mh: similar to maildir, but not really used
Alternate MDA: procmail: allows to filter mail automatically into different folders. Mail forwarding with ~/.forward: allows to redirect mail to a different address: just put the address you want to send to in the file ~/.forward.
POP or IMAP server Installation:
apt-get install dovecot
Configuration is in:
/etc/dovecot/dovecot.conf
The main thing that is needed is to enable the mail protocols you want:
protocols = imaps
Server monitoring To make all sorts of graphs:
apt-get install munin munin-node
Example: http://munin.ping.uio.no To compute more statistics:
  • anteater
  • isoqlog
  • mailgraph
Monitor system logs: logcheck:
  • sends you mail with abnormal log lines
  • It's important to customize what is normal and you do it with regular expressions
Filtering viruses and spam clamav - Virus scanner Virus scanning:
  • Postfix gives the mail to clamav that scans it and gives it back if it's clean.
  • Strategies for infected mail:
    • silently delete it
    • refuse the mail and send a notification to the sender
    • refuse the mail and send a notification to the receiver
    • quarantine the e-mail
    • refuse delivery with a SMTP error
    • deliver with an extra header that says that it's a virus
spamassassin - Spam filter Spam scanning:
  • Postfix gives the mail to spamd that scans it and gives it back with some spam information.
  • Strategies for spam mail:
    • silently delete it
    • refuse the mail and send a notification to the sender
    • refuse the mail and send a notification to the receiver
    • quarantine the e-mail
    • refuse delivery with a SMTP error
    • deliver with an extra header that says that it's spam
  • New techniques:
    • greylisting: when you receive a mail from a host you've never seen before, refuse it with a temporary error, and accept it the second time (after some time delay). Spammers normally don't retry, and implementing retry would increase their cost of sending e-mail.
    • crossassassin: if more than some amount of your users receive a mail with the same message ID, throw it away. Sending mails with different headers would increase the cost of sending e-mail.
Man pages and sections Man pages are divided in sections:
  • man man shows all the sections of the manpages
  • man 5 postconf shows the postconf manpage in the "configuration file" section
  • Normally manpages are referred as manpage(section) (e.g. postconf(5) )

Enrico Zini: Fifth day in Addis

Samba To get samba:
apt-get install samba samba-doc smbclient
To get the Samba Web Administration Tool:
apt-get install swat netkit-inetd
The configuration is in /etc/samba:
  • One [global] section with the general settings
  • One section per share
One could use swat at http://localhost:901/ but it does not work easily on Ubuntu. To see what is shared:
smbclient -L localhost
To access a share:
smbclient //localhost/name-of-the-share
To add a new user:
sudo smbpasswd -a username
To change the password of a user:
sudo smbpasswd username
To test accessing a share as a user:
smbclient //localhost/web -U yared
Documentation:
man smb.conf
To force the user or group used to access a share:
force user = enrico
force group = www-data
To set the unix permissions for every created file:
# For files
create mask = 0664
# For directories
directory mask = 0775
Example share configuration for a webspace:
mkdir /var/www/public
chgrp www-data /var/www/public
chmod 0775 /var/www/public
Then, in /etc/samba/smb.conf:
[web]
   comment = Webspace
   path = /var/www
   writable = yes
   public = no
   force group = www-data
   create mask = 0664
   directory mask = 0775
Example share configuration for a read only directory where only a limited group of people can write:
[documents]
   comment = Documents
   path = /home/enrico/Desktop/documents
   force user = enrico
   public = yes
   writable = no
   write list = enrico, yared
Print server (CUPS) Installation:
apt-get install cupsys
Configuration:
  • On the web (not enabled in Ubuntu):

    http://localhost:631/
    
  • On the desktop:

    System/Administration/Printing
    
Example IPP URIs:
ipp://server[:port]/printers/queue
http://server:631/printers/queue
ipp://server[:port]/...
For example:
ipp://server/printers/laserjet
"This printer uri scheme can be used to contact local or remote print services to address a particular queue on the named host in the uri. The "ipp" uri scheme is specified in the Internet Print Protocol specifications and is actually much more free form that listed above. All Solaris and CUPS based print queues will be accessed using the formats listed above. Access to print queues on other IPP based print servers requires use of the server supported ipp uri format. Generally, it will be one of the formats listed above."
GRUB The configuration file is in /boot/grub/menu.lst. The documentation can be accessed as info grub after installing the package grub-doc. Quick list of keys for info:
arrows
move around
enter
enters a section
l
goes back
u
goes up one node
q
quit
/
search
Grub trick to have a memory checker:
  1. apt-get install memtest86+

  2. Add this to /boot/grub/menu.lst:

    title Memory test
        root (hd0,5)
        kernel /boot/memtest86+.bin
    
Firewall With iptables:
man iptables
# Only allow in input the network packets
# that are going to the web server
iptables -P INPUT DROP
iptables -A INPUT --protocol tcp --destination port 80 -j ACCEPT
# To reset the input chain as the default
iptables -F INPUT
iptables -P INPUT ACCEPT
Some links:
Squid Installation:
apt-get install squid
The configuration is in /etc/squid/squid.conf. To allow the local network to use the proxy:
# Add this before "http_access deny all"
acl our_networks src 10.4.15.0/24
http_access allow our_networks
To use a parent proxy:
cache_peer proxy.aau.edu.et     parent    8080  0  proxy-only no-query
Pay attention because /var/spool/squid will grow as the cache is used. The maximum cache size is set in the directive cache_dir. Information about squid access control is at http://www.squid-cache.org/Doc/FAQ/FAQ-10.html To check that the configuration has no syntactic errors: squid -k parse. To match urls:
acl forbiddensites url_regex [-i] regexp
For info about regular expressions:
man regex
Example filtering by regular expression:
acl skype url_regex -i [^A-Za-z]skype[^A-Za-z]
http_access deny skype
Transparent proxy setup: http://www.tldp.org/HOWTO/TransparentProxy.html
Problems found today Hiccups of the day:
  • swat does not run on Ubuntu because Ubuntu does not have inetd
  • swat does not allow root login on Ubuntu because root does not have a password
  • smbpasswd -a does not seem to update the timestamp of /var/lib/samba/passwd.tdb
  • cups web admin does not work on Ubuntu
  • LDAP is still not so intuitive to set up

Enrico Zini: Fourth day in Addis

Unix file permissions:
drwxr-xr-x   2 root root    38 2006-07-14
 
+- Is a directory
drwxr-xr-x   2 root root    38 2006-07-14
 ---
   
  +- User permissions (u)
drwxr-xr-x   2 root root    38 2006-07-14
    ---
      
     +- Group permissions (g)
drwxr-xr-x   2 root root    38 2006-07-14
       ---
         
        +- Permissions for others (o)
drwxr-xr-x   2 root root    38 2006-07-14
               ----
                 
                +- Owner user
drwxr-xr-x   2 root root    38 2006-07-14
                    ----
                      
        Owner group -+
Other bits:
  • 4000 Set user ID:
    • For executable files: run as the user who owns the file, instead of the user who runs the file
    • For directories: I think it's not used
  • 2000 Set group ID:
    • For executable files: run as the group who owns the file, instead of the group of the user who runs the file
    • For directories: when a file is created inside the directory, it belongs to the group of the directory instead of the default group of the user who created the file
  • 1000 Sticky bit:
    • For files: I think it's not used anymore
    • For directories: only the owner of a file can delete or rename the file
The executable bit for directories means "can access the files in the directory". If a directory is readable but not executable, then I can see the list of files (with ls) but I cannot access the files. To access a file, all the directories of its path up to / need to be executable. Commands to manipulate permissions:
  • chown - change file owner and group
  • chgrp - change group ownership
  • chmod - change file access permissions
  • sudo adduser enrico www-data adds the user enrico to the group www-data.
Example setup for a website for students:
# Create the group 'students'
mkdir /var/www/students
chgrp students /var/www/students
chmod 2775 /var/www/students
# If you don't want other users to read the files of the students:
chmod 2770 /var/www/students
adduser www-data students
 (this way the web server can read the
  pages)
# when you add a user to a group, it does not affect running processes:
 - users need to log out and in again
 - servers need to be restarted
Apache:
  • To install apache2 without a graphical interface:

    apt-cache search apache2   less
    sudo apt-get install apache2
    
  • By default, /var/www is where is the static website.

  • By default, ~/public_html is the personal webspace for every user, accessible as: http://localhost/~user

  • By default, /usr/lib/cgi-bin contains scripts that are executed when someone browses http://website/cgi-bin/script

  • By default, apache reads the server name from the DNS. If we don't have a name in the DNS and we want to use the IP, we need to set:

    ServerName 10.4.15.158
    
    in /etc/apache/apache2.conf (set it to your IP address)
  • To access the Apache manual: http://localhost/doc/apache2-doc/manual/

  • http://localhost/doc/apache2-doc/manual/mod/mod_access.html The access control module

  • http://localhost/doc/apache2-doc/manual/mod/mod_auth.html The user authentication module

  • To edit a user password file, use:

    htpasswd - Manage user files for basic authentication
    
  • Example .htaccess file to password protect a directory:

    AuthUserFile /etc/apache2/students
    AuthType Basic
    AuthName "Students"
    Require valid-user
    
  • Information about .htaccess is in http://localhost/doc/apache2-doc/manual/howto/htaccess.html

  • If you need to tell apache to listen on different ports, add a Listen directive to /etc/apache2/ports.conf. Then you can use:

    <VirtualHost www.training.aau.edu.et:9000>
    [...]
    </VirtualHost>
    
  • To setup an HTTPS website:

    • Documentation is in http://localhost/doc/apache2-doc/manual/ssl/

    • How to create a certificate: http://www.tc.umn.edu/~brams006/selfsign.html

    • Create a certificate:

      /usr/sbin/apache2-ssl-certificate -days 365
      
    • Create a virtual host on port 443:

      <VirtualHost www.training.aau.edu.et:443>
      [...]
      </VirtualHost>
      
    • Enable SSL in the VirtualHost:

      SSLEngine On
      SSLCertificateFile /etc/apache2/ssl/apache.pem
      
    • Enable listening on the HTTPS port (/etc/apache2/ports.conf):

      Listen 443
      
Apache troubleshooting:
  • check that there are no errors in the configuration file:

    apache2ctl configtest
    
    This it is always a good thing to do before restarting or reloading apache.
  • read logs in /var/log/apache2/

  • if you made a change but you don't see it on the web, it can be that you have the old page in the cache of the browser: try reloading a few times.

To install PHP
  • apt-get install libapache2-module-php5

  • then by default, every file .php is executed as php code

  • Small but useful test php file:

    <? phpinfo() ?>
    
To install MySQL
  • apt-get install mysql-client mysql-server

  • for administration run mysql as root:

    • Create a database with:

      create database students
      
  • Give a user access to the database:

    # Without password
    grant all on students.* to enrico;
    # With password
    grant all on students.* to enrico identified by "SECRET";
    
  • More information can be found at http://www-css.fnal.gov/dsg/external/freeware/mysqlAdmin.html

To use MySQL from PHP:
apt-get install php5-mysqli php5-mysql
Problems found today:
  • the apache2 manual in /usr/share/doc/manual can only be viewed using apache because it uses MultiView. So you need to have a working apache to read how to have a working apache.
  • chmod does not have examples in the manpage.

Enrico Zini: Third day in Addis

Believe it or not, a network that fails often is the best thing to have when you are teaching network troubleshooting. Various tools useful for networking:
  • ifconfig - configure a network interface

  • dnsmasq - Simple DNS and DHCP server

  • host - DNS lookup utility

  • route - show / manipulate the IP routing table

  • arping - send ARP REQUEST to a neighbour host

  • mii-tool - view, manipulate media-independent interface status (IOW, see if the cable works)

  • nmap - Network exploration tool and security / port scanner Examples:

    # Look at what machines are active in the local network:
    nmap -sP 10.5.15.0/24
    # Look at what ports are open in a machine:
    nmap 10.5.15.26
    
  • tcpdump - dump traffic on a network It can be used to see if there is traffic, and to detect traffic that shouldn't be there.

Useful tip:
# Convert a unix timestamp to a readable date
date -d @1152841341
What happens when you browse a web page:
  1. type the address www.google.com in the browser
  2. the browser needs the IP address of the web server:
  1. look for the DNS address in /etc/resolv.conf (/etc/resolv.conf is created automatically by the DHCP client)
  2. try all the DNS servers in /etc/resolv.conf until one gives you the IP address of www.google.com
  3. take the first address that comes from the DNS (in our case was 64.233.167.104)
  1. figure out how to connect to 64.233.167.104:
  1. consult the routing table to see if it's in the local network:
  1. if it's in the local network, then look for the MAC address (using ARP - Address Resolution Protocol)
  2. if it'd not in the local network, then send through the gateway (again using ARP to find the MAC address of the gateway)
  1. Send out the HTTP request to the local web server or through the gateway, using the Ethernet physical protocol, and the MAC address to refer to the other machine.
Troubleshooting network problems:
  1. See if the network driver works:
  1. With ifconfig, see if you see the HWaddr:. If you do not see it, then the linux driver for the network card is not working. Unfortunately there's no exact way to say that it works perfectly
  1. See if you have an IP address with ifconfig. If you find out that you need to rerun DHCP (for example, if the network cable was disconnected when the system started), then you can do it either by deactivating/reactivating the Ethernet interface using System/Administration/Networking or, on a terminal, running:

    # ifdown eth0
    # ifup eth0
    
    If you don't get an IP, try to see if the DHCP server is reachable by running:
    $ arping -D [address of DHCP server]
    
  2. See if the local physical network works:

  1. With sudo mii-tool, see if the cable link is ok. If it's not, then it's a problem in the cable or the plugs, or simply the device at the other end of the cable is turned off.
  2. Try arping or ping -n on a machine in the local network (like the gateway) to see if the local network works.
  1. See if the DNS works:
  1. Find out the DNS address:

    cat /etc/resolv.conf
    
  2. If it's local, arping it

  3. If it's not local, ping -n it

  4. Try to resolve a famous name using that DNS:

    $ host [name] [IP address of the DNS]
    
  5. Try to resolve the name of the machine you're trying to connect. If you can resolve a famous name but not the name you need, then it's likely a problem with their DNS.

  1. If you use a proxy, see if the proxy is reachable: check if the proxy name resolves to an IP, if you can ping it, if you can telnet to the proxy address and port:

    $ telnet [proxy address] [proxy port]
    
    you quit telnet with ^]quit.
  2. If you can connect directly to the web server, try to see if it answers:

    $ telnet [address] 80
    
    If you are connected, you can confirm that it's a web server:
    GET / HTTP/1.0 (then Enter twice)
    
    If it's a web server, it should give you something like a webpage or an HTTP redirect.
When you try to setup a service and it doesn't work:
  1. check that it's running:

    $ ps aux   grep dnsmasq
    
  2. check that it's listening on the right port:

    $ sudo netstat -lp
    
  3. check that it's listening from the outside:

    $ nmap [hostname]
    
  4. check for messages in /var/log/daemon.log or /var/log/syslog

  5. check that the configuration is correct and reload or restart the server to make sure it's running with the right configuration:

    # /etc/init.d/dnsmasq restart
    
dnsmasq:
By default: works as a DNS server that serves the data in /etc/hosts. By default: uses /etc/resolv.conf to find addresses of other DNS to use when a name is not found in /etc/hosts. To enable the DHCP server, uncomment:
dhcp-range=192.168.0.50,192.168.0.150,12h
in /etc/dnsmasq.conf and set it to the range of addresses you want to serve. Pay attention to never put two DHCP servers on the same local network, or they will interfere with each others. To test if the DHCP server is working, use dhcping (not installed by default on Ubuntu). To communicate other information like DNS, gateway and netmask to the clients, use this piece of dnsmasq.conf:
# For reference, the common options are:
# subnet mask - 1
# default router - 3
# DNS server - 6
# broadcast address - 28
dhcp-option=1,255.255.255.0
dhcp-option=3,192.168.0.1
dhcp-option=6,192.168.0.1
dhcp-option=28,192.168.0.255
Problems found today:
  • changing the name of the local machine in /etc/hosts breaks sudo, and without sudo it's impossible to edit the file. The only way to fix this is a reboot in recovery mode.
  • dhclient -n -w is different than dhclient -nw
Quick start examples with tar:
# Create an archive
tar zcvf nmap.tar.gz *.deb
# Extract an archive
tar zxvf nmap.tar.gz
# Look at the contents of an archive
tar ztvf nmap.tar.gz
Quick & dirty way to send a file between two computers without web server, e-mail, shared disk space or any other infrastructure:
# To send
nc -l -p 12345 -q 1 < nmap.tar.gz
# To receive
nc 10.5.15.123 12345 > nmap.tar.gz
# To repeat the send command 20 times
for i in  seq 1 20 ; do nc -l -p 12345 -q 1 < nmap.tar.gz ; done

Luciano Bello: the WTF survey

Sometimes, if you don t blog for a while, you need a good excuse for do it. Today I have one.


Click to see the whole page


You don t speak Spanish? You should :P.

It s a survey (or an opinion poll, i m not sure). The newspaper La Raz n ask to the readers: Are you agree that the tubal ligation and the vasectomy are allowed?. The answers are: between 9 and 7 hours (49.8%), between 6 and 4 hours (46.1%), ten or more (2.6%) and neither, I can t sleep (1.5%).

17 July 2006

Evan Prodromou: 29 Messidor CCXIV

We had a great trip to wt:Mauricie this weekend. It was quite an adventure -- we took off from Montreal on Saturday in the middle of a downpour, with no idea if it was going to clear before we got to the R serve faunique de St.-Maurice. But the rain slowed as we passed through wt:Yamachiche and by the time we were in wt:Shawinigan it was clear. Driving up the beautiful St.-Maurice River is real pleasure, especially when it's freshly-rained-on. The river is wide and slow (I think because of the hydro plant at Shawinigan) and sailboats and houseboats run up and down it. We got to the welcome center for the park too late -- it was closed -- so we had to go across the $12 (!) toll bridge into the reserve on faith alone. Which took a lot, since it's all dirt roads on the other side. It was a beautiful ride, but about 30 bone-jarring kilometers to our final campsite. We entertained the baby by making long aaaaaaaaaah sounds over the grading bumps -- by the end of the ride she was doing it herself. We saw a huge black bear on the way up -- maybe twice my size, long and lanky, loping across the road like it owned the place. Maj's face dropped when she saw it -- she'd never seen a bear before. It was an impressive animal, and it looked like Amita straight-leg crawling. We had a little hassle at the front gate of the Lac Normand campground, but we got settled down OK, put the baby in an insect-proof playpen, and Maj made a great pasta dinner while I put up the tent. I finished up the dishes and Maj and Amita walked down to the beach ("one of the best in Quebec", gushes S paq, and I'm inclined to believe them), and I was just starting a campfire when another storm hit. So we played in the tent for a while and went to sleep. Sunday we had a quick breakfast and headed down to la plage. The lake was beautiful, with a 4-5km sand beach ringing it on the north side. All three of us went for a swim in the water; Amita held onto Maj's back and made motorboat noises. On the drive out we tried to cut across the Parc de la Mauricie and the R serve faunique de Mastigouche to get to wt:Saint-Michel-des-Saints, which is a town in wt:Lanaudi re that I've wanted to visit for a while. But we took a wrong turn at poorly-marked crossroads, and ended up at the safe and sane exit instead. How often does that happen? Drive home was nice, and we had sushi from the Sushi Shop on Mont Royal. The cats were mad, but they always are. tags:

Kerblog Norsola Johnson passed along a link to Mazen Kerbaj's difficult Kerblog. Kerbaj is a expatriate Lebanese musician and comics artist caught in Beirut by the blockade and the destruction of the airport there. Stuck in the city under bombardment, he took the opportunity to start a comics blog. tags:

S3 I wrote about Amazon's crazy S3 (Simple Storage Service), which provides a Web interface for data storage. I was interested to see that Om Malik had an entry about it: StartUps Embracing Amazon S3. I guess it's a cheap and easy way to get a lot of storage when you don't have a lot of time to work on things like getting storage. Does it mean I've gone bad if I read Om Malik? He was a speaker at the Mesh Conference a couple of months ago, and I didn't like his talk. Now, I read his blog daily. Gar. tags:

Oh No Robot I just caught up with the latest Spamusement comic, and saw a "Transcribe This Comic!" link at the bottom. It goes to the fine Oh No Robot comics search and transcription service. I guess the idea is that readers transcribe comics into searchable text. A great idea, and a nice example of user-generated metadata. tags:

MicroID So, when ClaimID asked me to embed a MicroID into my home page to verify it, I thought they were nuts. But I did it anyways. Now, I'm trying to wrap my head around MicroID, and I'm starting to think it's very clever. I dunno... maybe a MediaWiki extension is in order... tags:

A couple of hours later... I dicked around a bit more with MicroID, and made my own MicroID verifier in Python. You're welcome to use it; see Software/Python/MicroID. tags:

Moon language From sneakums on #tron: I cannot speak your crazy moon language from Maciej Ceglowski a nice story about language and life in Montreal. tags:

Matt Brown: Patching the sc520 watchdog to reboot Soekris devices

We use a lot of Soekris devices at work which often get deployed to remote locations where its not easy to access them if things go wrong. I’ve recently been writing some software to act as a watchdog on the machine. As a very last resort this software makes use of the watchdog chipset provided by the Soekris boards so that if userspace goes away the machine will get rebooted back into what is hopefully a sane state. Unfortunately the in kernel sc520 watchdog device doesn’t actually reboot the Soekris when the heartbeat ping from userspace is lost. This makes it virtually useless. There has been a patch floating around since 2004 that fixes this problem, but it hasn’t made it in to the kernel for one reason or another. I think it’s Soekris specific… I’ve updated the patch to apply cleanly against the sc520 driver found in 2.6.16 and you can find it below. http://www.mattb.net.nz/patches/linux/soekris-sc520wd.patch

16 July 2006

Jordi Mallach: New SMS received

Message 1
I am in Spain!
What a fine country.
+1310XXXXXXX
2006/07/16 18:10:18
Dear sender,
I'm happy about you being so enthusiastic about your visit to Spain. I would probably be able to share the moment with you if I had the slightest clue about your identity.
Have a great night at nearly 30 C,
Jordi

15 July 2006

Andreas Barth: xorg-x11 removed from testing

Yesterday, we pushed some packages with force into testing:
force-hint gnome-media/2.14.2-1 sound-juicer/2.14.4-2 nautilus-cd-burner/2.14.2-1 libsexy/0.1.8-1 rhythmbox/0.9.5-1 gnome-python-extras/2.14.0-2 gnome-system-monitor/2.14.4-1 libgtop2/2.14.1-2 libgksu/1.9.4-3 wmfire/1.2.3-2 gnome-python-desktop/2.14.0-2 totem/1.4.1-2 evolution-data-server/1.6.2-3 gnome-panel/2.14.2-1 evolution/2.6.2-4 That means that all these source packages (and associated binary packages) are put into testing independent of what they break - but we assured before that they won't break anything bad. So, the result of that was that a few packages were broken on a few architectures, but we prevented that this large transition sticked together with the next large transition. These changes allowed us today to remove xorg-x11 (6.9) from testing, which brough uninstallabilities down to a very good level on most architectures: If you are interested in the detailed stats, they are at http://ftp-master.debian.org/~aba/test-issues/. If you are a porter of a not-as-good arch, you might be interessted to work on reducing the amount of broken packages - our goal is to have none at release. After this task is done, the next difficult one is already approaching.

Holger Levsen: the joys of spam and css

Unwanted cc:s on debian mailinglists clearly qualifies as "unsolicited bulk e-mail" for me, aka spam. So from now on I will send personal replies to those people asking them to stop (because very often people reply to those emails and keep the cc:s). Also I will keep them in a list of shame. Please, read your email before sending and include the headers in this reading. Thanks.

In completly unrelated news yesterday I finally (after nine month) fixed the layout of this blog, so you can sensibly read it with a webbrowser as

Ian Murdock

Simon Phipps: “Stewart is spot-on with Flickr’s policy and paradoxically will keep my business by allowing me to leave at any time.” Absolutely. This post nicely explains something I’ve noticed about my own actions recently, namely why I’ve remained a del.icio.us user even as I’ve become less than enthralled with its new owner: I can export my data at any time from del.icio.us and take it to a competing service. As Simon so eloquently explains, I remain a customer because it’s easy to stop being one. To compete, a rival service needs to not only provide a better product but also make it equally simple to take my business elsewhere, and most service providers don’t seem to be in any hurry to let me do that. Furthermore, the usual reasons for moving data elsewhere, better integration with other providers’ services or more compelling features in a competing service, are less important when the service I’m already using has an open, well-documented API that allows me to do whatever I want with my data without needing to export it.

Lars Wirzenius: Random thought: Understanding, accepting, agreeing

"I can understand why he said it." "Why do you defend him? How can you of all people agree with him?" In everyday language, to "understand something" is often interpreted as "agree with something" or "accept something". Usually, that is also what is meant, but it would be better if it weren't. There is a subtle difference between these meanings, and it is annoying when this is lost in the communication and misunderstandings happen. For example, I can definitely understand the urge of breaking into computers without permission. It is an intellectual, technical challenge, and there's the intense thrill of doing something you're not supposed to be doing. However, I do not, emphatically do not, agree, accept, condone, or otherwise suggest that it should be allowed. On the contrary, crackers should be taken out and spanked. Let's agree to understand and accept this subtlety. On the other hand, if making a point, it's good to know that not everyone understands subtlety, especially not all the time. Making the distinction clearer can prevent unnecessary disagreements.

Erich Schubert: JSP-like C++ webpages

TNTnet, just uploaded to Debian, looks interesting. Basically it offers JSP-/PHP-like embedding of C++ code into web pages (or the other way round). Well, if you want to be precise, the syntax is actually much closer to HTML::Mason, a perl-based web application framework I've used some years ago. You can bet that these pages, compiled into native code, are hard to beat speed-wise... Think of compiling your website into your webserver. It's obviously not suited for wide audiences - you can have your users upload PHP files, but you probably won't allow them to upload C++ code - but more for high-performance sites with specialized applications. I have, however, one major gripe with it: The syntax is not XML. I wish they would have picked true XML as data format, instead on basically using string processing. I hate how many people treat webpages as mere character strings with some special characters instead of structured data serialized in some way or another. I don't want to be stuck using a line-based editor for structured data forever. I'm still using vim, and I'll probably continue for a while, but for XML I'm actually just waiting for an editor with a true understanding of XML and the power and speed-of-use of vim... KID remains my favourite way of generating web pages. It's templating transparently embedded in a XHTML file using a separate XML namespace. And it's actually compiling the templates into python modules. P.S. if your ever need the Dummy 16 and Dummy17 functions, get them from microsoft (excel) now! No need to write them yourself anymore! (Dummy 1-15 are apparently discontinued.) P.P.S. I just wanted to point out, that I'm convinced that embedding much code into a layout file is a bad idea (one of the reasons why I despise PHP). Keep logics and layout separate. KID for example is quite good at that. In my DNSoupdate tool for example, all the template contains is loop constructs and simple case disinctions. This also brings me to my biggest gripe with KID: it depends on python. I don't see a way of using the same template with a different language. So maybe I'll actually go back to some TAL solution or so that is a bit more portable templating-wise (and still perfectly reuseable XML).

14 July 2006

Julien Danjou: Blogging is hard today

It began with heavy spam load in comments, last month someone tried to DDoS my blog (had to block 50 IP), and now I am fighting against spam in referers. YOU WON'T MAKE ME SILENT !!! THE WORLD WILL KNOW THE TRUTH !!!

13 July 2006

Andreas Barth: xorg-x11 removed from testing

Yesterday, we pushed some packages with force into testing:
force-hint gnome-media/2.14.2-1 sound-juicer/2.14.4-2 nautilus-cd-burner/2.14.2-1 libsexy/0.1.8-1 rhythmbox/0.9.5-1 gnome-python-extras/2.14.0-2 gnome-system-monitor/2.14.4-1 libgtop2/2.14.1-2 libgksu/1.9.4-3 wmfire/1.2.3-2 gnome-python-desktop/2.14.0-2 totem/1.4.1-2 evolution-data-server/1.6.2-3 gnome-panel/2.14.2-1 evolution/2.6.2-4 That means that all these source packages (and associated binary packages) are put into testing independent of what they break - but we assured before that they won't break anything bad. So, the result of that was that a few packages were broken on a few architectures, but we prevented that this large transition sticked together with the next large transition. These changes allowed us today to remove xorg-x11 (6.9) from testing, which brough uninstallabilities down to a very good level on most architectures: If you are interested in the detailed stats, they are at http://ftp-master.debian.org/~aba/test-issues/. If you are a porter of a not-as-good arch, you might be interessted to work on reducing the amount of broken packages - our goal is to have none at release. After this task is done, the next difficult one is already approaching.

Next.

Previous.