Prepare DNS

• Change the TTL on the DNS entry for libravatar.org to 3600 seconds.
• Remove the mirrors I don't control from the DNS load balancer (cdn and seccdn).
• Remove the main server from cdn and seccdn in DNS.

Preparing the new server

• Setup the new server.
• Copy the database from the old site and restore it.
• Copy /var/lib/libravatar from the old site.
• Hack my local /etc/hosts file to point to the new server's IP address:

  xxx.xxx.xxx.xxx www.libravatar.org stats.libravatar.org cdn.libravatar.org
  

• Test all functionality on the new site.

Preparing the old server

• Prepare a static "under migration" Apache config in /etc/apache2/sites-enables.static/:

  <VirtualHost *:80>
      RewriteEngine On
      RewriteRule ^ https://www.libravatar.org [redirect=301,last]
  </VirtualHost>
  <VirtualHost *:443>
      SSLEngine on
      SSLProtocol TLSv1
      SSLHonorCipherOrder On
      SSLCipherSuite RC4-SHA:HIGH:!kEDH
      SSLCertificateFile /etc/libravatar/www.crt
      SSLCertificateKeyFile /etc/libravatar/www.pem
      SSLCertificateChainFile /etc/libravatar/www-chain.pem
      RewriteEngine On
      RewriteRule ^ /var/www/migration.html [last]
      <Directory /var/www>
          Allow from all
          Options -Indexes
      </Directory>
  </VirtualHost>
  

• Put this static file in /var/www/migration.html:

  <html>
  <body>
  <p>We are migrating to a new server. See you soon!</p>
  <p>- <a href="http://identi.ca/libravatar">@libravatar</a></p>
  </body>
  </html>
  

• Enable the rewrite module:

  a2enmod rewrite
  

• Prepare an Apache config proxying to the new server in /etc/apache2/sites-enabled.proxy/:

  <VirtualHost *:80>
      RewriteEngine On
      RewriteRule ^ https://www.libravatar.org [redirect=301,last]
  </VirtualHost>
  <VirtualHost *:443>
      SSLEngine on
      SSLProtocol TLSv1
      SSLHonorCipherOrder On
      SSLCipherSuite RC4-SHA:HIGH:!kEDH
      SSLCertificateFile /etc/libravatar/www.crt
      SSLCertificateKeyFile /etc/libravatar/www.pem
      SSLCertificateChainFile /etc/libravatar/www-chain.pem
      SSLProxyEngine on
      ProxyPass / https://www.libravatar.org/
      ProxyPassReverse / https://www.libravatar.org/
  </VirtualHost>
  

• Enable the proxy-related modules for Apache:

  a2enmod proxy
  a2enmod proxy_connect
  a2enmod proxy_http
  

Migrating servers

• Tweet and dent about the upcoming migration.
• Enable the static file config on the old server (disabling the Django app).
• Copy the database from the old server and restore it on the new server.
• Copy /var/lib/libravatar from the old server to the new one.

Disable mirror sync

• Log into each mirror and comment out the sync cron jobs in /etc/cron.d/libravatar-slave.
• Make sure mirrors are no longer able to connect to the old server by deleting /var/lib/libravatar/master/.ssh/authorized_keys on the old server.

Testing the main site

• Hack my local /etc/hosts file to point to the new server's IP address:

  xxx.xxx.xxx.xxx www.libravatar.org stats.libravatar.org cdn.libravatar.org
  

• Test all functionality on the new site.
• If testing is successful, update DNS to point to the new server with a short TTL (in case we need to revert).
• Enable the proxy config on the old server.
• Hack my local /etc/hosts file to point to the old server's IP address.
• Test basic functionality going through the proxy.
• Remove local /etc/hosts/ hacks.

Re-enable mirror sync

• Build a new libravatar-slave package with an updated known_hosts file for the new server.
• Log into each server I control and update that package.
• Test the connection to the master (hacking /etc/hosts on the mirror if needed):

  sudo -u libravatar-slave ssh libravatar-master@0.cdn.libravatar.org
  

• Uncomment the sync cron jobs in /etc/cron.d/libravatar-slave.
• An hour later, make sure that new images are copied over and that the TLS certs are still working.
• Remove /etc/hosts hacks from all mirrors.

Post migration steps

• Tweet and dent about the fact that the migration was successful.
• Send a test email to the support address included in the tweet/dent.
• Take a backup of config files and data on the old server in case I forgot to copy something to the new one.
• Get in touch with mirror owners to tell them to update libravatar-slave package and test ssh configuration.
• Add third-party controlled mirrors back to the DNS load-balancer once they are up to date.
• A few days later, change the TTL for the main site back to 43200 seconds.
• A week later, kill the proxy on the old server by shutting it down.

• TI cc1111
• Atmel ATmega32U4
• Atmel ATiny85
• STMicroelectronics STM32L151/STM32L152
• NXP LPC11U14

• Free software. I m long past the age at which I trust hardware vendors to deliver credible binary software. If you want me to use your silicon, make sure the tools required to use it are all free software.
• Debian support required. Bdale and I do everything on Debian, so any processor we use needs to have tools that run under Debian. We re more than willing to package software for Debian, so if it s just generic Linux-compatible free software, that s fine.
• Built-in USB. Everything we build needs to talk to a computer for configuration and data download. Don t even think of asking me to attach another chip for USB support. Ok, so the ATtiny85 doesn t qualify here, and that project does use an external chip for USB which is connected to the ATtiny85 through an LED and phototransistor. I was excited when TI came out with the CC340 chips because the MSP430 is a much nicer CPU than the 8051. But, they still haven t added USB. If I m going to add another chip for USB, it ll be a CPU anyways, at which point I ll use a stand-alone RF chip.
• Cheap developer boards. TI sells a tiny little cc1111 developer board; STmicroelectronics and NXP both sell boards that include a programmer and a target CPU for prototyping software. And, there are a million Arduino clones with various Atmel processors on them. Cheap means I can buy them without thinking too hard; having a built-in programmer means that early OS development can be done without first having to figure out how to wire up a programmer.
• Probably 32 bits, probably an ARM. Yeah, the cc1111 and Atmel processors were fun to bring up, but I m pretty much over the 8-bit microcontroller now. ARM processors are far more sophisticated in terms of power management and built-in device capability, plus they re a ton faster at doing any math. The ATtiny85 remains an exception here; it s smaller than any ARM I can get, and so it s still useful for projects too small for an ARM.
• Available in small quantities from a reasonable on-line retailer like Digikey, Mouser or even Avnet Express. I d love to try out the new low-cost STM32L100 parts, but the only one available is the largest one in the 64LQFP package, and I really don t need that many pins for most products.

• It s small (5mm x 5mm)
• It s cheap ($1.48 q100)

• Publish the programmer source code. The programming dongle is a tool for selling your silicon, not a separate revenue stream. Publish the source under a free software license (GPL recommended, of course). We ll take it, make it better, and ensure that you get the changes back to help your other customers. It s a win-win plan.
• Document the flash programming registers. Yeah, I could probably disassemble the mask-rom bits that are on the chip, but I have better things to do with my time. Heck, go wild and document any other hidden bits in the silicon. Again, we re embedded developers, we don t like magic hardware hidden from view any more than PC developers like the BIOS and SMM.

$ cat meta-data
instance-id: iid-local01;
local-hostname: myhost;

$ cat user-data
#cloud-config
password: mypassword
ssh_pwauth: True
chpasswd:   expire: False  
ssh_authorized_keys:
  - ssh-rsa ... foo@foo.com (insert ~/.ssh/id_rsa.pub here)

$ genisoimage -output init.iso -volid cidata -joliet -rock user-data meta-data

Required-Start:    $network

start on (local-filesystems and net-device-up IFACE!=lo)
stop on runlevel [!2345]
expect fork
respawn
exec nmbd -D

start on net-device-up IFACE=wlan0
stop on net-device-down IFACE=wlan0
expect daemon
respawn
exec mydaemon

$ cat /etc/init/tftpd-hpa.conf
start on runlevel [2345]
stop on runlevel [!2345]
expect fork
respawn
script
        if [ -f $ DEFAULTS  ]; then
                . $ DEFAULTS 
        fi
        exec /usr/sbin/in.tftpd --listen  --user $ TFTP_USERNAME  --address $ TFTP_ADDRESS  $ TFTP_OPTIONS  $ TFTP_DIRECTORY 
end script
$ cat /etc/init/tftpd-hpa-restart.conf
start on net-device-up
task
instance $IFACE
script
        status tftpd-hpa   grep -q start/   stop
        restart tftpd-hpa
end script

imon 4-4:1.0: imon_incoming_packet: unknown keypress, code 0x100000e

• the problem was known
• the fix was in the Linus kernel git repository
• but not in any released kernel

% sudo input-kbd 6
0x100007f = 106  # KEY_RIGHT
0x1000080 = 105  # KEY_LEFT
0x1007f00 = 108  # KEY_DOWN
0x1008000 = 103  # KEY_UP
0x1010000 = 272  # BTN_LEFT
0x1010080 = 272  # BTN_LEFT
0x1020000 = 273  # BTN_RIGHT
0x1020080 = 273  # BTN_RIGHT
0x200001e = 513  # KEY_NUMERIC_1
0x200001f = 514  # KEY_NUMERIC_2

0x200001e  KP1
0x200001f  KP2
0x2000020  KP3
0x2000021  KP4
0x2000022  KP5
0x2000023  KP6
0x2000024  KP7
0x2000025  KP8
0x2000026  KP9
0x2000027  KP0
0x2800000  MENU
0x288795b7 PAGEDOWN
0x289395b7 PAGEUP

• You don't need to put there the scancode and keycode you don't divert,
• the interesting scancode are found by input-kbd
• you can also use input-events to find which scancode correspond to one key
• the interesting keycode are in /usr/include/linux/input.h, note that this is not the same thing than X keycode...

ACTION=="remove", GOTO="mykeyboard_end"
KERNEL!="event*", GOTO="mykeyboard_end"
ENV ID_INPUT_KEY =="", GOTO="mykeyboard_end"
SUBSYSTEMS=="bluetooth", GOTO="mykeyboard_end"
SUBSYSTEMS=="usb", IMPORT builtin ="usb_id"
ENV ID_VENDOR =="15c2", ENV ID_MODEL_ID =="0042", RUN+="keymap $name /etc/udev/imon"
LABEL="mykeyboard_end"

Bus 004 Device 014: ID 15c2:0042 SoundGraph Inc.

1. Look like a lisp, DTRT with it (e.g. dashes turn to underscores, earmuffs turn to all-caps.)
2. We're still Python. Most of the internals translate 1:1 to Python internals.
3. Use unicode everywhere.
4. Tests or it doesn't exist.
5. Fix the bad decisions in Python 2 when we can (see true_division)
6. When in doubt, defer to Python.
7. If you're still unsure, defer to Clojure
8. If you're even more unsure, you're fucked (see also: defer to Common Lisp)

;; good:
(with [fd (open "/etc/passwd")]
    (print (.readlines fd)))
;; bad:
(with [fd (open "/etc/passwd")]
    (print (fd.readlines)))

;; good:
(import [sh [cat grep]])
(-> (cat "/usr/share/dict/words") (grep "-E" "tag$"))
;; bad:
(import [sh [cat grep]])
(grep (cat "/usr/share/dict/words") "-E" "tag$")

;; good (and prefered):
(defn fib [n]
  (if (<= n 2)
      n
      (+ (fib (- n 1)) (fib (- n 2)))))
;; still OK:
(defn fib [n]
  (if (<= n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
;; still OK:
(defn fib [n]
  (if (<= n 2)
    n
    (+ (fib (- n 1)) (fib (- n 2)))))
;; Stupid as hell
(defn fib [n]
    (if (<= n 2)
            n
      (+ (fib (- n 1)) (fib (- n 2)))))

;; good (and prefered):
(defn fib [n]
  (if (<= n 2)
      n
      (+ (fib (- n 1)) (fib (- n 2)))))
;; Stupid as hell
(defn fib [n] 
    (if (<= n 2)
      n
      (+ (fib (- n 1)) (fib (- n 2)))
    )
)  ; GAH, BURN IT WITH FIRE

;; bad (and evil)
(defn foo (x) (print x))
(foo 1)

;; good (and prefered):
(defn foo [x] (print x))
(foo 1)

Planning Before you fly out, make sure you look up the visa requirements for your country of citizenship. Many western countries will require a visa and you will need to visit the local Brazillian embassy ahead of time to get one. Next, have a look at the list of recommended immunizations. As with most destinations, it is recommended that your routine immunizations be up to date, but there are also other specialized ones such as Yellow Fever that are recommended by the Brazillian government. You should therefore visit a travel clinic a few weeks ahead of time. Other than that, I suggest reading up on the country and keeping an eye on the various travel advisories.

Arrival You will be flying at the Porto Alegre airport. If you need to exchange overseas money for Brazilian Reals, you can do that there. You'll probably also want to pick up a power adapter at the airport if you intend to charge your laptop while you're in the country Brazil has both 127V and 220V outlets using Type N sockets. Privacy note: using the free airport wifi will require giving your Passport details as part of the registration process.

Language If you don't speak Portuguese, expect a few challenges since most of the people you'll meet (including taxi drivers, many airport workers, some hotel staff) won't speak English. I highly recommend getting a phrase book before you leave and printing paper maps of where you are planning to go (to show to taxi drivers when you get lost). Native Spanish speakers seem to get by speaking Spanish to Portuguese speakers and understanding enough Portuguese to hold a conversation. I wouldn't count on it unless your Spanish is quite good though. Also, the official conference blog posts get eventually translated to English, but there is a delay, so you may want to subscribe to the Portuguese feed and use Google Translate to keep up with FISL news before you get there.

The conference FISL is a large conference and it has a very "decentralized" feel to it. From the outside, it looks like it's organized by an army of volunteers where everyone is taking care of some portion of it without a whole lot of top-down direction. It seems to work quite well! What this means for you as a foreign speaker however is that you're unlikely to be provided with a lot of information or help finding your way around the conference (i.e. no "speaker liaison"). There is a separate registration desk for speakers but that's about all of the attention you'll receive before you deliver your talk. So make sure you know where to go and show up in your assigned room early to speak with the person introducing you. If your talk is in English, it will be live-transated by an interpreter. It's therefore a good idea to speak a bit more slowly and to pause a bit more. Other than that, the organizers make an effort to schedule an English talk in each timeslot so non-Portuguese speakers should still be able to get a lot out of the conference. FISL was a lot of fun for me and I hope that some of these tips will help you enjoy the biggest FLOSS gathering in the southern hemisphere!

• Switched roaming workstation profiles from wicd to network-manager for network configuration, as wicd didn't work any more.
• Changed version numbers of patched gosa and libpam-mklocaluser packages to make sure our locally patched versions will be replaced by the official packages when they are released from Debian. Those installing alpha version need to reinstall or manually downgrade gosa and libpam-mklocaluser.
• Added bluetooth tools to the default desktop (bluedevil, blueman).
• Added tools for sharing the desktop on KDE (krdc, krfb).
• Added valgrind to the default installation for easier debugging of crash bugs.

• Fixed artwork package to work with gnome, no longer break desktop=gnome installations.
• Adjusted installer to now work when forced to use a proxy with the netinst CD.
• Fixed code detecting and setting/loading hardware specific setup/firmware to work more robust out of the box.
• Adjusted Kerberos setup to detect realm and server settings at install time instead of dynamically at run time. This avoid a crash with krb5-auth-dialog on diskless workstations without a DNS name.
• Worked around misfeature in network-manager not calling the dhclient exit hooks, causing automatic proxy configuration and automatic host name setting at run time to work again.
• Fixed feature setting the default Iceweasel start page from URL fetched from LDAP, to allow schools to set the global default by updating the dc=skole,dc=skolelinux,dc=no LDAP object.
• Changed default host name on all networked machines to be unique (generated from MAC or reverse DNS) after boot.
• Adjusted partition sizes to make sure they are big enough.

• Grub is missing the new artwork.
• KDE fail to understand the wpad.dat file provided, causing it to not use the http proxy as it should.
• Chromium also fail to use the proxy.

• ftp://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso
• http://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso
• rsync -avzP ftp.skolelinux.org::skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso .

• ftp://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso
• http://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso
• rsync -avzP ftp.skolelinux.org::skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso .

• Authentication for private repos? (-p)
• Other clone mechanisms? (-c)
• A help function? (-h)
• Other features?

0 comment(s) this blog is flattr enabled

There will come a time when you believe everything is finished. That will be the beginning. Louis L Amour

People speak of misfortunes and sufferings, remarked Pierre, but if at this moment I were asked: Would you rather be what you were before you were taken prisoner, or go through all this again? then for heaven s sake let me again have captivity and horseflesh! We imagine that when we are thrown out of our usual ruts all is lost, but it is only then that what is new and good begins. While there is life there is happiness. There is much, much before us. Tolstoy, War and Peace

• Updating involves multiple copies of the window contents, first from back buffer to composite redirect buffer, thence from the composite buffer to the screen back buffer, and then from the screen back buffer to the scanout buffer. The last copy is amenable to page flipping, but that would require the EGL buffer age extension so that the compositor could reduce the cost of presenting following frames by being able to track the contents of the back buffers.
• The application is not informed about when the actual screen presentation occurs.

1. Provide accurate information to applications about when presentation to the screen actually occurs. In particular, GLX applications using GLX_OML_sync_control should receive the correct information in terms of UST and MSC for each Swap Buffers request.
2. Ensure that applications still receive correct information as to the contents of their buffers, in particular we want to be able to implement EGL_EXT_buffer_age in a useful manner.
3. Avoid needing to un-redirect full-screen windows to get page flipping behavior.
4. Eliminate all extra copies. A windowed application may still perform one copy from back buffer to scanout buffer, but there shouldn t be any reason to copy contents to the composite redirection buffer or the compositing manager back buffer.

1. Application creates back buffer, draws new frame to it.
2. Application executes PresentRegion. In this example, the valid and update parameters are None , indicating that the full window should be redrawn.
3. The server captures the PresentRegion request and constructs a PresentRedirectNotify event containing sufficient information for the compositor to place that image correctly on the screen:
   • target window for the presentation
   • source pixmap containing the new image
   • idle fence to notify when the source pixmap is no longer in use.
   • serial number from the request.
   • target MSC for the presentation. This should probably just be the computed absolute MSC value, and not the original target/divisor/numerator values provided by the application.
4. The compositing manager receives this event and constructs a new PresentRegion request using the provided source pixmap, but this time targeting the root window, and constructing a valid region which clips the pixmap to the shape of the window on the screen. This request would use the original application s idle fence value so that when complete, the application would get notified. This request would need to also include the original target window and serial number so that a suitable PresentCompleteNotify event can be constructed and delivered when the final presentation is complete.
5. The server executes this new PresentRegion operation. When complete, it delivers PresentCompleteNotify events to both the compositing manager and the application.
6. Once the source pixmap is no longer in use (either the copy has completed, or the screen has flipped away from this pixmap), the server triggers the idle fence.

1. The compositing manager receives this event. Using the new pixmap, along with pixmaps for the other involved windows and graphical elements, the compositing manager constructs an updated view for the affected portion of the screen in a back buffer pixmap. Once complete, a PresentRegion operation that uses this back buffer pixmap is sent to the X server. Again, the original target window and serial number are also delivered to the server so that a suitable PresentCompleteNotify event can be delivered to the application.
2. The server executes this new PresentRegion operation; PresentCompleteNotify events are delivered, and idle fences triggered as appropriate.

1. Full-screen flipped applications. In this case, the application s idle fence can be triggered once the application provides a new frame and the X server has flipped to that new frame, or some other scanout buffer.
2. Windowed, copied applications. In this case, the application s idle fence can be triggered once the application provides a new frame to the compositing manager, and the X server doesn t have any presentations queued.

1. Have the X server change how Present redirection works when some other application selects for Automatic redirection on a window. It would copy the source pixmap into the window buffer and then send (a modified?) PresentRedirectNotify event to the compositing manager.
2. Include a flag in the PresentRedirectNotify event that the composite redirect buffer needs to eventually get the contents of the source pixmap, and then expect the compositing manager to figure out what to do.

PRESENTNOTIFY  
    window: WINDOW
    serial: CARD32
     
 
    PresentRegion
    window: WINDOW
    pixmap: PIXMAP
    serial: CARD32
    valid-area: REGION or None
    update-area: REGION or None
    x-off, y-off: INT16
    idle-fence: FENCE
    target-crtc: CRTC or None
    target-msc: CARD64
    divisor: CARD64
    remainder: CARD64
    notifies: LISTofPRESENTNOTIFY
 
    Errors: Window, Pixmap, Match

 
    PresentRedirectNotify
    type: CARD8         XGE event type (35)
    extension: CARD8        Present extension request number
    length: CARD16          2
    evtype: CARD16          Present_RedirectNotify
    eventID: PRESENTEVENTID
    event-window: WINDOW
    window: WINDOW
    pixmap: PIXMAP
    serial: CARD32
    valid-area: REGION
    valid-rect: RECTANGLE
    update-area: REGION
    update-rect: RECTANGLE
    x-off, y-off: INT16
    target-crtc: CRTC
    target-msc: CARD64
    idle-fence: FENCE
    update-window: BOOL
 

1. Insert a command into the ring which blocks execution until a suitable time immediately preceding the blt operation.
2. Queue the blt operation at vblank time so that it executes before the scanout starts.

1. Queue the blt from the interrupt handler
2. Queue the blt from a kernel thread running in response to the interrupt
3. Send an event up to user space and have the X server construct the blt command.

1. The presented pixmap is owned by the application, and so it shouldn t be used except when the presented window covers the whole screen. When the window gets reconfigured, we end up copying the window s pixmap to the regular screen pixmap.
2. The kernel flipping API is asynchronous, and doesn t provide any abort mechanism. This isn t usually much of an issue; we simply delay reporting the actual time of flip until the kernel sends the notification event to the X server. However, if the window is reconfigured or destroyed while the flip is still pending, cleaning that up must wait until the flip has finished.
3. The applications buffer remains busy until it is no longer being used for scanout; that means that applications will have to be aware of this and ensure that they don t deadlock waiting for the current scanout buffer to become idle before switching to a new scanout buffer.

• GLX_EXT_swap_control_tear. This will provide applications with the ability to avoid dropping frames when pushing the hardware just over the frame rate limit.
• EGL_EXT_buffer_age. (I assume we ll probably want a GLX version as well?) This will allow compositors to more efficiently perform partial updates in a flipping environment, and is enabled by having all of the buffer management within the GL library.

• git://people.freedesktop.org/~keithp/dri3proto.git master. DRI3 protocol specification and X server headers.
• git://people.freedesktop.org/~keithp/presentproto.git master. Present protocol specification and X server headers.
• git://people.freedesktop.org/~keithp/xcb/proto.git dri3. XCB protocol defines for both DRI3 and Present.
• git://people.freedesktop.org/~keithp/xcb/libxcb.git dri3. XCB library changes for file descriptor passing.
• git://people.freedesktop.org/~keithp/xserver.git dri3. X server with file descriptor passing, DRI3 and Present support.
• git://people.freedesktop.org/~keithp/mesa.git dri3 Mesa with DRI3/Present support for GLX.
• git://people.freedesktop.org/~keithp/drm.git dri3. DRM library with defines for async flipping.
• git://people.freedesktop.org/~keithp/xf86-video-intel.git dri3. Intel driver with DRI3, Present and async flipping support.
• git://git.kernel.org/pub/scm/linux/kernel/git/keithp/linux-2.6 dri3. Kernel with async flipping.

ssh-keygen -t rsa

cp ~/.ssh/id_rsa.pub ~/

mkdir -p ~/.ssh
ssh jamie-share@chavez.mayfirst.org "cat id_rsa.pub" >> ~/.ssh/authorized_keys

sudo apt-get install openssh-server screen

ssh -R 2222:localhost:22 jamie-share@chavez.mayfirst.org

ssh <their-username>localhost -p 2222