Writing a File Atomically from Python

I wanted to be able to write to a file, but there was a possibility that others could be reading from it while I was writing to it. So I came up with this code below (which I release as public domain) which should guarantee that the file gets written without anyone being able to read it until it is completely written to disk.

import os, os.path
import errno

def write(filename, data, mode=0600):
    """ guarantees that noone can read filename with only partial data """
    fd = None
    seq = 1
    dn = os.path.dirname(filename)
    bn = os.path.basename(filename)
    while fd == None:
        try:
            fn = os.path.join(dn,".%s.%d"%(bn,seq))
            fd = os.open(fn, os.O_CREAT | os.O_EXCL | os.O_WRONLY, mode)
        except OSError, e:
            if e.errno == errno.EEXIST:
                seq += 1
                continue
            else:
                raise

    # write data to temporary file
    fo = os.fdopen(fd, "w")
    fo.write(str(data))
    fo.close
    #os.close(fd)

    # move file into place now that it is written
    os.rename(fn, filename)

Outputting HTML5 via XML/XSLT

One of my web applications uses XSLT to transform some XML data into output. For a long time, I was convinced XHTML was the future. But now I've seen the light and HTML5 is the way to go. I want my app to be able to take advantage of all the HTML5 goodness while still retaining the content/view separation that using XML and XSLT has allowed me.

Content With Style had a good blog post highlighting this problem, but it seems that the doctype has since changed. The current HTML5 spec outlines the current accepted doctype including the doctype that XSLT-generated documents can use.

So here's the xsl:output string I needed to use:

    <xsl:output method="html"
         doctype-system="about:legacy-compat"
         indent="yes"/>

Palm not syncing

My Palm wasn't connecting correctly to my Linux netbook. I was getting some weird errors in the logs:

usb 2-1: new full speed USB device using uhci_hcd and address 2
usb 2-1: device descriptor read/64, error -71
usb 2-1: device descriptor read/64, error -71
usb 2-1: new full speed USB device using uhci_hcd and address 3
usb 2-1: device descriptor read/64, error -71
usb 2-1: device descriptor read/64, error -71
usb 2-1: new full speed USB device using uhci_hcd and address 4
usb 2-1: device not accepting address 4, error -71
usb 2-1: new full speed USB device using uhci_hcd and address 5
usb 2-1: device not accepting address 5, error -71
hub 2-0:1.0: unable to enumerate USB device on port 1

After hunting and hunting (and not saving the page that gave me the answer), I discovered this is related to a bug on the Palm. The page I found suggested a reset. So I did, and it synced up right after that.

Screen Locking Shortcut in LXDE

I liked having a keyboard shortcut, ctrl-alt-l, to lock the screen under Gnome. But in LXDE, it was a bit trickier to setup. There isn't a fancy GUI for setting these options (that I know of), one has to edit Openbox's configuration file: ~/.config/openbox/lxde-rc.xml

Find the "<keyboard>" section and add the following:

    <keybind key="C-A-l">
      <action name="Execute">
        <command>xscreensaver-command -lock</command>
      </action>
    </keybind>

Suspend on Lid Close with LXDE

LXDE has been great on my Asus EEE 1001P. It's fast and usable. I've had some issues with it, but the most frustrating problem is that Fedora 12 with LXDE doesn't suspend by default when you close the lid. I did some googling and discovered this is known and the easy to fix it is to install the gnome-power-manager. But that kind of defeats the point of LXDE, right? I'm trying to avoid installing and using gnome heaviness to stay lean and mean and fast.

So, after lots of googling around, I finally came up with the optimum configuration which uses what I hope are lighter-weight services.

First, I had to install acpid to respond to lid close/open events (my code is based on this post):

yum install acpid

I hunted through the LXDE source code and discovered that lxde-logout uses a dbus call to HAL to trigger the suspend. I found a great blog post describing how to do this from a script. I implemented that by doing creating the following files:

/etc/acpi/events/lid (update: don't put a dot in the filename for Fedora 14+, it seems a new version of acpid ignores all files with dots in their names)

event=button/lid
action=/etc/acpi/actions/lid.sh

/etc/acpi/actions/lid.sh

#!/bin/bash

# maybe this sleep isn't needed, but just to make sure the lid close event doesn't trigger immediate wake-up
sleep 2

# make sure the lid is closed
grep -q open /proc/acpi/button/lid/LID/state && exit 0

# suspend
dbus-send --system --dest=org.freedesktop.Hal --type=method_call --print-reply /org/freedesktop/Hal/devices/computer org.freedesktop.Hal.Device.SystemPowerManagement.Suspend int32:0


Germany Bike Maps - For Free!

I've known about the OpenStreetMap project for some time, but I've never taken advantage of it before. But as I was hunting around for a Linux mapping program (and found "qlandkarte gt"), I stumbled across OpenStreetMap-based maps that you could use off-line. And among those were detailed bike trail maps of Germany. But loading them on to my GPS proved a bit complicated because I don't have a Windows box around!

First, I found the maps here: http://openmtbmap.org/download/

Then I had to download 7zip in order to unpack the map files (which are bundled in a Windows executable). qlandkarte was immediately able to use these map files locally. And they looked very detailed and accurate! But how to get them on to my Garmin Etrex Vista C (an older handheld model)?

Unfortunately, Garmin only provides map loading software for Windows and Mac. There appears to be an open-source project to do this, but it didn't look well supported or easy. Since I have a Mac, I downloaded Basecamp (i found many recommendations to skip Garmin's RoadTrip project and use the more stable Basecamp).

Next, you have to convert the Windows maps into Mac maps (why oh why does Garmin have separate formats for different programs?). Garmin provides a converter program that runs under windows (http://www8.garmin.com/support/download_details.jsp?id=3897), but fortunately, there is an open-source project called Gmapibuilder that does the same thing.

Next, I ran the Garmin Map Manager on my Mac (came bundled with Basecamp) to load the maps into Basecamp. Then, I used Garmin's Map Installer applications (also bundled with Basecamp) to select my GPS unit and load a subset of the map on to it. I was happy to discover that I was able to pick a subset because my eTrex Vista C only has 24MB of memory and no expansion slots. So I was able to reduce the map to just the area that I wanted. And voila! I was done!

I got some good help in this process from openmtbmap.org, but since it seemed missing a couple steps, I thought I'd document it all here.

chown in Python

The built-in chown in Python (in the os module) only uses uid/gid instead of username/groupname. So here's a couple of (simple) versions that can take usernames and groupnames. I can't remember where I borrowed some of this code from, but it was heavily inspired by code samples I found while google searching:

def chown(path,user,group):
    uid = pwd.getpwnam(user)[2]
    gid = grp.getgrnam(group)[2]
    os.chown(path,uid,gid)

def chown_recursive(path,user,group):
    if path is None or path in ['/','/root','/usr','/tmp','/bin','/sbin']:
        raise Error("path isn't right...")
    uid = pwd.getpwnam(user)[2]
    gid = grp.getgrnam(group)[2]
    os.chown(root, uid, gid)
    for root, dirs, files in os.walk(path):
        for name in dirs:
            os.chown(os.path.join(root,name), uid, gid)
        for name in files:
            os.chown(os.path.join(root,name), uid, gid)