Paths and line numbers are given for Ubuntu 7.04 Feisty Fawn, so if you’re on another distribution or another version of Ubuntu, your mileage may vary.

What we’ll do is add an XkbOption to the keyboard configuration files. At first, add the following at the end of /usr/share/X11/xkb/symbols/capslock:

partial hidden modifier_keys
xkb_symbols "escape" {
    key <CAPS> {        [       Escape  ]       };
    key <ESC>  {        [       None    ]       };
};

This defines an option that maps the CAPS and ESC key codes to the Escape and None symbols respectively. If you don’t want to disable the original Escape key, leave the corresponding line out. Now let’s give it an option name: insert the following line to /usr/share/X11/xkb/rules/base, somewhere around line 810, where the other capslock lines are:

caps:escape          =       +capslock(escape)

Now let’s add that option to the keyboard configuration in /etc/X11/xorg.conf: Find the InputDevice section for the keyboard and add the following line to the section:

        Option          "XkbOptions"    "caps:escape"

Now restart X by logging out and typing Ctrl-Alt-Backspace at the login prompt. That’s it. CapsLock is gone and on its place Escape stood in.

I wasn’t successful in convincing the XKeyboardConfig maintainer Sergej Udaltsov to accept a patch for this, but I attempted only once yet :)

Epiphany

1. Create a bookmark of a result page of the search engine (Ctrl-D)
2. Replace the search term with %s in the URL

Firefox

1. Write up an XML file according to the OpenSearch spec
2. Figure out how to load that file into Firefox
3. Figure out how to get that XML file right – Firefox just says that it doesn’t support this search engine
4. …

End of rant.

The outcome of the procedure is that you’ll have a second network card for your virtual machine, which you can configure from inside the virtual machine as if it was a real network card connected to the same ethernet as your real network card.

I assume that you either

• get your IP address with DHCP
• or have a static IP address

If you’re on DSL or cable modem with PPPoE you probably don’t want host networking, but NAT, which works out of the box.

We’re going to

1. create a virtual network interface
2. set up an ethernet bridge
3. add your physical network interface and the virtual interfaces to the bridge

Preparations

At first let’s install the needed software:

$ sudo apt-get install bridge-utils uml-utilities

The virtual network interface is configured with the package uml-utilities. uml stands for User Mode Linux, so understandably the package is preconfigured for use with User Mode Linux. In order to work correctly with VirtualBox, in /etc/network/if-pre-up.d/uml-utilities replace

chown root:uml-net /dev/net/tun

with

chown root:vboxusers /dev/net/tun

As we’re going to change the network configuration, we’re going to shut down the network. So don’t close this browser window :)

$ sudo /etc/init.d/networking stop

Creating a Virtual Network Interface

Now on to configuring the virtual network interface. Ubuntu and Debian store network configuration in the file /etc/network/interfaces. Assuming your network card is eth0, you’ll find a stanza beginning with the following in the file:

auto eth0
iface eth0 inet .....

Actually at install Ubuntu writes several such stanzas into the file so that they automatically work. Create a backup copy of /etc/network/interfaces (Now.) Then comment out all interfaces you don’t use so that NetworkManager doesn’t change the network configuration when we don’t want to. Leave in the lo interface, it’s required for local IP communication.

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet .....

#auto eth1
#iface eth1 inet dhcp

#auto ath0
#iface ath0 inet dhcp

#auto wlan0
#iface wlan0 inet dhcp

The next step is to create a virtual network interface. This is the network interface that will be used by the virtual machine. We don’t configure it – the virtual machine will do it. Add the following stanza under the lo interface:

auto tap0
iface tap0 inet manual
tunctl_user markus
uml_proxy_arp markus.mynetwork.loc
uml_proxy_ether eth0

Replace markus.mynetwork.loc with the host name or IP address you get from DHCP for your host machine (not the virtual machine). Also replace eth0 with your real network interface, if that is not eth0, and markus with your user name on the system. The user name set here will be the only user with permission to work with the virtual network interface.

Static IP address

If you have a static IP address, replace markus.mynetwork.loc instead with your static IP address.

Creating the bridge

Now we’ll set up the bridge. Add the following stanza after the tap0 stanza:

auto br0
iface br0 inet dhcp
bridge_ports eth0 tap0
bridge_maxwait 0

Static IP address

If you have a static IP address, you have to carry over the network settings from eth0 to br0. Assuming your eth0 stanza looks something like this:

auto eth0
iface eth0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.254

change the br0 stanza to look like this:

auto br0
iface br0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.254
bridge_ports eth0 tap0
bridge_maxwait 0

In any case, change the stanza for eth0 to the following:

auto eth0
iface eth0 inet manual

That’s it. Enable networking again:

$ sudo /etc/init.d/networking start

You can use ifconfig -a to see if all devices were correctly configured. You can now use host networking in VirtualBox. Enter tap0 in the interface name field. The fields setup application and terminate application can be left empty.

More than one virtual network interface

You can run more than one OS at the same time and have all them use host networking. Add a tap interface for all of them. Here’s an example for three virtual network interfaces:

auto tap0
iface tap0 inet manual
tunctl_user markus
uml_proxy_arp markus.mynetwork.loc
uml_proxy_ether eth0

auto tap1
iface tap1 inet manual
tunctl_user markus
uml_proxy_arp markus.mynetwork.loc
uml_proxy_ether eth0

auto tap2
iface tap2 inet manual
tunctl_user markus
uml_proxy_arp markus.mynetwork.loc
uml_proxy_ether eth0

auto br0
iface br0 inet dhcp
bridge_ports eth0 tap0 tap1 tap2
bridge_maxwait 0

Have fun, and let me know if that post was helpful to you :)

The reason is that windows software that adds meta tags to music files uses the default russian 8-bit encoding CP1251. All ID3 versions except for the newest ones only allow ISO-8859-1 as the tag encoding. So muine, according to the standard, interprets the tags in ISO-8859-1. Let’s change that.

I’m using Ubuntu 6.10. Let’s have a look at the muine sources:

~/src/deb$ apt-get install build-essential
~/src/deb$ apt-get source muine
...
dpkg-source: extracting muine in muine-0.8.5
dpkg-source: unpacking muine_0.8.5.orig.tar.gz
dpkg-source: applying ./muine_0.8.5-1ubuntu4.diff.gz
~/src/deb$ cd muine-0.8.5/
~/src/deb/muine-0.8.5$ ls src/
...
AddWindowEntry.cs             DndUtils.cs      Metadata.cs          SkipToWindow.cs
...

The file Metadata.cs looks like it’s responsible for the ID3 tags. Searching it for title shows the following lines:

                // Properties :: Title (get;)
                [DllImport ("libmuine")]
                private static extern IntPtr metadata_get_title (IntPtr metadata);

DllImport imports a binary library file. The next line declares a function metadata_get_title which is implemented in libmuine. Let’s look at that.

~/src/deb/muine-0.8.5$ ls libmuine/
...
gsequence.c  metadata.c   player-gst-0.8.c  rb-cell-renderer-pixbuf.c
...

Searching for title in metadata.c gives us the following line:

        metadata->title = get_mp3_comment_value (tag, ID3_FRAME_TITLE, 0);

Which leads us to get_mp3_comment_value. Let’s look at its definition:

get_mp3_comment_value (struct id3_tag *tag,
                       const char *field_name,
                       int index)
{
...
        frame = id3_tag_findframe (tag, field_name, 0);
...
        field = id3_frame_field (frame, 1);
...
        ucs4 = id3_field_getstrings (field, index);
...
        utf8 = id3_ucs4_utf8duplicate (ucs4);
...
}

get_mp3_comment_value calls a lot of functions the name of which starts with id3. The functions are not defined in metadata.c. They aren’t defined anywhere in the muine source code:

~/src/deb/muine-0.8.5$ grep -r id3_field_getstrings .
./libmuine/metadata.c:  latin1 = id3_ucs4_latin1duplicate (id3_field_getstrings (field, 0));
./libmuine/metadata.c:  ucs4 = id3_field_getstrings (field, index);

Only calls to that function. In metadata.c there’s an include statement that includes id3tag.h. Looks like what we need. Let’s download the source for the corresponding library:

~/src/deb/muine-0.8.5$ apt-cache search id3tag
libid3tag0 - ID3 tag reading library from the MAD project
libid3tag0-dev - ID3 tag reading library from the MAD project
mp3rename - Rename mp3 files based on id3tags
somaplayer - player audio for the soma suite
~/src/deb/muine-0.8.5$ cd ..
~/src/deb$ apt-get source libid3tag0
...
dpkg-source: extracting libid3tag in libid3tag-0.15.1b
dpkg-source: unpacking libid3tag_0.15.1b.orig.tar.gz
dpkg-source: applying ./libid3tag_0.15.1b-8.diff.gz
~/src/deb$ cd libid3tag-0.15.1b/
~/src/deb/libid3tag-0.15.1b$ grep -r id3_field_getstrings .
...
./field.c:id3_ucs4_t const *id3_field_getstrings(union id3_field const *field,
...

The function is defined in field.c. It accesses an array stringlist. That array is filled in the function id3_field_parse. This function calls another function, id3_parse_string that extracts the string values of a field.

~/src/deb/libid3tag-0.15.1b$ grep -r id3_parse_string *
parse.c:id3_ucs4_t *id3_parse_string(id3_byte_t const **ptr, id3_length_t length,
parse.h:id3_ucs4_t *id3_parse_string(id3_byte_t const **, id3_length_t,

This function is defined in parse.c. For ISO-8859-1 fields it calls id3_latin1_deserialize.

~/src/deb/libid3tag-0.15.1b$ grep -r id3_latin1_deserialize *
latin1.c:id3_ucs4_t *id3_latin1_deserialize(id3_byte_t const **ptr, id3_length_t length)
...

id3_latin1_deserialize is defined in latin1.c. It calls id3_latin1_decode to convert the latin1 string to UCS-4, which in turn calls id3_latin1_decodechar to convert a single character. We’re there: we have found the place we have to change:

/*
 * NAME:        latin1->decodechar()
 * DESCRIPTION: decode a (single) latin1 char into a single ucs4 char
 */
id3_length_t id3_latin1_decodechar(id3_latin1_t const *latin1,
                                   id3_ucs4_t *ucs4)
{
  *ucs4 = *latin1;

  return 1;
}

The function is very simple: ISO-8859-1 is a subset of unicode, so only a direct assignment is needed. For CP1251 things are different. Looking at the wikipedia page for CP1251, we see that the letters of the russian alphabet start at 0xC0 with the upper case letters, followed by the lower case letters to 0xFF. Using gnome-character-map, we find that the corresponding unicode code points are U+0410 through U+044F and that the letters are in the same order. Very convenient. Let’s change the function to return the correct unicode values for the CP1251 letters 0xC0 through 0xFF:

id3_length_t id3_latin1_decodechar(id3_latin1_t const *latin1,
				   id3_ucs4_t *ucs4)
{
  if (*latin1 >= 0xc0)
    *ucs4 = 0x410 + (*latin1 - 0xc0);
  else
    *ucs4 = *latin1;

  return 1;
}

The unicode encoding used here, UCS-4, just packs the unicode code point in a 32 bit integer, so we can just directly assign the unicode value. Now on to compiling the changed libid3tag.

~/src/deb/libid3tag-0.15.1b$ sudo apt-get build-dep libid3tag0
...
~/src/deb/libid3tag-0.15.1b$ sudo apt-get install fakeroot
...
~/src/deb/libid3tag-0.15.1b$ fakeroot dpkg-buildpackage -uc -us
...
~/src/deb/libid3tag-0.15.1b$ sudo dpkg -i ../libid3tag0_0.15.1b-8_i386.deb
...

Now let’s delete the muine song database so that it re-reads the metadata.

~/src/deb/libid3tag-0.15.1b$ rm ~/.gnome2/muine/*

Start muine and import the music file:

Победа! :)

Update: Added install of build-essential at the beginning.

I can generally recommend Marius Gdeminas’ blog, I found quite a lot of interesting things in there, including a link to his GTimeLog, Bash prompts: showing the command in the window title, xsel: the power of Unix, …

svn: Won't delete locally modified directory '.'
svn: Left locally modified or unversioned files

svn status looks weird after such a failed switch:

markus@markus:/var/www/community$ svn status
!      .
    S  app
!   S  admin
!      admin/tmp
?      admin/tmp/cache/models
!      admin/tmp/cache
    S  sql
...

What’s going on? Let’s assume that I checked out a branch and want to switch to the trunk. In the branch a directory admin/tmp/cache/models was added and its svn:ignore was set to *, that is, all files inside that directory are ignored. Now there appeared files inside this directory. I issue the switch to branch command and got the above error message. That means, that there were some files inside models that are ignored by subversion. The switch operation deleted the .svn directory in admin/tmp/cache/models but didn’t remove the directory itself because it won’t delete files that it doesn’t control – specificially the ignored files in admin/tmp/cache/models. How to get out of that situation?

At first, switch back to where you were, i.e. to the branch. This will give you the following error:

...
svn: Failed to add directory 'admin/tmp/cache/models': object of the same name already exists

svn status looks much better now:

markus@markus:/var/www/community$ svn status
!      .
!      admin
!      admin/tmp
?      admin/tmp/cache/models
!      admin/tmp/cache

Now delete the unversioned models directory and switch again to the branch:

...
A    admin/tmp/cache/models
...

Now delete all ignored files by hand. To find out which files exist that are ignored, use svn status --no-ignore. Note that ignored files interfere with the switch only if they are in a directory that exists only in the branch. Now the switch to the trunk should work without a glitch.

What do you need to do to have your database and web application be all UTF-8? MySQL offers a lot of places to configure character set (character encoding) and character collation (used for sorting and comparing text).

The most important rule is: Don’t rely on server configuration – you may not control it on the server you application will be running. When you write a web application that needs to run on a variety of operating systems and Linux distributions, all with their own default database configuration, you must make as little assumptions as possible about the system it will run on and its configuration. You may have configured your MySQL server to run your application correctly, but you may not have permission or the opportunity to reconfigure the MySQL server your application will run on. Fortunately you can specify all character set configuration in MySQL in places that you control: in the SQL scripts and the source code of your web application.

Stored data

One side of the problem is the data in the tables. If you control the CREATE DATABASE statement, you should specify the character set there:

CREATE DATABASE webapp
        DEFAULT CHARACTER SET utf8;

On some web hosts, web applications have to use the one database that came with the hosting. The database has already been created for you. In that case you have to specify the character set in the individual CREATE TABLE statements:

CREATE TABLE gadgets (
    name VARCHAR(255) PRIMARY KEY,
    rating INT
) DEFAULT CHARACTER SET utf8;

It doesn’t hurt to specify the character set in both locations. (Except you’re violating the DRY Principle.) Now MySQL knows that the data in your tables is in UTF-8.

Communication with the database

The other side of the problem is the data that comes from and gets sent to the client. MySQL offers a lot of features here; you can have different character sets at almost every stage of data processing. To be all UTF-8, issue the following statement just after you’ve made the connection to the database server:

SET NAMES utf8;

This sets the character_set_client, character_set_connection and character_set_results variables to utf8. See below for the meaning of each of these variables.

Communication with the database also concerns SQL files you read with the MySQL command line client, or upload with phpMyAdmin. Put the statement at the top of every SQL file, like this:

SET NAMES utf8;
INSERT INTO TABLE gadgets (name, rating) VALUES ('iPod', 45);

If you’re talking to mysql from a command line that doesn’t understand UTF-8 (likely in Windows and older Linuxes), use the following statement to tell MySQL which character set you’re using on the client side:

SET CHARACTER SET cp1250;

This sets the character_set_client and character_set_results variables to cp1250. Upon arriving on the server, your data will be converted from CP1250 to UTF-8. Results returned to you will be converted from UTF-8 to CP1250.

Which character set do you need for the command line?

utf8

Modern Linux (Fedora, Ubuntu since 5.04, recent SuSE, recent Mandriva) for all languages. No need to issue a SET CHARACTER SET command here.

latin1

Western European Windows and Linux. English, Spanish, German, French, …

cp1250

Central European Windows. Polish, Czech, Slovak, Hungarian, Slovene, Croatian, Romanian, Albanian.

latin2

Central European Linux. Polish, Czech, Slovak, Hungarian, Slovene, Croatian, Romanian, Albanian.

cp1251

Cyrillic Windows. Russian, Ukrainian, Bulgarian, Belorussian, …

koi8r

Russian Linux.

koi8u

Ukrainian Linux.

cp1256

Arabic Windows.

cp1257

Baltic Windows. Estonian, Latvian, Lithuanian.

latin7

Baltic Linux. Estonian, Latvian, Lithuanian.

latin5

Turkish Linux.

Character set and collation variables

What do these character_set_* variables mean?

Variables concerning communication

character_set_client

This informs MySQL about the character set data from the client is encoded in.

character_set_connection

This is the character set MySQL converts incoming data to. It converts from character_set_client.

character_set_result

This is the character set MySQL converts outgoing data to. It converts from the character set specified for the data in the database / tables / columns.

Variables concerning data storage

character_set_server

The character set used for new databases if none is specified in the CREATE DATABASE statement. It can be set from the server configuration file, on the command line for mysqld and interactively in a database session.

character_set_database

The character set used for new tables if none is specified in the CREATE TABLE statement. It is set to character_set_server by default.

character_set_system

The character set for meta-data: database, table and column names. Its value is always utf8.

Feedback

Please leave a comment if this information was helpful to you. Also don’t hesitate to ask questions concerning the above. I’m not the best writer and some things are probably phrased incomprehensibly.

Last but not least I’m always interested in other topics concerning web programming that I could write about. What are you most interested in? What information do you need most?

Links

• Unicode and UTF-8 FAQ for Unix/Linux by Markus Kuhn

to

With metacity and GNOME, you can set for example Shift-Alt-Enter to toggle full screen mode. This is similar to Alt-Enter for toggling maximized mode, so it’s easy to remember:

The idea is to remember the text not word by word, but to remember just the key words in each line and build a mental story with these words. The story doesn’t have to be convincing or anything; it’s just a tool to help you remember. Do try this at home! I’ll offer my key words and my mental story for part of this song. I only had time for two stanzas:

(I won’t reproduce the whole text here as it’s probably copyrighted.)

highway
hair
colitas
rising up
distance
shimmering light
head heavy
night

doorway
mission bell
thinking
heaven hell
candle
show way
voices
heard

(refrain)

tiffany
mercedes
pretty
friends
dance
sweet
remember
forget

captain
wine
spirits
1969
voices
night
say

I wrote these words down from memory, about 14 hours after coming up with them in the marshrutka on my way to work. That took me about 15 minutes. Do repeat the text without looking at it after coming up with your words / story. After work I could recall everything. I now have more trouble remembering the 8 accords that you have to play in a stanza than the text :)

The scene (mental story) I imagine for second stanza goes like this: It’s a warm sunny day, some rich women went by car to a street cafe where she sits now. Because she’s rich, pretty guys desire her. She calls them friends, although these guys are only after her money. They dance for her in the courtyard, she finds that sweet and they sweat. It’s summer. Some of the guys dance well (to remember), some not so well (to forget). Then the first-person narrator calls the waiter (the captain is the waiter in my story) to ask him to bring him his wine. Wine contains alcohol — spirits. The waiter tells his story.

I don’t have a story for the last four lines, as they are pretty easy to remember by itself. The words were enough.

I encourage you to try this at home :)

This is buggy, but I’ll fix it later. It’s also racy, but who cares, I don’t understand why, anyway. Let’s just hope that this will never happen. Or ?!?!

You can also search for buffer overflows.

• Install hugs
• Create an empty file foo.hs
• start hugs

markus@katerina2:~/src/haskell$ touch foo.hs
markus@katerina2:~/src/haskell$ hugs
__   __ __  __  ____   ___      _________________________________________
||   || ||  || ||  || ||__      Hugs 98: Based on the Haskell 98 standard
||___|| ||__|| ||__||  __||     Copyright (c) 1994-2005
||---||         ___||           World Wide Web: http://haskell.org/hugs
||   ||                         Report bugs to: hugs-bugs@haskell.org
||   || Version: 20050308       _________________________________________

Haskell 98 mode: Restart with command line option -98 to enable extensions

Type :? for help
Hugs.Base> :load foo.hs
Main> :edit

:edit calls the editor with the last loaded file – here foo.hs. Put the following in the file:

times 1 x = x
times n x = times (n-1) x ++ x

This defines a function times which takes two arguments: one from which 1 can be subtracted and another one that can be concatenated with itself. Haskell infers this from how you use the parameters in the function. times concatenates its second argument n times with itself.

Main> times 2 [3]
[3,3]
Main> times 2 [42]
[42,42]

Now in order to produce the seg fault (or core dump) in hugs, you have to call times with a nonpositive number of times:

Main> times 0 [42]
Segmentation fault

Nothing earth-shattering, but I find it interesting to have found a possibility to crash Haskell in half an hour after reading about its core dump immunity.

This is meant humoristically, not as a rant. Take it with a grain of salt :)

Gossip has been my favourite Jabber client for a long time. I like the simplicity and non-intrusiveness of its interface. But at work the standard IM network is ICQ. I’ve been using Gaim to connect to ICQ, but its interface has always bugged me.

The Jabber protocol provides for so-called transports, which enable you to talk to other IM networks while using Jabber on the client side. Gossip doesn’t support transports. But although that sounds like a deal-breaker, it isn’t. Transports work like this: you register on a jabber server and then sign up for a transport, providing your ICQ credentials. Then your ICQ contacts are available through UIN@transport.jabber.example.com, for example. After signing up for the transport using a transport-supporting client, you can use any Jabber client you like, and that includes Gossip. All your ICQ contacts will appear on the new account, and you can add other ICQ accounts using UIN@transport.jabber.example.com.

Now step-by-step for Ubuntu Dapper users:

1. Shut down Gossip.
2. Install gajim:
   1. Applications -> Add/Remove Software
   2. check “show applications without tech support”
   3. in the search field enter “gajim”
   4. check gajim and click OK
3. Start gajim through the Applications menu. It’s in “Internet”.
4. Add your existing jabber account to gajim.
5. Add a new account on a server that has a transport for ICQ (or whatever you need). I used jabber.freenet.de. This server has transports for AIM/ICQ, MSN, Yahoo, ICQv2 and IRC. You can see a list of accounts in Edit -> Accounts -> New -> Forward.

   Use the “Discover services” window to find out which transports are offered by a specific server.

   

   You can have both windows open at the same time.

6. Close the “Discover services” window and open it with your new jabber account.
7. Select the transport you want and click “Edit” or “Register”. You’ll be asked to provide the credentials for your ICQ/AIM/whatever account.
8. Close gajim.
9. Start gossip and add your account on the new server.

That’s it! All your ICQ contacts appear in Gossip. Add new ICQ contacts using UIN@icq.jabber.freenet.de (replace icq.jabber.freenet.de with the transport server you selected).

Digg this

• Find a computer, sit down in front of it and play Minesweeper.
• Teleport yourself to a foreign country the language of which you don’t know and try to understand the people, talk to them and write in their language.

Playing Minesweeper is especially interesting: You play against yourself. Or rather your awareness observes how your ego in a dream plays against a computer that exists only in your ego’s dream. Try to catch your brain on making a logical mistake while you open more and more fields on the mine field.

I’ll add to this list as I think of more challenges. Leave your ideas for challenges in lucid dreams in the comments.

I was walking by a bus or tram stop and saw a transport network plan. I decided to read it. As it turns out, I was in Cottbus, a middle-sized town in East Germany. I have never been to Cottbus in real life. The network plan contained all the stations the buses, or trams, in Cottbus stop at. Suddenly I had an idea: I wanted to challenge my brain, challenge the completeness and correctness of its imagination. I tried to read the station names, with the intent to later, when I was awake, find them on the real transport network plan of Cottbus.

As it stands, my mind couldn’t lie to my awareness: I plainly wasn’t able to read the text. I saw lines that resembled something that could be text in some other language, but it weren’t latin letters. (The text “Cottbus” was written readably with latin letters.) Everytime I looked at one specific bus stop label, it seemed to hav changed from the last time. For this reason I had problems remembering a specific label. I do remember though how the script generally looked like: It contained strokes, mainly from top to bottom and diagonal ones, and arcs like in C or ). These elements were less connected that latin letters; in a word of the length of 10 latin letters there were maybe 20 such strokes and arcs.

Why did my mind allow me to read the plan in the first place? As I’ve never been to Cottbus and have never seen the plan of public transport, it couldn’t possibly show me the real plan. I knew that, as well as my mind. (That sentence sounds a little insane, so let’s rephrase it: My awareness knew that as well as the mind of my physical ego, which was dreaming at the time. Read more about you, awareness and ego on Steve Pavlinas web log.)

On the time line in the dream, I first saw the plan and only then decided to take a detailed look at it, so it was too late for my mind to not let me see it. And I hadn’t thought of the challenge yet. So I didn’t expect to see something that I will later compare with reality. Before looking at the station labels though, I already thought of the challenge. And my mind knew that I was specifically after tricking it. So it decided to do something about it. After all this is a dream, and in a dream it’s possible for text to be just not readable. In the end my mind tricked my awareness.

Selenium, a framework that supports functional unit testing of web applications, simulates a user clicking links, filling out forms and submitting them. It can determine whether a blog post appeared in a list, after you used a form to submit it. It can test whether your AJAX screenname availability check works. It supports cookies. And all that because it runs directly in your browser. The Javascript that runs in your browser receives commands like “click the submit button” or “wait for the spinner to appear” or “make sure this error message didn’t appear on the page”.

Selenium comes in several flavours: one is Selenium Core, which runs directly and only in the browser. As a language for writing tests it supports only selenese, which can express only a simple sequence of actions and tests. This is of course too little for most web applications. But there’s a flavour of Selenium that can be integrated with traditional unit testing frameworks like SimpleTest (for PHP) and PyUnit: Selenium Remote Control.

Selenium Remote Control is essentially a server written in Java with two functions: it acts as a proxy server for the browser and takes commands via HTTP that will be executed in the browser. The proxy server is needed because browsers rightfully don’t allow Javascript from one site to run on another site. Read Selenium Remote Control: The Same Origin Policy if you’d like to know more.

Selenium Remote Control comes with client drivers for Python, Ruby, Perl and Java – a PHP client driver has not made it in the official release yet.

To start using Selenium Remote Control for doing functional tests of your web application, download the current release (http://openqa.org/selenium-rc/download.action), unpack it and start the server:

wget selenium-remote-control-0.8.1.zip
unzip selenium-remote-control-0.8.1.zip
cd selenium-remote-control-0.8.1/server
java -jar selenium-server.jar

This starts the two servers: the proxy server for the browser and the server that takes the commands. Then write a unit test case, using one of the client drivers provided in the archive. I’ll use Python here:

mkdir functional-tests
cd functional-tests
cp path/to/selenium-remote-control-0.8.1/python/selenium.py .

and put this in a file test-login.py

import unittest

class LoginTest(unittest.TestCase):
seleniumHost = 'localhost'
seleniumPort = '4444'
browserStartCommand = '*firefox /usr/lib/firefox/firefox-bin'
browserURL = 'http://community'

def setUp(self):
self.selenium = selenium('localhost', '4444', '*firefox /usr/lib/firefox/firefox-bin', 'http://www.example.com')
self.selenium.start()

def tearDown(self):
self.selenium.stop()

def testLogin(self):
self.selenium.open("/login")
self.selenium.type("name=username", "john")
self.selenium.type("name=password", "johnspassword")
self.selenium.click("id=Submit")
self.selenium.wait_for_page_to_load(15000)
self.failUnless(self.selenium.is_text_present("Login successful"))

if __name__ == '__main__':
unittest.main()

Then run

python ./test-login.py

and watch selenium opening a Firefox instance for you, loading the login page, typing username and password, submitting the form and finally checking for the message about successful login. After that Firefox is closed and PyUnit reports that the test ran successfully.

Links

• Practical Testing PHP Applications with Selenium
• Automate acceptance tests with Selenium: A detailed tutorial with real life examples
• Web Testing Your Apps with Selenium-Core