Django+nginx+uwsgi+supervisor

Because I've heard so many people talking about switching from Apache to this combination, I thought I would give it a try. After scouring the Internet and reading some blog posts, I came up with a setup, which I'm documenting here, because there were some gotchas for me.

I use Ubuntu 11.04, so this is the list of required packages:
• python-pip
• python-virtualenv
• postgresql
• postgresql-dev
• python2.7-dev (uwsgi requirement)
• libxml2-dev (uwsgi requirement)
• libpcre3 (nginx requirement)
• libpcre3-dev (nginx requirement)
• nginx

I install nginx with apt-get to have the necessary directory created in /etc/, but I'll replace the binary with the one I get after building the source of the latest version:

wget http://nginx.org/download/nginx-1.0.11.tar.gz
tar -xzf nginx-1.0.11.tar.gz
cd nginx-1.0.11/
./configure --conf-path=/etc/nginx/nginx.conf\
--sbin-path=/usr/sbin\
--without-http-cache
make
make install

The configuration options are so few because this is what I need in development, but for production more should be added.

Next, I just use django-startproject, to create a Django project and start working. My fork of this application does some extra things. First, it assumes very similar development and production environments, so the same stack is also used for development (nginx, uwsgi, supervisor, postgresql). Second, it automates a number of tasks done at the start of each project or on a new deployment. A typical work-flow is like this:
1. Install django-startproject:
    pip install -e git+https://github.com/LucianU/django-startproject#egg=django-startproject

2. Go to the location where you want to create your new project and run:
    django-startproject.py projectname
    At this point, you are prompted for some values and then the setup is done. During the setup, all the placeholders are replaced in the project template with the values corresponding to the current machine and the ones provided at the prompt. Afterwards, a virtual env is created and the packages used for development are installed: Django, psycopg2, supervisor, uwsgi, django-debug-toolbar. I prefer installing uwsgi and supervisor in a virtual env and not globally.

3. Create a database and put your credentials in the project's settings/local.py file.

4. Activate your environment and run django-admin.py syncdb

5. Symlink project/confs/development/nginx.conf into the 'sites-enabled' directory of your nginx installation and restart the server

If there weren't any problems, you should be done and running supervisord should start uwsgi, and you should be able to access the Django greeting page.

django-startproject also offers a script that can be used for deployment. For example, if someone wants to start working on a project created with django-startproject, they just clone the repository, run bin/bootstrap.py dev, and then follow the quickstart from step 3 onwards.

Last, I want to mention some things about nginx's configuration system because this was unclear to me at the start. nginx has a global configuration file (located at /etc/nginx/nginx.conf in case of Ubuntu) and two directories, sites-available and sites-enabled. Generally, in nginx.conf are found settings applying to the server as a whole, like used modules, compression, and so on. On the other hand, the configuration files placed in sites-enabled are specific to the served app, containing the domain name and the location of application and its logs.

Setting the terminal default editing mode

I had the unexpected surprise after I installed 64-bit Ubuntu, to discover that the terminal wasn't set to Emacs editing mode by default. It's not that I'm a big lover of Emacs (I use Vim), but I've gotten used to the Emacs key bindings in the terminal. After some searching in ~/.bashrc and /etc/bash.bashrc, I found that the "problem" came from ~/.inputrc. The first line of that file was 'set editing-mode vi', so I commented it and all was well again.

How to setup wireless for Inspiron N3010 in Ubuntu 11.04

It took me a day and the help of allandee from #ubuntu to be able to setup wireless on my laptop, so I decided to write down the steps and put them here in case I or someone else needs them.

First, the laptop comes with Broadcom Corporation BCM4313 802.11b/g/n Wireless LAN Controller (rev 01), so the instructions are for it.

Here are the steps:

 1. sudo apt-get install git (in case it's not installed)
 2. git clone git://git.bu3sch.de/b43-tools.git
 3. cd b43-tools
 4. make #build the b43-fwcutter binary

 5. cd
 6. mkdir tmp #create a temporary dir for the driver
 7. cd tmp
 8. wget http://mirror2.openwrt.org/sources/broadcom-wl-5.10.56.27.3_mipsel.tar.bz2
 9. tar -xjf broadcom-wl-5.10.56.27.3_mipsel.tar.bz2 #extract the archive
 10. sudo ~/b43-tools/b43-fwcutter -w /libs/firmware/ 
~/tmp/broadcom-wl-5.10.56.27.3_mipsel/driver/wl_apsta/wl_prebuilt.o #extract the firmware
 #the line above is wrapped to properly show up on screen 
 11. modprob b43
 a restart might be needed afterwards.
 

This is it.

Geospatial Applications

I've started learning about the topic, so I'm getting familiar with PostGIS, OpenLayers, Mapnik, and GIS in general. I want to make some blog posts detailing the steps I've gone through when rendering my own tiles for example.
Anyway, I find the topic fascinating because it allows you to visually link statistics with locations on the globe, which I think is very cool. To be continued...

How to upgrade the SQLite that comes with Python

If you're using Python with SQLite and you want to be able to read and write to the database concurrently, you need to enable WAL. The problem is that only SQLite versions 3.7.x support this new functionality, and Python comes with older versions (at least Python 2.6.x does).

So, you need to upgrade SQLite, but fortunately this is surprisingly easy. If you're on Windows, like I am, you should go to the SQLite download page and download the sqlite.dll. Then, you simply take that dll, and put it in Your_Python_Installation\DLLs, where Your_Python_Installation is the folder where Python is installed, in my case Python26.

After you've done that, you can simply do

PRAGMA journal_mode=wal

and that should activate WAL. To be sure, check the return of value of curs.fetchone() which should be (u'wal',).

Installing oursql on Windows

I had some difficulties installing oursql on Windows, so I thought I'd write down the steps, in case I need to do it again in the future or anyone else has the problems.

First, I tried installing with pip, but that gave me some problems (something about a missing file) even after I installed Cython.

On second try, I noticed that there are already some binaries available, so I tried with those. The archive which can be downloaded from here http://www.habnabit.org/software/oursql/ contains an installer which in turn contains an egg and a .pyd file. Those go in the site-packages directory. I'm saying this in case anyone wants to install oursql in a virtualenv, which is what I wanted to do and I couldn't because the installer only detected my main site-packages and wouldn't let me choose.

I put those in their place, but when trying to import oursql, I got a "ImportError: DLL load failed...". Apparently, I needed the MySQL C Connector. I installed that, but I still got the error, which I solved by copying libmySQL.dll from my WAMP package into site-packages(supposedly, this isn't necessary when MySQL has been installed directly).

Ok, to recap. The steps were:
1) Download an archive with the binaries from here http://www.habnabit.org/software/oursql/
2) Extract the archive and also the installer.
3) Put the egg and the .pyd in the site-packages.
4) Install MySQL C Connector
5) Copy libmySQL.dll into site-packages.
6) Done.

Update

I thought I'd come back and write something here because I haven't done so in a month, I think. One of the things I did in the mean time was starting to learn assembly. My main goal is to cover my bases, so to speak, and learn the fundamentals of computers and computer science, and I thought that learning assembly would provide that. I haven't started writing code because the book that I'm learning from (Assembly Language Step-by-Step: Programming with Linux) is big, but I'm close to that point and I'm looking forward to it.

So far I like the book. It starts really slow, which is a bit annoying but to be expected considering it's for people with assembly as their first programming language (I wonder how many people start with assembly), but then goes into a lot of detail and careful explanation in order to make you understand what's going on. It has analogies and a lot of drawings which are so useful for visual learners. That's all I can say for now. I'll add to this once I finish the book.

Oh, and here's something inspiring. A guy who wrote his own compiler, editor, OS and a few others.