Archive for the ‘gnu-linux’ Category

Installing django 1.3 on DreamHost

UPDATE: I’ve rewrited the post since the instructions are much easier than I did put here (and more secure)

You will probably say: “Hey!, what’s in your mind to need django 1.3, it has been released just a week ago!”. The answer is: long-term development. Currently I’m working on e-cidadania, an e-democracy tool designed for participative processes. The development will take some time, and we’ve just started. It’s better to update everything now than later, but to the point, installing django 1.3 in dreamhost.

NOTE: I assume you know how to install a django project in DH. If not, please see this wiki

Let’s start.

STEP 1: CREATE A VIRTUALENV

You can create a virtualenv with the system-wide python installation (currently 2.5.2) or install your own python interpreter if you need it. We will use the system-wide python for convenience, since it’s still supported by django 1.3.

This part is extracted from here

NOTE: These commands must be typed at your home directory

$ wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.5.2.tar.gz
$ tar xzf virtualenv-1.5.2.tar.gz
$ python virtualenv-1.5.2/virtualenv.py $HOME/env

This will create a directory called “env” with all the stuff you need to work. Let’s do this virtual environment our default.

STEP 2: STABLISH THE VIRTUALENV PYTHON AS DEFAULT

The following instructions about stablishing the virtual python as default only work at shell level. For passenger the system-wide python is the default.

On ~/.bash_profile write:

export PATH=$PATH:$HOME/env/bin

This will stablish the directory ~/env/bin (where our virtual python is) before the system-wide python. To work right now without logging in again, we can do:

$ source .bash_profile

That will make the executables in ~/env/bin available for using right now, and between them, it’s easy_install, properly configured to install any python module locally without problem.

STEP 3: INSTALL DJANGO 1.3 AND ANY OTHER PACKAGES

Let’s install django 1.3 and some common python packages needed in almost every project:

$ easy_install -U django

For the moment we will install just django, because the rest of python packages are provided by the DH server. If you need non standard modules (p.e. python-dateutil) or other version of an already provided package, you must install them after django.

If you try to execute some django project right now you’ll see that the system-wide installation of django is the preferred. Let’s change that.

STEP 4: CONFIGURING PASSENGER_WSGI.PY

In our passenger_wsgi.py we must modify the PYTHONPATH, let’s do it:

import sys, os
 
# Python has no prepend function, so we must do an insert. We insert the key directories
# BEFORE ANYTHING, so the python installation sees them before the system-wide libraries.
 
sys.path.insert(0,'/home/USERNAME/env/bin')
sys.path.insert(0,'/home/USERNAME/env/lib/python2.5/site-packages/Django-1.3-py2.5.egg')
sys.path.insert(0,'/home/USERNAME/env/lib/python2.5/site-packages')
 
os.environ['DJANGO_SETTINGS_MODULE'] = "YOURPROJECT.settings"
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Remember that if any of your modules fails, python will not fall back to the system-wide installation. Once a module is loaded, python does not seek another. Also, if you installed another package apart from django, you must do an insert including it.

That’s it! Now when you receive a visitor, Passenger will load you desired django version and packages.

Another tool: pyce

This is maybe one of the most useless and handy tools that I will ever make, a password/passphrase generator for non critical environments.

It’s based on the Diceware method and works like a charm for generating passphrases.

It’s a 15 minutes implementation that will generate passphrases for you using a file named “wordlist.txt” in the format that diceware website has in it’s files.

Download it here (has the spanish word list):

https://gitorious.org/misc-tools/pyce

Migrating from bzr to git + cgit + nginx

Well, after some months of “I don’t know… I just do small projects… git is too much for me” I decided to migrate from Bazaar (bzr) to git, the massively accepted and widespread VCS.

I already used git for grabbing code from the net, but never for my own projects. First of all was to test if I could migrate all the bzr repos to git, and it was quite easy!

I grabbed a copy of fast-export (comes with fast-import) into the bzr plugins directory:

mkdir -p ~/.bazaar/plugins
cd ~/.bazaar/plugins
bzr co lp:bzr-fastimport fastimport

After that it’s time to copy the repository:

git init new_git_project_folder
cd new_git_project_folder
bzr fast-export --plain ~/path/to/bzr/branch | git fast-import
git checkout master

Now it’s time for the real work, installing nginx, fastcgi, cgit and “configure ‘em all!

In gentoo it was quite easy, just do:

emerge -auND fcgiwrap spawn-fcgi nginx cgit

Be careful, for this you will need to unmask packages (cgit) and activate some USE flags like “nginx_modules_http_fastcgi” for nginx. I leave that work up to you, after all is not that difficult.

In my use case, nginx and the cgi will be on the same server, and that allows me to ommit a few fastcgi options. In this example I assume that your server root is /var/www/localhost.

First, configure the spawn-fcgi:

ln -s spawn-fcgi /etc/init.d/spawn-fcgi.cgit
cp /etc/conf.d/spawn-fcgi /etc/conf.d/spawn-fcgi.cgit

Edit /etc/conf.d/spawn-fcgi.cgit to look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Since I have everything on the same machine, I'll just use a socket
FCGI_SOCKET=/var/run/spawn-fcgi/localhost_spawn-fcgi
 
# We set this to nothing because if they're deleted, spawn-fcgi defaults them
# to their initial values.
FCGI_ADDRESS=
FCGI_PORT=
 
# Tell spawn-fcgi where is fcgiwrap (check this, it can be on another location)
FCGI_PROGRAM=/usr/sbin/fcgiwrap
 
FCGI_CHILDREN=1
FCGI_CHROOT=
FCGI_CHDIR=
 
# Defaults to root, so I change it to nginx
FCGI_USER=nginx
FCGI_GROUP=nginx
 
ALLOWED_ENV="PATH"

Copy cgit to the web root. Since gentoo’s webapp-config does not support nginx you’ll have to do this by hand:

cp /usr/share/webapps/cgit/0.8.3.1-r1/hostroot/cgi-bin/cgit.cgi /var/www/localhost/htdocs
cp /usr/share/webapps/cgit/0.8.3.1-r1/htdocs/cgit.css /var/www/localhost/htdocs
cp /usr/share/webapps/cgit/0.8.3.1-r1/htdocs/cgit.png /var/www/localhost/htdocs

We must now configure cgit to look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
virtual-root=/
cache-size=1000
css=/cgit.css
enable-index-links=1
enable-log-filecount=1
enable-log-linecount=1
logo=/cgit.png
root-title=Silvana Git Browser
root-desc=It is all done by electricity, my dear Arronax, by electricity! - Capt. Nemo
cache-root=/var/cache/cgit
cache-root-ttl=0
cache-repo-ttl=0
cache-dynamic-ttl=0
cache-static-ttl=0
snapshots=tar.gz tar.bz2
# As recommended by cgit, we put the repositories in another file
include=/etc/cgitrc.local

Let’s create the repositories file /etc/cgitrc.local:

1
2
3
4
5
6
7
8
# List of git repositories
section=code # Not necessary but very useful to classify repos
repo.url=gitproject # The url in the web browser
repo.desc=The description 
repo.path=/path/to/your/repo
repo.owner=Your name or email
# I don't use this but in this field you will write the repo url to grab the code 
# repo.clone-url=

Finally, configure the nginx server. Add this line to the HTTP section of /etc/nginx/nginx.conf

1
2
# Use up to 10 Megabytes for the 'code' cache, keep inactive cache around for 1h
fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=code:10m inactive=1h max_size=100m;

And now create this server section:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
server {
    listen 80;
    server_name localhost;
 
    # Serve static files
    location ~* ^.+\.(css|png|ico)$ {
        root /var/www/localhost/htdocs;
        expires 30d;
    }
 
    location / {
        # I tried with some rewrites and only this one worked fine
        rewrite ^/([^?/]+/[^?]*)?(?:\?(.*))?$ /cgit.cgi?url=$1&$2 last;
 
        fastcgi_cache      code;
        fastcgi_cache_valid 200 5m;
        fastcgi_cache_use_stale off;
 
        fastcgi_pass            unix:/var/run/spawn-fcgi/localhost_spawn-fcgi-1;
        fastcgi_read_timeout    5m;
        fastcgi_index   /;
 
        fastcgi_param    DOCUMENT_ROOT    /var/www/localhost/htdocs;
        fastcgi_param    SCRIPT_FILENAME  /var/www/localhost/htdocs/cgit.cgi;
        fastcgi_param    QUERY_STRING  $query_string;
        fastcgi_param    REQUEST_METHOD  $request_method;
        fastcgi_param    CONTENT_TYPE  $content_type;
        fastcgi_param    CONTENT_LENGTH  $content_length;
        fastcgi_param    GATEWAY_INTERFACE  CGI/1.1;
        fastcgi_param    SERVER_SOFTWARE  nginx;
        fastcgi_param    SCRIPT_NAME  $fastcgi_script_name;
        fastcgi_param    REQUEST_URI  $request_uri;
        fastcgi_param    DOCUMENT_URI  $document_uri;
        fastcgi_param    DOCUMENT_ROOT  $document_root;
        fastcgi_param    SERVER_PROTOCOL  $server_protocol;
        fastcgi_param    REMOTE_ADDR  $remote_addr;
        fastcgi_param    REMOTE_PORT  $remote_port;
        fastcgi_param    SERVER_ADDR  $server_addr;
        fastcgi_param    SERVER_PORT  $server_port;
        fastcgi_param    SERVER_NAME  $server_name;
    }
 
    access_log /var/log/nginx/myrepos_access_log combined;
    error_log /var/log/nginx/myrepos_error_log warn;
}

Hope this article will help you. It is a mixup of some articles I found that helped me, specially the “Running cgit under nginx” article by Russell