Archive for the ‘git’ Category

Cidadania has its own GitHub account

It is not very professional to have all the code repositories of a compny in your personal profile, even if the code is open source.

So, again I’m extending the cidadania infrastructure making a global account in GitHub. You can access here: http://github.com/cidadania and it’s own page at github (still working on it) http://cidadania.github.com

Starting development of Supervise 2

If you remember, a few months ago I started a project called “Supervise”. It was a project management tool based on django, aimed to be the next “Redmine”, but for some personal reasons it had to be stopped (I’ve got a job!). Some new releases of django were made, and with them, hot new features were added.

The Supervise project was in an early stage of devlopment, so I said myself: “Why not start from scratch? It’s not that difficult”. So here it is: Supervise 2, a new version made on top of django 1.3. The list of features will be much the same as Supervise:

  • Multiple project management like Redmine.
  • Various SCM support: GIT, Bazaar, SVN, Mercurial. There will be more supported SCMs but for now that covers most of the developers needs.
  • Issue tracking
  • Wiki
  • News
  • User profiles

That’s for version 0.1-stable (which will come in a couple months I think)

For now you can watch the development or fork the code in both Gitorious and GitHub

Installing django 1.3 on DreamHost

UPDATE: I’ve rewrited the post since the instructions are much easier than I did put here (and more secure)

You will probably say: “Hey!, what’s in your mind to need django 1.3, it has been released just a week ago!”. The answer is: long-term development. Currently I’m working on e-cidadania, an e-democracy tool designed for participative processes. The development will take some time, and we’ve just started. It’s better to update everything now than later, but to the point, installing django 1.3 in dreamhost.

NOTE: I assume you know how to install a django project in DH. If not, please see this wiki

Let’s start.

STEP 1: CREATE A VIRTUALENV

You can create a virtualenv with the system-wide python installation (currently 2.5.2) or install your own python interpreter if you need it. We will use the system-wide python for convenience, since it’s still supported by django 1.3.

This part is extracted from here

NOTE: These commands must be typed at your home directory

$ wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.5.2.tar.gz
$ tar xzf virtualenv-1.5.2.tar.gz
$ python virtualenv-1.5.2/virtualenv.py $HOME/env

This will create a directory called “env” with all the stuff you need to work. Let’s do this virtual environment our default.

STEP 2: STABLISH THE VIRTUALENV PYTHON AS DEFAULT

The following instructions about stablishing the virtual python as default only work at shell level. For passenger the system-wide python is the default.

On ~/.bash_profile write:

export PATH=$PATH:$HOME/env/bin

This will stablish the directory ~/env/bin (where our virtual python is) before the system-wide python. To work right now without logging in again, we can do:

$ source .bash_profile

That will make the executables in ~/env/bin available for using right now, and between them, it’s easy_install, properly configured to install any python module locally without problem.

STEP 3: INSTALL DJANGO 1.3 AND ANY OTHER PACKAGES

Let’s install django 1.3 and some common python packages needed in almost every project:

$ easy_install -U django

For the moment we will install just django, because the rest of python packages are provided by the DH server. If you need non standard modules (p.e. python-dateutil) or other version of an already provided package, you must install them after django.

If you try to execute some django project right now you’ll see that the system-wide installation of django is the preferred. Let’s change that.

STEP 4: CONFIGURING PASSENGER_WSGI.PY

In our passenger_wsgi.py we must modify the PYTHONPATH, let’s do it:

import sys, os
 
# Python has no prepend function, so we must do an insert. We insert the key directories
# BEFORE ANYTHING, so the python installation sees them before the system-wide libraries.
 
sys.path.insert(0,'/home/USERNAME/env/bin')
sys.path.insert(0,'/home/USERNAME/env/lib/python2.5/site-packages/Django-1.3-py2.5.egg')
sys.path.insert(0,'/home/USERNAME/env/lib/python2.5/site-packages')
 
os.environ['DJANGO_SETTINGS_MODULE'] = "YOURPROJECT.settings"
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Remember that if any of your modules fails, python will not fall back to the system-wide installation. Once a module is loaded, python does not seek another. Also, if you installed another package apart from django, you must do an insert including it.

That’s it! Now when you receive a visitor, Passenger will load you desired django version and packages.

Another tool: pyce

This is maybe one of the most useless and handy tools that I will ever make, a password/passphrase generator for non critical environments.

It’s based on the Diceware method and works like a charm for generating passphrases.

It’s a 15 minutes implementation that will generate passphrases for you using a file named “wordlist.txt” in the format that diceware website has in it’s files.

Download it here (has the spanish word list):

https://gitorious.org/misc-tools/pyce

“Guess the film” revisited

Yes, I know I posted the other entry last night, but after that I kept messing around with the code. One of my friend requests was to have a GUI (seriously?). Ok, after thinking about it, and modifying some code here is the version 0.7 of this magnificent and almighty implementation of guess the film.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim:fenc=utf-8
#
# Copyright (c) 2010 Oscar Carballal Prego <info@oscarcp.com>
#
# Distributed under terms of the MIT license.
 
# TODO:
# - Solve the codification problem in Windows.
# - Improve the films list to have a director and description.
 
# FIXME:
# - _get_film(self) does not print 5 spaces before text in format string
 
# Import python modules
import sys
import random
import datetime
 
 
class Game():
 
    def __init__(self):
        """
        Start everything. Declare dictionaries and lists, open files and get
        all the content ready.
        """
        self.films_file = open('films.txt', 'r')
        self.films = []
        self.players = {}
 
        [self.films.append([line]) for line in self.films_file]
        self.films_file.close()
 
    def _end(self):
        """
        This function terminates the game, after showing a basic statistic of
        the game.
        """
        self.end_time = datetime.datetime.now()
        self.t = str((self.end_time - self.start_time)).split(':')
        self.counter = len(self.players) - 1
        self.p = self.players.items()
        self.fwinner = max(self.players, key = lambda a: self.players.get(a))
        print "\n\n In a game of {0} players, results are:\n".format(len(self.players))
        print " Time: {0} hours, {1} minutes, {2} seconds.\n".format(self.t[0],
                                                                     self.t[1],
                                                                     self.t[2].split('.')[0])
        print " Points\n ------\n"
        while self.counter >= 0:
            print " {0} won {1} points.".format(self.p[self.counter][0],
                                                self.p[self.counter][1])
            self.counter -= 1
        print "\n ******* Winner: {0} *******\n".format(self.fwinner)
        # Clear everything
        del self.films[:]
        self.players.clear()
        sys.exit(0)
 
    def _get_players(self):
        """
        This function asks the number of players and their names. Names are
        stored in a dictionary as a key with a value of zero for every player.
        """
        self.i = 0
        self.check = 0
        while self.check == 0:
            try:
                self.n_players = int(raw_input('* How many players? '))
                self.check = 1
            except ValueError:
                print '* ERROR: Must type a number!'
        if self.n_players == 1:
            print '* Are you kidding? This can\'t be played alone!.'
            sys.exit(0)
        while self.i < self.n_players:
            self.player_name = raw_input('Player {0} name: '.format(self.i))
            self.players[self.player_name] = 0
            self.i += 1
 
    def _get_points(self):
        """
        This function stores the points earned by a player. It simply increases
        the number of the value by one in the pair [key,value].
        """
        try:
            self.winner = raw_input('* Who guessed? (player name, case sensitive): ')
            self.players[self.winner] += 1
        except:
            self._get_points()
 
    def _get_film(self):
        """
        This function gets a random film from the film list.
        """
        print '\n {0:>5}'.format(random.choice(self.films)[0])
 
    def start(self):
        """
        This function starts the game. It gets the time when the game is
        started and enters a loop asking to generate a new film. If the player
        does not want a new film _end() is executed.
        """
        self._get_players()
        self.start_time = datetime.datetime.now()
        while True:
            self.generate = raw_input('* Generate film? (s/n) ')
            if self.generate == 'n' or self.generate == 'no':
                self._end()
            else:
                self._get_film()
                self._get_points()
 
if __name__ == '__main__':
    game = Game()
    game.start()

Migrating from bzr to git + cgit + nginx

Well, after some months of “I don’t know… I just do small projects… git is too much for me” I decided to migrate from Bazaar (bzr) to git, the massively accepted and widespread VCS.

I already used git for grabbing code from the net, but never for my own projects. First of all was to test if I could migrate all the bzr repos to git, and it was quite easy!

I grabbed a copy of fast-export (comes with fast-import) into the bzr plugins directory:

mkdir -p ~/.bazaar/plugins
cd ~/.bazaar/plugins
bzr co lp:bzr-fastimport fastimport

After that it’s time to copy the repository:

git init new_git_project_folder
cd new_git_project_folder
bzr fast-export --plain ~/path/to/bzr/branch | git fast-import
git checkout master

Now it’s time for the real work, installing nginx, fastcgi, cgit and “configure ‘em all!

In gentoo it was quite easy, just do:

emerge -auND fcgiwrap spawn-fcgi nginx cgit

Be careful, for this you will need to unmask packages (cgit) and activate some USE flags like “nginx_modules_http_fastcgi” for nginx. I leave that work up to you, after all is not that difficult.

In my use case, nginx and the cgi will be on the same server, and that allows me to ommit a few fastcgi options. In this example I assume that your server root is /var/www/localhost.

First, configure the spawn-fcgi:

ln -s spawn-fcgi /etc/init.d/spawn-fcgi.cgit
cp /etc/conf.d/spawn-fcgi /etc/conf.d/spawn-fcgi.cgit

Edit /etc/conf.d/spawn-fcgi.cgit to look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Since I have everything on the same machine, I'll just use a socket
FCGI_SOCKET=/var/run/spawn-fcgi/localhost_spawn-fcgi
 
# We set this to nothing because if they're deleted, spawn-fcgi defaults them
# to their initial values.
FCGI_ADDRESS=
FCGI_PORT=
 
# Tell spawn-fcgi where is fcgiwrap (check this, it can be on another location)
FCGI_PROGRAM=/usr/sbin/fcgiwrap
 
FCGI_CHILDREN=1
FCGI_CHROOT=
FCGI_CHDIR=
 
# Defaults to root, so I change it to nginx
FCGI_USER=nginx
FCGI_GROUP=nginx
 
ALLOWED_ENV="PATH"

Copy cgit to the web root. Since gentoo’s webapp-config does not support nginx you’ll have to do this by hand:

cp /usr/share/webapps/cgit/0.8.3.1-r1/hostroot/cgi-bin/cgit.cgi /var/www/localhost/htdocs
cp /usr/share/webapps/cgit/0.8.3.1-r1/htdocs/cgit.css /var/www/localhost/htdocs
cp /usr/share/webapps/cgit/0.8.3.1-r1/htdocs/cgit.png /var/www/localhost/htdocs

We must now configure cgit to look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
virtual-root=/
cache-size=1000
css=/cgit.css
enable-index-links=1
enable-log-filecount=1
enable-log-linecount=1
logo=/cgit.png
root-title=Silvana Git Browser
root-desc=It is all done by electricity, my dear Arronax, by electricity! - Capt. Nemo
cache-root=/var/cache/cgit
cache-root-ttl=0
cache-repo-ttl=0
cache-dynamic-ttl=0
cache-static-ttl=0
snapshots=tar.gz tar.bz2
# As recommended by cgit, we put the repositories in another file
include=/etc/cgitrc.local

Let’s create the repositories file /etc/cgitrc.local:

1
2
3
4
5
6
7
8
# List of git repositories
section=code # Not necessary but very useful to classify repos
repo.url=gitproject # The url in the web browser
repo.desc=The description 
repo.path=/path/to/your/repo
repo.owner=Your name or email
# I don't use this but in this field you will write the repo url to grab the code 
# repo.clone-url=

Finally, configure the nginx server. Add this line to the HTTP section of /etc/nginx/nginx.conf

1
2
# Use up to 10 Megabytes for the 'code' cache, keep inactive cache around for 1h
fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=code:10m inactive=1h max_size=100m;

And now create this server section:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
server {
    listen 80;
    server_name localhost;
 
    # Serve static files
    location ~* ^.+\.(css|png|ico)$ {
        root /var/www/localhost/htdocs;
        expires 30d;
    }
 
    location / {
        # I tried with some rewrites and only this one worked fine
        rewrite ^/([^?/]+/[^?]*)?(?:\?(.*))?$ /cgit.cgi?url=$1&$2 last;
 
        fastcgi_cache      code;
        fastcgi_cache_valid 200 5m;
        fastcgi_cache_use_stale off;
 
        fastcgi_pass            unix:/var/run/spawn-fcgi/localhost_spawn-fcgi-1;
        fastcgi_read_timeout    5m;
        fastcgi_index   /;
 
        fastcgi_param    DOCUMENT_ROOT    /var/www/localhost/htdocs;
        fastcgi_param    SCRIPT_FILENAME  /var/www/localhost/htdocs/cgit.cgi;
        fastcgi_param    QUERY_STRING  $query_string;
        fastcgi_param    REQUEST_METHOD  $request_method;
        fastcgi_param    CONTENT_TYPE  $content_type;
        fastcgi_param    CONTENT_LENGTH  $content_length;
        fastcgi_param    GATEWAY_INTERFACE  CGI/1.1;
        fastcgi_param    SERVER_SOFTWARE  nginx;
        fastcgi_param    SCRIPT_NAME  $fastcgi_script_name;
        fastcgi_param    REQUEST_URI  $request_uri;
        fastcgi_param    DOCUMENT_URI  $document_uri;
        fastcgi_param    DOCUMENT_ROOT  $document_root;
        fastcgi_param    SERVER_PROTOCOL  $server_protocol;
        fastcgi_param    REMOTE_ADDR  $remote_addr;
        fastcgi_param    REMOTE_PORT  $remote_port;
        fastcgi_param    SERVER_ADDR  $server_addr;
        fastcgi_param    SERVER_PORT  $server_port;
        fastcgi_param    SERVER_NAME  $server_name;
    }
 
    access_log /var/log/nginx/myrepos_access_log combined;
    error_log /var/log/nginx/myrepos_error_log warn;
}

Hope this article will help you. It is a mixup of some articles I found that helped me, specially the “Running cgit under nginx” article by Russell