Archive for the ‘migration’ Category

Howto generic class-based views in django 1.3

This is a reminder for myself, to know how the new class-based generic views work. The code belongs to the GPL project e-cidadania. I’ll be posting as I code.

DETAILVIEW

The function-based code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
def view_space_index(request, space_name):
 
    """
    Show the index page for the requested space. This is a conglomerate of
    various modules.
    """
    place = get_object_or_404(Space, url=space_name)
 
    extra_context = {
        'entities': Entity.objects.filter(space=place.id),
        'documents': Document.objects.filter(space=place.id),
        'proposals': Proposal.objects.filter(belongs_to=place.id).order_by('-pub_date'),
        'publication': Post.objects.filter(post_space=place.id).order_by('-post_pubdate'),
    }
 
    return object_detail(request,
                         queryset = Space.objects.all(),
                         object_id = place.id,
                         template_name = 'spaces/space_index.html',
                         template_object_name = 'get_place',
                         extra_context = extra_context,
                        )

The class-based code:

class ViewSpaceIndex(DetailView):
 
    """
    Show the index page of a space. Get various extra contexts to get the
    information for that space.
 
    The get_object method searches in the user 'spaces' field if the current
    space is allowed, if not, he is redirected 
    """
    context_object_name = 'get_place'
    template_name = 'spaces/space_index.html'
 
    def get_object(self):
        space_name = self.kwargs['space_name']
 
        for i in self.request.user.profile.spaces.all():
            if i.url == space_name:
                return get_object_or_404(Space, url = space_name)
 
        self.template_name = 'not_allowed.html'
        return get_object_or_404(Space, url = space_name)
 
    def get_context_data(self, **kwargs):
        context = super(ViewSpaceIndex, self).get_context_data(**kwargs)
        place = get_object_or_404(Space, url=self.kwargs['space_name'])
        context['entities'] = Entity.objects.filter(space=place.id)
        context['documents'] = Document.objects.filter(space=place.id)
        context['proposals'] = Proposal.objects.filter(space=place.id).order_by('-pub_date')
        context['publication'] = Post.objects.filter(post_space=place.id).order_by('-post_pubdate')
        return context

DELETEVIEW

Function-based:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
@permission_required('spaces.delete_document')
def delete_doc(request, space_name, doc_id):
 
    """
    Delete an uploaded document
    """
    place = get_object_or_404(Space, url=space_name)
 
    return delete_object(request,
                         model = Document,
                         object_id = doc_id,
                         login_required = True,
                         template_name = 'spaces/document_delete.html',
                         template_object_name = 'doc',
                         post_delete_redirect = '/',
                         extra_context = {'get_place': place})

Class-based:

1
2
3
4
5
6
7
8
9
10
class DeleteDocument(DeleteView):
 
    """
    Delete an uploaded document.
    """
    model = Document
 
    def get_queryset(self):
        objects = Document.objects.all().filter(id=self.kwargs['doc_id'])
        return objects

LISTVIEW

Function-based:

1
2
3
4
5
6
7
8
9
10
11
12
def list_all_docs(request, space_name):
 
    """
    List all docuemnts stored within a space.
    """
    place = get_object_or_404(Space, url=space_name)
 
    return object_list(request,
                       queryset = Document.objects.all().filter(space=place.id).order_by('pub_date'),
                       template_name = 'spaces/document_list.html',
                       template_object_name = 'doc',
                       extra_context = {'get_place': place})

Class-based:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
class ListDocs(ListView):
 
    """
    List all documents stored whithin a space.
    """
    paginate_by = 25
    context_object_name = 'document_list'
 
    def get_queryset(self):
        place = get_object_or_404(Space, url=self.kwargs['space_name'])
        objects = Document.objects.all().filter(space=place.id).order_by('pub_date')
        return objects
 
    def get_context_data(self, **kwargs):
        context = super(ListDocs, self).get_context_data(**kwargs)
        context['get_place'] = get_object_or_404(Space, url=self.kwargs['space_name'])
        return context

REDIRECTVIEW

Function-based:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def go_to_space(request):
 
    """
    This view redirects to the space selected in the dropdown list in the
    index page. It only uses a POST petition.
 
    The 'raise Http404' isn't necessary, since if a space doesn't exist, it
    doesn't show, but just for security we will leave it.
    """
 
    place = get_object_or_404(Space, name = request.POST['spaces'])
 
    if request.POST:
        return redirect('/spaces/' + place.url)
 
    raise Http404

Class-based:

NOTE: Class-based redirects in django 1.3 only accepts GET petitions, instead of the recommended POST in django 1.2. This issue is fixed in 1.3.x

1
2
3
4
5
6
7
8
9
10
11
class GoToSpace(RedirectView):
 
    """
    This class redirects the user to a spaces after getting a GET petition.
 
    A 'raise Http404' is not necessary since a the objects the user can access
    are only the ones that are in the DB.
    """
    def get_redirect_url(self, **kwargs):
        self.place = get_object_or_404(Space, name = self.request.GET['spaces'])
        return '/spaces/{0}'.format(self.place.url)

Migrating from bzr to git + cgit + nginx

Well, after some months of “I don’t know… I just do small projects… git is too much for me” I decided to migrate from Bazaar (bzr) to git, the massively accepted and widespread VCS.

I already used git for grabbing code from the net, but never for my own projects. First of all was to test if I could migrate all the bzr repos to git, and it was quite easy!

I grabbed a copy of fast-export (comes with fast-import) into the bzr plugins directory:

mkdir -p ~/.bazaar/plugins
cd ~/.bazaar/plugins
bzr co lp:bzr-fastimport fastimport

After that it’s time to copy the repository:

git init new_git_project_folder
cd new_git_project_folder
bzr fast-export --plain ~/path/to/bzr/branch | git fast-import
git checkout master

Now it’s time for the real work, installing nginx, fastcgi, cgit and “configure ‘em all!

In gentoo it was quite easy, just do:

emerge -auND fcgiwrap spawn-fcgi nginx cgit

Be careful, for this you will need to unmask packages (cgit) and activate some USE flags like “nginx_modules_http_fastcgi” for nginx. I leave that work up to you, after all is not that difficult.

In my use case, nginx and the cgi will be on the same server, and that allows me to ommit a few fastcgi options. In this example I assume that your server root is /var/www/localhost.

First, configure the spawn-fcgi:

ln -s spawn-fcgi /etc/init.d/spawn-fcgi.cgit
cp /etc/conf.d/spawn-fcgi /etc/conf.d/spawn-fcgi.cgit

Edit /etc/conf.d/spawn-fcgi.cgit to look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Since I have everything on the same machine, I'll just use a socket
FCGI_SOCKET=/var/run/spawn-fcgi/localhost_spawn-fcgi
 
# We set this to nothing because if they're deleted, spawn-fcgi defaults them
# to their initial values.
FCGI_ADDRESS=
FCGI_PORT=
 
# Tell spawn-fcgi where is fcgiwrap (check this, it can be on another location)
FCGI_PROGRAM=/usr/sbin/fcgiwrap
 
FCGI_CHILDREN=1
FCGI_CHROOT=
FCGI_CHDIR=
 
# Defaults to root, so I change it to nginx
FCGI_USER=nginx
FCGI_GROUP=nginx
 
ALLOWED_ENV="PATH"

Copy cgit to the web root. Since gentoo’s webapp-config does not support nginx you’ll have to do this by hand:

cp /usr/share/webapps/cgit/0.8.3.1-r1/hostroot/cgi-bin/cgit.cgi /var/www/localhost/htdocs
cp /usr/share/webapps/cgit/0.8.3.1-r1/htdocs/cgit.css /var/www/localhost/htdocs
cp /usr/share/webapps/cgit/0.8.3.1-r1/htdocs/cgit.png /var/www/localhost/htdocs

We must now configure cgit to look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
virtual-root=/
cache-size=1000
css=/cgit.css
enable-index-links=1
enable-log-filecount=1
enable-log-linecount=1
logo=/cgit.png
root-title=Silvana Git Browser
root-desc=It is all done by electricity, my dear Arronax, by electricity! - Capt. Nemo
cache-root=/var/cache/cgit
cache-root-ttl=0
cache-repo-ttl=0
cache-dynamic-ttl=0
cache-static-ttl=0
snapshots=tar.gz tar.bz2
# As recommended by cgit, we put the repositories in another file
include=/etc/cgitrc.local

Let’s create the repositories file /etc/cgitrc.local:

1
2
3
4
5
6
7
8
# List of git repositories
section=code # Not necessary but very useful to classify repos
repo.url=gitproject # The url in the web browser
repo.desc=The description 
repo.path=/path/to/your/repo
repo.owner=Your name or email
# I don't use this but in this field you will write the repo url to grab the code 
# repo.clone-url=

Finally, configure the nginx server. Add this line to the HTTP section of /etc/nginx/nginx.conf

1
2
# Use up to 10 Megabytes for the 'code' cache, keep inactive cache around for 1h
fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=code:10m inactive=1h max_size=100m;

And now create this server section:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
server {
    listen 80;
    server_name localhost;
 
    # Serve static files
    location ~* ^.+\.(css|png|ico)$ {
        root /var/www/localhost/htdocs;
        expires 30d;
    }
 
    location / {
        # I tried with some rewrites and only this one worked fine
        rewrite ^/([^?/]+/[^?]*)?(?:\?(.*))?$ /cgit.cgi?url=$1&$2 last;
 
        fastcgi_cache      code;
        fastcgi_cache_valid 200 5m;
        fastcgi_cache_use_stale off;
 
        fastcgi_pass            unix:/var/run/spawn-fcgi/localhost_spawn-fcgi-1;
        fastcgi_read_timeout    5m;
        fastcgi_index   /;
 
        fastcgi_param    DOCUMENT_ROOT    /var/www/localhost/htdocs;
        fastcgi_param    SCRIPT_FILENAME  /var/www/localhost/htdocs/cgit.cgi;
        fastcgi_param    QUERY_STRING  $query_string;
        fastcgi_param    REQUEST_METHOD  $request_method;
        fastcgi_param    CONTENT_TYPE  $content_type;
        fastcgi_param    CONTENT_LENGTH  $content_length;
        fastcgi_param    GATEWAY_INTERFACE  CGI/1.1;
        fastcgi_param    SERVER_SOFTWARE  nginx;
        fastcgi_param    SCRIPT_NAME  $fastcgi_script_name;
        fastcgi_param    REQUEST_URI  $request_uri;
        fastcgi_param    DOCUMENT_URI  $document_uri;
        fastcgi_param    DOCUMENT_ROOT  $document_root;
        fastcgi_param    SERVER_PROTOCOL  $server_protocol;
        fastcgi_param    REMOTE_ADDR  $remote_addr;
        fastcgi_param    REMOTE_PORT  $remote_port;
        fastcgi_param    SERVER_ADDR  $server_addr;
        fastcgi_param    SERVER_PORT  $server_port;
        fastcgi_param    SERVER_NAME  $server_name;
    }
 
    access_log /var/log/nginx/myrepos_access_log combined;
    error_log /var/log/nginx/myrepos_error_log warn;
}

Hope this article will help you. It is a mixup of some articles I found that helped me, specially the “Running cgit under nginx” article by Russell