Yeah, way more than you can think. In this case I’m talking about the e-cidadania permission system. We all know that django provides some basic permission system at model level, but in the case of e-cidadania it wasn’t enough. We needed very detailed object level permissions.

First of all let me tell you that the e-cidadania project is run by only one person: me. At the time of designing the base of the project the object permission systems available were really outdated or didn’t even work, so I had to make a workaround that at the moment seemed a good idea, even after trying some example cases, it was based on three fields in what I call the “father” model, a model that doesn’t control everything but is the base of all the rest of the modules (if you want to take a look, it’s the Space module).

As I was developing everything went fine, then the class-based views came out and I couldn’t make the proper validations, so I had to make another workaround in the get_object method. It turned out to be “consistent” but I didn’t had enough time to test it, and I continued developing the platform, in some cases with the help of some people that joined temporarily the project (GSoC 2012, pre-GSoC 2013) and everything seemed fine, until a week ago.

Let me tell you, e-cidadania has almost 60Ksloc (stats here) and currently has 12 modules, there are still a couple of modules to develop. I was so centered in development that totally forgot to test all the user roles in the platform (it would take me 4 times the time it was taking) and I convinced my workmates  (which are not related to IT, that’s a different story) to make some basic usage testing. What we discovered was that most of the permissions failed, and the validations didn’t work as expected, allowing in one of the use cases an anonymous user to enter in private spaces and even interact in them!

And I had t redesign the permission system. From scratch. Luckily django-guardian is up-to-date now and we’re using it, but I didn’t know all the work we had to do until I finished planning the new design. Not only we had to redo all the views of the platform, we also had to modify all the permissions in the models, all the checks in the templates, everything from ground up! And that will take quite a while, and we planned to release e-cidadania 0.1.9 on April, and there’s a lot of work to do yet, and a lot to polish, and only a pair of hands is working on it.

So be careful with sofware design, it can cost you more than you think.

After seeing this post from Kenneth Reitz, I thought I should make my own also, so everyone knows that not only the US and northern Europe has great companies to work with.

I’ve been working for two years now at Cidadania, a very small SME from Santiago de Compostela, Spain and we are just 9 people there. I must say that even if it’s not the best paid company I worked with, they compensate that with very good working conditions, which are not very common in Spain except for maybe… 20 companies in all the country?

Since I entered in Cidadania, I didn’t have any schedule, I’ve had a 7h shift and I can work remotely if needed. I also learned huge amounts of stuff related with my work and traveled a lot.  I’ve been at Lleida (Catalonia), Helsinki (Finland), California (US), Chicago (US), Tallin (Estonia) and lot of different places around Spain.

Some details about this two years:

• I’ve travelled more than 28.500KM
• 25 days spent in hotels.
• 8 meetups attended
• 3 talks given.
• 3 countries visited.
• I’ve met hundreds of people.

I’ve met also tons of people from other countries and other ways of thinking (which has opened my mind quite a bit). Cidadania is a great place to work, people is ALWAYS nice to you, and you get angry at some points like “don’t you have blood in your veins or what?” (just joking).

Of course being a non-IT consultacy sometimes it gets in the way, because they don’t know how to handle IT stuff or projects, and this sometimes is annoying, but believe me if I say that I won’t leave this company unless I create my own company or I get out of Spain (well, maybe if Igalia hires me I’ll stay).

Things I’ve learned

• You’ll get pretty fucking lonely sometimes (I agree with Kenneth in that)
• Get some time to rest, believe me, you’ll have to put order in your mind sometimes.
• Don’t be shy. DON’T. You have to loosen the ropes that tie you to the habits and prejudices of yourself and/or your country.
• Don’t hesitate to ask whatever it is. (I’ve been told that a lot in school, and I’m applying it 20 years late)
• You may think that you’re no one in the world and the programmers around you are the greatest. Don’t be a fool, surely in the same room there someone who thinks the same and he/she is taking you as an example to follow.

Probably my workmates won’t be reading this post, but I just want to say them that they’re awesome. So, here to 2013, kippis!

Ok, I will be honest, I’m excited for achieving this thing. I’ll explain here all the process I did to get Trac 0.12.3 with AccountManagerPlugin 0.2.1 (yes, I know the current is 0.3.2 but it’s not on the python cheese shop) on a shared server via One-Click Install.

THESE INSTRUCTIONS REQUIRE ADVANCED KNOWLEDGE, YOU’RE ON YOUR OWN HERE. AT MARCH 31, 2012 THEY’RE STILL UNTESTED IN THE WAY THEY’RE WRITTEN HERE.

Requirements:

- Have a Working virtualenv (in my case an environment with Python 2.7)

Assumptions:

- You have a user with SSH access.
- You will not use a SVN repository at installation time.

1.- Create a new domain

I’m sure you know to do this. Just go to “Manage Domains” and create one.

2.- Activate HtAccess on the domain/folder

In my use case the domain was the root of the trac project but you can use a folder inside the domain to handle your project, just create the directory. Go to “Goodies -> HtAccess/WebDAV“, pick your domain and click “Set up a new directory“.

If you use the domain as the trac directory you don’t have to put anything in “Directory name” field. If you use a directory, put it there.

After that you need to activate “Password protected directory” and put your users in the “User accounts for this area” in the format that comes in the example. After that click on “Configure this directory”

3.- One Click Install Trac 0.11.4

For starters: there is a known bug in the DreamHost server that will make the Python interpreter to execute the system Trac, which is 0.11.1.

Go to “Goodies -> One Click Installers“. Select Trac and install it on your domain, do not select any repository. After you get the mails everything will be installed.

4.- Trying our new trac and messing around with HtAccess

If you try to access your new Trac you will see that asks for a username and password even before showing up: ouch!, that’s not what we want. We want a publicly visible Trac.

Now you need to access your account via SSH and enter your trac directory or domain. You will realize that there are two important files: .htaccess and .htpasswd.

We only need the .htpasswd file for trac, so we will tell the DreamHost server that we don’t want the .htaccess messing with us. Open the .htaccess file and you will see a section like this:

1
2
3
4
5
6

### Generated by Dreamhost. DO NOT modify!!! ###
AuthType Basic
AuthUserFile /home/username/yourtracdomain/.htpasswd
AuthName "members"
require valid-user
################################################

Comment it entirely (don’t delete it, you may need to use it later). Now go and rename the .htpasswd file to trac.htpasswd.

I’m not sure about this part, at this step I was struggling with AccountManager and already modified the config thousands of times

Open your trac.ini file (located at yourpirjectdir/conf/trac.ini) and modify the route to the .htpasswd file to your new trac.htpasswd.

Now you can access to your trac publicly. We just need to create an admin user. You can do it with:

trac-admin yourtracdir/ permission add USERNAME_IN_HTPASSWD TRAC_ADMIN

Now we have our administrator. I’m sure you want to enjoy your newly created Trac instance, but there is still work to do.

From here on, the method I followed is not the best, in fact it’s a WTF! hack, but I didn’t find a better way to do it.

5.- Installing Trac (what?) and AccountManagerPlugin

Ok, we need to upgrade our Trac installation, and for that we will need to install it again, but this time, on the virtualenv. As you might already noticed, our One-Click Trac is installed in a directory with the same name as the domain but with underscores, that’s where DreamHost messed everything. Do not delete that directory unless you REALLY know what you’re doing and how to fix the mess it will cause.

So, this is the easy part:

pip install -U trac tracaccountmanager

It will install (as of March 31, 2012) Trac 0.12.3 and AccountManager 0.2.1-dev.

You may notice two things, the first is that the python interpreter keeps using Trac 0.11.1 and second, it does not detect the AccountManagerPlugin.

6.- Warrior’s March.

We need to “redirect” all the petitions that FastCGI makes to the new Trac in our virtualenv. This will include modifying several files.

If you don’t know it I’ll tell you that when your server receives a petition to the trac, it loads the file “index.fcgi” located at your trac directory, so we will start there.

You will need to add the virtualenv directory to the PATH, so edit the “index.fcgi” file and add (right after TRAC_ENV):

export PYTHONPATH=”/home/YOUR_USER/YOUR_VIRTUALENV/lib/python2.7/site-packages”

After this, you can see in the file that the petition is redirected to the One-Click Trac, at the cgi-bin directory. Let’s go there, we will find two files: trac.fcgi and trac.cgi that contains almost the same code.

This is actually the real trigger of the trac installation, we need to tell him that when executed, it must import the trac module from our virtualenv. For that we will need to specify what interpreter to use and where to look for that module.

In both files you need to put at the first line this:

#!/home/YOUR_USER/YOUR_VIRTUALENV/bin/python

and just after the comment we must specify him a new route for the site-packages. If we don’t do it, it will load the system-wide Trac module.

1
2

import sys
sys.path.insert(0, '/home/YOUR_USER/YOUR_VIRTUALENV/lib/python2.7/site-packages/Trac-0.12.3-py2.7.egg')

This should be enough for the virtualenv trac to load, but we’re not finished. We didn’t update the Trac environment!!

First we need to check if your virtualenv works ok, so enter your home directory and check what trac-admin binary will be executed:

which trac-admin

If your virtualenv trac-admin will be executed there’s no problem, if not, you will have to specify the complete route to the executable. Now execute this:

trac-admin yourtracdir/ upgrade

It should upgrade your trac DB and stuff to the 0.12.3 version and also suggest you to update the wiki. Do it, you never know.

OK, we’re ready to go, aren’t we?

7.- Testing our Trac installation

We can now load our Trac 0.12.3 installation. Everything should be ok, but wait, what’s that?

<acct_mgr.web_ui.MessageWrapper? object at 0xa2a102c>

I don’t want that on my Trac!! I can’t access the admin! I don’t see anything working ok!

I haven’t investigated about this, but it’s a known bug in the AccountManager plugin that can be solved editing your trac.ini and adding this line:

acct_mgr.web_ui.emailverificationmodule = disabled

That’s it. Now you have a working Trac 0.12.3 in a shared DreamHost server.
Please make me know if it worked for you or if you had any troubles. I’ll do what I can to help you.

It is not very professional to have all the code repositories of a compny in your personal profile, even if the code is open source.

So, again I’m extending the cidadania infrastructure making a global account in GitHub. You can access here: http://github.com/cidadania and it’s own page at github (still working on it) http://cidadania.github.com

Hi! long time no see, specially regarding non programming stuff. This year I will go to the TEDx Galicia conference for first time. The main topic is not really my best, but hey, maybe I find some interesting stuff (and interesting people) there!

Hi! It has been some time since the last post…

As you know I’m developing an open source e-democracy platform called e-cidadania. The strong point of the platform is the debate system. It’s nothing like the forums we are used to see in the internet, this time is an ordered debate system.

The thing is, I’ve never developed something that complex and I’m not used to ajax interactions, but after our partners dissapeared (they were supposed to do that module) I had to start the development by myself. The debate system is very much like a kanban or scrum board (in fact I plan to revamp it for that) and I’m almost finishing the frontend. Since I have no experience with this maybe I’ve got the wrong approach, but “it works” for now (if you want to give me some advice you’re very welcome).

The current frontend is based on a table, jQuery sortables and a bunch of javascript to give the table the needed flexibility. Here goes a screenshot!

One of the modules for e-cidadania project was a basic event calendar, meaning that the event will be marked on the respective modules (proposals, spaces, debates, etc.) and will be gathered by the calendar module. In the first moment i thought to use jquery directly, but after some research I found that it was easier to generate the calendar locally and serve it to the client, so here it is, a multilanguage, event-driven HTML calendar:

The basic explanation is that we create a class inheriting the LocaleHTMLCalendar (a localized version of the HTMLCalendar module) and override some functions like formatmonth() then in the view we stablish the current user locale, some basic protection (month number) and convert the month and year numbers to integers, since they come as strings from the URL.

models.py (I put it on models for convenience but it really should be on views.py)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

import calendar
from datetime import date
from itertools import groupby
 
from django.utils.html import conditional_escape as esc
 
class EventCalendar(calendar.LocaleHTMLCalendar):
 
    """
    Event calendar is a basic calendar made with HTMLCalendar module.
    """
    def __init__(self, events, *args, **kwargs):
        self.events = self.group_by_day(events)
        super(EventCalendar, self).__init__(*args, **kwargs)
 
    def formatday(self, day, weekday):
        if day != 0:
            cssclass = self.cssclasses[weekday]
            if date.today() == date(self.year, self.month, day):
                cssclass += ' today'
            if day in self.events:
                cssclass += ' filled'
                body = ['<ul>']
                for event in self.events[day]:
                    body.append('<li>')
                    body.append('<a href="%s">' % event.get_absolute_url())
                    body.append(esc(event.title))
                    body.append('</a></li>')
                body.append('<ul>')
                return self.day_cell(cssclass, '%d %s' % (day, ''.join(body)))
            return self.day_cell(cssclass, day)
        return self.day_cell('noday', '&nbsp;')
 
    def formatmonth(self, year, month):
        self.year, self.month = year, month
        return super(EventCalendar, self).formatmonth(year, month)
 
    def group_by_day(self, events):
        field = lambda event: event.meeting_date.day
        return dict(
            [(day, list(items)) for day, items in groupby(events, field)]
        )
 
    def day_cell(self, cssclass, body):
        return '<td class="%s">%s</td>' % (cssclass, body)

views.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

from django.shortcuts import render_to_response, get_object_or_404
from django.utils.safestring import mark_safe
from django.template import RequestContext
from django.utils import translation
 
from e_cidadania.apps.spaces.models import Meeting, Space
from e_cidadania.apps.cal.models import EventCalendar
from e_cidadania import settings
 
def calendar(request, space_name, year, month):
 
    # Avoid people writing wrong numbers or any program errors.
    if int(month) not in range(1, 13):
        return render_to_response('cal/error.html',
                                  context_instance=RequestContext(request))
 
    place = get_object_or_404(Space, url=space_name)
    next_month = int(month) + 1
    prev_month = int(month) - 1
 
    meetings = Meeting.objects.order_by('meeting_date') \
                              .filter(space = place,
                                      meeting_date__year = year,
                                      meeting_date__month = month)
 
    cur_lang = translation.get_language()
    cur_locale = translation.to_locale(cur_lang)+'.UTF-8' #default encoding with django
    cal = EventCalendar(meetings, settings.FIRST_WEEK_DAY, cur_locale).formatmonth(int(year), int(month))
 
    return render_to_response('cal/calendar.html',
                              {'calendar': mark_safe(cal),
                               'nextmonth': '%02d' % next_month,
                               'prevmonth': '%02d' % prev_month,
                               'get_place': place},
                               context_instance = RequestContext(request))

If you remember, a few months ago I started a project called “Supervise”. It was a project management tool based on django, aimed to be the next “Redmine”, but for some personal reasons it had to be stopped (I’ve got a job!). Some new releases of django were made, and with them, hot new features were added.

The Supervise project was in an early stage of devlopment, so I said myself: “Why not start from scratch? It’s not that difficult”. So here it is: Supervise 2, a new version made on top of django 1.3. The list of features will be much the same as Supervise:

• Multiple project management like Redmine.
• Various SCM support: GIT, Bazaar, SVN, Mercurial. There will be more supported SCMs but for now that covers most of the developers needs.
• Issue tracking
• Wiki
• News
• User profiles

That’s for version 0.1-stable (which will come in a couple months I think)

For now you can watch the development or fork the code in both Gitorious and GitHub

UPDATE: I’ve rewrited the post since the instructions are much easier than I did put here (and more secure)

You will probably say: “Hey!, what’s in your mind to need django 1.3, it has been released just a week ago!”. The answer is: long-term development. Currently I’m working on e-cidadania, an e-democracy tool designed for participative processes. The development will take some time, and we’ve just started. It’s better to update everything now than later, but to the point, installing django 1.3 in dreamhost.

NOTE: I assume you know how to install a django project in DH. If not, please see this wiki

Let’s start.

STEP 1: CREATE A VIRTUALENV

You can create a virtualenv with the system-wide python installation (currently 2.5.2) or install your own python interpreter if you need it. We will use the system-wide python for convenience, since it’s still supported by django 1.3.

This part is extracted from here

NOTE: These commands must be typed at your home directory

$ wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.5.2.tar.gz
$ tar xzf virtualenv-1.5.2.tar.gz
$ python virtualenv-1.5.2/virtualenv.py $HOME/env

This will create a directory called “env” with all the stuff you need to work. Let’s do this virtual environment our default.

STEP 2: STABLISH THE VIRTUALENV PYTHON AS DEFAULT

The following instructions about stablishing the virtual python as default only work at shell level. For passenger the system-wide python is the default.

On ~/.bash_profile write:

export PATH=$PATH:$HOME/env/bin

This will stablish the directory ~/env/bin (where our virtual python is) before the system-wide python. To work right now without logging in again, we can do:

$ source .bash_profile

That will make the executables in ~/env/bin available for using right now, and between them, it’s easy_install, properly configured to install any python module locally without problem.

STEP 3: INSTALL DJANGO 1.3 AND ANY OTHER PACKAGES

Let’s install django 1.3 and some common python packages needed in almost every project:

$ easy_install -U django

For the moment we will install just django, because the rest of python packages are provided by the DH server. If you need non standard modules (p.e. python-dateutil) or other version of an already provided package, you must install them after django.

If you try to execute some django project right now you’ll see that the system-wide installation of django is the preferred. Let’s change that.

STEP 4: CONFIGURING PASSENGER_WSGI.PY

In our passenger_wsgi.py we must modify the PYTHONPATH, let’s do it:

import sys, os
 
# Python has no prepend function, so we must do an insert. We insert the key directories
# BEFORE ANYTHING, so the python installation sees them before the system-wide libraries.
 
sys.path.insert(0,'/home/USERNAME/env/bin')
sys.path.insert(0,'/home/USERNAME/env/lib/python2.5/site-packages/Django-1.3-py2.5.egg')
sys.path.insert(0,'/home/USERNAME/env/lib/python2.5/site-packages')
 
os.environ['DJANGO_SETTINGS_MODULE'] = "YOURPROJECT.settings"
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Remember that if any of your modules fails, python will not fall back to the system-wide installation. Once a module is loaded, python does not seek another. Also, if you installed another package apart from django, you must do an insert including it.

That’s it! Now when you receive a visitor, Passenger will load you desired django version and packages.

This is more a reminder than a proper post, but let’s see how do we do this. I’ve been weeks wondering how the heck I could change the language on the fly in a website with the django i18n system, and the worst is that IT’S DOCUMENTED, but it’s sparse across the documentation and the source code docs (hey guys, you could write down all the documentation from time to time).

Let’s start:

To have the translation system working you need to add this middleware to your settings:

1

    django.middleware.locale.LocaleMiddleware

After that, on your urls.py file you need to include the i18n view:

1

    (r'^i18n/', include('django.conf.urls.i18n')),

That view only accepts POST requests, so if for any reason you need a GET, you will better be prepared, because you have to modify a core view, or create your own one. Ok, that view will put the url “i18n/setlang” to our service. Let’s go to the interesting part, the HTML code:

1
2
3
4
5
6
7
8
9

    <form action="/i18n/setlang/" method="post">{% csrf_token %}
      <input name="next" type="hidden" value="/" />
      <select name="language">
        {% for lang in LANGUAGES %}
          <option value="{{ lang.0 }}">{{ lang.1 }}</option>
        {% endfor %}
      </select>
      <input type="submit" value="Go" />
    </form>

This code will make a dropdown list with ALL the languages supported by django (and there are quite a few) so if you want to restrict the languages you must set up the LANGUAGES variable in your settings.py with something like this:

1
2
3
4
5

LANGUAGES = (
    ('es', 'Spanish'),
    ('en', 'English'),
    ('gl', 'Galician'),
)

Be warned, this language list is NOT translated, that means that wherever you are, that list will be always in english. To translate it you must make use of gettext directly. Do not attempt to use the translation utils from django, that will cause a circular import.

UPDATE:

Aleck gave us another way of doing the i18n selector, using links instead of a dropdown (which is very useful :) In this use case, we use list items for selecting from English and Dutch languages. This HTML code assumes you have this code in your settings.py:

1
2
3
4

LANGUAGES = (
    ('en', 'English'),
    ('nl', 'Dutch'),
)

HTML code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<li>
    <form name="setLangEnglish" action="/i18n/setlang/" method="POST">{% csrf_token %}
        <input name="next" type="hidden" value="/" />
        <input type="hidden" name="language" value="en" />
        <a href="#" onclick="document.setLangEnglish.submit();return false;">English</a>
    </form>
</li>
<li>
    <form name="setLangDutch" action="/i18n/setlang/" method="POST">{% csrf_token %}
        <input name="next" type="hidden" value="/" />
        <input type="hidden" name="language" value="nl" />
        <a href="#" onclick="document.setLangDutch.submit();return false;">Dutch</a>
    </form>                
</li>

Thank you very much Aleck!

UPDATE 2:

These days I’m updating the CSS of e-cidadania to Bootstrap 2.0 so I thought it could be a nice moment to change the language selector for something more stylish. Here is the code to use a list selector with bootstrap 2.0:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

<ul class="nav pull-right">
    <li class="dropdown">
        <a href="#" class="dropdown-toggle" data-toggle="dropdown">{% trans "Language" %}<b class="caret"></b></a>
        <ul class="dropdown-menu">
	    {% for lang in LANGUAGES %}
	        <li>
                    <form name="setLang{{ lang.1}}" action="/i18n/setlang/" method="POST">{% csrf_token %}
                        <input name="next" type="hidden" value="/" />
                        <input type="hidden" name="language" value="{{ lang.0 }}" />
                        <a href="#" onclick="document.setLang{{ lang.1 }}.submit();return false;">{{ lang.1 }}</a>
                    </form>
                </li>
            {% endfor %}    
	</ul>
    </li>
</ul>