Initial setup Django 1.5 with django-celery 3.0 and rabbitmq broker

System/Requirements – Django 1.5 with django-celery 3.0 and a rabbitmq broker on ubuntu 13.10 (Xubuntu)

  • Assumption – you already have a functioning django site up and running, you can see my previous post.

The end goal is to be able to observe and track tasks (especially longer running tasks) from the django admin. For example, to use the following

from celery import task

def add(x, y):
    return x + y

Then observe tasks being recorded in the admin site.


Continue reading

Using Bootstrap with Django

Part 2

This is a continuation from the previous post on using django-skel to create a new django project.


In this post we will walk through using Bootstrap (a front-end framework for developing responsive projects) github.

Assumptions: you have already completed the previous post and have an initial working django site.

Goal: The result of this tutorial / walk-through is a django page with the basic template from the guide.


Continue reading

Using django-skel to create a new Django site

Part 1

This is the first step – in Part 2 I cover adding bootstrap support.


Create a django site using django-skel on xubuntu 13.10 using django 1.5.

I have not created a new django site for a while (having been consumed with opencv and TurboGears), so I wanted to start a fresh walkthrough (also to prepare for startup weekend By the end I was able to get an admin page going, a good layout for the project, and lots of helpful developer tools in place.

Following the excellent documentation provided for django-skel:

Continue reading

Python depth first search with graphviz

Graphviz is an awesome tool to help visualize graphs, and there is a handy interface for python This is a basic example of how to do a depth first search in python and then visual the results. I will walk through the basic steps to get graphviz and what I did to make it work.

NOTE: graphviz provides search implementations, I have created a beginner/toy implementation for demonstration purposes.


  1. Linux – Xubuntu 13.10 – can be made to work for other systems but this is what I am using for this guide.
  2. Python 2.7 – assumed that you got it as part of xubuntu.
  3. virtualenv – could do without but I have used it here.

Example Results

What you can expect at the end – generate a png for each step of the depth first search


Continue reading

Working through MongoEngine Tutorial

Having reviewed several available options – I decided it would be fun to try MongoEngine with my new Django site.

I will be walking through this tutorial to start with – I will start where I left off with the skeleton site.

Getting Started

The suggested installing it with “$ pip install mongoengine”

(lazybower)rcstats@ubuntu:~/Desktop/lazybower/LazyBower$ pip install mongoengine
Downloading/unpacking mongoengine
 Downloading mongoengine-0.8.4.tar.gz (124kB): 124kB downloaded
 Running egg_info for package mongoengine

 no previously-included directories found matching 'docs/_build'
Requirement already satisfied (use --upgrade to upgrade): pymongo>=2.5 in /home/rcstats/Desktop/lazybower/lib/python2.7/site-packages (from mongoengine)
Installing collected packages: mongoengine
 Running install for mongoengine

 no previously-included directories found matching 'docs/_build'
Successfully installed mongoengine
Cleaning up...

I then went ahead and added it to the dev requirements (requirements/dev.txt).

python shell_plus
In [1]: from mongoengine import *

In [2]: connect('tumblelog')
Out[2]: MongoClient('localhost', 27017)

Complete the Tutorial

I ran through without incident just pumping the commands suggested in the tutorial into the django ipython shell.

(lazybower)rcstats@ubuntu:~/Desktop/lazybower/LazyBower$ python shell_plus


from mongoengine import *


class User(Document):
 email = StringField(required=True)
 first_name = StringField(max_length=50)
 last_name = StringField(max_length=50)

class Comment(EmbeddedDocument):
 content = StringField()
 name = StringField(max_length=120)

class Post(Document):
 title = StringField(max_length=120, required=True)
 author = ReferenceField(User, reverse_delete_rule=CASCADE)
 tags = ListField(StringField(max_length=30))
 comments = ListField(EmbeddedDocumentField(Comment))
 meta = {'allow_inheritance': True}

class TextPost(Post):
 content = StringField()

class ImagePost(Post):
 image_path = StringField()

class LinkPost(Post):
 link_url = StringField()

Mongo Cheat Sheet

So you can double check your work, here is the cheat sheet I use:



That gives an overview of my experience getting started with MongoEngine. I was fairly pleased and will move on to integrating it with a Django app in the following post.


Starting a new Django site


Start a new django site using one of the available skeleton/templates for django projects on a Linux Ubuntu system. This will be a record of the steps and modifications I needed to make this run on my developer system.

Assumptions – this guide uses xubuntu 13.04 and assumes you already have django installed. This walkthrough was just intended to be a record of the steps required to use a skeleton project. You can find the django install guide here along with their excellent tutorial.

Continue reading

Django Memcache


I want to increase the performance of several pages on, there are a few problematic pages that are taking several seconds to load. They have rarely changing content, but somewhat complex queries to the DB underneath them. I have decided to use memcache and will be using the following references as a guide:

Continue reading

Fixing an bug in the race uploader (txt file to PostgreSQL)


When I implemented the automatic file uploader I neglected to add a test case for the time field that stores the race time (it comes in in the format <hour>:<minute>:<second>.<microseconds>). My first implementation had worked correctly and I had never had any tests to verify this particular field was getting process correctly.

Now I needed to fix the uploader and try to fix the incorrect data in the database without having to reprocess all the old results.

Continue reading

Performance Tweaks


This is simply a record of my investigation into merging and compressing my static resources (css and js files) on a personal website. What follows is the resources I looked at and how I hoped to use them on my site.

In summary, I have not found a good solution and I will revisit the subject in the future.

Continue reading

Django Basic File Upload

As part of the ongoing improvements to the web site, I have decided that I want contributors to be able to upload race results. Currently I am using a python script to upload the results myself.

Some Back Ground on The Current Process

Currently I use set of python scripts and pgdb to parse the race results and feed them into the postgresql database. The code to feed results into the database was a bit rough and certainly not robust enough enough for contributors to work with (it has been a bit of bootstrapping as I learned more about security with respect to django and postgresql).

This was a design decision I made early on. That I would prioritize getting results into the system quickly, the consequence being that I would need to manually run script to upload races results every time they became available. Now that the site is live and I am able to start getting user feedback, the time has come for me to clean up this process.

References To Start From

Initial Prototype

Using the django documentation and some stackoverflow examples I arrived at the following code.


{% extends "base.html" %}

{% block pagetitle %}Upload Race Results{% endblock %}
{% block pagetitle_description %}Upload race results from the scoring system.{% endblock %}

{% block content %}

<section class="full_width">
<h2>{{ error_status }}</h2>
	<form action="/upload_start/" method="post" enctype="multipart/form-data">
	{% csrf_token %}
	<input type="file" name="file" />
	<input type="submit" name="submit" value="Upload" />

{{ form.errors }}
{{ form.non_field_errors }}
</section><!-- /.full_width -->
{% endblock %}

class UploadFileForm(forms.Form):
    #title = forms.CharField(max_length=50)
    file  = forms.FileField()

def upload_start(request):
    if request.method == 'POST':
        form = UploadFileForm(request.POST, request.FILES)

        # What causes the form to not be valid?
        if form.is_valid():
            # Need to make sure the key used for FILES[ ] matches up with the
            # form in the template.
            return HttpResponseRedirect('/upload_start')
            error = "Failed to upload file:" + request.FILES['file'].name
            return render_to_response('upload_start.html', {'form':form, 'error_status': error}, context_instance=RequestContext(request))

        form = UploadFileForm()
    return render_to_response('upload_start.html', {'form': form}, context_instance=RequestContext(request))

def handle_uploaded_file(f):
    #print "name:",
    #print "size:", f.size
    with open('/home/username/Desktop/rcstats/rcdata_media/fileupload/testfile.txt', 'wb+') as destination:
        for chunk in f.chunks():

Some of the initial stumbling points I hit that are worth a note:


At this stage, this is sufficient to result in a file getting placed in the assigned directory. I am in the process of designing the E2E user experience through this process, I will follow up with an additional post to cover the next steps.