<![CDATA[geeknam's blog]]> 2012-06-27T03:16:31+10:00 http://blog.namis.me/ Octopress <![CDATA[Fibonacci with caching]]> 2012-05-29T19:21:00+10:00 http://blog.namis.me/2012/05/29/fibonacci-with-caching Here is my attempt to compare the performance of fibonacci with different ways of caching the results. First implementation uses memoization decorator whereas the second one makes use of defaultdict for caching. You can read my previous blog post on caching with defaultdict.

Fibonacci 1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import timeit

def memoize(function):
    memo = {}

    def wrapper(*args):
        if args in memo:
            return memo[args]
        else:
            rv = function(*args)
            memo[args] = rv
            return rv

    return wrapper


@memoize
def fib(num):
    if num == 0:
        return 0
    elif num == 1:
        return 1
    else:
        return fib(num - 1) + fib(num - 2)


t = timeit.Timer(stmt="fib(150)", setup="from __main__ import fib")
print t.timeit()

# Approximate Result: 0.482068061829 seconds
Fibonacci 2
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

from collections import defaultdict
import timeit

class FibCache(defaultdict):

    def __init__(self, fn):
        self.fn = fn

    def __missing__(self, num):
        self[num] = self.fn(num, self)
        return self[num]


def fib(num, cache):
    if num == 0:
        return 0
    elif num == 1:
        return 1
    else:
        return cache[num - 1] + cache[num - 2]


fib_cache = FibCache(fib)

t = timeit.Timer(stmt="fib_cache[150]", setup="from __main__ import fib_cache")
print t.timeit()

# Approximate Result: 0.268569946289 seconds

Note: Default number of loops for timeit is 1000000

If you’re looking to hire a Python programmer, this would be a great programming question. Challenge all the candidates to write the fastest fibonacci function. Hire the one whose code is the fastest.

Feel free to suggest better/faster implementations if you know any, I’d be glad to add more to the collection.

]]> <![CDATA[Writing unit tests for Django middleware]]> 2012-05-13T02:21:00+10:00 http://blog.namis.me/2012/05/13/writing-unit-tests-for-django-middleware Sometimes, it is a pain in the ass to write tests for Django apps (even though Django provides some pretty awesome testing tools out of box). One of the scenarios is testing middleware. Let’s take a look at this middleware.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

from .cart import Cart

class CartMiddleware:
    def process_request(self, request):
        '''Load the current cart from the session, or create a new one'''
        cart = request.session.get('cart', None)

        if cart is None:
            cart = Cart()
        else:
            cart = Cart(**cart)

        setattr(request, 'cart', cart)
        return None

What the middleware does:
- It gets attributes/values of the cart from the session
- It creates a cart with those unpacked arguments or creates a fresh cart and attaches this cart to the request object.

Now, let’s analyze what needs to be tested:
1. Assert that if the cart exists in the session, it should be set as an attribute of the request
2. Assert that if the cart doesn’t exist in the session, a new cart is created and attached to the request object
3. Assert that all attributes of the cart can be accessed after the cart is attached to the request object

How about dependencies? What do we need in order to write this test?
- CartMiddleware class (this can be easily imported)
- request object as an argument in process_request(). This one is hard to obtain, and since we are writing a unit test, let’s just mock it.

We are now ready to write the test

1
2
3
4
5
6
7
8
9
10

from django.test import TestCase
from mock import Mock
from .cart import CartMiddleware, Cart

class CartMiddlewareTest(TestCase):

    def setUp(self):
        self.cm = CartMiddleware()
        self.request = Mock()
        self.request.session = {}

This sets up an instance of CartMiddleware and mocks a request. I’m using Michael Foord’s mock library to assist me with this. Since we know session is a dictionary like object, we can mock it with an empty dictionary for now.

1
2
3
4
5
6
7
8
9
10

def test_process_request_without_cart(self):
    self.assertEqual(self.cm.process_request(self.request), None)
    self.assertIsInstance(self.request.cart, Cart)

def test_process_request_with_cart(self):
    data = {'postcode': '3050'}
    self.request.session = {'cart': data}
    self.assertEqual(self.cm.process_request(self.request), None)
    self.assertIsInstance(self.request.cart, Cart)
    self.assertEqual(self.request.cart.postcode, data['postcode'])

The first test asserts that (without a cart in the session):
- process_request returns None
- Cart object has been attached to request as an attribute

The second test asserts that (with a cart in the session):
- process_request returns None
- Dictionary containing cart attributes in session is unpacked and used to create a Cart object.
- The cart attribute (postcode in this case) can be accessed

Conclusion
As you could see, I didn’t have to use django.test.client.Client to create requests, responses or sessions. When writing unit tests, one shouldn’t care much about creating a precise dependency because, eventually what needs to be tested is the behaviour of the code and not the dependency itself. Therefore, it is more advisable to mock those dependencies in order to satisfy the conditions of the unit of code we test.

]]> <![CDATA[Burn your fixtures, use model factories]]> 2012-04-21T16:46:00+10:00 http://blog.namis.me/2012/04/21/burn-your-fixtures This blog post is inspired by one of the talks in PyCon US 2012 by Carl Meyer named Testing and Django in which he discusssed the guidelines for writing good tests.

After watching the whole talk, I realised what I’ve been doing is all wrong. The fact that tests I write run within minutes shows how terrible my unit-test-fu is. Using fixtures was a really bad idea. Each test case I wrote would load some fixtures to setup necessary dependencies. Sometimes, this creates a problem when the model has been redesigned. The fixtures would have to be edited due to the change of the model otherwise they won’t be installed properly.

Instead of using fixtures, Carl introduced model factories which would create a model instance with default values. Of course, you will have a flexibility to change those default values to suit each of your test because naturally, when you test a behaviour of a class, you definitely need it to be in different conditions.

Here is an example of a model factory that was used in the talk:

1
2
3
4
5
6
7
8
9
10

def create_profile(**kwargs):
    defaults = {
        "likes_cheese": True,
        "age": 32,
        "address": "3815 Brookside Dr",
    }
    defaults.update(kwargs)
    if "user" not in defaults:
        defaults["user"] = create_user()
    return Profile.objects.create(**defaults)

And this is how you could use it in the test:

1
2
3
4

def test_can_vote(self):
    """A user age 18+ can vote in the US."""
    profile = create_profile(age=18)
    self.assertTrue(profile.can_vote)

This approach solves the problem with maintaining the fixtures to suit your model design. If your model changes, the only thing you have to edit is the default values of the factory. Therefore model factories are way easier to maintain. If you need a large data set, you could easily write a for loop that wraps the model factory. So burn all your fixture right now !!!

]]> <![CDATA[Django custom email form field for Email Pie]]> 2012-03-27T01:14:00+11:00 http://blog.namis.me/2012/03/27/django-custom-email-form-field-for-email-pie As you know Django’s forms.EmailField() is capable of validating email addresses but what if you wanted to validate the existence of a particular email address? I’ve found a pretty neat solution for that, it’s called Email Pie. Email Pie is a wonderful little JSON API that gives you a simple way to validate email addresses. We’ll let you know if that email address is legit or not.. You could either use the service hosted on emailpie.com or run your own instance (it’s open source). In this post, I’m going to show you how to write a custom form field that makes use of Email Pie.

Make sure you pip install requests and simplejson before proceeding. Now create a fields.py file with the following content:

fields.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

import requests
import simplejson

from django import forms
from django.core.exceptions import ValidationError


EMAIL_PIE_HOST = 'http://emailpie.com'  # change this if you run your own instance

def validate_email_pie(value):
    params = {'email': value}
    response = requests.get('%s/v1/check' % EMAIL_PIE_HOST, params=params)
    response = simplejson.loads(response.content)

    if not response['success']:
        raise ValidationError(response['errors'][0]['message'])


class EmailPieField(forms.EmailField):
    default_validators = [validate_email_pie]

validate_email_pie method makes GET request to Email Pie and gets back a response. The service checks for email format and mx records, it also catches misspellings (you can get it with response['didyoumean']). validate_email_pie raises ValidationError if the service detects an invalid email address. Now in your forms you can use EmailPieField like this:

forms.py
1
2
3
4
5
6

from django import forms
from .fields import EmailPieField

class ContactForm(forms.Form):
    name = forms.CharField()
    email = EmailPieField()

For those interested, here are some tests for the field :)

tests.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

from django.test import TestCase
from django.core.exceptions import ValidationError
from .fields import EmailPieField


class EmailPieFieldTest(TestCase):

    def test_invalid_email(self):
        email_field = EmailPieField()
        self.assertRaises(ValidationError, email_field.clean, 'email@totest')

    def test_invalid_mx(self):
        email_field = EmailPieField()
        self.assertRaises(ValidationError, email_field.clean, 'email@totest.com')

    def test_valid_email(self):
        email_field = EmailPieField()
        self.assertEquals(email_field.clean('nam@namis.me'), 'nam@namis.me')

]]> <![CDATA[Python class factory]]> 2012-02-16T12:05:00+11:00 http://blog.namis.me/2012/02/16/python-class-factory Today’s post will be on factory design pattern in Python. You can read more about it in Wiki. This design pattern allows you to create objects without specifying the exact class of object that will be created. Let’s say, you want to create an object based on a provided string.

Here are some classes. Both of them are pets. We want to dynamically create a pet from one of those classes based on a string.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class Pony(object):

    def __init__(self, name):
        self.name = name
        self.hp = 100

    def magic_power(self):
        # Attack opponent with super powers
        pass

    def heal(self):
        # Increase hp (health point)
        pass

class Snake(object):
    # Blah blah same as Pony
    pass

Let’s write a factory method to make this happen:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

def create_class(pet_class_name):
    # pet_class_name decides what class will be returned
    if pet_class_name is 'pony':
        class InheritedClass(Pony):
            # You can add additional methods or attributes if needed
            pass
        return InheritedClass
    elif pet_class_name is 'snake':
        # Or you can return the exact class
        return Snake

# Create MyPet from a string
MyPet = create_class('pony')

# MyPet is now a subclass of Pony
# It inherits everything from Pony
django = MyPet('django')

# Kaboom !!!
django.magic_power()

]]> <![CDATA[Python caching using defaultdict]]> 2012-02-15T10:46:00+11:00 http://blog.namis.me/2012/02/15/python-caching-using-defaultdict Today, I’ve learnt about defaultdict container from my colleague, who uses it for caching. defaultdict container is part of collections module which provides high-performance container datatypes. You can find more about it here. Note: New in version 2.5.

Let’s see the code without caching mechanism:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

import csv
from models import Pony, PonySwag

# Getting some data from csv file
swags = csv.DictReader('pony_swag.csv', 'rU'), dialect='excel')

for swag in swags:
    # Get or create a pony
    pony = Pony.objects.get_or_create(
        name=swag['pony_name']
    )[0]

    # Create pony swag which belongs to a pony
    PonySwag.objects.get_or_create(
        name=swag['swag_name'],
        power=swag['swag_power'],
        pony=pony
    )[0]

In this scenario, for every entry in csv, the code hits the database to get a pony (based on the name) to assign it to a corresponding swag. Now if we cache the created ponies, we would not have to query the database anymore.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import csv
from collections import defaultdict
from models import Pony, PonySwag

class PoniesCache(defaultdict):
    # If dict lookup fails this will be called
    def __missing__(self, key):
        # Insert a pony in cache dict and return it
        pony = Pony.objects.get_or_create(name=key)[0]
        self[key] = pony
        return pony

ponies_cache = PoniesCache()
# Getting some data from csv file
swags = csv.DictReader('pony_swag.csv', 'rU'), dialect='excel')

for swag in swags:
    pony = ponies_cache[swag['pony_name']]

    # Create pony swag which belongs to a pony
    PonySwag.objects.get_or_create(
        name=swag['swag_name'],
        power=swag['swag_power'],
        pony=pony
    )[0]

As you might have read, defaultdict is a subclass of dict and it overrides a method called __missing__.
__missing__ is called by the __getitem__() method of the dict class when the requested key is not found (when swag['pony_name'] value is not found in ponies_cache). In our case, we override __missing__ to create a pony and store it in cache (if it doesn’t exist yet) and return it. This results in a pony being inserted in the dictionary with a key swag['pony_name'].

]]> <![CDATA[Python strptime is not thread-safe]]> 2012-02-14T15:30:00+11:00 http://blog.namis.me/2012/02/14/python-strptime-is-not-thread-safe When using Python to automate your work, threading is one thing you need to know of. I’ve got into a problem with threading when parsing time. To be exact: datetime.datetime.strptime() function was not thread-safe. Luckily, I found a solution on StackOverflow.

This is how the problem occurred:

1
2
3
4
5
6
7
8
9

from datetime import datetime
import threading

def parse_time(value):
    # Parse time with strptime()
    datetime.strptime(value], "%Y-%m-%d %H:%M:%S.%f")

t = threading.Thread(target=parse_time, args=(value))
t.start()

Fixing it was pretty straightforward. Just make a dummy call on strptime() before creating a thread:

1
2
3

datetime.strptime("2011-04-05 18:40:58.525996", "%Y-%m-%d %H:%M:%S.%f")
t = threading.Thread(target=format_time, args=(some_values))
t.start()
]]> <![CDATA[Regenerating Google's ClientLogin AuthToken in Django]]> 2012-02-14T12:47:00+11:00 http://blog.namis.me/2012/02/14/regenerating-googles-clientlogin-authtoken-in-django If your application makes use of Google API, you will most likely have to deal with AuthToken expiry (unless you use oAuth). As far as I know, AuthToken generated with ClientLogin authorization expires after 2 weeks. Therefore, it’s good to have a solution to auto regenerate it. First of all, create a model that stores the token and the last updated day.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

from django.db import models
from datetime import datetime, timedelta

class AuthToken(models.Model):
    token = models.TextField()
    last_updated = models.DateTimeField(auto_now_add=True)

    def token_update(self):
        # Check if the token's lifespan is more than n days (n=1 in this case)
        # You can adjust n value to whatever you want

        if datetime.now() - self.last_updated > timedelta(days=1):
            return True
        return False

Now let’s write a function that regenerates AuthToken and saves it back to database.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

from models import AuthToken

CLIENT_LOGIN_URL = 'https://www.google.com/accounts/ClientLogin'

def regenerate_authtoken():
    auth_token = AuthToken.objects.all()[0]
    if auth_token.token_update():
        data = urllib.urlencode(
            {
                'Email': 'youremail@gmail.com',
                'Passwd': 'yourpassword',
                'service': 'service_name',
                'accountType': 'GOOGLE'
            }
        )
        req = urllib2.Request(CLIENT_LOGIN_URL, data=data)
        resp = urllib2.urlopen(req)
        resp_body = resp.read()

        resp_dict = dict(
            x.split('=') for x in resp_body.split('\n') if x
        )
        auth_token.token = resp_dict['Auth']
        auth_token.save()

With this approach you would have to call regenerate_authtoken() every time you make API requests. Let me know if you have a better solution.

]]> <![CDATA[More on Django FormView]]> 2012-02-13T21:49:00+11:00 http://blog.namis.me/2012/02/13/more-on-django-formview I’ve been trying to limit myself to use class based generic views as much as possible. Django’s generic views were developed to take away the monotony of repeating certain patterns again and again. Some generics views are properly documented (like ListView and DetailView), however I wanted to make use of FormView to process my forms. Unfortunately, there were no FormView examples provided. So here it is.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

from django.views import generic

class EmailPreferenceView(generic.FormView):
    form_class = EmailPreferenceForm

    def get(self, *args, **kwargs):
        # You can access url variables from kwargs
        # url: /email_preferences/geeknam > kwargs['username'] = 'geeknam'
        # Assign to self.subscriber to be used later
        self.subscriber = get_subscriber(kwargs['username'])

    def post(self, request, *args, **kwargs):
        # Process view when the form gets POSTed
        pass

    def get_initial(self):
        # Populate ticks in BooleanFields
        initial = {}
        for s in self.subscriber.events.all():
            initial[s.value_id] = True
        return initial

    def get_form(self, form_class):
        # Initialize the form with initial values and the subscriber object
        # to be used in EmailPreferenceForm for populating fields
        return form_class(
            initial=self.get_initial(),
            subscriber=self.subscriber
        )

    def get_success_url(self):
        # Redirect to previous url
        return self.request.META.get('HTTP_REFERER', None)

    def form_valid(self, form):
        messages.info(
            self.request,
            "You have successfully changed your email notifications"
        )
        return super(EmailPreferenceView, self).form_valid(form)

    def form_invalid(self, form):
        messages.info(
            self.request,
            "Your submission has not been saved. Try again."
        )
        return super(EmailPreferenceView, self).form_invalid(form)

email_preferences = EmailPreferenceView.as_view()

]]> <![CDATA[Django dynamic unsubscribe forms]]> 2012-02-13T15:40:00+11:00 http://blog.namis.me/2012/02/13/django-dynamic-unsubscribe-forms The normal usecase of Django forms is to have predefined fields in forms like this:

1
2
3
4
5
6

from django import forms

class EmailPreferenceForm(forms.Form):
    daily_digest = forms.BooleanField(required=False)
    weekly_news = forms.BooleanField(required=False)
    monthly_summary = forms.BooleanField(required=False)

What if we wanted to add more fields to the form like weekly_digest and so on? Well, of course you would have to add a new field to the form and maybe handle extra work in your views. Let’s make this more dynamic.

Let’s create Event model which will store all events. Entries from this model will be generated into fields in the form later:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

from django.db import models

class Event(models.Model):

    title = models.CharField(
        max_length=100,
        help_text='e.g: Weekly News'
    )
    title_id = models.CharField(
        max_length=100,
        blank=True,
        editable=False
    )

    def __unicode__(self):
        return self.title

    def save(self, *args, **kwargs):
        if not self.title_id:
            # Generate title_id based on title
            # Weekly News > weekly_news
            self.title_id = self.title.lower().replace(' ', '_').replace("'", '')

And now the form that will generate fields based on all entries in Event model

1
2
3
4
5
6
7
8
9
10

from django import forms
from models import Event

class EmailPreferenceForm(forms.Form):
    # Auto generate fields based on title_id in Event
    def __init__(self, *args, **kwargs):
        super(EmailPreferenceForm, self).__init__(*args, **kwargs)
        fields = Event.objects.all()
        for f in fields:
            self.fields[f.title_id] = forms.BooleanField(required=False)

Let’s say you have a Subscriber model that has many Events. This piece of code will initialize the checkboxes based on subscribers’ preferences.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

from forms import EmailPreferenceForm

def email_preferences(request):
    if request.method == 'POST':
        email_preferences = EmailPreferenceForm(request.POST)
        # Process the form
    else:
        # Initialize the preferences. This will make the boxes checked.
        initial = {}
        for e in subscriber.events.all():
            initial[e.title_id] = True
        email_preferences = EmailPreferenceForm(initial=initial)

    # Context and HTTPResponses here
]]> <![CDATA[Loading fixtures in Django tests]]> 2012-02-13T14:22:00+11:00 http://blog.namis.me/2012/02/13/loading-fixtures-in-django-tests When writing tests in Django apps, I have an urge to reuse code in setUp() across all tests (most of my apps use the same setUp). So I come up with a GlobalSetup class that other tests should inherit from (it’s located at the project level). However, this approach creates a problem with loading fixtures. According to Django docs:

Once you’ve created a fixture and placed it in a fixtures directory in one of your INSTALLED_APPS, you can use it in your unit tests by specifying a fixtures class attribute on your django.test.TestCase subclass

And you would normally do this to load the fixtures:

1
2
3
4
5
6
7

from django.test import TestCase

class PoniesTest(TestCase):
    fixtures = ['ponies']

    def test_ponies(self):
        call_some_test_code()

How should we load the fixtures now, when the test file is not at the apps level? Instead of creating fixures folder and placing fixtures in them (in every app), a better and neater way to handle this would be to put all fixtures in one folder at the project level and load them. You can do this with call_command as following:

1
2
3
4
5
6
7
8

from django.test import TestCase
from django.core.management import call_command

class GlobalSetup(self):
    # Create some ponies and snakes here
    def setUp(self):
        # Load fixtures
        call_command('loaddata', 'tests/ponies', verbosity=0)

Invoking this call_command is equivalent to running manage.py loaddata /path/to/fixtures The test in each app can now inherit GlocalSetup and reuse setUp():

1
2
3
4
5
6
7
8

from project.tests import GlobalSetup

class PoniesTest(GlobalSetup):
    def setUp(self):
        super(PoniesTest, self).setUp()

    def test_ponies(self):
        call_some_test_code()

You could also limit each test to load different fixtures by passing a list of fixtures name ['ponies', 'snakes'] to setUp(). Hope this will help you to DRY up your tests.

]]> <![CDATA[First post from Octopress]]> 2012-02-11T21:42:00+11:00 http://blog.namis.me/2012/02/11/first-post-from-octopress Hey there! This is my new blog powered by Octropress and hosted on Github. Octopress was exactly what I was looking for in a blog engine (Yes, code snippets should be built out of the box in a blog). I’ve had Blogger previously but ever since I switched it to the new theme, all my github gists were not able to display. So I abandoned it. I might migrate the posts from Blogger here, however I’ll mainly focus on writing about Python and Django in this one. That’s about it, stay tuned for the next post. :wq

]]> <![CDATA[Fabric deployment]]> 2011-11-12T13:28:00+11:00 http://blog.namis.me/2011/11/12/fabric-deployment Having played with Capistrano before, I suddenly had an urge to find its Pythonic alternative. After few google searches, I stumbled upon Fabric. It’s basically a “python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks”. Spent few minutes trying to write functions to deploy Django applications to my Linux server and I have to say, Fabric’s API is really easy to use, very customizable (it’s as easy as wrapping up your shell commands in a python function) and more importantly it has Django integration which I’m gonna need the most. Having said all these, here’s the video demo I recorded while messing around with Fabric, enjoy:

P/S: django-nose, nosegrowl2 and tdaemon were used in order to show notification of testing results (just FYI)

]]> <![CDATA[Fatal error: Call to undefined function _make_cat_compat()]]> 2011-11-05T13:23:00+11:00 http://blog.namis.me/2011/11/05/fatal-error-call-to-undefined-function-make-cat-compat If you have the following error after uploading your WordPress, and re-uploading category-template.php didn’t work, you could try pasting this function at the of the file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<?php

function _make_cat_compat( &$category ) {
    if ( is_object( $category ) ) {
        $category->cat_ID = &$category->term_id;
        $category->category_count = &$category->count;
        $category->category_description = &$category->description;
        $category->cat_name = &$category->name;
        $category->category_nicename = &$category->slug;
        $category->category_parent = &$category->parent;
    }
    elseif ( is_array( $category ) && isset( $category['term_id'] ) ) {
        $category['cat_ID'] = &$category['term_id'];
        $category['category_count'] = &$category['count'];
        $category['category_description'] = &$category['description'];
        $category['cat_name'] = &$category['name'];
        $category['category_nicename'] = &$category['slug'];
        $category['category_parent'] = &$category['parent'];
    }
}

?>
]]> <![CDATA[Aliases for Google Storage]]> 2011-06-11T13:13:00+10:00 http://blog.namis.me/2011/06/11/aliases-for-google-storage I’ve just registered an account on Google Storage last month and I totally love it as. I particularly like gsutil since I’m most of my time on Terminal. Here are two aliases I use with gsutil in my .bash_profile. Of course, remember to replace your bucket name

1
2
3
4
5
6
7

function gsup(){
    gsutil cp "$@" gs://bucketname.com
}

function gsdown(){
    gsutil cp gs://bucketname.com/"$@" ~/Desktop
}
]]> <![CDATA[Using PHP to parse git history]]> 2011-05-08T13:19:00+10:00 http://blog.namis.me/2011/05/08/using-php-to-parse-git-history Here is how you can store git command output in a good manner after which you will be able to easily retrieve necessary information. This snippet belongs to the project I’m currently working on.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

<?php
// Author: Ngo Minh Nam

$dir = "/path/to/your/repo/";
$output = array();
chdir($dir);
exec("git log",$output);
$history = array();
foreach($output as $line){
    if(strpos($line, 'commit')===0){
        if(!empty($commit)){
            array_push($history, $commit);
            unset($commit);
        }
        $commit['hash']   = substr($line, strlen('commit'));
    }
    else if(strpos($line, 'Author')===0){
        $commit['author'] = substr($line, strlen('Author:'));
    }
    else if(strpos($line, 'Date')===0){
        $commit['date']   = substr($line, strlen('Date:'));
    }
    else{
        $commit['message']  .= $line;
    }
}

print_r($history);

?>
]]> <![CDATA[GAE XMPPHandler]]> 2010-12-25T15:57:00+11:00 http://blog.namis.me/2010/12/25/gae-xmpphandler Finally got time to continue developing Push Contacts. I’ve gotten a nerve-wrecking problem while developing with C2DM applications. When the app supposed to get the push message, it didn’t respond to it. Looking at the logs I encountered an error stack related to GTalk (with TalkProvider tag). Decided to google the problem and the solution was to sign out of GTalk on the phone.

Ok now a tip related to Google App Engine development. I just added a new feature to Push Contacts which is allowing users to reply to the latest SMS received. I had to retrieve the email of the user the bot is chatting with to find which phone number to send the SMS to. Took a look at guru.appspot.com source code and decided to use xmpp_handlers.CommandHandler. Used this method text_message(self, message) and message.sender supposed to return the sender of a message but instead it returned an IM formatted string. Here is how you can retrieve the email.

1
2
3
4
5

class XMPPHandler(xmpp_handlers.CommandHandler):
    def text_message(self, message):
        idx  = message.sender.index('/')
        user = message.sender[0:idx]
        logging.debug(user)
]]> <![CDATA[Fix screwed up keyboard layout on Ubuntu 9.04]]> 2010-12-05T13:34:00+11:00 http://blog.namis.me/2010/12/05/fix-screwed-up-keyboard-layout-on-ubuntu-9-dot-04 Got Gnome desktop installed on TonidoPlug but when trying to vnc to it I realised the keyboard layout is messed up. This is how you fix it:

  1. Kill existing vnc session: vncserver -kill :1 (1 here is the pid)
  2. Edit with: nano /root/.vnc/xstartup
  3. Add this line: export XKL_XMODMAP_DISABLE=1
  4. Restart vncserver
]]> <![CDATA[TonidoPlug]]> 2010-11-28T15:19:00+11:00 http://blog.namis.me/2010/11/28/tonidoplug Yes, TonidoPlug is a new toy that I’ve gotten last week. It is a basically a plug computer that can act likea home server. The hardware specs are the same as SheevaPlug with 1.2GHz ARM and 512MB RAM and 512MB Flash. The best thing about it is the energy consumption ranging from 5-10 Watts.

So what’s the difference between TonidoPlug and SheevaPlug? The main and the most noticeable difference is the built-in software that TonidoPlug provides which makes it really easy to get started with. It’s seriously just plug-and-play and you will get all kinds of stuffs you need: torrent (yes torrenting 24/7 is leet), music player (that allows streaming), sharing files and photos, mounting drives.

But the problem with this is the limitation of applications you can install. What if you want to run an rsync server or install a LAMP stack? Luckily there is an open source project called PlugApps that allows users to have more flexibility. Some kind souls have compiled Arch Linux and port it to work on a SheevaPlug hardware. They named it Plugbox. SheevaPlug naturally runs on Ubuntu 9.04 but apparently Ubuntu has dropped support for ARM that’s why Arch Linux would be a perfect suit.

I’m currently trying to run Plugbox from my external hard drive that has 2 partitions but TonidoPlug doesn’t seem to boot from it. I’m gonna dive deeper into this. Will update the blog once I get it to work.

Edited
Ok, like finally I got TonidoPlug booting from external hard drive but it still runs on Tonido (Ubuntu 9.04). There’s no way I can make it boot from external hard drive with PlugBox. The problem was with reboot command, I had to make a cold boot (plug and unplug the power source) in order for the plug to boot into my Samsung G3. I’ve recently installed awesome stuff like:

  • webmin (for sysadmin)
  • gitosis (for git repos)
  • mt-daapd (iTunes Server)
  • ettercap (to monitor my home network)
  • lsyncd (to synchronize files)
  • nginx (for reverse proxy and web server)
  • django (web apps running with gunicorn and postgres)
  • transmission (torrent server)
  • vncserver
]]> <![CDATA[Simple web service with Google App Engine]]> 2010-06-13T13:44:00+10:00 http://blog.namis.me/2010/06/13/simple-web-service-with-google-app-engine It’s been a while since I last posted an entry. Nothing much has changed in my life. The two-week holiday has just passed by in a blink of an eye. Has been busy with Android assignment, Cypher IGs camp and Pycon Singapore 2010. Now after Pycon, I’m pretty much convinced about the power of simplicity of Python. Knowing Google App Engine (GAE) quite a while ago, I decided to use GAE to help me out in my assignment (yes, yes, Google to the rescue) by writing a simple GAE web service which stores/retrieves feedbacks in form of JSON. The following tutorial is simple enough for beginners to follow.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

import urllib
import json
from google.appengine.api import mail
from google.appengine.ext import webapp, db
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp.util import run_wsgi_app


class Feedback(db.Model):
    user     = db.StringProperty()
    heritage = db.StringProperty()
    comment  = db.StringProperty(multiline=True)
    star     = db.IntegerProperty()
    date     = db.DateTimeProperty(auto_now_add=True)

    #Convert feedback object into a dictionary
    def toDict(self):
        return {'user': self.user,'heritage': self.heritage,
                    'comment': self.comment,'star':  self.star,'date': self.date}


class FeedbackGetter(webapp.RequestHandler):
    def get(self,heritage):
    #Retrieve feedbacks of a particular heritage requested
        heritage = urllib.unquote(heritage)
        feedbacks = db.GqlQuery("SELECT * FROM Feedback WHERE heritage = :1", heritage)

        feedDict = {}
        for feedback in feedbacks:
            feedDict[str(feedback.key())] = feedback.toDict()

        self.response.out.write(json.encode(feedDict));

class FeedbackStore(webapp.RequestHandler):
    def post(self):
    #Storing feedback in datastore
        feedback = Feedback()
        feedback.user     = self.request.get('email')
        feedback.heritage = self.request.get('heritage')
        feedback.comment  = self.request.get('comment')
        feedback.star     = int(self.request.get('star'))
        feedback.put()

        #Sending confirmation email to the user
        myemail         = mail.EmailMessage()
        myemail.sender  = "World Heritage Site <noreply@androidnam.appspotmail.com>"
        myemail.to      = feedback.user
        myemail.subject = "Feedback on " + feedback.heritage
        myemail.body    = '''
        Thank you for submitting your feedback, \n
        Heritage: %s \n
        Comment: %s \n
        Rating: %s 
        ''' % (feedback.heritage, feedback.comment, feedback.star)
        myemail.send()

        self.redirect('/get/'+feedback.heritage)

class FeedbackForm(webapp.RequestHandler):
    def get(self):
        self.response.out.write("""
        <html>
          <body>
            <form action="/store" method="post">
              Heritage:<div><input name="heritage" /></div>
              Email:<div><input name="email" /></div>
              Comment:<div><input name="comment" /></div>
              Star:<div><input name="star" /></div>
              <div><input type="submit" value="Submit"></div>
            </form>
          </body>
        </html>
        """)


application = webapp.WSGIApplication([(r'/get/(.*)', FeedbackGetter),
                      ('/store', FeedbackStore),
                      ('/feedback', FeedbackForm)],
                                      debug=True)


def main():
    run_wsgi_app(application)

if __name__ == "__main__":
    main()
]]>