Knockout Advanced Tips: Binding Custom Parameters

It is possible in Knockout to bind custom parameters to a call in JavaScript. I believe this is intrinsic to JavaScript. It also kind of reminds me of currying though at the moment I am not entirely sure.

For example, suppose you wanted to call myFunction with some parameters besides event and item (because those are what you get normally by Knockout). I need to do this because I want a callback after the click event. So here is my function:

var myFunction = function(callback, data, event) {


   //value of "this" would be $data
};

Here is how I would do the call:

click: myFunction.bind($data, myCallbackFunction);

The first argument to "bind" is the context (value of "this") that you want the function to execute with. Any arguments after that will be passed to the function prior to the actual arguments.

Data and event are always passed after any extra arguments that you provided to "bind".

Architecting KnockoutJS Applications

One of my favorite front end javascript libraries is KnockoutJS.

KnockoutJS is a javascript library that helps you bind javascript variables to DOM data. When you change the javascript variable the HTML automatically changes to the new value of the variable.

The magic behind how this happens isn't particularly important. If you really want to know you can read the source. The basic pattern is commonly referred to as event pooling.

Some of the best parts:

 - You should use knockout if you have a bunch of $(element).click(function()) functions and want to clean it up. If your frontend code is even a bit complicated you probably know what I am talking about.
 - It is easy to integrate with an existing project. AngularJS is probably best off when you are starting from scratch. Knockout is great because you can optionally use it when you have a really complicated frontend.
 - Really quick to learn. I found knockout super easy to pick up. The thing you have to get over is your elements looking like:

<span data-bind="text: airplane"></span>


It irked me a bit to see a data-bind attribute when it isn't really HTML. Yes you are mixing presentation logic with code. Honestly though I've seen learned that it is pretty pragmatic. Lots of HTML editors play nice with knockout and people who only know HTML and CSS can pick it up quickly.


Some of the parts I don't particular like about Knockout:

 - Lack of a good PJAX/history.js style navigation. You get this with AngularJS.
 - Getting your view models to be semi reusable is a pain in the butt. I used a combination of server side script includes and knockout statements.

When first getting started you may wonder how should you architect your application so it is somewhat less like spaghetti. A good way to organize your Knockout code is first think of all the objects in your application. If you are using Rails (or in my case Django) you will have a bunch of models.

For each of these models it is nice to have some sort of front end analogue in Javascript.

Here is an example of an object:


function Airplane(viewModel, data) {
  var self = this;
  ko.mapping.fromJS(data, {}, self);

  self.visible = ko.observable(true);
  self.selected = ko.observable(false);

  self.land = function(item, event) {
    $.ajax({
      url: "/api/airplane/" + self.id() + "/land",
      type: 'POST',
      success: function() {
        // Update myself with the fresh data from the server
        ko.mapping.fromJS(data, self);
      },
      error: function(jqXHR, textStatus, errorThrown) {
      }
    });
  }
}

I usually like to stick this in some namespace that is available globally. A few comments about this object. I take in the viewModel which created it so I have access to the parent (this is useful later). I also take in this variable called data which is used for an awesome knockoutjs plugin called ko.mapping. It basically iterates through your javascript object and creates ko.obserables() out of all of attributes. Super useful and you want to use this. I didn't use it before and I had to do self.variable = ko.observable(data.variable) over and over. Yeah not a good idea.

Another awesome thing about the plugin is when your AJAX REST API call returns you can update the object in one line by doing ko.mapping.fromJS(data, self).

Hope you give knockout a try. If you have any questions I've noticed the knockout community is awesome. I am also available so just drop me a note.

-Jeff

Introducing Django Friendlyurls

Django Friendly URLs

Django Friendly URLs is a Django application allows you to create urls that directly return views. There is no 302 Redirect.

Why would you want such a thing?

The most common use case is that you have URLs which return user profiles such as http://www.example.com/user/1

Friendly URLs allow you to create a URL so that the user can also be accessed by going to:

http://www.example.com/jeff

The key is that the address bar in the browser does not change to http://www.example.com/user/1

The web server returns the same HTML document as http://www.example.com/user/1

Flow

The application looks up the string 'jeff' If it finds a friendly url match it will read out the absolute_path and attempt to resolve it by putting it through the URL resolver a 2nd time.

If it finds a resolution it will return that view.

Usage

Here is an example of how to implement friendly_urls for a user:

I put this in models.py:

from django.contrib.contenttypes import generic from friendlyurls import UrlMapping from django.contrib.auth.models import User

def get_absolute_url(self):

return u'/user/%s' % self.id

User.add_to_class('friendly_urls', generic.GenericRelation(UrlMapping)) User.get_absolute_url = get_absolute_url

This basically monkey patches the User model to add a field. You can then do

user = User.objects.get(pk=1) user.friendly_urls.all()

This will get a list of all the friendly urls.

You shouldn't have to monkey patch all the models.

Problems / Questions

Don't forget to run syncdb because friendlyurls creates a table to store the URL mappings.

Additional Settings

FRIENDLYURLS_IGNORE_CASE - Set this to True if you want to ignore the case when trying to find a path. Default: False

Todo Features

• Need to cache the results returned by a lookup for performance.

Probably do a from django.core.cache import cache

https://docs.djangoproject.com/en/dev/topics/cache/?from=olddocs

Update to backing up Google Apps Accounts

It turns out there is an even easier way to backup Google Apps Accounts.

Since we are still on the free tier I had to manually go in to each account and enable IMAP.

After this was done I found a tool called imap2maildir.

https://github.com/rtucker/imap2maildir/ 

$ ./imap2maildir -u jeff@example.com -d jeff@example.com --create

Created a directory called "jeff@example.com".

Backing up and Archiving Google Mail Accounts

The easiest way to backup Google Mail accounts is to enable IMAP on the account and download the messages using getmail.

Once getmail is installed you need to create a file called getmailrc. If you plan to download multiple gmail accounts then you might want to create a directory for each account and point the getmail script to that directory. Here is an example of a getmailrc file for Google Mail:


[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmail.com
username = username@example.com
password = examplepassword
mailboxes = ("[Gmail]/All Mail",)
port = 993

[destination]
type = Maildir
path = ~/username@example.com/

[options]
received = false
delivered_to = false
read_all = false
verbose = 1


After this file is saved you can procede to run getmail. I needed getmail to run all night and in the background. I outputted all the stdout to a logfile so I used the following command:

getmail --getmaildir . > output.txt 2>&1 &

Don't forget to create the directories cur, new, tmp as these are the directories that are needed for IMAP.


Now that you have your mail in a Maildir format what do you do with it? In my case I wanted to delete the account off Google Apps but still be able to search the mail if I needed it at a later date.

The strategy I came up with to bring up a copy of courier and serve the Maildir using a webmail script (in this case Roundcube).

I installed PHP through Nginx first. The easiest way to get a PHP environment up and running on Nginx is to use the Ubuntu packages:

php5-cgi
php5-common

For additional functionality such as PostgreSQL support you can install the package:
php5-pgsql

I then configured nginx with the following script:


server {
listen 80;
server_name webmail.example.com;
access_log /var/log/nginx/access.log;
log_subrequest off;

location / {
root /www/webmail.example.com;
index index.php;

location ~ \.php$ {
include fastcgi_params;
fastcgi_pass localhost:9000;
fastcgi_param SCRIPT_FILENAME /www/webmail.example.com/$fastcgi_script_name;
}
}
}

I then created the script: /etc/init.d/php-fcgi
BIND=127.0.0.1:9000
USER=www-data
PHP_FCGI_CHILDREN=15
PHP_FCGI_MAX_REQUESTS=1000

PHP_CGI=/usr/bin/php-cgi
PHP_CGI_NAME=`basename $PHP_CGI`
PHP_CGI_ARGS="- USER=$USER PATH=/usr/bin PHP_FCGI_CHILDREN=$PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS=$PHP_FCGI_MAX_REQUESTS $PHP_CGI -b $BIND"
RETVAL=0

start() {
echo -n "Starting PHP FastCGI: "
start-stop-daemon --quiet --start --background --chuid "$USER" --exec /usr/bin/env -- $PHP_CGI_ARGS
RETVAL=$?
echo "$PHP_CGI_NAME."
}
stop() {
echo -n "Stopping PHP FastCGI: "
killall -q -w -u $USER $PHP_CGI
RETVAL=$?
echo "$PHP_CGI_NAME."
}

case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
*)
echo "Usage: php-fastcgi {start|stop|restart}"
exit 1
;;
esac
exit $RETVAL


After php was up and running I installed courier:
apt-get install courier-imap courier-imap-ssl

I then downloaded roundcube and configured roundcube as necessary.

Google Apps Split Delivery for Email - Have your cake and eat it too

Split delivery for e-mail is when you have a single piece of mail but want a copy of it sent to multiple destinations. There are a couple of reasons you would want to do this:

- You are getting ready to migrate to Google Apps but don't want to go all in yet with your current e-mail server. This is understandable since you want to test out Google Apps first to see if it will work.

- You like Google Apps but don't want to pay the yearly fee. You rather just stay under the limit of the free accounts but still want to have e-mail accounts on your domain (i.e. @example.com).

- You have other special circumstances where you want a copy of all the mail that comes in and have it delivered to some other server.


This post is actually more toward the 2nd point. 90% of my e-mails are on Google Apps but there is a remaining 10% that I rather not have a Google Apps account. However I still want them to have e-mail via some webmail client.


To get this working I am using Ubuntu with Postfix running the primary mail server for example.com. I set the MX records for example.com to this:
10 mail.example.com
20 ALT1.ASPMX.L.GOOGLE.COM

Pretty straight forward so far. The trick is that you need to get Postfix to forward a copy of all Google Apps mail to their mail servers. To do this I use Postfix's Before-Queue Content Filter: http://www.postfix.org/SMTPD_PROXY_README.html.

This allows me to create a SMTP server that Postfix will delivery a copy of the mail to Google Apps. In the file /etc/postfix/master.cf I put the following at the end:

# =============================================================
# service type private unpriv chroot wakeup maxproc command
# (yes) (yes) (yes) (never) (100)
# =============================================================
#
# Before-filter SMTP server. Receive mail from the network and
# pass it to the content filter on localhost port 10025.
#
smtp inet n - n - 20 smtpd
-o smtpd_proxy_filter=127.0.0.1:10025
-o smtpd_client_connection_count_limit=10


This makes Postfix delivery a copy of the incoming mail to the SMTP server at 127.0.0.1:10025.

Okay but now you are asking where do I get an SMTP server to do the processing? It so happens Python comes with a smtp server library. I wrote a script that basicallyhttp://www.blogger.com/img/blank.gif inherits the SMTPServer (called CustomSMTPServer). It also implements the one command that is expected by Postfix (EHLO) because Postfix actually speaks ESMTP. I did this by subclassing the smtpd.SMTPChannel class. One caveat is that I had to use the _SMTPChannel__variablename syntax because some variables like fqdn and greeting were made private by the SMTPChannel class. So with Python you use the special syntax of prepending the class name to access it. This is generally bad practice but in this case it was all I had.

You can download the script here:
http://dl.dropbox.com/u/2177278/pymailforwarder.py


Simply run the script to start a basic SMTP server listening on port 10025 and localhost. The script simply accepts a piece of mail and forwards it to Google App's mail server.

So what does this let you do? In my case this lets me run Roundcube, Horde, or SquirrelMail for users that don't need a Google Apps e-mail account. For those that do I simply create that user on Google Apps.

Things to watch out for with this type of deployment:

- You have to edit /etc/postfix/main.cf and add the value
local_recipient_maps =
(Yes that is a blank or equals nothing). This makes Postfix accept the mail even though the recipient is not in the list. This can be bad. Postfix says this in the documentation:

With this setting, the Postfix SMTP server will not reject mail with "User unknown in local recipient table". Don't do this on systems that receive mail directly from the Internet. With today's worms and viruses, Postfix will become a backscatter source: it accepts mail for non-existent recipients and then tries to return that mail as "undeliverable" to the often forged sender address.

To get around this you should really have an alias map file. For temporary testing though this setting will work wonders.

- The Python SMTP relay should not be exposed to the internet. It listens on localhost but ideally it would be nice to modify the script to accept authentication of some sort.

Locales in Ubuntu

The list of locales that you have installed on a system are in the directory:

/usr/lib/locale

To generate a locale you can run:

locale-gen en_US.UTF-8

This is very useful when you need to generate a UTF-8 locale.