Urban Hafner

On node.js: Part 0 - Introduction

[Posted by Urban Hafner on 26 Sep 2009]

node.js is JavaScript for the server side. There are other server side JavaScript frameworks (like narwhal or helma), too. But I don’t like them and here’s why: All of these other solutions try to introduce threads as a way of allowing many different connections (if we assume we are talking about a web server). This is bad in several ways:

It’s threads!

Using threads is very, very hard in any language. It’s almost impossible to write a program that uses threads (in a language that has shared mutable state). Also, threads are rather expensive. Both in startup and in memory footprint.

It’s not the JavaScript way

Using threads isn’t really the JavaScript way. JavaScript in the browser doesn’t have threads. Introducing them on the server side doesn’t really “break” the language, but it doesn’t really work the way JavaScript normally works. So, you’re totally switching the paradigm and you can’t really use the knowledge and intuition you already have.

The Event Loop

So what’s the “proper” way to do things in JavaScript? Events, of course! When you run JavaScript in the browser it isn’t in control of the browser, but it only runs in certain intervals or when certain events happen. This works, because the browser has an event loop running that switches between all the different JavaScript code snippets that are supposed to run at the moment.

How does that translate to the server side?

So, why would you need that on the server side? Well, if you’re trying to work on the server side, you need access to the hard disk or have network connections. Compared to CPU and RAM usage that’s really, really, really, …, really slow (many orders of magnitudes in fact). And if you’re trying to run a web server for example that’s bad, because you don’t have threads and you would block all other connections.

And that’s where node.js steps in. For example, if you’re trying to open a file node.js doesn’t block. You just ask node.js to open the file and tell you when it’s done:

node.fs.readdir("/").addCallback(function (files) {
  puts(files);
});

If you’re coming from an language but JavaScript this seems very weird. But it has the following advantages:

It works just like JavaScript in the browser. You attach events just like you do there.
It gives you a form of concurrency (at least for I/O) without the headaches of using threads.

Now for the big example: The web server

Now that we’ve heard of all the good things that is node.js, let’s see a bigger example. How about a web server that returns “hello world”:

node.http.createServer(function (request, response) {
  response.sendHeader(200, {"Content-Type": "text/plain"});
  response.sendBody("Hello World\n");
  response.finish();
}).listen(8000);

Wait a minute! That’s just too short, that’s not a real example. OK, well, … you’ll have to wait for the next part of the series where I’ll walk you through some bigger example (probably a video streaming site or something similar).

But I want more, NOW!

But if you’re interested in more code you can find more information in the following places:

If there are any questions don’t hesitate to ask on the mailing list.

Tags nodejs, node.js, javascript, event, events, io, v8, server, server-side, server side

Real validates_uniqueness_of in Rails

[Posted by Urban Hafner on 19 Aug 2009]

We all know that validates_uniqueness_of doesn’t provide a real guarantee for unique database entries. For most cases though it’s good enough. But there are cases where you either get so much traffic that this problem becomes an issue, or it’s absolutely critical that the values in the database are unique.

If this is the case there’s no way this can be done entirely in Rails. After all, how should Rails know that the values in the database are unique? That’s the job of the database. So we have to go that route. In my case I had to generate a unique token for a user. The code started out as:

class User < ActiveRecord::Base

  validates_uniqueness_of :token
  before_validation_on_create :generate_token
  
  private
  
  def generate_token
    self.token = rand(36**10).to_s(36)
  end
end

In this case I might run into the above mentioned race condition. Luckily though I can also just retry saving the record and (hopefully) will end up with a unique token if the saving fails the first time.

So first, we’ll need to ensure that the database keeps the entries in the token column of the user table unique. To ensure this, we’ll add an unique index on this field:

class AddUniqueTokenIndexToUser < ActiveRecord::Migration
  def self.up
    add_index :users, :token, :unique => true
  end

  def self.down
    remove_index :users, :token
  end
end

Now, whenever Rails tries to save an row into the database that contains a duplicate entry for the token column a ActiveRecord::StatementInvalid exception will be thrown. All though that’s not ideal (as this exception might have an entirely different cause) we can catch that exception and try saving again.

We’d have to that that in every place where we want to create/modify a User though. So wouldn’t it be much nicer if save (and save!, …) would just do that for us? Indeed it would be nice so let’s get our meta programming toolkit out and write the code for that:

# Use "extend DbUnique" to get this functionality
module DbUnique
  
  DUPLICATE_ERROR_MESSAGES = [
    "Duplicate entry",           # MySQL
    /column (\w+) is not unique/ # SQLite
  ]
  
  def db_unique(*methods)
    methods.each do |method|
      module_eval <<-"end;"
        alias old_#{method} #{method}
        def #{method}(*args)
          retries = 0
          begin
            old_#{method}(*args)
          rescue ActiveRecord::StatementInvalid => error
            if error.message =~ Regexp.union(*DUPLICATE_ERROR_MESSAGES)
              retries += 1
              if retries < 5
                retry
              else
                raise
              end
            else
              raise
            end
          end
        end
      end;
    end
  end
  
end

We can then just include that code into our ActiveRecord model and overwrite the methods we want with versions that retry writing to the database a number of times:

class User < ActiveRecord::Base

  extend DbUnique  
  db_unique :save, :save!
  
end

Tags ruby, rails, validates_uniqueness_of, unique, uniqueness

Turning man pages into nicely printable PS files

[Posted by Urban Hafner on 16 Jun 2009]

As seen on Twitter:

man -t program > program.ps

Turns the man page of a program into a nicely formatted PostScript file.

Tags man, ps, print

Relaunched my Blog using Jekyll

[Posted by Urban Hafner on 03 Jun 2009]

After Euruko I started rewriting my blog using Jekyll. That’s because it generated static HTML files that can be easily hosted on my TextDrive shared hosting plan (i.e. no expensive Ruby processes).

Have a look at my github repository and the Jekyll Wiki for information on how to do it yourself.

Tags software, blog, jekyll, ruby

Announcing Dacelo

[Posted by Urban Hafner on 15 Mar 2009]

With this post I’m announcing Dacelo. My personal Blog engine. It’s in its infancy right now but I hope to have a useable version in a few weeks (Yes, that long as I don’t have much time).

It’s mostly for my personal use as Typo is just getting to heavy and feature rich for me. That means that it’s goal is to be as minimal as possible and that it should only have the needed features, where needed is defined by me ;) The current feature list looks like this:

Blog posts and comments
Login via OpenID
Textile formating of the posts
Secure admin area (using https)
Easy way to setup your blog as your OpenID URL (whatever the terminology for that may be)
Tags and a tag cloud in the sidebar
Highlighting for your code snippets
Live preview of your edits on the posts
Atom/RSS feeds for the blog and comments (and maybe for the comments on a blog post)
Comment management (i.e. Spam protection, maybe Akismet support)
And anything else that I’ve now forgotten and will “need”.

Feel free to check out the code at http://github.com/ujh/dacelo/tree/master and the tickets at http://www.assembla.com/spaces/dacelo.

Tags ruby, rails, blog, engine

Multisite ActionMailer Ruby on Rails plugin

[Posted by Urban Hafner on 26 Feb 2009]

I just released a new Ruby on Rails plugin. Here’s the contents of the README:


If you want ActionMailer templates handled the same way as views when
you are using the rails-multisite plugin use this plugin.

Right now it only works in conjunction with the msales fork of
rails-multisite [1] and is in conflict with action_mailer_layouts [2]

[1] http://github.com/msales/rails-multisite/tree/master
[2] http://github.com/willcodeforfoo/action_mailer_layouts/tree/master

You can get it at http://github.com/msales/multisite_action_mailer/tree/master. Patches welcome.

Tags ruby, rubyonrails, rails, plugin, git, github, actionmailer, mail, multisite

Rails partials and their dependencies

[Posted by Urban Hafner on 21 Feb 2009]

Have you ever wondered how your Ruby on Rails views and partials relate? Did you want to know which views change when you change that one partial?

I don’t know about you but I had that problem on a project recently. That why I created a Rails plugin called partial_dependencies. It creates a graphical representation of your views and partials for you.

Get it now while it’s still hot :)

Tags rails, rubyonrails, ruby, partials, graphviz, dependencies, plugin

Fink, Mysql, and Mysql Ruby gem

[Posted by Urban Hafner on 13 Jan 2009]

Installing the ruby mysql gem with a mysql installed via fink worked for me using the following command:

sudo gem install mysql -- --with-mysql-include=/sw/include/mysql \
 --with-mysql-lib=/sw/lib --with-mysql-config=/sw/bin/mysql_config

Tags fink, ruby, mac, osx, rubygems

Bad bad Firefox

[Posted by Urban Hafner on 12 Jun 2008]

A few days ago I bought a new DSLR (a Nikon D40) and started to upload more pictures to flickr (see http://flickr.com/photos/ujh for my photo stream). I also started looking at much more pictures on flickr from other people. And in one of those the photographer was complaining how bad the pictures looked on flickr. In a comment someone else explained that the problem is not flickr but Firefox! And true enough. The difference between Firefox and for example Safari is so big even I could see it! So if you like to look at pictures don’t use Firefox.

Tags firefox, browser, flickr

For all the bass players out there ...

[Posted by Urban Hafner on 28 Dec 2007]

I just happened to run across a guy recording himself playing bass on youtube. Here’s a sample:

It makes we want to start playing again. Maybe I can get somewhere close (well maybe not).

Tags bass, youtube

nodejs node.js javascript event events io v8 server server-side server side ruby rails validates_uniqueness_of unique uniqueness man ps print software blog jekyll ruby ruby rails blog engine ruby rubyonrails rails plugin git github actionmailer mail multisite rails rubyonrails ruby partials graphviz dependencies plugin fink ruby mac osx rubygems firefox browser flickr bass youtube cgos go ai game computer computer go ruby computer-go baduk weiqi go computer computer go ruby computer-go baduk weiqi ruby conferenz euruko vienna austria fun video greece fire burn death destruction german democracy grundgesetz lighttpd port webserver podcast ITConversations ratings flickr photo web 2.0 fun pragmatic programmers erlang book mac tiger novelties developer osx youtube john coltrane coltrane video planet computer go baduk weiqi go ai bot baduk weiqi housebot computer-go mac osx apple lookupd web typo blog textdrive stable memory web fun meme munich cocoa apple macosx cocoadevhouse podcast nature nature magazine science book review chad fowler pragmatic programmer IT offshoring genomics metagenomics paleogenomics sequencing mammoth science bioinformatics paleontology article ipod reboot bioinformatics CTM exercise book mozart oz dataflow concurrency CTM book exercise mozart oz programmed trigger higher-order programming CTM exercise book mozart oz thread scheduling podcast german mac CTM exercise book mozart oz dataflow wait CTM exercise book mozart oz concurrency dataflow variable CTM exercise book mozart oz concurrency fibonacci CTM book exercise thread garbage collection mozart oz CTM book exercise mozart oz thread semantics bioinformatics wiki evolution genetic marker human migration bioinformatics genographic book CTM mozart oz exercise quicksort sorting divide conquer difference list book CTM mozart oz exercise queue FIFO book CTM mozart oz exercise list matrix book CTM mozart oz execution time difference list book CTM mozart oz exercise difference list book CTM mozart oz exercise list book CTM mozart oz exercise iterative dataflow book CTM mozart oz exercise iterative book CTM exercise mozart oz termination book CTM exercise mozart oz state invariant CTM exercise book mozart oz iterative computation book CTM oz mozart exercise book CTM oz mozart exercise book CTM oz mozart exercise book CTM oz mozart exercise book CTM oz book CTM oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz fun NARCL project lisp machine learning data mining CTM book oz NARCL project lisp data mining machine learning fun web tool lisp book fico project ruby havannah board game fico project havannah ruby board game idea concurrent lisp del.icio.us NARCL lisp project data mining machine learning CTM wiki CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz CTM book oz