Concurrency and Erlang

A Presentation by Andre Pang

By 2009, modern CPUs will possess over a teraflop of computing power. However, they'll also be divided up into 20+ CPU cores that are capable of executing 80+ hardware threads concurrently. How do you deal with such a beast? Even today, concurrency is arguably the most complex problem that programmers have to face, and it'll only get worse: the pipe dream of auto-vectorising compilers that will automatically parallelise tasks won't be arriving anytime in the near (or far) future.

In this seminar, we'd like to introduce Erlang: a programming language based on the functional programming paradigm specialised at writing large, concurrent applications. It has a number of unique features, making it extremely compelling for writing servers that are required to be high-performing, have ridiculous availability (99.999% uptime or better), can scale well to deal with huge load, and can be distributed (even geographically). In addition, since it has an industrial focus, it has an excellent standard library that a lot of other more academic languages lack (such as Ocaml and Haskell), so you can code in it without having to hand-roll your own XML or HTTP parser. Erlang is quite a different beast from traditional application development languages such as C/C++/Java, and also takes a different approach to solving problems than the plethora of dynamic languages such as Python, Perl and Ruby. With an emphasis on real-world usage, we'll be using ejabberd -- the Erlang-based Jabber server that powers jabber.org -- as a running example of how you can build massive, reliable systems that regularly serves tens of thousands of users (with less than half a dozen programmers).

Even if you don't use Erlang, you'll be able to take away a lot of good practices and techniques from the talk and merge them into your own project so you can better deal with concurrency (such as preferring message passing for concurrency vs using shared mutable state), and gain a better understanding of systems-level debates such as why userland thread scheduling is actually useful.

Note: I have my own homepage now for the talk, at http://www.algorithm.com.au/talks/concurrency-erlang

Direct link to video