Comments on: How to Use Concurrency to Improve Response Time in ErlyWeb Facebook Apps

By: S

Fri, 25 Jan 2008 04:31:50 +0000

But python is not limited to OS thread! What about stackless and/or greenlets. Those do not cost as much as OS thread and do the same as erlang.

By: Yariv

Yariv — Wed, 23 Jan 2008 06:00:40 +0000

@Igor It’s not just the capabillity of spawning processes with few LOCs that makes Erlang interesting. It’s also 1) the fact that those processes are very cheap and there’s no scalability or performance risk in spawning them at request processing time (the same can’t be said for OS threads, which Python and Java use — Google “apache vs yaws” to see what I mean) and 2) the inherent safety in the Erlang concurrency model which guarantees that multiple threads would never try to modify the same data because all data is immutable. These two factors effectively give you a carte blanche for spawning as many processes as you want without worrying about artificial scalability bottlenecks and thorny concurrency issues.

By: Bryan

Bryan — Wed, 23 Jan 2008 01:59:15 +0000

Totally with you, Yariv. Many actions that might take a lot of time, I’ve been spawning new processes for. Updating Facebook profiles, or even just flushing content to the database - as long as I can show the user a reasonably consistent view (by leaving the in-memory data temporarily out of sync), I let those updates happen in the background.

@Igor: Erlang *does* help with the failure cases. Even beyond Kevin’s note that inter-process communication is simple, Erlang also has a method for linking processes together, such that even if the background process dies unexpectedly, you can handle the failure in a live process that might know what to do about it.

By: Kevin Smith

Kevin Smith — Tue, 22 Jan 2008 21:32:56 +0000

@Igor - Making the async call reliable in a Java webapp is, I think, a fairly ugly proposition. Actually, I think its ugly in any language that doesn’t have Erlang-like message passing semantics.

In one of these other languages, say you’re off in another thread making FB call and Bad Things Happen. How do you propagate the failure back to correct owner? I know how to do that in Erlang - just send a message. Java or Python will require me to write more infrastructure to handle the failure correctly.

By: Igor

Igor — Tue, 22 Jan 2008 19:37:53 +0000

Most languages these days include capabilities for kicking off background threads. In Java it takes half a dozen lines of boilerplate code (the number of lines is testament to languages verbosity). Python is comparable to Erlang.

The hard problem is making it reliable. What happens if FB is not available. What about the case where machine running background task crashed? Does Erlang help with any of these?

By: Harish Mallipeddi

Harish Mallipeddi — Tue, 22 Jan 2008 11:28:54 +0000

Forget about being responsive, you absolutely have to make those profile FBML and newsfeed calls asynchronously because Facebook imposes very stringent timeout periods. I was implementing my app in Python/Django and had to resort to writing my own little daemon to handle these calls.