Comments on: The Erlang Challenge

By: developers.hover.in » Somethings to rejoice about

developers.hover.in » Somethings to rejoice about — Mon, 18 May 2009 08:24:29 +0000

[...] compared some techniques from many pmap implementations out there, and here are some [...]

By: SQL Tutorials

SQL Tutorials — Fri, 01 May 2009 02:37:19 +0000

Does anyone know if there is another language or set of commands beside SQL for talking with databases?

I’m working on a project and am doing some research thanks

By: SQL Tutorials

SQL Tutorials — Fri, 01 May 2009 02:37:02 +0000

You know, the thing about SQL is, that there is virtually nothing that can replace it.

Does anyone know if a substitute exists for sql? I mean besides MS SQL and Oracle and all that jazz. Thanks.

By: The (Unofficial) Erlang Blog » Blog Archive » Erlang Challenge Announced

Sun, 17 Feb 2008 22:59:12 +0000

[...] last week’s Arc release and subsequent Arc challenge, Yariv Sadan (ErlyWeb) has announced the Erlang Challenge. The goal is to design & program an advanced “parallel map” function that can [...]

By: Martial

Martial — Mon, 11 Feb 2008 10:19:48 +0000

I do like Erlang programming but the Smalltalk continuation-based solution with Seaside looks like the simpler way:
http://www.lukas-renggli.ch/blog/take-the-arc-challenge
Erlang has the advantage of low memory consumption and ‘infinite’ scalability.

By: Yariv’s Blog » Blog Archive » More Erlang Fun: Distributed, Fault Tolerant MapReduce

Mon, 11 Feb 2008 02:13:05 +0000

[...] the Erlang Challenge posting, I showed how to implement a simple distributed parallel map function. You can pass it a [...]

By: Yariv

Yariv — Sat, 09 Feb 2008 23:42:29 +0000

@Bob There is a slight difference between the use of SpawnFun in this example and calling pmap(Fun, List, [node() | nodes()]) — in this example, if the list of nodes is not empty, the current node won’t be included in the call to spawn(). Obviously it’s not a very important difference — when I designed the example, I was just trying to illustrate how in Erlang, spawning processes on remote VM’s is as simple as spawning them on the local VM.

@wingedsubmariner I know about plists. It’s great.

By: wingedsubmariner

wingedsubmariner — Fri, 08 Feb 2008 23:31:39 +0000

I wrote a library to do this a while ago:

http://code.google.com/p/plists/

It will also parallelize many other operations. It supports proper error handling as well, trying to make itself as invisible as possible, while still not sending large lists across processes except when necessary (separate heaps == potential inefficiency). It’s capable of autodetecting the number of of threads Erlang is configured to use on each node and to spawn that many processes — effectively maxing out a machine.

It’s over one thousand lines of code though, so it fails succinctness.

By: Xichekolas

Xichekolas — Fri, 08 Feb 2008 22:55:50 +0000

@sjs: Ahhh, yeah. That makes total sense.

By: Bob Ippolito

Bob Ippolito — Fri, 08 Feb 2008 21:57:04 +0000

It seems most people are missing the fact that this will distribute the work over all nodes (usually nodes are run on different machines). The data parallel languages cited in the comments will do a great job of this too, almost certainly better than Erlang, but I believe that most of them will only do computation on the local machine.

The equivalent single-node pmap is for Erlang is short too, but less interesting (remove all but the last 3 statements, change SpawnFun to spawn, and remove Nodes from the argument list).

There isn’t really a different method for process creation on the local node versus any other node. It looks like SpawnFun is just handling a case that it really doesn’t need to (bad input). spawn(node(), F) would work just fine (node() is the local node). spawn(F) is just shorter to write, but it’s not necessary.

Realistically you’d probably call pmap(Fun, List, [node() | nodes()]) which would distribute the computation over all nodes that the local node can see, including itself node. He could’ve just as well checked for an empty Nodes and used [node()] instead.

As far as mailbox size goes, it depends on what you mean by “properly”. The answer is probably not — this isn’t an optimal pmap for very large lists due to that issue.