John got me incredibly excited about Grand Central Dispatch. I'm almost tempted to start tinkering with Macruby to see if I can't get that functionality without having to relearn C.
And there you have it: a for loop replaced with a concurrency-enabled equivalent with one line of code. No preparation, no additional variables, no impossible decisions about the optimal number of threads, no extra work required to wait for all the independent tests to complete. (The dispatch_apply() call will not return until all the tasks it has dispatched have completed.) Stunning.
It doesn't. It, hopefully, alleviates you from counting on explicit thread numbers and threadpool mamagement - offsetting those to the OS, which presumably knows better since it has a larger scope picture of what is going on in the system.
Basically a threadpool as any other, but managed by the OS - not you.
The whole point of GCD is that it is a central entity that knows how much work the entire system is doing on each individual processor core, and can adjust the number of threads accordingly.
Does your Erlang example take into account the overall work the entire OS is doing on each individual processor core, adjusting the number of threads as the amount of overall work changes — or does it just take the Erlang-program's own work into account?
> The whole point of GCD is that it is a central entity that knows how much work the entire system is doing on each individual processor core, and can adjust the number of threads accordingly.
No. That's a side effect and benefit. The whole point of GCD is that it's trivial to use, therefore making it easy to benefit from multicore machines.
> Does your Erlang example take into account the overall work the entire OS is doing on each individual processor core, adjusting the number of threads as the amount of overall work changes — or does it just take the Erlang-program's own work into account?
In the erlang view of the world, the erlang runtime is the OS. And within that context, Erlang takes in account the overall work of itself (in fact, it more precisely doesn't give a damn: it spawns a scheduler thread per logical core -- or more or less if you ask it to -- and then dynamically maps Erlang processes on these OS threads the way GCD dispatches queued tasks to its threadpool).
The first two letters stand for Grand Central. That's not a side-effect, it's a core purpose, without which it would not do half of what it does, nor do it half as well.
While Erlangers may view the Erlang environment as the universe, it decidedly isn't, as it runs on an OS that does other things as well. Your Erlang example, while I'm sure it's really great for you and other Erlangers, simply doesn't do what GCD does.
> Your Erlang example, while I'm sure it's really great for you and other Erlangers, simply doesn't do what GCD does.
That's very debatable, but thank goodness it wasn't my point in the first place. My point was simply to do what cubicle67's example demonstrates, namely (in his words)
> And there you have it: a for loop replaced with a concurrency-enabled equivalent with one line of code. No preparation, no additional variables, no impossible decisions about the optimal number of threads, no extra work required to wait for all the independent tests to complete. (The dispatch_apply() call will not return until all the tasks it has dispatched have completed.) Stunning.
That's very precisely what the Erlang example does.
As a fan of Erlang, it might have been better to stick to your initial point rather than trying to disagree with tjogin about what is/isn't a central purpose of GCD. That tangent was orthogonal to the syntactic elegance you wanted to highlight.
Of course, if that sort of thing was a core design goal from the outset it damned well should be more elegant! It's much harder to bolt closures on to C, and a bit of syntactic elegance had to be traded off to make it happen, no doubt. I'd say they did a fine job given the constraints. My favorite touch: they chose a glyph for the block pointer that was actually pointy: ^
I don't want to detract from Erlang, because you were right to highlight what it does so well...but it's nice to see these ideas make it into the C foundation of a mass-market operating system in a manner that is both central and grand, even if the syntax isn't quite as tight.
> but it's nice to see these ideas make it into the C foundation of a mass-market operating system in a manner that is both central and grand, even if the syntax isn't quite as tight.
Absolutely. And I find the syntax pretty tight anyway.
Keep in mind, though, that if your DoWork function has some large amount of data in it (like, it's looking up each element of the Data list in a large orddict), every process created by pmap will get a _copy_ of that large orddict. Erlang's awesome, but it's not perfect; one of the current VM decisions is that no memory is shared by two processes, so it's sometimes a problem when blindly trying to convert a map to a pmap.
This is what excited me most too. It is a very similar approach than what we did in a project when we had to add multithreading later on, namely to use tasks and hierarchical queues.
But with GCD, it has the knowledge of the number of cores and running threads and processes so it can allocate the whole system's resources much better.
The other thing I like is how well it is integrated with other new features like blocks, OpenCL and Clang. I expect to see something similar on Linux soon, or (wishful thinking) if the GCD interface was ported to other unices so we could write portable multicore application in a sane way.
Since it doesn't seem anybody addressed your original suggestion of tinkering with MacRuby, it's worth noting that the MacRuby API for GCD landed on trunk 2 days ago. Happy hacking!
