Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

monoids are perfectly parallelized. map/reduce are built on top of that.


Parallelising things is the easy part. Making them run faster than the single threaded version is often the harder part because the overhead will eat all your gains if your workload isn't large enough.

The easy cases are already covered by things like OpenMP and the difficult problems also usually benefit from being written in a language that allows you to tweak your memory layout and allocation patterns manually.

I just wish that higher level languages would give me this freedom without sacrificing productiviy or safety. I'm not even asking for completely unreasonable things. At the bare minimum all I really need are off gc heap allocation (with reasonable safety via reference counting or whatever rust does), value types and a way to launch threads that bypass the garbage collector stop the world pause. It's just a checkbox of features and the latter is not that important because you already can sort of do that by calling into C and launching an OS thread. For some reason though nobody is checking those boxes.

I still stick to C++ and C but it gets old to spend hours fixing memory leaks and other possibly security related bugs from other developers.

I feel like I'm writing too much semi-offtopic things just because I have no other place to express them.


Which is great, if you're theorising about why map/reduce work or trying to recreate it from first principles. I don't think many of us are trying to do that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: