Why We Don’t Teach Parallel Computing in Software Carpentry
Konrad Hinsen recently wrote a blog post that explains why teaching parallel computing with Python is hard. To make a long story short, Python’s multiprocessing module can fail on simple problems in a whole bunch of ways that require fairly advanced understanding to diagnose and repair—and that’s even before you try doing things on Windows.
Do you think that parallel computing is and always will be beyond the scope of Software Carpentry, or is it just waiting for the tools to come along that will make it more manageable? It seems you have at least thought about teaching parallel computing: I for one would love to see how you would have fleshed out the notes at http://software-carpentry.org/4_0/hpc/ .
My experience of parallel computing is limited to multithreading in Java, which I guess comes under “Shared Memory with Threads and Why Not”. Maybe it is “really, really hard”, but I think it is getting easier (with Java’s new executor framework and other toolkits) and more important (with numbers of cores per machine on the increase). The most important lesson, I think, is not to make objects mutable unless they really need it, and only then to worry about synchronization. The problem is that by the time people realize they want their application to be multithreaded they have already created a structure of endlessly flexible, mutable and highly thread-unsafe classes. Is there a way to encourage immutable object design early on even without teaching multithreading? Or am I being hopelessly static-typed and UnPythonic?
@Ben Waugh I don’t know about “always” — I think restricted models of parallelism, like the data parallelism of MATLAB or NumPy, are already very usable — but real MIMD parallelism is intrinsically harder than sequential programming, so I don’t know if it will be in reach of Software Carpentry in my working lifetime. Of course, I’d like to be pleasantly surprised…