Defenestrating Thoughts from the Bivouac

I recently read through bits and pieces of Parallel and Distributed Programming Using C++ because I wanted to brush up on some parallel computing concepts. Given how important this topic is now, I was surprised to find very few books written on this topic.

In software, we’re finding that our free lunch is pretty much over. As long as processors (and the rest of the system) got faster and faster, we didn’t have to do anything to make our software faster, rather just go along for the ride. In fact, it’s pretty common in any software shop to hear, “Sure it’s slow now, but when it ships in 2 years it will be fine, because machines will be a lot faster.”

But now raw processor speed (clock speeds, instruction throughput) is not increasing as much, we’re instead getting more than one processor per chip (multi-threading, multi-core). In order to actually use this, we need to design our programs very differently.

On the hyper-threaded or multi-core processors , I see great performance gains when running two CPU-intensive apps. But when the CPUs are doing tasks that were assumed (but not required) to be serial, I actually see some significant performance loss in some cases in parallel, resulting in some bad cache thrashing.

The challenge is that probably around 5 to 10% of programmers understand and can write good concurrent code. As an industry, we’re going to have to put a lot of effort into training developers and the development of libraries and platforms to abstract a lot of the difficulty away.

Given that background, I was disappointed by the textbook. While it was great for introductions to lots of important topics, I didn’t feel that the depth was sufficient. Aside from the physical book form factor, you could easily get the same information if you Google for it.

I didn’t pay for it (yay for libraries!) and I like to read physical books rather than stare at a screen, so I found the book enjoyable, if not directly applicable to anything I actually do. :-)