I think this is a great question. So, I've begun a series of blog posts about it here.
Dmckee's answer is correct in the narrowest sense. Let me rephrase in my own words here, implicitly including some of the comments:
There is no value in parallelizing
operations that are not CPU bound.
There is little value in parallelizing
operations that are only CPU bound for
short periods of time, say, less than
a few hundred milliseconds. Indeed,
doing so will most likely cause a
program to be more complex, and buggy.
Learning how to implement fine grained
parallelism is complicated and doing
it well is difficult.
That is true as far as it goes, but I belive the answer is richer for a broader set of programs. Indeed, There are many reasons to use multi-threaded, and then implicitly multi-core techniques in your production applications. For example, it is a huge benefit to your users to move disk and network I/O operations off your user interface thread.
This has nothing to do with increasing the throughput of compute bound operations, and everything to do with keeping a program's user interface responsive. Note, you don't need a graphical UI here - command line programs, services, and server based applications, can benefit for this as well.
I completely agree that taking a CPU bound operation and paralyzing it can often be a complex task - requiring knowledge of fine grained synchronization, CPU caching, CPU instruction pipelines, etc. etc. Indeed, this can be classically 'hard'.
But, I would argue that the need to do his is rare; there are just not that many problems that need this kind of fine grained parallelism. Yes! they do exist and you may deal this this every day, but I would argue that in the day to day life of most developers, this is pretty rare.
Even so, there are good reasons to learn the fundamentals of multi-threaded, and thus multi-core development.
- It can make your program more responsive from a user perspective by moving longer operations off the message loop thread.
- Even for things that are not CPU bound, it can often make sense to do them in parallel.
- It can break up complex single threaded state machines into simpler, more procedural code.
Indeed, the OS already does a lot for you here, and you can use libraries that are multi-core enabled (like Intel's stuff). But, operating systems and libraries are not magic - I argue that it is valuable for most develops to learn the basics of multi-threaded programming. This will let you write better software that your users are happier with.
Of course, not every program should be multi-threaded, or multi-core enabled. It is just fine for some things to be implemented in a simple single threaded manner. So, don’t take this as advice that every program should be multi-threaded – use your own good judgment here. But, it can often be a valuable technique and very beneficial in many regards. As mentioned above, I plan on blogging about this a bit starting here. Feel free to follow along and post comments there as you feel inclined