In computing, if we want to harvest this potential for surprise, see results this interesting, we have to write complex systems.
This doesn't mean we want to write complicated systems.
Wait, isn't that contradictory? Complex systems that are not complicated?
Ah, but that is the whole point of complexity theory.
Herbert Simon described a complex system as being composed of hierarchy and near-decomposable parts. Just as an economy is composed of many humans making decisions independently, a piece of software can be composed of many parts that can't see inside each other. This preserves the potential for producing complex behavior, solving a complex problem. Yet, each part within the system may provide a simple abstraction.
The difference between complex and complicated - or, as Rich Hickey calls it, complected - is intertwining the different modules in the system. An application may consist of many uniform modules, each of which knows about the inner workings of the other -- complicated. Or it may consist of even more modules, each of which interacts only with the abstractions exposed by the others -- simpler, and yet with potentially complex results.
The key is breaking the software into independent modules, each of which says to the others "I don't know how you do what you do, and I don't care." Each module or component can be developed by a separate team with different coding standards or in a different language. Uniformity is compromised *gasp*. No single architect understands all parts of the system. Strict top-down, God-like orchestration is sacrificed. Instead, component interactions are abstract. Teams interact at higher levels, without knowledge of each others' code.
Each component handles versioning, security, releases, backwards compatibility - everything, in its own way. Reuse is at the component level, at the level of release, not at the class or function level.
A system based on independent components like these probably contains more code than a functionality-equivalent system that is orchestrated top-down. That's okay. It has potential to support much greater solution complexity.
A fact of software development:
A fact of software development:
For every 25 percent increase in problem complexity, there is a 100 percent increase in complexity of the software solution. That's not a condition to try to change (even though reducing complexity is always a desirable thing to do); that's just the way it is. This is true because every feature we add potentially impacts every feature we already support. The increase in solution complexity is combinatorial, not linear. The way to combat this is to reduce the complectedness of our software. Stop intertwining all the bits -- a coherent architecture is more efficient in lines of code, but it is inherently limiting. It is limited by what the God Architect can hold in his head.
Break the software into simpler components. Some of these may be open-source components, which someone has to learn well enough to configure and implement. Others will be custom. Break them off into teams and give those teams autonomy. Let each team implement its piece in the simplest way possible for that particular problem.
Meanwhile, the systemwide architects don't know the particulars of each solution. In fact, they should not know the particulars. That would allow them to base solutions on implementation details. Don't let that happen! Raise the level of abstraction. Components must interact with abstractions, not with each other. Knowledge of inner workings of other modules is a negative.
Jeff Bezos forced this strategy at Amazon around 2002 by decree: all teams will interact via service interfaces only; they can use whatever technology they want; and all interfaces must be externalizable, ready to be exposed to the outside world. The result? The largest online bookseller became the largest vendor of cloud computing. Did anyone predict that ten years ago?
When you and I interact, our brains do not interact directly. Rather, we both interact with the physical world. I say something, you hear it. Abstractions on both sides reduce the granularity of the interaction, but maintain the independence of our individual brains. No mind-melds allowed. While this seems limiting, observation shows that amazing unpredictable structures - nations, communities, economies - emerge from these limited interactions.
Complicated systems are limited in growth. Complex systems have even greater potential for growth than the designers of the components conceived. Keep your components simple and independent, and there is no end to the problems we can solve.
 Complexity, A Guided Tour, by Melanie Mitchell, chapter 7
 Facts and Fallacies of Software Engineering, by Robert L. Glass, p. 58