Why Technologies Fail

Technologies usually fail in a gentle way, compared to other human endeavors. A new technology supplants an existing technology. To look at recent trends, the buzzwords ten years ago were "blog" and "myspace" and the buzzwords of today are "twitter" and "facebook". The respective intents -- communication and social networking -- are parallel across these, but we have gotten an upgrade in medium. Compared to contemporary advances in software development, the differences between the technologies is not marked. Blogging has become a paradigm of its own, but has failed in the sense that it no longer is predominantly used for the quick (and often pointless) communication provided by twitter (this blog being an exception). MySpace, except for music, has failed almost completely in comparison to FaceBook. While both sites hired prominent engineers and MySpace had a huge early competitive lead, FaceBook is this one of the most traficked sites on the internet, while MySpace isn't in the top 20.

So we can ask then, why do technologies fail so slowly or why are the originators of original technologies so seldom those that ride the wave when the obseleting technology is evident?

Much of this comes from a fear of innovation. Due to financial and social pressures, maintainence of the status quo supercedes in priority the investment needed in driving technologies forward. Innovation needs two stimuli:

1. Research and development of superceding technologies
2. Development of infrastructure to perfect process

Since the one predictable facet of the future is that it will be unknown, the state of the art must be immersed. It is an oft-mistaken notion that future technologies appear abruptly. Except in certain cases, this is mistaken. As a historical example, Thomas Edison is said to have invented the electric light and that overnight the world was changed. In fact, several forms of electric lighting existing before Edison's incandescent bulb, which was not an unknown idea to scientists. Likewise, today the technologies that will drive the future of the web are not strangers to those in the field. But bringing them to market in a way usable to consumers is what changes the world.

Since engineers at the forefront of their industry are generally very bright, and since known good development methodologies are common knowledge, why does this process seem so slow and jagged to consumers? We've been promised the electric car, transparent computer interopability, and houses in space for years now. None of these is beyond the reach of the state of the art. And yet they are not palatable to the market, despite high demand.

In order to extend the state of the art, the state of the art must be full. That is to say, innovation saturates. In order to develop superseding technologies, the existing technology must flood the market. Until then, the practical points of implementation will not be perceptable as needs.

Taking each of our examples, the electric car has not become a dominant technology because of the investments of a legacy dominant technology, that of fossil fuels. The energy industry, first saturated by fossil fuels in the 19th century, has become a global power. Like all powers, a key component of it is preservation of its prevalence. So prices are kept (yes, even today!) artificially low. As a byproduct (or conspiracy, by others' counts, but quite inaccurately for the most part), gasoline powered cars become more practical, even if the absolute cost to the economy is a lack of innovation and slower progress in adopting future forms of transportation. We see a technology -- fossil fuels -- that due to a overarching demand for its product can continue such demand until scarcity forces alternatives to dominate. Its infrastructure is its product, and there can be no real competition because its infrastructure has such a high total capital value of the world.

What of the promise for computers to help organize our lives? Technologies exist today to put documents on the internet, to communicate about whatever we want to talk about, and to keep track of our schedule. However, while in one sense it can be said that the integration of these technologies is quite innovative (e.g. adding an event to your calender due to an ICS attachment in your email that was automatically rendered from the sender's message), the practical reality is the lack of seemless integration makes computers an appliance for doing a discrete number of things and less about engaging in a process that is information exchange. In software development, process equals product: the outcome of a seemless communication platform, allowing people to manifest intent with their machines, is symmetric with the manifestation of intent via the software development process. In this sense, programming is speaking very specifically to a computer in order to get one's ideas across. Just as there is no unbiased reporter, the code that drives computer programs is a direct manifestation of the engineer's comprehension of the problem to be solved.

Since so many engineers are really good at what they do, and since there is a large group of integral known problems that prohibit a direct path forward, why aren't these problems solved? Again, this is a result of a lack of saturation of the state of the art. On one side, this takes the form of consumers of technology being most keenly aware of the problems that confront them everyday. If a technology is satisfying, then there is no need to innovate. If it is known how to work around a deficiency, this is often seen as preferable to an expensive upgrade often of unknown quality and ease of adoption. Consumers and producers of technology don't move forward because the path forward is laden with holes.

What fills in these holes, ultimately, is investment in infrastructure. A system with an ideal infrastructure makes solving problems always a step forward. This is the densest communication vector. A system with a non-ideal infrastructure forces side-innovation. Steps forward may be achieved but only by concurrent side stepping to avoid difficulties. For a pure consumer, side stepping is only undesirable in that it causes displeasure. For a producer whose goal is furtherment of the technological state of the art (while, notice, like oil companies, some producers are not), this is a direct impediment to progress. A poor infrastructure allows only side innovation. The current state of technology is such that maintaining its state requires full-time investment.

Investment in infrastructure is often overlooked in the drive for innovation. Expectations are such that a great idea is something to implement and profit from. Such a discrete view of technology is ultimately noninnovative as it produces no direct growth, only an instance of a technological implementation. In order to grow, the fires of innovation must be continuously stoked. Problems must be continually fixed, build systems must be continuously kept up with the holes on the horizon. For technology, process equals product. If "progress" is just one-offs or work-arounds, the producer is pigeon-holed into a well of product maintainence instead of the ability to create.

Which brings us to houses in space. Space-faring technology has not become consumer accessible in over half a century of innovation, again by the best and brightest. The Apollo moon landing is held as one of the wonders of the modern world. Yet, in our history, it remains but a blip. Particularly in America, the cost as a response to demand of putting people into space has risen over the years despite investment in space technologies. The potential return on such an investment is vast, and yet it has met with no takers. The problem is that there has been no investment in the infrastructure needed to make this a reality. Scientists play with state of the art probes and ion drives, but the focus is not on beginning to fill the market, but the seeking of esoteric data with the unreasonable hope that the studies towards the mysteries of the universe can somehow be marketed. Sadly, this self-promotion has not received the same investment as professional sports. Houses in space will come, but only when we realize they are ours to have