We are so enamored with technology today, what it can do, the excitement it brings to us as technologists that we have lost sight of the fact that technology isn't the end game for a business. Why? Because if your organization could figure out a way to run their business so that it wasn't dependent upon technology they would do it in a heartbeat. The people who own and run businesses don't care about the underlying technology, even those who classify themselves as technology companies. At the end of the day technology enables the business to develop product. They aren't in the business of developing technology as end product and I think that goes for companies who deliver 'technical' products.
Many teams I've worked on in the past 10 years have isolated themselves from the organization, instead telling stakeholders they were focusing on technical excellence. Leaving the stakeholders wondering why they weren't delivering what they wanted all in the name of technical excellence.
I worked on one project that was so late that the business ending up exiting the market that they were in. In response the Development manager said if only the stakeholders knew how cool the technology was that they had worked on they would change their mind. Really? They could care less, all they wanted was a product that would keep them in the market and making money.
Though the amount of information we can process has exploded we aren't really solving really new problems, they are just much larger in scale and are required with much higher availability. At the end of the day business runs on information that drives product decisions that build value. Like any other part of the organization we are responsible for delivering value, not technology.
If we could deliver data to our business without all of this cool technology like hadoop, etc....they would be just as satisfied.
We should spend much more time on how we can deliver what we need for the business with less over building systems that continually build in complexity because we aren't product focused. Think about it, organizations that deliver at scale, such as car manufacturers continue to refine their processes so that complexity is removed, whereas we seem to have to add complexity in order to deliver value. They add technology where it makes sense not just because it's the latest thing.
Technology isn't important but our technical product is. We have architects, tech leads, dev managers, however I don't often see them actually defining the technical product that the business needs, rather they evaluate specific pieces of it, SQL server over Oracle, Java over .Net, etc.....and with each iteration individual personalities can influence the type of technology being implemented, often times in direct competition with the documented tech stack that company operates in.
What we end up with is a technology product that doesn't scale well and adds cost to the organization in both resources and revenue delays. When confronted with the mess we've made we tell the organization that can solve the problem with a new platform. Unfortunately we often fail to fully deliver on that promise for the same reasons that we got into the situation in the first place.
This is not to say that we don't try to build systems right. Executives also need to bear responsibility for not providing the Engineers you have hired the support and time necessary to actually build technical products that deliver value to your business product, they are one and the same in today's world.
When your technical team is forced to meet dates, the inherent compromises that need to be made typically result in short cuts to your technical product in order to deliver the most for your business product. You can do this in the short term, but if you don't provide your teams time to shore up these short cuts your technical debt continues to grow. And like any debt you eventually have to pay it. Pay it at the end of every release like a credit card you won't incur finance charges. Continue to pay just the minimum and you eventually can go bankrupt.