Contrary to how they are perceived in the media as lumbering dinosaurs, most large organisations have loads of clever people and good ideas.The trouble is these organisations can’t deliver against the ideas. The answer is not “Blue Sky” R&D departments or “BlackOps” teams. The answer is to make the BlackOps approach standard; create an innovation platform built from microservices with new services deployed swiftly into production.
Corporate context
Large organisations generally lack the ability to pivot quickly or easily (or both). There are a couple of organisational characteristics that prevent rapid development.
- Technical platform: Large organisations often maintain a single “official” development platform or set of technologies. Typically these involve large enterprise application tiers, with slow release cycles.
- Cross project dependencies: Large organisations often align their teams in such a way as to ensure new ideas that require cooperation from multiple departments/teams are stifled due to aligning projects and/or team resources.
Blue Sky R&D is not the answer
Some large organisations try to address these issues by putting innovation into a “Blue Sky” R&D team. These teams have all the fun but wield little power to push ideas through to production. Three common outcomes of this scenario are:
- Idea stays on the shelf
- The Blue Sky boys commission an external agency that has no knowledge of how to deliver within the organisation
- Ship the greenfield stuff out the door; a surefire talent-loss exercise
BlackOps is a symptom
In large organisations it’s also pretty standard to have a non-standard, unofficial set of technologies, often wielded by the ever-present “BlackOps” guerrilla development teams. As a general rule, these developers crave simplicity and tooling that allows them to quickly and easily prototype, and are often frustrated by large, enterprise application tiers that are untestable, therefore hard to release and hard to innovate against.
These BlackOps teams highlight that the problem is the technology stack. The BlackOps approach demonstrates that organisations can move swiftly to production but at the cost of a limited, known, easily supportable (and therefore a constrained) set of technologies. At a higher level it is the uber-services, owned and protected by stop-sign-wielding product owners, that are an ultimate stranglehold on the ability of the organisation to innovate against its own platform.
Some of the most successful organisations in the innovation space have become so by harnessing the power of the BlackOps approach. So much so that they no longer have a separate BlackOps team.
Micro Services
There are several key requirements for a piloting platform. At the very least, a set of attributes that inform success (scale, analytics, flexibility, audience mechanics etc), and more importantly, a sanitised, known and above all minimal time and cost route to production.
Consider moving towards small, lightweight, interconnected services. Its the classic Lego scenario. The more granular the service, the simpler it is, and the number of options for working with that service becomes considerably greater. At the same time, if each component is a service within its own right, with a set of rules about these services interconnect (think about lego connectors on every brick – in this context, HTTP is a good start), then you can consider starting to build these services in more than just your adopted enterprise language to see if the proposal holds water.
Design-wise, this approach is known as “Micro Services”. These pluggable, highly version-able components can be used, just as Lego, to build new ideas with simplistic interfaces. Micro Services also adhere to the single responsibility model, where each service should do one and only one thing, and this in itself can lead to simplistic, small code bases. Can you imagine a world where a given service is simplistic enough for your team to be able to iterate and release it safely into a new product idea without being bottlenecked by the team who “own” the service?
Of course, it’s not quite that simple. The simplicity gained at the Service tier can only work within a well designed and powerful tooling and build pipeline. As with all new projects, you seriously want to consider the effort required to establish that framework.
Netflix, perhaps the originators in this space and certainly the most willing to share, use a powerful framework of components that publish and control routing to different versions of a given service to allow A/B/multi-variate testing of new ideas. This is the classic “measure” part of the build/measure/learn loop in Eric Ries’s “The Lean Startup”. Whilst Micro service blog posts are prevalent on the internet, if you have the time I would recommend you consider looking at the some of the originators. Critically, they arrived where they are because of a problem they had to solve, rather than a trend. Netflix in particular has a technical blog with a wealth of information, and has also published a large number of its tooling a!round Micro Service architectures as OSS. Mandatory reading for anyone moving in this direction.
Case study of Innovation in a corporation
Some time ago I did a large piece of foundation work for the BBC. The goal was to look at a technical platform which can deliver against the classic “succeed quick, fail-faster” scenario. Some SMEs, startups in particular, often do most of these things out the box, but its much, much harder for large organisations to do so.
When we started BBC Connected Studio, we knew that one of our key challenges was frustration with the BBC’s “Forge” Technical Platform. And we knew that rumours about these frustrations had breached Camp BBC and were rife in the wider development community – the very people we wanted to partner with in our innovation projects.
Each project’s funding was relatively small, and we knew we had to allow partners to work within their area of expertise. Asking them to adhere to a constrained set of technologies and integration into an existing complex, corporate technology stack – that even the internal developers didn’t like – would result in lack of delivery of an idea we could pilot, thus killing the initiative. Remember these partners weren’t proposing ideas that constituted an entire development platform, but ideas that could be iterated to produce new products and services cheaply and quickly.
We had to convince the business to allow new services into production in “new” language implementations, perhaps in vertical, self sufficient slices.
As with all modern technology projects, the technology is no longer the complexity in such a move – a proliferation of frameworks and an ability to dynamically create and destroy infrastructure quickly and cheaply means actually building the ideas should be entirely possible in the way it wasn’t event 5 years ago.
It is still organisational complexity that is the differentiating factor between companies who can quickly and easily develop services, and those that can’t. But that’s for another post.