Ossification and Software Delivery

Posted by Phil on June 25, 2017

The software delivery service has a very different appearance to clients looking from the outside in than it does to those for whom it's a day job. To most non-technical customers, the complexities behind a pleasantly-designed user interface are hidden and abstract. Something that was delivered several years ago and has worked well should continue to work well and suggestions that it's becoming harder or more fraught to change their solution are often not well-received. To the uninitiated, professional software delivery services are a technical constant: a specialism involving moving business data from Point A to Point B and if the technology and platform are fundamentally unchanged, then the degree of difficulty and error should also remain unchanged.

"It's not rocket science!", "It was working fine for years before you added these enhancements", "It used to take you two hours but now it takes half a day for the same job". Clients aren't just being unreasonable for some sadistic pleasure they derive from complaining. Their concerns are completely valid from their point of view. They paid a good amount of money for their solution years ago and why are they being expected to pay even more now to do simple changes today that are taking twice as long as they used to?

The expectation that a change or fix should be identically reproducible given the system is (outwardly) unchanged in terms of functionality is a reasonable assumption given the degrees the developers have are in Computer Science or Information Science. Having said that, why are the terms "computer science" and "software development" not used interchangeably in the same way as "programming", "development" and "software engineering"?

Is the process a science like maths or engineering or chemistry? If not, then we have property developers as a job. Is software development the same sort of thing where constructing a building is a known process and should have predictable costs, inputs and results?

Perception and Language

Part of the difficulty we have between client expectations and actually delivering to them is perception. Science is about measuring or detecting and explaining objective reality. It's about gathering data and working to understand it. Science as a discipline is not concerned with the people-oriented outcomes (at least in the first instance. Science communication is an entirely different issue altogether). Computer science is a mathematical, "algorithmic" discipline that considers ideas such as efficiency, performance and logic. It's unsurprising that clients should expect features, fixes and enhancements to be delivered consistently over a period of years if this is the definition they associate with the software profession.

Yet software development is much more than those things when taken in context with the wider problem of creating a solution to a given problem. Coding requires creativity, imagination and innovation to be truly effective, so what things in our industry contribute to that and what factors create an environment where the process becomes harder, more rigid and inflexible?

To avoid being burned by production mistakes, angry clients or expensive design decisions, we often try to protect against those events by adding more process, implementing policies around design and training our people to think within those risk-averse paradigms so they don't wander off the garden path. But adding all these protections is a false economy. A little is probably a good thing, but there needs to be very judicious use of constraints, process and cost cuts because left unchecked, these things contribute to an even harder environment in which to delivery good results.

Ossification 

Ossification occurs when the environment begins to support conditions where change and growth become harder or riskier and freedom of movement and adaptability are reduced. As adaptability or evolution is constrained, the system becomes more rigid and hard to maintain.

Ossification can occur in any part of the software development ecosystem:

  • the code
  • the developer personally (skills or knowledge)
  • the process or culture of the organisation

Software development should be organic: constantly-evolving and adapting to changes in technology, business / market demands or development team requirements. But as a system grows in complexity, the impacts of making larger changes grows with it unless the conditions are set to encourage and respond to change.

The code base itself needs constant gardening as it grows in size. Technical debt, external dependencies, packages / libraries and features all press down on a system's maintainability such that over time, the tendrils of each functional area; each class dependency spread and become ingrained into other parts of the code base. There is a cost in refreshing and maintaining these dependencies and interactions but if the client or business is unwilling to invest the time - seeing it as effort better spent elsewhere - then these problems continue to compound to a point where change becomes practically impossible.

The developers on the team also require growth. Without training in new technologies, ideas and alternative pursuits, their interest will stagnate, their skills will become myopic and their motivation will quickly dwindle. If you're only granting an hour a week for staff training, or prescribing when, how and which options are considered "approved" then your people aren't going to realise their full potential and they'll become unhappy and restless very quickly; even if the work itself is challenging and engaging. If your people aren't adequately empowered to broaden their horizons and knowledge, their ability to adapt to change - and apply new thinking and skills to the code base they're assigned to - suffers as a direct result.

Finally - the process. If the company has a culture of separating or segregating its people from different stages of the software development cycle, then it becomes much harder to respond to problems or challenges when they arise. It also becomes harder to remedy issues around knowledge transfer, training and experience between members of different teams if they're not enabled to see the whole system. The natural reaction for organisations who see mistakes or failures creeping in to their development process is - perversely - to add more process: to bandage and cauterise the wound rather than address the cause.

This actually has the opposite of the desired effect: people are less able to work across teams, teams are siloed and separated further and time (and therefore costs) for delivery increase as more process and paperwork is imposed under the guise of "protecting the innocent". Innovation is stifled, knowledge isn't shared, visibility is lost across the system as a whole and customers become frustrated at the lack of results they'd come to expect. As more process is added, the adaptability of the system is reduced and the focus on the important task - shipping features - is substituted for chargeability, cost savings or "efficiency".

The true task of leadership is to get organisational systems working. Managers must do the hard work of understanding the market, their customers, and make difficult choices about what work to do and especially what work not to do. Most importantly though, managers need to study how teams and organisations really function and then create systems and environments that don’t hinder people from achieving great results for themselves and for the client.

Refactoring is something that should be second nature for code, but a team and its leadership should also take time to reflect and refactor so that they're given the best possible shot at ensuring the balance between delivering good outcomes for the client and investing in strong growth as a team and for the company are not mutually exclusive.