One thing I have to explain a lot is what are the costs of software development. Why are things taking so long? Why is there any needed for maintenance and support? Why are developers spending significant amount of their time looking over the existing code base and why we can not just add the next and the next feature?
Today I have an example of this – and these are “dependencies”.
The goal of this article is to give people more understanding on how the “tech works.”. I’ve seen that every line of code and every dependency that we add to a project will inevitably result in further costs down the road so we should really keep free of unnecessary dependencies and features.
Daily builds
Many contemporary professional software projects have a daily build. This means that every day at least once the project is “built” from zero, all the tests are run and we automatically validate that the customers could use it.
Weekly dependencies updates
Every software project depends on libraries that implement common functionality and features. Having few dependencies is healthy for the project, but having no dependencies and implementing everything on your own is just not viable in today’s world.
These libraries and frameworks that we depend on also regularly release new versions.
My general rule that I follow in every project is that we check for new versions of the dependencies every Wednesday at around 08:00 in the morning. We check for new dependencies, we download them, we build the project and we run the specs/tests. If the tests fail this means that the new dependencies that we’ve downloaded have somehow changed the behavior of the project.
Dependencies change
Most of the time dependencies are changed in a way that does not break any of the functionality of your project. This week was not such a week. A new dependency came along and it broke a few of the projects.
The problem came from a change in two dependencies:
We have installed new versions of two of the dependencies “websocket-driver” and “mustache-js-rails’
These two dependencies broke the builds.
Why should we keep up to date
Now out of the blue we should resolve this problem. This takes time. Sometimes it is 5 minutes. Sometimes it could be an hour or two. If we don’t do it, it will probably result in more time at a later stage. As the change is new in ‘mustache-js-rails’ we have the chance to get in touch with the developers of the library and resolve the issue while it is fresh for them and they are still “in the context” of what they were doing.
Given the large number of dependencies that each software project has there is a constant need to keep up to date with new recent versions of your dependencies.
What if we don’t keep up to date?
I have one such platform. We decided 6-7 years ago not to invest any further in it. It is still working but it is completely out of date. Any new development will cost the same as basically developing the platform as brand new. That’s the drawback of not keeping up to date. And it happens even with larger systems on a state level with the famous search for COBOL developers because a state did not invest in keeping their platform up to date for some 30+ years.
The subject of technical debt is interesting for me. I recently got a connection on linkedin offering me to help us identify, track and resolve technical debt and this compelled me to further write this article and to give some perspective on how we manage technical debt in our platforms and frameworks by specifically stopping on a few examples from the fllcasts.com and buildin3d.com platforms along with the Instructions Steps (IS) framework that we are developing. Hope it is useful for you all.
What is technical debt?
Here is the definition on the first source of wikipedia. It is pretty straightforward –
Technical debt is a concept in programming that reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution.
The way I think about technical debt is – we write certain structures like for example “if” which in many cases is a Technical Debt. Of course there are many other types, but I will stop at this one for the article.
// In this example we do a simple if for a step when visualizing 3d assembly instructions
// if there is animation for the specific step created by the author and persisted in the file we play this animation, but if there is no animation in the file we create a default animation.
if(step.hasAnimation()) {
return new AnimationFromFile(step)
} else {
return new DefaultAnimation(step)
}
The problem with technical dept here is that there later in the development of the project two new cases might arise.
Case 1 – we want to provide the functionality for AnimationFromFile(step) only to paying customers, otherwise we return just the default animation. The code then becomes:
// We check if the user is subscribed and only then provide the better experience
if(step.hasAnimation() && customer.hasSubscription()) {
return new AnimationFromFile(step)
} else {
return new DefaultAnimation(step)
}
Why is this bad? Where is the debt? – the debt is that we are now coupling the logic for playing animation with the logic for customers that have subscriptions. This means that when there are changes to the subscription logic and API we must also change the logic for handling animations.
Case 2 – we introduce a third type of animation that is only for users with a certain WebGL feature in their browser. The code becomes:
// We check if the browser supports the WebGL feature in question and then return a FancyAnimation.
if(step.hasAnimation() && customer.hasSubscription()) {
return new AnimationFromFile(step)
} else {
if (webGlFeaturePresent()) {
return new FancyAnimation();
else {
return new DefaultAnimation(step)
}
}
Now we have a logic that knows about creating DefaultAnimation, about reading from files, about what a subscription is and when are users subscribed and it also knows much about the browsers and their support for WebGL.
At a certain point in time we would have to refactor this logic and separate it into more decoupled pieces. That is a technical debt.
How was Technical Debt created (in the example above)?
We took the easy path now by placing one more if in the logic, but we knew that at some point we would have to refactor the logic.
Should we omit Technical Debt?
I think a good architecture could prevent a lot of the technical debt that is occurring. Good decoupled architecture with small units with clear boundaries and no state will result in 0 technical debt and we should strive to create such systems. Practically the world is not perfect. In a team of engineers even if you spend all your time on fighting with Technical Debt it is enough for only one colleague at one instance to take the easy path and add one more if to “fix this in 5 minutes instead of 3 hours” and the Technical Debt is already there. You’ve borrowed from the future.
How do we track technical debt?
I have personally learned to live with some technical debt. If I now do
$ git grep "FIXME"
in one of our platforms we would get 37 results. These are 37 places where we think we could have made the implementation better and we even have some idea how, but we’ve actively decided that it is not the time now for this. Probably this part of the code is not used often enough. Or we are waiting for specific requirements from specific clients to come and we would address this them – when there is someone to “pay for it”. Can we address it now? – but of course. It would take us two, three days, but the question is why? Why should we address this now. Would it bring us more customers, would it bring more value to the customers? It would surely make our life easier. So it is a balance.
Our balance
I can summarize our balance like this.
We identify part of the code as a technical Debt (because we do regular code reviews).
We try to look at “what went wrong” and understand how this could be implemented better. We might even try in a different branch but we do not spend that much resources on this.
We then know “what when wrong” and we agree to be more careful and not to take debt the next time but to instead implement it in the right way the first time.
After that we decide if it is worth it to refactor the current issue – are the new clients coming that would ask us for modifications on these parts of the code?
That’s it.
Simple “FIXME”, “TODO”, “NOTE”, “IMPORTANT”, “SECURITY” tags in the code, git grep to see where we are and balance with trying to learn how to do it correctly next time.
How can we solve Technical Debt for the example above?
In buildin3d we have a framework with an event-driven plugin architecture. So for us it was simply a matter of registering a different plugin for the different features.
// Pseudo code is
framework.register(new FancyAnimation())
framework.register(new AnimationFromFile())
framework.register(new DefaultAnimationExtension())
framework.ariveOnStep((step)=> {
...ask all the extensions for animation and play the first animation that is returned
}})
The question is at which stage do you invest in a framework.
What about MVP(s)?
The greater balance is sometimes between an MVP and a working product. On one occasion we had an open source tool that was doing exactly what we needed. It was converting one 3D file format to a different 3D file format. We started the project. We used the open source tool. We delivered a working MVP in about a month and we took a lot of debt, because this tool came with other dependencies and was clearly not developed to be supported and extended. It was clear from the beginning that once new client requirements started coming we would have to re-write almost everything. And we waited. We waited for about 2 years. For 2 years we were extending the initial implementation and one day a client came with a requirement that we could no longer support. Then, it took us about 6 months to re-write the whole implementation in a completely new, much more extensible way and could easily accommodate new requirements.
Conclusion
Try not to take technical debt.
If you have to then at least try to learn why it happens and learn how not to do it in the future. You will exponentially become better.
Write down a comment in the code about why do you think this is a debt and how it should be approached. Spend some time reviewing and resolving debts if it pays off.
Reply
You must be logged in to post a comment.