The Negative Outcomes of Building Chaotic Systems
In the past I spoke a little about building chaotic systems that gets released to unsuspecting customers which inevitably and negatively impacts their lives. In this article, I’m going to outline in detail the negative impact chaotic software causes when left unchecked without a warning of its maturity and the state of quality.
Let’s first define chaotic software.
A chaotic system is a system that doesn’t follow the best practices or standards in building software. These systems then get released out to an audience under the impression that it is production ready. Chaotic systems are just a reflection of either dishonesty or mediocracy, sometimes even both.
A lot of engineering teams don’t mind adapting and following best practices for building proper standardized systems. But the problem here comes at the point where these teams don’t know which best practices to follow, and what standards to adapt.
We will talk later in this article about how to prevent the next chaotic system from being born into our world and ensure every new system follows best practices for the safety of everyone involved.
Reasons for Chaos
Systems that are built without standardization are built this way for three different reasons. It can be unawareness, misbelieves or simply conflict. We will talk in detail about each one of these reasons, and how they can be remedied in the short and long terms.
Unawareness or Laziness
The most common reason is engineering teams’ unawareness of the importance of standardization and following best practices. I’ve seen several teams throughout my career where their final “product” is a mesh-mash of conflicting patterns and designs that lack cohesion and stability.
These teams may have the best intentions at heart. But due to the continuous state of inexperience of the industry they get so easily confused and lost as of which standard to follow and which quality measure to adhere to.
There are other teams that fully understand the best practices and standards. But they choose to actively reject following these standards and adhering to these practices. The reasons for these actions may vary from one team to another. Sometimes it’s just laziness. Following standards requires a ton of hard work, study, and research. It requires hearts passionate about the industry and minds eager to learn and evolve. These teams are usually comprised of employees that just want to score a quick buck and go home. These types of employees are the most dangerous to themselves, to the business and more importantly to their customers.
Solving the issue with unawareness and laziness is quite simple. It takes strong leadership to adhere to the vision and enforce these best practices until it becomes the standard for the entire startup or company then eventually it drives mass adaptation and less excuses for those who choose to be lazy to learn it.
Misbelief
For some of these teams, they reject standardization due to misbelief that cutting corners will get them to where they want to go “faster”. They lie to themselves and say: “let’s just cut these corners now, and later we will go back and fix it”. These teams never “go back” and they surely never “fix” anything. I’ve even heard some that say: “If it works don’t fix it” — If you’ve ever heard that term, know that there’s a big chaos lying behind whatever system these so-called engineers are building.
It can be quite difficult to distinguish between these teams and their counterparts of engineering groups. The type of groups that adhere to startup mentality where they truly continuously refactor and evolve their systems. Several years ago, I worked in a startup where engineering teams never take off the tag “beta” or “alpha” from the logos of their products until everything is rewritten and properly tested according to a standard. I’ve even seen teams put a disclaimer saying things like: “This product is experimental use at your own risk”.
I have tremendous respect for these teams because they fully understand and realize the importance of honestly and truthfully informing their end-users with the true state of their systems. They also recognize that engineering standards are in a continuous state of evolution and their systems must be continuously evolving for as long they are being used.
Leaders of chaos are the ones who mainly drive that misbelief. They need to be held accountable for the outcomes of their decisions with data. They need to be exposed so they can truly understand the risks they bring upon themselves, the business, and the end-users. It takes discussions and bravely to expose these mindsets and bring them back to the right path.
Leaders of chaos and their origins will require its own article which I will be discussing sometime soon.
Conflict
The third reason some teams decide not to follow best practices and standardization is conflict. These teams experience continuous waging wars between engineers trying to push their own interpretation of what best practices are and what measure of quality they need to follow. It becomes even worse when the agendas are different, and intentions are not always leaning towards quality and engineering excellence.
Nothing in our industry today can ruin the potential future of standardized non-chaotic systems like personal agendas and selfish desires of temporary and fake wins. Especially when the people carrying these agendas are in authority and the people fighting for standardization lack faith and courage to withstand these authoritarian oppressors.
These situations can be quite difficult and troubling. Junior engineers being indoctrinated into chaos and fighters for what’s true and right being oppressed, bullied, and even silenced to give up their principles and fall into what they truly believe to be falsehood.
Conflicts like these can be resolved with three types of solutions. The easiest of them all is migration. To leave it all behind and let it all burn itself into its own chaos (and it usually does). The slightly harder solution is continuing to speak out against what’s wrong and never be silenced or bullied into holding back the truth. The last one is to bring rain of fire from above. Enforcements of supporters that push back and shift the direction towards success and prosperity.
It usually takes tremendous amount of influence horizontally (peers) and vertically (leaders and directs) and even diagonally (cross teams and customers) to bring about a true change. It usually takes faith, intellect, and power to truly make any measurable change in our world. This case here isn’t any different.
Now, let’s talk about the outcomes of building chaotic non-standardized systems.
The Business
Chaotic non-standardized systems hurt the business that hosts and develops them the most. Yahoo! For instance used to be a promising company with massive influence and presence on the web. A string of poor business choices has ultimately led to the company’s demise. By the testimony of Yahoo’s own engineers Elliott-McCrea wrote “I recently pulled up a worklog I was keeping in 2008–2009, and I found 18 meetings scheduled over a 9-month period discussing why Flickr’s API was poorly designed and when we’d be shutting it down and migrating it to the YOS Web Services Standard.”
Chad Dickerson, former Yahoo developer evangelist and the current CTO of Etsy comments, “In my experience, entrepreneurs moving into Yahoo! often got stuck doing PowerPoints about “strategy” instead of writing code and shipping products.”
Poorly designed non-standardized systems were a big part of the reason why Yahoo failed. Yahoo’s chaotic systems cost the company everything. Alison Griswold wrote in the stunning collapse of Yahoo how the company went from $100 billion valuation to only a fraction of $4 billion valuation due to chaos.
The Customers
Businesses can always recover and get back on their feet if the right vision and leadership are put in place. However, the impact of chaotic systems on customers at Yahoo! was exponentially much worse than the business impact.
Yahoo has faced two major data breaches, one of them being the largest one on the record! In 2016, it reported a data breach of over 500 million user accounts in 2014. In December 2016, it also reported another data breach that occurred around 2013 in which hackers impacted about 3 billion of its user accounts!
Impacted users account and stolen data is what we call in the industry an irreparable damage. Data breaches can massively impact the safety of customers, exposing their information, location, family and often times puts these users as targets for being blackmailed.
The situation gets much worse when the systems being developed directly impacts people’s lives. Like medical systems that are developed to detect health issues and warrants to alarm/alert medical health professionals to immediately intervene and save people’s lives.
Chaotic systems can also threaten the safety of children when they allow wrongdoers to manipulate them into luring innocent children to leave their homes and getting kidnapped and harmed.
The Engineers
The last aspect of impact of chaotic systems are the engineers involved into building these systems. Building chaotic systems can be very attractive at the beginning. Everything gets done so fast. No testing, no design and no quality assurance. And then that sweet short-lived “win” fades away when the system starts to fail. Angry customers, failing business and frustrated engineers.
Engineers go through mental torment maintaining chaotic systems. No amount of money, benefits, perks or insurance can possibly remedy the negative impact chaotic systems have on the mental, physical and spiritual health of the engineers running and maintaining these systems.
Lena Kozar wrote an article where she talked about the growing number of developers who are experiencing work-related, mental health disorders. These include stress, exhaustion, fatigue, and loss of motivation.
Lena goes on to say: When developers begin coding, they leverage all four basic elements of the cognitive sphere. Feelings and perceptions are needed to download data from the screen into the brain, our thinking is then activated to process the data, imagination kicks in to help find creative solutions, and our memory stores already existing solutions for us to use when they are suitable. A developer’s work is closely related to engineering, which also requires focused thinking. The modern scientific theory assumes we are only able to work in a focused setting for 2–4 hours. Of course, our brain is rather flexible and adaptive. It actually allows us to work productively while being overloaded, for 1–3 months. However, if someone continues assigning an excessive number of tasks to be processed during those 2–4 hours of focused thinking, it will likely throw developers into an abyss of chronic exhaustion.
Standardized non-chaotic systems solve this issue. It allows our brains to switch from focused and unfocused mode to prevent these dangerous symptoms from impacting engineers and their mental health.
Standardized systems create a culture of understanding, collaboration and better communication when the goal is unified, and the path forward is well-defined and well-practiced and adopted.
The Future
A growing number of engineers are becoming more aware of the importance of standardization. The industry is beginning to form communities around best practices and continues to provide engineers with answers throughout several platforms, visual, vocal, or written.
It’s the responsibility of every engineer out there to seek out standardization. It all begins with a core belief in the importance of standardization. Then learning and practicing it themselves so can gain their own knowledge from their own experiences. Then go on to preach that knowledge to other engineers and other communities where the awareness of standardization has not arrived yet.
Writing code that works is something anyone could do today. Writing standardized systems is a massive burden and worthy endeavor for engineers everywhere to chase and accomplish.
As a seasoned engineer myself, I see a growing number of engineers from everywhere joining and contributing to The Standard Community. They continue to enrich the overall human experience to setup the measures of quality and craftmanship.
Wherever I go and whoever I talk to, I see eyes eager to see the future. Hearts eager to believe in the mission. And minds eager to learn and walk the path. It’s going to take the whole of the engineering community to drive such initiatives and optimize our path forward as human species to hop on to the next level of our evolutionary journey.
