Dark Architecture

Fog lights are standard accessories on many recent car models. They are inexpensive, look exotic, and illuminate the ground in foggy conditions. (Driving through Fog - Guide for a Safe Return, Autoevolution.com). But some drivers use fog lights to fight the night rather than fog. Whether these night drivers don't understand fog lights or they want to extend their sight to drive faster, using fog lights in this way is little different from driving with your high beams on: they blind other drivers night vision and create road glare so other drivers cannot see the pavement. While the driver with fog lights has increased visibility while the vision of approaching drivers is reduced. This can actually increase the chance of a night collision instead of reducing it as one might think. (How Much Light is Too Much Light?, Wheels.ca).) In some countries, driving with fog lights without fog is illegal. (Fog Light Fury, CarAdvice.com.au).

In a previous The Lone Coder article, I asked if companies considered good computer design a priority for a successful business (Paper, Plastic or Sound Design, The Lone Coder). A rapidly moving society valuing immediate results and short-term solutions is like a driver with his fog lights on, ignoring long-term risks.

Good computer professionals, when companies permit, minimize long-term risk and eliminate root problems. Some positions carry titles like "engineer" or "architect". Both these terms were invented over 40 years ago and, aside from the legal issues over who can use the term "engineer", the definitions in Wikipedia are very abstract.

Wikipedia describes software engineering in this way: "Since the field is still relatively young compared to its sister fields of engineering, there is still much debate around what software engineering actually is, and if it conforms to the classical definition of engineering. It has grown organically out of the limitations of viewing software as just programming." (Software Engineering, Wikipedia)

Wikipedia also defines software architcture: "The software architecture of a program or computing system is the structure or structures of the system, which comprise software components, the externally visible properties of those components, and the relationships between them. The term also refers to documentation of a system's software architecture. Documenting software architecture facilitates communication between stakeholders, documents early decisions about high-level design, and allows reuse of design components and patterns between projects." (Software Architecture, Wikipedia)

So if good developers understand good design, what is the difference between a software developer, a software engineer, and a sotware (or application) architect and how did they relate to design?

I asked a software developer.

"An IT Architect", he said, "is a person who don't actually know how to code, just what it should look like."

Those who can, develop.

Those who can't develop, engineer.

Those who can't engineer, architect.

So did design play no role in any of these positions?

I asked a manager who worked for several large Canadian companies.

"Understanding business, database, software and hardware requirements?" the manager mused. "Experience in different sized environments. Always looking beyond the obvious, looking at the long-term consequences, looking for the next challenge. Seeing a problem and taking ownership of it. That person is an enterprise architect."

"An 'enterprise architect'?" I repeated. "What's that?"

"One of a small number of people that big corporations rely on to save them millions of dollars. Someone to give them a strategic advantage over their competition. A lot of people think they have what it takes but not everyone has the skills to be an enterprise architect. The money is good but, expect to spend a lot of your time unemployed. In fact, companies often fail to understand why they need architects."

In a culture where "you require X years experience in Y", or other meaningless assessments of skills ("IT Hiring: We Don't Want the Best", The Lone Coder), a designer spends so much time on such the broad issues that he lacks narrow skills sought after by many recruiters and HR departments. So I could understand that.

"You, Ken, definitely 80% of an enterprise architect...except you are missing vital characteristic for success."

"What's that?" I asked.

"You're not an extrovert."

I was perplexed. "An extrovert?"

"Yes. You must be a disrespectful, in-your-face, extroverted, take control of a situation, threaten people, deceive them, humiliate them and ram projects down their throats. You have to know that, as an architect, you are superior to other computer professionals and to know that the end justifies the means when it's for their own good."

Whatever enterprise architects were, they didn't sound like trustworthy, responsible team players. I didn't understand why an enterprise architect had to be a school yard bully. Surely leadership comes in many forms ("Five Different Types of Leaders", Steve Denning, "The Leadership Crash Course: Seven Types of Leaders", Dr. Paul Taffinder) and a bully is not usually on the list, and that's for good reasons. To oversee and understand large, complex systems you have to understand your own limits and empower the people around you.

So I approached another manager, this one a TOGAF certified architect (TOGAF Home Page). Perhaps he could explain architecture to me.

He began by telling me about the long years of work, labouring unrecognized and unappreciated. He told me that some day I would understand, when I was experienced enough and had proven myself worthy to know. I expected him to tell me that he walked 10 miles in the snow to work each day (Origin of the Phrase).

He used "architected" as a verb, showed me thick official looking documents with carefully constructed UML diagrams. But, when I looked past the documents and reviewed the designs, many of designs were flawed and many of the project had failed. Was architecture, then, just documentation?

When some people talk about design patterns (or algorithms), they sometimes use the term "anti-pattern" or "dark pattern". This is a sarcastic way of discussing common practices that destroy good design. (Anti-pattern, Wikipedia defining Dark Patterns). An example of a dark pattern is "gold plating": polishing the documentation and the source code layout, while leaving the poor software designs within unchallenged and broken.

Perhaps, like dark patterns, there was also "dark architecture", common practices opposing good high-level design, breaking real business, database, software and hardware requirements and the processes to deploy them. One could imagine some of its incarnations:

• Accredition Laurels - Using past successes to get a free pass on bad designs. "Sure it looks wrong, but he's an architect from a famous company so there's no need to review or test the design."

• Architectural Slums - Portions of a design that no longer meets the system requirements but can't be upgraded so it's left disenfranchised. "We have a 20-year-old program on an old server sitting in a corner that does this already. Why should we spend good money to replace it with something that does the same thing but fits into our new systems?"

• Buzzword Hypocracy - Hiding bad practices under the guise of using common but not well understood terminology. "This is architected using the agile methodology: that means we can eliminate all testing."

• Chronological Snobbery - The newest design fads are given preferential treatment while older, well-tested techniques are shunned. A term coined by C.S. Lewis. "That solution is so 1980's. Where's the Java SOA?"

• Ego-Driven Design - Obsession with redesigning things that have no requirement for changes purely to prove one's skills. "This program uses tabs instead of spaces. What garbage! We need to rebuild it and make it right."

• Family Loyalty - Choosing designs to endear yourself to people you know because it's beneficial for you personally or your career. "My husband created this monolythic monster so we're releasing it."

• Hardware Immunity - Purchasing expensive hardware and then blaming performance issues on software or database design, without investigating the hardware performance. "This NAS cost $20,000, so it couldn't be the performance bottleneck".

• Language Limbo - Using the wrong tools because they're the company standard, even though qualified people are on staff who know the right tools. "Let's build this high-performance application using shell scripts because we don't have any C++ positions at our company."

• Lowest Common Denominator - Oversimplfying a design because you believe your staff are incompetent. "Don't purchase a SAN: our guys would never be able to meet the challenge of deploying it."

• Micro-Thinking - Designing for a single development microcomputer or vitual machine but deploying or expecting to deploy onto several real servers without understanding the impact.

• Paperwork Fog - Creating large volumes of gold-plated, overly-detailed documentation in order to hide the weaknesses in a design. "These UML diagrams are detailed and professional but I can't follow them. It looks like a thorough design: let's go with it."

• Performance Pinhole Optimization - Focusing on the performance of small areas while ignoring the wider issues of the system as a whole. "Don't use comments in your scripts: that will slow down the performance of our overloaded server."

• Plug-in Faith - The belief that good design is not important because there's a magic solution developed by a higher power available for download. "Download and add a caching module. That will solve the performance problem in our design."

• Standard Fixation - Sticking to an enterprise's architectural standard even when it was never intended to apply to the current situation. "We're using MySQL even though it's not optimized for large-scale transactions because that's our standard."

• Superman Syndrome - Believing an architect can design a solution without proper requirements and no matter how complex the system is, solely on the grounds that they are an experienced architect. "I have no experience in JSP on 200 servers: but I can solve this problem without investigating it. That's just how good I am."

• Teddy Bear Designs - Choosing design to reassuring users rather than developing an appropriate design. "Don't worry...this is an industry standard and things will go forward without problems."

• Throwing Hardware - For a design that doesn't meet requirements, add additional equipment until it functions for the short-term, even though fixing the design problem would be faster and cheaper.

• Tower of Babel - Creating unnecessarily overly complicated designs that introduce large beaucratic overhead and make things incomprehensible to humans, often done to move an issue into the designer's skill domain. "We'll wrap all our database queries in giant data access objects that build the SQL queries from dozens function calls with long parameter lists. That will fix the problems with the database."

• Transformation Terror - If something is meeting the current requirements, don't redesign it to meet upcoming requirements for fear of introducing mistakes. "If you install a second database server to handle the extra load, and it doesn't work, I could lose my job!"

• Un-Reality Show - Managers demanding new business requirements based on what saw on a TV show or at the movies. "They did this on Star Trek: how soon can we add voice recognition to our web product?"

The list was seemingly endless.

Perhaps it wasn't design that I didn't understand, but rather the forces that worked against design were the things that I didn't understand. It was dark architecture that I didn't understand: so many people driving with their fog lighs on, obscuring what was really important: good design.

My chat program popped open. "Hey, do you know a fast algorithm for collision detection?"

"Hey, Architect!" someone called behind me. "Do you think this is a design flaw?"

I was a guy who understood business, database, software and hardware requirements, always looking beyond the obvious, looking at the long-term consequences, looking for a challenge, taking ownership of problems, making designs that made sense.

I turned around. "Sorry, I'm not an architect...". Nor was I an engineer. Nor a developer. Nor a DBA. I wasn't sure what I was. On the other hand..."OK," I said, "let's a have a look at it."

November 22, 2009