PegaSoft Canada White Paper
Draft - 2003
By Ken O. Burtch
Wal-Mart spends billions on IT [Cringely, ASP)
In over half of all business that have developed IT departments, there has been no improvement in productivity with computerization. Half of the business sectors that did improve productivity were tech-oriented companies. [Rethinking]
There is an English proverb: “A good archer is known not by his arrows but by his aim.” [BR13, p. 117] The concept of marketable “skills”, that a person’s capabilities can be summarized by a few words, is a myth. Experience at a particular area of IT is usually irrelevant. Software evolves so much that personnel officers have no clue which products, languages, or talents are related to others. They don't realize, for example, that C++ or even Ada makes a foundation for Java.
It is a rule of thumb that any experience beyond 2 years with an area in computers is usually inconsequential. There was a time I saw an job ad for "8 years Java experience". Java was less than 6 years old at the time. The “years” was simply used to weed out candidates—in this case, it weeded out all the honest ones.
When deceit becomes the norm, there is no banner to rally behind. Richard Stallman attempts to take credit for Linux. Bill Gates attempts to take credit for the Open Source movement. Truth is, after all, the first casualty of war.
The suppression of truth is no more visible than in corporate IT departments. According to a 1998 survey, only 26% of IT projects were successfully completed. It's not surprising when the average employment time for an IT worker is 2 or 3 years.
But without truth, there is nothing left but Andromeda-esque Nietzschean. No employee loyalty. No customer loyalty. Not that it matters to the Venture Capitalists funding dot.coms. But what about Open Source where, without loyalty, there would be no contributors to projects?
Some would say that hatred is enough to fuel the Linux revolution. 20 years ago operating systems were free. Now you have to pay an additional $400 if you don't want your PC to be an expensive doorstop. Purchase a commercial UNIX for thousands of dollars and essentially purchase 20-year-old technology. Sure, there's plenty to hate about the unfair practices of software companies. But as Morden would say on Babylon 5, "So you want the Centauri dead. But then what?" Like G'Kar, the slashdot crowd has no answer. Hatred supplies motive without direction.
With any revolution, the fight is futile without the presence of a good leader. Linus Torvalds doesn't hate. Linus Torvalds doesn't lie. Others gather around his flag even though they don't share Linus' values. Without a leader like Linus, Linux would only be another Excite@Home slight-of-hand.
Don't be afraid to look foolish. Canadians look foolish to Americans because of their tiny armed forces. Canadians, on the other hand, believe that they are tougher than Americans because, although they have the ability to build nuclear weapons, they choose not to so as not to jeopardize U.S. relations. Like two cowboys meeting on the range, Canada decided to be the one to drop its gun first, putting itself at the disadvantage. The first one to drop the gun is the one with the guts. Canadians don't want a Cubian Missle Crisis.
At Comdex/Canada, a Linux company put out a "donations for Linux" box. People would put in spare change to support the open source movement. What did they do with the money? They went out and spent it on beer. They betrayed the trust of Linux supporters as well as wasted an opportunity to put Linux's momentum to good use.
A company is made of people, and people need a moral center. The pursuit of profit is never a moral center.
Your employees will spend the majority of their lives at your company: they are people, not human resources. People have fundamental needs that must be met. Resources don't.
"The first obligation of a genuinely loving person will always be to his or her marital or parental relationships." [Peck, pg. 159]
"In approximately a quarter of our cases, whether patients are adults or children, considerable and even dramatic improvement is shown during the first few months of psychotherapy, before any of the roots of problems have been uncovered or significant interpretations have been made. There are several reasons for this phenomenon, but chief among them, I believe, is the patient's sense that he or she is being truly listened to, often for the first time in years, and perhaps for the first time ever." [Peck, pg.129]
Software is a model solution for real world problems, and the models are designed by individuals. “There is only one thing that creates software, and that is the human brain. There is only one thing that creates lots of software; lots of human brains working together.” [OMP] Component and prefab software has largely failed. Have 10 programmers tackle one project and you’ll get 10 different solutions with different strengths and weaknesses.
Feature-orient products designed to fail can't be blamed on a single company. A TV repairman told me that sets today are designed to fail after a few years. You need to spend thousands to get the same quality of TV set your parent's owned. A local boat trailer manufacturer pointed out how other companies design boat trailers to fail, welding steel at the points of maximum stress. Trailer repairs are how they make their money. After all, if you spend thousands on a boat, how many people would examine the quality of the trailer included with the boat? It's the way business is conducted in North America.
Kathrine Hepburn said that "if you always do what interests you, at least one person is pleased."
Apple discovered that focusing on advertising by sacrificing R-and-D was a failure. During a 1987 Inc. magazine interview, Sculley said, "Here we were, a new-products company in a new-products industry, and we could only turn out a new project every year or two-and then only by burning out our people."
A computer system is a virtual modem of a company.
Carl Sagan attributed the rise of the Ionian scientific society was due to its partial isolationism. Large empires caused conformity to wash out speculation and advancement. In this way, why would you expect large companies to be any different? Unless a large company takes steps to promote product improvements, there will be none. This is supported by the company design in the early days of Apple.
Apple focused so much on high-end machine in the late 80s and early 90s that they never asked who would be trained to use these machines. They were out of the price range for hobbyists and schools were switching to Intel machines. Who would be able to carry on the Apple legacy?
Software advances are, by in large, a straight-forward process. New advances are built on the success and failure of current products. If a company avidly pursues R&D, they will usually be the industry leader.
The larger problem is that, with a growing world population, the solutions to the problems of current technology is obvious to many.
Apple spent large amounts of money on R&D to stay in the lead, but there was no focus to development. Engineers and software designers had lots of money but had no direction and spent a lot of time spinning their wheels. It takes more than money to innovate. [Apple pg.82]
Apple developed the ability to play video clips on the Mac in 1987, years before any similar technology, but the project was inexplicably cancelled. mangers at Apple cancelled many similar advanced technology projects because they couldn't tell if a project was useful or not. How can a technically-deficient manager make such choices? There needs to be a different way.
Software is free to produce, free to distribute but labour intensive to conceive.
Customers depending on a product line require timely, accurate information.
Quality, late software sells better than timely, error-filled software. They will forgive your late rollout if the software works. But it is better to be both on-time and error-free.
It is better to ferret out all good ideas than to sabotage ideas the business can't develop.
Software is an intellectual model. It builds a philosophical relationship between you and your customers. Treat your customers
with respect, as when you share philosophy over a lunch meeting.
Lister's Law: "People under pressure don't think faster." Contrary to popular culture, pressure doesn't accelerate software projects. [Slack, p. 50]
[Slack, p11] Having workers 100% busy slows a company and doesn't save money: the cost of having workers with spare time becomes the cost of buffering and tracking work to keep employees busy. DeMarco estimates that 1/3 of North American companies are "overimproved". [Slack p.22]
Knowledge workers have an inherent task-switching overhead: it takes time to switch between projects. Programmers and writers often conceptualize so they have to immerse themselves into complex problems to come up with a solution. This is why workers are not freely interchangeable [Slack p13-16]. A study showed programmers spend an average of an hour a day switching tasks. [Slack, p20]
Pulling people in and out of teams causes confusion and disruption in the team's productivity, trust and cooperation.
[Slack pg. 25] Organizations that emphasize business actually slow down: employees work slower to make sure they don't run out of work, trying to appear to be busy.
Aggressive schedules undermine the foundations of a project. The idea that no harm will come from an aggressive schedule is wrong. [ S., p.56
There are only 24 hours in a day. [My hour shifting.]
There is no objective way to measure a programmer's productivity. [S. pg. 73]
Software development is enhanced with the proper equipment. Fast high-bandwidth throughput, easy-access to support specialists and abundant computing resources. [UCP]
Scacchi reports that IBM "programmers should be more productive when their system's response time is fast, consistent, and relatively predictable from the computing task at hand." [USP]
Software development is enhanced with the proper development tools. Use of modern languages, utilities, techniques, resuable components, proper language for the job and team software. [UCP]
The old adage was to surround yourself with people smarter than yourself. Today the adage is to surround yourself with people who are dumber than yourself, non-threatening.
The intellectual drain leads to the low morale and high IT staff turnover. Boredom with poor designs and frustration at not being able to fix them lead smart programmers to resort to consultant work. This means that a business' IT stable poor to mediocre developers.
An example of this is the "help desk" phenomenon. The idea of hiring poorly qualified staff at low pay to organize software rollouts and do maintenance is patently absurd. One CIO told me that his help desk had reduced problems significantly, but he failed to mention that his staff had taken to diagnosing their own problems and sending the step-by-step solutions along with the fix request to the help desk staff. It was the only way they could get problems resolved. Programmers are unlikely to remain at such a company.
Failure must be tolerated [S p.149] if people are to learn, change and grow. It is necessary in order to stop intellectual drain.
People won't purchase your product if they don't know about it, no matter how great your program is. In the case of the Macintosh, Apple was so confident of success that they never bothered to recruit Mac developers. As a result, there was virtually no software for the Mac, despite the fact the Mac was the best computer available at the time. Finally Apple hired 3 full-time "evangelists" who spent 18 months doing nothing but convincing hundreds of developers to support the Mac. Without this, the Mac would have failed utterly. [Apple pg. 24]
Apple's Pink project spiraled out of control because the team leader had no authority to stop new features being thrown in. Unlike UNIX, this was a case of a project being destroyed because there was too many cooks. [Apple pg. 99] Apple's Pink, Jaguar and QuickDraw GX projects all suffered from explosive, uncontrolled team growth. Teams must remain focused and not attempt to do too much at once. This is also what killed Linux 2.4. The ideal team size is about 5 people, large projects no more that 4 sub-teams of 5. The Mac was built by a team of 20.
According to CBC's Venture, Japan entered a recession because Japanese banks funded any new startup. The government bailed out banks that got into trouble. Although this created a large number of successful new businesses and products, it left the government in debt and eventually it was forced to stop bailing out the banks.
According to Dilbert, "a Mission Statement is defined as 'a long, awkward sentence that demonstrates management's inability to think clearly.' All good companies have one." [BR15, pg. 41]
Compare mission statement to the Bible. The Bible's mission statement is the golden rule, but it requires 4 gospels to explain what that rule means.
Piracy is a reality: turn it to your advantage instead of fighting it wherever possible.
Obsolescence is a reality: executables are obsolete in 2 years. Source code in 7 years.
Every boss creates a division because people are split into camps. Many bosses, the so-called "matrix" environment, creates many divisions. The divisions created by the command structure must promote clear goals and cooperation. Companies like H-P and Motorola organize people into divisions for particular products so that marketing, developers, etc. all sink or swim with the success of the product. [Apple, p 189]
Labour pools do not work. [Slack, p8] Combining workers into a resource pool reduces responsiveness, expertise, and creates divisions across projects.
"Always give trust slightly in advance of demonstrated trustworthiness." [S p.152]
Nortel had a program where top employees received sabbaticals. [BP]
The First Law of Bad Management: if something isn't working, do more of it. Jerry Weinberg. [S. p.80]
The Second Law of Bad management: put yourself in as your own infielder. [S p.81]
The ITT study suggests that management must keep on top of productivity issues and not be caught sleeping at the switch. [USP]
8/16 Magazine suggested that the Gospel of John is the best marketing guidebook.
"Is there anything of which one can say "Look! This is something new?" Especially true of computers, there's no such thing as a concept that is truly new. Everything is built upon or borrowed from someone else. Never become arrogant over an original design. Instead, give credit where credit is due.
"I denied myself nothing my eyes desired; I refused my heart no pleasure...but everything was meaningless." [Eccl. 2 10,11] Managers who exploit the company and those under them for personal gain lose sight of the big picture. In the end, they will be alone and realize their deeds got them nothing of real value.
"The wise man, like the fool, will not be remembered" [Eccl. 2:16] Although being foolish is well understood as a recipe for disaster, so is wisdom an end to itself. No matter how much you study, no matter how much knowledge you collect, and no matter how many white papers you publish, the pursuit of knowledge will not save your job, let alone save your life.
"I hated all the things I had toiled for under the sun, because I must leave them to one who comes after me. And who knows whether he will be a wise man or a fool?" [Eccl. 2:18] As a manager, to work hard and to create grand designs, projects and profits to be left to a successor is foolish. If the successor is an idiot, he can ruin all your fine accomplishments. Instead work your allotted time and apply your efforts for the benefits of real people who are under you in the present.
"What does a man get for all the toil and anxious striving with which he labours under the sun? All his days his work is bain and grief; even at night his mind does not rest." [Eccl 2:22,23] Work your alotted time because that what you promised the company and what the people who depend on you expect. But don't become obsessed over projects or people. Carrying your problems home only brings misery to your time away from work. You are not responsible for the actions of others.
"[God] has made everything beautiful in its time." [Eccl 3:9] Find the most positive spin on every situation. Don't be blind to the negatives, but if you're obsessed with the problems presented to you, you will be left with depression caused by the "pitter patter of little de-feats". Not everything that appears to be a major crisis happens to be one.
"God will bring to judgement both the righteous and the wicked, for there will be a time for every activity, a time for every deed."
[Eccl 3:17] Do not let things slide that need to be dealt with. Don't hope things will "just go away".
TRW Defense Systems Group reported forcing poor developers to work on advanced projects was the most costly decision of a software project. [USP]
"Small scale [project] programming productivity measurement often reveals more than an order of magnitude of variation." [USP]
A Bell Labs study showed that star developers tended to be more productive because they had a good rapport with other staff and could count on them to be helpful and responsive. [S pg.106]
The reason development teams are limited to small numbers of people is the paths of interaction increase exponentially as people are added to a project. A four-person project has 6 avenues of interaction. A five-person team has 10. DeMarco argues that development teams are more efficient if they have one clerk to assist them because the clerk doesn't have to interact with the others in the development process. [S p. 78]
The ideal size for a team is a maximum of 7 people. Larger teams should be broken down into subteams of 7 or less. [UCP]
Scacchi reports an Australian study that "adding staff beyond the optimal point decreases productivity." [USP] ie. Appleworks vs. Lotus.
Software teams are, by their nature, a learning environment. [S p. 166] When, then, is learning often stifled?
Teams are enhanced when they evaluate their own performance and are acknowledged for producing good work. [UCP]
In Activision's early says, each game had one author but the other programmers tested and gave suggestions. The original Pitfall had only one life but at the insistence of the others David Crane added more lives. [GDG]
Pillip G. Armour recommends:
Participate in only one team at a time.
Created a shared goal set.
Develop a common language.
Have a designated location to work.
Experience a collective discomfort.
Experience some team-oriented successes.
Share leadership.
Talk about your strengths.
Share resources and knowledge
Reward both the team and the people on it.
"Training is to practice by doing a new task much more slowly than an expert would do it."[S p. 178] This is why corporate training is a complete failure.
Management is not leadership. [OMP]
"Lawrence also found that programming experience beyond the first year on the job, structured programming and walktroughs contributed little to productivity improvement." [USP]
The best programmer managers are those with persuasion, negotiation, favours, and trustworthiness. Authority carries little weight. [S p. 185]
Team diversity creates innovation and adds resources for problem solving.
Credentials doesn’t mean anything if the software doesn’t hold up in day-to-day use.
The problem of the "Black Hole of Complexity" has been steadily growing for the past two decades. Consider the Pentium processor line. Through pipelining, cashing, branch prediction and heuristics, Pentium class processors succeed in making lousy programmers look good. But it also succeeds in making good programmers look mediocre. The chip is so complex that it's nearly impossible to write a predictably optimal program.
Complexity has also made obsolete the idea of a "guru". Systems have so many different competing standards, developers and projects that no one can truly be called a Linux guru. No one is truly a master of all things related to Linux because it is humanly impossible to do so.
In "Rebirth of the Computer Industry", Howard Lawson speculates that open source encourages what he calls the "Black Hole of Complexity". Like a Mandelbrot Set, the larger the system, the worse the details. In open source, with each person pushing his own agenda, nobody takes responsibility for simplifying Linux. Instead, companies use the complexity of Linux in order to sell support, making a profit at the user's expense. [Rebirth, pg. 26]
Given enough time, even Linux will spiral out of control at the hands of open source developers, leading Raymonds assertion that open source reduces errors in the dust. Given a finite number of programmers with a finite amount of time and projects that grow infinitely complex, it is (theoretical) that all projects will eventually spiral out of control.
Scacchi reports that NASA/SEL "higher productivity over the entire system life cycle with the use of a disciplined programming methodology." [USP]
U.S. bank study suggests that modifying structured programs takes twice as long. [USP]
"The cost of code defects...costs less to repair a poor quality program [than a quality one]." Problem of overengineering.