Programming Is Mostly Thinking

Pretend you have a really great programming day. 

You only have to attend a few meetings, have only a few off-topic conversations, don't get distracted or interrupted much, don't have to do a bunch of status or time reporting, and you put in a good six hours of serious programming [note: this RARELY happens in an 8-10 hour day]. 

I want to review your work in the morning, so I print out a diff of your day's work before going home. 

Sadly, overnight the version control system crashes and they have to recover from the previous day's backup. You have lost an entire day's work. 

If I give you the diff, how long will it take you to type the changes back into the code base and recover your six-hours' work?

Programming is 11/12ths Thinking

I've been touting this figure for some time now, and people keep asking me where the study is that produced such an odd number.

Well, it's not pulled out of thin air and it's not the result of a thorough scientific study.

I have done informal polls now for a few years, though I've not kept good records. My goal was not to become the scientist who cracks the statistical/mathematical code for programming activities. I was looking for a reasonable answer to a reasonable question.

However, this answer surprised me. In a long Quora post titled "How do programmers code so quickly?" one responder offered that it was a combination of physical skills (muscle memory, skill with tools, debugging skills, typing skill) and knowing where to search for info. His post was swamped and overwhelmed by posts explaining that typing and tools are not the most important aid to quick code production.

Software Factories

 

I have seen the stickers and slogans on stickers and social media for a long time that "typing is not the bottleneck" (though every once in a while the inability of some programmers to type is a bottleneck).

I am keenly aware that most management still subscribes to the idea that motion is work. They are fairly convinced that a lack of motion is a lack of work. That makes sense in a lawn care service, a factory assembly line, or a warehouse operation.

Nearly all of the visible work done in producing physical goods is motion. People roll steel, stamp, press, mill, pick and place, bolt/screw/rivet, and on. 

Modern factories produce goods with Computer Numerical Control machines, which produce perfect copies of an original model that may not even exist in real life. These machines work from abstract models -- just data, really -- and perform perfect motion. Humans tend the machines, rather than working the wood by hand.

I have some great guitars that were produced at affordable costs because of the degree of automation brought by such machines. 

Great boutique guitars are produced entirely by hand at higher cost and I don't put down that effort either. The world has room for both.

Software developers have perfected the factory. It runs flawlessly bit-perfect copies. You just click the "copy" or "download" button. It's so cheap that the purchasers happily cover the costs of the factory. Those who are cautious will double check the checksums that come with the download, but most people don't bother. The machines are reliable and efficient and quick and cheap. 

Once the initial model (really, just data) exists, then the marginal cost of all the bit-perfect copies is essentially zero. Yes, this is just copying and not creating, but that's what factories do. Custom shops might produce unique items (like guitars) but factories create copies of originals.

The software factory tends to give you a progress bar, so you can visualize the motion of bits, but in many ways you can say that the product doesn't really exist. It's a pattern of tiny charged v. uncharged areas of metal on a plate (well, probably) and you don't even pay for the plate or the magnet or the laser when you create the copy. It's already there.

Software is an intellectual good.

The Design Shop

In my years of working with Uncle Bob Martin, I heard him continually tell customers and students that software development is not a fabrication operation, but a design operation. Once the initial design is done, all the duplication is done by machines at nearly zero cost.

So what programmers and testers and POs and Scrum Masters and software management area all doing (if they're doing it right) is designing the data model that will later be used by the factory to create copies for use by customers, patrons, and other people in the community the software is intended to serve. 

Yet the mechanistic, Industrial-Age idea of software development as a factory persists, and developers dutifully try to make it look like they're doing physical labor at the detriment of the process. 

All intellectual activities are hard to observe and monitor. An idea that is 80% complete has no physical manifestation. It's an idea, and it's not done yet. Sometimes we have experiments or proof-of-concept code or notes, but they don't give an accurate "% complete" number as does physical work.

A chair being manufactured looks about 50% done at the 50% mark.  When it's done, it looks done.

A design for a chair may not exist on paper until it is more than 70% complete. And we don't know that it's really 70% done, because it's not finished being designed yet.

The Answer: Really?

I have asked this question at conventions, client companies, to my peers, to colleagues, and to strangers I have met for the first time when I find out they are programmers.

The answer I receive most often is "about half an hour."

I could use the 8-hour day, ignoring meetings and interruptions and status reports, but that feels like padding the answer. I stick to the six hours doing things that programmers identify as programming work.

There are twelve half-hours in six hours. One half-hour to retype all the changes made in six hours of hard programming work.

What in the world can that mean? How can it be so little?

The Meaning Behind the Answer

Right now I suspect a bunch of managers are about to go yell at their programmers for putting in a half-hour's work in an 8-hour day! That would be a horrible misunderstanding of what was actually happening.

What is really happening?

• Programmers were typing on and off all day. Those 30 minutes are to recreate the net result of all the work they wrote, un-wrote, edited, and reworked through the day. It is not all the effort they put in, it is only the residue of the effort.
• Programmers are avoiding defects as best they can. In order to do that, they have to be continuously evaluating the code as they write it, hypothesizing the kinds of defects or security vulnerabilities they might be introducing. After all, they receive their harshest criticism for introducing defects into the shared code base. 
• Programming is a kind of lossy compression. The code only says what the program must do when it is running. Why a programmer chose one particular way over others, how it influences the rest of the system, what errors were introduced and removed, and what pitfalls it avoids are not (generally) present in the text of the program.
• Most of the work is not in making the change, but in deciding how to make the change. Deciding requires us to understand the code that already exists. This is especially time-consuming when the code is messy or the design is not very obvious in the source code. 
• Programmers work in a social context since all their results are integrated into a shared code base (and most use pair programming or other "many eyes" techniques). Programmers may be helping other programmers or testers or operations people get a handle on their work. Connecting and communicating with others has benefits and costs that don't appear in the code. 

Six hours of intellectual work (reading, researching, deciding, confirming, validating, verifying) translates to about 30 minutes worth of net lines-of-code change to a code base.

That's not additional lines of code. We often have weeks when we fix bugs and add features and have fewer lines of code at end of the week than we had at the beginning of the week. I once got in trouble for having multiple weeks where we had negative lines of code -- we didn't know the 'grand boss' over our team was reporting SLOC as if it measured progress. Sigh. 

Programmers will gladly explain that the work they did was reading, learning, understanding, sometimes guessing, researching, debugging, testing, compiling, running, hypothesizing and disproving their ideas of what the code should look like. In short, they were thinking and deciding.

Most of what goes on is intellectual work.

One of the quora responders wrote:

You see the fingers flying over the keyboard; you don't see the hours spent in talking to users, discussing the problems with coworkers, doing research and thinking the problems through.

Another suggested:

I achieve it firstly (to the extent that I do) by 'helping' the customer to eliminate the unnecessary notions from their idea, which they often mistakenly call 'requirements' and sometimes even say they are 'must have'. This is the biggest possible acceleration in the delivery of a solution because I can do an infinite amount of no work in no time at all.

And yet another:

Really good developers do 90% or more of the work before they ever touch the  keyboard; really understanding the requirements and devising an appropriate solution.

These are not unique unusual answers. I find that most of the time, "knowing what not to write", "doing less," "working in smaller steps", and "having first figured out what to do" are common answers. Programming is much more about thinking than about typing.

I have examined a lot of the change logs (diffs). It has consistently looked like 30+/-10 minutes of change on a good day (at least to me). 

I'm confident enough to tout this number as effectively true, though I should mention that no company I work with has so far been willing to delete a whole day's work to prove or disprove this experiment yet.  Remember, I have only estimates and examinations of daily diffs to work from. The result here is not scientific.

I should also let you know that people who do more typing or more cut/paste are often doing less thinking and understanding, which results in more errors and more burden on other programmers to understand and correct their code.

Code is just the residue of the work.

So What?

If programming is 1/12th motion and 11/12ths thinking, then we shouldn't push people to be typing 11/12ths of the time. We should instead provide the materials, environment, and processes necessary to ensure that the thinking we do is of high quality. 

Doing otherwise is optimizing the system for the wrong effect.

What if we changed our tactics, and intentionally built systems for thinking together about software and making decisions easier to make? I think that productivity lies in this direction.

So I invite you: how can you experiment with learning on-the-job to create systems where the thinking is optimized? 

Dave Coplin Reimagines The Office

Understand your office situation better w/RSA Animate & David Coplin



Raised many interesting points. I still see value in being able to pair and mob, and would like to have heard more talk about that, but I think his idea about being in control of how you work is important.

Enjoy.

Christopher Avery and The Responsibility Process (vid)

Here is Christopher Avery shows us a mental model that will help us to become more responsible.

I found it concise and helpful. I hope you may also.

Getting Through To Each Other

Communication is a very human process.

A quick model

Every being has its own mental model of a domain

Connected to it is a hearing/understanding apparatus. When you tell me the sky is beautiful, my mental model suggests it is a nice shade of blue and had some light, interesting clouds. But it could be that we don't share a model, and you meant really intense lightning and fast-moving thunderheads.

Provided that there are not too many great disconnects, though, what you tell me may provide information that I can add to my mental model. This is true whether I understand the words I heard in the same way that you meant them or not.

Recognize there is a difference between what I hear, and what I understand. It is sometimes said that "memory is the residue of thought" so my memory of our conversation may not be my memory of the sounds and words used, but of my thoughts/interpretation of the sentences as they occurred. I probably remember what I was thinking while you were talking, rather than remembering what you were saying.

Here's a quick picture:









Also connected are the meaning/saying side of things. From within my model is a thought I want to share with you. To communicate that I have to pick the words that I use to say it. Each of those words is a citizen of my mental model that may not fill the same role in yours.

Think about the process of a thought I think translating to words I say, translated while listening or reading into the thought you are thinking, to what you assimilate into your memory.

This is an amazing, fragile, and vulnerable act.

If our models are not too disconnected, then you may apprehend a reasonably close interpretation of the same thought I'm expressing, and you can add it to your model. Then you will repeat the entire process from thought to words to my mind via the same lossy process.

Of course, we're built for it. Even in our first months of life we develop this process, and we continue to improve or degrade it through our lives. Still we feel a little thrill when we reach alignment and each enrich our mental models through conversation or written word. It is an accomplishment!

And of course, it's no wonder we reach a state of frustration when we have overly high expectations of other people to interpret our words in light of our mental model and apprehend our meanings instead of constructing their own.

Credit Where Credit is Due

Years after writing this up, I find that someone else did it earlier and better and has been teaching it for years and years. Of course, that person is the preeminent Virginia Satir, whose shoes I am unworthy to lace.

One would do well to go to Satir's own writing, such as in PeopleMaking, to get the deeper picture, but I find it is also described (in short blog form) by Katrina Clokie in her blog on the Satir Interaction Model for those who are short on time.

What if we're disconnected?

What if I tell you things that contradict your mental model? It is uncomfortable. It unsettles our ability to communicate and share. Do we need to start all over? Did we never understand each other?

What are the options?

• Perhaps I'm being ironic. 
• Maybe I'm wrong. 
• Maybe I'm lying to you. 
• Maybe I have different meanings for the words I'm using.  
• Maybe there is something wrong with at least one of our mental models. 

It's unsure, complicated, and fairly deep. We feel frustration, and confusion. Some people would break off the conversation. This is why a mentor of mine once told me "never tell anyone something that they cannot hear." He advised instead, bridging the gap first -- and taking the time to do that well.


We get cognitive dissonance if we assume neither of is ill-intentioned. It's easier to write the other off as a liar or kook or moron. But those are the easy, value-less way out of the bind. Better that we try to understand the other person's mental model.

Most people are mis-trained to shy away from upset and frustration that would possibly help them understand another person's model. They likewise avoid the opportunities to improve or correct their own mental models.

Add to that the old Left Brain Interpreter, whose job it is to interpret our memories of ourselves in the most flattering and heroic terms possible. It tells us we're right and that it's wrong of people to make us feel confused or upset.

I suspect this is one of the reasons that programmers tend to plateau and quit learning after their fifth year of experience; they have enough of a model built up, and believe strongly enough in it, that they are able to easily reject anything that doesn't clearly agree with what they already know.

Conversations need to focus compassionately on the differences we have in our terminology or mental model.

It's hard to be human, and it's not something that we can do alone.

What I'm learning from this:

• Every skull is a cultural boundary. 
• Frustration exists to help me; I should learn from it instead of avoiding it 
• Using words in a straight-forward way is a kindness 
• Using pictures will get us past many of our terminology issues 
• Recognizing a mismatch in terminology v. model is hard, and important 
• Avery's Responsibility Process is a helpful mental model to help me adjust my terminology and mental model. 
• We all live far beneath the endowments given to us mentally, and walk past dozens of lessons every day that could enrich our lives.

If you find the model flawed, or incomprehensible, or not particularly useful please join in conversation with me here. I'm happy to explain, rephrase, or even rebuild the whole model if it helps us communicate more clearly and freely. After all, you have a lot of information I could use.