Donald Knuth on work habits, problem solving, and happiness
Shuvomoy Das Gupta
April 13, 2020
Last update: May 2, 2022
Recently, I came across a few old and new interviews of Donald Knuth (sources: (i) Companion to the Papers of Donald Knuth, (ii) Interviews conducted by Lex Fridman Part 1 and Part 2), where he sheds light on his work habits, how he approaches problems, and his philosophy towards happiness. I really enjoyed reading the interviews. In this blog, I am recording his thoughts on approaching a problem, organizing daily activities, and the pursuit of happiness.
Seeing both the forest and the trees in research. "I've seen many graduate students working on their theses, over the years, and their research often follows a pattern that supports what I'm trying to explain. Suppose you want to solve a complicated problem whose solution is unknown; in essence you're an explorer entering into a new world. At first your brain is learning the territory, and you're making tiny steps, baby steps in the world of the problem. But after you've immersed yourself in that problem for awhile then you can start to make giant steps, bigger steps, and you can see many things at once, so your brain is getting ready for a new kind of work. You begin to see both the forest and the trees."
How Knuth works on a project. "When I start to investigate some topic, during the first days I fill up scratch paper like mad. I mean, I have a huge pile of paper at home, paper that's half-used, used on only one side; I've kept a lot of partially printed sheets instead of throwing them away, so that I can write on the back sides. And I'll use up 20 sheets or more per hour when I'm exploring a problem, especially at the beginning. For the first hour I'm trying all kinds of stuff and looking for patterns. Later, after internalizing those calculations or drawings or whatever they are, I don't have to write quite so much down, and I'm getting closer to a solution. The best test of when I'm about ready to solve a problem is whether or not I can think about it sensibly while swimming, without any paper or notes to help out. Because my mind is getting accustomed to the territory, and finally I can see what might possibly lead to the end. That's oversimplifying the truth a little bit, but the main idea is that, with all my students, I've noticed that they get into a mental state where they've become more familiar with a certain problem area than anybody else in the world."
Visualizers vs Symbolizers. "Well, you know, I’m visualizing the symbols. To me, the symbols are reality, in a way. I take a mathematical problem, I translate it into formulas, and then the formulas are the reality. I know how to transform one formula into another. That should be the subtitle of my book Concrete Mathematics: How to Manipulate Formulas. I’d like to talk about that a little.
I have a feeling that a lot of the brightest students don’t go into mathematics because–-curious thing–-they don’t need algebra at the level I did. I don’t think I was smarter than the other people in my class, but I learned algebra first. A lot of very bright students today don’t see any need for algebra. They see a problem, say, the sum of two numbers is 100 and the difference is 20, they just sort of say, “Oh, 60 and 40.” They’re so smart they don’t need algebra. They go on seeing lots of problems and they can just do them, without knowing how they do it, particularly. Then finally they get to a harder problem, where the only way to solve it is with algebra. But by that time, they haven’t learned the fundamental ideas of algebra. The fact that they were so smart prevented them from learning this important crutch that I think turned out to be important for the way I approach a problem. Then they say, “Oh, I can’t do math.” They do very well as biologists, doctors and lawyers."
What graduate students should do when they have expertise in a certain area. "When they [the students] reach this point [expertise in a certain area] I always tell them that now they have a responsibility to the rest of us. Namely, after they have solved their thesis problem and trained their brain for this problem area, they should look around for other, similar problems that require the same expertise. They should use their expertise now, while they have this unique ability, because they're going to lose it in a month. I emphasize that they shouldn't be satisfied with solving only one problem; they should also be thinking about other interesting problems that could be handled with the same methods."
On the importance of anthropomorphizing a problem. "Another aspect of role playing is considerably more important: We can often make advances by anthropomorphizing a problem, by saying that certain of its aspects are "bad guys" and others are "good guys," or that parts of a system are "talking to each other." This approach is helpful because our language has lots of words for human relationships, so we can bring more machinery to bear on what we're thinking about."
Why putting the discovery of a solution on paper is important. "Well, I have no sympathy with people who never write up an answer; it's selfish to keep beautiful discoveries a secret. But I can understand a reluctance to write something up when another problem has already grabbed your attention. I used to have three or four papers always in sort of a pipeline, waiting for their ideas to mature before I would finally prepare them for publication.
Frances Yao once described the situation very nicely. She said, you work very hard on a problem for a long time, and then you get this rush, this wonderful satisfaction when you've solved it. That lasts about an hour. And then you think of another problem, and you're consumed with curiosity about the answer to that new one. Again, your life isn't happy until you find the next answer."
The philosophy behind seeking solutions. "The process of seeking solutions is certainly a big part of a researcher's life, but really it's in everybody's life. I don't want to get deep into philosophy, but the book of Ecclesiastes in the Bible says essentially this:
Life is hard and then you die. You can, however, enjoy the process of living; don't worry about the fact that you're going to die. Some bad people have a good life, and some good people have a bad life, and that doesn't seem fair; but don't worry about that either. Just think about ways of enjoying the journey.
Again I'm oversimplifying, but that's the message I find in many parts of the Bible. For example, it turns up in Philippians 3:16, where the writer says that:
You don't race to get to the goal; the process of racing itself, of keeping the pace, is the real goal.
When I go on vacation, I like to enjoy the drive.
In Christian churches I am least impressed by a sermon that talks about how marvelous heaven is going to be at the end. To me that's not the message of Christianity. The message is about how to live now, not that we should live in some particular way because there's going to be pie in the sky some day. The end means almost nothing to me. I am glad it's there, but I don't see it as much of a motivating force, if any. I mean, it's the journey that's important."
Knuth's process of reading papers. "It turns out that I read everything at the same slow rate, whether I'm looking at light fiction or at highly technical papers. When I browse through a journal, the titles and abstracts of papers usually don't help me much, because they emphasize results rather than methods; therefore I generally go through page by page, looking at the illustrations, also looking for equations that are somehow familiar or for indications of useful techniques that are unfamiliar.
Usually a paper lies outside the scope of my books, because I've promised to write about only a rather small part of the entire field of computer science. In such cases there's nothing new for me to worry about, and I happily turn the pages, zipping to the end. But when I do find a potentially relevant paper, I generally read it only partway, only until I know where it fits into the table of contents of The Art of Computer Programming. Then I make myself a note, to read it later when I'm writing up that section. Sometimes, however—as happened last night with that paper about scheduling games of bridge—I get hooked on some question and try to explore it before I'm ready to move on to reading any other papers.
Eventually when I do begin to write a section of my book, I go into "batch mode" and read all of the literature for which my files point to that section, as well as all of the papers that those papers cite. I save considerable time by reading several dozen papers on the same topic all in the same week, rather than reading them one by one as they come out and trying to keep infinitely many things in my head all at once.
When I finally do get into batch mode, I go very carefully through the first two or three papers, trying to work the concepts out in my own mind and to anticipate what the authors are going to say before turning each page. I usually fail to guess what the next page holds, but the fact that I've tried and failed makes me more ready to understand why the authors chose the paths that they did. Frequently I'll also write little computer programs at this point, so that the ideas solidify in my head. Then, once I've gone slowly through the first few papers that I've accumulated about some topic, I can usually breeze through the others at a comparatively high speed. It's like the process of starting with baby steps and progressing to giant steps that I described earlier."
On parts of research that are much less fun. "Well, some parts of a job are always much less fun than others. But I've learned to grin and bear it, to bite the bullet and move on, to face the music, to take it in stride and make a virtue of necessity. (Excuse me for using so many clichés, but the number of different popular expressions tends to make my point.)"
On scheduling daily activities. "I schedule my activities in a somewhat peculiar way. Every day I look at the things that I'm ready to do, and choose the one that I like the least, the one that's least fun — the task that I would most like to procrastinate from doing, but for which I have no good reason for procrastination. This scheduling rule is paradoxical because you might think that I'm never enjoying my work at all; but precisely the opposite is the case, because I like to finish a project. It feels good to know that I've gotten through the hurdles."
My scheduling principle is to do the thing I hate most on my to-do list.
On pursuing a PhD. "A PhD is awarded for research, meaning that the student has contributed to the state of the world's knowledge. That's quite different from a bachelor's degree or a master's degree; those degrees are awarded for a mastery of existing knowledge. (In some non-science fields, like Art, a master's degree is more akin to a PhD; but I'm speaking now about the situation in mathematics and in the sciences.) My point is that it's a mistake to think of a PhD as a sort of next step after a BS or MS degree, like advancing further in some academic straight line. A PhD diploma is another animal entirely; it stands for a quite different kind of talent, which is orthogonal to one's ability to ace an examination. A lot of people who are extremely bright, with straight A+ grades as undergraduates, never get a PhD. They're smart in a way that's different from "research smart." I think of my parents, for example: I don't believe either one of them would have been a good PhD candidate, although both were extremely intelligent.
It's extremely misleading to rank people on an IQ scale with the idea that the smarter they are, the more suitable they are for a PhD degree; that's not it at all. People have talents in different dimensions, and a talent for research might even have a negative correlation with the ability to tie your own shoes."
Whether volunteering helps Knuth with his principal vocation. "Well, you're absolutely right. I can't do technical stuff all the time. I've found that I can write only a certain number of pages a day before running out of steam. When I reach this maximum number, I have no more ideas that day. So certainly within a 24-hour period, not all of it is going to be equally creative. Working in the garden, pulling weeds and so on, is a good respite. I recently got together with some friends at Second Harvest, repackaging food from one place to another. This kind of activity, using my hands, provides variety and doesn't really take away from the things I can do for the world."
On unhappiness. "I mean, if you didn't worry, and if you didn't go through some spells and crises, then you'd be missing a part of life. Even though such things aren't pleasant when you're doing them, they are the defining experiences — things to be glad about in retrospect because they happened. Otherwise you might be guilty of not feeling guilty!
On the other hand I've noticed in myself that there were times when my body was telling me to be unhappy, yet I sometimes couldn't readily figure out a reason for any unhappiness. I knew that I was feeling "down," but sometimes I had to go back several months to recall anything that anybody had said to me that might still be making me feel bad. One day, when I realized how hard it was to find any reason for my current unhappiness, I thought, "Wait a minute. I bet this unhappiness is really something chemical, not actually caused by circumstances.*" I began to speculate that my body was programmed to be unhappy a certain percentage of the time, and that hormones or something were the real reason behind moments of mild depression."
Why power corrupts. "When people have more power and they get richer, and they find themselves rich but still unhappy, they think, "Hmmm, I'll be happy if I only get rid of all the sources of my unhappiness." But the action of removing annoyances sometimes involves abusing their power. I could go on and on in this vein, I guess, because you find that in the countries where there is a great difference between rich and poor, the rich people have their problems, too. They haven't any motivation to change the way they're living, exploiting others, because as far as they can see, their own life isn't that happy. But if they would only realize that their unhappy spells are part of the way that they're made, and basically normal, they wouldn't make the mistake of blaming somebody else and trying to get even for imagined misdeeds."
Point eight is enough. "In fact I've concluded that it's really a good thing for people not to be 100% happy. I've started to live in accordance with a philosophy that can be summed up in the phrase "Point eight is enough," meaning "0.8 is enough."
You might remember the TV show from the 70s called "Eight is Enough," about a family with eight children. That's the source of my new motto. I don't know that 0.8 is the right number, but I do believe that when I'm not feeling 100% happy, I shouldn't feel guilty or angry, or think that anything unusual is occurring. I shouldn't set 100% as the norm, without which there must be something wrong. Instead, I might just as well wait a little while, and I'll feel better. I won't make any important decisions about my life at a time when I'm feeling less than normally good.
In a sense I tend now to suspect that it was necessary to leave the Garden of Eden. Imagine a world where people are in a state of euphoria all the time — being high on heroin, say. They'd have no incentive to do anything. What would get done? What would happen? The whole world would soon collapse. It seems like intelligent design when everybody's set point is somewhere less than 100%."
High minimum more important than high maximum. "I try to do a good job at whatever I'm doing, because it's more fun to do a good job than not. And when there's a choice between different things to spend time on, I try to look for things that will maximize the benefit without making me burn out.
For example, when I was working on the TeX project during the early 80s, hardly anybody saw me when I was sweeping the floor, mopping up the messes and carrying buckets of waste from the darkroom, cleaning the machines, and doing other such stuff. I did those things because I wouldn't have dared to ask graduate students to do menial tasks that were beneath them.
I know that every large project has some things that are much less fun than others; so I can get through the tedium, the sweeping or whatever else needs to be done. I just do it and get it over with, instead of wasting time figuring out how not to do it. I learned that from my parents. My mother is amazing to watch because she doesn't do anything efficiently, really: She puts about three times as much energy as necessary into everything she does. But she never spends any time wondering what to do next or how to optimize anything; she just keeps working. Her strategy, slightly simplified, is, "See something that needs to be done and do it." All day long. And at the end of the day, she's accomplished a huge amount.
Putting this another way, I think that the limiting thing — the thing that determines a person's success in life — is not so much what they do best, but what they do worst. I mean, if you rate every aspect of what someone does, considering everything that goes into a task, a high minimum is much more important than a high maximum. The TeX project was successful in large part because I quietly did things like mop the floor. The secret of any success that I've had, similarly, is that in all the projects I've worked on, the weakest link in my chain of abilities was still reasonably strong."
A person’s success in life is determined by having a high minimum, not a high maximum. If you can do something really well but there are other things at which you’re failing, the latter will hold you back. But if almost everything you do is up there, then you’ve got a good life. And so I try to learn how to get through things that others find unpleasant.
A guiding heuristic. "Don’t just do trendy stuff. If something is really popular, I tend to think: back off. I tell myself and my students to go with your own aesthetics, what you think is important. Don’t do what you think other people think you want to do, but what you really want to do yourself. That’s been a guiding heuristic for me all the way through."
Source of humility. "I wrote a couple of books, including Things a Computer Scientist Rarely Talks About, that are about theology — things you can’t prove — rather than mathematics or computer science. My life would not be complete if it was all about cut and dried things. The mystical things I don’t understand give me humility. There are things beyond my understanding.
In mathematics, I know when a theorem is correct. I like that. But I wouldn’t have much of a life if everything were doable. This knowledge doesn’t tear me apart. Rather, it ensures I don’t get stuck in a rut."
Meaning of life. "I personally think of my belief that God exists although I have no idea what that means. But I believe that there is something beyond human capabilities and it might be some AI. Whatever it is, I do believe that there is something that goes beyond human understanding but that I can try to learn more about how to resonate with whatever that being would like me to do. I strive for that (occasional glimpses of that being) not that I ever think I am going to get close to it. I try to imagine that I am following somebody's wishes and this AI or whatever it is, it is smart enough to give me clues."