1) Introduction

• You're going to spend half your professional life debugging
  • So you should learn how to do it systematically
• Talk about tools first
  • They'll make everything else less painful
• Then some techniques
• Assume for now that you built the right thing the wrong way
  • Requirements errors are actually a major cause of software project failure
  • But out of scope for this course

2) You Can Skip This Lecture If...

• You know how to use a symbolic debugger
  • Just knowing what one is doesn't count
• You know what a breakpoint is
• You know what conditional breakpoints are good for
• You know what logging is
• You follow Agans' Rules

3) Key Themes

• A debugger is a must-know tool
  • Not everything in software should be solved with yet-another-tool
  • But, a debugger is definitely one to learn.
• A few hours spent learning a debugger will literally save you hundreds of hours of wasted effort later
  • Learning the top 10 features of a debugger will get you 90% of the benefit
• A debugger is to a programmer, what a mitre saw is to a modern carpenter
  • With great(er) effort, you could probably build something without one
  • But, with one, you can get a lot more done, and with less grief

4) What's Wrong with Print Statements

• Many people still debug by adding print statements to their programs
• It's error-prone
  • Adding print statements is a good way to add typos
  • Particularly when you have to modify the block structure of your program
• And time-consuming
  • All that typing...
  • And (if you're using Java, C++, or Fortran) all that recompiling...
• And can be misleading
  • Moves things around in memory, changes execution timing, etc.
  • Common for bugs to hide when print statements are added, and reappear when they're removed

5) Symbolic Debuggers

• A debugger is a program that runs another program on your behalf
  • Sometimes called a symbolic debugger because it shows you the source code you wrote, rather than raw machine code
• While the target program (or debuggee) is running, the debugger can:
  • Pause, resume, or restart the target
  • Display or change values
  • Watch for calls to particular functions, changes to particular variables, etc.
• Do not need to modify the source of the target program!
  • Depending on your language, you may need to compile it with different flags
• And yes, the debugger modifies the target's layout in memory, and execution speed...
  • ...but a lot less than print statements...
  • ...with a lot less effort from you

6) Debugger Features

• Interactive debuggers typically show:
  • The source code
  • The call stack
  • The values of variables that are currently in scope
    • I.e., global variables, parameters to the current function call, and local variables in that function
  • A panel displaying what your program has printed to standard output and/or standard error
• We'll use [WingIDE] in this lecture

7) Kinds of Debuggers

• There may be several ways to get into the debugger
  • Launch the debugger, load the target program, and start work
  • Run the debugger with the target program as a command-line argument
  • Switch into debugging mode in the middle of an interactive session
• Sometimes also do post mortem debugging
  • When a program fails badly, it creates a core dump
    • Copies all of its internal state to a file on disk
  • Load that dump into the debugger, and see where the program was when it terminated
    • Not as good as watching it run...
    • ...but sometimes the best you can do

8) Integrated Development Environments

• Debuggers are usually part of integrated development environments (IDEs)
• Tools like this are available for every modern language
  • [Microsoft Visual Studio] on Windows
  • [Eclipse] for Java (and now C++)
• Also usually contain a class browser that presents an outline of the project's modules, classes, functions, variables, etc.
• For example, the WING IDE make it easy to browse functions (open file /usr/share/pyshared/numpy/lib/polynomial.py in the VMware image):

9) Command-Line Debuggers

• Many of today's debuggers are GUIs wrapped around older command-line debuggers
• Most widely used of these is [GDB]
  • Supports many languages, on many platforms
  • But no one ever said it was easy to learn
• Python comes with a simple debugger called pdb
  • Can be invoked by calling pdb.set_trace() inside a program

• Here's a program in file set-trace.py:

import pdb

base = "Na"

pdb.set_trace()

acid = "Cl"
salt = base + acid
print salt

• You can run it with the command, then single-step interactively:

user@user-desktop:~$ python set-trace.py

> /home/user/set-trace.py(7)<module>()
-> acid = "Cl"
(Pdb) n
> /home/user/set-trace.py(8)<module>()
-> salt = base + acid
(Pdb) n
> /home/user/set-trace.py(9)<module>()
-> print salt
(Pdb) n
NaCl
--Return--
> /home/user/set-trace.py(9)<module>()->None
-> print salt
(Pdb) n

10) Inspecting Values

• Use the debugger to set breakpoints in the target program
  • Tells the target program to pause when it reaches particular lines of code
• When the target program is paused, the debugger can display the contents of its memory
• Most debuggers can also evaluate expressions using the current values of variables
  • E.g., type in 2*x<0, debugger displays False

11) Controlling Execution

• The debugger can also:
  • Single-step (i.e., execute one statement at a time)
  • Step into function calls
  • Step over them, or
  • Run to the end of the current function
• Allows you to see:
  • How values are changing
  • Which branches the program is actually taking
  • Which functions are actually being called
• Debuggers really should be called "inspectors"

• Programs are "just" data
• Some bytes hold values that represent instructions
  • Each statement in the source program typically corresponds to several instructions
  • The compiler (or interpreter) keeps track of which lines of code produced which instructions
• Other bytes hold constants and variables
  • The static space contains constant strings, magic numbers, etc.
  • The call stack holds function parameters
  • The heap is all dynamically-created objects
• Some registers
  • An instruction pointer that keeps track of what to execute next
  • A stack pointer to keep track of the call stack
  • Miscellaneous other registers for doing arithmetic, etc.

12) Inspecting More Values

• Debugger lets you move up and down the call stack
  • Allows you to run to the problem, then figure out how you got there
• Also lets you modify values
  • If you have a theory about why a bug is occurring:
    • Run to that point
    • Set variables' values (e.g., set max_temp to -1)
    • Resume execution
  • Sometimes used to test error handling code
    • Easier to change time_spent_waiting to 600 seconds in debugger than to pull out the network cable and wait...

13) Conditional Breakpoints and Watchpoints

• The program only stops at a conditional breakpoint if some condition is met
  • E.g., loop index greater than 100, or filename argument is None
  • Much more efficient than single-stepping from the start of the program
• Some debuggers also support watchpoints
  • Have the debugger watch every write to memory
  • Halt when anything, anywhere, modifies a particular variable
  • Slow the program down a lot
    • Typically a factor of 100 or more
  • But sometimes the only practical way to find out when a particular list value is being overwritten

14) Agans' Rules

• Many people make debugging harder than it needs to be by:
  • Using inadequate tools
  • Not going about it systematically
  • Becoming impatient
• Agans' Rules [Agans 2002] describe how to apply the scientific method to debugging
  • Observe a failure
  • Invent a hypothesis explaining the cause
  • Test the hypothesis by running an experiment (i.e., a test)
  • Repeat until the bug has been found

15) Rule 0: Get It Right the First Time

• The simplest bugs to fix are the ones that don't exist
• Design, reflect, discuss, then code
  • "A week of hard work can sometimes save you an hour of thought."
• Design and build your code with testing and debugging in mind
  • Minimize the amount of "spooky action at a distance"
  • Minimize the number of things programmers have to keep track of at any one time
  • Train yourself to do things right, so that you'll code well even when you're tired, stressed, and facing a deadline
• "Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" (Brian Kernighan)

16) Rule 1: What Is It Supposed to Do?

• First step is knowing what the problem is
  • "It doesn't work" isn't good enough
  • What exactly is going wrong?
  • How do you know?
  • You will learn a lot by following execution in a debugger and trying to anticipate what the program is going to do next
• Requires you to know how the software is supposed to behave
  • Is this case covered by the specification?
  • If not:
    • Do you have enough knowledge to extrapolate?
    • Do you have the right to do so?
• Try not to let what you want to see influence what you actually observe
  • It's harder than you'd think [Hock 2004]

17) Rule 2: Is It Plugged In?

• Are you actually exercising the problem that you think you are?
  • Are you giving it the right test data?
  • Is it configured the way you think it is?
  • Is it the version you think it is?
  • Has the feature actually been implemented yet?
  • Why are you sure?
    • Maybe the reason you can't isolate the problem is that it's not there
• Another argument in favor of automatic regression tests
  • Guaranteed to rerun the test the same way each time
• Also a good argument against automatic regression tests
  • If the test is wrong, it will generate the same misleading result each time

18) Rule 3: Make It Fail

• You can only debug things when they go wrong
• So find a test case that makes the code fail every time
  • Then try to find a simpler one
  • Or start with a trivially simple test case that passes, then add complexity until it fails
• Each experiment becomes a test case
  • So that you can re-run all of them with a single command
  • How else are you going to know that the bug has actually been fixed?
• Use the scientific method
  • Formulate a hypothesis, make a prediction, conduct an experiment, repeat
  • Remember, it's computer science, not computer flip-a-coin

19) Alternatives

• What if you can't make it fail reliably?
  • Problem involves timing, network load, etc.
  • Or you just don't know enough about the cause yet
• Use post-mortem inspection
  • But then you have to reason backwards to figure out why the program crashed
• Or logging
  • But this can distort the program's behavior
  • And you'll have to wade through a lot of irrelevant information

20) Rule 4: Divide and Conquer

• The smaller the gap between cause and effect, the easier the relationship is to see
• So once you have a test that makes the system fail, use it isolate the faulty subsystem
  • Examine the input of the code that's failing
  • If that's wrong, look at the preceding code's input, and so on
• Use assert to check things that ought to be right
  • "Fail early, fail often"
  • A good way to stop yourself from introducing new bugs as you fix old ones
• When you do fix the bug, see whether you can add assertions to prevent it reappearing
  • If you made the mistake once, odds are that you, or someone, will make it again
• Another argument against duplicated code
  • Few things are as frustrating as fixing a bug, only to have it crop up again elsewhere

21) Rule 5: Change One Thing at a Time, For a Reason

• Replacing random chunks of code unlikely to do much good
  • If you got it wrong the first time, what makes you think you'll get it right the second? Or the ninth?
  • So always have a hypothesis before making a change
• Every time you make a change, re-run all of your tests immediately
  • The more things you change at once, the harder it is to know what's responsible for what
  • And the harder it is to keep track of what you've done, and what effect it had
  • Changes can also often uncover (or introduce) new bugs

22) Rule 6: Write It Down

• Science works because scientists keep records
  • "Did left followed by right with an odd number of lines cause the crash? Or was it right followed by left? Or was I using an even number of lines?"
• Records particularly useful when getting help
  • People are more likely to listen when you can explain clearly what you did

23) Rule 7: Be Humble

• If you can't find it in 15 minutes, ask for help
  • Just explaining the problem aloud is often enough
  • "Never debug standing up." (Gerald Weinberg)
• Don't keep telling yourself why it should work: if it doesn't, it doesn't
  • Never debug while grinding your teeth, either...
• Keep track of your mistakes
  • Just as runners keep track of their time for the 100 meter sprint
  • "You cannot manage what you cannot measure." (Bill Hewlett)
• And read [Zeller 2006] to learn more

24) Summary

• Debugging is not a black art
• Like medical diagnosis, it's a skill that can be studied and improved
• You're going to spend a lot of time doing it: you might as well learn how to do it well