Debugging
For debugging several debuggers available for the compiler packages Intel Cluster Studio and PGI Compiler. Additionally the graphical frontend ddd for the GNU debugger gdb is available as module.
Debugging using the GNU debugger
The GNU Project debugger (GDB) is a valuable tool that might help you to analyze what caused your program to crash, i.e. it assists you in the process of debugging. By means of GDB you can basically execute your program in a line-by-line manner, stop at specified positions and display or alter the state of all the variables therein. GDB is able to debug programs written in many compiled languages such as C and C++. Below we will consider a basic (malfunctioning) program written in C and see how GDB can be used to discover where the problem is located. This example is meant to illustrate several features of GDB
A basic malfunctioning example program
So as to illustrate how to use GDB in order to detect bugs in a program, consider the following source code for a malfunctioning program written in C (with annotated line numbers), here referred to as myExample.c:
1 #include <stdio.h>
2 #include <stdlib.h>
3
4
5 void listArray(int *, int);
6
7
8 void listArray(int *myArray, int N){
9 int i;
10
11 printf("array = [");
12 for(i=0; i<N; i++){
13 printf("%d ",myArray[i]);
14 }
15 printf("]\n");
16
17 }
18
19
20 int main(int argc, char *argv[]) {
21 int *myArray,N,i;
22
23 N=10;
24 myArray = (int *) malloc(-N*sizeof(int));
25
26 for(i=0; i<N; i++){
27 myArray[i]=i;
28 }
29
30 free(myArray);
31 return 0;
32 }
Besides the main function, which simply allocates an array and initializes its entries, the program implements one additional function called listArray, which lists a specified number of leading array entries.
Compiling for elaborate debugging information
So as to benefit from GDB you need to compile your program so that it provides further debugging information. Using gcc this is done by adding the compiler option -g. For the above example you might thus type
gcc myExample.c -o myExample -g
This then yields the executable myExample, compiled using further debugging symbols.
A debugging session using GDB
If you invoke the program in a straight foreward manner it will fail, resulting in
$ ./myExample Segmentation fault
So, why is this? Albeit you might have already spotted the reason for the segmentation fault, lets use GDB to hunt down the error! To start the GDB tool in order to debug the example program simply type
gdb myExample
on the command line. This will enter the interactive GDB mode, only. It will not run your program right away!
To have a look at the source code that referst to the executable myExample you might use the GDB command list in one of several ways. If you vaguely remember the name of a GDB command but cannot recall how to properly use it, you can use the help function to find out about the command. E.g. to obtain details on the list command, simply type:
(gdb) help list List specified function or line. With no argument, lists ten more lines after or around previous listing. "list -" lists the ten lines before a previous ten-line listing. One argument specifies a line, and ten lines are listed around that line. Two arguments with comma between specify starting and ending lines to list. Lines can be specified in these ways: LINENUM, to list around that line in current file, FILE:LINENUM, to list around that line in that file, FUNCTION, to list around beginning of that function, FILE:FUNCTION, to distinguish among like-named static functions. *ADDRESS, to list around the line containing that address. With two args if one is empty it stands for ten lines away from the other arg.
Hence to see the beginning of the main function one has to type
(gdb) list main
15 printf("]\n");
16
17 }
18
19
20 int main(int argc, char *argv[]) {
21 int *myArray,N,i;
22
23 N=10;
24 myArray = (int *) malloc(-N*sizeof(int));
which yields the 10 surrounding lines of the beginning of the main function. To get the next 10 lines you might just type enter.
Now, in order to execute your program type
(gdb) run Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/gdb/c/myExample Program received signal SIGSEGV, Segmentation fault. 0x000000000040070f in main (argc=1, argv=0x7fffffffe5a8) at myExample.c:27 27 myArray[i]=i;
So, there appears to be a problem at line 27 in the source code, at which point GDB reports a segmentation fault. To have a look at the context where the error occured you can type list 27 to find:
(gdb) list 27
22
23 N=10;
24 myArray = (int *) malloc(-N*sizeof(int));
25
26 for(i=0; i<N; i++){
27 myArray[i]=i;
28 }
29
30 free(myArray);
31 return 0;
One way to proceed is to clean restart and set a so called breakpoint at line
27. This will case the program flow to interupt as soon as it reaches line 27, right
before the command in that line, namely myArray[i]=i;, is executed:
(gdb) kill Kill the program being debugged? (y or n) y (gdb) break 27 Breakpoint 1 at 0x4006f8: file myExample.c, line 27. (gdb) run Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/gdb/c/myExample warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000 Breakpoint 1, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:27 27 myArray[i]=i;
Now, you might display the variables befor line 27 is executed. E.g. to see the current value of
the iteration variable i you can type print i to find
(gdb) print i $1 = 0
Similarly, the address in memory of the array myArray can be listed by typing
(gdb) print myArray $2 = (int *) 0x0
At this point note that the address of the array myArray looks awkward. Obviousely something went wrong!
It should rather look similar to something like 0x7fffffffe5a8, i.e. the address of the
array argv in the argument list of the main function! Now, 0x0 is the hexadecimal variant of 0. Hence, at line 27, myArray points to NULL. At this point, bear in mind that
whenever you try to allocate memory for a data structure and something went wrong during the allocation
procedure via malloc and no memory is reserved, the NULL pointer is given as a
return value by malloc. Hence, the
address 0x0 is a hint that something went wrong during the allocation procedure.
To bracket the error further, lets see how the allocation procedure affects the pointer myArray.
The memory allocation is done at line 24, so lets assign an additional break point via break 24
and do a clean restart by again calling kill. By the way, if you assigned several breaking points already and want to
have a detailed look at them or simply overview them, you can type
(gdb) info b
Num Type Disp Enb Address What
1 breakpoint keep y 0x00000000004006f8 in main at myExample.c:27
breakpoint already hit 1 time
2 breakpoint keep y 0x00000000004006d8 in main at myExample.c:24
Now, running the program will first stop prior to the command issued in line 24, where we might check the address of the array myArray before and after the attempted memory allcation:
(gdb) run
Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/gdb/c/myExample
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000
Breakpoint 2, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:24
24 myArray = (int *) malloc(-N*sizeof(int));
(gdb) print myArray
$3 = (int *) 0x7fffffffe5a0
(gdb) next
26 for(i=0; i<N; i++){
(gdb) print myArray
$4 = (int *) 0x0
And indeed, by looking at the arguments of the function malloc it should be clear that the minus sign
supplied with the size of the desired memory block caused the trouble.
Now, correcting the error by changing line 24 to
myArray = (int *) malloc(N*sizeof(int));
yields a functioning program.
Further features of GDB
To illustrate further features of GDB, consider the corrected variant of the example program. Starting a GDB session for the corrected program offers the possibility to further explore some of its features:
Modifying the value of variables
Say you set a breakpoint at line 24, looking up the value of N yields the answer 10 of course.
If you want to change the value of N for the remaining session to the value 15, you might type
set var N=15
Calling functions within GDB
Say you set a breakpoint at line 29, i.e. right after the values of the array are initialized. If you are
interested in whether all entries are properly initialized you can call the function listArray
declared in the source code by typing:
(gdb) break 29 Breakpoint 1 at 0x40071b: file myExample.c, line 29. (gdb) run Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/gdb/c/myExample warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000 Breakpoint 1, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:30 30 free(myArray); (gdb) call listArray(myArray,N) array = [0 1 2 3 4 5 6 7 8 9 ]
The possibility to directly use functions declared in the source code is especially useful if you want to display the content of intricate custom data structures.
Listing array entries
A more easy way to list single entris of an array and whole ranges of array entries is as follows (continuing the previous GDB session):
(gdb) print myArray[0]
$1 = 0
(gdb) print myArray[0]@5
$2 = {0, 1, 2, 3, 4}
(gdb) print myArray[0]@15
$3 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 135121, 0, 0, 0, 0}
Note that in the last line, also parts in the memory outside of the range of the array myArray are displayed, which is due to the particular way @X addresses the entries of the array. If you are familiar with the concept of pointers, you might immediately think of the correspondance between pointers and addresses in memory. By using the dereference operator, i.e. the prefix *, you can access the value stored at the respective memory address:
(gdb) print *(myArray) $4 = 0 (gdb) print *(myArray+1) $5 = 1 (gdb) print *(myArray+10) $6 = 135121
Executing source code line by line
Say you set a breakpoint at line 27. Then you can continue to execute the program line by line by typing the GDB command step. As an additional argument you can specify the number of times the command should be executed:
(gdb) break 27 Breakpoint 1 at 0x4006f6: file myExample.c, line 27. (gdb) run Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/gdb/c/myExample warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000 Breakpoint 1, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:27 27 myArray[i]=i; (gdb) print i $1 = 0 (gdb) step 2 Breakpoint 1, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:27 27 myArray[i]=i; (gdb) print i $2 = 1 (gdb) step 2 Breakpoint 1, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:27 27 myArray[i]=i; (gdb) print i $3 = 2
The command next executes the program line by line similar to step. However, when a subroutine is called, the next command treats the whole subroutine as one instruction (this is in contrast to the step command, which would pass through the subroutine also in line by line manner).
Conditional breakpoints
You can also provide a condition which needs to be met in order for the program to stop at a specified breakpoint. Say, e.g., you might want to put under scrutiny how the array entries change within the for loop (lines 26-28) when i==5. To achieve this, you can set a breakpoint at line 27 and execute the program flow until i==5 is met:
(gdb) break 27
Breakpoint 1 at 0x4006f6: file myExample.c, line 27.
(gdb) condition 1 i==5
(gdb) run
Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/gdb/c/myExample
Breakpoint 1, main (argc=1, argv=0x7fffffffe5a8) at myExample.c:27
27 myArray[i]=i;
(gdb) print i
$1 = 5
(gdb) print myArray[0]@N
$2 = {0, 1, 2, 3, 4, 0, 0, 0, 0, 0}
(gdb) step
26 for(i=0; i<N; i++){
(gdb) print myArray[0]@N
$3 = {0, 1, 2, 3, 4, 5, 0, 0, 0, 0}
Using DDD for debugging
Note that the HPC system also supports users that prefer the "Data Display Debugger" (DDD) over GDB, since it features an intuitive graphical user interface. Consequently you need to login to your HPC account using the -X option similar to
ssh -X abcd1234@hero.hpc.uni-oldenburg.de
to forward the X windows connection through the ssh connection. DDD is not available from scratch, you need to load the respective module first. This is done via
module load ddd
After this you can, e.g., run DDD on the example discussed above. For this you might simply naviagte to the respective working directory and type (remember to compile the program for elaborate debugging information)
ddd myExample
A snapshot from an actual debugging session (for the above example; breakpoint at line 23, program flow currently at line 27) using DDD is shown below.
Debugging using the Intel debugger
The local HPC system also features several Intel Cluster Studio (ics) addons, see here. Among those you can find the Intel debugger idb along with its command line variant idbc. So as to be able to use them you need to include the intel ics module at first:
module load intel/ics/2013.5.192/64
Then, you can compile the source code for debugging:
icc -o myExample myExample.c -g
(here, the program myExample.c is the same example program as the one used in the preceeding sections).
To invoke the Intel debugger for the resulting application myExample in command line mode you simply need to type
idbc myExample
similar to the proceeding in case of the GNU debugger. If you are familiar with the GNU debugger, you might use the Intel debugger in a straight forward manner since most of the keywords are the same. E.g. to list 10 surrounding lines for line number 8 you might just type
(idb) list 25
20 int main(int argc, char *argv[]) {
21 int *myArray,N,i;
22
23 N=10;
24 myArray = (int *) malloc(-N*sizeof(int));
25
26 for(i=0; i<N; i++){
27 myArray[i]=i;
28 }
29
Then, to set a breakpoint at line 24 and to run the code until that breakpoint is hit you might type
(idb) break 24 Breakpoint 1 at 0x400622: file /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/idb/c/myExample.c, line 24. (idb) run Starting program: /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/idb/c/myExample [New Thread 8593 (LWP 8593)] Breakpoint 1, main (argc=1, argv=0x7fffffffd8f8) at /user/fk5/ifp/agcompphys/alxo9476/wmwr/debug/idb/c/myExample.c:24 24 myArray = (int *) malloc(-N*sizeof(int));
Then, printing the address of the array myArray, stepping forward until the next line occurs and again printing the address of the array (which effectively locates the error in the sourcecode already) is done via
(idb) print myArray
$1 = (int *) 0x7fffffffd8f0
(idb) step
26 for(i=0; i<N; i++){
(idb) print myArray
$2 = (int *) 0x0
So, not only the commands are similar, also the response of the Intel debugger is similar to that of the GNU debugger, discussed above in more detail.
Albeit it is possible to also use the Intel debugger on programs compiled via a GNU compiler (this works by specifying an additonal command line option in the form idbc -gdb myExample), it is recommended to use debuggers along with their corresponting compilers (e.g., compiler icc with debugger icdbc and compiler gcc with debugger gdb).
More information on the Intel debugger can e.g. be found in the Intel Debugger Command Reference.
