linux/Documentation/dynamic-debug-howto.txt
<<
>>
Prefs
   1
   2Introduction
   3============
   4
   5This document describes how to use the dynamic debug (ddebug) feature.
   6
   7Dynamic debug is designed to allow you to dynamically enable/disable kernel
   8code to obtain additional kernel information. Currently, if
   9CONFIG_DYNAMIC_DEBUG is set, then all pr_debug()/dev_dbg() calls can be
  10dynamically enabled per-callsite.
  11
  12Dynamic debug has even more useful features:
  13
  14 * Simple query language allows turning on and off debugging statements by
  15   matching any combination of:
  16
  17   - source filename
  18   - function name
  19   - line number (including ranges of line numbers)
  20   - module name
  21   - format string
  22
  23 * Provides a debugfs control file: <debugfs>/dynamic_debug/control which can be
  24   read to display the complete list of known debug statements, to help guide you
  25
  26Controlling dynamic debug Behaviour
  27===================================
  28
  29The behaviour of pr_debug()/dev_dbg()s are controlled via writing to a
  30control file in the 'debugfs' filesystem. Thus, you must first mount the debugfs
  31filesystem, in order to make use of this feature. Subsequently, we refer to the
  32control file as: <debugfs>/dynamic_debug/control. For example, if you want to
  33enable printing from source file 'svcsock.c', line 1603 you simply do:
  34
  35nullarbor:~ # echo 'file svcsock.c line 1603 +p' >
  36                                <debugfs>/dynamic_debug/control
  37
  38If you make a mistake with the syntax, the write will fail thus:
  39
  40nullarbor:~ # echo 'file svcsock.c wtf 1 +p' >
  41                                <debugfs>/dynamic_debug/control
  42-bash: echo: write error: Invalid argument
  43
  44Viewing Dynamic Debug Behaviour
  45===========================
  46
  47You can view the currently configured behaviour of all the debug statements
  48via:
  49
  50nullarbor:~ # cat <debugfs>/dynamic_debug/control
  51# filename:lineno [module]function flags format
  52/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup - "SVCRDMA Module Removed, deregister RPC RDMA transport\012"
  53/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init - "\011max_inline       : %d\012"
  54/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init - "\011sq_depth         : %d\012"
  55/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init - "\011max_requests     : %d\012"
  56...
  57
  58
  59You can also apply standard Unix text manipulation filters to this
  60data, e.g.
  61
  62nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control  | wc -l
  6362
  64
  65nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l
  6642
  67
  68Note in particular that the third column shows the enabled behaviour
  69flags for each debug statement callsite (see below for definitions of the
  70flags).  The default value, no extra behaviour enabled, is "-".  So
  71you can view all the debug statement callsites with any non-default flags:
  72
  73nullarbor:~ # awk '$3 != "-"' <debugfs>/dynamic_debug/control
  74# filename:lineno [module]function flags format
  75/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012"
  76
  77
  78Command Language Reference
  79==========================
  80
  81At the lexical level, a command comprises a sequence of words separated
  82by whitespace characters.  Note that newlines are treated as word
  83separators and do *not* end a command or allow multiple commands to
  84be done together.  So these are all equivalent:
  85
  86nullarbor:~ # echo -c 'file svcsock.c line 1603 +p' >
  87                                <debugfs>/dynamic_debug/control
  88nullarbor:~ # echo -c '  file   svcsock.c     line  1603 +p  ' >
  89                                <debugfs>/dynamic_debug/control
  90nullarbor:~ # echo -c 'file svcsock.c\nline 1603 +p' >
  91                                <debugfs>/dynamic_debug/control
  92nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
  93                                <debugfs>/dynamic_debug/control
  94
  95Commands are bounded by a write() system call.  If you want to do
  96multiple commands you need to do a separate "echo" for each, like:
  97
  98nullarbor:~ # echo 'file svcsock.c line 1603 +p' > /proc/dprintk ;\
  99> echo 'file svcsock.c line 1563 +p' > /proc/dprintk
 100
 101or even like:
 102
 103nullarbor:~ # (
 104> echo 'file svcsock.c line 1603 +p' ;\
 105> echo 'file svcsock.c line 1563 +p' ;\
 106> ) > /proc/dprintk
 107
 108At the syntactical level, a command comprises a sequence of match
 109specifications, followed by a flags change specification.
 110
 111command ::= match-spec* flags-spec
 112
 113The match-spec's are used to choose a subset of the known dprintk()
 114callsites to which to apply the flags-spec.  Think of them as a query
 115with implicit ANDs between each pair.  Note that an empty list of
 116match-specs is possible, but is not very useful because it will not
 117match any debug statement callsites.
 118
 119A match specification comprises a keyword, which controls the attribute
 120of the callsite to be compared, and a value to compare against.  Possible
 121keywords are:
 122
 123match-spec ::= 'func' string |
 124               'file' string |
 125               'module' string |
 126               'format' string |
 127               'line' line-range
 128
 129line-range ::= lineno |
 130               '-'lineno |
 131               lineno'-' |
 132               lineno'-'lineno
 133// Note: line-range cannot contain space, e.g.
 134// "1-30" is valid range but "1 - 30" is not.
 135
 136lineno ::= unsigned-int
 137
 138The meanings of each keyword are:
 139
 140func
 141    The given string is compared against the function name
 142    of each callsite.  Example:
 143
 144    func svc_tcp_accept
 145
 146file
 147    The given string is compared against either the full
 148    pathname or the basename of the source file of each
 149    callsite.  Examples:
 150
 151    file svcsock.c
 152    file /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c
 153
 154module
 155    The given string is compared against the module name
 156    of each callsite.  The module name is the string as
 157    seen in "lsmod", i.e. without the directory or the .ko
 158    suffix and with '-' changed to '_'.  Examples:
 159
 160    module sunrpc
 161    module nfsd
 162
 163format
 164    The given string is searched for in the dynamic debug format
 165    string.  Note that the string does not need to match the
 166    entire format, only some part.  Whitespace and other
 167    special characters can be escaped using C octal character
 168    escape \ooo notation, e.g. the space character is \040.
 169    Alternatively, the string can be enclosed in double quote
 170    characters (") or single quote characters (').
 171    Examples:
 172
 173    format svcrdma:         // many of the NFS/RDMA server dprintks
 174    format readahead        // some dprintks in the readahead cache
 175    format nfsd:\040SETATTR // one way to match a format with whitespace
 176    format "nfsd: SETATTR"  // a neater way to match a format with whitespace
 177    format 'nfsd: SETATTR'  // yet another way to match a format with whitespace
 178
 179line
 180    The given line number or range of line numbers is compared
 181    against the line number of each dprintk() callsite.  A single
 182    line number matches the callsite line number exactly.  A
 183    range of line numbers matches any callsite between the first
 184    and last line number inclusive.  An empty first number means
 185    the first line in the file, an empty line number means the
 186    last number in the file.  Examples:
 187
 188    line 1603       // exactly line 1603
 189    line 1600-1605  // the six lines from line 1600 to line 1605
 190    line -1605      // the 1605 lines from line 1 to line 1605
 191    line 1600-      // all lines from line 1600 to the end of the file
 192
 193The flags specification comprises a change operation followed
 194by one or more flag characters.  The change operation is one
 195of the characters:
 196
 197-
 198    remove the given flags
 199
 200+
 201    add the given flags
 202
 203=
 204    set the flags to the given flags
 205
 206The flags are:
 207
 208f
 209    Include the function name in the printed message
 210l
 211    Include line number in the printed message
 212m
 213    Include module name in the printed message
 214p
 215    Causes a printk() message to be emitted to dmesg
 216t
 217    Include thread ID in messages not generated from interrupt context
 218
 219Note the regexp ^[-+=][flmpt]+$ matches a flags specification.
 220Note also that there is no convenient syntax to remove all
 221the flags at once, you need to use "-flmpt".
 222
 223
 224Debug messages during boot process
 225==================================
 226
 227To be able to activate debug messages during the boot process,
 228even before userspace and debugfs exists, use the boot parameter:
 229ddebug_query="QUERY"
 230
 231QUERY follows the syntax described above, but must not exceed 1023
 232characters. The enablement of debug messages is done as an arch_initcall.
 233Thus you can enable debug messages in all code processed after this
 234arch_initcall via this boot parameter.
 235On an x86 system for example ACPI enablement is a subsys_initcall and
 236ddebug_query="file ec.c +p"
 237will show early Embedded Controller transactions during ACPI setup if
 238your machine (typically a laptop) has an Embedded Controller.
 239PCI (or other devices) initialization also is a hot candidate for using
 240this boot parameter for debugging purposes.
 241
 242
 243Examples
 244========
 245
 246// enable the message at line 1603 of file svcsock.c
 247nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
 248                                <debugfs>/dynamic_debug/control
 249
 250// enable all the messages in file svcsock.c
 251nullarbor:~ # echo -n 'file svcsock.c +p' >
 252                                <debugfs>/dynamic_debug/control
 253
 254// enable all the messages in the NFS server module
 255nullarbor:~ # echo -n 'module nfsd +p' >
 256                                <debugfs>/dynamic_debug/control
 257
 258// enable all 12 messages in the function svc_process()
 259nullarbor:~ # echo -n 'func svc_process +p' >
 260                                <debugfs>/dynamic_debug/control
 261
 262// disable all 12 messages in the function svc_process()
 263nullarbor:~ # echo -n 'func svc_process -p' >
 264                                <debugfs>/dynamic_debug/control
 265
 266// enable messages for NFS calls READ, READLINK, READDIR and READDIR+.
 267nullarbor:~ # echo -n 'format "nfsd: READ" +p' >
 268                                <debugfs>/dynamic_debug/control
 269
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.