linux/Documentation/trace/events.txt
<<
>>
Prefs
   1                             Event Tracing
   2
   3                Documentation written by Theodore Ts'o
   4                Updated by Li Zefan and Tom Zanussi
   5
   61. Introduction
   7===============
   8
   9Tracepoints (see Documentation/trace/tracepoints.txt) can be used
  10without creating custom kernel modules to register probe functions
  11using the event tracing infrastructure.
  12
  13Not all tracepoints can be traced using the event tracing system;
  14the kernel developer must provide code snippets which define how the
  15tracing information is saved into the tracing buffer, and how the
  16tracing information should be printed.
  17
  182. Using Event Tracing
  19======================
  20
  212.1 Via the 'set_event' interface
  22---------------------------------
  23
  24The events which are available for tracing can be found in the file
  25/sys/kernel/debug/tracing/available_events.
  26
  27To enable a particular event, such as 'sched_wakeup', simply echo it
  28to /sys/kernel/debug/tracing/set_event. For example:
  29
  30        # echo sched_wakeup >> /sys/kernel/debug/tracing/set_event
  31
  32[ Note: '>>' is necessary, otherwise it will firstly disable
  33  all the events. ]
  34
  35To disable an event, echo the event name to the set_event file prefixed
  36with an exclamation point:
  37
  38        # echo '!sched_wakeup' >> /sys/kernel/debug/tracing/set_event
  39
  40To disable all events, echo an empty line to the set_event file:
  41
  42        # echo > /sys/kernel/debug/tracing/set_event
  43
  44To enable all events, echo '*:*' or '*:' to the set_event file:
  45
  46        # echo *:* > /sys/kernel/debug/tracing/set_event
  47
  48The events are organized into subsystems, such as ext4, irq, sched,
  49etc., and a full event name looks like this: <subsystem>:<event>.  The
  50subsystem name is optional, but it is displayed in the available_events
  51file.  All of the events in a subsystem can be specified via the syntax
  52"<subsystem>:*"; for example, to enable all irq events, you can use the
  53command:
  54
  55        # echo 'irq:*' > /sys/kernel/debug/tracing/set_event
  56
  572.2 Via the 'enable' toggle
  58---------------------------
  59
  60The events available are also listed in /sys/kernel/debug/tracing/events/ hierarchy
  61of directories.
  62
  63To enable event 'sched_wakeup':
  64
  65        # echo 1 > /sys/kernel/debug/tracing/events/sched/sched_wakeup/enable
  66
  67To disable it:
  68
  69        # echo 0 > /sys/kernel/debug/tracing/events/sched/sched_wakeup/enable
  70
  71To enable all events in sched subsystem:
  72
  73        # echo 1 > /sys/kernel/debug/tracing/events/sched/enable
  74
  75To enable all events:
  76
  77        # echo 1 > /sys/kernel/debug/tracing/events/enable
  78
  79When reading one of these enable files, there are four results:
  80
  81 0 - all events this file affects are disabled
  82 1 - all events this file affects are enabled
  83 X - there is a mixture of events enabled and disabled
  84 ? - this file does not affect any event
  85
  862.3 Boot option
  87---------------
  88
  89In order to facilitate early boot debugging, use boot option:
  90
  91        trace_event=[event-list]
  92
  93event-list is a comma separated list of events. See section 2.1 for event
  94format.
  95
  963. Defining an event-enabled tracepoint
  97=======================================
  98
  99See The example provided in samples/trace_events
 100
 1014. Event formats
 102================
 103
 104Each trace event has a 'format' file associated with it that contains
 105a description of each field in a logged event.  This information can
 106be used to parse the binary trace stream, and is also the place to
 107find the field names that can be used in event filters (see section 5).
 108
 109It also displays the format string that will be used to print the
 110event in text mode, along with the event name and ID used for
 111profiling.
 112
 113Every event has a set of 'common' fields associated with it; these are
 114the fields prefixed with 'common_'.  The other fields vary between
 115events and correspond to the fields defined in the TRACE_EVENT
 116definition for that event.
 117
 118Each field in the format has the form:
 119
 120     field:field-type field-name; offset:N; size:N;
 121
 122where offset is the offset of the field in the trace record and size
 123is the size of the data item, in bytes.
 124
 125For example, here's the information displayed for the 'sched_wakeup'
 126event:
 127
 128# cat /sys/kernel/debug/tracing/events/sched/sched_wakeup/format
 129
 130name: sched_wakeup
 131ID: 60
 132format:
 133        field:unsigned short common_type;       offset:0;       size:2;
 134        field:unsigned char common_flags;       offset:2;       size:1;
 135        field:unsigned char common_preempt_count;       offset:3;       size:1;
 136        field:int common_pid;   offset:4;       size:4;
 137        field:int common_tgid;  offset:8;       size:4;
 138
 139        field:char comm[TASK_COMM_LEN]; offset:12;      size:16;
 140        field:pid_t pid;        offset:28;      size:4;
 141        field:int prio; offset:32;      size:4;
 142        field:int success;      offset:36;      size:4;
 143        field:int cpu;  offset:40;      size:4;
 144
 145print fmt: "task %s:%d [%d] success=%d [%03d]", REC->comm, REC->pid,
 146           REC->prio, REC->success, REC->cpu
 147
 148This event contains 10 fields, the first 5 common and the remaining 5
 149event-specific.  All the fields for this event are numeric, except for
 150'comm' which is a string, a distinction important for event filtering.
 151
 1525. Event filtering
 153==================
 154
 155Trace events can be filtered in the kernel by associating boolean
 156'filter expressions' with them.  As soon as an event is logged into
 157the trace buffer, its fields are checked against the filter expression
 158associated with that event type.  An event with field values that
 159'match' the filter will appear in the trace output, and an event whose
 160values don't match will be discarded.  An event with no filter
 161associated with it matches everything, and is the default when no
 162filter has been set for an event.
 163
 1645.1 Expression syntax
 165---------------------
 166
 167A filter expression consists of one or more 'predicates' that can be
 168combined using the logical operators '&&' and '||'.  A predicate is
 169simply a clause that compares the value of a field contained within a
 170logged event with a constant value and returns either 0 or 1 depending
 171on whether the field value matched (1) or didn't match (0):
 172
 173          field-name relational-operator value
 174
 175Parentheses can be used to provide arbitrary logical groupings and
 176double-quotes can be used to prevent the shell from interpreting
 177operators as shell metacharacters.
 178
 179The field-names available for use in filters can be found in the
 180'format' files for trace events (see section 4).
 181
 182The relational-operators depend on the type of the field being tested:
 183
 184The operators available for numeric fields are:
 185
 186==, !=, <, <=, >, >=
 187
 188And for string fields they are:
 189
 190==, !=
 191
 192Currently, only exact string matches are supported.
 193
 1945.2 Setting filters
 195-------------------
 196
 197A filter for an individual event is set by writing a filter expression
 198to the 'filter' file for the given event.
 199
 200For example:
 201
 202# cd /sys/kernel/debug/tracing/events/sched/sched_wakeup
 203# echo "common_preempt_count > 4" > filter
 204
 205A slightly more involved example:
 206
 207# cd /sys/kernel/debug/tracing/events/signal/signal_generate
 208# echo "((sig >= 10 && sig < 15) || sig == 17) && comm != bash" > filter
 209
 210If there is an error in the expression, you'll get an 'Invalid
 211argument' error when setting it, and the erroneous string along with
 212an error message can be seen by looking at the filter e.g.:
 213
 214# cd /sys/kernel/debug/tracing/events/signal/signal_generate
 215# echo "((sig >= 10 && sig < 15) || dsig == 17) && comm != bash" > filter
 216-bash: echo: write error: Invalid argument
 217# cat filter
 218((sig >= 10 && sig < 15) || dsig == 17) && comm != bash
 219^
 220parse_error: Field not found
 221
 222Currently the caret ('^') for an error always appears at the beginning of
 223the filter string; the error message should still be useful though
 224even without more accurate position info.
 225
 2265.3 Clearing filters
 227--------------------
 228
 229To clear the filter for an event, write a '0' to the event's filter
 230file.
 231
 232To clear the filters for all events in a subsystem, write a '0' to the
 233subsystem's filter file.
 234
 2355.3 Subsystem filters
 236---------------------
 237
 238For convenience, filters for every event in a subsystem can be set or
 239cleared as a group by writing a filter expression into the filter file
 240at the root of the subsystem.  Note however, that if a filter for any
 241event within the subsystem lacks a field specified in the subsystem
 242filter, or if the filter can't be applied for any other reason, the
 243filter for that event will retain its previous setting.  This can
 244result in an unintended mixture of filters which could lead to
 245confusing (to the user who might think different filters are in
 246effect) trace output.  Only filters that reference just the common
 247fields can be guaranteed to propagate successfully to all events.
 248
 249Here are a few subsystem filter examples that also illustrate the
 250above points:
 251
 252Clear the filters on all events in the sched subsystem:
 253
 254# cd /sys/kernel/debug/tracing/events/sched
 255# echo 0 > filter
 256# cat sched_switch/filter
 257none
 258# cat sched_wakeup/filter
 259none
 260
 261Set a filter using only common fields for all events in the sched
 262subsystem (all events end up with the same filter):
 263
 264# cd /sys/kernel/debug/tracing/events/sched
 265# echo common_pid == 0 > filter
 266# cat sched_switch/filter
 267common_pid == 0
 268# cat sched_wakeup/filter
 269common_pid == 0
 270
 271Attempt to set a filter using a non-common field for all events in the
 272sched subsystem (all events but those that have a prev_pid field retain
 273their old filters):
 274
 275# cd /sys/kernel/debug/tracing/events/sched
 276# echo prev_pid == 0 > filter
 277# cat sched_switch/filter
 278prev_pid == 0
 279# cat sched_wakeup/filter
 280common_pid == 0
 281
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.