linux/Documentation/trace/mmiotrace.txt
<<
>>
Prefs
   1                In-kernel memory-mapped I/O tracing
   2
   3
   4Home page and links to optional user space tools:
   5
   6        http://nouveau.freedesktop.org/wiki/MmioTrace
   7
   8MMIO tracing was originally developed by Intel around 2003 for their Fault
   9Injection Test Harness. In Dec 2006 - Jan 2007, using the code from Intel,
  10Jeff Muizelaar created a tool for tracing MMIO accesses with the Nouveau
  11project in mind. Since then many people have contributed.
  12
  13Mmiotrace was built for reverse engineering any memory-mapped IO device with
  14the Nouveau project as the first real user. Only x86 and x86_64 architectures
  15are supported.
  16
  17Out-of-tree mmiotrace was originally modified for mainline inclusion and
  18ftrace framework by Pekka Paalanen <pq@iki.fi>.
  19
  20
  21Preparation
  22-----------
  23
  24Mmiotrace feature is compiled in by the CONFIG_MMIOTRACE option. Tracing is
  25disabled by default, so it is safe to have this set to yes. SMP systems are
  26supported, but tracing is unreliable and may miss events if more than one CPU
  27is on-line, therefore mmiotrace takes all but one CPU off-line during run-time
  28activation. You can re-enable CPUs by hand, but you have been warned, there
  29is no way to automatically detect if you are losing events due to CPUs racing.
  30
  31
  32Usage Quick Reference
  33---------------------
  34
  35$ mount -t debugfs debugfs /sys/kernel/debug
  36$ echo mmiotrace > /sys/kernel/debug/tracing/current_tracer
  37$ cat /sys/kernel/debug/tracing/trace_pipe > mydump.txt &
  38Start X or whatever.
  39$ echo "X is up" > /sys/kernel/debug/tracing/trace_marker
  40$ echo nop > /sys/kernel/debug/tracing/current_tracer
  41Check for lost events.
  42
  43
  44Usage
  45-----
  46
  47Make sure debugfs is mounted to /sys/kernel/debug.
  48If not (requires root privileges):
  49$ mount -t debugfs debugfs /sys/kernel/debug
  50
  51Check that the driver you are about to trace is not loaded.
  52
  53Activate mmiotrace (requires root privileges):
  54$ echo mmiotrace > /sys/kernel/debug/tracing/current_tracer
  55
  56Start storing the trace:
  57$ cat /sys/kernel/debug/tracing/trace_pipe > mydump.txt &
  58The 'cat' process should stay running (sleeping) in the background.
  59
  60Load the driver you want to trace and use it. Mmiotrace will only catch MMIO
  61accesses to areas that are ioremapped while mmiotrace is active.
  62
  63During tracing you can place comments (markers) into the trace by
  64$ echo "X is up" > /sys/kernel/debug/tracing/trace_marker
  65This makes it easier to see which part of the (huge) trace corresponds to
  66which action. It is recommended to place descriptive markers about what you
  67do.
  68
  69Shut down mmiotrace (requires root privileges):
  70$ echo nop > /sys/kernel/debug/tracing/current_tracer
  71The 'cat' process exits. If it does not, kill it by issuing 'fg' command and
  72pressing ctrl+c.
  73
  74Check that mmiotrace did not lose events due to a buffer filling up. Either
  75$ grep -i lost mydump.txt
  76which tells you exactly how many events were lost, or use
  77$ dmesg
  78to view your kernel log and look for "mmiotrace has lost events" warning. If
  79events were lost, the trace is incomplete. You should enlarge the buffers and
  80try again. Buffers are enlarged by first seeing how large the current buffers
  81are:
  82$ cat /sys/kernel/debug/tracing/buffer_size_kb
  83gives you a number. Approximately double this number and write it back, for
  84instance:
  85$ echo 128000 > /sys/kernel/debug/tracing/buffer_size_kb
  86Then start again from the top.
  87
  88If you are doing a trace for a driver project, e.g. Nouveau, you should also
  89do the following before sending your results:
  90$ lspci -vvv > lspci.txt
  91$ dmesg > dmesg.txt
  92$ tar zcf pciid-nick-mmiotrace.tar.gz mydump.txt lspci.txt dmesg.txt
  93and then send the .tar.gz file. The trace compresses considerably. Replace
  94"pciid" and "nick" with the PCI ID or model name of your piece of hardware
  95under investigation and your nickname.
  96
  97
  98How Mmiotrace Works
  99-------------------
 100
 101Access to hardware IO-memory is gained by mapping addresses from PCI bus by
 102calling one of the ioremap_*() functions. Mmiotrace is hooked into the
 103__ioremap() function and gets called whenever a mapping is created. Mapping is
 104an event that is recorded into the trace log. Note that ISA range mappings
 105are not caught, since the mapping always exists and is returned directly.
 106
 107MMIO accesses are recorded via page faults. Just before __ioremap() returns,
 108the mapped pages are marked as not present. Any access to the pages causes a
 109fault. The page fault handler calls mmiotrace to handle the fault. Mmiotrace
 110marks the page present, sets TF flag to achieve single stepping and exits the
 111fault handler. The instruction that faulted is executed and debug trap is
 112entered. Here mmiotrace again marks the page as not present. The instruction
 113is decoded to get the type of operation (read/write), data width and the value
 114read or written. These are stored to the trace log.
 115
 116Setting the page present in the page fault handler has a race condition on SMP
 117machines. During the single stepping other CPUs may run freely on that page
 118and events can be missed without a notice. Re-enabling other CPUs during
 119tracing is discouraged.
 120
 121
 122Trace Log Format
 123----------------
 124
 125The raw log is text and easily filtered with e.g. grep and awk. One record is
 126one line in the log. A record starts with a keyword, followed by keyword-
 127dependent arguments. Arguments are separated by a space, or continue until the
 128end of line. The format for version 20070824 is as follows:
 129
 130Explanation     Keyword Space-separated arguments
 131---------------------------------------------------------------------------
 132
 133read event      R       width, timestamp, map id, physical, value, PC, PID
 134write event     W       width, timestamp, map id, physical, value, PC, PID
 135ioremap event   MAP     timestamp, map id, physical, virtual, length, PC, PID
 136iounmap event   UNMAP   timestamp, map id, PC, PID
 137marker          MARK    timestamp, text
 138version         VERSION the string "20070824"
 139info for reader LSPCI   one line from lspci -v
 140PCI address map PCIDEV  space-separated /proc/bus/pci/devices data
 141unk. opcode     UNKNOWN timestamp, map id, physical, data, PC, PID
 142
 143Timestamp is in seconds with decimals. Physical is a PCI bus address, virtual
 144is a kernel virtual address. Width is the data width in bytes and value is the
 145data value. Map id is an arbitrary id number identifying the mapping that was
 146used in an operation. PC is the program counter and PID is process id. PC is
 147zero if it is not recorded. PID is always zero as tracing MMIO accesses
 148originating in user space memory is not yet supported.
 149
 150For instance, the following awk filter will pass all 32-bit writes that target
 151physical addresses in the range [0xfb73ce40, 0xfb800000[
 152
 153$ awk '/W 4 / { adr=strtonum($5); if (adr >= 0xfb73ce40 &&
 154adr < 0xfb800000) print; }'
 155
 156
 157Tools for Developers
 158--------------------
 159
 160The user space tools include utilities for:
 161- replacing numeric addresses and values with hardware register names
 162- replaying MMIO logs, i.e., re-executing the recorded writes
 163
 164
 165
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.