linux/Documentation/infiniband/user_mad.txt
<<
>>
Prefs
   1USERSPACE MAD ACCESS
   2
   3Device files
   4
   5  Each port of each InfiniBand device has a "umad" device and an
   6  "issm" device attached.  For example, a two-port HCA will have two
   7  umad devices and two issm devices, while a switch will have one
   8  device of each type (for switch port 0).
   9
  10Creating MAD agents
  11
  12  A MAD agent can be created by filling in a struct ib_user_mad_reg_req
  13  and then calling the IB_USER_MAD_REGISTER_AGENT ioctl on a file
  14  descriptor for the appropriate device file.  If the registration
  15  request succeeds, a 32-bit id will be returned in the structure.
  16  For example:
  17
  18        struct ib_user_mad_reg_req req = { /* ... */ };
  19        ret = ioctl(fd, IB_USER_MAD_REGISTER_AGENT, (char *) &req);
  20        if (!ret)
  21                my_agent = req.id;
  22        else
  23                perror("agent register");
  24
  25  Agents can be unregistered with the IB_USER_MAD_UNREGISTER_AGENT
  26  ioctl.  Also, all agents registered through a file descriptor will
  27  be unregistered when the descriptor is closed.
  28
  29Receiving MADs
  30
  31  MADs are received using read().  The receive side now supports
  32  RMPP. The buffer passed to read() must be at least one
  33  struct ib_user_mad + 256 bytes. For example:
  34
  35  If the buffer passed is not large enough to hold the received
  36  MAD (RMPP), the errno is set to ENOSPC and the length of the
  37  buffer needed is set in mad.length.
  38
  39  Example for normal MAD (non RMPP) reads:
  40        struct ib_user_mad *mad;
  41        mad = malloc(sizeof *mad + 256);
  42        ret = read(fd, mad, sizeof *mad + 256);
  43        if (ret != sizeof mad + 256) {
  44                perror("read");
  45                free(mad);
  46        }
  47
  48  Example for RMPP reads:
  49        struct ib_user_mad *mad;
  50        mad = malloc(sizeof *mad + 256);
  51        ret = read(fd, mad, sizeof *mad + 256);
  52        if (ret == -ENOSPC)) {
  53                length = mad.length;
  54                free(mad);
  55                mad = malloc(sizeof *mad + length);
  56                ret = read(fd, mad, sizeof *mad + length);
  57        }
  58        if (ret < 0) {
  59                perror("read");
  60                free(mad);
  61        }
  62
  63  In addition to the actual MAD contents, the other struct ib_user_mad
  64  fields will be filled in with information on the received MAD.  For
  65  example, the remote LID will be in mad.lid.
  66
  67  If a send times out, a receive will be generated with mad.status set
  68  to ETIMEDOUT.  Otherwise when a MAD has been successfully received,
  69  mad.status will be 0.
  70
  71  poll()/select() may be used to wait until a MAD can be read.
  72
  73Sending MADs
  74
  75  MADs are sent using write().  The agent ID for sending should be
  76  filled into the id field of the MAD, the destination LID should be
  77  filled into the lid field, and so on.  The send side does support
  78  RMPP so arbitrary length MAD can be sent. For example:
  79
  80        struct ib_user_mad *mad;
  81
  82        mad = malloc(sizeof *mad + mad_length);
  83
  84        /* fill in mad->data */
  85
  86        mad->hdr.id  = my_agent;        /* req.id from agent registration */
  87        mad->hdr.lid = my_dest;         /* in network byte order... */
  88        /* etc. */
  89
  90        ret = write(fd, &mad, sizeof *mad + mad_length);
  91        if (ret != sizeof *mad + mad_length)
  92                perror("write");
  93
  94Transaction IDs
  95
  96  Users of the umad devices can use the lower 32 bits of the
  97  transaction ID field (that is, the least significant half of the
  98  field in network byte order) in MADs being sent to match
  99  request/response pairs.  The upper 32 bits are reserved for use by
 100  the kernel and will be overwritten before a MAD is sent.
 101
 102P_Key Index Handling
 103
 104  The old ib_umad interface did not allow setting the P_Key index for
 105  MADs that are sent and did not provide a way for obtaining the P_Key
 106  index of received MADs.  A new layout for struct ib_user_mad_hdr
 107  with a pkey_index member has been defined; however, to preserve
 108  binary compatibility with older applications, this new layout will
 109  not be used unless the IB_USER_MAD_ENABLE_PKEY ioctl is called
 110  before a file descriptor is used for anything else.
 111
 112  In September 2008, the IB_USER_MAD_ABI_VERSION will be incremented
 113  to 6, the new layout of struct ib_user_mad_hdr will be used by
 114  default, and the IB_USER_MAD_ENABLE_PKEY ioctl will be removed.
 115
 116Setting IsSM Capability Bit
 117
 118  To set the IsSM capability bit for a port, simply open the
 119  corresponding issm device file.  If the IsSM bit is already set,
 120  then the open call will block until the bit is cleared (or return
 121  immediately with errno set to EAGAIN if the O_NONBLOCK flag is
 122  passed to open()).  The IsSM bit will be cleared when the issm file
 123  is closed.  No read, write or other operations can be performed on
 124  the issm file.
 125
 126/dev files
 127
 128  To create the appropriate character device files automatically with
 129  udev, a rule like
 130
 131    KERNEL=="umad*", NAME="infiniband/%k"
 132    KERNEL=="issm*", NAME="infiniband/%k"
 133
 134  can be used.  This will create device nodes named
 135
 136    /dev/infiniband/umad0
 137    /dev/infiniband/issm0
 138
 139  for the first port, and so on.  The InfiniBand device and port
 140  associated with these devices can be determined from the files
 141
 142    /sys/class/infiniband_mad/umad0/ibdev
 143    /sys/class/infiniband_mad/umad0/port
 144
 145  and
 146
 147    /sys/class/infiniband_mad/issm0/ibdev
 148    /sys/class/infiniband_mad/issm0/port
 149
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.