linux/Documentation/networking/eql.txt
<<
>>
Prefs
   1  EQL Driver: Serial IP Load Balancing HOWTO
   2  Simon "Guru Aleph-Null" Janes, simon@ncm.com
   3  v1.1, February 27, 1995
   4
   5  This is the manual for the EQL device driver. EQL is a software device
   6  that lets you load-balance IP serial links (SLIP or uncompressed PPP)
   7  to increase your bandwidth. It will not reduce your latency (i.e. ping
   8  times) except in the case where you already have lots of traffic on
   9  your link, in which it will help them out. This driver has been tested
  10  with the 1.1.75 kernel, and is known to have patched cleanly with
  11  1.1.86.  Some testing with 1.1.92 has been done with the v1.1 patch
  12  which was only created to patch cleanly in the very latest kernel
  13  source trees. (Yes, it worked fine.)
  14
  15  1.  Introduction
  16
  17  Which is worse? A huge fee for a 56K leased line or two phone lines?
  18  It's probably the former.  If you find yourself craving more bandwidth,
  19  and have a ISP that is flexible, it is now possible to bind modems
  20  together to work as one point-to-point link to increase your
  21  bandwidth.  All without having to have a special black box on either
  22  side.
  23
  24
  25  The eql driver has only been tested with the Livingston PortMaster-2e
  26  terminal server. I do not know if other terminal servers support load-
  27  balancing, but I do know that the PortMaster does it, and does it
  28  almost as well as the eql driver seems to do it (-- Unfortunately, in
  29  my testing so far, the Livingston PortMaster 2e's load-balancing is a
  30  good 1 to 2 KB/s slower than the test machine working with a 28.8 Kbps
  31  and 14.4 Kbps connection.  However, I am not sure that it really is
  32  the PortMaster, or if it's Linux's TCP drivers. I'm told that Linux's
  33  TCP implementation is pretty fast though.--)
  34
  35
  36  I suggest to ISPs out there that it would probably be fair to charge
  37  a load-balancing client 75% of the cost of the second line and 50% of
  38  the cost of the third line etc...
  39
  40
  41  Hey, we can all dream you know...
  42
  43
  44  2.  Kernel Configuration
  45
  46  Here I describe the general steps of getting a kernel up and working
  47  with the eql driver.  From patching, building, to installing.
  48
  49
  50  2.1.  Patching The Kernel
  51
  52  If you do not have or cannot get a copy of the kernel with the eql
  53  driver folded into it, get your copy of the driver from
  54  ftp://slaughter.ncm.com/pub/Linux/LOAD_BALANCING/eql-1.1.tar.gz.
  55  Unpack this archive someplace obvious like /usr/local/src/.  It will
  56  create the following files:
  57
  58
  59
  60       ______________________________________________________________________
  61       -rw-r--r-- guru/ncm      198 Jan 19 18:53 1995 eql-1.1/NO-WARRANTY
  62       -rw-r--r-- guru/ncm      30620 Feb 27 21:40 1995 eql-1.1/eql-1.1.patch
  63       -rwxr-xr-x guru/ncm      16111 Jan 12 22:29 1995 eql-1.1/eql_enslave
  64       -rw-r--r-- guru/ncm      2195 Jan 10 21:48 1995 eql-1.1/eql_enslave.c
  65       ______________________________________________________________________
  66
  67  Unpack a recent kernel (something after 1.1.92) someplace convenient
  68  like say /usr/src/linux-1.1.92.eql. Use symbolic links to point
  69  /usr/src/linux to this development directory.
  70
  71
  72  Apply the patch by running the commands:
  73
  74
  75       ______________________________________________________________________
  76       cd /usr/src
  77       patch </usr/local/src/eql-1.1/eql-1.1.patch
  78       ______________________________________________________________________
  79
  80
  81
  82
  83
  84  2.2.  Building The Kernel
  85
  86  After patching the kernel, run make config and configure the kernel
  87  for your hardware.
  88
  89
  90  After configuration, make and install according to your habit.
  91
  92
  93  3.  Network Configuration
  94
  95  So far, I have only used the eql device with the DSLIP SLIP connection
  96  manager by Matt Dillon (-- "The man who sold his soul to code so much
  97  so quickly."--) .  How you configure it for other "connection"
  98  managers is up to you.  Most other connection managers that I've seen
  99  don't do a very good job when it comes to handling more than one
 100  connection.
 101
 102
 103  3.1.  /etc/rc.d/rc.inet1
 104
 105  In rc.inet1, ifconfig the eql device to the IP address you usually use
 106  for your machine, and the MTU you prefer for your SLIP lines. One
 107  could argue that MTU should be roughly half the usual size for two
 108  modems, one-third for three, one-fourth for four, etc...  But going
 109  too far below 296 is probably overkill. Here is an example ifconfig
 110  command that sets up the eql device:
 111
 112
 113
 114       ______________________________________________________________________
 115       ifconfig eql 198.67.33.239 mtu 1006
 116       ______________________________________________________________________
 117
 118
 119
 120
 121
 122  Once the eql device is up and running, add a static default route to
 123  it in the routing table using the cool new route syntax that makes
 124  life so much easier:
 125
 126
 127
 128       ______________________________________________________________________
 129       route add default eql
 130       ______________________________________________________________________
 131
 132
 133  3.2.  Enslaving Devices By Hand
 134
 135  Enslaving devices by hand requires two utility programs: eql_enslave
 136  and eql_emancipate (-- eql_emancipate hasn't been written because when
 137  an enslaved device "dies", it is automatically taken out of the queue.
 138  I haven't found a good reason to write it yet... other than for
 139  completeness, but that isn't a good motivator is it?--)
 140
 141
 142  The syntax for enslaving a device is "eql_enslave <master-name>
 143  <slave-name> <estimated-bps>".  Here are some example enslavings:
 144
 145
 146
 147       ______________________________________________________________________
 148       eql_enslave eql sl0 28800
 149       eql_enslave eql ppp0 14400
 150       eql_enslave eql sl1 57600
 151       ______________________________________________________________________
 152
 153
 154
 155
 156
 157  When you want to free a device from its life of slavery, you can
 158  either down the device with ifconfig (eql will automatically bury the
 159  dead slave and remove it from its queue) or use eql_emancipate to free
 160  it. (-- Or just ifconfig it down, and the eql driver will take it out
 161  for you.--)
 162
 163
 164
 165       ______________________________________________________________________
 166       eql_emancipate eql sl0
 167       eql_emancipate eql ppp0
 168       eql_emancipate eql sl1
 169       ______________________________________________________________________
 170
 171
 172
 173
 174
 175  3.3.  DSLIP Configuration for the eql Device
 176
 177  The general idea is to bring up and keep up as many SLIP connections
 178  as you need, automatically.
 179
 180
 181  3.3.1.  /etc/slip/runslip.conf
 182
 183  Here is an example runslip.conf:
 184
 185
 186
 187
 188
 189
 190
 191
 192
 193
 194
 195
 196
 197
 198
 199  ______________________________________________________________________
 200  name          sl-line-1
 201  enabled
 202  baud          38400
 203  mtu           576
 204  ducmd         -e /etc/slip/dialout/cua2-288.xp -t 9
 205  command        eql_enslave eql $interface 28800
 206  address        198.67.33.239
 207  line          /dev/cua2
 208
 209  name          sl-line-2
 210  enabled
 211  baud          38400
 212  mtu           576
 213  ducmd         -e /etc/slip/dialout/cua3-288.xp -t 9
 214  command        eql_enslave eql $interface 28800
 215  address        198.67.33.239
 216  line          /dev/cua3
 217  ______________________________________________________________________
 218
 219
 220
 221
 222
 223  3.4.  Using PPP and the eql Device
 224
 225  I have not yet done any load-balancing testing for PPP devices, mainly
 226  because I don't have a PPP-connection manager like SLIP has with
 227  DSLIP. I did find a good tip from LinuxNET:Billy for PPP performance:
 228  make sure you have asyncmap set to something so that control
 229  characters are not escaped.
 230
 231
 232  I tried to fix up a PPP script/system for redialing lost PPP
 233  connections for use with the eql driver the weekend of Feb 25-26 '95
 234  (Hereafter known as the 8-hour PPP Hate Festival).  Perhaps later this
 235  year.
 236
 237
 238  4.  About the Slave Scheduler Algorithm
 239
 240  The slave scheduler probably could be replaced with a dozen other
 241  things and push traffic much faster.  The formula in the current set
 242  up of the driver was tuned to handle slaves with wildly different
 243  bits-per-second "priorities".
 244
 245
 246  All testing I have done was with two 28.8 V.FC modems, one connecting
 247  at 28800 bps or slower, and the other connecting at 14400 bps all the
 248  time.
 249
 250
 251  One version of the scheduler was able to push 5.3 K/s through the
 252  28800 and 14400 connections, but when the priorities on the links were
 253  very wide apart (57600 vs. 14400) the "faster" modem received all
 254  traffic and the "slower" modem starved.
 255
 256
 257  5.  Testers' Reports
 258
 259  Some people have experimented with the eql device with newer
 260  kernels (than 1.1.75).  I have since updated the driver to patch
 261  cleanly in newer kernels because of the removal of the old "slave-
 262  balancing" driver config option.
 263
 264
 265  o  icee from LinuxNET patched 1.1.86 without any rejects and was able
 266     to boot the kernel and enslave a couple of ISDN PPP links.
 267
 268  5.1.  Randolph Bentson's Test Report
 269
 270
 271
 272
 273
 274
 275
 276
 277
 278
 279
 280
 281
 282
 283
 284
 285
 286
 287
 288
 289
 290
 291
 292
 293
 294
 295
 296
 297
 298
 299
 300
 301
 302
 303
 304
 305
 306
 307
 308
 309
 310
 311
 312
 313
 314
 315
 316
 317
 318
 319
 320
 321
 322
 323
 324
 325
 326
 327
 328
 329
 330
 331  From bentson@grieg.seaslug.org Wed Feb  8 19:08:09 1995
 332  Date: Tue, 7 Feb 95 22:57 PST
 333  From: Randolph Bentson <bentson@grieg.seaslug.org>
 334  To: guru@ncm.com
 335  Subject: EQL driver tests
 336
 337
 338  I have been checking out your eql driver.  (Nice work, that!)
 339  Although you may already done this performance testing, here
 340  are some data I've discovered.
 341
 342  Randolph Bentson
 343  bentson@grieg.seaslug.org
 344
 345  ---------------------------------------------------------
 346
 347
 348  A pseudo-device driver, EQL, written by Simon Janes, can be used
 349  to bundle multiple SLIP connections into what appears to be a
 350  single connection.  This allows one to improve dial-up network
 351  connectivity gradually, without having to buy expensive DSU/CSU
 352  hardware and services.
 353
 354  I have done some testing of this software, with two goals in
 355  mind: first, to ensure it actually works as described and
 356  second, as a method of exercising my device driver.
 357
 358  The following performance measurements were derived from a set
 359  of SLIP connections run between two Linux systems (1.1.84) using
 360  a 486DX2/66 with a Cyclom-8Ys and a 486SLC/40 with a Cyclom-16Y.
 361  (Ports 0,1,2,3 were used.  A later configuration will distribute
 362  port selection across the different Cirrus chips on the boards.)
 363  Once a link was established, I timed a binary ftp transfer of
 364  289284 bytes of data. If there were no overhead (packet headers,
 365  inter-character and inter-packet delays, etc.) the transfers
 366  would take the following times:
 367
 368      bits/sec  seconds
 369      345600    8.3
 370      234600    12.3
 371      172800    16.7
 372      153600    18.8
 373      76800     37.6
 374      57600     50.2
 375      38400     75.3
 376      28800     100.4
 377      19200     150.6
 378      9600      301.3
 379
 380  A single line running at the lower speeds and with large packets
 381  comes to within 2% of this.  Performance is limited for the higher
 382  speeds (as predicted by the Cirrus databook) to an aggregate of
 383  about 160 kbits/sec.  The next round of testing will distribute
 384  the load across two or more Cirrus chips.
 385
 386  The good news is that one gets nearly the full advantage of the
 387  second, third, and fourth line's bandwidth.  (The bad news is
 388  that the connection establishment seemed fragile for the higher
 389  speeds.  Once established, the connection seemed robust enough.)
 390
 391  #lines  speed mtu  seconds    theory  actual  %of
 392         kbit/sec      duration speed   speed   max
 393  3     115200  900     _       345600
 394  3     115200  400     18.1    345600  159825  46
 395  2     115200  900     _       230400
 396  2     115200  600     18.1    230400  159825  69
 397  2     115200  400     19.3    230400  149888  65
 398  4     57600   900     _       234600
 399  4     57600   600     _       234600
 400  4     57600   400     _       234600
 401  3     57600   600     20.9    172800  138413  80
 402  3     57600   900     21.2    172800  136455  78
 403  3     115200  600     21.7    345600  133311  38
 404  3     57600   400     22.5    172800  128571  74
 405  4     38400   900     25.2    153600  114795  74
 406  4     38400   600     26.4    153600  109577  71
 407  4     38400   400     27.3    153600  105965  68
 408  2     57600   900     29.1    115200  99410.3 86
 409  1     115200  900     30.7    115200  94229.3 81
 410  2     57600   600     30.2    115200  95789.4 83
 411  3     38400   900     30.3    115200  95473.3 82
 412  3     38400   600     31.2    115200  92719.2 80
 413  1     115200  600     31.3    115200  92423   80
 414  2     57600   400     32.3    115200  89561.6 77
 415  1     115200  400     32.8    115200  88196.3 76
 416  3     38400   400     33.5    115200  86353.4 74
 417  2     38400   900     43.7    76800   66197.7 86
 418  2     38400   600     44      76800   65746.4 85
 419  2     38400   400     47.2    76800   61289   79
 420  4     19200   900     50.8    76800   56945.7 74
 421  4     19200   400     53.2    76800   54376.7 70
 422  4     19200   600     53.7    76800   53870.4 70
 423  1     57600   900     54.6    57600   52982.4 91
 424  1     57600   600     56.2    57600   51474   89
 425  3     19200   900     60.5    57600   47815.5 83
 426  1     57600   400     60.2    57600   48053.8 83
 427  3     19200   600     62      57600   46658.7 81
 428  3     19200   400     64.7    57600   44711.6 77
 429  1     38400   900     79.4    38400   36433.8 94
 430  1     38400   600     82.4    38400   35107.3 91
 431  2     19200   900     84.4    38400   34275.4 89
 432  1     38400   400     86.8    38400   33327.6 86
 433  2     19200   600     87.6    38400   33023.3 85
 434  2     19200   400     91.2    38400   31719.7 82
 435  4     9600    900     94.7    38400   30547.4 79
 436  4     9600    400     106     38400   27290.9 71
 437  4     9600    600     110     38400   26298.5 68
 438  3     9600    900     118     28800   24515.6 85
 439  3     9600    600     120     28800   24107   83
 440  3     9600    400     131     28800   22082.7 76
 441  1     19200   900     155     19200   18663.5 97
 442  1     19200   600     161     19200   17968   93
 443  1     19200   400     170     19200   17016.7 88
 444  2     9600    600     176     19200   16436.6 85
 445  2     9600    900     180     19200   16071.3 83
 446  2     9600    400     181     19200   15982.5 83
 447  1     9600    900     305     9600    9484.72 98
 448  1     9600    600     314     9600    9212.87 95
 449  1     9600    400     332     9600    8713.37 90
 450
 451
 452
 453
 454
 455  5.2.  Anthony Healy's Report
 456
 457
 458
 459
 460
 461
 462
 463  Date: Mon, 13 Feb 1995 16:17:29 +1100 (EST)
 464  From: Antony Healey <ahealey@st.nepean.uws.edu.au>
 465  To: Simon Janes <guru@ncm.com>
 466  Subject: Re: Load Balancing
 467
 468  Hi Simon,
 469          I've installed your patch and it works great. I have trialed
 470          it over twin SL/IP lines, just over null modems, but I was
 471          able to data at over 48Kb/s [ISDN link -Simon]. I managed a
 472          transfer of up to 7.5 Kbyte/s on one go, but averaged around
 473          6.4 Kbyte/s, which I think is pretty cool.  :)
 474
 475
 476
 477
 478
 479
 480
 481
 482
 483
 484
 485
 486
 487
 488
 489
 490
 491
 492
 493
 494
 495
 496
 497
 498
 499
 500
 501
 502
 503
 504
 505
 506
 507
 508
 509
 510
 511
 512
 513
 514
 515
 516
 517
 518
 519
 520
 521
 522
 523
 524
 525
 526
 527
 528
 529
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.