1 2 3EDAC - Error Detection And Correction 4 5Written by Doug Thompson <dougthompson@xmission.com> 67 Dec 2005 717 Jul 2007 Updated 8 9(c) Mauro Carvalho Chehab <mchehab@redhat.com> 1005 Aug 2009 Nehalem interface 11 12EDAC is maintained and written by: 13 14 Doug Thompson, Dave Jiang, Dave Peterson et al, 15 original author: Thayne Harbaugh, 16 17Contact: 18 website: bluesmoke.sourceforge.net 19 mailing list: bluesmoke-devel@lists.sourceforge.net 20 21"bluesmoke" was the name for this device driver when it was "out-of-tree" 22and maintained at sourceforge.net. When it was pushed into 2.6.16 for the 23first time, it was renamed to 'EDAC'. 24 25The bluesmoke project at sourceforge.net is now utilized as a 'staging area' 26for EDAC development, before it is sent upstream to kernel.org 27 28At the bluesmoke/EDAC project site is a series of quilt patches against 29recent kernels, stored in a SVN repository. For easier downloading, there 30is also a tarball snapshot available. 31 32============================================================================ 33EDAC PURPOSE 34 35The 'edac' kernel module goal is to detect and report errors that occur 36within the computer system running under linux. 37 38MEMORY 39 40In the initial release, memory Correctable Errors (CE) and Uncorrectable 41Errors (UE) are the primary errors being harvested. These types of errors 42are harvested by the 'edac_mc' class of device. 43 44Detecting CE events, then harvesting those events and reporting them, 45CAN be a predictor of future UE events. With CE events, the system can 46continue to operate, but with less safety. Preventive maintenance and 47proactive part replacement of memory DIMMs exhibiting CEs can reduce 48the likelihood of the dreaded UE events and system 'panics'. 49 50NON-MEMORY 51 52A new feature for EDAC, the edac_device class of device, was added in 53the 2.6.23 version of the kernel. 54 55This new device type allows for non-memory type of ECC hardware detectors 56to have their states harvested and presented to userspace via the sysfs 57interface. 58 59Some architectures have ECC detectors for L1, L2 and L3 caches, along with DMA 60engines, fabric switches, main data path switches, interconnections, 61and various other hardware data paths. If the hardware reports it, then 62a edac_device device probably can be constructed to harvest and present 63that to userspace. 64 65 66PCI BUS SCANNING 67 68In addition, PCI Bus Parity and SERR Errors are scanned for on PCI devices 69in order to determine if errors are occurring on data transfers. 70 71The presence of PCI Parity errors must be examined with a grain of salt. 72There are several add-in adapters that do NOT follow the PCI specification 73with regards to Parity generation and reporting. The specification says 74the vendor should tie the parity status bits to 0 if they do not intend 75to generate parity. Some vendors do not do this, and thus the parity bit 76can "float" giving false positives. 77 78In the kernel there is a PCI device attribute located in sysfs that is 79checked by the EDAC PCI scanning code. If that attribute is set, 80PCI parity/error scanning is skipped for that device. The attribute 81is: 82 83 broken_parity_status 84 85as is located in /sys/devices/pci<XXX>/0000:XX:YY.Z directories for 86PCI devices. 87 88FUTURE HARDWARE SCANNING 89 90EDAC will have future error detectors that will be integrated with 91EDAC or added to it, in the following list: 92 93 MCE Machine Check Exception 94 MCA Machine Check Architecture 95 NMI NMI notification of ECC errors 96 MSRs Machine Specific Register error cases 97 and other mechanisms. 98 99These errors are usually bus errors, ECC errors, thermal throttling 100and the like. 101 102 103============================================================================ 104EDAC VERSIONING 105 106EDAC is composed of a "core" module (edac_core.ko) and several Memory 107Controller (MC) driver modules. On a given system, the CORE 108is loaded and one MC driver will be loaded. Both the CORE and 109the MC driver (or edac_device driver) have individual versions that reflect 110current release level of their respective modules. 111 112Thus, to "report" on what version a system is running, one must report both 113the CORE's and the MC driver's versions. 114 115 116LOADING 117 118If 'edac' was statically linked with the kernel then no loading is 119necessary. If 'edac' was built as modules then simply modprobe the 120'edac' pieces that you need. You should be able to modprobe 121hardware-specific modules and have the dependencies load the necessary core 122modules. 123 124Example: 125 126$> modprobe amd76x_edac 127 128loads both the amd76x_edac.ko memory controller module and the edac_mc.ko 129core module. 130 131 132============================================================================ 133EDAC sysfs INTERFACE 134 135EDAC presents a 'sysfs' interface for control, reporting and attribute 136reporting purposes. 137 138EDAC lives in the /sys/devices/system/edac directory. 139 140Within this directory there currently reside 2 'edac' components: 141 142 mc memory controller(s) system 143 pci PCI control and status system 144 145 146============================================================================ 147Memory Controller (mc) Model 148 149First a background on the memory controller's model abstracted in EDAC. 150Each 'mc' device controls a set of DIMM memory modules. These modules are 151laid out in a Chip-Select Row (csrowX) and Channel table (chX). There can 152be multiple csrows and multiple channels. 153 154Memory controllers allow for several csrows, with 8 csrows being a typical value. 155Yet, the actual number of csrows depends on the electrical "loading" 156of a given motherboard, memory controller and DIMM characteristics. 157 158Dual channels allows for 128 bit data transfers to the CPU from memory. 159Some newer chipsets allow for more than 2 channels, like Fully Buffered DIMMs 160(FB-DIMMs). The following example will assume 2 channels: 161 162 163 Channel 0 Channel 1 164 =================================== 165 csrow0 | DIMM_A0 | DIMM_B0 | 166 csrow1 | DIMM_A0 | DIMM_B0 | 167 =================================== 168 169 =================================== 170 csrow2 | DIMM_A1 | DIMM_B1 | 171 csrow3 | DIMM_A1 | DIMM_B1 | 172 =================================== 173 174In the above example table there are 4 physical slots on the motherboard 175for memory DIMMs: 176 177 DIMM_A0 178 DIMM_B0 179 DIMM_A1 180 DIMM_B1 181 182Labels for these slots are usually silk screened on the motherboard. Slots 183labeled 'A' are channel 0 in this example. Slots labeled 'B' 184are channel 1. Notice that there are two csrows possible on a 185physical DIMM. These csrows are allocated their csrow assignment 186based on the slot into which the memory DIMM is placed. Thus, when 1 DIMM 187is placed in each Channel, the csrows cross both DIMMs. 188 189Memory DIMMs come single or dual "ranked". A rank is a populated csrow. 190Thus, 2 single ranked DIMMs, placed in slots DIMM_A0 and DIMM_B0 above 191will have 1 csrow, csrow0. csrow1 will be empty. On the other hand, 192when 2 dual ranked DIMMs are similarly placed, then both csrow0 and 193csrow1 will be populated. The pattern repeats itself for csrow2 and 194csrow3. 195 196The representation of the above is reflected in the directory tree 197in EDAC's sysfs interface. Starting in directory 198/sys/devices/system/edac/mc each memory controller will be represented 199by its own 'mcX' directory, where 'X' is the index of the MC. 200 201 202 ..../edac/mc/ 203 | 204 |->mc0 205 |->mc1 206 |->mc2 207 .... 208 209Under each 'mcX' directory each 'csrowX' is again represented by a 210'csrowX', where 'X' is the csrow index: 211 212 213 .../mc/mc0/ 214 | 215 |->csrow0 216 |->csrow2 217 |->csrow3 218 .... 219 220Notice that there is no csrow1, which indicates that csrow0 is 221composed of a single ranked DIMMs. This should also apply in both 222Channels, in order to have dual-channel mode be operational. Since 223both csrow2 and csrow3 are populated, this indicates a dual ranked 224set of DIMMs for channels 0 and 1. 225 226 227Within each of the 'mcX' and 'csrowX' directories are several 228EDAC control and attribute files. 229 230============================================================================ 231'mcX' DIRECTORIES 232 233 234In 'mcX' directories are EDAC control and attribute files for 235this 'X' instance of the memory controllers. 236 237For a description of the sysfs API, please see: 238 Documentation/ABI/testing/sysfs/devices-edac 239 240 241============================================================================ 242'csrowX' DIRECTORIES 243 244When CONFIG_EDAC_LEGACY_SYSFS is enabled, the sysfs will contain the 245csrowX directories. As this API doesn't work properly for Rambus, FB-DIMMs 246and modern Intel Memory Controllers, this is being deprecated in favor 247of dimmX directories. 248 249In the 'csrowX' directories are EDAC control and attribute files for 250this 'X' instance of csrow: 251 252 253Total Uncorrectable Errors count attribute file: 254 255 'ue_count' 256 257 This attribute file displays the total count of uncorrectable 258 errors that have occurred on this csrow. If panic_on_ue is set 259 this counter will not have a chance to increment, since EDAC 260 will panic the system. 261 262 263Total Correctable Errors count attribute file: 264 265 'ce_count' 266 267 This attribute file displays the total count of correctable 268 errors that have occurred on this csrow. This 269 count is very important to examine. CEs provide early 270 indications that a DIMM is beginning to fail. This count 271 field should be monitored for non-zero values and report 272 such information to the system administrator. 273 274 275Total memory managed by this csrow attribute file: 276 277 'size_mb' 278 279 This attribute file displays, in count of megabytes, of memory 280 that this csrow contains. 281 282 283Memory Type attribute file: 284 285 'mem_type' 286 287 This attribute file will display what type of memory is currently 288 on this csrow. Normally, either buffered or unbuffered memory. 289 Examples: 290 Registered-DDR 291 Unbuffered-DDR 292 293 294EDAC Mode of operation attribute file: 295 296 'edac_mode' 297 298 This attribute file will display what type of Error detection 299 and correction is being utilized. 300 301 302Device type attribute file: 303 304 'dev_type' 305 306 This attribute file will display what type of DRAM device is 307 being utilized on this DIMM. 308 Examples: 309 x1 310 x2 311 x4 312 x8 313 314 315Channel 0 CE Count attribute file: 316 317 'ch0_ce_count' 318 319 This attribute file will display the count of CEs on this 320 DIMM located in channel 0. 321 322 323Channel 0 UE Count attribute file: 324 325 'ch0_ue_count' 326 327 This attribute file will display the count of UEs on this 328 DIMM located in channel 0. 329 330 331Channel 0 DIMM Label control file: 332 333 'ch0_dimm_label' 334 335 This control file allows this DIMM to have a label assigned 336 to it. With this label in the module, when errors occur 337 the output can provide the DIMM label in the system log. 338 This becomes vital for panic events to isolate the 339 cause of the UE event. 340 341 DIMM Labels must be assigned after booting, with information 342 that correctly identifies the physical slot with its 343 silk screen label. This information is currently very 344 motherboard specific and determination of this information 345 must occur in userland at this time. 346 347 348Channel 1 CE Count attribute file: 349 350 'ch1_ce_count' 351 352 This attribute file will display the count of CEs on this 353 DIMM located in channel 1. 354 355 356Channel 1 UE Count attribute file: 357 358 'ch1_ue_count' 359 360 This attribute file will display the count of UEs on this 361 DIMM located in channel 0. 362 363 364Channel 1 DIMM Label control file: 365 366 'ch1_dimm_label' 367 368 This control file allows this DIMM to have a label assigned 369 to it. With this label in the module, when errors occur 370 the output can provide the DIMM label in the system log. 371 This becomes vital for panic events to isolate the 372 cause of the UE event. 373 374 DIMM Labels must be assigned after booting, with information 375 that correctly identifies the physical slot with its 376 silk screen label. This information is currently very 377 motherboard specific and determination of this information 378 must occur in userland at this time. 379 380============================================================================ 381SYSTEM LOGGING 382 383If logging for UEs and CEs are enabled then system logs will have 384error notices indicating errors that have been detected: 385 386EDAC MC0: CE page 0x283, offset 0xce0, grain 8, syndrome 0x6ec3, row 0, 387channel 1 "DIMM_B1": amd76x_edac 388 389EDAC MC0: CE page 0x1e5, offset 0xfb0, grain 8, syndrome 0xb741, row 0, 390channel 1 "DIMM_B1": amd76x_edac 391 392 393The structure of the message is: 394 the memory controller (MC0) 395 Error type (CE) 396 memory page (0x283) 397 offset in the page (0xce0) 398 the byte granularity (grain 8) 399 or resolution of the error 400 the error syndrome (0xb741) 401 memory row (row 0) 402 memory channel (channel 1) 403 DIMM label, if set prior (DIMM B1 404 and then an optional, driver-specific message that may 405 have additional information. 406 407Both UEs and CEs with no info will lack all but memory controller, 408error type, a notice of "no info" and then an optional, 409driver-specific error message. 410 411 412============================================================================ 413PCI Bus Parity Detection 414 415 416On Header Type 00 devices the primary status is looked at 417for any parity error regardless of whether Parity is enabled on the 418device. (The spec indicates parity is generated in some cases). 419On Header Type 01 bridges, the secondary status register is also 420looked at to see if parity occurred on the bus on the other side of 421the bridge. 422 423 424SYSFS CONFIGURATION 425 426Under /sys/devices/system/edac/pci are control and attribute files as follows: 427 428 429Enable/Disable PCI Parity checking control file: 430 431 'check_pci_parity' 432 433 434 This control file enables or disables the PCI Bus Parity scanning 435 operation. Writing a 1 to this file enables the scanning. Writing 436 a 0 to this file disables the scanning. 437 438 Enable: 439 echo "1" >/sys/devices/system/edac/pci/check_pci_parity 440 441 Disable: 442 echo "0" >/sys/devices/system/edac/pci/check_pci_parity 443 444 445Parity Count: 446 447 'pci_parity_count' 448 449 This attribute file will display the number of parity errors that 450 have been detected. 451 452 453============================================================================ 454MODULE PARAMETERS 455 456Panic on UE control file: 457 458 'edac_mc_panic_on_ue' 459 460 An uncorrectable error will cause a machine panic. This is usually 461 desirable. It is a bad idea to continue when an uncorrectable error 462 occurs - it is indeterminate what was uncorrected and the operating 463 system context might be so mangled that continuing will lead to further 464 corruption. If the kernel has MCE configured, then EDAC will never 465 notice the UE. 466 467 LOAD TIME: module/kernel parameter: edac_mc_panic_on_ue=[0|1] 468 469 RUN TIME: echo "1" > /sys/module/edac_core/parameters/edac_mc_panic_on_ue 470 471 472Log UE control file: 473 474 'edac_mc_log_ue' 475 476 Generate kernel messages describing uncorrectable errors. These errors 477 are reported through the system message log system. UE statistics 478 will be accumulated even when UE logging is disabled. 479 480 LOAD TIME: module/kernel parameter: edac_mc_log_ue=[0|1] 481 482 RUN TIME: echo "1" > /sys/module/edac_core/parameters/edac_mc_log_ue 483 484 485Log CE control file: 486 487 'edac_mc_log_ce' 488 489 Generate kernel messages describing correctable errors. These 490 errors are reported through the system message log system. 491 CE statistics will be accumulated even when CE logging is disabled. 492 493 LOAD TIME: module/kernel parameter: edac_mc_log_ce=[0|1] 494 495 RUN TIME: echo "1" > /sys/module/edac_core/parameters/edac_mc_log_ce 496 497 498Polling period control file: 499 500 'edac_mc_poll_msec' 501 502 The time period, in milliseconds, for polling for error information. 503 Too small a value wastes resources. Too large a value might delay 504 necessary handling of errors and might loose valuable information for 505 locating the error. 1000 milliseconds (once each second) is the current 506 default. Systems which require all the bandwidth they can get, may 507 increase this. 508 509 LOAD TIME: module/kernel parameter: edac_mc_poll_msec=[0|1] 510 511 RUN TIME: echo "1000" > /sys/module/edac_core/parameters/edac_mc_poll_msec 512 513 514Panic on PCI PARITY Error: 515 516 'panic_on_pci_parity' 517 518 519 This control files enables or disables panicking when a parity 520 error has been detected. 521 522 523 module/kernel parameter: edac_panic_on_pci_pe=[0|1] 524 525 Enable: 526 echo "1" > /sys/module/edac_core/parameters/edac_panic_on_pci_pe 527 528 Disable: 529 echo "0" > /sys/module/edac_core/parameters/edac_panic_on_pci_pe 530 531 532 533======================================================================= 534 535 536EDAC_DEVICE type of device 537 538In the header file, edac_core.h, there is a series of edac_device structures 539and APIs for the EDAC_DEVICE. 540 541User space access to an edac_device is through the sysfs interface. 542 543At the location /sys/devices/system/edac (sysfs) new edac_device devices will 544appear. 545 546There is a three level tree beneath the above 'edac' directory. For example, 547the 'test_device_edac' device (found at the bluesmoke.sourceforget.net website) 548installs itself as: 549 550 /sys/devices/systm/edac/test-instance 551 552in this directory are various controls, a symlink and one or more 'instance' 553directorys. 554 555The standard default controls are: 556 557 log_ce boolean to log CE events 558 log_ue boolean to log UE events 559 panic_on_ue boolean to 'panic' the system if an UE is encountered 560 (default off, can be set true via startup script) 561 poll_msec time period between POLL cycles for events 562 563The test_device_edac device adds at least one of its own custom control: 564 565 test_bits which in the current test driver does nothing but 566 show how it is installed. A ported driver can 567 add one or more such controls and/or attributes 568 for specific uses. 569 One out-of-tree driver uses controls here to allow 570 for ERROR INJECTION operations to hardware 571 injection registers 572 573The symlink points to the 'struct dev' that is registered for this edac_device. 574 575INSTANCES 576 577One or more instance directories are present. For the 'test_device_edac' case: 578 579 test-instance0 580 581 582In this directory there are two default counter attributes, which are totals of 583counter in deeper subdirectories. 584 585 ce_count total of CE events of subdirectories 586 ue_count total of UE events of subdirectories 587 588BLOCKS 589 590At the lowest directory level is the 'block' directory. There can be 0, 1 591or more blocks specified in each instance. 592 593 test-block0 594 595 596In this directory the default attributes are: 597 598 ce_count which is counter of CE events for this 'block' 599 of hardware being monitored 600 ue_count which is counter of UE events for this 'block' 601 of hardware being monitored 602 603 604The 'test_device_edac' device adds 4 attributes and 1 control: 605 606 test-block-bits-0 for every POLL cycle this counter 607 is incremented 608 test-block-bits-1 every 10 cycles, this counter is bumped once, 609 and test-block-bits-0 is set to 0 610 test-block-bits-2 every 100 cycles, this counter is bumped once, 611 and test-block-bits-1 is set to 0 612 test-block-bits-3 every 1000 cycles, this counter is bumped once, 613 and test-block-bits-2 is set to 0 614 615 616 reset-counters writing ANY thing to this control will 617 reset all the above counters. 618 619 620Use of the 'test_device_edac' driver should any others to create their own 621unique drivers for their hardware systems. 622 623The 'test_device_edac' sample driver is located at the 624bluesmoke.sourceforge.net project site for EDAC. 625 626======================================================================= 627NEHALEM USAGE OF EDAC APIs 628 629This chapter documents some EXPERIMENTAL mappings for EDAC API to handle 630Nehalem EDAC driver. They will likely be changed on future versions 631of the driver. 632 633Due to the way Nehalem exports Memory Controller data, some adjustments 634were done at i7core_edac driver. This chapter will cover those differences 635 6361) On Nehalem, there are one Memory Controller per Quick Patch Interconnect 637 (QPI). At the driver, the term "socket" means one QPI. This is 638 associated with a physical CPU socket. 639 640 Each MC have 3 physical read channels, 3 physical write channels and 641 3 logic channels. The driver currently sees it as just 3 channels. 642 Each channel can have up to 3 DIMMs. 643 644 The minimum known unity is DIMMs. There are no information about csrows. 645 As EDAC API maps the minimum unity is csrows, the driver sequencially 646 maps channel/dimm into different csrows. 647 648 For example, supposing the following layout: 649 Ch0 phy rd0, wr0 (0x063f4031): 2 ranks, UDIMMs 650 dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400 651 dimm 1 1024 Mb offset: 4, bank: 8, rank: 1, row: 0x4000, col: 0x400 652 Ch1 phy rd1, wr1 (0x063f4031): 2 ranks, UDIMMs 653 dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400 654 Ch2 phy rd3, wr3 (0x063f4031): 2 ranks, UDIMMs 655 dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400 656 The driver will map it as: 657 csrow0: channel 0, dimm0 658 csrow1: channel 0, dimm1 659 csrow2: channel 1, dimm0 660 csrow3: channel 2, dimm0 661 662exports one 663 DIMM per csrow. 664 665 Each QPI is exported as a different memory controller. 666 6672) Nehalem MC has the hability to generate errors. The driver implements this 668 functionality via some error injection nodes: 669 670 For injecting a memory error, there are some sysfs nodes, under 671 /sys/devices/system/edac/mc/mc?/: 672 673 inject_addrmatch/*: 674 Controls the error injection mask register. It is possible to specify 675 several characteristics of the address to match an error code: 676 dimm = the affected dimm. Numbers are relative to a channel; 677 rank = the memory rank; 678 channel = the channel that will generate an error; 679 bank = the affected bank; 680 page = the page address; 681 column (or col) = the address column. 682 each of the above values can be set to "any" to match any valid value. 683 684 At driver init, all values are set to any. 685 686 For example, to generate an error at rank 1 of dimm 2, for any channel, 687 any bank, any page, any column: 688 echo 2 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/dimm 689 echo 1 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/rank 690 691 To return to the default behaviour of matching any, you can do: 692 echo any >/sys/devices/system/edac/mc/mc0/inject_addrmatch/dimm 693 echo any >/sys/devices/system/edac/mc/mc0/inject_addrmatch/rank 694 695 inject_eccmask: 696 specifies what bits will have troubles, 697 698 inject_section: 699 specifies what ECC cache section will get the error: 700 3 for both 701 2 for the highest 702 1 for the lowest 703 704 inject_type: 705 specifies the type of error, being a combination of the following bits: 706 bit 0 - repeat 707 bit 1 - ecc 708 bit 2 - parity 709 710 inject_enable starts the error generation when something different 711 than 0 is written. 712 713 All inject vars can be read. root permission is needed for write. 714 715 Datasheet states that the error will only be generated after a write on an 716 address that matches inject_addrmatch. It seems, however, that reading will 717 also produce an error. 718 719 For example, the following code will generate an error for any write access 720 at socket 0, on any DIMM/address on channel 2: 721 722 echo 2 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/channel 723 echo 2 >/sys/devices/system/edac/mc/mc0/inject_type 724 echo 64 >/sys/devices/system/edac/mc/mc0/inject_eccmask 725 echo 3 >/sys/devices/system/edac/mc/mc0/inject_section 726 echo 1 >/sys/devices/system/edac/mc/mc0/inject_enable 727 dd if=/dev/mem of=/dev/null seek=16k bs=4k count=1 >& /dev/null 728 729 For socket 1, it is needed to replace "mc0" by "mc1" at the above 730 commands. 731 732 The generated error message will look like: 733 734 EDAC MC0: UE row 0, channel-a= 0 channel-b= 0 labels "-": NON_FATAL (addr = 0x0075b980, socket=0, Dimm=0, Channel=2, syndrome=0x00000040, count=1, Err=8c0000400001009f:4000080482 (read error: read ECC error)) 735 7363) Nehalem specific Corrected Error memory counters 737 738 Nehalem have some registers to count memory errors. The driver uses those 739 registers to report Corrected Errors on devices with Registered Dimms. 740 741 However, those counters don't work with Unregistered Dimms. As the chipset 742 offers some counters that also work with UDIMMS (but with a worse level of 743 granularity than the default ones), the driver exposes those registers for 744 UDIMM memories. 745 746 They can be read by looking at the contents of all_channel_counts/ 747 748 $ for i in /sys/devices/system/edac/mc/mc0/all_channel_counts/*; do echo $i; cat $i; done 749 /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm0 750 0 751 /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm1 752 0 753 /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm2 754 0 755 756 What happens here is that errors on different csrows, but at the same 757 dimm number will increment the same counter. 758 So, in this memory mapping: 759 csrow0: channel 0, dimm0 760 csrow1: channel 0, dimm1 761 csrow2: channel 1, dimm0 762 csrow3: channel 2, dimm0 763 The hardware will increment udimm0 for an error at the first dimm at either 764 csrow0, csrow2 or csrow3; 765 The hardware will increment udimm1 for an error at the second dimm at either 766 csrow0, csrow2 or csrow3; 767 The hardware will increment udimm2 for an error at the third dimm at either 768 csrow0, csrow2 or csrow3; 769 7704) Standard error counters 771 772 The standard error counters are generated when an mcelog error is received 773 by the driver. Since, with udimm, this is counted by software, it is 774 possible that some errors could be lost. With rdimm's, they displays the 775 contents of the registers 776