x3850M2 PCI error 1801

Added by Neal K. Todd, last edited by Neal K. Todd on Nov 10, 2008
Configuration details for the new backup server

0.  IBM System x3850 M2.
    2 1440W Power Supplies.
    4 dual core Xeon processors (2.4GHz)
    16GB DDR2 SDRAM.
    4 146GB 10K Hot Swap SAS Hard Drives.
    2 Emulex 4GB FC Dual-Port PCIe Controllers.
    3 IBM Ultra 320 SCSI PCIe Controllers.

1.  PCI ROM errors on boot

On original bootup there were PCI ROM POST errors:  POST 1801 error: A PCI adapter has requested memory resources that are not available.  The system   ROM space is limited by PC architecture to 128 KB; any requests that will push the system beyond this limit cannot be serviced, and will generate this error message.  Referencing the document "IBM Support Information - Solving 1801    Errors", the simplest solution to this problem is to reduce the base system ROM requirements to the minimum necessary.  Typically, this can be achieved by disabling the PXE (Network boot) ROM capability of the onboard Ethernet.  This does not disable the device in the operating system,it only disables its capability to perform a network boot.

Configuration sequence for disabling PXE:
    Note: Assumes default settings are loaded.
    1.  While booting the system, hit the F1 key at the IBM splash screen.
    2.  Select 'Start Options' and press 'Enter'.
    3.  Select 'Planar Ethernet PXE/DHCP'.
    4.  Use right arrow to set 'Disable' (this will only disable PXE booting for this device).
    5.  Press Esc key to return to the main setup screen.
    6.  Select 'Save Settings' and press 'Enter'.  Press 'Enter' again to confirm.
    7.  Select 'Exit Setup' and press 'Enter'.  Select 'Yes' to confirm.

Once this action was taken, the system booted without the PCI ROM error notice.

    Additional note:  disabling the PXE boot option on the onboard Ethernet may not clear off enough PCI ROM to clear the POST error.  From the BIOS screen, select 'Advanced Options', then change the      PCI device configuration settings to disable the ROM on each PCI device.  This will free up enough PCI ROM to ensure the system boots without the POST error.

2.  Raid 6 configuration on disk array

In the bootup cycle hit cntl-H to bring up a web-bios interface to set up the ServeRAID-MR10k SAS/SATA raid controller.  The choices were only Raid 0, Raid 5
and Raid 6 for this controller.  We chose to setup Raid 6, which gave us 278472 MB space.

3.  Apply fix to mkinitrd

In order to guarantee that scsi drivers were preloaded so that the LTO tapes would come up consistently on system boot:  mkinitrd -f --preload ioatdma --preload megaraid_sas --preload ata_piix --preload aic79xx --preload lpfc /boot/initrd-2.6.18-53.1.14.el5.img 2.6.18-53.1.14.el5
    
In order to guarantee that this was not necessary to redo every time a new kernel was loaded on the system, added exclude=kernel* to /etc/yum.conf so that new kernels were not installed on a yum update.

4.  Version of Redhat -- 5.3 server

5.  File system setup:

120M /boot  -- /dev/sda1.  Boot partition must be on non-LVM device.
16G  /root  -- /dev/mapper/VolGroup00-LogVol00.  Operating system.
16G  swap   -- /dev/mapper/VolGroup00-LogVol01
16G  /spare -- /dev/mapper/VolGroup00-LogVol02.  Location of daily Arkeia config backup.
197G /opt   -- /dev/mapper/VolGroup00-LogVol03.  Location of Arkeia backup software installation and database.  This filesystem was built using the command:
'mkfs.ext3 -s 1 -m 0 -b 1024 -i 1024 /dev/mapper/VolGroup00-LogVol03' in order to maximize the number of inodes for Arkeia use.  Experience has shown that this parameter must be maximized in order to back up certain file systems without running out of inodes.  The file system created has 235864064 inodes.

Upgrades to the Arkeia backup software led to a major problem with backups:  the daily incremental backups began to take longer than a 24-hour period to complete.  Arkeia uses a file indexing scheme to keep track of their backups, and the algorithm used for daily incremental backups was slowing disk access to a crawl.  We purchased a Fusion-IO PCXe solid-state drive card, which we used to replace the disk /opt file system.     Fusion-io ioDrive 160GB Solid State Storage Device [PCIe x4, Low Profile].

Entry in /etc/fstab:        /dev/fioa1              /opt                    ext3    defaults,noauto         0 0

Df -h:  /dev/fioa1          132G   31G  101G  24% /opt

Df -i:  /dev/fioa1           157140984 20133790 137007194   13% /opt

6.  Version of Arkeia
        Arkeia Network Backup Version 8.0.2