RAID 5 in Windows on Mac Pro

[pipomolo42]

Hello,

I've posted this to InsanelyMac a few days ago, I'm reposting it here, because It's more about "running things on a mac" than "running OS X on things" ... I hope you don't mind.

I've been looking for a solution to setup a raid system in my mac pro that would work equally in Linux, Windows and maybe OS X.

the ESB2 southbridge supports 3 different modes:
- Legacy IDE (as PCI vendor id 2680)
- AHCI (as PCI vendor id 2681)
- RAID (as PCI vendor id 2682)

It seems that when the system is booted, AHCI mode is used (I checked the status in rEFIt's efi console), and Mac OS X also reports that it's using AHCI mode.

Unfortunately, when booting Linux or Windows through the apple CSM, the ESB2 is switched back to Legacy IDE mode, so you cannot use the RAID features of the board.

While reading through the ESB2 specification (http://www.intel.com/design/chipsets...hts/313082.htm) it seems that AHCI mode supports RAID 0 and 1, and RAID mode supports RAID 0, 1 and 5. but RAID mode also requires an "Option ROM" to be on the mainboard. Specification also states that it is possible to switch the chipset's mode, by just writting to offset 0x90 of the ide controller PCI configuration space (the PCI device at address 00:00:1f.2).

I confirmed this mode switching under Linux, using the setpci tool to write to the config space and to check the vendor id afterwards.

Eventually, the goal would be to be able to set up a RAID 5 array that would be accessible from both Windows and Linux (don't know if OS X would support it, as it is not provided as a standard feature, but I only use OS X for firmware updates anyways)

The question I still have is : what about the "Option ROM" ? is it only used for array configuration ? In which case Linux and Windows can achieve the same, or is it really necessary for the array's "sanity" ?

Then, my guess would be that Apple's CSM is responsible for setting this controller mode (They probably decided to use the legacy IDE mode as AHCI and RAID mode require a specific driver to be loaded from a floppy at windows installation, and the Mac Pro does not have a floppy drive).

Actually the driver could also be slipstreamed into the installation CD. And Vista supports "F6 loading" from a USB key or CD as well as floppy. And the driver is included in the linux kenel for some versions now. So, it wouldn't have been that difficult to offer "bootcampers" a better option than legacy IDE.

The options I see to restore the full fearured behaviour would be:

a) patch a bootloader (like grub or lilo) to write to the PCI config space, and use this bootloader to boot all operating systems that require BIOS emulation. The problem is that these bootloader generally use the BIOS to access the hard drives, so I don't know how safe it is to switch the controller mode outside of the bios.

b) patch the apple CSM so that it does not set the controller to Legacy IDE mode. I first thought this would require reading/writing to the firmware directly, but actually, I think it could be achieved more easily from rEFIt, just writting to RAM between the LoadImage and StartImage calls.

Anyone has ideas or hints about this ?

[pipomolo42]

Hello,

For your information, I successfully managed to install and boot Linux and WinXP with the ESB2 in RAID mode. I patched grub's stage1 so that it writes to the correct pci configuration space address. patch and resulting stage1 are here: http://boeglin.org/static/macpro/ .

For Linux to get it to work, it's as simple as:
- make sure your kernel has the ahci module
- reboot

For windows it's like:
- add an entry to grub to chainload to your windows partition
- add an entry to grub following this trick: http://www.lrz-muenchen.de/~bernhard/grub-chain-cd.html

I then simply used this last entry to chainload to the windows installation CD (in which I previously slipstreamed the chipset drivers (infinst) and AHCI drivers available here: http://downloadcenter.intel.com/Deta...stems&lang=eng , using nlite)


After next reboot, I simply used "refit -> grub -> windows entry" instead "refit ->windows"
to get to a winXP capable of RAID.

Unfortunately, it doesn't completely work: the Intel RAID configuration utility appears to be confused as no Option ROM was loaded by Apple's CSM, and it only offers RAID 0 (maybe as a "default" first entry, as it could not get the capabilities from the Option ROM INT13h functions) ...

From there, there are several possible options:

- simply write 0x40 instead of 0x80 to the PCI config space to benefit at least from AHCI instead of RAID. (it can simply be achieved by replacing the 0x80 by a 0x40 at offset 0x5d of the stage1 image I provided, or directly in the first disk sector in which you installed grub). According to Intel's spec, AHCI mode would allow at least the use of RAID 0 and 1 in Windows (I didn't have time to test yet).

- "if" the Option ROM is present in Apple's CMS and "if" the CSM respects the BIOS specs and "if" I manage to set the ESB2 in RAID mode before the CSM tries to load the Optopn ROM, it should be executed, and RAID should then be working 100% (maybe writing a EFI driver that will patch the CSM in memory before it is loaded).

- other possible way (?): the EFI API has a method named LoadOptionRom, I haven't checked yet what it does exactly.

- yet other possible way (?): there are many mainboards or systems by TYAN, Intel, HP, Dell, IBM using the same ESB2 southbridge and with downloadable bios file. We could just extract the Option ROM from one of these, and see how we can load it to memory.

- create an online petition and whine at apple so that they publish and document a new firmware for the mac pro including the Option ROM and the ability to select in which mode we want the ESB2 to boot (using a NVRAM variable, or known memory address).

Edit: 0x40 has to be written at offset 0x5d, not 0x5b as I previously wrote, sorry for the mistake (but anyway, there is no 0x80 at 0x5b in the file, so that was easy to guess, I hope ;) )

[pipomolo42]

(Could one of the admins or moderators change the topic to something like "RAID 5 in Windows on Mac Pro", maybe it will attract more people with knowledge or ideas about this. Thanks!)

[zarmanto]

No problem; 'tis edited as requested.

[pipomolo42]

Ok, so I finally tried AHCI mode: it does not provide any option for handling RAID arrays, so it seems that hooking an Option ROM to int13h is the only way to go for this ...

On the other hand, AHCI should still be a much better option than the legacy IDE selected by Apple: it supports NCQ, and it also allows DMA transfers (whereas Legacy IDE mode uses only PIO, which consumes CPU), and the Intel spec also claims that AHCI offers a better power managment ...

Now I need to think a bit more about the possibilities to get an option rom and have it "executed" at bootup.

( and thanks for the edit )

[pipomolo42]

Hi,

I managed to get the option rom from a bios image of a motherboard using the samle chipset.

I also started to look more closely at the Mac Pro EFI firmware:

The compatibility BIOS is included in it, as GUID 1547B4F3-3E8A-4FEF-81C8-328ED647AB1A. It starts with the strings "Copyright (C) 1998-2004 Insyde Software Corp.", "SYS CORE", "5.31.DT", "07/29/05".

This GUID is referenced in the DXE driver BC6D08DC-865D-4FFE-8B7A-FB5FB04F12F1, which is in turn referenced in the EFI application 2B0585EB-D8B8-49A9-8B8C-E21B01AEF2B7, which is the legacy OS loader

The next step will be to find where in these files is the controller put in legacy IDE mode, and put it in RAID mode instead, and then load the option rom in the option rom space (0xc0000-0xf0000).

[Beliyaal]

This is great news. I was struggling with this when I first installed my Mac Pro and gave up. I only need AHCI as I use Windows software raid 0 only.

Hopefully it's possible to change the driver without reinstalling Windows.

When I did my testing I noticed that the throughput was shared by two SATA ports. This meant if I put two drives in the same raid on port 0 and 1, or port 2 and 3 it wouldn't perform as well. Because AHCI wasn't supported (port 4 and 5 isn't recognized by windows) I ended up buying an PCIE sata card so I could get 3 channels.

Would this throughput limit be fixed by putting the controller in AHCI mode or is it a hardware limitation?

[pipomolo42]

Hi,

As far as I know, it's not possible to switch from IDE to AHCI or RAID mode without reinstalling, as winXP seems to have issues finding the disk to boot on ...

Edit: actually, it *could* work if you install the AHCI driver in IDE mode and then reboot to AHCI mode, but I'll have to try first to confirm

Edit2: I just tested, and it works:
- update your IDE controller (the one whose device ID is 2680) with the AHCI driver in http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&Inst=Yes&ProductID=2308&Dwn ldID=11207&strOSs=All&OSFullName=All%20Operating%2 0Systems&lang=eng
- reboot windows, using the patched grub as a bootstrap, that will set the controller to AHCI mode

But I think it'll not work for RAID, as winXP registers it as a different controller class (SCSI/RAID)

If you have good disks benchmarks to point me to, I could run them in all modes, and if they show a big difference in performances, maybe try to use them as an argument to ask Apple to provide a firmware that fully supports its hardware capabilities ... Would be easier than me trying to fix it on my own

[pipomolo42]

Here is also one link you'ld want to read before switching to AHCI (although I didn't need to apply it, forcing the AHCI driver installation was enough):
http://support.microsoft.com/kb/922976

Edit: Ah, sorry, this one's only for Vista ... Well, Wikipedia also contains good advices:
http://en.wikipedia.org/wiki/AHCI

[pipomolo42]

So, I finally made some benchmarks, using Intel Iometer (http://www.iometer.org/).

I plugged in two hard disks, in the first two Sata slots, and setup 4 workers on each disk, and tested using 512B, 2kB and 64kB blocks.

As you can see in the results I posted on http://www.flickr.com/photos/1069507...7601084232467/, the AHCI mode completely beats the legacy IDE mode.


with 512B blocks: Total I/O per second: +24%

with 2kB blocks: Total I/O per second: +22%

with 64kB blocks: Total I/O per second: +19%

So, the more the controller is stressed, the better it performs in AHCI ...

Also, with 64kB blocks, it's interresting to note that the 19% increase also means that the Total MB per second goes from 17.09 to 20.32!