pantz.org banner
Limits of RAID cards, file systems, and OS's
Posted on 08-02-2005 15:22:00 UTC | Updated on 06-03-2008 17:25:47 UTC
Section: /software/linux/ | Permanent Link

I've been doing research on new RAID solutions that can use over 2TB of disk space. I have come upon some interesting info on the limits of the different parts of the system. It seems that there is a big line you have to follow when you want to address large volumes of disk space. Each part of the chain has a limit. Which means the smallest part of the chain is what you are limited to. The chain goes like this Raid card-> OS Kernel-> File system -> Partitioning program.

First the RAID card has to support disks (arrays) or logical volumes larger than 2 terabytes (TB). This is usually done by having the card support 64 bit Logical Block Addressing (LBA). Most cards and OS's unless they are very new only support 32 bit LBA's. This is what limits them to 2TB's (2^32 * 512 ~ 2TB).

Next, the OS needs to be able to format and address a logical volume or disk greater than 2TB. Right now some OS's that do this are Windows 2003 Server SP1, XP 64bit, and the 2.6.x linux kernel. The logical volume can be split into partitions but my example here is using the logical volume being used as one big partition.

Then, you will need a file system that can support a disk/volume over 2TB. The thing with file systems is that they can theoretically get to a certain size but usually there seems to be a limit with another part of the OS that keeps them from reaching this size. Like a driver (NTFS) or a kernel block size limit (linux). FAT16 partitions are limited to a maximum of 2Gb. FAT32's limit is theoretically 8TB but Windows 2000 and Windows XP cannot FORMAT a volume larger than 32 GB in size using the native FAT32 file system. A quick note on FAT32. The file size limit is 4Gig so copying that full dual layer DVD won't work. So to get around all this you'll want to use NTFS. Using the default cluster size of 4 KB for large volumes, the maximum NTFS volume size windows allows is 256 terabytes. This is limited by the windows driver.

In the 2.6 linux kernel the maximum limit of block devices with a 32-bit CPU and kernel is 16TB with 4 KB block sizes. The linux kernel page cache index limit (32 bits) == filesystem block number. 2^32 * 4 KB = 16 TiB. With a 64-bit CPU and 64-bit 2.6 kernel the max block device size is 8 Exabytes (EB). Some filesystems can go to the Exabyte range others can't. For example the Ext3 filesystem on linux can only support a maximum volume size up to 32TB on 64-bit systems. That means you can have a partition max of 32TB. With the XFS filesystem on linux the (partitions) limit is the kernel's block device limit of 16TB on 32-bit and 8EB on 64-bit. The XFS filesystem itself can handle an maximum file size of 8EB and can handle a maximum volume size of 8EB. See the Wikipedia page on comparison of filesystems for more info on the limits of many filesystems. Just remember that just because the filesystem says it can handle something does not mean the kernel or software of the os your running can handle it.

Lastly, you will need a partitioning program that can address as disk/volume over 2TB. For linux as of this writing fdisk can not do this. You would need to use a linux program called parted. With the windows OS's that support the larger LBA's I assume the disk manager can deal with theses new sizes. Just note that Disk devices with more than 2 TB of disk space must be converted to GPT (GUID Partition Tables) format for all of the disk space to be usable. If the device uses MBR format, the disk space beyond 2 TB will be unusable.

The following is a example of parted commands to make raid volume that is accessed as /dev/sda. It uses XFS a the file system.:

parted /dev/sda

Now you should be at a Parted prompt. Next we make the GPT label.

mklabel gpt

Then we see how big our volume is by printing info about it.

p

Disk geometry for /dev/sda: 0.000-2384185.000 megabytes
Disk label type: gpt
Minor       Start       End       Filesystem Name       Flags

Now we make our partition using the number we got from our print out of the volume.

mkpart
Partition type? [primary]? [press enter]
File system type? [ext2]? xfs
Start? 0
End? 2384185

Then we quit.

q

Next make your file system (Ex. mkfs.xfs -f -b size=4k -l size=64m /dev/sda). Don't forget to update /etc/fstab, if necessary.

Del.icio.us! | Digg Me! | Reddit!

Related stories


RSS Feed RSS feed logo
About


3com
3ware
alsa
alsactl
alsamixer
amd
android
apache
areca
arm
ati
auditd
awk
badblocks
bash
bind
bios
bonnie
cable
carp
cat5
cdrom
cellphone
centos
chart
chrome
cifs
cisco
cloudera
comcast
commands
comodo
compiz-fusion
corsair
cpufreq
cpufrequtils
cpuspeed
cron
crontab
crossover
cu
cups
cvs
database
dbus
dd
dd_rescue
ddclient
debian
decimal
dhclient
dhcp
diagnostic
diskexplorer
disks
dkim
dns
dos
dovecot
drac
dsniff
dvdauthor
e-mail
echo
editor
emerald
ethernet
expect
ext3
ext4
fat32
fedora
fetchmail
fiber
filesystems
firefox
firewall
flac
flexlm
floppy
flowtools
fonts
format
freebsd
ftp
gdm
gmail
gnome
greasemonkey
greylisting
growisofs
grub
hacking
hadoop
harddrive
hba
hex
hfsc
html
html5
http
https
idl
ie
ilo
intel
ios
iperf
ipmi
iptables
ipv6
irix
javascript
kde
kernel
kickstart
kmail
kprinter
krecord
kubuntu
kvm
lame
ldap
linux
logfile
lp
lpq
lpr
maradns
matlab
memory
mencoder
mhdd
mkinitrd
mkisofs
moinmoin
motherboard
mouse
movemail
mplayer
multitail
mutt
myodbc
mysql
mythtv
nagios
nameserver
netflix
netflow
nginx
nic
ntfs
ntp
nvidia
odbc
openbsd
openntpd
openoffice
openssh
openssl
openvpn
opteron
parted
partimage
patch
perl
pf
pfflowd
pfsync
photorec
php
pop3
pop3s
ports
postfix
power
procmail
proftpd
proxy
pulseaudio
putty
pxe
python
qemu
r-studio
raid
recovery
redhat
router
rpc
rsync
ruby
saltstack
samba
schedule
screen
scsi
seagate
seatools
sed
sendmail
sgi
shell
siw
smtp
snort
solaris
soundcard
sox
spam
spamd
spf
sql
sqlite
squid
srs
ssh
ssh.com
ssl
su
subnet
subversion
sudo
sun
supermicro
switches
symbols
syslinux
syslog
systemrescuecd
t1
tcpip
tcpwrappers
telnet
terminal
testdisk
tftp
thttpd
thunderbird
timezone
ting
tls
tools
tr
trac
tuning
tunnel
ubuntu
unbound
vi
vpn
wget
wiki
windows
windowsxp
wireless
wpa_supplicant
x
xauth
xfree86
xfs
xinearama
xmms
youtube
zdump
zeromq
zic
zlib