[LUNI] file system limitations

[d.w. harks]

Since directories are represented in most Unix filesystems as files,
which get an inode just like any other file, to my knowledge most
filesystems don't treat them terribly specially in any way that
would introduce directory-specific hard limits.

What you end up running into as you scale is either file
size limitations or inode allocation limitations -- in a fixed space
for a superblock, you can only store a table of so many inodes
before you run out of space. Whether those inodes are allocated for
files or directories (which are really just files that store a list
of inodes of other files 'contained' within them) wouldn't be
significant. 

However, some of the work going in to optimizing ext2/3 does include
work with sparse files and where to physically store data on disk
with relation to logical filesystem branches -- to reduce the disk
seeks needed to get from one 'related' file to another. So, if I'm
recursing through a subdirectory tree, are the directory inodes
physically close to their related data inodes so that I can get to
the data without sending the read head flying all over the disk?

Some other info on how Journaling Filesystems work is available at: 

http://bulmalug.net/body.phtml?nIdNoticia=1154&nIdPage=3

http://www.linuxgazette.com/issue68/dellomodarme.html

The design of ext2fs:

http://web.mit.edu/tytso/www/linux/ext2intro.html

I do know that XFS is excellent in terms of scalability, and have
seen it handle terabytes of data with 100,000s of directories on SGI
IRIX without breaking a sweat. The Linux implementation of XFS is of
quite high quality, although it's not quite as easy to get going
with as is ext3. XFS on a RAID system with a (fast) external journal
has killer performance.

dave

In our last episode, Erik Lickerman expounded thusly:
> Are there any limits on the number of directories you can have in a linux
> filesystem?  Does performance signifigantly decrease if you have 1000
> directories?  10,000 directories?  100,000 directories?  Does it depend on
> which filesystem you use?  Obviously in asking this question I am assuming
> you have the hardware to support it.
> 
> Erik
> 
> ______________________________________________________________________
> Linux Users Of Northern Illinois - Technical Discussion 
> luni at luni.org
> http://luni.org/mailman/listinfo/luni

-- 
{----------------d.w. harks----------------}
 dave at psys.org	   http://www.psys.org/dw/
{------------------------------------------}
   Blog: http://dwblog.psys.org <- NEW!