[LUNI] file system limitations
elickerman at ameritech.net
Sun Nov 17 16:45:01 CST 2002
What is in my mind here? It is a medical application. Because there is
never a need to do a join across different patients, rather than put all
patients in a single relational database, I have divided it up so each
patient's chart is a directory containing a handful of data files. To
access a particular patient, you look him up in an LDAP directory. This
gives you the chart id and the address of the server the chart is on. You
then contact the server (an EJB running on JBoss) and pass it the chart id
which is the name of the directory the patient's chart is in.
The advantage of this setup, versus the one large database approach, is
scalability, both with more powerful hardware and with additonal machines.
It also makes it simple to transfer a chart to another physical location.
Just move the directory, and change the server address in the directory.
If a practice has 20,000 patients, that means 20,000 directories on the
server, but in any given half hour period, probably only fifty directories
would be accessed. Gievn what you and the others have told me, it sounds as
if this is no problem.
From: luni-admin at luni.org [mailto:luni-admin at luni.org]On Behalf Of
Sent: Sunday, November 17, 2002 1:46 PM
To: luni at luni.org
Subject: Re: [LUNI] file system limitations
On Sun, Nov 17, 2002 at 09:52:29AM -0600, Erik Lickerman wrote:
> Are there any limits on the number of directories you can have in a linux
Well, inodes are a finite resources, and every directory consumes one
inode, as does every regular, device, or special file. But if you know
you're going to need unusually many (in proportion to the size of the
filesystem) you can set that when you initialize the filesystem.
> Does performance signifigantly decrease if you have 1000 directories?
> 10,000 directories? 100,000 directories? Does it depend on which
> filesystem you use?
Having many directories per se doesn't matter much. Having a great
many directories (or files) contained in one directory (that is being
accessed) matters to different filesystems to different amounts.
Having a huge number of directories that are being actively used
(searched, say) will place an awful lot of pressure on the in-memory
buffer pool; I would think this would be relatively independent of
filesystem type, but that's a quite wooly guess.
> Obviously in asking this question I am assuming you have the hardware
> to support it.
Let's see. 100K directories of modest size might consume only a single
4K block each plus their inode (smaller than a block; smaller than the
traditional 512 byte sector, I think, but I can't call to mind an
actual number). Surely it's well under 500MB, which doesn't seem
likely to be very significant relative to the disk space needed to hold
their contents. OTOH, that's quite a bit to have to buffer in memory
in order to avoid heavy disk accesses if the entire set is being
accessed - though, again, not in comparison to the bufferring for the
contents of files that would probably also be being used in anything
remotely like normal use.
So what is it that's in your mind here? Sounds interesting!
He that questioneth much shall learn much, and content much; but
especially if he apply his questions to the skill of the persons whom
he asketh; for he shall give them the occasion to please themselves
in speaking, and himself shall continually gather knowledge. But let
his questions not be troublesome, for that is fit for a poser; and
let him be sure to leave other men their turns to speak. - Francis Bacon
Linux Users Of Northern Illinois - Technical Discussion
luni at luni.org
More information about the luni