[d-kernel] [Fwd: Re: [Unionfs] UnionFS as step towards a big/reliable/fast virtual disk?]

[Anton Farygin <rider@altlinux.com>]

[-- Attachment #1: Type: text/plain, Size: 73 bytes --]

FYI.

Интересно для тех, то закладывается на unionfs в своих решениях.



[-- Attachment #2: Re: [Unionfs] UnionFS as step towards a big/reliable/fast virtual	disk? --]
[-- Type: message/rfc822, Size: 5856 bytes --]

From: "Charles P. Wright" <cwright@cs.sunysb.edu>
To: "Gordon Mohr (Internet Archive)" <gojomo@archive.org>
Cc: unionfs@filer.fsl.cs.sunysb.edu
Subject: Re: [Unionfs] UnionFS as step towards a big/reliable/fast virtual disk?
Date: Thu, 17 Mar 2005 19:01:47 -0500
Message-ID: <1111104108.21904.12.camel@localhost.localdomain>

On Thu, 2005-03-17 at 15:27 -0800, Gordon Mohr (Internet Archive) wrote:
> The first application that came to my mind after learning about
> UnionFS was using it to create a big and reliable and/or fast
> virtual disk out of many smaller disks -- a sort of
> filesystem-level RAID (raifs?).
We are developing something similar called RAIF (Redundant Array of
Independent Filesystems), but it is not likely to be released for some
time (if ever).  It is actually based on Unionfs, but has been changed
to the point of being completely different.

In general, Unionfs probably won't fit (at least in its current state)
into what you want it to do.  But, I am going to answer your other
questons on a bit lower-level, to describe the Unionfs properties you
are asking about anyway since others might be curious.

> Has there been any work (or speculation) in this direction for
> UnionFS?
> 
> Some questions that come to mind for such usage:
> 
> - Is there any hard limit to the branch fanout? (10? 100? 1000? more?)
Currently it is related to MAX_FD because we use an fd_set for managing
a branch bitmap under some failure situations.  Performance will be bad
enough well before then that this limit won't matter.

> - How would performance be expected to degrade with high fanout?
It depends on the operation.  The real big ones are lookup and readdir.
The rub with lookup is that most other operations are preceded by a
lookup.  Lookup requires O(n) for directories in every case.  For
opening files, the common case of a file being in the first branch that
its parent exists in is relatively quick (since only one lookup needs to
be done).  If you then need to do a more complex operation you'll need
to do lookup on each underlying branch.

It also depends on your name spaces, if you have one flat namespace
performance will suffer more than if you have a quickly treeing name
space such that a directory only exists in relatively few branches.

We have some performance numbers in our tech report at:
http://www.fsl.cs.sunysb.edu/docs/unionfs-tr/index.html

> - How would performance degrade or memory usage rise if a particular
>    directory in one branch, or across multiple branches, had millions
>    or tens of millions of files (assuming the underlying filesystem
>    handles such quantities acceptably)?
The memory usage will increase for each in memory object based on the
number of branches (not files within them).

In the midst of a readdir you will need on the order of 64 bytes per
file name in the directory.  This is unavoidable if you want duplicate
elimination (you need to remember what you already returned to the
user).

> - How does UnionFS handle detected disk failures on one of the
>    underlying branches? (Could a remove/rebuild/replace cycle be
>    triggered?)
Right now it will pass up the error.   In reality, it is possible that
it will get confused depending on the exact circumstance.

> - Are there inherent factors which would prevent a future
>    revision of UnionFS from...
>    - optionally operating under the assumption that the same path
>      on different branches represents an identical file, and
>      fanning-out reads for greater speed?
We have a replication fan-out system that does precisely that (it was
folded into the RAIF project that I mentioned above).

>    - choosing from more than one r/w branch to write a particular
>      file (or at least offering some way to force a particular
>      open/copy into a deeper branch)?
You would need to change user-space as well to do it, but it would
probably be possible to do it with an ioctl that would invoke a create
on one of the branches, then open that file.

Charles

_______________________________________________
unionfs mailing list
unionfs@mail.fsl.cs.sunysb.edu
http://www.fsl.cs.sunysb.edu/mailman/listinfo/unionfs