File persistence with NAS (NFS)

[hsp <pi...@ibest.com.br>]

Hi;

We are using the api jackrabbit 2.8.
Recently the environment was migrated to use a NAS (Isilon).
So, the server was mapped to use the resource by NFS (SUSE SLES).
At first would be transparent to jackrabbit, so the configuration
(repository.xml/workspace.xml) was not changed.
The application got ok, but, when the jackrabbit is writing a file the
iowait parameter in server rises too much, and files bigger than 10MB wont
be saved because session timeout .
We have tested the speed in copy files from server to NAS directly and it
was in 30MBps, so why when the jackrabbit is sending the file to write in
NAS the speed goes down in that way (2 - 5 MBps... )

Apreciate some help in this scenario. Regards!


Just for information, that's our repository.xml:







--
View this message in context: http://jackrabbit.510166.n4.nabble.com/File-persistence-with-NAS-NFS-tp4662164.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.

Re: File persistence with NAS (NFS)

[hsp <pi...@ibest.com.br>]

We have plans to migrate to oak, in future.
But, nowadays, I did an edit directly in FileDataStore to support my needs.
(Actually I will do an overryde class...)

The way the FileDataStore receives de InputStream (as a SequenceInputStream)
it is not possible to send over java.nio directly, so I persist the stream
in a tmpfile in tmp server local drive (calculating the size and hash), and
if that file not yet exists in ASN the transfer is triggered with java.nio.


There is the code:





--
View this message in context: http://jackrabbit.510166.n4.nabble.com/File-persistence-with-NAS-NFS-tp4662164p4662273.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.

Re: File persistence with NAS (NFS)

[Thomas Mueller <mu...@adobe.com>]

Hi,

The FileDataStore is not optimized for that case. Patches would be welcome!

Please note in Oak, there is a new content addressable storage
(FileBlobStore) that does not have those problems: it doesn't create temp
files because it splits binaries into chunks of 2 MB (configurable).

Regards,
Thomas





On 24/03/15 14:39, "hsp" <pi...@ibest.com.br> wrote:

>The class org.apache.jackrabbit.core.data.FileDataStore uses
>apache.IOUtils
>to send a file to the filesystem. In my case this is done using a NFS
>resource mapped to a NAS directory.
>Would the digest (SHA-1) calculation be done in local tmp instead a remote
>tmp (in my case) be faster?
>And, instead use IOUtils, use the java.nio to transfer in blocks (better
>than stream...)?
>
>Any thougths? 
>Thanks in advanced.
>Helio.
>
>
>
>--
>View this message in context:
>http://jackrabbit.510166.n4.nabble.com/File-persistence-with-NAS-NFS-tp466
>2164p4662169.html
>Sent from the Jackrabbit - Users mailing list archive at Nabble.com.

Re: File persistence with NAS (NFS)

[hsp <pi...@ibest.com.br>]

The class org.apache.jackrabbit.core.data.FileDataStore uses apache.IOUtils
to send a file to the filesystem. In my case this is done using a NFS
resource mapped to a NAS directory.
Would the digest (SHA-1) calculation be done in local tmp instead a remote
tmp (in my case) be faster?
And, instead use IOUtils, use the java.nio to transfer in blocks (better
than stream...)?

Any thougths? 
Thanks in advanced.
Helio.



--
View this message in context: http://jackrabbit.510166.n4.nabble.com/File-persistence-with-NAS-NFS-tp4662164p4662169.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.