large files (5-50 GB)

[Jeff Wang <je...@mac.com>]

Hello,

I've gotten JackRabbit 1.5 working on Tomcat 6.x. I've configured the  
datastore to handle large files.  I am testing the datastore by mounting
a webdav mount on the same computer.  I can upload files to about 1GB  
but when I try to upload a 7GB video file, it says:

"Sorry, operation could not be completed because an unexpected error  
occurred (Error Code 0)"

Is there a maximum file size limit? We may need to store video files  
up to 50 GB so I am thinking that taking the
message digest of that is going to take a really long time.  That is  
what prompted these tests.

thanks,
Jeff

Re: large files (5-50 GB)

[Thomas Müller <th...@day.com>]

Hi,

> a webdav mount on the same computer.  I can upload files to about 1GB but
> > when I try to upload a 7GB video file, it says:
>

What file system do you use? I have updated the documentation in the wiki to
say "The file system must supports files as large as the largest object you
want to store. Please note that the file size limit of FAT32 is 2 GB". If
you need FAT32 support, could you open a JIRA issue? It's possible to work
around this limitation, but I don't know how many still use FAT32.

Regards,
Thomas

Re: large files (5-50 GB)

[Greg Klebus <gr...@klebus.com>]

Hi Jeff

On Tue, Feb 17, 2009 at 8:29 PM, Jeff Wang <je...@mac.com> wrote:
> I've gotten JackRabbit 1.5 working on Tomcat 6.x. I've configured the
> datastore to handle large files.  I am testing the datastore by mounting
> a webdav mount on the same computer.  I can upload files to about 1GB but
> when I try to upload a 7GB video file, it says:
>
> "Sorry, operation could not be completed because an unexpected error
> occurred (Error Code 0)"

We'd need more information, as there are many potential reasons for a
problem like this.

Of special interest:

- PM user (Derby?), DataStore configuration (DB- or FileDataStore?)
- log files, including tomcat log files (ideally parts with relevant
exceptions or messages)

> Is there a maximum file size limit?

Generally no, although various sub-components or external dependencies
might impose their own limits. Two most likely culprits:

- servlet container might choke on large files (google for keywords:
tomcat large file upload 2GB)
- AFAIR Derby has 2GB limit for blobs (if DBDataStore used)

> We may need to store video files up to
> 50 GB so I am thinking that taking the
> message digest of that is going to take a really long time.  That is what
> prompted these tests.

Right approach IMO - defining your benchmark and runing respective
tests. If only more people followed these rules instead of asking
"what is the maximum value of X that your system can bear with"... :)

Cheers
Greg