You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Gadbury <ga...@googlemail.com> on 2009/07/15 10:42:56 UTC
DataStore - File or Database for an Ecommerce Web Application?
Hi all,
I am currently working on an e-commerce web application. As we could be
storing large binary files, I am investigating the use of a DataStore. I am
trying to decide whether to use a file data store or a database data store.
I have read the Jackrabbit article on data stores
(http://wiki.apache.org/jackrabbit/DataStore) but there are some aspects I
would like to clarify. One concern I have is what would happen when
multiple users download large files from the store? For example, lets say
100 users are downloading the same 100MB file from the Datastore. For file
data stores, the article states:
"The file data store does not use any local cache, that means content is
directly read from the files as needed. New content is first stored in a
temporary file, and later renamed / moved."
I assume this means that with 100 users downloading the same 100MB file, all
downloading users will stream directly from the file and that no local /
temporary copies of the file would be created on the server. I'm not quite
sure what it means by new content; does it mean that when a new file is
added to the repository (data store), a temporary copy will be created on
the server? When reading and downloading, are no temporary copies of the
file created on the server when using a file data store?
If I was to use a database data store and apply the same scenario, would 100
local copies (on the server) of the 50MB file be created for each user
requesting the file for download? Or would there be just one temporary
instance of the same file? If that is the case, this would present a
problem if those 100 users were each downloading different 100MB files.
Please would someone be able to advise? I just want to be sure that I have
understood the difference between file and database data stores, and the
implications of each. I am concerned about the memory constraints and the
server becoming overwhelmed.
Thanks for your time.
--
View this message in context: http://www.nabble.com/DataStore---File-or-Database-for-an-Ecommerce-Web-Application--tp24494051p24494051.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: DataStore - File or Database for an Ecommerce Web Application?
Posted by Alexander Klimetschek <ak...@day.com>.
On Wed, Jul 15, 2009 at 10:42 AM, Gadbury<ga...@googlemail.com> wrote:
> "The file data store does not use any local cache, that means content is
> directly read from the files as needed. New content is first stored in a
> temporary file, and later renamed / moved."
This text is 100% correct (for the file datastore).
> I assume this means that with 100 users downloading the same 100MB file, all
> downloading users will stream directly from the file and that no local /
> temporary copies of the file would be created on the server. I'm not quite
> sure what it means by new content; does it mean that when a new file is
> added to the repository (data store), a temporary copy will be created on
> the server?
Upon saving a new binary stream via the JCR API, it will first be
saved in a temporary file internally because the datastore a) only
stores full files (there could be an error while streaming a large
binary) and b) it needs to calculate the MD5 of the file to use it as
identifier.
> When reading and downloading, are no temporary copies of the
> file created on the server when using a file data store?
Right.
> If I was to use a database data store and apply the same scenario, would 100
> local copies (on the server) of the 50MB file be created for each user
> requesting the file for download? Or would there be just one temporary
> instance of the same file? If that is the case, this would present a
> problem if those 100 users were each downloading different 100MB files.
AFAIK this depends on the JDBC driver of the database, but I think
most of them spool binary blobs in temporary local files. In general,
a file datastore has always very good performance and a database
datastore cannot be better - it will most likely be slower.
Regards,
Alex
--
Alexander Klimetschek
alexander.klimetschek@day.com
Re: DataStore - File or Database for an Ecommerce Web Application?
Posted by Thomas Müller <th...@day.com>.
Hi,
I have added the answers of your questions to
http://wiki.apache.org/jackrabbit/DataStore
Regards,
Thomas
Re: DataStore - File or Database for an Ecommerce Web Application?
Posted by Alexander Klimetschek <ak...@day.com>.
On Thu, Jul 16, 2009 at 10:58 AM, Gadbury<ga...@googlemail.com> wrote:
> With a configuration such as the one above (note: using a db data store), I
> assume that all workspace metadata and versioning data will be stored (using
> the MySQL persistance manager) in the MySQL database 'ecommerce_content',
> and any binary data over the size of the param minRecordLength will be
> stored in the MySQL database 'ecommerce_datastore'...? Without a data store
> specified, I assume that the binary data would be stored in the
> ecommerce_content.wsp_binval table...? I guess the idea is that one can
> specify different storage mechanisms for different aspects of the repository
> (versioning, workspace metadata, large binary data, etc.) ...? Please could
> someone correct me if I am not understanding correctly :)
Exactly, I couldn't have summarized that better ;-)
Regards,
Alex
--
Alexander Klimetschek
alexander.klimetschek@day.com
Re: DataStore - File or Database for an Ecommerce Web Application?
Posted by Gadbury <ga...@googlemail.com>.
Hi,
Thank you for the response, Alex. Also, thank you Thomas for updating the
DataStore FAQ. It really helped although I am sorry to say that I have a
few more questions to ask.
What is the difference between using a database datastore and not specifying
a data store but using a db persistance manager? My configuration file is
as follows:
----------------------------------------------
<Repository name="repository">
<FileSystem
class="org.apache.jackrabbit.core.fs.db.DbFileSystem"></FileSystem>
<Security appName="Jackrabbit"></Security>
<Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="wsp"/>
<Workspace name="wsp">
<FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
...
</FileSystem>
<PersistenceManager
class="org.apache.jackrabbit.core.persistence.bundle.MySqlPersistenceManager">
...
</PersistenceManager>
<SearchIndex
class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
<FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
...
</FileSystem>
</SearchIndex>
</Workspace>
<Versioning rootPath="${rep.home}/version">
<FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
...
</FileSystem>
<PersistenceManager
class="org.apache.jackrabbit.core.persistence.bundle.MySqlPersistenceManager">
...
</PersistenceManager>
</Versioning>
<SearchIndex
class="org.apache.jackrabbit.core.query.lucene.SearchIndex"></SearchIndex>
<DataStore class="org.apache.jackrabbit.core.data.db.DbDataStore">
...
</DataStore>
</Repository>
---------------------------------------------------
With a configuration such as the one above (note: using a db data store), I
assume that all workspace metadata and versioning data will be stored (using
the MySQL persistance manager) in the MySQL database 'ecommerce_content',
and any binary data over the size of the param minRecordLength will be
stored in the MySQL database 'ecommerce_datastore'...? Without a data store
specified, I assume that the binary data would be stored in the
ecommerce_content.wsp_binval table...? I guess the idea is that one can
specify different storage mechanisms for different aspects of the repository
(versioning, workspace metadata, large binary data, etc.) ...? Please could
someone correct me if I am not understanding correctly :)
Thanks again for your time. Regards,
James
Gadbury wrote:
>
> Hi all,
>
> I am currently working on an e-commerce web application. As we could be
> storing large binary files, I am investigating the use of a DataStore. I
> am trying to decide whether to use a file data store or a database data
> store.
>
> I have read the Jackrabbit article on data stores
> (http://wiki.apache.org/jackrabbit/DataStore) but there are some aspects I
> would like to clarify. One concern I have is what would happen when
> multiple users download large files from the store? For example, lets say
> 100 users are downloading the same 100MB file from the Datastore. For
> file data stores, the article states:
>
> "The file data store does not use any local cache, that means content is
> directly read from the files as needed. New content is first stored in a
> temporary file, and later renamed / moved."
>
> I assume this means that with 100 users downloading the same 100MB file,
> all downloading users will stream directly from the file and that no local
> / temporary copies of the file would be created on the server. I'm not
> quite sure what it means by new content; does it mean that when a new file
> is added to the repository (data store), a temporary copy will be created
> on the server? When reading and downloading, are no temporary copies of
> the file created on the server when using a file data store?
>
> If I was to use a database data store and apply the same scenario, would
> 100 local copies (on the server) of the 50MB file be created for each user
> requesting the file for download? Or would there be just one temporary
> instance of the same file? If that is the case, this would present a
> problem if those 100 users were each downloading different 100MB files.
>
> Please would someone be able to advise? I just want to be sure that I
> have understood the difference between file and database data stores, and
> the implications of each. I am concerned about the memory constraints
> and the server becoming overwhelmed.
>
> Thanks for your time.
>
--
View this message in context: http://www.nabble.com/DataStore---File-or-Database-for-an-Ecommerce-Web-Application--tp24494051p24512779.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.