multiple webapps many embedded vs single network

[Ron Chan <rc...@i-tao.com>]

Hi

For multiple webapps on a single tomcat instance, is it better to have
separate embedded databases in each webapp or have one derby network server
serving them all?

The webapps don't share data, they have their own database files.

Thanks
Ron

-- 
View this message in context: http://www.nabble.com/multiple-webapps-many-embedded-vs-single-network-tf2522906.html#a7037681
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: multiple webapps many embedded vs single network

[Ron Chan <rc...@i-tao.com>]

Thanks Dan, that was what i was looking for, the discussion seems to have
taken off in a slightly different tangent to my original question, but it is
still good information to know, so thanks all for your input.

To give a little bit more background, the application is being designed so
that it can be downloaded and run without any other dependencies (ie tomcat
or jetty running single webapp with derby embedded).  However, there is also
an option for hosting it, therefore we started thinking about hosting a
number of copies of the same webapp in the same tomcat - that's when we
wanted to know if it is better then to have a single network server derby
serving all the webapps.

Thanks
Ron



Dan Scott wrote:
> 
> Well, this seems like flamebait to me. The proof is in the Apache
> Derby that exists right now: it has a small foot print, you can set it
> up in Network Server mode, grant privileges to discrete users and
> separate applications through the use of schemas. It seems to perform
> quite well, although nobody to date has stepped up to fund an official
> TPC-C or TPC-H benchmark (which shouldn't surprise anyone, as MySQL
> and PostgreSQL haven't had official benchmarks set here either). But
> the original poster's question was not about performance benchmarks or
> competing with IDS, Oracle, or DB2; it was a simple question about
> memory usage.
> 
> To answer the original poster's question, Network Server mode is
> undoubtedly more memory efficient if you're using it as a central
> database for more than a couple of applications. The reason the
> threshold is a couple of applications is that the Derby database
> itself runs in one JVM, while the Network Server runs in a separate
> JVM. Thus, your memory break-even point is effectively two
> applications hosted in a single Network Server mode Derby database.
> After that, you're saving yourself one JVM per application (subject,
> of course, to some increasing memory overhead due to the number of
> concurrent connections your Network Server may have to handle).
> 
> In terms of separating your applications at a later date, I wouldn't
> worry too much about that. Use schemas to separate your application
> and you can pretty easily shift an application to point at another
> database at any given time; just export the schema for the given
> application, load it in the new database, and change your connection
> string to hit the new database.
> 
> Dan
> 
> 

-- 
View this message in context: http://www.nabble.com/multiple-webapps-many-embedded-vs-single-network-tf2522906.html#a7071921
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: multiple webapps many embedded vs single network

[Dan Scott <de...@gmail.com>]

On 29/10/06, Michael Segel <ms...@segel.com> wrote:
>
> > -----Original Message-----
> > From: Jean T. Anderson [mailto:jta@bristowhill.com]
> > Sent: Sunday, October 29, 2006 1:32 PM
> > To: Derby Discussion
> > Subject: Re: multiple webapps many embedded vs single network
> >
> > Michael Segel wrote:
> > ...
> > > Derby wasn't designed to be a central database to multiple apps. So its
> > not
> > > efficient in that role. Note: This is in comparison to IDS, DB2, Oracle.
> > > Derby is not one of those. It lacks the features that they have to act
> > as a
> > > centralized DB, however it does have a much smaller footprint.
> >
> > I disagree with this statement, but perhaps I didn't read this thread
> > carefully and am missing some context.
> >
> [mjs]
> Knowing you, you haven't really thought about what I was saying.
>
> > Derby fully supports multi-user, multi-application concurrent access,
> > even in embedded mode. It complies with the ACID (Atomic, Consistent,
> > Isolation, Durable) properties expected of relational databases.
> >
> [mjs]
>
> And what does this have to do with performance?
>
> Now Jean you should know better. But I don't believe you're truly a heritage
> Informix person. ;-)
>
> I would suggest that talk to Mohan and ask him about items in DB2 and
> Informix that improve performance. Just how IDS stores data is way more
> advanced. Raw partitions/chucks, table spaces, table portioning, detatched
> indexes. It's the whole layout.
>
> > Olav Sandstaa's ApacheCon US 2005 performance presentation is here and
> > includes results for Derby in both embedded and client/server modes
> > (plus MySQL and PostgreSQL):
> > http://wiki.apache.org/apachecon-
> > data/attachments/Us2005OnlineSessionSlides/attachments/ApacheCon05usDerbyP
> > erformance.pdf
> >
> [mjs]
> Ooooh! MySQL, now that's a barn burner when it comes to looking for a
> centralized database?
>
> Hint: Compare MySQL to DB2, Oracle, Informix on performance issues.
>
> > I don't know of any similar results for comparing Derby performance to
> > IDS, DB2, and Oracle. If anyone knows of any such studies, please post a
> > pointer.
> >
>
> [mjs]
> You won't see it.
> Is there a Derby TPC-C benchmark?
>
> In case you've missed it, Derby was designed with a small footprint. Adding
> features that make databases perform well where there are multiple
> instances/databases per server, improved data storage techniques etc are
> missing from Derby.
>
> Is that a bad thing? No. Derby was designed for a specific niche and not a
> general purpose centralized database. You want those features, then you
> increase your footprint and you lose your ability to be embedded in a web
> page.
>
>
> We've covered this ground before.
> Does the community want a 100% Java, and a *free* database that could
> compete with Oracle, DB2 and Informix? Are there people in the community
> that are capable of designing and implementing this?
>
> And how do you still maintain the small foot print for those that want to
> imbed it?

Well, this seems like flamebait to me. The proof is in the Apache
Derby that exists right now: it has a small foot print, you can set it
up in Network Server mode, grant privileges to discrete users and
separate applications through the use of schemas. It seems to perform
quite well, although nobody to date has stepped up to fund an official
TPC-C or TPC-H benchmark (which shouldn't surprise anyone, as MySQL
and PostgreSQL haven't had official benchmarks set here either). But
the original poster's question was not about performance benchmarks or
competing with IDS, Oracle, or DB2; it was a simple question about
memory usage.

To answer the original poster's question, Network Server mode is
undoubtedly more memory efficient if you're using it as a central
database for more than a couple of applications. The reason the
threshold is a couple of applications is that the Derby database
itself runs in one JVM, while the Network Server runs in a separate
JVM. Thus, your memory break-even point is effectively two
applications hosted in a single Network Server mode Derby database.
After that, you're saving yourself one JVM per application (subject,
of course, to some increasing memory overhead due to the number of
concurrent connections your Network Server may have to handle).

In terms of separating your applications at a later date, I wouldn't
worry too much about that. Use schemas to separate your application
and you can pretty easily shift an application to point at another
database at any given time; just export the schema for the given
application, load it in the new database, and change your connection
string to hit the new database.

Dan

Re: multiple webapps many embedded vs single network

["Jean T. Anderson" <jt...@bristowhill.com>]

Michael Segel wrote:
>>-----Original Message-----
>>From: Jean T. Anderson [mailto:jta@bristowhill.com]
>>Sent: Sunday, October 29, 2006 1:32 PM
>>To: Derby Discussion
>>Subject: Re: multiple webapps many embedded vs single network
>>
>>Michael Segel wrote:
>>...
>>
>>>Derby wasn't designed to be a central database to multiple apps. So its
>>not
>>>efficient in that role. Note: This is in comparison to IDS, DB2, Oracle.
>>>Derby is not one of those. It lacks the features that they have to act
>>as a
>>>centralized DB, however it does have a much smaller footprint.
>>
>>I disagree with this statement, but perhaps I didn't read this thread
>>carefully and am missing some context.
> 
> [mjs] 
> Knowing you, you haven't really thought about what I was saying.

Actually, I did think about it, but perhaps narrowly to address a common
misconception I encounter (more in the next paragraph). The sentence
that caught my attention was "Derby wasn't designed to be a central
database to multiple apps."

I frequently encounter the misconception that Derby is a single user
database, especially in embedded mode. For those who are confused, I'd
like to make sure that they understand that even in embedded mode Derby
is an ACID-compliant, multiuser database. Derby *is* designed to provide
 database access to multiple apps and users.

 -jean

RE: multiple webapps many embedded vs single network

[Michael Segel <ms...@segel.com>]

> -----Original Message-----
> From: Jean T. Anderson [mailto:jta@bristowhill.com]
> Sent: Sunday, October 29, 2006 1:32 PM
> To: Derby Discussion
> Subject: Re: multiple webapps many embedded vs single network
> 
> Michael Segel wrote:
> ...
> > Derby wasn't designed to be a central database to multiple apps. So its
> not
> > efficient in that role. Note: This is in comparison to IDS, DB2, Oracle.
> > Derby is not one of those. It lacks the features that they have to act
> as a
> > centralized DB, however it does have a much smaller footprint.
> 
> I disagree with this statement, but perhaps I didn't read this thread
> carefully and am missing some context.
> 
[mjs] 
Knowing you, you haven't really thought about what I was saying.

> Derby fully supports multi-user, multi-application concurrent access,
> even in embedded mode. It complies with the ACID (Atomic, Consistent,
> Isolation, Durable) properties expected of relational databases.
> 
[mjs] 

And what does this have to do with performance?

Now Jean you should know better. But I don't believe you're truly a heritage
Informix person. ;-)

I would suggest that talk to Mohan and ask him about items in DB2 and
Informix that improve performance. Just how IDS stores data is way more
advanced. Raw partitions/chucks, table spaces, table portioning, detatched
indexes. It's the whole layout. 

> Olav Sandstaa's ApacheCon US 2005 performance presentation is here and
> includes results for Derby in both embedded and client/server modes
> (plus MySQL and PostgreSQL):
> http://wiki.apache.org/apachecon-
> data/attachments/Us2005OnlineSessionSlides/attachments/ApacheCon05usDerbyP
> erformance.pdf
> 
[mjs] 
Ooooh! MySQL, now that's a barn burner when it comes to looking for a
centralized database?

Hint: Compare MySQL to DB2, Oracle, Informix on performance issues.

> I don't know of any similar results for comparing Derby performance to
> IDS, DB2, and Oracle. If anyone knows of any such studies, please post a
> pointer.
> 

[mjs] 
You won't see it.
Is there a Derby TPC-C benchmark?

In case you've missed it, Derby was designed with a small footprint. Adding
features that make databases perform well where there are multiple
instances/databases per server, improved data storage techniques etc are
missing from Derby.

Is that a bad thing? No. Derby was designed for a specific niche and not a
general purpose centralized database. You want those features, then you
increase your footprint and you lose your ability to be embedded in a web
page.


We've covered this ground before.
Does the community want a 100% Java, and a *free* database that could
compete with Oracle, DB2 and Informix? Are there people in the community
that are capable of designing and implementing this?

And how do you still maintain the small foot print for those that want to
imbed it?

Re: multiple webapps many embedded vs single network

["Jean T. Anderson" <jt...@bristowhill.com>]

Michael Segel wrote:
...
> Derby wasn't designed to be a central database to multiple apps. So its not
> efficient in that role. Note: This is in comparison to IDS, DB2, Oracle.
> Derby is not one of those. It lacks the features that they have to act as a
> centralized DB, however it does have a much smaller footprint.

I disagree with this statement, but perhaps I didn't read this thread
carefully and am missing some context.

Derby fully supports multi-user, multi-application concurrent access,
even in embedded mode. It complies with the ACID (Atomic, Consistent,
Isolation, Durable) properties expected of relational databases.

Olav Sandstaa's ApacheCon US 2005 performance presentation is here and
includes results for Derby in both embedded and client/server modes
(plus MySQL and PostgreSQL):
http://wiki.apache.org/apachecon-data/attachments/Us2005OnlineSessionSlides/attachments/ApacheCon05usDerbyPerformance.pdf

Slide 21 is especially interesting, which compares main-memory and
disk-based databases for 1-100 clients. While Derby does not perform as
well as MySQL and PostgreSQL for main-memory databases, it does perform
better that MySQL for disk-based databases. Note that this presentation
doesn't reflect the performance enhancements in 10.2.

I don't know of any similar results for comparing Derby performance to
IDS, DB2, and Oracle. If anyone knows of any such studies, please post a
pointer.

regards,

-jean

RE: multiple webapps many embedded vs single network

[Michael Segel <ms...@segel.com>]

> -----Original Message-----
> From: Ron Chan [mailto:rchan@i-tao.com]
> Sent: Saturday, October 28, 2006 5:52 AM
> To: derby-user@db.apache.org
> Subject: Re: multiple webapps many embedded vs single network
> 
> 
> I guess either way, derby allows me to do this easily, i just need to move
> the folder where the data is held
> 
> I was more refering to what is more efficient, if i run multiple embedded
> derby instances does it need greater memory requirements than a single
> network server instance serving all the databases?
> 
> 
[mjs] 
I understand. 
But the "efficiency" is not a main issue when you look at the larger
picture.

If your applications are database heavy, I would consider them tightly
coupled and I'd run multiple instances (One per app). You can then either
split the app from the database server, or you can split the apps and derby
to a different server as they grow.

If your applications are tightly coupled to each other, I'd run a single
instance of Derby for those applications. If Derby requires more resources,
then I'd separate Derby to a different host from your app server.

This should be your underlying concern as you design your hardware around
your application.

Today, you can buy a dual processor mother board and then each processor has
dual cores. That's a lot of horsepower, especially on a Linux Server. With
RAID built in, and using SATA drives you can build a custom box for a
relatively low price point. That's why it's a moot point. Your maintenance
cost of the application is going to be more expensive than doing a hardware
mod.


But to answer your question....

Derby wasn't designed to be a central database to multiple apps. So its not
efficient in that role. Note: This is in comparison to IDS, DB2, Oracle.
Derby is not one of those. It lacks the features that they have to act as a
centralized DB, however it does have a much smaller footprint.

Does that help?



> 
> Michael Segel-2 wrote:
> >
> > Depends on how tightly coupled the web apps are.
> >
> > Also how tightly do you want to couple the database to the app...
> >
> > Suppose at a later date you have some growth. You need to move some of
> > your apps to a new machine.
> >
> > What do you do?
> >
> >
> > Sent via BlackBerry.
> >
> > -Mike Segel
> > Principal
> > MSCC
> > 312 952 8175
> >
> >
> > -----Original Message-----
> > From: Ron Chan <rc...@i-tao.com>
> > Date: Fri, 27 Oct 2006 13:23:27
> > To:derby-user@db.apache.org
> > Subject: multiple webapps many embedded vs single network
> >
> >
> > Hi
> >
> > For multiple webapps on a single tomcat instance, is it better to have
> > separate embedded databases in each webapp or have one derby network
> > server
> > serving them all?
> >
> > The webapps don't share data, they have their own database files.
> >
> > Thanks
> > Ron
> >
> > --
> > View this message in context:
> > http://www.nabble.com/multiple-webapps-many-embedded-vs-single-network-
> tf2522906.html#a7037681
> > Sent from the Apache Derby Users mailing list archive at Nabble.com.
> >
> >
> 
> --
> View this message in context: http://www.nabble.com/multiple-webapps-many-
> embedded-vs-single-network-tf2522906.html#a7047639
> Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: multiple webapps many embedded vs single network

[Ron Chan <rc...@i-tao.com>]

I guess either way, derby allows me to do this easily, i just need to move
the folder where the data is held

I was more refering to what is more efficient, if i run multiple embedded
derby instances does it need greater memory requirements than a single
network server instance serving all the databases?



Michael Segel-2 wrote:
> 
> Depends on how tightly coupled the web apps are.
> 
> Also how tightly do you want to couple the database to the app...
> 
> Suppose at a later date you have some growth. You need to move some of
> your apps to a new machine.
> 
> What do you do?
> 
> 
> Sent via BlackBerry.
> 
> -Mike Segel
> Principal
> MSCC
> 312 952 8175
> 
> 
> -----Original Message-----
> From: Ron Chan <rc...@i-tao.com>
> Date: Fri, 27 Oct 2006 13:23:27 
> To:derby-user@db.apache.org
> Subject: multiple webapps many embedded vs single network
> 
> 
> Hi
> 
> For multiple webapps on a single tomcat instance, is it better to have
> separate embedded databases in each webapp or have one derby network
> server
> serving them all?
> 
> The webapps don't share data, they have their own database files.
> 
> Thanks
> Ron
> 
> -- 
> View this message in context:
> http://www.nabble.com/multiple-webapps-many-embedded-vs-single-network-tf2522906.html#a7037681
> Sent from the Apache Derby Users mailing list archive at Nabble.com.
> 
> 

-- 
View this message in context: http://www.nabble.com/multiple-webapps-many-embedded-vs-single-network-tf2522906.html#a7047639
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: multiple webapps many embedded vs single network

[Michael Segel <ms...@mycingular.blackberry.net>]

Depends on how tightly coupled the web apps are.

Also how tightly do you want to couple the database to the app...

Suppose at a later date you have some growth. You need to move some of your apps to a new machine.

What do you do?


Sent via BlackBerry.

-Mike Segel
Principal
MSCC
312 952 8175


-----Original Message-----
From: Ron Chan <rc...@i-tao.com>
Date: Fri, 27 Oct 2006 13:23:27 
To:derby-user@db.apache.org
Subject: multiple webapps many embedded vs single network


Hi

For multiple webapps on a single tomcat instance, is it better to have
separate embedded databases in each webapp or have one derby network server
serving them all?

The webapps don't share data, they have their own database files.

Thanks
Ron

-- 
View this message in context: http://www.nabble.com/multiple-webapps-many-embedded-vs-single-network-tf2522906.html#a7037681
Sent from the Apache Derby Users mailing list archive at Nabble.com.