A generic question about JackRabbit

[Rami Ojares <ra...@gmail.com>]

Hi,

I only now stumbled onto JackRabbit project.
And a basic question about it rose in me spontaneously.

Early in the seventies Ted Codd published relational model as an improvement
over the hierarchical model that had been used for data storage until then.

So why do we still need a hierarchical model for data storage?

Features mentioned like full text search, versioning, transactions, 
observation, and more
can be just as well implemented with the relational model.

And what does unstructured data mean?
It sounds to me like random noise.

- rami

Re: A generic question about JackRabbit

[ChadDavis <ch...@gmail.com>]

On Fri, Mar 26, 2010 at 8:34 AM, Rami Ojares <ra...@gmail.com> wrote:
> Hi,
>
> I only now stumbled onto JackRabbit project.
> And a basic question about it rose in me spontaneously.
>
> Early in the seventies Ted Codd published relational model as an improvement
> over the hierarchical model that had been used for data storage until then.
>
> So why do we still need a hierarchical model for data storage?
>

Some data is more suited to hierarchical data.  I'm no expert, but I
am a senior Java engineer and I had plenty of database theory classes.
 JCR just exposes a persistence layer at a level that reflects a
object based, hierarchical view of data, which meshes well with
current application usages.

> Features mentioned like full text search, versioning, transactions,
> observation, and more
> can be just as well implemented with the relational model.
>

Sure.  And, in many cases Jackrabbit is using a relational db
underneath the hood.  I consider the JCR as simply a higher level data
persistence.


> And what does unstructured data mean?

RDMBS development is also unstructured.  But typically you have to
redefined DDL and rebuild the DB each time you modify the structure.
Again, JCR provides a higher level persistence layer that gives
benefits to application during their "fluid" data structure period,
and this period can frequently be lengthier than development schedules
would like to portray.

> It sounds to me like random noise.
>
> - rami
>

Re: A generic question about JackRabbit

[ChadDavis <ch...@gmail.com>]

On Fri, Mar 26, 2010 at 8:47 AM, Joseph Ottinger <dr...@gmail.com> wrote:
> And I'm a moron - research shows that E.F. Codd went by Ted as well. Sorry,
> folks. :)
>

Not as much of a moron as I.  I was going to suggest that his name was
"Boyce-Codd" . . .

Re: A generic question about JackRabbit

[Joseph Ottinger <dr...@gmail.com>]

And I'm a moron - research shows that E.F. Codd went by Ted as well. Sorry,
folks. :)

On Fri, Mar 26, 2010 at 10:46 AM, Joseph Ottinger <dr...@gmail.com>wrote:

> It was E.F.Codd, for one thing, and not Ted. :)
>
> But... relational models work very well for *reporting* but not so much for
> other things. They're kinda slow, for example. Hierarchical and set-related
> models (document-based, ODBMS, and CODASYL dbs, respectively) are much, much
> faster for some operations, and slower for others.
>
> Relational models excel in reporting and warehousing. The rest... meh,
> they're kinda slow on updates and set relationships. We're just so used to
> the lack of speed that we think it's the norm.
>
> However, if you're doing a lot of updates or your set modifications are
> common, you'll typically find the other models faster, and - depending on
> your taste - easier.
>
> Certainly transaction processing is easier with nonrelational.
>
> That's not to say that relational databases are *bad* -- they're just very
> good at structured data, used for reporting, which is what they were used
> for most often. Now they're seen as part of the background noise of data
> storage, so everyone converts their problem into something the relational
> database can use - as opposed to finding the solution that fits the problem
> well.
>
> We've decided to let our tools dictate our solutions, instead of picking
> our solutions to fit our problems.
>
>
> On Fri, Mar 26, 2010 at 10:34 AM, Rami Ojares <ra...@gmail.com>wrote:
>
>> Hi,
>>
>> I only now stumbled onto JackRabbit project.
>> And a basic question about it rose in me spontaneously.
>>
>> Early in the seventies Ted Codd published relational model as an
>> improvement
>> over the hierarchical model that had been used for data storage until
>> then.
>>
>> So why do we still need a hierarchical model for data storage?
>>
>> Features mentioned like full text search, versioning, transactions,
>> observation, and more
>> can be just as well implemented with the relational model.
>>
>> And what does unstructured data mean?
>> It sounds to me like random noise.
>>
>> - rami
>>
>
>
>
> --
> Joseph B. Ottinger
> http://enigmastation.com
>



-- 
Joseph B. Ottinger
http://enigmastation.com

Re: A generic question about JackRabbit

[Thomas Müller <th...@day.com>]

Hi,

You can't say one is better than the other, they are just different.
JCR is more geared towards file systems (which are also hierarchical)
and web content (urls, which are also hierarchical). It's easier to
map this kind of content into the JCR model than to map it to the
relational model. Then, nodes don't have a fixed set of properties.
This better matches to the 'object oriented' world than a relational
model. It also better matches XML / JSON use cases. JCR has some
features that are not / less standardized in the relational world (for
example versioning).

> if you're doing a lot of updates or your set modifications are
> common, you'll typically find the other models faster,

Yes and no. One advantage of the relational model is that it usually
better compresses the data because of normalization. But that only
applies in the small scale. If the data doesn't fit into one system,
other approaches may scale better. That's the reason for Google
BigTable, Hadoop, Cassandra, and so on. It also applies to JCR. The
current Jackrabbit implementation is not yet 'highly scalable' (that
wasn't the goal yet), but the JCR model is. I'm not sure if the
relational model does limit the scalability of SQL databases, but
current relational database implementations just don't scale very
well.

Regards,
Thomas

Re: A generic question about JackRabbit

[Joseph Ottinger <dr...@gmail.com>]

On Fri, Mar 26, 2010 at 11:07 AM, Rami Ojares <ra...@gmail.com> wrote:

> On 26.3.2010 16:46, Joseph Ottinger wrote:
>
>> It was E.F.Codd, for one thing, and not Ted. :)
>>
>
> Ted was his nickname.
> I was trying to be casual :-)
> (But I see you realized this while I was writing this email)
>
> I don't know him casually (or formally, so...) My mistake. :)

>
>  But... relational models work very well for *reporting* but not so much
>> for
>> other things. They're kinda slow, for example. Hierarchical and
>> set-related
>> models (document-based, ODBMS, and CODASYL dbs, respectively) are much,
>> much
>> faster for some operations, and slower for others.
>>
>
> Yes the speed issue.
> It is true that if your data is structured hierarchically and you access it
> in your application then you can create
> the fastest application by storing it in the fashion that is good for the
> first application.
> It just so happens that other applications come along that want to use the
> same data also (to avoid redundancy).
> And for them the hierarchical model might prove to be a nightmare.
>
> Sure, it might. That's a big word to slip in there; I *might* get hit by a
bus tomorrow, I *might* win the lottery, I *might* grow a third eye. But
it's academic until I actually have those things happen.


> The key idea of relational model is that it is a generic way to model any
> kind of data.
> New data structures can be derived from the relations the data has.
> Ad hoc data models do not have this property.
> But of course are completely valid if only one application uses them.
>
> No reason you can't combine the two models, either. That's where the NoSQL
camp gets things wrong, often; they forget to remember that you can use an
RDMS for the things for which it's appropriate, and a NoSQL datastore for
the things for which IT is appropriate.

>
>  Relational models excel in reporting and warehousing. The rest... meh,
>> they're kinda slow on updates and set relationships. We're just so used to
>> the lack of speed that we think it's the norm.
>>
>
> The lack of speed comes out of users wish to have constraints on the data
> model.
> If you allow any kind of data into your relations then I don't see why
> relational model would be slow.
> The only reason would be bad implementations.
>
> Sure, of course constraints are part of the reason you use an RDMS in the
first place...


> Further relational model is a MODEL not implementation.
> So implementation can be anything as long as it produces the relational
> model.
>
> Sure. How many literal implementations have there been?


>  However, if you're doing a lot of updates or your set modifications are
>> common, you'll typically find the other models faster, and - depending on
>> your taste - easier.
>>
>
> I would like Thomas Mueller (developer of H2) to comment on this.
> Is the speed somehow related to the relational model?
> As far as I understand the speed suffers because of ACID requirements, not
> so much because of the relational model.
> And the poor usability comes from standardization (SQL).
>

The poor usability comes from how people typically design relational
schemas, and yes, ACID factors in. But if you're willing to explain away the
flaws by saying "the flaws are there only if you use the things that expose
the flaws," you're not saying much of anything at all.

For most people, because of how they use RDMSes, the databases are slower
than they could be. That's because they use various normalization levels.
That's because they need transactions that work. They use SQL.

That's the norm. So that's what defines how well an RDMS works for most
people. Certainly you can use alternatives that work better based on a
specific problem domain or an alternative solution from the hand of a master
DBA for the specific database you use... but that's *not* the norm.

>
> Standards are good because of compatibility but they rarely if ever produce
> the optimal user experience.


Sure... but when generalizing you pretty much have to go with the standards.
IMO.


>  Certainly transaction processing is easier with nonrelational.
>>
>
> Why?


Because the locking is (usually) more limited to only those records
affected, and relationship-checking is usually scoped better.


>  That's not to say that relational databases are *bad* -- they're just very
>> good at structured data, used for reporting, which is what they were used
>> for most often. Now they're seen as part of the background noise of data
>> storage, so everyone converts their problem into something the relational
>> database can use - as opposed to finding the solution that fits the
>> problem
>> well.
>>
>
> I think people use the relational model because projects who produce data
> that is never reused in other applications are rare.
> Also I don't see non-relational models as "bad" :-)


*nod* If your data is going to be consumed by a wide variety of applications
in a general form, a general-purpose data structure is good. Of course, REST
was designed to help normalize this... but REST wouldn't be grand for
generating grist for the reporting mill, either.


>  We've decided to let our tools dictate our solutions, instead of picking
>> our
>> solutions to fit our problems.
>>
>
> Tools no policy.
> Well I am all for this one.
>

?

>
>
-- 
Joseph B. Ottinger
http://enigmastation.com

Re: A generic question about JackRabbit

[Rami Ojares <ra...@gmail.com>]

On 26.3.2010 16:46, Joseph Ottinger wrote:
> It was E.F.Codd, for one thing, and not Ted. :)
>    

Ted was his nickname.
I was trying to be casual :-)
(But I see you realized this while I was writing this email)

> But... relational models work very well for *reporting* but not so much for
> other things. They're kinda slow, for example. Hierarchical and set-related
> models (document-based, ODBMS, and CODASYL dbs, respectively) are much, much
> faster for some operations, and slower for others.
>    

Yes the speed issue.
It is true that if your data is structured hierarchically and you access 
it in your application then you can create
the fastest application by storing it in the fashion that is good for 
the first application.
It just so happens that other applications come along that want to use 
the same data also (to avoid redundancy).
And for them the hierarchical model might prove to be a nightmare.

The key idea of relational model is that it is a generic way to model 
any kind of data.
New data structures can be derived from the relations the data has.
Ad hoc data models do not have this property.
But of course are completely valid if only one application uses them.

> Relational models excel in reporting and warehousing. The rest... meh,
> they're kinda slow on updates and set relationships. We're just so used to
> the lack of speed that we think it's the norm.
>    

The lack of speed comes out of users wish to have constraints on the 
data model.
If you allow any kind of data into your relations then I don't see why 
relational model would be slow.
The only reason would be bad implementations.

Further relational model is a MODEL not implementation.
So implementation can be anything as long as it produces the relational 
model.

> However, if you're doing a lot of updates or your set modifications are
> common, you'll typically find the other models faster, and - depending on
> your taste - easier.
>    

I would like Thomas Mueller (developer of H2) to comment on this.
Is the speed somehow related to the relational model?
As far as I understand the speed suffers because of ACID requirements, 
not so much because of the relational model.
And the poor usability comes from standardization (SQL).

Standards are good because of compatibility but they rarely if ever 
produce the optimal user experience.

> Certainly transaction processing is easier with nonrelational.
>    

Why?

> That's not to say that relational databases are *bad* -- they're just very
> good at structured data, used for reporting, which is what they were used
> for most often. Now they're seen as part of the background noise of data
> storage, so everyone converts their problem into something the relational
> database can use - as opposed to finding the solution that fits the problem
> well.
>    

I think people use the relational model because projects who produce 
data that is never reused in other applications are rare.
Also I don't see non-relational models as "bad" :-)

> We've decided to let our tools dictate our solutions, instead of picking our
> solutions to fit our problems.
>    

Tools no policy.
Well I am all for this one.

- rami

Re: A generic question about JackRabbit

[Joseph Ottinger <dr...@gmail.com>]

It was E.F.Codd, for one thing, and not Ted. :)

But... relational models work very well for *reporting* but not so much for
other things. They're kinda slow, for example. Hierarchical and set-related
models (document-based, ODBMS, and CODASYL dbs, respectively) are much, much
faster for some operations, and slower for others.

Relational models excel in reporting and warehousing. The rest... meh,
they're kinda slow on updates and set relationships. We're just so used to
the lack of speed that we think it's the norm.

However, if you're doing a lot of updates or your set modifications are
common, you'll typically find the other models faster, and - depending on
your taste - easier.

Certainly transaction processing is easier with nonrelational.

That's not to say that relational databases are *bad* -- they're just very
good at structured data, used for reporting, which is what they were used
for most often. Now they're seen as part of the background noise of data
storage, so everyone converts their problem into something the relational
database can use - as opposed to finding the solution that fits the problem
well.

We've decided to let our tools dictate our solutions, instead of picking our
solutions to fit our problems.

On Fri, Mar 26, 2010 at 10:34 AM, Rami Ojares <ra...@gmail.com> wrote:

> Hi,
>
> I only now stumbled onto JackRabbit project.
> And a basic question about it rose in me spontaneously.
>
> Early in the seventies Ted Codd published relational model as an
> improvement
> over the hierarchical model that had been used for data storage until then.
>
> So why do we still need a hierarchical model for data storage?
>
> Features mentioned like full text search, versioning, transactions,
> observation, and more
> can be just as well implemented with the relational model.
>
> And what does unstructured data mean?
> It sounds to me like random noise.
>
> - rami
>



-- 
Joseph B. Ottinger
http://enigmastation.com

Re: A generic question about JackRabbit

[Alexander Klimetschek <ak...@day.com>]

On Sat, Mar 27, 2010 at 00:16, Rami Ojares <ra...@gmail.com> wrote:
> But Alex said a few radical comments so I just can't refrain myself from
> commenting them a little.
>
>> I think normalization is often thought to be absolutely fundamental
>> for any data schema, because it is a central part of RDBMS and this is
>> basically the only thing that was taught the last 20-30 years (went
>> this route myself). But a major reason for normalization was simply
>> the space constraint, it's not fundamental at all.
>>
>
> Data normalization is concerned with issues of how the relational model
> should be used.
> If you don't have the data in 1st normal form then the relational operators
> and other tools
> that the relational model provides are unable to query the data.

Yes, and all I was saying that because of space constraints, a storage
model around normalization won. I didn't say that normalization is
generally bad, and I didn't question the obvious fact that
normalization is a central concept of an RDBMS.

>> The same story with the ACID constraints... banking accounts are not
>> the only software application nowadays ;-)
>>
>
> Wow, that came out of the bush and hit me right there in the back between my
> shoulder blades ;-)
>
> Basically what you are saying is that when you update data in your storage
> system (that could any data model)
> you don't really care if only part of the data you sent there got updated
> (Atomicity)
> You don't care that if you have set some rules for your data that these
> rules are respected. (Consistency)
> If many threads are accessing the data it does not concern you if they see
> each other's updates
> partially and modify each others data while their operations are underway.
> (Isolation)
> And certainly you don't give a toss about whether your data really stays in
> the storage once you have put it there. (Durability)
>
> The reason why acid properties are related to banking industry is that
> people demand correctness when banks deal with their money.

Yes, I didn't question that, I just noted that banking is a very
special case. And it comes at a cost that most banking applications
are blocked an entire night to be able to safely process transactions.
It's just that banking is the number one sample and teaching schema
for RDBMS, because it's very good at it. However, this affects how
people think about data storage, as if ACID is a given requirement for
everything. It isn't. I think this opens up quite a bit nowadays in
light of the Nosql movement. See the CAP theorem [1].

Most applications don't need all those strict constraints, eg. can
perfectly live with eventual consistency. The real world is not
perfect either.

All I am saying is that it's just necessary to detach from certain
principles in the RDBMS world if you want to see the big picture.
Whatever you chose in the end depends on the needs of your
application. And there are many applications that don't benefit from
the (strict) RDBMS model, rather they loose flexibility, performance,
etc.

[1] http://en.wikipedia.org/wiki/CAP_theorem

Regards,
Alex

-- 
Alexander Klimetschek
alexander.klimetschek@day.com

Re: A generic question about JackRabbit

[Rami Ojares <ra...@gmail.com>]

Thank you very much for all your answers.
I think I got the overall flavour of how people see the data management 
issues on this list.
But Alex said a few radical comments so I just can't refrain myself from 
commenting them a little.

> I think normalization is often thought to be absolutely fundamental
> for any data schema, because it is a central part of RDBMS and this is
> basically the only thing that was taught the last 20-30 years (went
> this route myself). But a major reason for normalization was simply
> the space constraint, it's not fundamental at all.
>    

Data normalization is concerned with issues of how the relational model 
should be used.
If you don't have the data in 1st normal form then the relational 
operators and other tools
that the relational model provides are unable to query the data.
(Putting a comma separated list as a value of an attribute makes it very 
hard to formulate a query
"Give me rows where one of the list's components is smaller than X")

2NF gives guidance on the meaning of a canditate key in a relation
And so forth.

The point being that normalization could never have been about saving 
space because it
is concerned with how you should use the relational model so you don't 
get redundant
data in your model that is "out-of-sync" with other data in your system.

In fact it is not concerned at all with the issue of how much disk space 
is used because that
is left completely to the IMPLEMENTATION of the relational model to decide.

Things like materialized views have nothing to do with the relational 
model. They are just an
example that sql is used also to guide the implementation of how data is 
stored.

I can easily imagine a RDBMS that implements all views as materialized 
into let's say RAM, disk or
a network of 10 million servers. But from the user's (application's) 
point of view this is of no concern.

But what does concern the user is: Do I get the correct value when I 
query it? Can I update this piece of data
without making the whole system inconsistent.

I think people rarely make the difference between data MODEL and it's 
various IMPLEMENTATION strategies.
Most of the time people are only concerned with the implementation and 
the data model is just a consequence
of how things were implemented. This is how I think the situation is 
with JackRabbit.

But before you start crucifying me let me just say that I don't see 
anything wrong with this approach.
It is just a bit too ad hoc for my own state of mind.

> The same story with the ACID constraints... banking accounts are not
> the only software application nowadays ;-)
>    

Wow, that came out of the bush and hit me right there in the back 
between my shoulder blades ;-)

Basically what you are saying is that when you update data in your 
storage system (that could any data model)
you don't really care if only part of the data you sent there got 
updated (Atomicity)
You don't care that if you have set some rules for your data that these 
rules are respected. (Consistency)
If many threads are accessing the data it does not concern you if they 
see each other's updates
partially and modify each others data while their operations are 
underway. (Isolation)
And certainly you don't give a toss about whether your data really stays 
in the storage once you have put it there. (Durability)

The reason why acid properties are related to banking industry is that 
people demand correctness when banks deal with their money.
Otherwise they don't give their money to the banks.
I have a feeling that other kind of software could also benefit from 
correctness.
But somehow the sql based databases have done a poor job delivering on 
this promise.

According to me the reason is twofold.
1) Poor sql database implementations
2) Users who have been using these implementations poorly

Once more ... before you start calling me a pedantic and overscrupulous 
let me just say that you can let go
of many of these checks if you know how your data is updated and you 
have full control of how the storage is used.

But in the generic case I don't think so.

Summa summarum: I could take JackRabbit and use it as my implementation 
of the relational model or let's say implementation
of an XML database (yes there used to be that kind of "hot stuff" on the 
market a few years back :).
The data model of JackRabbit is the JCR api (if you can call a java api 
a data model).
And the project's deliverables are it's implementation(s).

- rami

Re: A generic question about JackRabbit

[ChadDavis <ch...@gmail.com>]

> I think normalization is often thought to be absolutely fundamental
> for any data schema, because it is a central part of RDBMS and this is
> basically the only thing that was taught the last 20-30 years (went
> this route myself). But a major reason for normalization was simply
> the space constraint, it's not fundamental at all.

Interesting discussion.  But I thought normalization was about
preventing data corruption.

Re: A generic question about JackRabbit

[Alexander Klimetschek <ak...@day.com>]

Going very Nosql-ly here ;-)

On Fri, Mar 26, 2010 at 19:23, ChadDavis <ch...@gmail.com> wrote:
> Perhaps, OT, but wouldn't we still say that if data integrity is very
> important, then we still would prefer a well normalized RDBMS to a
> object oriented persistence mechanism?

Yes, but that depends on the application. There is nothing wrong by
eg. reusing a certain string (eg. "us", "uk", "switzerland") instead
of having it as a foreign key to some country table (*). This
effectively moves the ownership of the data towards the application
(developer) and away from a database maintainer / dba, which is a good
thing for fast and flexible application development as experience
shows ;-)

(*) The implementation can still intern strings to save space, and
some parts of jackrabbit do that already.

I think normalization is often thought to be absolutely fundamental
for any data schema, because it is a central part of RDBMS and this is
basically the only thing that was taught the last 20-30 years (went
this route myself). But a major reason for normalization was simply
the space constraint, it's not fundamental at all.

The same story with the ACID constraints... banking accounts are not
the only software application nowadays ;-)

Regards,
Alex

-- 
Alexander Klimetschek
alexander.klimetschek@day.com

Re: A generic question about JackRabbit

[ChadDavis <ch...@gmail.com>]

On Fri, Mar 26, 2010 at 12:21 PM, Alexander Klimetschek
<ak...@day.com> wrote:
> On Fri, Mar 26, 2010 at 15:34, Rami Ojares <ra...@gmail.com> wrote:
>> Early in the seventies Ted Codd published relational model as an improvement
>> over the hierarchical model that had been used for data storage until then.
>>
>> So why do we still need a hierarchical model for data storage?
>
> In addition to what others have said already: a major requirement for
> going from files to RDBMS in the 70s was the big cost of storage back
> then. Hence normalization was considered important, and the relational
> model made the best use of the disk space available.
>
> But nowadays this is different, so "denormalization" is ok. It makes
> reads faster. And it is absolutely needed for large databases that
> span multiple servers.

Perhaps, OT, but wouldn't we still say that if data integrity is very
important, then we still would prefer a well normalized RDBMS to a
object oriented persistence mechanism?

>
> Regards,
> Alex
>
> --
> Alexander Klimetschek
> alexander.klimetschek@day.com
>

Re: A generic question about JackRabbit

[Alexander Klimetschek <ak...@day.com>]

On Fri, Mar 26, 2010 at 15:34, Rami Ojares <ra...@gmail.com> wrote:
> Early in the seventies Ted Codd published relational model as an improvement
> over the hierarchical model that had been used for data storage until then.
>
> So why do we still need a hierarchical model for data storage?

In addition to what others have said already: a major requirement for
going from files to RDBMS in the 70s was the big cost of storage back
then. Hence normalization was considered important, and the relational
model made the best use of the disk space available.

But nowadays this is different, so "denormalization" is ok. It makes
reads faster. And it is absolutely needed for large databases that
span multiple servers.

Regards,
Alex

-- 
Alexander Klimetschek
alexander.klimetschek@day.com