You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@polygene.apache.org by Niclas Hedhman <ni...@hedhman.org> on 2017/05/06 08:25:28 UTC
ORM EntityStore
Gang,
I have gotten overly angry with Hibernate on $dayjob, and decided to take a
look at what a ORM-ish implementation in Polygene would look like. And it
was easier than I expected, so...
Pretty simple,
0. One "types table" that keep tracks of all types. Content of this is
probably fully cached.
1. One type table for each MixinType (except HasIdentity)
2. One "base entity" table with the Identity, lastmodifed and so on stuff.
3. Use a different (internal) identity for the "value", which makes it easy
to fully support non-destructive updates.
4. Property->SQL on basic types, and Serialization kick in on
ValueComposites.
5. Association->SQL is simply a VARCHAR
6. Named/ManyAssociation->SQL needs an intermediate table per assoc (naming
is trouble some)
7. get() becomes a SELECT with one join per mixin type
8. new/update is bunch of INSERT (if non-destructive is used) SQL
statement, one for each mixin type plus an UPDATE for "base entity".
9. Named/ManyAssociations will also be an INSERT with many values.
10. JOOQ is awfully helpful in making this, and I am trying to modularize
it so that people can customize it if needed, at Composite Methods level,
either by Concerns or overriding Mixin methods.
11. IF destructive mode is wanted, I suggest that DELETE statements are
issued to remove old stuff, which makes both modes semantically identical,
and not a big deal if something is accidentally not removed.
And that is pretty much it. Fairly straight forward, and a far cry from the
complexity of Hibernate, guessing what it does and why.
NOTE, this is only for "Java Model drives SQL model", but in an
enterprise-friendly way. I hope that this can be a big "adoption driver".
WDYAT?
Cheers
--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Re: ORM EntityStore
Posted by Niclas Hedhman <ni...@hedhman.org>.
I assume that you are referring to (from sample project)
public static Person findPersonNamed(String firstName, String lastName)
{
Operation findByFirstNameAndLastName =
PersonFinder.firstName().eq(firstName)
.and(PersonFinder.lastName().eq(lastName));
return PersonFinder.findOne(findByFirstNameAndLastName);
}
which on paper is somewhat similar.
However, there seems to be rather large hurdles;
1. They start with an app model in XML
2. They generate source code from that XML in build time, and quite a bit
of it [1].
3. The PersonFinder has "Polygene naming", and it is a class that is
regenerated on every build.
4. Person is a one-time generated class, extending a every time generated
AbstractPerson that follows "getter/setter naming".
5. Static methods... why?
6. It seems that transactions are wrapped in a single method, and that
doesn't fit the UnitOfWork workflow [2]
7. A lot of the generated stuff is about providing a query interface.
The codebase is non-trivial, I give them that. And it might be a lot better
than JPA, I mean it is hard not to be worse.
Now, using it out of the box is probably impossible, and I don't think it
is possible to make this generic enough to be able to leverage Polygene
models, so the alternative would be to fork it and massage it into what we
would need. More effort than _I_ care to go for...
HTH
Niclas
[1]
https://goldmansachs.github.io/reladomo-kata/main-kata-presentation/ReladomoKata.xhtml#(11)
https://goldmansachs.github.io/reladomo-kata/main-kata-presentation/ReladomoKata.xhtml#(12)
[2]
https://goldmansachs.github.io/reladomo-kata/main-kata-presentation/ReladomoKata.xhtml#(65)
On Sat, Jul 22, 2017 at 4:31 PM, Jiri Jetmar <ju...@gmail.com>
wrote:
> Hi guys,
>
> there is an interesting project open sourced by Goldman Sachs:
>
> https://github.com/goldmansachs/reladomo
>
> It is a basically a Java ORM with some pretty unique features and the Query
> DSL is in some ways similar to what we are using in Apache Polygene :
>
> Person.createPerson("Taro", "Tanaka", "JPN");
> Person person = Person.findPersonNamed("Taro", "Tanaka");
>
> Evtl. this can be used as ORM in Apache Polygene ? Looks promising, so lets
> see..
>
> Cheers,
> Jiri
>
>
>
> 2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
>
> > Gang,
> >
> > I have gotten overly angry with Hibernate on $dayjob, and decided to
> take a
> > look at what a ORM-ish implementation in Polygene would look like. And it
> > was easier than I expected, so...
> >
> > Pretty simple,
> >
> > 0. One "types table" that keep tracks of all types. Content of this is
> > probably fully cached.
> >
> > 1. One type table for each MixinType (except HasIdentity)
> >
> > 2. One "base entity" table with the Identity, lastmodifed and so on
> stuff.
> >
> > 3. Use a different (internal) identity for the "value", which makes it
> easy
> > to fully support non-destructive updates.
> >
> > 4. Property->SQL on basic types, and Serialization kick in on
> > ValueComposites.
> >
> > 5. Association->SQL is simply a VARCHAR
> >
> > 6. Named/ManyAssociation->SQL needs an intermediate table per assoc
> (naming
> > is trouble some)
> >
> > 7. get() becomes a SELECT with one join per mixin type
> >
> > 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> > statement, one for each mixin type plus an UPDATE for "base entity".
> >
> > 9. Named/ManyAssociations will also be an INSERT with many values.
> >
> > 10. JOOQ is awfully helpful in making this, and I am trying to modularize
> > it so that people can customize it if needed, at Composite Methods level,
> > either by Concerns or overriding Mixin methods.
> >
> > 11. IF destructive mode is wanted, I suggest that DELETE statements are
> > issued to remove old stuff, which makes both modes semantically
> identical,
> > and not a big deal if something is accidentally not removed.
> >
> > And that is pretty much it. Fairly straight forward, and a far cry from
> the
> > complexity of Hibernate, guessing what it does and why.
> >
> > NOTE, this is only for "Java Model drives SQL model", but in an
> > enterprise-friendly way. I hope that this can be a big "adoption driver".
> >
> > WDYAT?
> >
> >
> > Cheers
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
> >
>
--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Re: ORM EntityStore
Posted by Jiri Jetmar <ju...@gmail.com>.
Hi guys,
there is an interesting project open sourced by Goldman Sachs:
https://github.com/goldmansachs/reladomo
It is a basically a Java ORM with some pretty unique features and the Query
DSL is in some ways similar to what we are using in Apache Polygene :
Person.createPerson("Taro", "Tanaka", "JPN");
Person person = Person.findPersonNamed("Taro", "Tanaka");
Evtl. this can be used as ORM in Apache Polygene ? Looks promising, so lets
see..
Cheers,
Jiri
2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
> Gang,
>
> I have gotten overly angry with Hibernate on $dayjob, and decided to take a
> look at what a ORM-ish implementation in Polygene would look like. And it
> was easier than I expected, so...
>
> Pretty simple,
>
> 0. One "types table" that keep tracks of all types. Content of this is
> probably fully cached.
>
> 1. One type table for each MixinType (except HasIdentity)
>
> 2. One "base entity" table with the Identity, lastmodifed and so on stuff.
>
> 3. Use a different (internal) identity for the "value", which makes it easy
> to fully support non-destructive updates.
>
> 4. Property->SQL on basic types, and Serialization kick in on
> ValueComposites.
>
> 5. Association->SQL is simply a VARCHAR
>
> 6. Named/ManyAssociation->SQL needs an intermediate table per assoc (naming
> is trouble some)
>
> 7. get() becomes a SELECT with one join per mixin type
>
> 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> statement, one for each mixin type plus an UPDATE for "base entity".
>
> 9. Named/ManyAssociations will also be an INSERT with many values.
>
> 10. JOOQ is awfully helpful in making this, and I am trying to modularize
> it so that people can customize it if needed, at Composite Methods level,
> either by Concerns or overriding Mixin methods.
>
> 11. IF destructive mode is wanted, I suggest that DELETE statements are
> issued to remove old stuff, which makes both modes semantically identical,
> and not a big deal if something is accidentally not removed.
>
> And that is pretty much it. Fairly straight forward, and a far cry from the
> complexity of Hibernate, guessing what it does and why.
>
> NOTE, this is only for "Java Model drives SQL model", but in an
> enterprise-friendly way. I hope that this can be a big "adoption driver".
>
> WDYAT?
>
>
> Cheers
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
>
Re: ORM EntityStore
Posted by Niclas Hedhman <ni...@hedhman.org>.
Thanks for the views, and Slick seems to be (correct me if I am wrong) the
equivalent of JOOQ in Java. I would even argue that Slick is more "masking
it with another custom language" than JOOQ is trying to do;
Result<Record> baseEntityResult = dsl
.select()
.from( entitiesTable )
.where( identityColumn.eq( reference.toURI() ) )
.fetch();
BUT, the interesting point is whether the best way forward is to see if it
is possible to make JOOQ handle Polygene entities instead of Java Beans.
JOOQ operates in two "modes", either the lower level (which I intended to
use) where the programmer needs to ensure that everything is correct, or in
a higher/generated mode, where type-safety is achieved with generation of
code from a model "somehow" (I haven't studied the details, so know very
little about it). This is likely a great undertaking, and probably more
effort than I can cope with in the short run.
Another "option" is to simply expose the "Result-to-Object" mapping phase,
so people could take any arbitrary JOOQ Result<Record> and push that into
Polygene entities. That is very easy to do, BUT one risk that data is
missing in the Result<Record> and hence corrupts the data over time. This
could be leveraged by map into Values instead, so they can't be modified
and can't be saved. That would have the interesting side-effect that it is
incredibly useful for a View Model in a CQRS set up. Also, if that Value
has ManyAssociation, then getting those (within a UnitOfWork) would neatly
retrieve those entities (albeit one-by-one). Not sure it would be possible
to have a ManyAssocation returned as a query result, but maybe it is also
an option.
What I don't like is to only provide "manual" handling, since that will
lead to data corruption as people forget to include fields and handle
mapping tables (Named/ManyAssociation) correctly. So I can contemplate the
second option above, into Values. That seems rather safe, yet opens the
query engine (JOOQ) to full potential, I think.
Cheers
Niclas
On Sun, May 7, 2017 at 2:26 AM, Sandro Martini <sa...@gmail.com>
wrote:
> Hi Niclas,
>
> I was tired of Hibernate too :-) ...
> I think that your proposal looks very interesting, an approach more
> related to code to me looks better (and more Java oriented) than many
> current products ... but an hard point could be to find a way to let
> developers write SQL code if/when needed, instead of masking it with
> another (custom) language like HQL or others similar.
>
> Just to have some idea, there is a great product from the Scala stack:
> [Slick](http://slick.lightbend.com/) that has a similar approach,
> maybe some idea can arise.
>
> Hope this helps.
>
> Bye
>
> 2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
> > Gang,
> >
> > I have gotten overly angry with Hibernate on $dayjob, and decided to
> take a
> > look at what a ORM-ish implementation in Polygene would look like. And it
> > was easier than I expected, so...
> >
> > Pretty simple,
> >
> > 0. One "types table" that keep tracks of all types. Content of this is
> > probably fully cached.
> >
> > 1. One type table for each MixinType (except HasIdentity)
> >
> > 2. One "base entity" table with the Identity, lastmodifed and so on
> stuff.
> >
> > 3. Use a different (internal) identity for the "value", which makes it
> easy
> > to fully support non-destructive updates.
> >
> > 4. Property->SQL on basic types, and Serialization kick in on
> > ValueComposites.
> >
> > 5. Association->SQL is simply a VARCHAR
> >
> > 6. Named/ManyAssociation->SQL needs an intermediate table per assoc
> (naming
> > is trouble some)
> >
> > 7. get() becomes a SELECT with one join per mixin type
> >
> > 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> > statement, one for each mixin type plus an UPDATE for "base entity".
> >
> > 9. Named/ManyAssociations will also be an INSERT with many values.
> >
> > 10. JOOQ is awfully helpful in making this, and I am trying to modularize
> > it so that people can customize it if needed, at Composite Methods level,
> > either by Concerns or overriding Mixin methods.
> >
> > 11. IF destructive mode is wanted, I suggest that DELETE statements are
> > issued to remove old stuff, which makes both modes semantically
> identical,
> > and not a big deal if something is accidentally not removed.
> >
> > And that is pretty much it. Fairly straight forward, and a far cry from
> the
> > complexity of Hibernate, guessing what it does and why.
> >
> > NOTE, this is only for "Java Model drives SQL model", but in an
> > enterprise-friendly way. I hope that this can be a big "adoption driver".
> >
> > WDYAT?
> >
> >
> > Cheers
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
>
--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Re: ORM EntityStore
Posted by Sandro Martini <sa...@gmail.com>.
Hi Niclas,
I was tired of Hibernate too :-) ...
I think that your proposal looks very interesting, an approach more
related to code to me looks better (and more Java oriented) than many
current products ... but an hard point could be to find a way to let
developers write SQL code if/when needed, instead of masking it with
another (custom) language like HQL or others similar.
Just to have some idea, there is a great product from the Scala stack:
[Slick](http://slick.lightbend.com/) that has a similar approach,
maybe some idea can arise.
Hope this helps.
Bye
2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
> Gang,
>
> I have gotten overly angry with Hibernate on $dayjob, and decided to take a
> look at what a ORM-ish implementation in Polygene would look like. And it
> was easier than I expected, so...
>
> Pretty simple,
>
> 0. One "types table" that keep tracks of all types. Content of this is
> probably fully cached.
>
> 1. One type table for each MixinType (except HasIdentity)
>
> 2. One "base entity" table with the Identity, lastmodifed and so on stuff.
>
> 3. Use a different (internal) identity for the "value", which makes it easy
> to fully support non-destructive updates.
>
> 4. Property->SQL on basic types, and Serialization kick in on
> ValueComposites.
>
> 5. Association->SQL is simply a VARCHAR
>
> 6. Named/ManyAssociation->SQL needs an intermediate table per assoc (naming
> is trouble some)
>
> 7. get() becomes a SELECT with one join per mixin type
>
> 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> statement, one for each mixin type plus an UPDATE for "base entity".
>
> 9. Named/ManyAssociations will also be an INSERT with many values.
>
> 10. JOOQ is awfully helpful in making this, and I am trying to modularize
> it so that people can customize it if needed, at Composite Methods level,
> either by Concerns or overriding Mixin methods.
>
> 11. IF destructive mode is wanted, I suggest that DELETE statements are
> issued to remove old stuff, which makes both modes semantically identical,
> and not a big deal if something is accidentally not removed.
>
> And that is pretty much it. Fairly straight forward, and a far cry from the
> complexity of Hibernate, guessing what it does and why.
>
> NOTE, this is only for "Java Model drives SQL model", but in an
> enterprise-friendly way. I hope that this can be a big "adoption driver".
>
> WDYAT?
>
>
> Cheers
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
Re: ORM EntityStore
Posted by Niclas Hedhman <ni...@hedhman.org>.
Stan,
yes it has struck me, and I think the answer is that indexing would then
become an issue of figuring what indexes to create in SQL. Yet to be
explored, but probably possible.
On Mon, May 8, 2017 at 9:23 PM, Stanislav Muhametsin <
stanislav.muhametsin@zest.mail.kapsi.fi> wrote:
> I agree with JJ here, but SQL-leads-Polygene approach is extremely hard to
> do, considering that creating SQL indexing store to fully support all stuff
> that Polygene query engine does is quite demanding already.
> But I'm not saying it is impossible - hardly anything is. :)
> And it's good that Niclas is starting with simple first step - that way,
> one might learn things that will come handy when making more complex,
> further steps.
>
> Without knowing much about current code structure and big entity-related
> principles of Polygene, I think one big aspect to really try and make work
> is synergy between SQL ES and Indexing.
> Currently those are acting as two completely different databases with
> different schemas etc.
> But what Niclas talked about in the first e-mail, it sounds like ORM
> approach already covers a big chunk of what current SQL indexing does.
> So maybe those two can be e.g. using common schema logic (and thus share
> common code related to RDB initialization etc) at some point?
>
>
> On 8.5.2017 12:47, Jiri Jetmar wrote:
>
>> Ok, see your point. So what you are saying now is that the first step
>> would
>> be a Polygene model that would lead/specify the SQL model in a more
>> SQL-ish
>> way.
>>
>> That means a more natural/classical SQL schema will be generated - unlike
>> the key/value approach that is implemented now.
>>
>> I personally would still prefer an SQL-leads-Polygene model, simply
>> because
>> generations of developers
>> 'trust' in a DB/SQL schema. Further organizational structures within
>> companies rely on strict data ownership based on relational schemas. There
>> are also tons of tools that support relational schemas - from the
>> generation of documentation.. till migration of complex schemas.
>>
>> I know that Hibernate is exactly doing the mapping from the OO-World to
>> the
>> R-World, offering a large number of features like schema generations,
>> etc..
>> And we all know how complex and unpredictable this tool is. There are also
>> people that already give up and are simply arguing that when you are in
>> enterprise and you have to deal with relational repositories and you have
>> simply pay the price and deal with the complexity (complexity ==
>> 'business
>> value' and any other competitor has to deal with the same complexity as
>> well.)
>>
>> In general, this ORM thing has eventually too many points that must be
>> considered, so that a 'perfect' and universal solution does not exist...
>>
>> Cheers,
>> jj
>>
>>
>>
>>
>>
>> 2017-05-08 10:28 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
>>
>> What I mean is that this starts with a Polygene model and it will create
>>> SQL tables.
>>>
>>> Why not the other way around?
>>> Well, you will immediately descend into the rabbit hole where Hibernate
>>> is.
>>> With incomprehensibly complex mappings of tables to arbitrary sections of
>>> code, simply not robust enough to ensure solid design. But the SQL model
>>> that we end up with is neat and in the spirit of SQL, unlike the current
>>> KeyValue use of SQL in the entitystore-sql.
>>>
>>> That said, adding some feature to the above (such as table/column name
>>> mappings) would allow creation of tool (interactive or declarative) to
>>> take
>>> a DB schema and generate the Polygene model from it, to the extent that
>>> it
>>> is possible. However, there will be constructs not achievable to fit the
>>> Polygene model, and it is likely that any DB would need additional tables
>>> and columns to accommodate a Polygene model, for instance things like
>>> entity versioning, lastmodifed, and others.
>>>
>>> Furthermore, it might be reasonable to capture additional requirements
>>> via
>>> JOOQ, and increase the support of consuming/feeding JOOQ, either by
>>> getting
>>> JOOQ to understand Polygene models, or the other way around. Getting JOOQ
>>> to understand Polygene seems possible, albeit not trivial, so I am not
>>> sure
>>> of the ambition level here. What's important to realize is that it is
>>> outside the domain of "Entity Stores" and UnitOfWork managed persistence.
>>> (I am already feeling the pain, that JOOQ seems to require the
>>> transaction
>>> to be within a single JOOQ method call, when we need the prepare() and
>>> commit() in separate calls, initiated from UnitOfWork)
>>>
>>> So, loads of work to do, but I want to focus on one step at a time, a
>>> step
>>> that is not endlessly complex to satisfy.
>>>
>>>
>>> Cheers
>>> Niclas
>>>
>>>
>>>
>>> On Mon, May 8, 2017 at 3:59 PM, Jiri Jetmar <ju...@gmail.com>
>>> wrote:
>>>
>>> Hi Niclas,
>>>>
>>>> thank you for taking over the ORM topic. We definitely require a
>>>> 'generally' accepted approach how to deal with persistence !
>>>>
>>>> 'NOTE, this is only for "Java Model drives SQL model", but in an
>>>> enterprise-friendly way.'
>>>>
>>>> I;m a bit confused with the above expression. From my understanding we
>>>>
>>> need
>>>
>>>> the opposite direction - a relational model drivers the application
>>>>
>>> related
>>>
>>>> model. And as we are using Java, we end up with a O-R-M. Or do
>>>> I misunderstood something ?
>>>>
>>>> Thank you.
>>>>
>>>> Cheers,
>>>> jj
>>>>
>>>>
>>>>
>>>> 2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
>>>>
>>>> Gang,
>>>>>
>>>>> I have gotten overly angry with Hibernate on $dayjob, and decided to
>>>>>
>>>> take a
>>>>
>>>>> look at what a ORM-ish implementation in Polygene would look like. And
>>>>>
>>>> it
>>>
>>>> was easier than I expected, so...
>>>>>
>>>>> Pretty simple,
>>>>>
>>>>> 0. One "types table" that keep tracks of all types. Content of this is
>>>>> probably fully cached.
>>>>>
>>>>> 1. One type table for each MixinType (except HasIdentity)
>>>>>
>>>>> 2. One "base entity" table with the Identity, lastmodifed and so on
>>>>>
>>>> stuff.
>>>>
>>>>> 3. Use a different (internal) identity for the "value", which makes it
>>>>>
>>>> easy
>>>>
>>>>> to fully support non-destructive updates.
>>>>>
>>>>> 4. Property->SQL on basic types, and Serialization kick in on
>>>>> ValueComposites.
>>>>>
>>>>> 5. Association->SQL is simply a VARCHAR
>>>>>
>>>>> 6. Named/ManyAssociation->SQL needs an intermediate table per assoc
>>>>>
>>>> (naming
>>>>
>>>>> is trouble some)
>>>>>
>>>>> 7. get() becomes a SELECT with one join per mixin type
>>>>>
>>>>> 8. new/update is bunch of INSERT (if non-destructive is used) SQL
>>>>> statement, one for each mixin type plus an UPDATE for "base entity".
>>>>>
>>>>> 9. Named/ManyAssociations will also be an INSERT with many values.
>>>>>
>>>>> 10. JOOQ is awfully helpful in making this, and I am trying to
>>>>>
>>>> modularize
>>>
>>>> it so that people can customize it if needed, at Composite Methods
>>>>>
>>>> level,
>>>
>>>> either by Concerns or overriding Mixin methods.
>>>>>
>>>>> 11. IF destructive mode is wanted, I suggest that DELETE statements are
>>>>> issued to remove old stuff, which makes both modes semantically
>>>>>
>>>> identical,
>>>>
>>>>> and not a big deal if something is accidentally not removed.
>>>>>
>>>>> And that is pretty much it. Fairly straight forward, and a far cry from
>>>>>
>>>> the
>>>>
>>>>> complexity of Hibernate, guessing what it does and why.
>>>>>
>>>>> NOTE, this is only for "Java Model drives SQL model", but in an
>>>>> enterprise-friendly way. I hope that this can be a big "adoption
>>>>>
>>>> driver".
>>>
>>>> WDYAT?
>>>>>
>>>>>
>>>>> Cheers
>>>>> --
>>>>> Niclas Hedhman, Software Developer
>>>>> http://polygene.apache.org - New Energy for Java
>>>>>
>>>>>
>>>
>>> --
>>> Niclas Hedhman, Software Developer
>>> http://polygene.apache.org - New Energy for Java
>>>
>>>
>
--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Re: ORM EntityStore
Posted by Stanislav Muhametsin <st...@zest.mail.kapsi.fi>.
I agree with JJ here, but SQL-leads-Polygene approach is extremely hard
to do, considering that creating SQL indexing store to fully support all
stuff that Polygene query engine does is quite demanding already.
But I'm not saying it is impossible - hardly anything is. :)
And it's good that Niclas is starting with simple first step - that way,
one might learn things that will come handy when making more complex,
further steps.
Without knowing much about current code structure and big entity-related
principles of Polygene, I think one big aspect to really try and make
work is synergy between SQL ES and Indexing.
Currently those are acting as two completely different databases with
different schemas etc.
But what Niclas talked about in the first e-mail, it sounds like ORM
approach already covers a big chunk of what current SQL indexing does.
So maybe those two can be e.g. using common schema logic (and thus share
common code related to RDB initialization etc) at some point?
On 8.5.2017 12:47, Jiri Jetmar wrote:
> Ok, see your point. So what you are saying now is that the first step would
> be a Polygene model that would lead/specify the SQL model in a more SQL-ish
> way.
>
> That means a more natural/classical SQL schema will be generated - unlike
> the key/value approach that is implemented now.
>
> I personally would still prefer an SQL-leads-Polygene model, simply because
> generations of developers
> 'trust' in a DB/SQL schema. Further organizational structures within
> companies rely on strict data ownership based on relational schemas. There
> are also tons of tools that support relational schemas - from the
> generation of documentation.. till migration of complex schemas.
>
> I know that Hibernate is exactly doing the mapping from the OO-World to the
> R-World, offering a large number of features like schema generations, etc..
> And we all know how complex and unpredictable this tool is. There are also
> people that already give up and are simply arguing that when you are in
> enterprise and you have to deal with relational repositories and you have
> simply pay the price and deal with the complexity (complexity == 'business
> value' and any other competitor has to deal with the same complexity as
> well.)
>
> In general, this ORM thing has eventually too many points that must be
> considered, so that a 'perfect' and universal solution does not exist...
>
> Cheers,
> jj
>
>
>
>
>
> 2017-05-08 10:28 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
>
>> What I mean is that this starts with a Polygene model and it will create
>> SQL tables.
>>
>> Why not the other way around?
>> Well, you will immediately descend into the rabbit hole where Hibernate is.
>> With incomprehensibly complex mappings of tables to arbitrary sections of
>> code, simply not robust enough to ensure solid design. But the SQL model
>> that we end up with is neat and in the spirit of SQL, unlike the current
>> KeyValue use of SQL in the entitystore-sql.
>>
>> That said, adding some feature to the above (such as table/column name
>> mappings) would allow creation of tool (interactive or declarative) to take
>> a DB schema and generate the Polygene model from it, to the extent that it
>> is possible. However, there will be constructs not achievable to fit the
>> Polygene model, and it is likely that any DB would need additional tables
>> and columns to accommodate a Polygene model, for instance things like
>> entity versioning, lastmodifed, and others.
>>
>> Furthermore, it might be reasonable to capture additional requirements via
>> JOOQ, and increase the support of consuming/feeding JOOQ, either by getting
>> JOOQ to understand Polygene models, or the other way around. Getting JOOQ
>> to understand Polygene seems possible, albeit not trivial, so I am not sure
>> of the ambition level here. What's important to realize is that it is
>> outside the domain of "Entity Stores" and UnitOfWork managed persistence.
>> (I am already feeling the pain, that JOOQ seems to require the transaction
>> to be within a single JOOQ method call, when we need the prepare() and
>> commit() in separate calls, initiated from UnitOfWork)
>>
>> So, loads of work to do, but I want to focus on one step at a time, a step
>> that is not endlessly complex to satisfy.
>>
>>
>> Cheers
>> Niclas
>>
>>
>>
>> On Mon, May 8, 2017 at 3:59 PM, Jiri Jetmar <ju...@gmail.com>
>> wrote:
>>
>>> Hi Niclas,
>>>
>>> thank you for taking over the ORM topic. We definitely require a
>>> 'generally' accepted approach how to deal with persistence !
>>>
>>> 'NOTE, this is only for "Java Model drives SQL model", but in an
>>> enterprise-friendly way.'
>>>
>>> I;m a bit confused with the above expression. From my understanding we
>> need
>>> the opposite direction - a relational model drivers the application
>> related
>>> model. And as we are using Java, we end up with a O-R-M. Or do
>>> I misunderstood something ?
>>>
>>> Thank you.
>>>
>>> Cheers,
>>> jj
>>>
>>>
>>>
>>> 2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
>>>
>>>> Gang,
>>>>
>>>> I have gotten overly angry with Hibernate on $dayjob, and decided to
>>> take a
>>>> look at what a ORM-ish implementation in Polygene would look like. And
>> it
>>>> was easier than I expected, so...
>>>>
>>>> Pretty simple,
>>>>
>>>> 0. One "types table" that keep tracks of all types. Content of this is
>>>> probably fully cached.
>>>>
>>>> 1. One type table for each MixinType (except HasIdentity)
>>>>
>>>> 2. One "base entity" table with the Identity, lastmodifed and so on
>>> stuff.
>>>> 3. Use a different (internal) identity for the "value", which makes it
>>> easy
>>>> to fully support non-destructive updates.
>>>>
>>>> 4. Property->SQL on basic types, and Serialization kick in on
>>>> ValueComposites.
>>>>
>>>> 5. Association->SQL is simply a VARCHAR
>>>>
>>>> 6. Named/ManyAssociation->SQL needs an intermediate table per assoc
>>> (naming
>>>> is trouble some)
>>>>
>>>> 7. get() becomes a SELECT with one join per mixin type
>>>>
>>>> 8. new/update is bunch of INSERT (if non-destructive is used) SQL
>>>> statement, one for each mixin type plus an UPDATE for "base entity".
>>>>
>>>> 9. Named/ManyAssociations will also be an INSERT with many values.
>>>>
>>>> 10. JOOQ is awfully helpful in making this, and I am trying to
>> modularize
>>>> it so that people can customize it if needed, at Composite Methods
>> level,
>>>> either by Concerns or overriding Mixin methods.
>>>>
>>>> 11. IF destructive mode is wanted, I suggest that DELETE statements are
>>>> issued to remove old stuff, which makes both modes semantically
>>> identical,
>>>> and not a big deal if something is accidentally not removed.
>>>>
>>>> And that is pretty much it. Fairly straight forward, and a far cry from
>>> the
>>>> complexity of Hibernate, guessing what it does and why.
>>>>
>>>> NOTE, this is only for "Java Model drives SQL model", but in an
>>>> enterprise-friendly way. I hope that this can be a big "adoption
>> driver".
>>>> WDYAT?
>>>>
>>>>
>>>> Cheers
>>>> --
>>>> Niclas Hedhman, Software Developer
>>>> http://polygene.apache.org - New Energy for Java
>>>>
>>
>>
>> --
>> Niclas Hedhman, Software Developer
>> http://polygene.apache.org - New Energy for Java
>>
Re: ORM EntityStore
Posted by Jiri Jetmar <ju...@gmail.com>.
Ok, see your point. So what you are saying now is that the first step would
be a Polygene model that would lead/specify the SQL model in a more SQL-ish
way.
That means a more natural/classical SQL schema will be generated - unlike
the key/value approach that is implemented now.
I personally would still prefer an SQL-leads-Polygene model, simply because
generations of developers
'trust' in a DB/SQL schema. Further organizational structures within
companies rely on strict data ownership based on relational schemas. There
are also tons of tools that support relational schemas - from the
generation of documentation.. till migration of complex schemas.
I know that Hibernate is exactly doing the mapping from the OO-World to the
R-World, offering a large number of features like schema generations, etc..
And we all know how complex and unpredictable this tool is. There are also
people that already give up and are simply arguing that when you are in
enterprise and you have to deal with relational repositories and you have
simply pay the price and deal with the complexity (complexity == 'business
value' and any other competitor has to deal with the same complexity as
well.)
In general, this ORM thing has eventually too many points that must be
considered, so that a 'perfect' and universal solution does not exist...
Cheers,
jj
2017-05-08 10:28 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
> What I mean is that this starts with a Polygene model and it will create
> SQL tables.
>
> Why not the other way around?
> Well, you will immediately descend into the rabbit hole where Hibernate is.
> With incomprehensibly complex mappings of tables to arbitrary sections of
> code, simply not robust enough to ensure solid design. But the SQL model
> that we end up with is neat and in the spirit of SQL, unlike the current
> KeyValue use of SQL in the entitystore-sql.
>
> That said, adding some feature to the above (such as table/column name
> mappings) would allow creation of tool (interactive or declarative) to take
> a DB schema and generate the Polygene model from it, to the extent that it
> is possible. However, there will be constructs not achievable to fit the
> Polygene model, and it is likely that any DB would need additional tables
> and columns to accommodate a Polygene model, for instance things like
> entity versioning, lastmodifed, and others.
>
> Furthermore, it might be reasonable to capture additional requirements via
> JOOQ, and increase the support of consuming/feeding JOOQ, either by getting
> JOOQ to understand Polygene models, or the other way around. Getting JOOQ
> to understand Polygene seems possible, albeit not trivial, so I am not sure
> of the ambition level here. What's important to realize is that it is
> outside the domain of "Entity Stores" and UnitOfWork managed persistence.
> (I am already feeling the pain, that JOOQ seems to require the transaction
> to be within a single JOOQ method call, when we need the prepare() and
> commit() in separate calls, initiated from UnitOfWork)
>
> So, loads of work to do, but I want to focus on one step at a time, a step
> that is not endlessly complex to satisfy.
>
>
> Cheers
> Niclas
>
>
>
> On Mon, May 8, 2017 at 3:59 PM, Jiri Jetmar <ju...@gmail.com>
> wrote:
>
> > Hi Niclas,
> >
> > thank you for taking over the ORM topic. We definitely require a
> > 'generally' accepted approach how to deal with persistence !
> >
> > 'NOTE, this is only for "Java Model drives SQL model", but in an
> > enterprise-friendly way.'
> >
> > I;m a bit confused with the above expression. From my understanding we
> need
> > the opposite direction - a relational model drivers the application
> related
> > model. And as we are using Java, we end up with a O-R-M. Or do
> > I misunderstood something ?
> >
> > Thank you.
> >
> > Cheers,
> > jj
> >
> >
> >
> > 2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
> >
> > > Gang,
> > >
> > > I have gotten overly angry with Hibernate on $dayjob, and decided to
> > take a
> > > look at what a ORM-ish implementation in Polygene would look like. And
> it
> > > was easier than I expected, so...
> > >
> > > Pretty simple,
> > >
> > > 0. One "types table" that keep tracks of all types. Content of this is
> > > probably fully cached.
> > >
> > > 1. One type table for each MixinType (except HasIdentity)
> > >
> > > 2. One "base entity" table with the Identity, lastmodifed and so on
> > stuff.
> > >
> > > 3. Use a different (internal) identity for the "value", which makes it
> > easy
> > > to fully support non-destructive updates.
> > >
> > > 4. Property->SQL on basic types, and Serialization kick in on
> > > ValueComposites.
> > >
> > > 5. Association->SQL is simply a VARCHAR
> > >
> > > 6. Named/ManyAssociation->SQL needs an intermediate table per assoc
> > (naming
> > > is trouble some)
> > >
> > > 7. get() becomes a SELECT with one join per mixin type
> > >
> > > 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> > > statement, one for each mixin type plus an UPDATE for "base entity".
> > >
> > > 9. Named/ManyAssociations will also be an INSERT with many values.
> > >
> > > 10. JOOQ is awfully helpful in making this, and I am trying to
> modularize
> > > it so that people can customize it if needed, at Composite Methods
> level,
> > > either by Concerns or overriding Mixin methods.
> > >
> > > 11. IF destructive mode is wanted, I suggest that DELETE statements are
> > > issued to remove old stuff, which makes both modes semantically
> > identical,
> > > and not a big deal if something is accidentally not removed.
> > >
> > > And that is pretty much it. Fairly straight forward, and a far cry from
> > the
> > > complexity of Hibernate, guessing what it does and why.
> > >
> > > NOTE, this is only for "Java Model drives SQL model", but in an
> > > enterprise-friendly way. I hope that this can be a big "adoption
> driver".
> > >
> > > WDYAT?
> > >
> > >
> > > Cheers
> > > --
> > > Niclas Hedhman, Software Developer
> > > http://polygene.apache.org - New Energy for Java
> > >
> >
>
>
>
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
>
Re: ORM EntityStore
Posted by Niclas Hedhman <ni...@hedhman.org>.
What I mean is that this starts with a Polygene model and it will create
SQL tables.
Why not the other way around?
Well, you will immediately descend into the rabbit hole where Hibernate is.
With incomprehensibly complex mappings of tables to arbitrary sections of
code, simply not robust enough to ensure solid design. But the SQL model
that we end up with is neat and in the spirit of SQL, unlike the current
KeyValue use of SQL in the entitystore-sql.
That said, adding some feature to the above (such as table/column name
mappings) would allow creation of tool (interactive or declarative) to take
a DB schema and generate the Polygene model from it, to the extent that it
is possible. However, there will be constructs not achievable to fit the
Polygene model, and it is likely that any DB would need additional tables
and columns to accommodate a Polygene model, for instance things like
entity versioning, lastmodifed, and others.
Furthermore, it might be reasonable to capture additional requirements via
JOOQ, and increase the support of consuming/feeding JOOQ, either by getting
JOOQ to understand Polygene models, or the other way around. Getting JOOQ
to understand Polygene seems possible, albeit not trivial, so I am not sure
of the ambition level here. What's important to realize is that it is
outside the domain of "Entity Stores" and UnitOfWork managed persistence.
(I am already feeling the pain, that JOOQ seems to require the transaction
to be within a single JOOQ method call, when we need the prepare() and
commit() in separate calls, initiated from UnitOfWork)
So, loads of work to do, but I want to focus on one step at a time, a step
that is not endlessly complex to satisfy.
Cheers
Niclas
On Mon, May 8, 2017 at 3:59 PM, Jiri Jetmar <ju...@gmail.com>
wrote:
> Hi Niclas,
>
> thank you for taking over the ORM topic. We definitely require a
> 'generally' accepted approach how to deal with persistence !
>
> 'NOTE, this is only for "Java Model drives SQL model", but in an
> enterprise-friendly way.'
>
> I;m a bit confused with the above expression. From my understanding we need
> the opposite direction - a relational model drivers the application related
> model. And as we are using Java, we end up with a O-R-M. Or do
> I misunderstood something ?
>
> Thank you.
>
> Cheers,
> jj
>
>
>
> 2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
>
> > Gang,
> >
> > I have gotten overly angry with Hibernate on $dayjob, and decided to
> take a
> > look at what a ORM-ish implementation in Polygene would look like. And it
> > was easier than I expected, so...
> >
> > Pretty simple,
> >
> > 0. One "types table" that keep tracks of all types. Content of this is
> > probably fully cached.
> >
> > 1. One type table for each MixinType (except HasIdentity)
> >
> > 2. One "base entity" table with the Identity, lastmodifed and so on
> stuff.
> >
> > 3. Use a different (internal) identity for the "value", which makes it
> easy
> > to fully support non-destructive updates.
> >
> > 4. Property->SQL on basic types, and Serialization kick in on
> > ValueComposites.
> >
> > 5. Association->SQL is simply a VARCHAR
> >
> > 6. Named/ManyAssociation->SQL needs an intermediate table per assoc
> (naming
> > is trouble some)
> >
> > 7. get() becomes a SELECT with one join per mixin type
> >
> > 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> > statement, one for each mixin type plus an UPDATE for "base entity".
> >
> > 9. Named/ManyAssociations will also be an INSERT with many values.
> >
> > 10. JOOQ is awfully helpful in making this, and I am trying to modularize
> > it so that people can customize it if needed, at Composite Methods level,
> > either by Concerns or overriding Mixin methods.
> >
> > 11. IF destructive mode is wanted, I suggest that DELETE statements are
> > issued to remove old stuff, which makes both modes semantically
> identical,
> > and not a big deal if something is accidentally not removed.
> >
> > And that is pretty much it. Fairly straight forward, and a far cry from
> the
> > complexity of Hibernate, guessing what it does and why.
> >
> > NOTE, this is only for "Java Model drives SQL model", but in an
> > enterprise-friendly way. I hope that this can be a big "adoption driver".
> >
> > WDYAT?
> >
> >
> > Cheers
> > --
> > Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
> >
>
--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java
Re: ORM EntityStore
Posted by Jiri Jetmar <ju...@gmail.com>.
Hi Niclas,
thank you for taking over the ORM topic. We definitely require a
'generally' accepted approach how to deal with persistence !
'NOTE, this is only for "Java Model drives SQL model", but in an
enterprise-friendly way.'
I;m a bit confused with the above expression. From my understanding we need
the opposite direction - a relational model drivers the application related
model. And as we are using Java, we end up with a O-R-M. Or do
I misunderstood something ?
Thank you.
Cheers,
jj
2017-05-06 10:25 GMT+02:00 Niclas Hedhman <ni...@hedhman.org>:
> Gang,
>
> I have gotten overly angry with Hibernate on $dayjob, and decided to take a
> look at what a ORM-ish implementation in Polygene would look like. And it
> was easier than I expected, so...
>
> Pretty simple,
>
> 0. One "types table" that keep tracks of all types. Content of this is
> probably fully cached.
>
> 1. One type table for each MixinType (except HasIdentity)
>
> 2. One "base entity" table with the Identity, lastmodifed and so on stuff.
>
> 3. Use a different (internal) identity for the "value", which makes it easy
> to fully support non-destructive updates.
>
> 4. Property->SQL on basic types, and Serialization kick in on
> ValueComposites.
>
> 5. Association->SQL is simply a VARCHAR
>
> 6. Named/ManyAssociation->SQL needs an intermediate table per assoc (naming
> is trouble some)
>
> 7. get() becomes a SELECT with one join per mixin type
>
> 8. new/update is bunch of INSERT (if non-destructive is used) SQL
> statement, one for each mixin type plus an UPDATE for "base entity".
>
> 9. Named/ManyAssociations will also be an INSERT with many values.
>
> 10. JOOQ is awfully helpful in making this, and I am trying to modularize
> it so that people can customize it if needed, at Composite Methods level,
> either by Concerns or overriding Mixin methods.
>
> 11. IF destructive mode is wanted, I suggest that DELETE statements are
> issued to remove old stuff, which makes both modes semantically identical,
> and not a big deal if something is accidentally not removed.
>
> And that is pretty much it. Fairly straight forward, and a far cry from the
> complexity of Hibernate, guessing what it does and why.
>
> NOTE, this is only for "Java Model drives SQL model", but in an
> enterprise-friendly way. I hope that this can be a big "adoption driver".
>
> WDYAT?
>
>
> Cheers
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
>