You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by Mark Struberg <st...@yahoo.de.INVALID> on 2021/05/04 06:24:16 UTC

nearing a 3.2.0 release

hi folks!

We are nearing a release. We just have to polish the documentation and prepare.

I've successfully did run all tests green for the following databases:

* Derby
* h2
* PostgreSQL
* Oracle
* MySQL
* MariaDB
* MS SQLServer

* Hypersonic (HSQLDB) has a few test glitches since many moons :( Nothing tragical, only things like missing delimiters when DBIdentifiers has spaces, etc. We need to work on it, but I fear it will not be ready for this release. Same tests have been failing for the last releases as well.

I had to revert a few delimiter changes which came as part of the HerdDB support. We need to do a general re-evaluation of DBDictionary#delimitAll. This needs more time and we must make sure it breaks no other existing databases. Not sure if we can find time before 3.2.0 or rather fix it for 3.2.1 and only ship preliminary support for HerdDB for now?

Any opinions?

txs and LieGrue,
strub

Re: nearing a 3.2.0 release

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hmm, code looks wrong anyway since
https://github.com/apache/openjpa/blob/dd9bce0cc909bd96d706a9438bdc2a58111c481f/openjpa-jdbc/src/main/java/org/apache/openjpa/jdbc/identifier/DBIdentifierUtilImpl.java#L286
handles delimiter now so shouldnt be handled twice probably.

> The problem is that it only creates a problem if you let this run
multiple times

Does it mean when a DB exists and you restart the app (or another instance
starts) problem accurs?
This must clearly not be I agree.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 4 mai 2021 à 11:53, Enrico Olivelli <eo...@gmail.com> a écrit :

> Il giorno mar 4 mag 2021 alle ore 11:19 Mark Struberg
> <st...@yahoo.de.invalid> ha scritto:
> >
> > Hi!
> >
> >
> https://github.com/apache/openjpa/commit/dd9bce0cc909bd96d706a9438bdc2a58111c481f
> <
> https://github.com/apache/openjpa/commit/dd9bce0cc909bd96d706a9438bdc2a58111c481f
> >
> >
> > The problem is that it only creates a problem if you let this run
> multiple times. PostgreSQL internally converts all delimited columns to
> lowercase. But somewhere along the way it corrupted the information,
> leading to some columns not being able to allow dropping the whole table
> due to not matching anymore.
> >
> > The problematic area seems to be this code part. It does not look wrong,
> but somehow it creates a problem with PostgreSQL:
> >
> > protected Column addPrimaryKeyColumn(Table table) {
> >     DBDictionary dict = _conf.getDBDictionaryInstance();
> >     DBIdentifier delimitedColumnName =
> dict.fromDBName(getPrimaryKeyColumn(),
> DBIdentifier.DBIdentifierType.COLUMN);
> >     Column pkColumn = table.addColumn(dict.getValidColumnName
> >         (delimitedColumnName, table));
> >
> > previously (and now again) it was:
> > Column pkColumn =
> table.addColumn(dict.getValidColumnName(getPrimaryKeyColumnIdentifier(),
> table));
> >
> > Happy to help with HerdDB, but stability for existing databases must be
> guaranteed.
>
>
> I have run a minimal set of tests on HerdDB with current OpenJPA
> master and my tests are passing.
>
> ButI am sure that I did those changes for some reason, probably for
> the tests we were running with Romain in preparation to ApacheCon.
> I cannot find them anymore (I changed my laptop/work in the meantime).
>
> Probably the problem was about recreating the
> tables/columns/sequences, like the problems you are pointing out with
> PGSQL
>
> Enrico
>
> > Let's try to work this out till the weekend!
> >
> > LieGrue,
> > strub
> >
> >
> >
> > > Am 04.05.2021 um 08:52 schrieb Romain Manni-Bucau <
> rmannibucau@gmail.com>:
> > >
> > > Hi,
> > >
> > > The HSQLDB changes are okish and does not break existing applications
> > > (which is the most important) so only people working on generated DDL
> > > directly can have issues and due to the actual impact it should be fine
> > > anyway.
> > >
> > > However, breaking HerdDB is not an option so I agree with Enrico we
> should
> > > ensure it works - at least as HSQL where the runtime is not broken - or
> > > delay the 3.2 until we can do it.
> > >
> > > @Mark did you see why we didn't catch these issues before since build
> was
> > > green when we merged HerdDB integration? Is it profile related (and
> should
> > > we enable them by default with N surefire executions) or something not
> > > covered we should add? Goal is indeed to avoid it to happen again.
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> > >
> > >
> > > Le mar. 4 mai 2021 à 08:47, Enrico Olivelli <eo...@gmail.com> a
> écrit :
> > >
> > >> Il giorno mar 4 mag 2021 alle ore 08:24 Mark Struberg
> > >> <st...@yahoo.de.invalid> ha scritto:
> > >>>
> > >>> hi folks!
> > >>>
> > >>> We are nearing a release. We just have to polish the documentation
> and
> > >> prepare.
> > >>>
> > >>> I've successfully did run all tests green for the following
> databases:
> > >>>
> > >>> * Derby
> > >>> * h2
> > >>> * PostgreSQL
> > >>> * Oracle
> > >>> * MySQL
> > >>> * MariaDB
> > >>> * MS SQLServer
> > >>>
> > >>> * Hypersonic (HSQLDB) has a few test glitches since many moons :(
> > >> Nothing tragical, only things like missing delimiters when
> DBIdentifiers
> > >> has spaces, etc. We need to work on it, but I fear it will not be
> ready for
> > >> this release. Same tests have been failing for the last releases as
> well.
> > >>>
> > >>> I had to revert a few delimiter changes which came as part of the
> HerdDB
> > >> support.
> > >>
> > >> Can you please point out what has been reverted ?
> > >> I believe Romain had some test case applications that break due to
> > >> reserved Words.
> > >>
> > >> I am sorry that I did not have time to complete the work on making all
> > >> tests pass on HerdDB.
> > >>
> > >> We need to do a general re-evaluation of DBDictionary#delimitAll. This
> > >> needs more time and we must make sure it breaks no other existing
> > >> databases. Not sure if we can find time before 3.2.0 or rather fix it
> > >> for 3.2.1 and only ship preliminary support for HerdDB for now?
> > >>>
> > >>> Any opinions?
> > >>
> > >> I can try to build current Master and run the few tests cases we have
> > >> in HerdDB repo that test compatibility with OpenJPA
> > >>
> > >> Enrico
> > >>
> > >>>
> > >>> txs and LieGrue,
> > >>> strub
> > >>
> >
>

Re: nearing a 3.2.0 release

Posted by Enrico Olivelli <eo...@gmail.com>.
Il giorno mar 4 mag 2021 alle ore 11:19 Mark Struberg
<st...@yahoo.de.invalid> ha scritto:
>
> Hi!
>
> https://github.com/apache/openjpa/commit/dd9bce0cc909bd96d706a9438bdc2a58111c481f <https://github.com/apache/openjpa/commit/dd9bce0cc909bd96d706a9438bdc2a58111c481f>
>
> The problem is that it only creates a problem if you let this run multiple times. PostgreSQL internally converts all delimited columns to lowercase. But somewhere along the way it corrupted the information, leading to some columns not being able to allow dropping the whole table due to not matching anymore.
>
> The problematic area seems to be this code part. It does not look wrong, but somehow it creates a problem with PostgreSQL:
>
> protected Column addPrimaryKeyColumn(Table table) {
>     DBDictionary dict = _conf.getDBDictionaryInstance();
>     DBIdentifier delimitedColumnName = dict.fromDBName(getPrimaryKeyColumn(), DBIdentifier.DBIdentifierType.COLUMN);
>     Column pkColumn = table.addColumn(dict.getValidColumnName
>         (delimitedColumnName, table));
>
> previously (and now again) it was:
> Column pkColumn = table.addColumn(dict.getValidColumnName(getPrimaryKeyColumnIdentifier(), table));
>
> Happy to help with HerdDB, but stability for existing databases must be guaranteed.


I have run a minimal set of tests on HerdDB with current OpenJPA
master and my tests are passing.

ButI am sure that I did those changes for some reason, probably for
the tests we were running with Romain in preparation to ApacheCon.
I cannot find them anymore (I changed my laptop/work in the meantime).

Probably the problem was about recreating the
tables/columns/sequences, like the problems you are pointing out with
PGSQL

Enrico

> Let's try to work this out till the weekend!
>
> LieGrue,
> strub
>
>
>
> > Am 04.05.2021 um 08:52 schrieb Romain Manni-Bucau <rm...@gmail.com>:
> >
> > Hi,
> >
> > The HSQLDB changes are okish and does not break existing applications
> > (which is the most important) so only people working on generated DDL
> > directly can have issues and due to the actual impact it should be fine
> > anyway.
> >
> > However, breaking HerdDB is not an option so I agree with Enrico we should
> > ensure it works - at least as HSQL where the runtime is not broken - or
> > delay the 3.2 until we can do it.
> >
> > @Mark did you see why we didn't catch these issues before since build was
> > green when we merged HerdDB integration? Is it profile related (and should
> > we enable them by default with N surefire executions) or something not
> > covered we should add? Goal is indeed to avoid it to happen again.
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <https://www.packtpub.com/application-development/java-ee-8-high-performance>
> >
> >
> > Le mar. 4 mai 2021 à 08:47, Enrico Olivelli <eo...@gmail.com> a écrit :
> >
> >> Il giorno mar 4 mag 2021 alle ore 08:24 Mark Struberg
> >> <st...@yahoo.de.invalid> ha scritto:
> >>>
> >>> hi folks!
> >>>
> >>> We are nearing a release. We just have to polish the documentation and
> >> prepare.
> >>>
> >>> I've successfully did run all tests green for the following databases:
> >>>
> >>> * Derby
> >>> * h2
> >>> * PostgreSQL
> >>> * Oracle
> >>> * MySQL
> >>> * MariaDB
> >>> * MS SQLServer
> >>>
> >>> * Hypersonic (HSQLDB) has a few test glitches since many moons :(
> >> Nothing tragical, only things like missing delimiters when DBIdentifiers
> >> has spaces, etc. We need to work on it, but I fear it will not be ready for
> >> this release. Same tests have been failing for the last releases as well.
> >>>
> >>> I had to revert a few delimiter changes which came as part of the HerdDB
> >> support.
> >>
> >> Can you please point out what has been reverted ?
> >> I believe Romain had some test case applications that break due to
> >> reserved Words.
> >>
> >> I am sorry that I did not have time to complete the work on making all
> >> tests pass on HerdDB.
> >>
> >> We need to do a general re-evaluation of DBDictionary#delimitAll. This
> >> needs more time and we must make sure it breaks no other existing
> >> databases. Not sure if we can find time before 3.2.0 or rather fix it
> >> for 3.2.1 and only ship preliminary support for HerdDB for now?
> >>>
> >>> Any opinions?
> >>
> >> I can try to build current Master and run the few tests cases we have
> >> in HerdDB repo that test compatibility with OpenJPA
> >>
> >> Enrico
> >>
> >>>
> >>> txs and LieGrue,
> >>> strub
> >>
>

Re: nearing a 3.2.0 release

Posted by Mark Struberg <st...@yahoo.de.INVALID>.
Hi!

https://github.com/apache/openjpa/commit/dd9bce0cc909bd96d706a9438bdc2a58111c481f <https://github.com/apache/openjpa/commit/dd9bce0cc909bd96d706a9438bdc2a58111c481f>

The problem is that it only creates a problem if you let this run multiple times. PostgreSQL internally converts all delimited columns to lowercase. But somewhere along the way it corrupted the information, leading to some columns not being able to allow dropping the whole table due to not matching anymore.

The problematic area seems to be this code part. It does not look wrong, but somehow it creates a problem with PostgreSQL:

protected Column addPrimaryKeyColumn(Table table) {
    DBDictionary dict = _conf.getDBDictionaryInstance();
    DBIdentifier delimitedColumnName = dict.fromDBName(getPrimaryKeyColumn(), DBIdentifier.DBIdentifierType.COLUMN);
    Column pkColumn = table.addColumn(dict.getValidColumnName
        (delimitedColumnName, table));

previously (and now again) it was:
Column pkColumn = table.addColumn(dict.getValidColumnName(getPrimaryKeyColumnIdentifier(), table));

Happy to help with HerdDB, but stability for existing databases must be guaranteed.
Let's try to work this out till the weekend!

LieGrue,
strub

 

> Am 04.05.2021 um 08:52 schrieb Romain Manni-Bucau <rm...@gmail.com>:
> 
> Hi,
> 
> The HSQLDB changes are okish and does not break existing applications
> (which is the most important) so only people working on generated DDL
> directly can have issues and due to the actual impact it should be fine
> anyway.
> 
> However, breaking HerdDB is not an option so I agree with Enrico we should
> ensure it works - at least as HSQL where the runtime is not broken - or
> delay the 3.2 until we can do it.
> 
> @Mark did you see why we didn't catch these issues before since build was
> green when we merged HerdDB integration? Is it profile related (and should
> we enable them by default with N surefire executions) or something not
> covered we should add? Goal is indeed to avoid it to happen again.
> 
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
> 
> 
> Le mar. 4 mai 2021 à 08:47, Enrico Olivelli <eo...@gmail.com> a écrit :
> 
>> Il giorno mar 4 mag 2021 alle ore 08:24 Mark Struberg
>> <st...@yahoo.de.invalid> ha scritto:
>>> 
>>> hi folks!
>>> 
>>> We are nearing a release. We just have to polish the documentation and
>> prepare.
>>> 
>>> I've successfully did run all tests green for the following databases:
>>> 
>>> * Derby
>>> * h2
>>> * PostgreSQL
>>> * Oracle
>>> * MySQL
>>> * MariaDB
>>> * MS SQLServer
>>> 
>>> * Hypersonic (HSQLDB) has a few test glitches since many moons :(
>> Nothing tragical, only things like missing delimiters when DBIdentifiers
>> has spaces, etc. We need to work on it, but I fear it will not be ready for
>> this release. Same tests have been failing for the last releases as well.
>>> 
>>> I had to revert a few delimiter changes which came as part of the HerdDB
>> support.
>> 
>> Can you please point out what has been reverted ?
>> I believe Romain had some test case applications that break due to
>> reserved Words.
>> 
>> I am sorry that I did not have time to complete the work on making all
>> tests pass on HerdDB.
>> 
>> We need to do a general re-evaluation of DBDictionary#delimitAll. This
>> needs more time and we must make sure it breaks no other existing
>> databases. Not sure if we can find time before 3.2.0 or rather fix it
>> for 3.2.1 and only ship preliminary support for HerdDB for now?
>>> 
>>> Any opinions?
>> 
>> I can try to build current Master and run the few tests cases we have
>> in HerdDB repo that test compatibility with OpenJPA
>> 
>> Enrico
>> 
>>> 
>>> txs and LieGrue,
>>> strub
>>

Re: nearing a 3.2.0 release

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi,

The HSQLDB changes are okish and does not break existing applications
(which is the most important) so only people working on generated DDL
directly can have issues and due to the actual impact it should be fine
anyway.

However, breaking HerdDB is not an option so I agree with Enrico we should
ensure it works - at least as HSQL where the runtime is not broken - or
delay the 3.2 until we can do it.

@Mark did you see why we didn't catch these issues before since build was
green when we merged HerdDB integration? Is it profile related (and should
we enable them by default with N surefire executions) or something not
covered we should add? Goal is indeed to avoid it to happen again.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 4 mai 2021 à 08:47, Enrico Olivelli <eo...@gmail.com> a écrit :

> Il giorno mar 4 mag 2021 alle ore 08:24 Mark Struberg
> <st...@yahoo.de.invalid> ha scritto:
> >
> > hi folks!
> >
> > We are nearing a release. We just have to polish the documentation and
> prepare.
> >
> > I've successfully did run all tests green for the following databases:
> >
> > * Derby
> > * h2
> > * PostgreSQL
> > * Oracle
> > * MySQL
> > * MariaDB
> > * MS SQLServer
> >
> > * Hypersonic (HSQLDB) has a few test glitches since many moons :(
> Nothing tragical, only things like missing delimiters when DBIdentifiers
> has spaces, etc. We need to work on it, but I fear it will not be ready for
> this release. Same tests have been failing for the last releases as well.
> >
> > I had to revert a few delimiter changes which came as part of the HerdDB
> support.
>
> Can you please point out what has been reverted ?
> I believe Romain had some test case applications that break due to
> reserved Words.
>
> I am sorry that I did not have time to complete the work on making all
> tests pass on HerdDB.
>
> We need to do a general re-evaluation of DBDictionary#delimitAll. This
> needs more time and we must make sure it breaks no other existing
> databases. Not sure if we can find time before 3.2.0 or rather fix it
> for 3.2.1 and only ship preliminary support for HerdDB for now?
> >
> > Any opinions?
>
> I can try to build current Master and run the few tests cases we have
> in HerdDB repo that test compatibility with OpenJPA
>
> Enrico
>
> >
> > txs and LieGrue,
> > strub
>

Re: nearing a 3.2.0 release

Posted by Enrico Olivelli <eo...@gmail.com>.
Il giorno mar 4 mag 2021 alle ore 08:24 Mark Struberg
<st...@yahoo.de.invalid> ha scritto:
>
> hi folks!
>
> We are nearing a release. We just have to polish the documentation and prepare.
>
> I've successfully did run all tests green for the following databases:
>
> * Derby
> * h2
> * PostgreSQL
> * Oracle
> * MySQL
> * MariaDB
> * MS SQLServer
>
> * Hypersonic (HSQLDB) has a few test glitches since many moons :( Nothing tragical, only things like missing delimiters when DBIdentifiers has spaces, etc. We need to work on it, but I fear it will not be ready for this release. Same tests have been failing for the last releases as well.
>
> I had to revert a few delimiter changes which came as part of the HerdDB support.

Can you please point out what has been reverted ?
I believe Romain had some test case applications that break due to
reserved Words.

I am sorry that I did not have time to complete the work on making all
tests pass on HerdDB.

We need to do a general re-evaluation of DBDictionary#delimitAll. This
needs more time and we must make sure it breaks no other existing
databases. Not sure if we can find time before 3.2.0 or rather fix it
for 3.2.1 and only ship preliminary support for HerdDB for now?
>
> Any opinions?

I can try to build current Master and run the few tests cases we have
in HerdDB repo that test compatibility with OpenJPA

Enrico

>
> txs and LieGrue,
> strub