You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "jan.swaelens" <ja...@sofico.be> on 2016/03/30 16:40:13 UTC
Ignite Schema Import Utility - Mismatch when loading data between
Date and Timestamp
Hello,
I am experimenting with the auto schema generator functionality and
experience the following issue. I have a table with a declared 'DATE' column
(Oracle). When loading the cache it actually gets a 'Timestamp' typed value
from the recordset.
Looking in the following method I see no logic to intercept this difference:
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.getColumnValue(ResultSet,
int, Class<?>)
The table is defined as follows:
CREATE TABLE "ACTIVITY"
( "ACTIVITY_ID" NUMBER(12,0),
..
"DUEDATE" DATE,
...
)
In there the 'due date' field is correctly typed but the actual query (and
resulting rs) contains a 'Timestamp'.
Adapting the generated code seems to resolve it (not completely sure as I am
hitting some other mapping mismatches), but that would be rather painful
when generating increments whenever we alter our model.
What would be the best approach here?
This is on latest Ignite release.
br
Jan
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
Thank you, planning a test session next week!
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p4152.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by Vasiliy Sisko <vs...@gridgain.com>.
Hello.
Issue with loading of Date data
https://issues.apache.org/jira/browse/IGNITE-2936 is closed.
It will be available at master branch or in next nightly build.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p4149.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
Thanks, out of office for two days but I will test asap when I'm back!
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3966.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data between
Date and Timestamp
Posted by Alexey Kuznetsov <ak...@gridgain.com>.
Jan,
I merged IGNITE-2936 into master today and I think it will be available
next day in nightly build or you could build from master by yourself.
On Fri, Apr 1, 2016 at 7:33 PM, jan.swaelens <ja...@sofico.be> wrote:
> Thank you!
>
> I registered two additional threads for the problems, referring back to
> this
> one in terms of generated collaterals, hope that is ok.
>
> br
> jan
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3859.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
Thank you!
I registered two additional threads for the problems, referring back to this
one in terms of generated collaterals, hope that is ok.
br
jan
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3859.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data between
Date and Timestamp
Posted by Alexey Kuznetsov <ak...@gridgain.com>.
Jan, I think it will be better to start separate threads.
As of IGNITE-2936 I will review Vasiliy code and merge into master in a
couple of days.
On Fri, Apr 1, 2016 at 6:26 PM, jan.swaelens <ja...@sofico.be> wrote:
> Thanks!
>
> For the two other cases (case mismatch on index and binary mismatch),
> should
> I start a separate thread or will we continue this in the current one?
>
> best regards
> jan
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3855.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
Thanks!
For the two other cases (case mismatch on index and binary mismatch), should
I start a separate thread or will we continue this in the current one?
best regards
jan
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3855.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by Vasiliy Sisko <vs...@gridgain.com>.
Hello @jan.swaelens
I reproduce you problem. Issue
https://issues.apache.org/jira/browse/IGNITE-2936 is created.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3850.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
Thanks!
About the Long vs long, indeed after experimenting with changing all Long
types to long in the generated code I found out that there was a problem
when it tried to set NULL on a primitive long. So they should remain Long
where NULL is allowed as you indicate indeed (otherwise it would cause
problems with the SQL queries if it would use for example - since that
would invalidate the use of the IS NULL clause).
The reason why I got on that long track is that I get an error (after
correcting the index uppercase and date/timestamp in code) in
'BinaryUtils.mergeMetadata' when it tries to merge two values for the
'nameMlid' field. At a certain point it generates a mismatch error because
newFieldTypeName is set to long and OldFieldTypeName is set to Object. Since
the field is typed as 'Long' and it is displaying 'long' there I assumed
that the problem orignated from the types used on the definition - which now
doesn't seem to be the case.
So I will have to add another question I'm afraid, any idea why it is being
confused here. Attached a log with the full trace as well.
binaryMismatch.log
<http://apache-ignite-users.70518.x6.nabble.com/file/n3813/binaryMismatch.log>
I'll have a go with the web console, if I can get it working through our
proxy that is. In the end I will probably be generating meta data using our
own meta repository but this looks like a nice feature to kickstart indeed.
Thanks again
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3813.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data between
Date and Timestamp
Posted by Alexey Kuznetsov <ak...@gridgain.com>.
Jan, thanks for test case we will debug and answer with our findings.
As of long and Long datatypes.
Schema import utility by default generate primitive types (long, int,...)
for NOT NULL fields and classes (Long, Integer,...) for nullable fields.
Type could be changed in UI by clicking on cell with type and select
appropriate type for your case.
Also could you try our GridGain web console? This utility also could import
database metadata and generate POJO + configuration files.
See https://ignite.apache.org/addons.html for more details.
On Thu, Mar 31, 2016 at 1:10 PM, jan.swaelens <ja...@sofico.be>
wrote:
> In addition to the above, I also find it strange that it generates both
> long
> and Long types for what seem to be very similar fields, for example:
>
> /** Value for predecessortypeEnumid. */
> private long predecessortypeEnumid;
>
> /** Value for successortypeEnumid. */
> private long successortypeEnumid;
>
> /** Value for durationunitEnumid. */
> private Long durationunitEnumid;
>
> This might be the cause of another problem I am facing - I'll get back with
> more details on it later, just wanted to indicate this as well in case it
> might be relevant to the other two observations.
>
> best regards
> jan
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3807.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
In addition to the above, I also find it strange that it generates both long
and Long types for what seem to be very similar fields, for example:
/** Value for predecessortypeEnumid. */
private long predecessortypeEnumid;
/** Value for successortypeEnumid. */
private long successortypeEnumid;
/** Value for durationunitEnumid. */
private Long durationunitEnumid;
This might be the cause of another problem I am facing - I'll get back with
more details on it later, just wanted to indicate this as well in case it
might be relevant to the other two observations.
best regards
jan
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3807.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data
between Date and Timestamp
Posted by "jan.swaelens" <ja...@sofico.be>.
Hello,
Please find the attached script to create the table and the generated pojo
file. I also included an insert for the record which is giving problems, in
case the data matters.
This is running against an Oracle Database 11g Release 11.2.0.1.0
activity_DDL.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/n3806/activity_DDL.txt>
Activity.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n3806/Activity.java>
ActivityKey.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n3806/ActivityKey.java>
CacheConfig.CacheConfig
<http://apache-ignite-users.70518.x6.nabble.com/file/n3806/CacheConfig.CacheConfig>
Now that we are at it, I have another issue with this particular generated
POJO.
Looking in the CacheConfig java file you see query indexes being defined
which reference uppercase column names, this indeed matches with the table
definition. However the pojo and defined fields are not completely in
uppercase, which causes a failure when the H2 tables are being created
during the definition of the cache.
error.log
<http://apache-ignite-users.70518.x6.nabble.com/file/n3806/error.log>
Thanks!
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790p3806.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Schema Import Utility - Mismatch when loading data between
Date and Timestamp
Posted by Alexey Kuznetsov <ak...@gridgain.com>.
Jan, could you provide a simple test case with one table (sql script to
create table in Oracle) and generated class.
We will reproduce and try to resolve this issue.
Also, please specify what version of Oracle do you use.
Thanks.
On Wed, Mar 30, 2016 at 9:40 PM, jan.swaelens <ja...@sofico.be>
wrote:
> Hello,
>
> I am experimenting with the auto schema generator functionality and
> experience the following issue. I have a table with a declared 'DATE'
> column
> (Oracle). When loading the cache it actually gets a 'Timestamp' typed value
> from the recordset.
>
> Looking in the following method I see no logic to intercept this
> difference:
>
> org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.getColumnValue(ResultSet,
> int, Class<?>)
>
> The table is defined as follows:
> CREATE TABLE "ACTIVITY"
> ( "ACTIVITY_ID" NUMBER(12,0),
> ..
> "DUEDATE" DATE,
> ...
> )
>
> In there the 'due date' field is correctly typed but the actual query (and
> resulting rs) contains a 'Timestamp'.
>
> Adapting the generated code seems to resolve it (not completely sure as I
> am
> hitting some other mapping mismatches), but that would be rather painful
> when generating increments whenever we alter our model.
>
> What would be the best approach here?
>
> This is on latest Ignite release.
>
> br
> Jan
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Schema-Import-Utility-Mismatch-when-loading-data-between-Date-and-Timestamp-tp3790.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com