You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Byunghwa Yun (JIRA)" <ji...@apache.org> on 2016/10/05 05:07:20 UTC
[jira] [Updated] (NIFI-2867) The ColumnSize of datetime type is 19
length in MySQL JDBC.
[ https://issues.apache.org/jira/browse/NIFI-2867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Byunghwa Yun updated NIFI-2867:
-------------------------------
Description:
When I use the ConvertJSONToSQL, it changes the field value for data capacity of source datastore.
/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java: 430
if (colSize != null && fieldValue.length() > colSize) {
fieldValue = fieldValue.substring(0, colSize);
}
But the type of field is datetime in MySQL DB, JDBC driver returns 19 column size. So above code truncate field value to 2016-10-05 12:00:00 from 2016-10-05 12:00:00.0.
PutSQL only resolve the TIMESTAMP type using yyyy-MM-dd HH:mm:ss.SSS SimpleDateFormat. So it throws below exception.
2016-10-05 13:05:56,526 ERROR [Timer-Driven Process Thread-14] o.apache.nifi.processors.standard.PutSQL
org.apache.nifi.processor.exception.ProcessException: The value of the sql.args.7.value is '2016-10-05 13:05:43', which cannot be converted to a timestamp
at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:630) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:241) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
Caused by: java.text.ParseException: Unparseable date: "2016-10-05 13:05:43"
at java.text.DateFormat.parse(DateFormat.java:366) ~[na:1.8.0_51]
at org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:798) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:626) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
... 13 common frames omitted
Thank you.
was:
When I use the ConvertJSONToSQL, it changes the field value for data capacity of source datastore.
/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java: 430
if (colSize != null && fieldValue.length() > colSize) {
fieldValue = fieldValue.substring(0, colSize);
}
But the type of field is datetime in MySQL DB, JDBC driver returns 19 column size. So above code truncate field value to 2016-10-05 12:00:00 from 2016-10-05 12:00:00.0.
PuSQL only resolve the TIMESTAMP type using yyyy-MM-dd HH:mm:ss.SSS SimpleDateFormat. So it throws below exception.
2016-10-05 13:05:56,526 ERROR [Timer-Driven Process Thread-14] o.apache.nifi.processors.standard.PutSQL
org.apache.nifi.processor.exception.ProcessException: The value of the sql.args.7.value is '2016-10-05 13:05:43', which cannot be converted to a timestamp
at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:630) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:241) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
Caused by: java.text.ParseException: Unparseable date: "2016-10-05 13:05:43"
at java.text.DateFormat.parse(DateFormat.java:366) ~[na:1.8.0_51]
at org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:798) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:626) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
... 13 common frames omitted
Thank you.
> The ColumnSize of datetime type is 19 length in MySQL JDBC.
> -----------------------------------------------------------
>
> Key: NIFI-2867
> URL: https://issues.apache.org/jira/browse/NIFI-2867
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.1.0
> Reporter: Byunghwa Yun
> Priority: Minor
>
> When I use the ConvertJSONToSQL, it changes the field value for data capacity of source datastore.
> /nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java: 430
> if (colSize != null && fieldValue.length() > colSize) {
> fieldValue = fieldValue.substring(0, colSize);
> }
> But the type of field is datetime in MySQL DB, JDBC driver returns 19 column size. So above code truncate field value to 2016-10-05 12:00:00 from 2016-10-05 12:00:00.0.
> PutSQL only resolve the TIMESTAMP type using yyyy-MM-dd HH:mm:ss.SSS SimpleDateFormat. So it throws below exception.
> 2016-10-05 13:05:56,526 ERROR [Timer-Driven Process Thread-14] o.apache.nifi.processors.standard.PutSQL
> org.apache.nifi.processor.exception.ProcessException: The value of the sql.args.7.value is '2016-10-05 13:05:43', which cannot be converted to a timestamp
> at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:630) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:241) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_51]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_51]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> Caused by: java.text.ParseException: Unparseable date: "2016-10-05 13:05:43"
> at java.text.DateFormat.parse(DateFormat.java:366) ~[na:1.8.0_51]
> at org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:798) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:626) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> ... 13 common frames omitted
> Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)