[jira] [Assigned] (NIFI-5888) QueryRecord processor handling timestamp

["Koji Kawamura (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Koji Kawamura reassigned NIFI-5888:
-----------------------------------

    Assignee: Koji Kawamura

> QueryRecord processor handling timestamp
> ----------------------------------------
>
>                 Key: NIFI-5888
>                 URL: https://issues.apache.org/jira/browse/NIFI-5888
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.8.0
>            Reporter: Gideon Korir
>            Assignee: Koji Kawamura
>            Priority: Major
>
> Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
>  
> {code:java}
> @Test public void testTimestampColumns() throws InitializationException { 
> final MockRecordParser parser = new MockRecordParser(); 
> parser.addSchemaField("name", RecordFieldType.STRING); 
> parser.addSchemaField("date_added", RecordFieldType.TIMESTAMP); 
> parser.addRecord("Tom", Timestamp.valueOf("2018-12-03 09:12:00")); 
> parser.addRecord("Jerry", Timestamp.valueOf("2018-12-04 10:26:00")); 
> parser.addRecord("Tom", Timestamp.valueOf("2017-01-03 11:22:00")); 
> final List<String> colNames = new ArrayList<>(); 
> colNames.add("name"); 
> colNames.add("day_added"); 
> final ResultSetValidatingRecordWriter writer = new ResultSetValidatingRecordWriter(colNames); 
> TestRunner runner = getRunner(); 
> runner.addControllerService("parser", parser); 
> runner.enableControllerService(parser); 
> runner.addControllerService("writer", writer); 
> runner.enableControllerService(writer); 
> runner.setProperty(REL_NAME, "select name, {fn YEAR(date_added)} as day_added from FLOWFILE"); runner.setProperty(QueryRecord.RECORD_READER_FACTORY, "parser"); 
> runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer"); 
> runner.enqueue(""); 
> runner.run(); 
> runner.assertTransferCount(REL_NAME, 1); 
> }
> {code}
>  
> *Expected*: Test will pass
> *Actual*: fails with ClassCastException with message "java.sql.Timestamp cannot be cast to java.lang.Long".
> //Also used sql: "select name, CAST(date_added, DATE) as day_added from FLOWFILE" and failed
>  
> I (think) I've traced the issue to the NiFi usage of Apache Calcite; according to Julian Hyde's comment on [this issue|https://issues.apache.org/jira/browse/CALCITE-1427].
> Disclaimer: My knowledge on Calcite is zero to none so I could be completely wrong.
> Currently, QueryRecord uses FlowFileTable which only implements QueryableTable and TranslateableTable and therefore can not work with java.sql.Timestamp (at least based on Hyde's comment on the issue). This will mean almost all RecordReader services (Avro, Csv) would not work with QueryRecord.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)