[jira] [Created] (HUDI-485) Check for where clause is wrong in HiveIncrementalPuller

["Pratyaksh Sharma (Jira)" <ji...@apache.org>]

Pratyaksh Sharma created HUDI-485:
-------------------------------------

             Summary: Check for where clause is wrong in HiveIncrementalPuller
                 Key: HUDI-485
                 URL: https://issues.apache.org/jira/browse/HUDI-485
             Project: Apache Hudi (incubating)
          Issue Type: Sub-task
          Components: Incremental Pull, newbie
            Reporter: Pratyaksh Sharma
            Assignee: Pratyaksh Sharma


HiveIncrementalPuller checks the clause in incrementalSqlFile like this -> 

if (!incrementalSQL.contains("`_hoodie_commit_time` > '%targetBasePath'")) {
 LOG.info("Incremental SQL : " + incrementalSQL
 + " does not contain `_hoodie_commit_time` > %targetBasePath. Please add "
 + "this clause for incremental to work properly.");
 throw new HoodieIncrementalPullSQLException(
 "Incremental SQL does not have clause `_hoodie_commit_time` > '%targetBasePath', which "
 + "means its not pulling incrementally");
}

Basically we are trying to add a placeholder here which is later replaced with config.fromCommitTime here - 

incrementalPullSQLtemplate.add("incrementalSQL", String.format(incrementalSQL, config.fromCommitTime));

Hence, the above check needs to replaced with `_hoodie_commit_time` > %targetBasePath



--
This message was sent by Atlassian Jira
(v8.3.4#803005)