[jira] [Created] (GOBBLIN-1579) Propagate HiveTableLocationNotMatchException to fail the job execution

["Venkatesh (Jira)" <ji...@apache.org>]

Venkatesh created GOBBLIN-1579:
----------------------------------

             Summary: Propagate HiveTableLocationNotMatchException to fail the job execution
                 Key: GOBBLIN-1579
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1579
             Project: Apache Gobblin
          Issue Type: Bug
            Reporter: Venkatesh


When existing target table path is different from desired path for an hive data movement  the below mentioned code in UnpartitionedTableFileSet class gets executed. 
default:
  log.error("Source and target table are not compatible. Aborting copy of table " + this.helper.getTargetTable(),
      new HiveTableLocationNotMatchException(this.helper.getTargetTable().getDataLocation(),
          existingTargetTable.get().getDataLocation()));
  multiTimer.close();
  return Lists.newArrayList();
HiveTableLocationNotMatchException is just logged as an Error and empty list is returned. Though the workunits computation failed, because of the empty arraylist, the created workunits is zero and the job status is set as COMMITTED.

 

Error log:

2021-11-16 11:37:04 PST ERROR [request-allocator-0] org.apache.gobblin.data.management.copy.hive.UnpartitionedTableFileSet  - Source and target table are not compatible. Aborting copy of table employeeorg.apache.gobblin.data.management.copy.hive.HiveTableLocationNotMatchException: Desired target location [hdfs://localhost:8020/tmp/hive/tc_db_1_copy/employee] and already registered target location file:/tmp/hive/tc_db_copy/employee/data/employee do not agree.         at org.apache.gobblin.data.management.copy.hive.UnpartitionedTableFileSet.generateCopyEntities(UnpartitionedTableFileSet.java:86)

 

Solution:

Throw HiveTableLocationNotMatchException so that the exception will be propogated to the caller and the job run will result in failure.

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)