[jira] [Created] (SQOOP-3467) sqoop incremental job: util.AppendUtils: Cannot append files to target dir; no such directory

["Mike Bridge (Jira)" <ji...@apache.org>]

Mike Bridge created SQOOP-3467:
----------------------------------

             Summary: sqoop incremental job: util.AppendUtils: Cannot append files to target dir; no such directory
                 Key: SQOOP-3467
                 URL: https://issues.apache.org/jira/browse/SQOOP-3467
             Project: Sqoop
          Issue Type: Bug
    Affects Versions: 1.4.7
         Environment:  
{noformat}
 {noformat}
 
            Reporter: Mike Bridge


I am running sqoop to export from postgresql to avro as a hadoop job on Google and exporting to a google Cloud Storage.  When I export a whole table it works perfectly, but exporting incrementally gives me an error.

 

 
{code:java}
gcloud dataproc jobs submit hadoop \
 --cluster="$CLUSTER_NAME" \
 --class=org.apache.sqoop.Sqoop \
 --properties=mapreduce.job.classloader=true \
 --jars="$UBER_JAR,$HSQLDB_JAR" \
 --region="$CLUSTER_REGION" \
 -- job --create "$job_name" \
 -- import \
 --connect="${CONNECTION_STRING}" \
 --username="${SOURCE_USER}" \
 --password="${SOURCE_PASSWORD}" \
 --target-dir="gs://$WAREHOUSE_BUCKET_NAME/${EXPORT_DIRNAME}/${job_name}" \
 --table="$table_name" \
 --as-avrodatafile $(echo "${map_column_args}") \
 --incremental=append \
 --split-by="${split_by}" \
 --check-column created \
 --last-value "2017-01-01 00:00:00.000000" \
 --verbose{code}
 

I can see it executes, but no data is produced.  There's a warning "Cannot append files to target dir" in the log:

 
{noformat}
20/03/13 20:52:18 INFO mapreduce.ImportJobBase: Transferred 4.6844 MB in 15.9306 seconds (301.106 KB/sec)
20/03/13 20:52:18 INFO mapreduce.ImportJobBase: Retrieved 27783 records.
20/03/13 20:52:18 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@7dc36524
20/03/13 20:52:18 WARN util.AppendUtils: Cannot append files to target dir; no such directory: _sqoop/df1bc552c9754b5aa2db3a6c04b03a75_insights_action
20/03/13 20:52:18 INFO tool.ImportTool: Incremental import complete! To run another incremental import of all data following this import, supply the following arguments:
20/03/13 20:52:18 INFO tool.ImportTool:  --incremental append
20/03/13 20:52:18 INFO tool.ImportTool:   --check-column created
20/03/13 20:52:18 INFO tool.ImportTool:   --last-value 2020-03-13 14:54:01.997784
20/03/13 20:52:18 INFO tool.ImportTool: (Consider saving this with 'sqoop job --create')
Job [1673b419f6c042d18dd8124f06e9c412] finished successfully.{noformat}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)