You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Cheolsoo Park (JIRA)" <ji...@apache.org> on 2012/05/04 08:38:50 UTC
[jira] [Commented] (SQOOP-443) Calling sqoop with hive import is
not working multiple times due to kept output directory
[ https://issues.apache.org/jira/browse/SQOOP-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268172#comment-13268172 ]
Cheolsoo Park commented on SQOOP-443:
-------------------------------------
Hi Nemon,
I like your idea. In fact, I often find myself removing output dir, so having such an option will be very useful.
That being said, I think that we should treat removing output dir in hive import differently from removing output dir in general sqoop import. In hive import, output dir is no more than temporary staging dir whereas it is a permanent destination in general sqoop import. It makes sense to me to clean up temporary staging dir when the job was successful because data is safely moved to hive.
So I think that it is better to handle your suggestion in a separate jira.
Thoughts?
> Calling sqoop with hive import is not working multiple times due to kept output directory
> ------------------------------------------------------------------------------------------
>
> Key: SQOOP-443
> URL: https://issues.apache.org/jira/browse/SQOOP-443
> Project: Sqoop
> Issue Type: Improvement
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Priority: Minor
> Attachments: SQOOP-443.patch
>
>
> Hive is not removing input directory when doing "LOAD DATA" command in all cases. This input directory is actually sqoop's export directory. Because this directory is kept, calling same sqoop command twice is failing on exception "org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory $table already exists".
> This issue might be easily overcome by manual directory removal, however it's putting unnecessary burden on users. It's also complicating executing saved jobs as there is additional script execution needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira