You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org> on 2011/08/19 20:52:29 UTC
[jira] [Commented] (SQOOP-319) The --hive-drop-import-delims option
should accept a replacement string
[ https://issues.apache.org/jira/browse/SQOOP-319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087888#comment-13087888 ]
jiraposter@reviews.apache.org commented on SQOOP-319:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1598/
-----------------------------------------------------------
Review request for Sqoop.
Summary
-------
I added a new option, --hive-delims-replacement, which lets you pass in a replacement string. I did it with a new option to remain backwards compatible with the existing interface.
This addresses bug SQOOP-319.
https://issues.apache.org/jira/browse/SQOOP-319
Diffs
-----
src/docs/user/hive-args.txt 7e6b7a0
src/docs/user/hive.txt 059d7cb
src/java/com/cloudera/sqoop/SqoopOptions.java d760d39
src/java/com/cloudera/sqoop/lib/FieldFormatter.java 41536e1
src/java/com/cloudera/sqoop/orm/ClassWriter.java dd3994e
src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 8f629f1
src/test/com/cloudera/sqoop/hive/TestHiveImport.java 35de2fd
testdata/hive/scripts/fieldWithNewlineReplacementImport.q PRE-CREATION
Diff: https://reviews.apache.org/r/1598/diff
Testing
-------
I added a unit test for the new option. I also tested the feature by hand. It works, but I found a bug when doing --direct (at least with MySQL). It doesn't end up calling the hiveStringDropDelims() function. Some other kind of escaping is going on. I'll file that as a separate JIRA.
Thanks,
Joey
> The --hive-drop-import-delims option should accept a replacement string
> -----------------------------------------------------------------------
>
> Key: SQOOP-319
> URL: https://issues.apache.org/jira/browse/SQOOP-319
> Project: Sqoop
> Issue Type: Bug
> Components: hive-integration
> Affects Versions: 1.3.0
> Reporter: Joey Echeverria
> Assignee: Joey Echeverria
> Priority: Minor
> Attachments: SQOOP-319-1.patch
>
>
> When importing data into hive, you have the option of dropping the hive delimiters in data fields. It would be more useful to replace the delimiters with a user defined string. Often times the dropped delimiters (like \n) are separating words. If I want to split on white space in my hive queries, I'll now get two words merged together. A more desirable behavior would be to replace it with a space. Making it user configurable will give the most flexibility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira