You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (JIRA)" <ji...@apache.org> on 2012/09/07 06:54:07 UTC
[jira] [Created] (HIVE-3444) Support reading columns containing
line separator
Navis created HIVE-3444:
---------------------------
Summary: Support reading columns containing line separator
Key: HIVE-3444
URL: https://issues.apache.org/jira/browse/HIVE-3444
Project: Hive
Issue Type: Improvement
Components: Serializers/Deserializers
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
Currently, LazySimpleSerde cannot handle columns including newline character cause hadoop splits rows by newline character. If the overhead counting fields by full scan and merging partial lines is tolerable, multi-lined column can be reconstructed at runtime.
But with this method, multi-lined column should not be located at the last of the row. This is just a idea for HIVE-1898.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3444) Support reading columns containing
line separator
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3444:
------------------------
Status: Patch Available (was: Open)
https://reviews.facebook.net/D5277
> Support reading columns containing line separator
> -------------------------------------------------
>
> Key: HIVE-3444
> URL: https://issues.apache.org/jira/browse/HIVE-3444
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3444.1.patch.txt
>
>
> Currently, LazySimpleSerde cannot handle columns including newline character cause hadoop splits rows by newline character. If the overhead counting fields by full scan and merging partial lines is tolerable, multi-lined column can be reconstructed at runtime.
> But with this method, multi-lined column should not be located at the last of the row. This is just a idea for HIVE-1898.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3444) Support reading columns containing
line separator
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3444:
------------------------
Attachment: HIVE-3444.1.patch.txt
> Support reading columns containing line separator
> -------------------------------------------------
>
> Key: HIVE-3444
> URL: https://issues.apache.org/jira/browse/HIVE-3444
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3444.1.patch.txt
>
>
> Currently, LazySimpleSerde cannot handle columns including newline character cause hadoop splits rows by newline character. If the overhead counting fields by full scan and merging partial lines is tolerable, multi-lined column can be reconstructed at runtime.
> But with this method, multi-lined column should not be located at the last of the row. This is just a idea for HIVE-1898.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3444) Support reading columns containing
line separator
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451276#comment-13451276 ]
Navis commented on HIVE-3444:
-----------------------------
Ashutosh Chauhan: It's good. I'll close this. But still this could be a temporal walk-around.
> Support reading columns containing line separator
> -------------------------------------------------
>
> Key: HIVE-3444
> URL: https://issues.apache.org/jira/browse/HIVE-3444
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3444.1.patch.txt
>
>
> Currently, LazySimpleSerde cannot handle columns including newline character cause hadoop splits rows by newline character. If the overhead counting fields by full scan and merging partial lines is tolerable, multi-lined column can be reconstructed at runtime.
> But with this method, multi-lined column should not be located at the last of the row. This is just a idea for HIVE-1898.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3444) Support reading columns containing
line separator
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3444:
------------------------
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
> Support reading columns containing line separator
> -------------------------------------------------
>
> Key: HIVE-3444
> URL: https://issues.apache.org/jira/browse/HIVE-3444
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3444.1.patch.txt
>
>
> Currently, LazySimpleSerde cannot handle columns including newline character cause hadoop splits rows by newline character. If the overhead counting fields by full scan and merging partial lines is tolerable, multi-lined column can be reconstructed at runtime.
> But with this method, multi-lined column should not be located at the last of the row. This is just a idea for HIVE-1898.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3444) Support reading columns containing
line separator
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450750#comment-13450750 ]
Ashutosh Chauhan commented on HIVE-3444:
----------------------------------------
Navis: I believe with MAPREDUCE-2254 in there, we can handle this more elegantly ?
> Support reading columns containing line separator
> -------------------------------------------------
>
> Key: HIVE-3444
> URL: https://issues.apache.org/jira/browse/HIVE-3444
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3444.1.patch.txt
>
>
> Currently, LazySimpleSerde cannot handle columns including newline character cause hadoop splits rows by newline character. If the overhead counting fields by full scan and merging partial lines is tolerable, multi-lined column can be reconstructed at runtime.
> But with this method, multi-lined column should not be located at the last of the row. This is just a idea for HIVE-1898.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira