You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hawq.apache.org by shivzone <gi...@git.apache.org> on 2017/12/12 17:37:28 UTC
[GitHub] incubator-hawq pull request #1322: Support Hive OpenCSVSerde
GitHub user shivzone opened a pull request:
https://github.com/apache/incubator-hawq/pull/1322
Support Hive OpenCSVSerde
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shivzone/incubator-hawq HAWQ-1571
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/1322.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1322
----
commit c13aa9f28b054e24f6deea9902e5219ebae7d486
Author: shivzone <sh...@gmail.com>
Date: 2017-12-12T17:36:58Z
Support Hive OpenCSVSerde
----
---
[GitHub] incubator-hawq issue #1322: Support Hive OpenCSVSerde
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on the issue:
https://github.com/apache/incubator-hawq/pull/1322
@sansanichfb based on team's suggestion, I've removed the explicit serde check from HiveLineBreakAccessor as it supports multiple custom serdes.
We can eventually do away with the strict serde class check upon further investigation with the other Accessors.
---
[GitHub] incubator-hawq pull request #1322: Support Hive OpenCSVSerde
Posted by denalex <gi...@git.apache.org>.
Github user denalex commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1322#discussion_r156481646
--- Diff: pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveLineBreakAccessor.java ---
@@ -43,7 +43,7 @@
public HiveLineBreakAccessor(InputData input) throws Exception {
super(input, new TextInputFormat());
((TextInputFormat) inputFormat).configure(jobConf);
- HiveUserData hiveUserData = HiveUtilities.parseHiveUserData(input, PXF_HIVE_SERDES.LAZY_SIMPLE_SERDE);
+ HiveUserData hiveUserData = HiveUtilities.parseHiveUserData(input, PXF_HIVE_SERDES.LAZY_SIMPLE_SERDE, PXF_HIVE_SERDES.CSV_SERDE);
--- End diff --
why do we need to keep a list of supported Serde ? If Serde is transparent to us, let support all of them and not restrict the users to use specific ones.
---
[GitHub] incubator-hawq pull request #1322: Support Hive OpenCSVSerde
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone closed the pull request at:
https://github.com/apache/incubator-hawq/pull/1322
---
[GitHub] incubator-hawq pull request #1322: Support Hive OpenCSVSerde
Posted by lavjain <gi...@git.apache.org>.
Github user lavjain commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1322#discussion_r156481970
--- Diff: pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveLineBreakAccessor.java ---
@@ -43,7 +43,7 @@
public HiveLineBreakAccessor(InputData input) throws Exception {
super(input, new TextInputFormat());
((TextInputFormat) inputFormat).configure(jobConf);
- HiveUserData hiveUserData = HiveUtilities.parseHiveUserData(input, PXF_HIVE_SERDES.LAZY_SIMPLE_SERDE);
+ HiveUserData hiveUserData = HiveUtilities.parseHiveUserData(input, PXF_HIVE_SERDES.LAZY_SIMPLE_SERDE, PXF_HIVE_SERDES.CSV_SERDE);
--- End diff --
I second that. We should support all serdes (Hive is responsible for them)
---