[jira] Commented: (HIVE-785) Add RecordWriter for ScriptOperator

["Zheng Shao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750039#action_12750039 ] 

Zheng Shao commented on HIVE-785:
---------------------------------

@HIVE-785.2.patch:

{code}
+++ common/src/java/org/apache/hadoop/hive/conf/HiveConf.java	(working copy)
+    HIVESCRIPTRECORDREADER("hive.script.recordreader", "org.apache.hadoop.hive.contrib.util.typedbytes.TextRecordReader"),
+    HIVESCRIPTRECORDWRITER("hive.script.recordwriter", "org.apache.hadoop.hive.contrib.util.typedbytes.TextRecordWriter"),
{code}
TextRecordReader/Writable is in org.apache.hadoop.hive.ql.exec package.


{code}
+public class TypedBytesWritableOutput {
...
+  public void write(Writable w) throws IOException {
+    if (w instanceof TypedBytesWritable) {
+      writeTypedBytes((TypedBytesWritable) w);
+    } else if (w instanceof BytesWritable) {
+      writeBytes((BytesWritable) w);
+    } else if (w instanceof ByteWritable) {
...
{code}

This write method is not very efficient. I am OK with leaving it as it is (I guess it's from hadoop?), or we can optimize it by using object + objectInspector.


> Add RecordWriter for ScriptOperator
> -----------------------------------
>
>                 Key: HIVE-785
>                 URL: https://issues.apache.org/jira/browse/HIVE-785
>             Project: Hadoop Hive
>          Issue Type: New Feature
>    Affects Versions: 0.5.0
>            Reporter: Zheng Shao
>            Assignee: Namit Jain
>         Attachments: hive.785.1.patch, hive.785.2.patch
>
>
> HIVE-708 added RecordReader, but it is hardcoding a "RecordWriter" that uses newline for Text and write out data directly for BytesWritable.
> We should make this configurable as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.