[jira] [Updated] (HIVE-14867) "serialization.last.column.takes.rest" does not work for MultiDelimitSerDe

["Liu Weizheng (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-14867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Liu Weizheng updated HIVE-14867:
--------------------------------
    Description: 
Create table with MultiDelimitSerde:
{code:java}
CREATE TABLE foo (a string, b string) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ("field.delim"="|@|","collection.delim"=":","mapkey.delim"="@") stored as textfile;
{code}
load data into table:
{code:java}
1|@|Lily|@|HW|@|abc
2|@|Lucy|@|LX|@|123
3|@|Lilei|@|XX|@|3434
{code}
select data from this table:
{code:java}
select * from foo;
+---------+----------------+--+
| foo.a  |     foo.b     |
+---------+----------------+--+
| 1       | Lily
^AHW^A
abc    |
| 2       | Lucy
^ALX
^A123    |
| 3       | Lilei
^AXX
^A3434  |
+---------+----------------+--+
3 rows selected (0.905 seconds)
{code}
You can see the last column takes all the data, and replace the delimiter to default ^A.

lastColumnTakesRestString should be false by default:
{code:java}
    String lastColumnTakesRestString = tbl
        .getProperty(serdeConstants.SERIALIZATION_LAST_COLUMN_TAKES_REST);
    lastColumnTakesRest = (lastColumnTakesRestString != null && lastColumnTakesRestString
        .equalsIgnoreCase("true"));
{code}
 

  was:
Create table with MultiDelimitSerde:
{code}
CREATE TABLE foo (a string, b string) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ("field.delim"="|@|","collection.delim"=":","mapkey.delim"="@") stored as textfile;
{code}

load data into table:
{code}
1|@|Lily|@|HW|@|abc
2|@|Lucy|@|LX|@|123
3|@|Lilei|@|XX|@|3434
{code}

select data from this table:
{code}
select * from foo;
+---------+----------------+--+
| foo.a  |     foo.b     |
+---------+----------------+--+
| 1       | Lily
^AHW^A
abc    |
| 2       | Lucy
^ALX
^A123    |
| 3       | Lilei
^AXX
^A3434  |
+---------+----------------+--+
3 rows selected (0.905 seconds)
{code}

You can see the last column takes all the data, and replace the delimiter to default ^A.

lastColumnTakesRestString should be false by default: 
{code}
    String lastColumnTakesRestString = tbl
        .getProperty(serdeConstants.SERIALIZATION_LAST_COLUMN_TAKES_REST);
    lastColumnTakesRest = (lastColumnTakesRestString != null && lastColumnTakesRestString
        .equalsIgnoreCase("true"));
{code}



> "serialization.last.column.takes.rest" does not work for MultiDelimitSerDe
> --------------------------------------------------------------------------
>
>                 Key: HIVE-14867
>                 URL: https://issues.apache.org/jira/browse/HIVE-14867
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 1.3.0
>            Reporter: Niklaus Xiao
>            Assignee: Niklaus Xiao
>            Priority: Major
>
> Create table with MultiDelimitSerde:
> {code:java}
> CREATE TABLE foo (a string, b string) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ("field.delim"="|@|","collection.delim"=":","mapkey.delim"="@") stored as textfile;
> {code}
> load data into table:
> {code:java}
> 1|@|Lily|@|HW|@|abc
> 2|@|Lucy|@|LX|@|123
> 3|@|Lilei|@|XX|@|3434
> {code}
> select data from this table:
> {code:java}
> select * from foo;
> +---------+----------------+--+
> | foo.a  |     foo.b     |
> +---------+----------------+--+
> | 1       | Lily
^AHW^A
abc    |
> | 2       | Lucy
^ALX
^A123    |
> | 3       | Lilei
^AXX
^A3434  |
> +---------+----------------+--+
> 3 rows selected (0.905 seconds)
> {code}
> You can see the last column takes all the data, and replace the delimiter to default ^A.
> lastColumnTakesRestString should be false by default:
> {code:java}
>     String lastColumnTakesRestString = tbl
>         .getProperty(serdeConstants.SERIALIZATION_LAST_COLUMN_TAKES_REST);
>     lastColumnTakesRest = (lastColumnTakesRestString != null && lastColumnTakesRestString
>         .equalsIgnoreCase("true"));
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)