You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Paul Yang (JIRA)" <ji...@apache.org> on 2010/09/21 03:21:32 UTC
[jira] Created: (HIVE-1658) Fix describe [extended] column
formatting
Fix describe [extended] column formatting
-----------------------------------------
Key: HIVE-1658
URL: https://issues.apache.org/jira/browse/HIVE-1658
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Paul Yang
Assignee: Thiruvel Thirumoolan
When displaying the column schema, the formatting should follow should be
name<TAB>type<TAB>comment<NEWLINE>
to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913668#action_12913668 ]
Ning Zhang commented on HIVE-1658:
----------------------------------
+1 on keeping the old format but add a "pretty operator" as the child of the explain, so that the execution plan for the EXPLAIN is an explain operator (with the old formatting) followed by an optional "pretty operator" taking the output and do further formatting.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916992#action_12916992 ]
Thiruvel Thirumoolan commented on HIVE-1658:
--------------------------------------------
Patch under works.
Changes:
1. 'describe' & 'describe extended' outputs will be the same as pre HIVE-558.
2. 'describe formatted' will use the new format for displaying columns and additional information.
Will implement the changes similar to how extended is implemented, using a boolean in DescTableDesc to denote the formatted keyword and formatting the output in DDLTask.describeTable.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917725#action_12917725 ]
He Yongqiang commented on HIVE-1658:
------------------------------------
+1. Looks good. Can you do the final patch?
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvel Thirumoolan updated HIVE-1658:
---------------------------------------
Attachment: HIVE-1658_1.patch
Cleaned up the code a little bit, will display UNKNOWN if time is 0. Will upload patch for test output tomm.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920406#action_12920406 ]
Ning Zhang commented on HIVE-1658:
----------------------------------
+1. Will commit if tests pass.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch, HIVE-1658_3.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913662#action_12913662 ]
Namit Jain commented on HIVE-1658:
----------------------------------
@Thiruvel, can we keep the new output in the old format.
I mean, we just have to make sure that the output has 3 columns separated by a delimiter.
So, if your current output is 'x', you can replace it with:
<TAB>x<TAB>
An implicit null at the beginning and end.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913315#action_12913315 ]
Paul Yang commented on HIVE-1658:
---------------------------------
Maybe another option is to have a pretty qualifier? To get your output, the user would have to type something like 'pretty describe <tablename>'. Otherwise everything defaults to normal.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: blob handling in hive
Posted by Jinsong Hu <ji...@hotmail.com>.
Yes. tentatively that is what I have to do. another way is to convert the
data to base64 encoded
string. after client received the data, it needs to decode the data back to
binary. this is a hack, but works.
If hive supports byte array as native data type, then the solution is much
more elegant.
Jimmy.
--------------------------------------------------
From: "Ted Yu" <yu...@gmail.com>
Sent: Tuesday, October 12, 2010 4:33 PM
To: <de...@hive.apache.org>
Subject: Re: blob handling in hive
> How about utf-8 encode your blob and store in Hive as String ?
>
> On Tue, Oct 12, 2010 at 4:20 PM, Jinsong Hu <ji...@hotmail.com>
> wrote:
>
>> I thought about that too. but then I need to write an bytes inspector and
>> stick that into hive inspector factory. we also need to create a new
>> datatype , such as blob , in hive's supported
>> data types. Adding a new supported data type to hive is a non-trivial
>> task,
>> as more code will need to be touched.
>>
>> I am just wondering if it is possible to get what I want to do without
>> such
>> big change.
>>
>>
>>
>> Jimmy.
>>
>> --------------------------------------------------
>> From: "Ted Yu" <yu...@gmail.com>
>> Sent: Tuesday, October 12, 2010 4:12 PM
>>
>> To: <de...@hive.apache.org>
>> Subject: Re: blob handling in hive
>>
>> How about creating org.apache.hadoop.hive.serde2.io.BytesWritable which
>>> wraps byte[] ?
>>>
>>> On Tue, Oct 12, 2010 at 3:49 PM, Jinsong Hu <ji...@hotmail.com>
>>> wrote:
>>>
>>> storing the blob in hbase is too costly. hbase compaction costs lots of
>>>> cpu. All I want to do is to be able to read the byte array out of a
>>>> sequence
>>>> file, and map that byte array to an hive column.
>>>> I can write a SerDe for this purpose.
>>>>
>>>> I tried to define the data to be array<tinyint>. I then tried to write
>>>> custom SerDe, after I get the byte array out of the disk, I need to
>>>> map
>>>> it,
>>>>
>>>> so I wrote the code:
>>>> columnTypes
>>>> =TypeInfoUtils.getTypeInfosFromTypeString("int,string,array<tinyint>");
>>>>
>>>> but then how to I convert the data in the row.set() method ?
>>>>
>>>> I tried this:
>>>>
>>>> byte [] bContent=ev.get_content()==null ? null :
>>>> (ev.get_content().getData()==null ? null : ev.get_content().getData());
>>>> org.apache.hadoop.hive.serde2.io.ByteWritable tContent =
>>>> bContent==null ? new org.apache.hadoop.hive.serde2.io.ByteWritable() :
>>>> new
>>>> org.apache.hadoop.hive.serde2.io.ByteWritable(bContent[0]) ;
>>>> row.set(2, tContent);
>>>>
>>>> this works for a single byte, but doesn't work for byte array.
>>>> Any way that I can get the byte array returned in sql is appreciated.
>>>>
>>>> Jimmy
>>>>
>>>> --------------------------------------------------
>>>> From: "Ted Yu" <yu...@gmail.com>
>>>> Sent: Tuesday, October 12, 2010 2:19 PM
>>>> To: <de...@hive.apache.org>
>>>> Subject: Re: blob handling in hive
>>>>
>>>>
>>>> One way is to store blob in HBase and use HBaseHandler to access your
>>>>
>>>>> blob.
>>>>>
>>>>> On Tue, Oct 12, 2010 at 2:14 PM, Jinsong Hu <ji...@hotmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>> I am using sqoop to export data from mysql to hive. I noticed that
>>>>>> hive
>>>>>> don't have blob data type yet. is there anyway I can do so hive can
>>>>>> store
>>>>>> blob ?
>>>>>>
>>>>>> Jimmy
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>
Re: blob handling in hive
Posted by Ted Yu <yu...@gmail.com>.
How about utf-8 encode your blob and store in Hive as String ?
On Tue, Oct 12, 2010 at 4:20 PM, Jinsong Hu <ji...@hotmail.com> wrote:
> I thought about that too. but then I need to write an bytes inspector and
> stick that into hive inspector factory. we also need to create a new
> datatype , such as blob , in hive's supported
> data types. Adding a new supported data type to hive is a non-trivial task,
> as more code will need to be touched.
>
> I am just wondering if it is possible to get what I want to do without such
> big change.
>
>
>
> Jimmy.
>
> --------------------------------------------------
> From: "Ted Yu" <yu...@gmail.com>
> Sent: Tuesday, October 12, 2010 4:12 PM
>
> To: <de...@hive.apache.org>
> Subject: Re: blob handling in hive
>
> How about creating org.apache.hadoop.hive.serde2.io.BytesWritable which
>> wraps byte[] ?
>>
>> On Tue, Oct 12, 2010 at 3:49 PM, Jinsong Hu <ji...@hotmail.com>
>> wrote:
>>
>> storing the blob in hbase is too costly. hbase compaction costs lots of
>>> cpu. All I want to do is to be able to read the byte array out of a
>>> sequence
>>> file, and map that byte array to an hive column.
>>> I can write a SerDe for this purpose.
>>>
>>> I tried to define the data to be array<tinyint>. I then tried to write
>>> custom SerDe, after I get the byte array out of the disk, I need to
>>> map
>>> it,
>>>
>>> so I wrote the code:
>>> columnTypes
>>> =TypeInfoUtils.getTypeInfosFromTypeString("int,string,array<tinyint>");
>>>
>>> but then how to I convert the data in the row.set() method ?
>>>
>>> I tried this:
>>>
>>> byte [] bContent=ev.get_content()==null ? null :
>>> (ev.get_content().getData()==null ? null : ev.get_content().getData());
>>> org.apache.hadoop.hive.serde2.io.ByteWritable tContent =
>>> bContent==null ? new org.apache.hadoop.hive.serde2.io.ByteWritable() :
>>> new
>>> org.apache.hadoop.hive.serde2.io.ByteWritable(bContent[0]) ;
>>> row.set(2, tContent);
>>>
>>> this works for a single byte, but doesn't work for byte array.
>>> Any way that I can get the byte array returned in sql is appreciated.
>>>
>>> Jimmy
>>>
>>> --------------------------------------------------
>>> From: "Ted Yu" <yu...@gmail.com>
>>> Sent: Tuesday, October 12, 2010 2:19 PM
>>> To: <de...@hive.apache.org>
>>> Subject: Re: blob handling in hive
>>>
>>>
>>> One way is to store blob in HBase and use HBaseHandler to access your
>>>
>>>> blob.
>>>>
>>>> On Tue, Oct 12, 2010 at 2:14 PM, Jinsong Hu <ji...@hotmail.com>
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>> I am using sqoop to export data from mysql to hive. I noticed that
>>>>> hive
>>>>> don't have blob data type yet. is there anyway I can do so hive can
>>>>> store
>>>>> blob ?
>>>>>
>>>>> Jimmy
>>>>>
>>>>>
>>>>>
>>>>
>>
Re: blob handling in hive
Posted by Jinsong Hu <ji...@hotmail.com>.
I thought about that too. but then I need to write an bytes inspector and
stick that into hive inspector factory. we also need to create a new
datatype , such as blob , in hive's supported
data types. Adding a new supported data type to hive is a non-trivial task,
as more code will need to be touched.
I am just wondering if it is possible to get what I want to do without such
big change.
Jimmy.
--------------------------------------------------
From: "Ted Yu" <yu...@gmail.com>
Sent: Tuesday, October 12, 2010 4:12 PM
To: <de...@hive.apache.org>
Subject: Re: blob handling in hive
> How about creating org.apache.hadoop.hive.serde2.io.BytesWritable which
> wraps byte[] ?
>
> On Tue, Oct 12, 2010 at 3:49 PM, Jinsong Hu <ji...@hotmail.com>
> wrote:
>
>> storing the blob in hbase is too costly. hbase compaction costs lots of
>> cpu. All I want to do is to be able to read the byte array out of a
>> sequence
>> file, and map that byte array to an hive column.
>> I can write a SerDe for this purpose.
>>
>> I tried to define the data to be array<tinyint>. I then tried to write
>> custom SerDe, after I get the byte array out of the disk, I need to
>> map
>> it,
>>
>> so I wrote the code:
>> columnTypes
>> =TypeInfoUtils.getTypeInfosFromTypeString("int,string,array<tinyint>");
>>
>> but then how to I convert the data in the row.set() method ?
>>
>> I tried this:
>>
>> byte [] bContent=ev.get_content()==null ? null :
>> (ev.get_content().getData()==null ? null : ev.get_content().getData());
>> org.apache.hadoop.hive.serde2.io.ByteWritable tContent =
>> bContent==null ? new org.apache.hadoop.hive.serde2.io.ByteWritable() :
>> new
>> org.apache.hadoop.hive.serde2.io.ByteWritable(bContent[0]) ;
>> row.set(2, tContent);
>>
>> this works for a single byte, but doesn't work for byte array.
>> Any way that I can get the byte array returned in sql is appreciated.
>>
>> Jimmy
>>
>> --------------------------------------------------
>> From: "Ted Yu" <yu...@gmail.com>
>> Sent: Tuesday, October 12, 2010 2:19 PM
>> To: <de...@hive.apache.org>
>> Subject: Re: blob handling in hive
>>
>>
>> One way is to store blob in HBase and use HBaseHandler to access your
>>> blob.
>>>
>>> On Tue, Oct 12, 2010 at 2:14 PM, Jinsong Hu <ji...@hotmail.com>
>>> wrote:
>>>
>>> Hi,
>>>> I am using sqoop to export data from mysql to hive. I noticed that
>>>> hive
>>>> don't have blob data type yet. is there anyway I can do so hive can
>>>> store
>>>> blob ?
>>>>
>>>> Jimmy
>>>>
>>>>
>>>
>
Re: blob handling in hive
Posted by Ted Yu <yu...@gmail.com>.
How about creating org.apache.hadoop.hive.serde2.io.BytesWritable which
wraps byte[] ?
On Tue, Oct 12, 2010 at 3:49 PM, Jinsong Hu <ji...@hotmail.com> wrote:
> storing the blob in hbase is too costly. hbase compaction costs lots of
> cpu. All I want to do is to be able to read the byte array out of a sequence
> file, and map that byte array to an hive column.
> I can write a SerDe for this purpose.
>
> I tried to define the data to be array<tinyint>. I then tried to write
> custom SerDe, after I get the byte array out of the disk, I need to map
> it,
>
> so I wrote the code:
> columnTypes
> =TypeInfoUtils.getTypeInfosFromTypeString("int,string,array<tinyint>");
>
> but then how to I convert the data in the row.set() method ?
>
> I tried this:
>
> byte [] bContent=ev.get_content()==null ? null :
> (ev.get_content().getData()==null ? null : ev.get_content().getData());
> org.apache.hadoop.hive.serde2.io.ByteWritable tContent =
> bContent==null ? new org.apache.hadoop.hive.serde2.io.ByteWritable() : new
> org.apache.hadoop.hive.serde2.io.ByteWritable(bContent[0]) ;
> row.set(2, tContent);
>
> this works for a single byte, but doesn't work for byte array.
> Any way that I can get the byte array returned in sql is appreciated.
>
> Jimmy
>
> --------------------------------------------------
> From: "Ted Yu" <yu...@gmail.com>
> Sent: Tuesday, October 12, 2010 2:19 PM
> To: <de...@hive.apache.org>
> Subject: Re: blob handling in hive
>
>
> One way is to store blob in HBase and use HBaseHandler to access your
>> blob.
>>
>> On Tue, Oct 12, 2010 at 2:14 PM, Jinsong Hu <ji...@hotmail.com>
>> wrote:
>>
>> Hi,
>>> I am using sqoop to export data from mysql to hive. I noticed that hive
>>> don't have blob data type yet. is there anyway I can do so hive can store
>>> blob ?
>>>
>>> Jimmy
>>>
>>>
>>
Re: blob handling in hive
Posted by Jinsong Hu <ji...@hotmail.com>.
storing the blob in hbase is too costly. hbase compaction costs lots of
cpu. All I want to do is to be able to read the byte array out of a sequence
file, and map that byte array to an hive column.
I can write a SerDe for this purpose.
I tried to define the data to be array<tinyint>. I then tried to write
custom SerDe, after I get the byte array out of the disk, I need to map
it,
so I wrote the code:
columnTypes
=TypeInfoUtils.getTypeInfosFromTypeString("int,string,array<tinyint>");
but then how to I convert the data in the row.set() method ?
I tried this:
byte [] bContent=ev.get_content()==null ? null :
(ev.get_content().getData()==null ? null : ev.get_content().getData());
org.apache.hadoop.hive.serde2.io.ByteWritable tContent =
bContent==null ? new org.apache.hadoop.hive.serde2.io.ByteWritable() : new
org.apache.hadoop.hive.serde2.io.ByteWritable(bContent[0]) ;
row.set(2, tContent);
this works for a single byte, but doesn't work for byte array.
Any way that I can get the byte array returned in sql is appreciated.
Jimmy
--------------------------------------------------
From: "Ted Yu" <yu...@gmail.com>
Sent: Tuesday, October 12, 2010 2:19 PM
To: <de...@hive.apache.org>
Subject: Re: blob handling in hive
> One way is to store blob in HBase and use HBaseHandler to access your
> blob.
>
> On Tue, Oct 12, 2010 at 2:14 PM, Jinsong Hu <ji...@hotmail.com>
> wrote:
>
>> Hi,
>> I am using sqoop to export data from mysql to hive. I noticed that hive
>> don't have blob data type yet. is there anyway I can do so hive can store
>> blob ?
>>
>> Jimmy
>>
>
Re: blob handling in hive
Posted by Ted Yu <yu...@gmail.com>.
One way is to store blob in HBase and use HBaseHandler to access your blob.
On Tue, Oct 12, 2010 at 2:14 PM, Jinsong Hu <ji...@hotmail.com> wrote:
> Hi,
> I am using sqoop to export data from mysql to hive. I noticed that hive
> don't have blob data type yet. is there anyway I can do so hive can store
> blob ?
>
> Jimmy
>
blob handling in hive
Posted by Jinsong Hu <ji...@hotmail.com>.
Hi,
I am using sqoop to export data from mysql to hive. I noticed that hive
don't have blob data type yet. is there anyway I can do so hive can store
blob ?
Jimmy
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920346#action_12920346 ]
Ning Zhang commented on HIVE-1658:
----------------------------------
Thanks Thiruvel! I will take a look. Does this patch subsume HIVE-1669?
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915437#action_12915437 ]
Ning Zhang commented on HIVE-1658:
----------------------------------
Another issue is that now 'desc extended' displays table/partition parameters in different lines. Since parameters is using a unordered map implementation, it will give non-deterministic display of those parameters. It will be great if the pretty operator will take care of ordering as well.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvel Thirumoolan updated HIVE-1658:
---------------------------------------
Attachment: HIVE-1658_3.patch
Changelog:
1. Updated code changes attached. Fixed a bug in DDLTask that was in last patch.
2. Added describe_table.q test to test various desc scenarios.
3. Updated create_view.q to also include 'desc formatted' for views.
4. hbase_stats.q had a bug and was failing from a fresh checkout too. Have fixed it.
5. All other changes are updates to test output.
desc table - This retains the old behavior.
<field name><tab><type><tab><comment>
<field name><tab><type><tab> # if no comment
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch, HIVE-1658_3.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912809#action_12912809 ]
Paul Yang commented on HIVE-1658:
---------------------------------
Actually, this JIRA is more about handling backward compatibility. It's possible that there things (JDBC? scripts?) that parsed the text output and depended on the old style.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919309#action_12919309 ]
Thiruvel Thirumoolan commented on HIVE-1658:
--------------------------------------------
Was writing shell scripts this week that rely on the schema output and there is no straight forward way to differentiate partitions and columns.
What do folks think on adding another column "flags" that displays "-"/"part"? We can add additional tags too later (sort..) if necessary. If I can't get to this tomm, will address it through another JIRA.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913043#action_12913043 ]
Thiruvel Thirumoolan commented on HIVE-1658:
--------------------------------------------
ok, will revert back the formatting of the columns (partitions also) and will leave the rest of the changes as is. That would mean the headers also will go away, the formatting doesnt look good with them and tab alone as separator.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920385#action_12920385 ]
Thiruvel Thirumoolan commented on HIVE-1658:
--------------------------------------------
@ Ning: Yes, HIVE-1658_3.patch includes HIVE-1669 changes too.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch, HIVE-1658_3.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917771#action_12917771 ]
He Yongqiang commented on HIVE-1658:
------------------------------------
one more thing, if the time information (create time, last access time etc) is 0, can you put some string like "unknown" to the output of desc format?
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920189#action_12920189 ]
Thiruvel Thirumoolan commented on HIVE-1658:
--------------------------------------------
The patch is ready and tests are running. Will upload patch when tests complete.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang resolved HIVE-1658.
------------------------------
Resolution: Fixed
Committed. Thanks Thiruvel!
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch, HIVE-1658_1.patch, HIVE-1658_3.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912846#action_12912846 ]
Namit Jain commented on HIVE-1658:
----------------------------------
Thiruvel, this is a show-stopper for HIVE-558.
The schema for describe and describe extended cannot be changed.
You can add NULLs at the beginning/end, but the number of columns have to be maintained
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916513#action_12916513 ]
Thiruvel Thirumoolan commented on HIVE-1658:
--------------------------------------------
Sorry folks, was out sick for more than a week. Will upload a patch tomorrow.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Thiruvel Thirumoolan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvel Thirumoolan updated HIVE-1658:
---------------------------------------
Attachment: HIVE-1658-PrelimPatch.patch
Preliminary patch on the above mentioned approach - felt this one to be easier. Comments welcome.
The code needs to be reorganized and cleaned, but I wanted to upload patch before I sign off for the day. Will proceed with test cases on confirmation of the approach.
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
> Attachments: HIVE-1658-PrelimPatch.patch
>
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1658) Fix describe [extended] column
formatting
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916207#action_12916207 ]
Namit Jain commented on HIVE-1658:
----------------------------------
Thiruvel, any updates on this - we need it urgently in order to deploy HIVE-558
> Fix describe [extended] column formatting
> -----------------------------------------
>
> Key: HIVE-1658
> URL: https://issues.apache.org/jira/browse/HIVE-1658
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Thiruvel Thirumoolan
>
> When displaying the column schema, the formatting should follow should be
> name<TAB>type<TAB>comment<NEWLINE>
> to be inline with the previous formatting style for backward compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.