You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Qiang Wang <ws...@gmail.com> on 2012/12/16 11:23:06 UTC
HiveHistory and HiveHistoryViewer
Hi All:
As I know, HiveHistory.log will replace '\n' with space before writing
Entry.value to history file:
* val = val.replace('\n', ' ');*
but HiveHistoryViewer use BufferedReader.readLine which takes '\n', '\r',
'\r\n' as line delimiter to parse history file
if val contains '\r', there is a high possibility
that HiveHistoryViewer.parseLine will fail, in which case usually
RecordTypes.valueOf(recType) will throw exception
'java.lang.IllegalArgumentException'
is this a bug ? Does HiveHistory.log need to replace '\r' with space as
well?
Re: HiveHistory and HiveHistoryViewer
Posted by Mark Grover <gr...@gmail.com>.
Qiang,
Good point. Uploaded a new patch.
Thanks!
On Mon, Dec 17, 2012 at 9:14 PM, Qiang Wang <ws...@gmail.com> wrote:
> "HiveHistory.parseHiveHistory use BufferedReader.readLine which takes '\n',
> '\r', '\r\n' as line delimiter to parse history file"
>
> And clients may be on mac, which takes '\r' as line delimiter
>
> So I think '\r' should also be replaced with space in HiveHistory.log, so
> that HiveHistory.parseHiveHistory could be consistent with HiveHistory.log
> and allow clients from mac
>
> Thanks!
>
>
> 2012/12/18 Mark Grover <gr...@gmail.com>
>>
>> Looks like a bug to me. This is the original JIRA that introduced this
>> change:
>> https://issues.apache.org/jira/browse/HIVE-176
>>
>> I don't think back in the day, we really cared about clients being on
>> windows.
>>
>> In any case, thanks for filing the JIRA, I have uploaded a patch which
>> I think doesn't break anything for linux clients and fixes things up
>> for Windows clients. Take a look, feedback welcome. The intent is the
>> same as your suggestions but the approach is a little more
>> conservative. If you feel strongly that it should be done according to
>> one of your suggestions, let me know, I will take another look.
>>
>> Thanks!
>> Mark
>>
>> On Mon, Dec 17, 2012 at 5:48 AM, Qiang Wang <ws...@gmail.com> wrote:
>> > anybody has an idea about this ?
>> >
>> > https://issues.apache.org/jira/browse/HIVE-3810
>> >
>> >
>> >
>> > 2012/12/16 Qiang Wang <ws...@gmail.com>
>> >>
>> >> glad to receive your reply!
>> >>
>> >> here is my point:
>> >> Firstly, I think HiveHistoryViewer is inconsistent with HiveHistory.
>> >> Secondly, hive server may be deloyed on linux, but client can be
>> >> anywhere.
>> >> hql from client will be logged into history file and hql may contails
>> >> '\r'
>> >>
>> >>
>> >> 2012/12/16 afancy <gr...@gmail.com>
>> >>>
>> >>> I don\t think it is a bug. If the program in hive writes logs to
>> >>> HiveHistory.log using '\n' to indicate the end of a line. Then, it is
>> >>> OK to
>> >>> use val = val.replace('\n', ' ');. Anyway,
>> >>> new line depends what on your OS: Hive is typically deployed on Linux.
>> >>> DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
>> >>>
>> >>> Unix & Mac OS X: \n, 0A, 10
>> >>> Macintosh (OS 9): \r, 0D, 13
>> >>>
>> >>> On Sun, Dec 16, 2012 at 11:23 AM, Qiang Wang <ws...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> '\n', '\r',
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >
>
>
Re: HiveHistory and HiveHistoryViewer
Posted by Qiang Wang <ws...@gmail.com>.
"HiveHistory.parseHiveHistory use BufferedReader.readLine which takes '\n',
'\r', '\r\n' as line delimiter to parse history file"
And clients may be on mac, which takes '\r' as line delimiter
So I think '\r' should also be replaced with space in HiveHistory.log, so
that HiveHistory.parseHiveHistory could be consistent with HiveHistory.log
and allow clients from mac
Thanks!
2012/12/18 Mark Grover <gr...@gmail.com>
> Looks like a bug to me. This is the original JIRA that introduced this
> change:
> https://issues.apache.org/jira/browse/HIVE-176
>
> I don't think back in the day, we really cared about clients being on
> windows.
>
> In any case, thanks for filing the JIRA, I have uploaded a patch which
> I think doesn't break anything for linux clients and fixes things up
> for Windows clients. Take a look, feedback welcome. The intent is the
> same as your suggestions but the approach is a little more
> conservative. If you feel strongly that it should be done according to
> one of your suggestions, let me know, I will take another look.
>
> Thanks!
> Mark
>
> On Mon, Dec 17, 2012 at 5:48 AM, Qiang Wang <ws...@gmail.com> wrote:
> > anybody has an idea about this ?
> >
> > https://issues.apache.org/jira/browse/HIVE-3810
> >
> >
> >
> > 2012/12/16 Qiang Wang <ws...@gmail.com>
> >>
> >> glad to receive your reply!
> >>
> >> here is my point:
> >> Firstly, I think HiveHistoryViewer is inconsistent with HiveHistory.
> >> Secondly, hive server may be deloyed on linux, but client can be
> anywhere.
> >> hql from client will be logged into history file and hql may contails
> '\r'
> >>
> >>
> >> 2012/12/16 afancy <gr...@gmail.com>
> >>>
> >>> I don\t think it is a bug. If the program in hive writes logs to
> >>> HiveHistory.log using '\n' to indicate the end of a line. Then, it is
> OK to
> >>> use val = val.replace('\n', ' ');. Anyway,
> >>> new line depends what on your OS: Hive is typically deployed on Linux.
> >>> DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
> >>>
> >>> Unix & Mac OS X: \n, 0A, 10
> >>> Macintosh (OS 9): \r, 0D, 13
> >>>
> >>> On Sun, Dec 16, 2012 at 11:23 AM, Qiang Wang <ws...@gmail.com>
> wrote:
> >>>>
> >>>> '\n', '\r',
> >>>
> >>>
> >>>
> >>>
> >>
> >
>
Re: HiveHistory and HiveHistoryViewer
Posted by Mark Grover <gr...@gmail.com>.
Looks like a bug to me. This is the original JIRA that introduced this change:
https://issues.apache.org/jira/browse/HIVE-176
I don't think back in the day, we really cared about clients being on windows.
In any case, thanks for filing the JIRA, I have uploaded a patch which
I think doesn't break anything for linux clients and fixes things up
for Windows clients. Take a look, feedback welcome. The intent is the
same as your suggestions but the approach is a little more
conservative. If you feel strongly that it should be done according to
one of your suggestions, let me know, I will take another look.
Thanks!
Mark
On Mon, Dec 17, 2012 at 5:48 AM, Qiang Wang <ws...@gmail.com> wrote:
> anybody has an idea about this ?
>
> https://issues.apache.org/jira/browse/HIVE-3810
>
>
>
> 2012/12/16 Qiang Wang <ws...@gmail.com>
>>
>> glad to receive your reply!
>>
>> here is my point:
>> Firstly, I think HiveHistoryViewer is inconsistent with HiveHistory.
>> Secondly, hive server may be deloyed on linux, but client can be anywhere.
>> hql from client will be logged into history file and hql may contails '\r'
>>
>>
>> 2012/12/16 afancy <gr...@gmail.com>
>>>
>>> I don\t think it is a bug. If the program in hive writes logs to
>>> HiveHistory.log using '\n' to indicate the end of a line. Then, it is OK to
>>> use val = val.replace('\n', ' ');. Anyway,
>>> new line depends what on your OS: Hive is typically deployed on Linux.
>>> DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
>>>
>>> Unix & Mac OS X: \n, 0A, 10
>>> Macintosh (OS 9): \r, 0D, 13
>>>
>>> On Sun, Dec 16, 2012 at 11:23 AM, Qiang Wang <ws...@gmail.com> wrote:
>>>>
>>>> '\n', '\r',
>>>
>>>
>>>
>>>
>>
>
Re: HiveHistory and HiveHistoryViewer
Posted by Qiang Wang <ws...@gmail.com>.
anybody has an idea about this ?
https://issues.apache.org/jira/browse/HIVE-3810
2012/12/16 Qiang Wang <ws...@gmail.com>
> glad to receive your reply!
>
> here is my point:
> Firstly, I think HiveHistoryViewer is inconsistent with HiveHistory.
> Secondly, hive server may be deloyed on linux, but client can be anywhere.
> hql from client will be logged into history file and hql may contails '\r'
>
>
> 2012/12/16 afancy <gr...@gmail.com>
>
>> I don\t think it is a bug. If the program in hive writes logs to
>> HiveHistory.log using '\n' to indicate the end of a line. Then, it is OK
>> to use *val = val.replace('\n', ' ');. Anyway, **
>> new line depends what on your OS:<https://ccrma.stanford.edu/~craig/utility/flip/> Hive
>> is typically deployed on Linux.
>> *
>> *
>> DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
>>
>> Unix & Mac OS X: \n, 0A, 10
>> Macintosh (OS 9): \r, 0D, 13
>> *
>> On Sun, Dec 16, 2012 at 11:23 AM, Qiang Wang <ws...@gmail.com> wrote:
>>
>>> '\n', '\r',
>>
>>
>>
>>
>>
>
Re: HiveHistory and HiveHistoryViewer
Posted by Qiang Wang <ws...@gmail.com>.
glad to receive your reply!
here is my point:
Firstly, I think HiveHistoryViewer is inconsistent with HiveHistory.
Secondly, hive server may be deloyed on linux, but client can be anywhere.
hql from client will be logged into history file and hql may contails '\r'
2012/12/16 afancy <gr...@gmail.com>
> I don\t think it is a bug. If the program in hive writes logs to
> HiveHistory.log using '\n' to indicate the end of a line. Then, it is OK
> to use *val = val.replace('\n', ' ');. Anyway, **
> new line depends what on your OS:<https://ccrma.stanford.edu/~craig/utility/flip/> Hive
> is typically deployed on Linux.
> *
> *
> DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
>
> Unix & Mac OS X: \n, 0A, 10
> Macintosh (OS 9): \r, 0D, 13
> *
> On Sun, Dec 16, 2012 at 11:23 AM, Qiang Wang <ws...@gmail.com> wrote:
>
>> '\n', '\r',
>
>
>
>
>
Re: HiveHistory and HiveHistoryViewer
Posted by afancy <gr...@gmail.com>.
I don\t think it is a bug. If the program in hive writes logs to
HiveHistory.log using '\n' to indicate the end of a line. Then, it is OK to
use *val = val.replace('\n', ' ');. Anyway, **
new line depends what on your
OS:<https://ccrma.stanford.edu/~craig/utility/flip/> Hive
is typically deployed on Linux.
*
*
DOS & Windows: \r\n 0D0A (hex), 13,10 (decimal)
Unix & Mac OS X: \n, 0A, 10
Macintosh (OS 9): \r, 0D, 13
*
On Sun, Dec 16, 2012 at 11:23 AM, Qiang Wang <ws...@gmail.com> wrote:
> '\n', '\r',