You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by 郭士伟 <gu...@gmail.com> on 2014/05/24 10:01:35 UTC
Hive can I contribute to Hive confluence wiki documents?
I have used a configuration property 'hive.cache.expr.evaluation' in hive
0.12 recently and found it useful. But I cannot find any documents about it
on the wiki page:
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
Indeed, I pick it up from the source code.
I think it maybe useful for other hive users if I can write some documents
about the property.
So, can anyone give me some pointers that how can I make the change happen ?
Thanks, anyway.
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by 郭士伟 <gu...@gmail.com>.
Sorry for no reply for so long.
I found this feature can cause some problem if some UDF did not implemented
properly.
When we using UDF, this feature requires that a UDF implement a
'getDisplayString' method which returns a unique string that can represent
the UDF and it's input parameters. The unique 'displayString' is used as
key for the evaluation cache. So, problem happen when those two criteria
are satisfied:
1. We use more than one UDF in one hql session
2. The 'getDisplayString' method of the UDFs in use return same string for
different UDFs.
I think it's worth to point this dependency out in the UDF related
documents or 'cache.expr.evaluation' document.
2014-07-04 14:37 GMT+08:00 Lefty Leverenz <le...@gmail.com>:
> The 'hive.cache.expr.evaluation' parameter is documented in the wikidoc Configuration
> Properties
> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>.
> Have you added other information in a different wikidoc? If so, I'll link
> the docs to each other.
>
> -- Lefty
>
>
> On Thu, May 29, 2014 at 5:31 AM, Lefty Leverenz <le...@gmail.com>
> wrote:
>
>> Thank you for pointing this out. The 'hive.cache.expr.evaluation'
>> parameter was added by HIVE-4209
>> <https://issues.apache.org/jira/browse/HIVE-4209> and it needs to be
>> included in the wiki. Its description is in hive-default.xml.template:
>>
>> If true, evaluation result of deterministic expression referenced twice
>>> or more will be cached. For example, in filter condition like ".. where key
>>> + 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached once and
>>> reused for following expression ("key + 10 = 0"). Currently, this is
>>> applied only to expressions in select or filter operator.
>>
>>
>> Do you want to provide more information than that? To contribute to the
>> wiki, create a Confluence account
>> <https://cwiki.apache.org/confluence/signup.action> and send a message
>> to this list asking for wiki editing privilege and giving your account
>> name. But if you just want the description to be added to the wiki, I can
>> take care of that.
>>
>> If you have additional information, it might not belong in Configuration
>> Properties
>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties> (although
>> once the parameter is there it can link to another page for details). The
>> Select
>> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>
>> page is one possibility, or maybe you'll find a better place for it.
>>
>> If you want to revise the description in hive-default.xml.template, you
>> need to open a JIRA and patch the file.
>>
>>
>> -- Lefty
>>
>>
>> On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
>>
>>> I have used a configuration property 'hive.cache.expr.evaluation' in
>>> hive 0.12 recently and found it useful. But I cannot find any documents
>>> about it on the wiki page:
>>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
>>> Indeed, I pick it up from the source code.
>>>
>>> I think it maybe useful for other hive users if I can write some
>>> documents about the property.
>>>
>>> So, can anyone give me some pointers that how can I make the change
>>> happen ?
>>>
>>> Thanks, anyway.
>>>
>>
>>
>
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by 郭士伟 <gu...@gmail.com>.
Yes, I think.
2014-07-16 14:52 GMT+08:00 Lefty Leverenz <le...@gmail.com>:
> HIVE-7314 is documented in the Configuration Properties wiki (
> hive.cache.expr.evaluation
> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>)
> and at the beginning of the Operators and UDFs
> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF>
> wiki. The latter includes a link to this email thread.
>
> Does that take care of it?
>
> -- Lefty
>
>
> On Sun, Jul 13, 2014 at 3:57 AM, Navis류승우 <na...@nexr.com> wrote:
>
>> For "cache.expr.evaluation" problem, it's fixed in
>> https://issues.apache.org/jira/browse/HIVE-7314 (will be included in
>> next release)
>>
>> But I agree that it can be a critical problem for users of 0.12.0 and
>> 0.13.x version and need proper warning on that.
>>
>> Thanks,
>> Navis
>>
>>
>> 2014-07-13 16:48 GMT+09:00 郭士伟 <gu...@gmail.com>:
>>
>> And I haven't add other information in the wikidoc yet.
>>>
>>>
>>> 2014-07-04 14:37 GMT+08:00 Lefty Leverenz <le...@gmail.com>:
>>>
>>>> The 'hive.cache.expr.evaluation' parameter is documented in the wikidoc Configuration
>>>> Properties
>>>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>.
>>>> Have you added other information in a different wikidoc? If so, I'll link
>>>> the docs to each other.
>>>>
>>>> -- Lefty
>>>>
>>>>
>>>> On Thu, May 29, 2014 at 5:31 AM, Lefty Leverenz <
>>>> leftyleverenz@gmail.com> wrote:
>>>>
>>>>> Thank you for pointing this out. The 'hive.cache.expr.evaluation'
>>>>> parameter was added by HIVE-4209
>>>>> <https://issues.apache.org/jira/browse/HIVE-4209> and it needs to be
>>>>> included in the wiki. Its description is in hive-default.xml.template:
>>>>>
>>>>> If true, evaluation result of deterministic expression referenced
>>>>>> twice or more will be cached. For example, in filter condition like "..
>>>>>> where key + 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached
>>>>>> once and reused for following expression ("key + 10 = 0"). Currently, this
>>>>>> is applied only to expressions in select or filter operator.
>>>>>
>>>>>
>>>>> Do you want to provide more information than that? To contribute to
>>>>> the wiki, create a Confluence account
>>>>> <https://cwiki.apache.org/confluence/signup.action> and send a
>>>>> message to this list asking for wiki editing privilege and giving your
>>>>> account name. But if you just want the description to be added to the
>>>>> wiki, I can take care of that.
>>>>>
>>>>> If you have additional information, it might not belong in Configuration
>>>>> Properties
>>>>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties> (although
>>>>> once the parameter is there it can link to another page for details). The
>>>>> Select
>>>>> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>
>>>>> page is one possibility, or maybe you'll find a better place for it.
>>>>>
>>>>> If you want to revise the description in hive-default.xml.template,
>>>>> you need to open a JIRA and patch the file.
>>>>>
>>>>>
>>>>> -- Lefty
>>>>>
>>>>>
>>>>> On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
>>>>>
>>>>>> I have used a configuration property 'hive.cache.expr.evaluation' in
>>>>>> hive 0.12 recently and found it useful. But I cannot find any documents
>>>>>> about it on the wiki page:
>>>>>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
>>>>>> Indeed, I pick it up from the source code.
>>>>>>
>>>>>> I think it maybe useful for other hive users if I can write some
>>>>>> documents about the property.
>>>>>>
>>>>>> So, can anyone give me some pointers that how can I make the change
>>>>>> happen ?
>>>>>>
>>>>>> Thanks, anyway.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by Lefty Leverenz <le...@gmail.com>.
HIVE-7314 is documented in the Configuration Properties wiki (
hive.cache.expr.evaluation
<https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>)
and at the beginning of the Operators and UDFs
<https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF> wiki.
The latter includes a link to this email thread.
Does that take care of it?
-- Lefty
On Sun, Jul 13, 2014 at 3:57 AM, Navis류승우 <na...@nexr.com> wrote:
> For "cache.expr.evaluation" problem, it's fixed in
> https://issues.apache.org/jira/browse/HIVE-7314 (will be included in next
> release)
>
> But I agree that it can be a critical problem for users of 0.12.0 and
> 0.13.x version and need proper warning on that.
>
> Thanks,
> Navis
>
>
> 2014-07-13 16:48 GMT+09:00 郭士伟 <gu...@gmail.com>:
>
> And I haven't add other information in the wikidoc yet.
>>
>>
>> 2014-07-04 14:37 GMT+08:00 Lefty Leverenz <le...@gmail.com>:
>>
>>> The 'hive.cache.expr.evaluation' parameter is documented in the wikidoc Configuration
>>> Properties
>>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>.
>>> Have you added other information in a different wikidoc? If so, I'll link
>>> the docs to each other.
>>>
>>> -- Lefty
>>>
>>>
>>> On Thu, May 29, 2014 at 5:31 AM, Lefty Leverenz <leftyleverenz@gmail.com
>>> > wrote:
>>>
>>>> Thank you for pointing this out. The 'hive.cache.expr.evaluation'
>>>> parameter was added by HIVE-4209
>>>> <https://issues.apache.org/jira/browse/HIVE-4209> and it needs to be
>>>> included in the wiki. Its description is in hive-default.xml.template:
>>>>
>>>> If true, evaluation result of deterministic expression referenced twice
>>>>> or more will be cached. For example, in filter condition like ".. where key
>>>>> + 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached once and
>>>>> reused for following expression ("key + 10 = 0"). Currently, this is
>>>>> applied only to expressions in select or filter operator.
>>>>
>>>>
>>>> Do you want to provide more information than that? To contribute to
>>>> the wiki, create a Confluence account
>>>> <https://cwiki.apache.org/confluence/signup.action> and send a message
>>>> to this list asking for wiki editing privilege and giving your account
>>>> name. But if you just want the description to be added to the wiki, I can
>>>> take care of that.
>>>>
>>>> If you have additional information, it might not belong in Configuration
>>>> Properties
>>>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties> (although
>>>> once the parameter is there it can link to another page for details). The
>>>> Select
>>>> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>
>>>> page is one possibility, or maybe you'll find a better place for it.
>>>>
>>>> If you want to revise the description in hive-default.xml.template, you
>>>> need to open a JIRA and patch the file.
>>>>
>>>>
>>>> -- Lefty
>>>>
>>>>
>>>> On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
>>>>
>>>>> I have used a configuration property 'hive.cache.expr.evaluation' in
>>>>> hive 0.12 recently and found it useful. But I cannot find any documents
>>>>> about it on the wiki page:
>>>>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
>>>>> Indeed, I pick it up from the source code.
>>>>>
>>>>> I think it maybe useful for other hive users if I can write some
>>>>> documents about the property.
>>>>>
>>>>> So, can anyone give me some pointers that how can I make the change
>>>>> happen ?
>>>>>
>>>>> Thanks, anyway.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by Navis류승우 <na...@nexr.com>.
For "cache.expr.evaluation" problem, it's fixed in
https://issues.apache.org/jira/browse/HIVE-7314 (will be included in next
release)
But I agree that it can be a critical problem for users of 0.12.0 and
0.13.x version and need proper warning on that.
Thanks,
Navis
2014-07-13 16:48 GMT+09:00 郭士伟 <gu...@gmail.com>:
> And I haven't add other information in the wikidoc yet.
>
>
> 2014-07-04 14:37 GMT+08:00 Lefty Leverenz <le...@gmail.com>:
>
>> The 'hive.cache.expr.evaluation' parameter is documented in the wikidoc Configuration
>> Properties
>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>.
>> Have you added other information in a different wikidoc? If so, I'll link
>> the docs to each other.
>>
>> -- Lefty
>>
>>
>> On Thu, May 29, 2014 at 5:31 AM, Lefty Leverenz <le...@gmail.com>
>> wrote:
>>
>>> Thank you for pointing this out. The 'hive.cache.expr.evaluation'
>>> parameter was added by HIVE-4209
>>> <https://issues.apache.org/jira/browse/HIVE-4209> and it needs to be
>>> included in the wiki. Its description is in hive-default.xml.template:
>>>
>>> If true, evaluation result of deterministic expression referenced twice
>>>> or more will be cached. For example, in filter condition like ".. where key
>>>> + 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached once and
>>>> reused for following expression ("key + 10 = 0"). Currently, this is
>>>> applied only to expressions in select or filter operator.
>>>
>>>
>>> Do you want to provide more information than that? To contribute to the
>>> wiki, create a Confluence account
>>> <https://cwiki.apache.org/confluence/signup.action> and send a message
>>> to this list asking for wiki editing privilege and giving your account
>>> name. But if you just want the description to be added to the wiki, I can
>>> take care of that.
>>>
>>> If you have additional information, it might not belong in Configuration
>>> Properties
>>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties> (although
>>> once the parameter is there it can link to another page for details). The
>>> Select
>>> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>
>>> page is one possibility, or maybe you'll find a better place for it.
>>>
>>> If you want to revise the description in hive-default.xml.template, you
>>> need to open a JIRA and patch the file.
>>>
>>>
>>> -- Lefty
>>>
>>>
>>> On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
>>>
>>>> I have used a configuration property 'hive.cache.expr.evaluation' in
>>>> hive 0.12 recently and found it useful. But I cannot find any documents
>>>> about it on the wiki page:
>>>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
>>>> Indeed, I pick it up from the source code.
>>>>
>>>> I think it maybe useful for other hive users if I can write some
>>>> documents about the property.
>>>>
>>>> So, can anyone give me some pointers that how can I make the change
>>>> happen ?
>>>>
>>>> Thanks, anyway.
>>>>
>>>
>>>
>>
>
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by 郭士伟 <gu...@gmail.com>.
And I haven't add other information in the wikidoc yet.
2014-07-04 14:37 GMT+08:00 Lefty Leverenz <le...@gmail.com>:
> The 'hive.cache.expr.evaluation' parameter is documented in the wikidoc Configuration
> Properties
> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>.
> Have you added other information in a different wikidoc? If so, I'll link
> the docs to each other.
>
> -- Lefty
>
>
> On Thu, May 29, 2014 at 5:31 AM, Lefty Leverenz <le...@gmail.com>
> wrote:
>
>> Thank you for pointing this out. The 'hive.cache.expr.evaluation'
>> parameter was added by HIVE-4209
>> <https://issues.apache.org/jira/browse/HIVE-4209> and it needs to be
>> included in the wiki. Its description is in hive-default.xml.template:
>>
>> If true, evaluation result of deterministic expression referenced twice
>>> or more will be cached. For example, in filter condition like ".. where key
>>> + 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached once and
>>> reused for following expression ("key + 10 = 0"). Currently, this is
>>> applied only to expressions in select or filter operator.
>>
>>
>> Do you want to provide more information than that? To contribute to the
>> wiki, create a Confluence account
>> <https://cwiki.apache.org/confluence/signup.action> and send a message
>> to this list asking for wiki editing privilege and giving your account
>> name. But if you just want the description to be added to the wiki, I can
>> take care of that.
>>
>> If you have additional information, it might not belong in Configuration
>> Properties
>> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties> (although
>> once the parameter is there it can link to another page for details). The
>> Select
>> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>
>> page is one possibility, or maybe you'll find a better place for it.
>>
>> If you want to revise the description in hive-default.xml.template, you
>> need to open a JIRA and patch the file.
>>
>>
>> -- Lefty
>>
>>
>> On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
>>
>>> I have used a configuration property 'hive.cache.expr.evaluation' in
>>> hive 0.12 recently and found it useful. But I cannot find any documents
>>> about it on the wiki page:
>>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
>>> Indeed, I pick it up from the source code.
>>>
>>> I think it maybe useful for other hive users if I can write some
>>> documents about the property.
>>>
>>> So, can anyone give me some pointers that how can I make the change
>>> happen ?
>>>
>>> Thanks, anyway.
>>>
>>
>>
>
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by Lefty Leverenz <le...@gmail.com>.
The 'hive.cache.expr.evaluation' parameter is documented in the
wikidoc Configuration
Properties
<https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.cache.expr.evaluation>.
Have you added other information in a different wikidoc? If so, I'll link
the docs to each other.
-- Lefty
On Thu, May 29, 2014 at 5:31 AM, Lefty Leverenz <le...@gmail.com>
wrote:
> Thank you for pointing this out. The 'hive.cache.expr.evaluation'
> parameter was added by HIVE-4209
> <https://issues.apache.org/jira/browse/HIVE-4209> and it needs to be
> included in the wiki. Its description is in hive-default.xml.template:
>
> If true, evaluation result of deterministic expression referenced twice or
>> more will be cached. For example, in filter condition like ".. where key +
>> 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached once and
>> reused for following expression ("key + 10 = 0"). Currently, this is
>> applied only to expressions in select or filter operator.
>
>
> Do you want to provide more information than that? To contribute to the
> wiki, create a Confluence account
> <https://cwiki.apache.org/confluence/signup.action> and send a message to
> this list asking for wiki editing privilege and giving your account name.
> But if you just want the description to be added to the wiki, I can take
> care of that.
>
> If you have additional information, it might not belong in Configuration
> Properties
> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties> (although
> once the parameter is there it can link to another page for details). The
> Select
> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>
> page is one possibility, or maybe you'll find a better place for it.
>
> If you want to revise the description in hive-default.xml.template, you
> need to open a JIRA and patch the file.
>
>
> -- Lefty
>
>
> On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
>
>> I have used a configuration property 'hive.cache.expr.evaluation' in hive
>> 0.12 recently and found it useful. But I cannot find any documents about it
>> on the wiki page:
>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
>> Indeed, I pick it up from the source code.
>>
>> I think it maybe useful for other hive users if I can write some
>> documents about the property.
>>
>> So, can anyone give me some pointers that how can I make the change
>> happen ?
>>
>> Thanks, anyway.
>>
>
>
Re: Hive can I contribute to Hive confluence wiki documents?
Posted by Lefty Leverenz <le...@gmail.com>.
Thank you for pointing this out. The 'hive.cache.expr.evaluation'
parameter was added by
HIVE-4209<https://issues.apache.org/jira/browse/HIVE-4209> and
it needs to be included in the wiki. Its description is in
hive-default.xml.template:
If true, evaluation result of deterministic expression referenced twice or
> more will be cached. For example, in filter condition like ".. where key +
> 10 > 10 or key + 10 = 0" "key + 10" will be evaluated/cached once and
> reused for following expression ("key + 10 = 0"). Currently, this is
> applied only to expressions in select or filter operator.
Do you want to provide more information than that? To contribute to the
wiki, create a Confluence
account<https://cwiki.apache.org/confluence/signup.action>and send a
message to this list asking for wiki editing privilege and
giving your account name. But if you just want the description to be added
to the wiki, I can take care of that.
If you have additional information, it might not belong in Configuration
Properties<https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties>
(although
once the parameter is there it can link to another page for details). The
Select<https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select>page
is one possibility, or maybe you'll find a better place for it.
If you want to revise the description in hive-default.xml.template, you
need to open a JIRA and patch the file.
-- Lefty
On Sat, May 24, 2014 at 1:01 AM, 郭士伟 <gu...@gmail.com> wrote:
> I have used a configuration property 'hive.cache.expr.evaluation' in hive
> 0.12 recently and found it useful. But I cannot find any documents about it
> on the wiki page:
> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties.
> Indeed, I pick it up from the source code.
>
> I think it maybe useful for other hive users if I can write some documents
> about the property.
>
> So, can anyone give me some pointers that how can I make the change happen
> ?
>
> Thanks, anyway.
>