You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Adam Kramer (JIRA)" <ji...@apache.org> on 2011/08/01 00:53:09 UTC
[jira] [Created] (HIVE-2327) UDFs should be made aware of their
arguments are constants.
UDFs should be made aware of their arguments are constants.
-----------------------------------------------------------
Key: HIVE-2327
URL: https://issues.apache.org/jira/browse/HIVE-2327
Project: Hive
Issue Type: Improvement
Reporter: Adam Kramer
There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2327) UDFs should be made aware when their
arguments are constants.
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073655#comment-13073655 ]
John Sichi commented on HIVE-2327:
----------------------------------
Is this different from HIVE-1360?
> UDFs should be made aware when their arguments are constants.
> -------------------------------------------------------------
>
> Key: HIVE-2327
> URL: https://issues.apache.org/jira/browse/HIVE-2327
> Project: Hive
> Issue Type: Improvement
> Reporter: Adam Kramer
>
> There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2327) UDFs should be made aware when their
arguments are constants.
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103150#comment-13103150 ]
John Sichi commented on HIVE-2327:
----------------------------------
OK, then I guess this issue should be renamed?
> UDFs should be made aware when their arguments are constants.
> -------------------------------------------------------------
>
> Key: HIVE-2327
> URL: https://issues.apache.org/jira/browse/HIVE-2327
> Project: Hive
> Issue Type: Improvement
> Reporter: Adam Kramer
>
> There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2327) UDFs should be made aware when their
arguments are constants.
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi resolved HIVE-2327.
------------------------------
Resolution: Duplicate
> UDFs should be made aware when their arguments are constants.
> -------------------------------------------------------------
>
> Key: HIVE-2327
> URL: https://issues.apache.org/jira/browse/HIVE-2327
> Project: Hive
> Issue Type: Improvement
> Reporter: Adam Kramer
>
> There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2327) UDFs should be made aware when their
arguments are constants.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach reopened HIVE-2327:
----------------------------------
HIVE-1360 made it possible for UDFs to detect constant parameters, but it didn't update any of the regex UDFs to take advantage of this feature.
> UDFs should be made aware when their arguments are constants.
> -------------------------------------------------------------
>
> Key: HIVE-2327
> URL: https://issues.apache.org/jira/browse/HIVE-2327
> Project: Hive
> Issue Type: Improvement
> Reporter: Adam Kramer
>
> There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2327) Optimize REGEX UDFs with constant
parameter information
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2327:
---------------------------------
Summary: Optimize REGEX UDFs with constant parameter information (was: UDFs should be made aware when their arguments are constants.)
> Optimize REGEX UDFs with constant parameter information
> -------------------------------------------------------
>
> Key: HIVE-2327
> URL: https://issues.apache.org/jira/browse/HIVE-2327
> Project: Hive
> Issue Type: Improvement
> Reporter: Adam Kramer
>
> There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2327) UDFs should be made aware when their
arguments are constants.
Posted by "Adam Kramer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kramer updated HIVE-2327:
------------------------------
Summary: UDFs should be made aware when their arguments are constants. (was: UDFs should be made aware of their arguments are constants.)
> UDFs should be made aware when their arguments are constants.
> -------------------------------------------------------------
>
> Key: HIVE-2327
> URL: https://issues.apache.org/jira/browse/HIVE-2327
> Project: Hive
> Issue Type: Improvement
> Reporter: Adam Kramer
>
> There are a lot of UDFs which would show major performance differences if one assumes that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would have to re-read the whole file if the filename changes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira