You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ning Zhang (JIRA)" <ji...@apache.org> on 2010/09/21 18:47:33 UTC
[jira] Created: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
parse_url_tuple: a UDTF version of parse_url
---------------------------------------------
Key: HIVE-1659
URL: https://issues.apache.org/jira/browse/HIVE-1659
Project: Hadoop Hive
Issue Type: New Feature
Reporter: Ning Zhang
The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913081#action_12913081 ]
Ning Zhang commented on HIVE-1659:
----------------------------------
parse_url currently support 2 signatures: parse_url(fullurl, '[QUERY|PATH|HOST|...]') and parse_url(fullurl, 'QUERY', '[ref|sk|...]'). In parse_url_tuple, the syntax is consolidated as parse_url_tuple(fullurl, 'HOST', 'PATH', 'QUERY:ref', 'QUERY:sk',...).
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Ning Zhang
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914358#action_12914358 ]
Ning Zhang commented on HIVE-1659:
----------------------------------
Xing, this patch doesn't apply cleanly with the latest trunk. Can you 'svn up' and regenerate the patch. You may need to resolve any conflicts after 'svn up'.
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Xing Jin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xing Jin updated HIVE-1659:
---------------------------
Attachment: HIVE-1659.patch2
generate patch after 'svn up' and in root directory
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch, HIVE-1659.patch2
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Xing Jin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xing Jin updated HIVE-1659:
---------------------------
Attachment: HIVE-1659.patch
This patch contains implementation of parse_url_tuple function, as well as a test file and a test output.
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914337#action_12914337 ]
Ning Zhang commented on HIVE-1659:
----------------------------------
Xing, the patch was not attached. Can you use the link "Attach file" in the left pane?
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914360#action_12914360 ]
Ning Zhang commented on HIVE-1659:
----------------------------------
Also when you generate the patch, you need to run 'svn diff' at the "root" directory of the hive trunk.
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914602#action_12914602 ]
Ning Zhang commented on HIVE-1659:
----------------------------------
Xing, there is a diff in show_functions.q. You need to overwrite the .out file with the addition of the new function. The following command will update the out file.
ant test -Dtestcase=TestCliDriver -Dqfile=show_functions.q -Doverwrite=true
Can you regenerate the patch after that?
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch, HIVE-1659.patch2
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1659:
-----------------------------
Status: Resolved (was: Patch Available)
Resolution: Fixed
Committed. Thanks Xing!
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch, HIVE-1659.patch2, HIVE-1659.patch3
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Xing Jin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xing Jin updated HIVE-1659:
---------------------------
Status: Patch Available (was: Open)
Affects Version/s: 0.5.0
Add new function parse_url_tuple for HIVE.
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1659) parse_url_tuple: a UDTF version of
parse_url
Posted by "Xing Jin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xing Jin updated HIVE-1659:
---------------------------
Attachment: HIVE-1659.patch3
update show_functions.q and 'svn up' again. Thanks. Do I need to run all unitests on my machine?
> parse_url_tuple: a UDTF version of parse_url
> ---------------------------------------------
>
> Key: HIVE-1659
> URL: https://issues.apache.org/jira/browse/HIVE-1659
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.5.0
> Reporter: Ning Zhang
> Attachments: HIVE-1659.patch, HIVE-1659.patch2, HIVE-1659.patch3
>
>
> The UDF parse_url take s a URL, parse it and extract QUERY/PATH etc from it. However it can only extract an atomic value from the URL. If we want to extract multiple piece of information, we need to call the function many times. It is desirable to parse the URL once and extract all needed information and return a tuple in a UDTF.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.