You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Grant Ingersoll (JIRA)" <ji...@apache.org> on 2011/07/07 20:27:16 UTC
[jira] [Created] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
----------------------------------------------------------------------------------------------------------------------------
Key: CASSANDRA-2869
URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
Project: Cassandra
Issue Type: Bug
Components: Contrib
Reporter: Grant Ingersoll
CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
I believe the fix lies in implementing
{code}
public void setStoreFuncUDFContextSignature(String signature)
{
}
{code}
and then using that signature when getting the UDFContext.
>From the Pig manual:
{quote}
setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
{quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069189#comment-13069189 ]
Hudson commented on CASSANDRA-2869:
-----------------------------------
Integrated in Cassandra-0.7 #534 (See [https://builds.apache.org/job/Cassandra-0.7/534/])
Use a UDF-specific context signature.
Patch by Jeremy Hanna, reviewed by brandonwilliams for CASSANDRA-2869
brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1149341
Files :
* /cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
> Fix For: 0.7.9, 0.8.2
>
> Attachments: 2869-2.txt, 2869.txt
>
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-2869:
----------------------------------------
Reviewer: brandon.williams
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
> Attachments: 2869.txt
>
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Hanna reassigned CASSANDRA-2869:
---------------------------------------
Assignee: Jeremy Hanna
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Hanna updated CASSANDRA-2869:
------------------------------------
Attachment: 2869-2.txt
Removed that String. Also removed adding mutation twice and put in the nested exception in putNext into the IOException. We've been meaning to add those last two items to one of these tickets.
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
> Attachments: 2869-2.txt, 2869.txt
>
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064149#comment-13064149 ]
Jeremy Hanna commented on CASSANDRA-2869:
-----------------------------------------
Yes. I was about to post an updated patch last night but got sidetracked. Do you mind removing that if it's otherwise good to go? Otherwise I can do that later today.
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
> Attachments: 2869.txt
>
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated CASSANDRA-2869:
---------------------------------------
Affects Version/s: 0.7.2
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Hanna updated CASSANDRA-2869:
------------------------------------
Attachment: 2869.txt
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
> Attachments: 2869.txt
>
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2869) CassandraStorage does not
function properly when used multiple times in a single pig script due to
UDFContext sharing issues
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063993#comment-13063993 ]
Brandon Williams commented on CASSANDRA-2869:
---------------------------------------------
Looks like we can remove UDFCONTEXT_SCHEMA_KEY_PREFIX now too, no?
> CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2869
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2869
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
> Affects Versions: 0.7.2
> Reporter: Grant Ingersoll
> Assignee: Jeremy Hanna
> Attachments: 2869.txt
>
>
> CassandraStorage appears to have threading issues along the lines of those described at http://pig.markmail.org/message/oz7oz2x2dwp66eoz due to the sharing of the UDFContext.
> I believe the fix lies in implementing
> {code}
> public void setStoreFuncUDFContextSignature(String signature)
> {
> }
> {code}
> and then using that signature when getting the UDFContext.
> From the Pig manual:
> {quote}
> setStoreFunc!UDFContextSignature(): This method will be called by Pig both in the front end and back end to pass a unique signature to the Storer. The signature can be used to store into the UDFContext any information which the Storer needs to store between various method invocations in the front end and back end. The default implementation in StoreFunc has an empty body. This method will be called before other methods.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira