You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Stephen Gargan (JIRA)" <ji...@apache.org> on 2010/12/30 01:21:46 UTC
[jira] Created: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
IPC Request Metadata written for transmission before processing by plugins
--------------------------------------------------------------------------
Key: AVRO-722
URL: https://issues.apache.org/jira/browse/AVRO-722
Project: Avro
Issue Type: Bug
Components: java
Affects Versions: 1.5.0
Reporter: Stephen Gargan
Priority: Minor
Fix For: 1.5.0
There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-722:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I committed this.
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-722.patch, rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978891#action_12978891 ]
Doug Cutting commented on AVRO-722:
-----------------------------------
Thanks, Stephen for noticing this bug, providing a fix for it, and including a great unit test!
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-722.patch, rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977482#action_12977482 ]
Doug Cutting commented on AVRO-722:
-----------------------------------
I found the other trace-related failure that motivated disabling these tests: AVRO-643.
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-722.patch, rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Gargan updated AVRO-722:
--------------------------------
Attachment: rpc-plugin.patch
I didn't realize that the trace tests had been excluded. I enabled them and worked out what was making them fail. The changes I'd made were effecting order that calls were being made to the plugin. I've added a test to insure that he order is maintained.
As a note, the TestFileSpanStorage is fairly time and load dependent and could be the source of the intermittent failures you're seeing. Its sleeps for arbitrary amounts waiting for the spans to be written asynchronously to disk. The current wait times are pretty tight and fail periodically on me, (though admittedly my machine is not very powerful).
What kind of error messages did it give when it failed before?
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-722:
------------------------------
Attachment: AVRO-722.patch
This works for me now with trace testing enabled. Thanks!
Here's a new version of the patch that also re-enables trace testing.
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-722.patch, rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Gargan updated AVRO-722:
--------------------------------
Attachment: rpc-plugin.patch
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting reassigned AVRO-722:
---------------------------------
Assignee: Stephen Gargan
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977014#action_12977014 ]
Doug Cutting commented on AVRO-722:
-----------------------------------
AVRO-644 reported problems with TestFileSpanStorage. I disabled trace tests in AVRO-646. I also saw failures in other trace tests but can't remember which. Should we re-enable trace testing and try to make TestFileSpanStorage more reliable?
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12976896#action_12976896 ]
Doug Cutting commented on AVRO-722:
-----------------------------------
This reverts a change that Patrick Wendell made in AVRO-595 when adding RPC tracing. I'm not sure why Patrick made that change there: perhaps it was essential but perhaps it was not. Unfortunately the trace tests are currently disabled in pom.xml, since they intermittently fail for reasons no one has yet debugged.
I re-enabled the trace tests in pom.xml and ran tests on unpatched trunk and they passed. Then I applied this patch and ran tests again and trace tests failed three times in a row. I reverted the patch and the trace tests then passed again. So superficially it appears that this patch breaks RPC tracing.
It would be nice to re-enable the trace tests by default. I think the problems are/were mostly caused by the use of fixed port numbers. In four runs they pass every time for me now.
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-722) IPC Request Metadata written for
transmission before processing by plugins
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-722:
------------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
I'll commit this unless there are objections.
> IPC Request Metadata written for transmission before processing by plugins
> --------------------------------------------------------------------------
>
> Key: AVRO-722
> URL: https://issues.apache.org/jira/browse/AVRO-722
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Stephen Gargan
> Assignee: Stephen Gargan
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-722.patch, rpc-plugin.patch, rpc-plugin.patch
>
>
> There is a small issue with the processing order of request metadata in an IPC call. The map is written for transmission before the context is passed to plugins for processing and as a result any changes that client plugins make to the metadata will not get transferred as the empty map has already been written.
> Also when the response metadata is read back in it is mistakenly read into the context's request metadata instead of the response metadata. The attached patch has fixes for both of these issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.