You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "David Allsopp (JIRA)" <ji...@apache.org> on 2011/07/03 13:42:22 UTC
[jira] [Created] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
InvalidRequestException when validating column data includes entire column value
--------------------------------------------------------------------------------
Key: CASSANDRA-2849
URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.1
Reporter: David Allsopp
Priority: Minor
Fix For: 0.8.2
If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "David Allsopp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Allsopp updated CASSANDRA-2849:
-------------------------------------
Attachment: cassandra-2849.diff
Attached diff moves the reason for the exception before the data, and truncates the column value data in the exception to 64K.
Unable to test this properly today since working on a fresh Windows box over the weekend and unable to get build working in Eclipse 8-(
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Priority: Minor
> Fix For: 0.8.2
>
> Attachments: cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "David Allsopp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059215#comment-13059215 ]
David Allsopp commented on CASSANDRA-2849:
------------------------------------------
A lot of classes use ByteBufferUtil.bytesToHex(), often for logging and exceptions, so may make more sense to add a method there for getting a truncated hex string.
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Priority: Minor
> Fix For: 0.8.2
>
> Attachments: cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083480#comment-13083480 ]
Hudson commented on CASSANDRA-2849:
-----------------------------------
Integrated in Cassandra-0.8 #272 (See [https://builds.apache.org/job/Cassandra-0.8/272/])
include column name in validation failure exceptions
patch by jbellis; reviewed by David Allsopp for CASSANDRA-2849
jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1156753
Files :
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Assignee: David Allsopp
> Priority: Minor
> Fix For: 0.8.5
>
> Attachments: 2849-v2.txt, cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "David Allsopp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083169#comment-13083169 ]
David Allsopp commented on CASSANDRA-2849:
------------------------------------------
Yes, that looks good to me, thanks.
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Priority: Minor
> Fix For: 0.8.5
>
> Attachments: 2849-v2.txt, cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "David Allsopp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059221#comment-13059221 ]
David Allsopp commented on CASSANDRA-2849:
------------------------------------------
On a related note, I just noticed that ByteBufferUtil.bytesToHex() is unnecessarily slow - will raise a new issue for this.
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Priority: Minor
> Fix For: 0.8.2
>
> Attachments: cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083131#comment-13083131 ]
Jonathan Ellis commented on CASSANDRA-2849:
-------------------------------------------
David, does v2 look ok to you?
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Priority: Minor
> Fix For: 0.8.5
>
> Attachments: 2849-v2.txt, cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2849) InvalidRequestException when
validating column data includes entire column value
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2849:
--------------------------------------
Attachment: 2849-v2.txt
v2 adds column name decoding and moves the value to a debug log (presumably the client knows what value it sent...)
> InvalidRequestException when validating column data includes entire column value
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2849
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.1
> Reporter: David Allsopp
> Priority: Minor
> Fix For: 0.8.2
>
> Attachments: 2849-v2.txt, cassandra-2849.diff
>
>
> If the column value fails to validate, then ThriftValidation.validateColumnData() calls bytesToHex() on the entire column value and puts this string in the Exception. Since the value may be up to 2GB, this is potentially a lot of extra memory. The value is likely to be logged (and presumably returned to the thrift client over the network?). This could cause a lot of slowdown or an unnecessary OOM crash, and is unlikely to be useful (the client has access to the full value anyway if required for debugging).
> Also, the reason for the exception (extracted from the MarshalException) is printed _after_ the data, so if there's any truncation in the logging system at any point, the reason will be lost.
> The reason should be displayed before the column value, and the column value should be truncated in the Exception message.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira