You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by GitBox <gi...@apache.org> on 2020/10/16 17:46:02 UTC
[GitHub] [cassandra] maedhroz opened a new pull request #781: CASSANDRA-16214 Improve Logging Downstream of DataOutputBuffer Overflows
maedhroz opened a new pull request #781:
URL: https://github.com/apache/cassandra/pull/781
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] maedhroz commented on a change in pull request #781: CASSANDRA-16214 Improve Logging Downstream of DataOutputBuffer Overflows
Posted by GitBox <gi...@apache.org>.
maedhroz commented on a change in pull request #781:
URL: https://github.com/apache/cassandra/pull/781#discussion_r506955792
##########
File path: src/java/org/apache/cassandra/db/rows/PartitionSerializationException.java
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.rows;
+
+import org.apache.cassandra.schema.TableMetadata;
+
+public class PartitionSerializationException extends RuntimeException
+{
+ public PartitionSerializationException(UnfilteredRowIterator partition, Throwable cause)
+ {
+ super(buildMessage(partition), cause);
+ }
+
+ private static String buildMessage(UnfilteredRowIterator partition)
+ {
+ TableMetadata table = partition.metadata();
+ Object readablePartitionkey = table.partitionKeyType.compose(partition.partitionKey().getKey());
Review comment:
> But for the composite partition key, the output is not so nice. It contains utf8 characters.
Looks like switching to `getString()` at minimum makes sense. I'm not sure if we should solve the composite problem. Will noodle on it...
> a test to verify if the message matches expectation
Agreed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] smiklosovic closed pull request #781: CASSANDRA-16214 Improve Logging Downstream of DataOutputBuffer Overflows
Posted by GitBox <gi...@apache.org>.
smiklosovic closed pull request #781:
URL: https://github.com/apache/cassandra/pull/781
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] maedhroz commented on pull request #781: CASSANDRA-16214 Improve Logging Downstream of DataOutputBuffer Overflows
Posted by GitBox <gi...@apache.org>.
maedhroz commented on pull request #781:
URL: https://github.com/apache/cassandra/pull/781#issuecomment-719077870
@yifan-c I've addressed your review comments and pushed one more commit. (Tests are in [progress](https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=CASSANDRA-16214)...)
I didn't expand the use of `PartitionSerializationException`, as serializing partitions to disk seems to be handled well enough by catching in `BigTableWriter` and serializing partitions over the wire in `UnfilteredRowIteratorSerializer`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] maedhroz commented on pull request #781: CASSANDRA-16214 Improve Logging Downstream of DataOutputBuffer Overflows
Posted by GitBox <gi...@apache.org>.
maedhroz commented on pull request #781:
URL: https://github.com/apache/cassandra/pull/781#issuecomment-711033237
> the PR does not cover all callers that can throw BufferOverflowException
@yifan-c Let me do a quick audit of the other things that might have to catch BOE.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] yifan-c commented on a change in pull request #781: CASSANDRA-16214 Improve Logging Downstream of DataOutputBuffer Overflows
Posted by GitBox <gi...@apache.org>.
yifan-c commented on a change in pull request #781:
URL: https://github.com/apache/cassandra/pull/781#discussion_r506731400
##########
File path: test/unit/org/apache/cassandra/io/util/DataOutputTest.java
##########
@@ -180,8 +181,9 @@ public void testDataOutputBufferMaxSizeFake() throws IOException
while (true)
write.publicFlush();
}
- catch (RuntimeException e) {
- if (e.getClass() == RuntimeException.class)
+ catch (RuntimeException e)
Review comment:
nit: only catch `BufferOverflowException` so it does not swallow other RTE, if any.
##########
File path: src/java/org/apache/cassandra/db/rows/PartitionSerializationException.java
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.rows;
+
+import org.apache.cassandra.schema.TableMetadata;
+
+public class PartitionSerializationException extends RuntimeException
+{
+ public PartitionSerializationException(UnfilteredRowIterator partition, Throwable cause)
+ {
+ super(buildMessage(partition), cause);
+ }
+
+ private static String buildMessage(UnfilteredRowIterator partition)
+ {
+ TableMetadata table = partition.metadata();
+ Object readablePartitionkey = table.partitionKeyType.compose(partition.partitionKey().getKey());
Review comment:
Looks like `partitionKeyType.getString()` is more commonly used in the code base to get the readable representation of partition key.
Btw, both `getString` and `compose and toString` produce the same result.
But for the **composite partition key**, the output is not so nice. It contains utf8 characters. For the table
```
CREATE TABLE ks.tbl (key ascii, name ascii, val ascii, val1 ascii, PRIMARY KEY ((key, name), val))
```
The partition key string for `key1, name1` is
```
\x00\x04key1\x00\x00\x04name1\x00
```
##########
File path: test/unit/org/apache/cassandra/io/util/DataOutputTest.java
##########
@@ -245,8 +246,8 @@ public void testDataOutputBufferBigReallocation() throws Exception
Assert.assertEquals(DataOutputBuffer.MAX_ARRAY_SIZE, write.validateReallocation(DataOutputBuffer.MAX_ARRAY_SIZE + 1L));
Assert.assertEquals(DataOutputBuffer.MAX_ARRAY_SIZE, write.validateReallocation(DataOutputBuffer.MAX_ARRAY_SIZE));
Assert.assertEquals(DataOutputBuffer.MAX_ARRAY_SIZE - 1, write.validateReallocation(DataOutputBuffer.MAX_ARRAY_SIZE - 1));
- checkThrowsRuntimeException(validateReallocationCallable( write, 0));
- checkThrowsRuntimeException(validateReallocationCallable( write, 1));
Review comment:
`checkThrowsRuntimeException()` is no longer reference. We can remove it.
##########
File path: src/java/org/apache/cassandra/db/rows/PartitionSerializationException.java
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.rows;
+
+import org.apache.cassandra.schema.TableMetadata;
+
+public class PartitionSerializationException extends RuntimeException
+{
+ public PartitionSerializationException(UnfilteredRowIterator partition, Throwable cause)
+ {
+ super(buildMessage(partition), cause);
+ }
+
+ private static String buildMessage(UnfilteredRowIterator partition)
+ {
+ TableMetadata table = partition.metadata();
+ Object readablePartitionkey = table.partitionKeyType.compose(partition.partitionKey().getKey());
Review comment:
It would be nice to have a test to verify if the message matches expectation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org