You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/06/09 06:32:19 UTC
[GitHub] [kafka] satishd opened a new pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
satishd opened a new pull request #10848:
URL: https://github.com/apache/kafka/pull/10848
- Updated transaction index as optional in `LogSegmentData`.
- Added a unit test for the introduced change.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] junrao commented on a change in pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
junrao commented on a change in pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#discussion_r648690860
##########
File path: storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java
##########
@@ -33,31 +34,32 @@
private final Path logSegment;
private final Path offsetIndex;
private final Path timeIndex;
- private final Path txnIndex;
+ private final Optional<Path> transactionIndex;
private final Path producerSnapshotIndex;
private final ByteBuffer leaderEpochIndex;
/**
* Creates a LogSegmentData instance with data and indexes.
- * @param logSegment actual log segment file
+ *
+ * @param logSegment actual log segment file
* @param offsetIndex offset index file
* @param timeIndex time index file
- * @param txnIndex transaction index file
+ * @param transactionIndex transaction index file, which can be null
* @param producerSnapshotIndex producer snapshot until this segment
* @param leaderEpochIndex leader-epoch-index until this segment
*/
public LogSegmentData(Path logSegment,
Path offsetIndex,
Path timeIndex,
- Path txnIndex,
+ Path transactionIndex,
Review comment:
Could we make transactionIndex Optional<Path> to make it clear?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#discussion_r649171869
##########
File path: storage/api/src/test/java/org/apache/kafka/server/log/remote/storage/LogSegmentDataTest.java
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.log.remote.storage;
+
+import org.apache.kafka.test.TestUtils;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.nio.ByteBuffer;
+
+public class LogSegmentDataTest {
+
+ @Test
+ public void testOptionalTransactionIndex() {
+ File dir = TestUtils.tempDirectory();
+ LogSegmentData logSegmentDataWithTransactionIndex = new LogSegmentData(
+ new File(dir, "log-segment").toPath(),
+ new File(dir, "offset-index").toPath(),
+ new File(dir, "time-index").toPath(),
+ new File(dir, "transaction-index").toPath(),
+ new File(dir, "producer-snapshot").toPath(),
+ ByteBuffer.allocate(1)
+ );
+ Assertions.assertTrue(logSegmentDataWithTransactionIndex.transactionIndex().isPresent());
Review comment:
Please use static imports to stick with the usual more concise assert style.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on a change in pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#discussion_r648768897
##########
File path: storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java
##########
@@ -33,31 +34,32 @@
private final Path logSegment;
private final Path offsetIndex;
private final Path timeIndex;
- private final Path txnIndex;
+ private final Optional<Path> transactionIndex;
private final Path producerSnapshotIndex;
private final ByteBuffer leaderEpochIndex;
/**
* Creates a LogSegmentData instance with data and indexes.
- * @param logSegment actual log segment file
+ *
+ * @param logSegment actual log segment file
* @param offsetIndex offset index file
* @param timeIndex time index file
- * @param txnIndex transaction index file
+ * @param transactionIndex transaction index file, which can be null
* @param producerSnapshotIndex producer snapshot until this segment
* @param leaderEpochIndex leader-epoch-index until this segment
*/
public LogSegmentData(Path logSegment,
Path offsetIndex,
Path timeIndex,
- Path txnIndex,
+ Path transactionIndex,
Review comment:
Passing `Optional` as arguments is not considered as a good practice. We still need to do null check for that Optional instance.
[SO answer from Brian Goetz](https://stackoverflow.com/questions/26327957/should-java-8-getters-return-optional-type/26328555#26328555) mentioned the right usage of `Optional`. I have also updated PR not to use it as a field.
```
You should almost never use it as a field of something or a method parameter.
```
[Javadoc of Optional ](https://docs.oracle.com/javase/10/docs/api/java/util/Optional.html)suggests returning as an argument mentioned below.
```
API Note:
Optional is primarily intended for use as a method return type where there is a clear need to represent "no result," and where using null is likely to cause errors. A variable whose type is Optional should never itself be null; it should always point to an Optional instance.
```
Having said that, I do not have strong opinions on the above. I am fine with the conventions that we are following in this project if we have any on `Optional` usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#discussion_r649171232
##########
File path: storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java
##########
@@ -33,31 +34,32 @@
private final Path logSegment;
private final Path offsetIndex;
private final Path timeIndex;
- private final Path txnIndex;
+ private final Optional<Path> transactionIndex;
private final Path producerSnapshotIndex;
private final ByteBuffer leaderEpochIndex;
/**
* Creates a LogSegmentData instance with data and indexes.
- * @param logSegment actual log segment file
+ *
+ * @param logSegment actual log segment file
* @param offsetIndex offset index file
* @param timeIndex time index file
- * @param txnIndex transaction index file
+ * @param transactionIndex transaction index file, which can be null
* @param producerSnapshotIndex producer snapshot until this segment
* @param leaderEpochIndex leader-epoch-index until this segment
*/
public LogSegmentData(Path logSegment,
Path offsetIndex,
Path timeIndex,
- Path txnIndex,
+ Path transactionIndex,
Review comment:
The Java reasoning is pretty arbitrary and is inconsistent with the Scala recommendation (which we've been following and has worked pretty well for us). I suggest we stick with our approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] junrao merged pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
junrao merged pull request #10848:
URL: https://github.com/apache/kafka/pull/10848
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
satishd commented on pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#issuecomment-858686600
Thanks @junrao @ijuma for the review. Addressed the review comments with the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on a change in pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#discussion_r648768897
##########
File path: storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java
##########
@@ -33,31 +34,32 @@
private final Path logSegment;
private final Path offsetIndex;
private final Path timeIndex;
- private final Path txnIndex;
+ private final Optional<Path> transactionIndex;
private final Path producerSnapshotIndex;
private final ByteBuffer leaderEpochIndex;
/**
* Creates a LogSegmentData instance with data and indexes.
- * @param logSegment actual log segment file
+ *
+ * @param logSegment actual log segment file
* @param offsetIndex offset index file
* @param timeIndex time index file
- * @param txnIndex transaction index file
+ * @param transactionIndex transaction index file, which can be null
* @param producerSnapshotIndex producer snapshot until this segment
* @param leaderEpochIndex leader-epoch-index until this segment
*/
public LogSegmentData(Path logSegment,
Path offsetIndex,
Path timeIndex,
- Path txnIndex,
+ Path transactionIndex,
Review comment:
Passing `Optional` as arguments is not considered as a good practice. We still need to do null check for that Optional instance.
[SO answer from Brian Goetz](https://stackoverflow.com/questions/26327957/should-java-8-getters-return-optional-type/26328555#26328555) mentioned the right usage of `Optional`. I have also update PR not to use it as a field.
```
You should almost never use it as a field of something or a method parameter.
```
[Javadoc of Optional ](https://docs.oracle.com/javase/10/docs/api/java/util/Optional.html)suggests returning as an argument mentioned below.
```
API Note:
Optional is primarily intended for use as a method return type where there is a clear need to represent "no result," and where using null is likely to cause errors. A variable whose type is Optional should never itself be null; it should always point to an Optional instance.
```
I do not have strong opinions on this, I am fine with the conventions that we are following in this project if we have any on `Optional` usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
satishd commented on pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#issuecomment-857423198
@junrao : Pl take a look at this minor PR making transaction index as optional in `LogSegmentData` as we discussed earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on pull request #10848: MINOR Updated transaction index as optional in LogSegmentData.
Posted by GitBox <gi...@apache.org>.
satishd commented on pull request #10848:
URL: https://github.com/apache/kafka/pull/10848#issuecomment-859319774
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org