You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/12/22 13:01:41 UTC
[GitHub] [kafka] satishd opened a new pull request, #13039: KAFKA 14480 Move the required dependant classes for ProducerStateManager
satishd opened a new pull request, #13039:
URL: https://github.com/apache/kafka/pull/13039
KAFKA 14480 Move the required dependant classes for ProducerStateManager
These include LogOffsetMetadata, Snapshot, LastRecord, TxnMetadata, BatchMetadata, ProducerStateEntry, ProducerAppendInfo
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #13039: KAFKA-14550 Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
ijuma commented on PR #13039:
URL: https://github.com/apache/kafka/pull/13039#issuecomment-1364033082
@satishd Can you please fix the conflicts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #13039: KAFKA-14550 Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
ijuma commented on PR #13039:
URL: https://github.com/apache/kafka/pull/13039#issuecomment-1368377953
@satishd did you see my review above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on a diff in pull request #13039: KAFKA-14550 Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
satishd commented on code in PR #13039:
URL: https://github.com/apache/kafka/pull/13039#discussion_r1059994296
##########
storage/src/main/java/org/apache/kafka/server/log/internals/SnapshotFile.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.log.internals;
+
+import org.apache.kafka.common.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+
+public class SnapshotFile {
+ private static final Logger log = LoggerFactory.getLogger(SnapshotFile.class);
+
+ public static long offsetFromFileName(String fileName) {
Review Comment:
Planned to do those changes as part of moving UnifiedLog/LocalLog to storage module. As you suggested, added `LogFileUtils` and `offsetFromFileName` in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #13039: KAFKA-14550: Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
ijuma commented on PR #13039:
URL: https://github.com/apache/kafka/pull/13039#issuecomment-1369037998
The build for all 3 JDK versions passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on pull request #13039: KAFKA-14550 Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
satishd commented on PR #13039:
URL: https://github.com/apache/kafka/pull/13039#issuecomment-1367740483
@ijuma These changes are updated with the latest trunk and resolved the conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] satishd commented on pull request #13039: KAFKA-14550 Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
satishd commented on PR #13039:
URL: https://github.com/apache/kafka/pull/13039#issuecomment-1368892291
@ijuma Sorry, Brave browser had some issues in refreshing the github PR page and missed your update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a diff in pull request #13039: KAFKA-14550 Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
ijuma commented on code in PR #13039:
URL: https://github.com/apache/kafka/pull/13039#discussion_r1059505967
##########
storage/src/main/java/org/apache/kafka/server/log/internals/SnapshotFile.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.log.internals;
+
+import org.apache.kafka.common.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+
+public class SnapshotFile {
+ private static final Logger log = LoggerFactory.getLogger(SnapshotFile.class);
+
+ public static long offsetFromFileName(String fileName) {
Review Comment:
Looks like this is duplicating some logic that is also present in `UnifiedLog`. Would it make sense to move it to a `LogFileUtils` class?
##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -1488,4 +1488,12 @@ public static String toLogDateTimeFormat(long timestamp) {
return Instant.ofEpochMilli(timestamp).atZone(ZoneId.systemDefault()).format(dateTimeFormatter);
}
+ /**
+ * Replace the given string suffix with the new suffix. If the string doesn't end with the given suffix throw an exception.
+ */
+ public static String replaceSuffix(String name, String oldSuffix, String newSuffix) {
Review Comment:
Nit: `name` should probably be `s` or something that makes it clear that it can be any string, not just a name.
Also, can we replace `CoreUtils.replaceSuffix` with this method and ensure we have unit tests for this new method?
##########
storage/src/main/java/org/apache/kafka/server/log/internals/SnapshotFile.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.log.internals;
+
+import org.apache.kafka.common.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+
+public class SnapshotFile {
+ private static final Logger log = LoggerFactory.getLogger(SnapshotFile.class);
+
+ public static long offsetFromFileName(String fileName) {
+ return Long.parseLong(fileName.substring(0, fileName.indexOf('.')));
+ }
+
+ private volatile File file;
+ public final long offset;
Review Comment:
Nit: let's place `public` fields first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma merged pull request #13039: KAFKA-14550: Move SnapshotFile and CorruptSnapshotException to storage module
Posted by GitBox <gi...@apache.org>.
ijuma merged PR #13039:
URL: https://github.com/apache/kafka/pull/13039
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org