You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/05/06 06:23:50 UTC

[GitHub] [kafka] satishd opened a new pull request #10638: MINOR Added `server-common` module to have server side common classes. 

satishd opened a new pull request #10638:
URL: https://github.com/apache/kafka/pull/10638


   MINOR Added `server-common` module to have server side common classes. 
   Moved ApiMessageAndVersion, RecordSerde, AbstractApiMessageSerde, and BytesApiMessageSerde to server-common module.
   
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628138696



##########
File path: build.gradle
##########
@@ -1585,6 +1644,7 @@ project(':shell') {
     implementation libs.jacksonJDK8Datatypes
     implementation libs.jline
     implementation libs.slf4jApi
+    implementation project(':server-common')

Review comment:
       Right, this is needed because `core` module does not expose `server-common` as a transitive module with gradle `api` dependency. It uses `implementation` dependency for `server-common` to be part of `core` module's compile/runtime classpaths.
   
   I am not sure whether we really have usages for `server-common` classes to be exposed as part of `core` module.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628222213



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    implementation project(':clients')

Review comment:
       I don't think we should remove the clients dependency since those modules rely on clients classes independently of their server-common dependency.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628138833



##########
File path: build.gradle
##########
@@ -779,6 +779,7 @@ project(':core') {
     api project(':clients')
     api libs.scalaLibrary
 
+    implementation project(':server-common')

Review comment:
       Right, it is needed as `server-common` is only compile/runtime dependency of `raft` and `metadata`. But it is not a transitive dependency.
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628228764



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"

Review comment:
       This is a good question and worth thinking about a bit more. I agree we can do it in a separate PR. I think the main thing to consider is how we differentiate between public classes exposed to users and common classes to be used by the kafka server modules.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mumrah commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r630302538



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    api project(':clients')
+    implementation libs.slf4jApi
+
+    testImplementation project(':clients')
+    testImplementation project(':clients').sourceSets.test.output
+    testImplementation libs.junitJupiter
+    testImplementation libs.mockitoCore
+
+    testRuntimeOnly libs.slf4jlog4j
+  }
+
+  task createVersionFile(dependsOn: determineCommitId) {
+    ext.receiptFile = file("$buildDir/kafka/$buildVersionFileName")
+    outputs.file receiptFile
+    outputs.upToDateWhen { false }
+    doLast {
+      def data = [
+              commitId: commitId,
+              version: version,
+      ]
+
+      receiptFile.parentFile.mkdirs()
+      def content = data.entrySet().collect { "$it.key=$it.value" }.sort().join("\n")
+      receiptFile.setText(content, "ISO-8859-1")
+    }
+  }
+
+  sourceSets {
+    main {
+      java {

Review comment:
       Are we only going to allow Java sources in this new module? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] junrao merged pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
junrao merged pull request #10638:
URL: https://github.com/apache/kafka/pull/10638


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma commented on pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#issuecomment-833529211


   cc @cmccabe 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mumrah commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r630302538



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    api project(':clients')
+    implementation libs.slf4jApi
+
+    testImplementation project(':clients')
+    testImplementation project(':clients').sourceSets.test.output
+    testImplementation libs.junitJupiter
+    testImplementation libs.mockitoCore
+
+    testRuntimeOnly libs.slf4jlog4j
+  }
+
+  task createVersionFile(dependsOn: determineCommitId) {
+    ext.receiptFile = file("$buildDir/kafka/$buildVersionFileName")
+    outputs.file receiptFile
+    outputs.upToDateWhen { false }
+    doLast {
+      def data = [
+              commitId: commitId,
+              version: version,
+      ]
+
+      receiptFile.parentFile.mkdirs()
+      def content = data.entrySet().collect { "$it.key=$it.value" }.sort().join("\n")
+      receiptFile.setText(content, "ISO-8859-1")
+    }
+  }
+
+  sourceSets {
+    main {
+      java {

Review comment:
       Are we only going to allow Java sources in this new module? (I think that's probably a good idea)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628139847



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"

Review comment:
       I plan to do that in another PR. I have not yet decided whether to have a separate sub module like `server-api` or part of `server-common` module. `server-api` module can contain all the server related public API classes including storage, security etc.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628139621



##########
File path: build.gradle
##########
@@ -1070,6 +1071,7 @@ project(':metadata') {
   archivesBaseName = "kafka-metadata"
 
   dependencies {
+    implementation project(':server-common')

Review comment:
       Good catch. It looks like these are already copied as part of `configurations.runtimeClasspath`. I raised https://github.com/apache/kafka/pull/10647 to fix that.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628139399



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    implementation project(':clients')

Review comment:
       I did not add `clients` as `api` dependency. We may need to do that as the users of `server-common` need `org.apache.kafka.common.protocol` classes as they are part of the APIs in `server-common` classes.
   
   After this change, we can remove `clients` dependency from wherever `server-common` module is directly used.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on pull request #10638: MINOR Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#issuecomment-833446234


   @junrao As discussed earlier, serde related classes are moved to the new `server-common` module. 
   We can add more server specific classes from other modules like `clients` into this module with followup PRs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628139004



##########
File path: build.gradle
##########
@@ -1267,11 +1269,13 @@ project(':raft') {
   archivesBaseName = "kafka-raft"
 
   dependencies {
+    implementation project(':server-common')

Review comment:
       Pl see the earlier [comment](https://github.com/apache/kafka/pull/10638#discussion_r628138833).




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628215953



##########
File path: build.gradle
##########
@@ -1585,6 +1644,7 @@ project(':shell') {
     implementation libs.jacksonJDK8Datatypes
     implementation libs.jline
     implementation libs.slf4jApi
+    implementation project(':server-common')

Review comment:
       core should generally not expose anything with api since core doesn't expose any public api.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#issuecomment-834307680


   Thanks @junrao for the review comments. Pl see inline replies and the commit https://github.com/apache/kafka/pull/10638/commits/1e259ffac16c4237565fa6be02c1e12df812e15a


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628139399



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    implementation project(':clients')

Review comment:
       We did not add `clients` as `api` dependency. We may need to do that as the users of server-common need `org.apache.kafka.common.protocol` classes as they are part of the APIs in `server-common` classes.
   
   After this change, we can remove `clients` dependency from wherever `server-common` module is directly used.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r629269557



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"

Review comment:
       Created https://issues.apache.org/jira/browse/KAFKA-12757 to discuss more on this. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628151162



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    implementation project(':clients')

Review comment:
       Added the change with the commit https://github.com/apache/kafka/pull/10638/commits/1e259ffac16c4237565fa6be02c1e12df812e15a




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628356562



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    implementation project(':clients')

Review comment:
       Good point @ijuma. It makes sense to me to have explicit client dependency irrespective of `server-common` dependency.  




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r628139847



##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"

Review comment:
       I plan to do that in another PR. I have not yet decided whether to have a separate sub module like `server-api` or part of `server-common` module.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] junrao commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
junrao commented on a change in pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#discussion_r627548721



##########
File path: build.gradle
##########
@@ -1585,6 +1644,7 @@ project(':shell') {
     implementation libs.jacksonJDK8Datatypes
     implementation libs.jline
     implementation libs.slf4jApi
+    implementation project(':server-common')

Review comment:
       Is this needed since core already depends on server-common?

##########
File path: build.gradle
##########
@@ -1267,11 +1269,13 @@ project(':raft') {
   archivesBaseName = "kafka-raft"
 
   dependencies {
+    implementation project(':server-common')

Review comment:
       Is this needed since server-common is included in metadata?

##########
File path: build.gradle
##########
@@ -1070,6 +1071,7 @@ project(':metadata') {
   archivesBaseName = "kafka-metadata"
 
   dependencies {
+    implementation project(':server-common')

Review comment:
       Not directly related to this PR. Do you know why we only include the storage module in releaseTarGz target, but not other server side modules?

##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"
+
+  dependencies {
+    implementation project(':clients')

Review comment:
       Since server-common depends on clients, perhaps we could remove clients from all modules that depend on server-common?

##########
File path: build.gradle
##########
@@ -1345,6 +1349,62 @@ project(':raft') {
   }
 }
 
+project(':server-common') {
+  archivesBaseName = "kafka-server-common"

Review comment:
       Do we want to move the classes in storage-api here? It helps reduce one module.

##########
File path: build.gradle
##########
@@ -779,6 +779,7 @@ project(':core') {
     api project(':clients')
     api libs.scalaLibrary
 
+    implementation project(':server-common')

Review comment:
       Is that needed since server-common is included in metadata/storage already?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] satishd edited a comment on pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

Posted by GitBox <gi...@apache.org>.
satishd edited a comment on pull request #10638:
URL: https://github.com/apache/kafka/pull/10638#issuecomment-834307680


   Thanks @junrao for the review comments. Pl see inline replies and the latest commits.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org