You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by "jsedding (via GitHub)" <gi...@apache.org> on 2025/08/14 14:41:43 UTC

[PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

jsedding opened a new pull request, #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "jsedding (via GitHub)" <gi...@apache.org>.
jsedding commented on code in PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#discussion_r2284833270


##########
oak-commons/src/main/java/org/apache/jackrabbit/oak/commons/concurrent/ForkJoinUtils.java:
##########
@@ -0,0 +1,120 @@
+/*

Review Comment:
   Added a unit-test.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "jsedding (via GitHub)" <gi...@apache.org>.
jsedding commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3200189706

   @reschke I have addressed your comments in commit 7290b84. Unless there are any other issues, I will merge the PR once the build completes successfully.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "jsedding (via GitHub)" <gi...@apache.org>.
jsedding commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3204252121

   @reschke can you please review again and confirm that my changes address your concern? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3188711851

   # Commit-Check ✔️


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3197257824

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [28.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&metric=new_coverage&view=list) (required ≥ 80%)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "jsedding (via GitHub)" <gi...@apache.org>.
jsedding commented on code in PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#discussion_r2284832792


##########
oak-commons/src/main/java/org/apache/jackrabbit/oak/commons/concurrent/package-info.java:
##########
@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-@Version("1.0.1")
+@Version("1.1.0")

Review Comment:
   Moved to `forkjoin` package and marked it as internal API.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "reschke (via GitHub)" <gi...@apache.org>.
reschke commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3204997310

   late comment: for consistency with other packages in commons (like "pio"), I'd make this "pconcurrent".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3196247286

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [28.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&metric=new_coverage&view=list) (required ≥ 80%)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3201057544

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [2 New issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [88.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "jsedding (via GitHub)" <gi...@apache.org>.
jsedding merged PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#issuecomment-3189101850

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [69.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454&metric=new_coverage&view=list) (required ≥ 80%)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2454)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "joerghoh (via GitHub)" <gi...@apache.org>.
joerghoh commented on code in PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#discussion_r2277022488


##########
oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tar/TarFiles.java:
##########
@@ -411,18 +413,28 @@ public void init() throws IOException {
         // iterates the indices in ascending order, but prepends - instead of
         // appending - the corresponding TAR readers to the linked list. This
         // results in a properly ordered linked list.
+        Stream.of(indices)
+                .parallel()
+                .map(index -> {
+                    try {
+                        if (readOnly) {
+                            return TarReader.openRO(map.get(index), tarRecovery, archiveManager);
+                        } else {
+                            return TarReader.open(map.get(index), tarRecovery, archiveManager);
+                        }
+                    } catch (IOException e) {
+                        log.warn("Unable to open TAR file: {}", map.get(index), e);
+                        throw new UncheckedIOException(e);

Review Comment:
   how is the exception handling for this UncheckedIOException? And would it possible to pass some more information (for example the name of the file (it's logged before).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "jsedding (via GitHub)" <gi...@apache.org>.
jsedding commented on code in PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#discussion_r2281844153


##########
oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tar/TarFiles.java:
##########
@@ -411,18 +413,28 @@ public void init() throws IOException {
         // iterates the indices in ascending order, but prepends - instead of
         // appending - the corresponding TAR readers to the linked list. This
         // results in a properly ordered linked list.
+        Stream.of(indices)
+                .parallel()
+                .map(index -> {
+                    try {
+                        if (readOnly) {
+                            return TarReader.openRO(map.get(index), tarRecovery, archiveManager);
+                        } else {
+                            return TarReader.open(map.get(index), tarRecovery, archiveManager);
+                        }
+                    } catch (IOException e) {
+                        log.warn("Unable to open TAR file: {}", map.get(index), e);
+                        throw new UncheckedIOException(e);

Review Comment:
   I have added handling to unwrap the original `IOException`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] OAK-11861 - segment store initialization should run in parallel [jackrabbit-oak]

Posted by "reschke (via GitHub)" <gi...@apache.org>.
reschke commented on code in PR #2454:
URL: https://github.com/apache/jackrabbit-oak/pull/2454#discussion_r2284005727


##########
oak-commons/src/main/java/org/apache/jackrabbit/oak/commons/concurrent/package-info.java:
##########
@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-@Version("1.0.1")
+@Version("1.1.0")

Review Comment:
   Please do not put new methods into public APIs unless they are really needed outside Oak. In the past we created new packages for that.
   
   See https://github.com/apache/jackrabbit-oak/pull/2457 which happens to do that.



##########
oak-commons/src/main/java/org/apache/jackrabbit/oak/commons/concurrent/ForkJoinUtils.java:
##########
@@ -0,0 +1,120 @@
+/*

Review Comment:
   There should be unit test coverage inside oak-commons.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org