You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "vogievetsky (via GitHub)" <gi...@apache.org> on 2025/04/01 21:30:48 UTC

[PR] Web console: add turboLoadingNodes (druid)

vogievetsky opened a new pull request, #17860:
URL: https://github.com/apache/druid/pull/17860

   The web console follow up to https://github.com/apache/druid/pull/17775
   
   <img width="534" alt="image" src="https://github.com/user-attachments/assets/aa9c0dfc-c024-41d2-a046-3a22a419ca0c" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Web console: add turboLoadingNodes (druid)

Posted by "vogievetsky (via GitHub)" <gi...@apache.org>.
vogievetsky merged PR #17860:
URL: https://github.com/apache/druid/pull/17860


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Web console: add turboLoadingNodes (druid)

Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe commented on PR #17860:
URL: https://github.com/apache/druid/pull/17860#issuecomment-2774337416

   
   > @cryptoe if we are wanting to call this out as experimental for druid 33, do we also need to update https://github.com/apache/druid/blob/master/docs/configuration/index.md to say the same in docs?
   Yup I think we should call it experimental in the docs as well. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Web console: add turboLoadingNodes (druid)

Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe commented on code in PR #17860:
URL: https://github.com/apache/druid/pull/17860#discussion_r2024096434


##########
web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx:
##########
@@ -156,6 +157,19 @@ export const COORDINATOR_DYNAMIC_CONFIG_FIELDS: Field<CoordinatorDynamicConfig>[
       </>
     ),
   },
+  {
+    name: 'turboLoadingNodes',
+    type: 'string-array',
+    info: (
+      <>
+        List of Historical servers to place in turbo loading mode. These servers use a larger
+        thread-pool to load segments faster but at the cost of query performance. For servers
+        specified in <Code>turboLoadingNodes</Code>,{' '}
+        <Code>druid.coordinator.loadqueuepeon.http.batchSize</Code> is ignored and the coordinator
+        uses the value of the respective <Code>numLoadingThreads</Code> instead.

Review Comment:
   ```suggestion
           uses the value of the respective <Code>numLoadingThreads</Code> instead. Please use this config with caution. All servers should eventually be removed from this list once the segment loading on the respective historicals is finished.
   ```



##########
web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx:
##########
@@ -156,6 +157,19 @@ export const COORDINATOR_DYNAMIC_CONFIG_FIELDS: Field<CoordinatorDynamicConfig>[
       </>
     ),
   },
+  {
+    name: 'turboLoadingNodes',
+    type: 'string-array',
+    info: (
+      <>
+        List of Historical servers to place in turbo loading mode. These servers use a larger

Review Comment:
   ```suggestion
           [Experimental] List of Historical servers to place in turbo loading mode. These servers use a larger
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Web console: add turboLoadingNodes (druid)

Posted by "capistrant (via GitHub)" <gi...@apache.org>.
capistrant commented on PR #17860:
URL: https://github.com/apache/druid/pull/17860#issuecomment-2772602496

   @cryptoe if we are wanting to call this out as experimental for druid 33, do we also need to update https://github.com/apache/druid/blob/master/docs/configuration/index.md to say the same in docs?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Web console: add turboLoadingNodes (druid)

Posted by "vogievetsky (via GitHub)" <gi...@apache.org>.
vogievetsky commented on code in PR #17860:
URL: https://github.com/apache/druid/pull/17860#discussion_r2025023145


##########
web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx:
##########
@@ -156,6 +157,19 @@ export const COORDINATOR_DYNAMIC_CONFIG_FIELDS: Field<CoordinatorDynamicConfig>[
       </>
     ),
   },
+  {
+    name: 'turboLoadingNodes',
+    type: 'string-array',
+    info: (
+      <>
+        List of Historical servers to place in turbo loading mode. These servers use a larger
+        thread-pool to load segments faster but at the cost of query performance. For servers
+        specified in <Code>turboLoadingNodes</Code>,{' '}
+        <Code>druid.coordinator.loadqueuepeon.http.batchSize</Code> is ignored and the coordinator
+        uses the value of the respective <Code>numLoadingThreads</Code> instead.

Review Comment:
   this text is just copied from the docs so the same change should probably be done there. I can do it.
   
   note: the philosophy I have is that the console is the docs that came to life. The docs should always be the source of truth.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Web console: add turboLoadingNodes (druid)

Posted by "vogievetsky (via GitHub)" <gi...@apache.org>.
vogievetsky commented on code in PR #17860:
URL: https://github.com/apache/druid/pull/17860#discussion_r2025026301


##########
web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx:
##########
@@ -156,6 +157,19 @@ export const COORDINATOR_DYNAMIC_CONFIG_FIELDS: Field<CoordinatorDynamicConfig>[
       </>
     ),
   },
+  {
+    name: 'turboLoadingNodes',
+    type: 'string-array',
+    info: (
+      <>
+        List of Historical servers to place in turbo loading mode. These servers use a larger

Review Comment:
   Experimental features get ` (experimental)` at the end of their label (not just the description). Will add. Let's add it to the docs also



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org