You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "jreyeshdez (via GitHub)" <gi...@apache.org> on 2024/02/12 11:15:24 UTC
[I] Peons uses base podTemplate for index tasks when specific podTemplate exists (druid)
jreyeshdez opened a new issue, #15886:
URL: https://github.com/apache/druid/issues/15886
Peon uses base pod template even though specific task type templates exist for index_kinesis and index_parallel and proper configurations have been set up in coordinator.
### Affected Version
Druid version 28.0.1 and 26.0.0
### Description
- Cluster size: 3x brokers, 3x coordinators, 3x historical, 3x routers
- Configurations in use: mm-less and zk-less
- Steps to reproduce the problem
Follow steps in https://druid.apache.org/docs/latest/development/extensions-contrib/k8s-jobs/#custom-template-pod-adapter
Set up druid to be mm-less and zk-less. In coordinator set up following env variables:
```
- name: druid_indexer_runner_k8s_podTemplate_base
value: "/opt/druid/conf/druid/basePodSpec.yaml"
- name: druid_indexer_runner_k8s_podTemplate_index_kinesis
value: "/opt/druid/conf/druid/indexPodSpec.yaml"
- name: druid_indexer_runner_k8s_podTemplate_index_parallel
value: "/opt/druid/conf/druid/indexPodSpec.yaml"
```
Regardless of the task_type being ejecutes in peons, the podTemplate used will be the one specified in `druid_indexer_runner_k8s_podTemplate_base`, mainly because the parsed env var looks like `druid.indexer.runner.k8s.podTemplate.{task.type}` instead of `druid.indexer.runner.k8s.podTemplate.{task_type}`.
List of tasks do not include index_kinesis, index_kafka, index_parallel when running unit tests for this line:
https://github.com/apache/druid/blob/master/extensions-contrib/kubernetes-overlord-extensions/src/main/java/org/apache/druid/k8s/overlord/taskadapter/PodTemplateTaskAdapter.java#L225
Linked issue https://github.com/apache/druid/issues/15875
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Peons uses base podTemplate for index tasks when specific podTemplate exists (druid)
Posted by "georgew5656 (via GitHub)" <gi...@apache.org>.
georgew5656 commented on issue #15886:
URL: https://github.com/apache/druid/issues/15886#issuecomment-1947299161
this pr should address the problem, still need to test in a druid environment https://github.com/apache/druid/pull/15915
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Peons uses base podTemplate for index tasks when specific podTemplate exists (druid)
Posted by "georgew5656 (via GitHub)" <gi...@apache.org>.
georgew5656 closed issue #15886: Peons uses base podTemplate for index tasks when specific podTemplate exists
URL: https://github.com/apache/druid/issues/15886
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Peons uses base podTemplate for index tasks when specific podTemplate exists (druid)
Posted by "georgew5656 (via GitHub)" <gi...@apache.org>.
georgew5656 commented on issue #15886:
URL: https://github.com/apache/druid/issues/15886#issuecomment-1946156509
makes sense, working on a fix for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org