You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/10/12 10:49:01 UTC
[GitHub] [druid] cryptoe opened a new issue #11791: Add support for hadoop 3
cryptoe opened a new issue #11791:
URL: https://github.com/apache/druid/issues/11791
### Motivation
As more and more clusters are getting created with hadoop 3.x.x, druid should start supporting hadoop 3.x.x as well.
A description of the problem.
### Proposed changes
We use combination of maven profiles and resource filtering to achieve this. Hadoop2 is supported by default and a new maven profile with the name hadoop3 is created. This will allow the user to choose the profile which is best suited for the use case.
Agreed, it will not help in the Hadoop dependency problems but does enable our users to use druid with multiple flavors.
Also with hadoop3, as @clintropolis mentioned, the dependencies come pre-shaded so we significantly reduce our effort in solving the dependency problems.
### Rationale
Discusion [thread](https://lists.apache.org/thread.html/r9e867429b44a694a77ff62351a0e92a4ca06c3f0184b8d9d5019c517%40%3Cdev.druid.apache.org%3E)
### Operational impact
This section should describe how the proposed changes will impact the operation of existing clusters. It should answer questions such as:
- Is anything going to be deprecated or removed by this change? How will we phase out old behavior?
We will phase out hadoop2 eventually and change the default profile to hadoop3.
- Is there a migration path that cluster operators need to be aware of?
Exisiting clusters will not be impacted,
- Will there be any effect on the ability to do a rolling upgrade, or to do a rolling _downgrade_ if an operator wants to switch back to a previous version?
Don't think so. It should have any impact.
### Test plan (optional)
Ut's and integration tests should pass
### Future work (optional)
We will phase out hadoop2 eventually and change the default profile to hadoop3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on issue #11791: Add support for hadoop 3
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on issue #11791:
URL: https://github.com/apache/druid/issues/11791#issuecomment-941080724
Thank you @cryptoe. can you clarify what is "resource filtering" here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 closed issue #11791: Add support for hadoop 3
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 closed issue #11791:
URL: https://github.com/apache/druid/issues/11791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on issue #11791: Add support for hadoop 3
Posted by GitBox <gi...@apache.org>.
cryptoe commented on issue #11791:
URL: https://github.com/apache/druid/issues/11791#issuecomment-941115452
Done added a link plus some more context .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on issue #11791: Add support for hadoop 3
Posted by GitBox <gi...@apache.org>.
cryptoe commented on issue #11791:
URL: https://github.com/apache/druid/issues/11791#issuecomment-941115452
Done added a link plus some more context .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org