You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "vtlim (via GitHub)" <gi...@apache.org> on 2025/08/04 20:11:32 UTC
Re: [PR] docs: add deprecation for hadoop and java11 (druid)
vtlim commented on code in PR #18286:
URL: https://github.com/apache/druid/pull/18286#discussion_r2252479310
##########
docs/ingestion/data-formats.md:
##########
@@ -962,11 +962,17 @@ Each line can be further parsed using [`parseSpec`](#parsespec).
### Avro Hadoop Parser
-:::info
- You need to include the [`druid-avro-extensions`](../development/extensions-core/avro.md) as an extension to use the Avro Hadoop Parser.
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
+
+You must now explicitly opt-in to using the deprecated `index_hadoop` task type. To opt-in, set `druid.indexer.task.allowHadoopTaskExecution` to `true` in your `common.runtime.properties` file. For more information, see [#18239](https://github.com/apache/druid/pull/18239)
Review Comment:
```suggestion
You must now explicitly opt in to using the deprecated `index_hadoop` task type. To opt in, set `druid.indexer.task.allowHadoopTaskExecution` to `true` in your `common.runtime.properties` file. For more information, see [#18239](https://github.com/apache/druid/pull/18239).
```
##########
docs/ingestion/data-formats.md:
##########
@@ -962,11 +962,17 @@ Each line can be further parsed using [`parseSpec`](#parsespec).
### Avro Hadoop Parser
-:::info
- You need to include the [`druid-avro-extensions`](../development/extensions-core/avro.md) as an extension to use the Avro Hadoop Parser.
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
+
+You must now explicitly opt-in to using the deprecated `index_hadoop` task type. To opt-in, set `druid.indexer.task.allowHadoopTaskExecution` to `true` in your `common.runtime.properties` file. For more information, see [#18239](https://github.com/apache/druid/pull/18239)
+
:::
:::info
+ You need to include the [`druid-avro-extensions`](../development/extensions-core/avro.md) as an extension to use the Avro Hadoop Parser.
Review Comment:
```suggestion
You need to include [`druid-avro-extensions`](../development/extensions-core/avro.md) as an extension to use the Avro Hadoop Parser.
```
##########
docs/operations/other-hadoop.md:
##########
@@ -23,6 +23,14 @@ title: "Working with different versions of Apache Hadoop"
-->
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
Review Comment:
same as above
##########
docs/ingestion/hadoop.md:
##########
@@ -23,6 +23,15 @@ sidebar_label: "Hadoop-based"
~ under the License.
-->
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
Review Comment:
same as above
##########
docs/ingestion/data-formats.md:
##########
@@ -962,11 +962,17 @@ Each line can be further parsed using [`parseSpec`](#parsespec).
### Avro Hadoop Parser
-:::info
- You need to include the [`druid-avro-extensions`](../development/extensions-core/avro.md) as an extension to use the Avro Hadoop Parser.
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
Review Comment:
```suggestion
Hadoop-based ingestion is deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
```
##########
docs/tutorials/cluster.md:
##########
@@ -133,8 +133,8 @@ The [basic cluster tuning guide](../operations/basic-cluster-tuning.md) has info
We recommend running your favorite Linux distribution. You will also need
-* [Java 11 or 17](../operations/java.md)
-* Python 2 or Python 3
+* [17](../operations/java.md)
Review Comment:
```suggestion
* [Java 17](../operations/java.md)
```
##########
docs/tutorials/tutorial-kerberos-hadoop.md:
##########
@@ -23,6 +23,14 @@ sidebar_label: Kerberized HDFS deep storage
~ under the License.
-->
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
Review Comment:
same as above
##########
docs/tutorials/index.md:
##########
@@ -40,8 +40,8 @@ You can follow these steps on a relatively modest machine, such as a workstation
The software requirements for the installation machine are:
* Linux, Mac OS X, or other Unix-like OS. (Windows is not supported)
-* [Java 11 or 17](../operations/java.md)
-* Python 3 (preferred) or Python 2
+* [17](../operations/java.md)
Review Comment:
```suggestion
* [Java 17](../operations/java.md)
```
##########
docs/tutorials/tutorial-batch-hadoop.md:
##########
@@ -23,6 +23,13 @@ sidebar_label: Load from Apache Hadoop
~ under the License.
-->
+:::caution[Deprecated]
+
+Hadoop-based ingestion deprecated. Use [SQL-based ingestion](../multi-stage-query/index.md) instead of MapReduce or [MiddleManager-less ingestion using Kubernetes](../development/extensions-core/k8s-jobs.md) instead of YARN.
Review Comment:
same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org