You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/06/23 15:43:15 UTC
[GitHub] [hadoop-ozone] elek opened a new pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
elek opened a new pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117
## What changes were proposed in this pull request?
With a few thousands issues ago Ozone was integral part of Hadoop/HDFS project. At that time there were two options to start datanode:
1. Start in a separated JVM
2. Start in the same JVM with the HDFS
Today only 1 is the standard way, this is tested and working. 2nd is not working any more but still documented.
I propose to drop the support of this use case as I can't see any benefit to support it anymore:
1. I think 100% of the users will use Ozone as a separated process not as a HDFS Datanode plugin
2. Fixing the classpath issues is significant effort as the classpath of HDFS and Ozone are diverged.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3858
## How was this patch tested?
Not required. One compose folder + docs are removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648715087
/ready
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] jnp commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
jnp commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648287597
The option to run them in same JVM was allowed, to support in-place upgrade from HDFS to Ozone in future. It was assumed that having the two Datanodes in the same JVM will certainly simplify Hdfs to Ozone upgrade.
Some design work happened in past, but not much progress since then. However, that is still a valid goal, IMO.
Once users are comfortable to switch to Ozone completely, an in-place upgrade option will be desirable to avoid data copy, and will be a way faster option to move to Ozone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648310628
/pending "Blocking merge ability as we definitely need an agreement first. Possible topic for Community Meeting."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648309355
> The option to run them in same JVM was allowed, to support in-place upgrade from HDFS to Ozone in future
@jnp Base on the last design in-place upgrade can work with different JVM on the same host. And it's possible to run HDFS and Ozone datanodes on the same node.
My problem is that maintaining this feature requires significant effort. If we would like to do it, it's a release blocker:
```
2020-06-23 15:22:52 WARN DataNode:983 - ServicePlugin org.apache.hadoop.ozone.HddsDatanodeService@5226e402 could not be started
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] arp7 commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
arp7 commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648338641
+1 for documenting that this is not supported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648715033
> +1 for documenting that this is not supported.
Yes, it's important that I wouldn't like to modify interfaces or code. Just remove visibility (remove docs + compose examples).
We can make it work any time, or we can provide specific plugin for in-place upgrade.
It seems that we have an agreement that we can hide this feature, and we can further discuss long-term strategies **before any code change**.
As we have the consensus, I will merge this PR, soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648247605
Ping everybody (especially @jnp @arp7 @xiaoyuyao @mukul1987 @nandakumar131 )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] maobaolong commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648521788
+1, it will make Ozone repo clearly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek edited a comment on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek edited a comment on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648309355
> The option to run them in same JVM was allowed, to support in-place upgrade from HDFS to Ozone in future
@jnp Base on the last design in-place upgrade can work with different JVM on the same host. And it's possible to run HDFS and Ozone datanodes on the same node.
My problem is that maintaining this feature requires significant effort. If we would like to do it, it's a release blocker:
```
2020-06-23 15:22:52 WARN DataNode:983 - ServicePlugin org.apache.hadoop.ozone.HddsDatanodeService@5226e402 could not be started
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
```
> Ozone HDFS Datanode Plugin is now an experimental feature
We can call it as *a broken feature* ;-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] jnp commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
jnp commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648338154
> Given the fact that it is broken currently, it makes sense to remove the document saying it is supported.
I agree.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
xiaoyuyao commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648306954
@elek thanks for bringing this up. Given the fact that it is broken currently, it makes sense to remove the document saying it is supported. Let's open a separate JIRA to track the issues such as classpath, etc. that prevent running HDDS datanode as plugin inside HDFS datanode JVM. They need to be fixed when we revisit in-place upgrade.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] dineshchitlangia commented on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648265724
This is a double edged sword :)
Some of my customers that I speak to have indicated they would not directly go to Ozone in an independent fashion and a few of them have indicated they want to explore Ozone by itself.
How about we leave it as is and mention a cautionary statement to indicate that Ozone HDFS Datanode Plugin is now an experimental feature and it not recommended from Production use.
This will allow some users to get their feet wet with ozone in a limited way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] dineshchitlangia edited a comment on pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
dineshchitlangia edited a comment on pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117#issuecomment-648265724
This is a double edged sword :)
Some of my customers that I speak to have indicated they would not directly go to Ozone in an independent fashion and a few of them have indicated they want to explore Ozone by itself.
How about we leave it as is and mention a cautionary statement to indicate that Ozone HDFS Datanode Plugin is now an experimental feature and it not recommended from Production use.
This will allow some users to get their feet wet with ozone in a limited way (if it works to some extent).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek merged pull request #1117: HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
Posted by GitBox <gi...@apache.org>.
elek merged pull request #1117:
URL: https://github.com/apache/hadoop-ozone/pull/1117
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org