You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ahmed Hussein (Jira)" <ji...@apache.org> on 2020/07/15 23:01:00 UTC
[jira] [Updated] (HADOOP-17098) Reduce Guava dependency in Hadoop
source code
[ https://issues.apache.org/jira/browse/HADOOP-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Hussein updated HADOOP-17098:
-----------------------------------
Description:
Relying on Guava implementation in Hadoop has been painful due to compatibility and vulnerability issues.
Guava updates tend to break/deprecate APIs. This made It hard to maintain backward compatibility within hadoop versions and clients/downstreams.
With 3.x uses java8+, the java 8 features should preferred to Guava, reducing the footprint, and giving stability to source code.
This jira should serve as an umbrella toward an incremental effort to reduce the usage of Guava in the source code and to create subtasks to replace Guava classes with Java features.
Furthermore, it will be good to add a rule in the pre-commit build to warn against introducing a new Guava usage in certain modules.
Any one willing to take part in this code refactoring has to:
# Focus on one module at a time in order to reduce the conflicts and the size of the patch. This will significantly help the reviewers.
# Run all the unit tests related to the module being affected by the change. It is critical to verify that any change will not break the unit tests, or cause a stable test case to become flaky.
# Merge should be done to the following branches: trunk, branch-3.3, branch-3.2, branch-3.1
A list of sub tasks replacing Guava APIs with java8 features:
{code:java}
com.google.common.io.BaseEncoding#base64() java.util.Base64
com.google.common.io.BaseEncoding#base64Url() java.util.Base64
com.google.common.base.Joiner.on() java.lang.String#join() or
java.util.stream.Collectors#joining()
com.google.common.base.Optional#of() java.util.Optional#of()
com.google.common.base.Optional#absent() java.util.Optional#empty()
com.google.common.base.Optional#fromNullable() java.util.Optional#ofNullable()
com.google.common.base.Optional java.util.Optional
com.google.common.base.Predicate java.util.function.Predicate
com.google.common.base.Function java.util.function.Function
com.google.common.base.Supplier java.util.function.Supplier
{code}
I also vote for the replacement of {{Precondition}} with either a wrapper, or Apache commons lang.
I believe you guys have dealt with Guava compatibilities in the past and probably have better insights. Any thoughts? [~weichiu], [~gabor.bota], [~stevel@apache.org], [~ayushtkn], [~busbey], [~jeagles], [~kihwal]
was:
Relying on Guava implementation in Hadoop has been painful due to compatibility and vulnerability issues.
Guava updates tend to break/deprecate APIs. This made It hard to maintain backward compatibility within hadoop versions and clients/downstreams.
With 3.x uses java8+, the java 8 features should preferred to Guava, reducing the footprint, and giving stability to source code.
This jira should serve as an umbrella toward an incremental effort to reduce the usage of Guava in the source code and to create subtasks to replace Guava classes with Java features.
Furthermore, it will be good to add a rule in the pre-commit build to warn against introducing a new Guava usage in certain modules.
Any one willing to take part in this code refactoring has to:
# Focus on one module at a time in order to reduce the conflicts and the size of the patch. This will significantly help the reviewers.
# Run all the unit tests related to the module being affected by the change. It is critical to verify that any change will not break the unit tests, or cause a stable test case to become flaky.
A list of sub tasks replacing Guava APIs with java8 features:
{code:java}
com.google.common.io.BaseEncoding#base64() java.util.Base64
com.google.common.io.BaseEncoding#base64Url() java.util.Base64
com.google.common.base.Joiner.on() java.lang.String#join() or
java.util.stream.Collectors#joining()
com.google.common.base.Optional#of() java.util.Optional#of()
com.google.common.base.Optional#absent() java.util.Optional#empty()
com.google.common.base.Optional#fromNullable() java.util.Optional#ofNullable()
com.google.common.base.Optional java.util.Optional
com.google.common.base.Predicate java.util.function.Predicate
com.google.common.base.Function java.util.function.Function
com.google.common.base.Supplier java.util.function.Supplier
{code}
I also vote for the replacement of {{Precondition}} with either a wrapper, or Apache commons lang.
I believe you guys have dealt with Guava compatibilities in the past and probably have better insights. Any thoughts? [~weichiu], [~gabor.bota], [~stevel@apache.org], [~ayushtkn], [~busbey], [~jeagles], [~kihwal]
> Reduce Guava dependency in Hadoop source code
> ---------------------------------------------
>
> Key: HADOOP-17098
> URL: https://issues.apache.org/jira/browse/HADOOP-17098
> Project: Hadoop Common
> Issue Type: Task
> Reporter: Ahmed Hussein
> Assignee: Ahmed Hussein
> Priority: Major
>
> Relying on Guava implementation in Hadoop has been painful due to compatibility and vulnerability issues.
> Guava updates tend to break/deprecate APIs. This made It hard to maintain backward compatibility within hadoop versions and clients/downstreams.
> With 3.x uses java8+, the java 8 features should preferred to Guava, reducing the footprint, and giving stability to source code.
> This jira should serve as an umbrella toward an incremental effort to reduce the usage of Guava in the source code and to create subtasks to replace Guava classes with Java features.
> Furthermore, it will be good to add a rule in the pre-commit build to warn against introducing a new Guava usage in certain modules.
> Any one willing to take part in this code refactoring has to:
> # Focus on one module at a time in order to reduce the conflicts and the size of the patch. This will significantly help the reviewers.
> # Run all the unit tests related to the module being affected by the change. It is critical to verify that any change will not break the unit tests, or cause a stable test case to become flaky.
> # Merge should be done to the following branches: trunk, branch-3.3, branch-3.2, branch-3.1
>
> A list of sub tasks replacing Guava APIs with java8 features:
> {code:java}
> com.google.common.io.BaseEncoding#base64() java.util.Base64
> com.google.common.io.BaseEncoding#base64Url() java.util.Base64
> com.google.common.base.Joiner.on() java.lang.String#join() or
> java.util.stream.Collectors#joining()
> com.google.common.base.Optional#of() java.util.Optional#of()
> com.google.common.base.Optional#absent() java.util.Optional#empty()
> com.google.common.base.Optional#fromNullable() java.util.Optional#ofNullable()
> com.google.common.base.Optional java.util.Optional
> com.google.common.base.Predicate java.util.function.Predicate
> com.google.common.base.Function java.util.function.Function
> com.google.common.base.Supplier java.util.function.Supplier
> {code}
>
> I also vote for the replacement of {{Precondition}} with either a wrapper, or Apache commons lang.
> I believe you guys have dealt with Guava compatibilities in the past and probably have better insights. Any thoughts? [~weichiu], [~gabor.bota], [~stevel@apache.org], [~ayushtkn], [~busbey], [~jeagles], [~kihwal]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org