Investigation on the diffusion of innovation along with java releases

[f....@virgilio.it.INVALID]

Dear Developers,

we are members of the ZEST research group (Zurich Empirical Software Engineering Team) based at the University of Zurich and the Delft University of Technology. We are conducting an investigation on the diffusion of innovations and we focus on the adoption of new language features. Our research is focused on how API producers adapt their interfaces to introduce support for Java 8’s lambdas. During the course of our investigation, we manually inspected Apache Hadoop’s source code and documentation to understand whether Java’s lambdas have widespread adoption. We would like to have your feedback on our findings.

Our study focuses primarily on Functional Interfaces and Lambda Expressions as these new features were introduced by the Java language and adopted the Java JDK API, as they reduce implementation complexity, improve readability, offer performance benefits and improve security contextualization.
Our analysis showed that though hadoop-common 3.2.0 did not explicitly introduce support for functional interfaces (e.g. by using the @FunctionalInterface annotation). We noticed that the API does provide compatibility with Java 8+ features, including lambda expressions (since the API’s build platform is now on Java 11). We would like to better understand as to why no major change was necessitated to facilitate the usage of lambda expressions with the API.

In most cases, developers choose to move to new releases to satisfy particular dependency requirements, to take advantage of new Java features (like streams and functional interfaces in the case of Java 8), or just to standardize their implementation to align the API with the Java JDK API. Can you provide us with more information about this?
How did you and your team tackle the choice to change the version of Java supported?
Which factors did you take into account when doing this?
Are there any documented sources (e.g. Jira tickets, or issue tracker issues) about that discussion you can provide us with?
Why were no explicit changes made to the interface to support lambda expressions?
Are there any future plans in place to make larger changes to the API such that lambda expressions would be supported?

We thank you for taking the time to answer our questions. If you would like to be posted about the results of this study, please let us know!

Kind Regards,
Fernando Petrulio.


[photo-logo] 	
Fernando Petrulio
PhD Student University of Zurich UZH
Department of Informatics

[linkedin] https://www.linkedin.com/in/fernando-petrulio

Re: Investigation on the diffusion of innovation along with java releases

[Steve Loughran <st...@cloudera.com.INVALID>]

On Sat, Aug 10, 2019 at 4:08 PM <f....@virgilio.it.invalid> wrote:

> Dear Developers,
>
> we are members of the ZEST research group (Zurich Empirical Software
> Engineering Team) based at the University of Zurich and the Delft
> University of Technology. We are conducting an investigation on the
> diffusion of innovations and we focus on the adoption of new language
> features. Our research is focused on how API producers adapt their
> interfaces to introduce support for Java 8’s lambdas. During the course of
> our investigation, we manually inspected Apache Hadoop’s source code and
> documentation to understand whether Java’s lambdas have widespread
> adoption. We would like to have your feedback on our findings.
>
> Our study focuses primarily on Functional Interfaces and Lambda
> Expressions as these new features were introduced by the Java language and
> adopted the Java JDK API, as they reduce implementation complexity, improve
> readability, offer performance benefits and improve security
> contextualization.
>

they complicate refactoring as it isnt always obvious when a l-exp
implements a specific interface; you cant-'t just ask an IDE for all
implementations and add a parameter

Our analysis showed that though hadoop-common 3.2.0 did not explicitly
> introduce support for functional interfaces (e.g. by using the
> @FunctionalInterface annotation). We noticed that the API does provide
> compatibility with Java 8+ features, including lambda expressions (since
> the API’s build platform is now on Java 11). We would like to better
> understand as to why no major change was necessitated to facilitate the
> usage of lambda expressions with the API.
>
> In most cases, developers choose to move to new releases to satisfy
> particular dependency requirements, to take advantage of new Java features
> (like streams and functional interfaces in the case of Java 8), or just to
> standardize their implementation to align the API with the Java JDK API.
> Can you provide us with more information about this?
>

updating to a new Java version is always trouble, especially as it forces
every cluster/app to upgrade. We've tended to lag, probably too long


> How did you and your team tackle the choice to change the version of Java
> supported?
> Which factors did you take into account when doing this?
> Are there any documented sources (e.g. Jira tickets, or issue tracker
> issues) about that discussion you can provide us with?
>
https://issues.apache.org/jira/browse/HADOOP-11090


> Why were no explicit changes made to the interface to support lambda
> expressions?
>

given they weren't needed, why bother?

we added stuff in tests first, because we can play there without changing
APIs; look at the git history of org.apache.hadoop.test.LambdaTestUtils for
an example

Are there any future plans in place to make larger changes to the API such
> that lambda expressions would be supported?
>

1. Public APIs are long term support responsibilities -we need more
internal use before rushing in to this.
2. HADOOP-15229 shows some of the work with Java CompletableFutures
3. Checked exceptions in java cripple us. we can't use APIs like
CompletableFuture without catching and wrapping all IOEs. This
makes providing APIs even more complex. Look in org.apache.hadoop.fs.impl
for some of that work
5.  https://issues.apache.org/jira/browse/HADOOP-11867

We thank you for taking the time to answer our questions. If you would like
> to be posted about the results of this study, please let us know!
>

please send a link when done

>
> Kind Regards,
> Fernando Petrulio.
>
>
> [photo-logo]
> Fernando Petrulio
> PhD Student University of Zurich UZH
> Department of Informatics
>
> [linkedin] https://www.linkedin.com/in/fernando-petrulio
>