You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/11/16 13:03:08 UTC
[GitHub] [beam] iemejia commented on a change in pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit
iemejia commented on a change in pull request #13287:
URL: https://github.com/apache/beam/pull/13287#discussion_r524046472
##########
File path: runners/spark/build.gradle
##########
@@ -41,10 +42,15 @@ evaluationDependsOn(":runners:core-java")
configurations {
validatesRunner
- hadoopVersion285
}
-def hadoop_285_version = '2.8.5'
+def hadoopVersions = [
Review comment:
Better :+1:
##########
File path: sdks/java/io/hadoop-common/build.gradle
##########
@@ -16,17 +16,21 @@
* limitations under the License.
*/
+import java.util.stream.Collectors
+
plugins { id 'org.apache.beam.module' }
applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.hadoop.common')
description = "Apache Beam :: SDKs :: Java :: IO :: Hadoop Common"
ext.summary = "Library to add shared Hadoop classes among Beam IOs."
-configurations {
- hadoopVersion285
-}
+def hadoopVersions = [
+ "285": "2.8.5",
+ "292": "2.9.2",
+ "2101": "2.10.1",
Review comment:
Are we API compatible to even include here "3.2.1"? Does it pass? if so we can maybe include it in a subsequent PR (just thinking out loud for this and the other Hadoop modules)
##########
File path: sdks/java/io/hadoop-common/build.gradle
##########
@@ -16,17 +16,21 @@
* limitations under the License.
*/
+import java.util.stream.Collectors
+
plugins { id 'org.apache.beam.module' }
applyJavaNature(automaticModuleName: 'org.apache.beam.sdk.io.hadoop.common')
description = "Apache Beam :: SDKs :: Java :: IO :: Hadoop Common"
ext.summary = "Library to add shared Hadoop classes among Beam IOs."
-configurations {
- hadoopVersion285
-}
+def hadoopVersions = [
+ "285": "2.8.5",
+ "292": "2.9.2",
+ "2101": "2.10.1",
Review comment:
I also was thinking if we could define this globally so we don't have this copy pasted everywhere as well as the hadoopVersions chunk, but my gradle fu is not so high to purpose a way to do it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org