You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/02/18 18:14:05 UTC
[GitHub] [iceberg] kbendick opened a new pull request #4167: Update release scripts to support Scala 2.13
kbendick opened a new pull request #4167:
URL: https://github.com/apache/iceberg/pull/4167
Noticed we added support for Scala 2.13 with Spark 3.2 recently, but the release scripts have not been updated to reflect that. The nightly release workflow has been updated to publish Scala 2.13 artifacts: https://github.com/apache/iceberg/pull/4009
This came up as a result of this discussion: https://github.com/apache/iceberg/pull/4157#discussion_r809710633. There's probably more work to do, and I can covert this to an issue if somebody else wants to take over.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on a change in pull request #4167: Update release scripts to support Scala 2.13
Posted by GitBox <gi...@apache.org>.
kbendick commented on a change in pull request #4167:
URL: https://github.com/apache/iceberg/pull/4167#discussion_r811481701
##########
File path: dev/stage-binaries.sh
##########
@@ -17,10 +17,16 @@
# specific language governing permissions and limitations
# under the License.
#
-
+SCALA_VERSION=2.12
FLINK_VERSIONS=1.12,1.13,1.14
SPARK_VERSIONS=2.4,3.0,3.1,3.2
HIVE_VERSIONS=2,3
-./gradlew -Prelease -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+./gradlew -Prelease -DscalaVersion=$SCALA_VERSION -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+
+# Also publish Scala 2.13 Artifacts for versions that support it.
+# Flink does not yet support 2.13 and Hive doesn't need a Scala specification.
+SCALA_VERSION=2.13
+SPARK_VERSIONS_2.13=3.2
Review comment:
Removing these anyway for now, but good call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4167: Update release scripts to support Scala 2.13
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4167:
URL: https://github.com/apache/iceberg/pull/4167#discussion_r810655206
##########
File path: dev/stage-binaries.sh
##########
@@ -17,10 +17,16 @@
# specific language governing permissions and limitations
# under the License.
#
-
+SCALA_VERSION=2.12
FLINK_VERSIONS=1.12,1.13,1.14
SPARK_VERSIONS=2.4,3.0,3.1,3.2
HIVE_VERSIONS=2,3
-./gradlew -Prelease -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+./gradlew -Prelease -DscalaVersion=$SCALA_VERSION -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+
+# Also publish Scala 2.13 Artifacts for versions that support it.
+# Flink does not yet support 2.13 and Hive doesn't need a Scala specification.
+SCALA_VERSION=2.13
+SPARK_VERSIONS_2.13=3.2
Review comment:
Adding `_2.13` is a bit confusing since the Scala version is parameterized. What about just redefining `SPARK_VERSIONS` like you do with `SCALA_VERSION`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4167: Update release scripts to support Scala 2.13
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4167:
URL: https://github.com/apache/iceberg/pull/4167#discussion_r810655109
##########
File path: dev/stage-binaries.sh
##########
@@ -17,10 +17,16 @@
# specific language governing permissions and limitations
# under the License.
#
-
+SCALA_VERSION=2.12
FLINK_VERSIONS=1.12,1.13,1.14
SPARK_VERSIONS=2.4,3.0,3.1,3.2
HIVE_VERSIONS=2,3
-./gradlew -Prelease -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+./gradlew -Prelease -DscalaVersion=$SCALA_VERSION -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+
+# Also publish Scala 2.13 Artifacts for versions that support it.
+# Flink does not yet support 2.13 and Hive doesn't need a Scala specification.
+SCALA_VERSION=2.13
+SPARK_VERSIONS_2.13=3.2
+./gradlew -P release -DscalaVersion=$SCALA_VERSION -DsparkVersions=$SPARK_VERSIONS_2.13 -DhiveVersions= -DflinkVersions= publishApachePublicationToMavenRepository
Review comment:
Nit: extra space after `release`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on a change in pull request #4167: Update release scripts to support Scala 2.13
Posted by GitBox <gi...@apache.org>.
kbendick commented on a change in pull request #4167:
URL: https://github.com/apache/iceberg/pull/4167#discussion_r811481642
##########
File path: dev/stage-binaries.sh
##########
@@ -17,10 +17,16 @@
# specific language governing permissions and limitations
# under the License.
#
-
+SCALA_VERSION=2.12
FLINK_VERSIONS=1.12,1.13,1.14
SPARK_VERSIONS=2.4,3.0,3.1,3.2
HIVE_VERSIONS=2,3
-./gradlew -Prelease -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+./gradlew -Prelease -DscalaVersion=$SCALA_VERSION -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+
+# Also publish Scala 2.13 Artifacts for versions that support it.
+# Flink does not yet support 2.13 and Hive doesn't need a Scala specification.
+SCALA_VERSION=2.13
+SPARK_VERSIONS_2.13=3.2
+./gradlew -P release -DscalaVersion=$SCALA_VERSION -DsparkVersions=$SPARK_VERSIONS_2.13 -DhiveVersions= -DflinkVersions= publishApachePublicationToMavenRepository
Review comment:
Updated to just be this call directly for now, with the comments but without the updated template variables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #4167: Update release scripts to support Scala 2.13
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #4167:
URL: https://github.com/apache/iceberg/pull/4167#discussion_r810655645
##########
File path: dev/stage-binaries.sh
##########
@@ -17,10 +17,16 @@
# specific language governing permissions and limitations
# under the License.
#
-
+SCALA_VERSION=2.12
FLINK_VERSIONS=1.12,1.13,1.14
SPARK_VERSIONS=2.4,3.0,3.1,3.2
HIVE_VERSIONS=2,3
-./gradlew -Prelease -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+./gradlew -Prelease -DscalaVersion=$SCALA_VERSION -DflinkVersions=$FLINK_VERSIONS -DsparkVersions=$SPARK_VERSIONS -DhiveVersions=$HIVE_VERSIONS publishApachePublicationToMavenRepository
+
+# Also publish Scala 2.13 Artifacts for versions that support it.
+# Flink does not yet support 2.13 and Hive doesn't need a Scala specification.
+SCALA_VERSION=2.13
+SPARK_VERSIONS_2.13=3.2
+./gradlew -P release -DscalaVersion=$SCALA_VERSION -DsparkVersions=$SPARK_VERSIONS_2.13 -DhiveVersions= -DflinkVersions= publishApachePublicationToMavenRepository
Review comment:
I think that this needs to exclude all of the non-Spark libraries, like we do in the workflows that just run Spark tests. That would look like this:
```bash
./gradlew -P release :iceberg-spark:iceberg-spark-3.2_2.13:publishApachePublicationToMavenRepository :iceberg-spark:iceberg-spark-extensions-3.2_2.13:publishApachePublicationToMavenRepository :iceberg-spark:iceberg-spark-runtime-3.2_2.13:publishApachePublicationToMavenRepository
```
You could parameterize that, but it may be worth just adding it and we can worry about updating it when we have more versions later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org