You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@iceberg.apache.org by Ryan Blue <bl...@apache.org> on 2022/10/09 22:42:20 UTC

[VOTE] Release Apache Iceberg 1.0.0 RC0

Hi Everyone,

I propose that we release the following RC as the official Apache Iceberg
1.0.0 release.

The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
* This corresponds to the tag: apache-iceberg-1.0.0-rc0
* https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
*
https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01

The release tarball, signature, and checksums are here:
* https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0

You can find the KEYS file here:
* https://dist.apache.org/repos/dist/dev/iceberg/KEYS

Convenience binary artifacts are staged on Nexus. The Maven repository URL
is:
* https://repository.apache.org/content/repositories/orgapacheiceberg-1106/

Please download, verify, and test.

This release is based on the latest 0.14.1 release. It includes changes to
remove deprecated APIs and the following additional bug fixes:
* Increase metrics limit to 100 columns
* Bump Spark patch versions for CVE-2022-33891
* Exclude Scala from Spark runtime Jars

Please vote in the next 72 hours.

[ ] +1 Release this as Apache Iceberg 1.0.0
[ ] +0
[ ] -1 Do not release this because...


-- 
Ryan Blue

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Ryan Blue <bl...@tabular.io>.

With 12 +1 votes (3 binding) and no -1, this passes.

Thanks, everyone! We'll start getting the rest of the release process going.

On Fri, Oct 14, 2022 at 6:10 AM leilei hu <hu...@gmail.com> wrote:

> +1(non-binding)
> verified(java 8):
>
> - Create table using HiveCatalog and HadoopCatalog
> - Spark Structured Streaming
> - Spark query with Spark’s DataSourceV2 API
> - Checksum, build and test
>
>
>
> 2022年10月11日 下午2:57，Jahagirdar, Amogh <ja...@amazon.com.INVALID> 写道：
>
> +1 (non-binding)
>
> Verified signature/checksums/license/build/tests with JDK 8
>
> Thanks,
>
> Amogh Jahagirdar
>
> *From: *Yufei Gu <fl...@gmail.com>
> *Reply-To: *"dev@iceberg.apache.org" <de...@iceberg.apache.org>
> *Date: *Monday, October 10, 2022 at 11:52 PM
> *To: *"dev@iceberg.apache.org" <de...@iceberg.apache.org>
> *Subject: *RE: [EXTERNAL][VOTE] Release Apache Iceberg 1.0.0 RC0
>
>
> *CAUTION*: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
> +1 (non-binding)
> - Verified Signature
> - Verified Checksum
> - Verified License
> - Verified build and test with Java 11
> - Test with Apache Spark 3.3
>
> Best,
>
> Yufei
>
> `This is not a contribution`
>
>
> On Mon, Oct 10, 2022 at 7:07 PM Rushan Jiang <ru...@andrew.cmu.edu>
> wrote:
>
> +1 (non-binding)
> - Verify Signature
> - Verify Checksums
> - Verify License Documentation
> - Verify Build and Test in Java 8 and 11
>
> On Mon, Oct 10, 2022 at 7:11 PM Daniel Weeks <dw...@apache.org> wrote:
>
> +1 (binding)
>
> Verified sigs/sums/license/build/test (java 8)
>
> Everything checks out,
> -Dan
>
> On Mon, Oct 10, 2022 at 3:25 PM Steven Wu <st...@gmail.com> wrote:
>
> Never mind. Missed the information that this 1.0.0 is based on the latest
> 0.14.1 release, which doesn't contain the PR 5318. I thought it was based
> on the latest master branch.
>
> +1 (non-binding)
> - Verify signature
> - Verify checksum
> - Tried SQL insert and query with Flink 1.15
>
> On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <st...@gmail.com> wrote:
>
> Ryan,
>
> It seems that this PR (merged on July 28) for the Flink FLIP-27 source is
> not included. https://github.com/apache/iceberg/pull/5318
>
> The commit ID still contains the old file in the old location. The new
> location should be "flink/source/IcebergTableSource.java" with FLIP-27
> config support.
>
> https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
>
> Thanks,
> Steven
>
> On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com>
> wrote:
>
> Whoops, sorry for the noise, I made a typo and was using the wrong scala
> version of the iceberg-spark-runtime jar, this works.
>
> +1 (non-binding)
> - Verify signature
> - Verify checksum
> - Verify license documentation
> - Tried with Spark 3.3
> - Ran unit tests
>
> Thanks
> Szehon
>
>
>
>
> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>
> wrote:
>
> Hi,
>
> I get a NoClassDefFoundError from IcebergSparkExtensions when running
> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
> jar doesn't contain scala classes, unlike previous jars
> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>
> scala> spark.sql("show databases").show
> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>   at
> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>   at
> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>   at
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>   at
> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>   at
> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>   at
> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>   at
> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>   at
> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>   at
> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>   at
> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>   at
> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>   at
> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>   at
> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>   at
> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>   at
> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>   at
> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>   at
> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at
> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>   at
> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>   at
> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>   at
> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>   at
> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>   ... 47 elided
> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>   at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>   at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>   ... 79 more
>
> Note, I usually verify by copying the spark-runtime jar to spark jars dir
> (can't usually get --packages flag to work as indicated on
> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
> version is not released yet), so let me know if I am using the wrong jar?
>
> Thanks
> Szehon
>
> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
> wrote:
>
> +1 (non-binding)
>
>    - validated checksum and signature
>    - checked license docs & ran RAT checks
>    - ran build and tests with JDK11
>
>
> Eduard
>
> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
> wrote:
>
> +1 (non-binding)
>
>    - Verified the Spark runtime jar contents.
>    - Checked license docs, ran RAT checks.
>    - Validated checksum and signature.
>
>
> Thanks,
> Ajantha
>
> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <pr...@gmail.com>
> wrote:
>
> Hello Everyone,
>
> Wanted to know your thoughts on whether we should also include the
> following bug fixes in this release as well:
>
> 1. MERGE INTO nullability fix, leads to query failure otherwise:
> *Reported instances :*
> a.
> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
> b. https://github.com/apache/iceberg/issues/5739
> c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>
> *PR's (Merged):*
> a. https://github.com/apache/iceberg/pull/5880
> b. https://github.com/apache/iceberg/pull/5679
>
> 2.  QueryFailure when running RewriteManifestProcedure on Date / Timestamp
> partitioned table when
> `spark.sql.datetime.java8API.enabled` is true.
> *Reported instances :*
> a. https://github.com/apache/iceberg/issues/5104
> b. https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>
> *PR* :
> a. https://github.com/apache/iceberg/pull/5860
>
> Regards,
> Prashant Singh
>
> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>
> +1 (binding)
>
>    - Checked license docs, ran RAT checks
>    - Validated checksum and signature
>    - Built and tested with Java 11
>    - Built binary artifacts with Java 8
>
>
> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>
> Hi Everyone,
>
> I propose that we release the following RC as the official Apache Iceberg
> 1.0.0 release.
>
> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
> *
> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>
> The release tarball, signature, and checksums are here:
> * https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>
> Convenience binary artifacts are staged on Nexus. The Maven repository URL
> is:
> *
> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>
> Please download, verify, and test.
>
> This release is based on the latest 0.14.1 release. It includes changes to
> remove deprecated APIs and the following additional bug fixes:
> * Increase metrics limit to 100 columns
> * Bump Spark patch versions for CVE-2022-33891
> * Exclude Scala from Spark runtime Jars
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Iceberg 1.0.0
> [ ] +0
> [ ] -1 Do not release this because...
>
>
> --
> Ryan Blue
>
>
>
> --
> Ryan Blue
>
>
>
> --
> *Rushan Jiang*
>
>
>

-- 
Ryan Blue
Tabular

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by leilei hu <hu...@gmail.com>.

+1(non-binding)
verified(java 8):

- Create table using HiveCatalog and HadoopCatalog
- Spark Structured Streaming
- Spark query with Spark’s DataSourceV2 API 
- Checksum, build and test



> 2022年10月11日 下午2:57，Jahagirdar, Amogh <ja...@amazon.com.INVALID> 写道：
> 
> +1 (non-binding)
> 
> Verified signature/checksums/license/build/tests with JDK 8
> 
> Thanks,
> 
> Amogh Jahagirdar
>  
> From: Yufei Gu <flyrain000@gmail.com <ma...@gmail.com>>
> Reply-To: "dev@iceberg.apache.org <ma...@iceberg.apache.org>" <dev@iceberg.apache.org <ma...@iceberg.apache.org>>
> Date: Monday, October 10, 2022 at 11:52 PM
> To: "dev@iceberg.apache.org <ma...@iceberg.apache.org>" <dev@iceberg.apache.org <ma...@iceberg.apache.org>>
> Subject: RE: [EXTERNAL][VOTE] Release Apache Iceberg 1.0.0 RC0
>  
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
>  
> +1 (non-binding) 
> - Verified Signature
> - Verified Checksum
> - Verified License
> - Verified build and test with Java 11
> - Test with Apache Spark 3.3
> 
> Best,
>  
> Yufei
>  
> `This is not a contribution`
>  
>  
> On Mon, Oct 10, 2022 at 7:07 PM Rushan Jiang <rushanj@andrew.cmu.edu <ma...@andrew.cmu.edu>> wrote:
> +1 (non-binding)
> - Verify Signature
> - Verify Checksums
> - Verify License Documentation
> - Verify Build and Test in Java 8 and 11
>  
> On Mon, Oct 10, 2022 at 7:11 PM Daniel Weeks <dweeks@apache.org <ma...@apache.org>> wrote:
> +1 (binding)
>  
> Verified sigs/sums/license/build/test (java 8)
>  
> Everything checks out,
> -Dan
>  
> On Mon, Oct 10, 2022 at 3:25 PM Steven Wu <stevenz3wu@gmail.com <ma...@gmail.com>> wrote:
> Never mind. Missed the information that this 1.0.0 is based on the latest 0.14.1 release, which doesn't contain the PR 5318. I thought it was based on the latest master branch.
>  
> +1 (non-binding)
> - Verify signature
> - Verify checksum
> - Tried SQL insert and query with Flink 1.15
>  
> On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <stevenz3wu@gmail.com <ma...@gmail.com>> wrote:
> Ryan, 
>  
> It seems that this PR (merged on July 28) for the Flink FLIP-27 source is not included. https://github.com/apache/iceberg/pull/5318 <https://github.com/apache/iceberg/pull/5318>
>  
> The commit ID still contains the old file in the old location. The new location should be "flink/source/IcebergTableSource.java" with FLIP-27 config support.
> https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java <https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java>
>  
> Thanks,
> Steven
>  
> On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <szehon.apache@gmail.com <ma...@gmail.com>> wrote:
> Whoops, sorry for the noise, I made a typo and was using the wrong scala version of the iceberg-spark-runtime jar, this works.
>  
> +1 (non-binding)
> - Verify signature
> - Verify checksum
> - Verify license documentation
> - Tried with Spark 3.3
> - Ran unit tests
>  
> Thanks
> Szehon
>  
>  
>  
>  
> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <szehon.apache@gmail.com <ma...@gmail.com>> wrote:
> Hi, 
>  
> I get a NoClassDefFoundError from IcebergSparkExtensions when running Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this jar doesn't contain scala classes, unlike previous jars iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>  
> scala> spark.sql("show databases").show
> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>   at org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>   at org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>   at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>   at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>   at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>   at org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>   at org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>   at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>   at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>   at org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>   at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>   at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>   at org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>   at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>   at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>   at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>   at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>   at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>   at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>   at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>   at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>   ... 47 elided
> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>   at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>   at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>   ... 79 more
>  
> Note, I usually verify by copying the spark-runtime jar to spark jars dir (can't usually get --packages flag to work as indicated onhttps://iceberg.apache.org/how-to-release/#verifying-with-spark <https://iceberg.apache.org/how-to-release/#verifying-with-spark>, as version is not released yet), so let me know if I am using the wrong jar?
>  
> Thanks
> Szehon
>  
> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <eduard@tabular.io <ma...@tabular.io>> wrote:
> +1 (non-binding)
> validated checksum and signature
> checked license docs & ran RAT checks
> ran build and tests with JDK11
>  
> Eduard
>  
> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <ajanthabhat@gmail.com <ma...@gmail.com>> wrote:
> +1 (non-binding)
> 
> Verified the Spark runtime jar contents.
> Checked license docs, ran RAT checks.
> Validated checksum and signature.
> 
> Thanks, 
> Ajantha
>  
> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <prashant010696@gmail.com <ma...@gmail.com>> wrote:
> Hello Everyone,
> 
> Wanted to know your thoughts on whether we should also include the following bug fixes in this release as well:
> 
> 1. MERGE INTO nullability fix, leads to query failure otherwise:
> Reported instances :
> a. https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep <https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep>
> b. https://github.com/apache/iceberg/issues/5739 <https://github.com/apache/iceberg/issues/5739>
> c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298 <https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298>
> 
> PR's (Merged):
> a. https://github.com/apache/iceberg/pull/5880 <https://github.com/apache/iceberg/pull/5880>
> b. https://github.com/apache/iceberg/pull/5679 <https://github.com/apache/iceberg/pull/5679>
> 
> 2.  QueryFailure when running RewriteManifestProcedure on Date / Timestamp partitioned table when
> `spark.sql.datetime.java8API.enabled` is true.
> Reported instances :
> a. https://github.com/apache/iceberg/issues/5104 <https://github.com/apache/iceberg/issues/5104>
> b. https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469 <https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469>
> 
> PR :
> a. https://github.com/apache/iceberg/pull/5860 <https://github.com/apache/iceberg/pull/5860>
> 
> Regards,
> Prashant Singh
>  
> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <blue@apache.org <ma...@apache.org>> wrote:
> +1 (binding)
> 
> Checked license docs, ran RAT checks 
> Validated checksum and signature 
> Built and tested with Java 11 
> Built binary artifacts with Java 8 
>  
> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <blue@apache.org <ma...@apache.org>> wrote:
> Hi Everyone,
> 
> I propose that we release the following RC as the official Apache Iceberg 1.0.0 release.
> 
> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0 <https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0>
> * https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01 <https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01>
> 
> The release tarball, signature, and checksums are here:
> * https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0 <https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0>
> 
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS <https://dist.apache.org/repos/dist/dev/iceberg/KEYS>
> 
> Convenience binary artifacts are staged on Nexus. The Maven repository URL is:
> * https://repository.apache.org/content/repositories/orgapacheiceberg-1106/ <https://repository.apache.org/content/repositories/orgapacheiceberg-1106/>
> 
> Please download, verify, and test. 
>  
> This release is based on the latest 0.14.1 release. It includes changes to remove deprecated APIs and the following additional bug fixes:
> * Increase metrics limit to 100 columns
> * Bump Spark patch versions for CVE-2022-33891
> * Exclude Scala from Spark runtime Jars
> 
> Please vote in the next 72 hours.
> 
> [ ] +1 Release this as Apache Iceberg 1.0.0
> [ ] +0
> [ ] -1 Do not release this because...
>  
>  
> -- 
> Ryan Blue
> 
>  
> -- 
> Ryan Blue
> 
>  
> -- 
> Rushan Jiang

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by "Jahagirdar, Amogh" <ja...@amazon.com.INVALID>.

+1 (non-binding)

Verified signature/checksums/license/build/tests with JDK 8

Thanks,

Amogh Jahagirdar

From: Yufei Gu <fl...@gmail.com>
Reply-To: "dev@iceberg.apache.org" <de...@iceberg.apache.org>
Date: Monday, October 10, 2022 at 11:52 PM
To: "dev@iceberg.apache.org" <de...@iceberg.apache.org>
Subject: RE: [EXTERNAL][VOTE] Release Apache Iceberg 1.0.0 RC0


CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.


+1 (non-binding)
- Verified Signature
- Verified Checksum
- Verified License
- Verified build and test with Java 11
- Test with Apache Spark 3.3

Best,

Yufei

`This is not a contribution`


On Mon, Oct 10, 2022 at 7:07 PM Rushan Jiang <ru...@andrew.cmu.edu>> wrote:
+1 (non-binding)
- Verify Signature
- Verify Checksums
- Verify License Documentation
- Verify Build and Test in Java 8 and 11

On Mon, Oct 10, 2022 at 7:11 PM Daniel Weeks <dw...@apache.org>> wrote:
+1 (binding)

Verified sigs/sums/license/build/test (java 8)

Everything checks out,
-Dan

On Mon, Oct 10, 2022 at 3:25 PM Steven Wu <st...@gmail.com>> wrote:
Never mind. Missed the information that this 1.0.0 is based on the latest 0.14.1 release, which doesn't contain the PR 5318. I thought it was based on the latest master branch.

+1 (non-binding)
- Verify signature
- Verify checksum
- Tried SQL insert and query with Flink 1.15

On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <st...@gmail.com>> wrote:
Ryan,

It seems that this PR (merged on July 28) for the Flink FLIP-27 source is not included. https://github.com/apache/iceberg/pull/5318

The commit ID still contains the old file in the old location. The new location should be "flink/source/IcebergTableSource.java" with FLIP-27 config support.
https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java

Thanks,
Steven

On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com>> wrote:
Whoops, sorry for the noise, I made a typo and was using the wrong scala version of the iceberg-spark-runtime jar, this works.

+1 (non-binding)
- Verify signature
- Verify checksum
- Verify license documentation
- Tried with Spark 3.3
- Ran unit tests

Thanks
Szehon




On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>> wrote:
Hi,

I get a NoClassDefFoundError from IcebergSparkExtensions when running Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this jar doesn't contain scala classes, unlike previous jars iceberg-spark-runtime-3.3_2.12-0.14.1.jar.

scala> spark.sql("show databases").show
java.lang.NoClassDefFoundError: scala/collection/SeqOps
  at org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
  at org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
  at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
  at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
  at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
  at scala.collection.TraversableLike.map(TraversableLike.scala:286)
  at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
  at scala.collection.AbstractTraversable.map(Traversable.scala:108)
  at org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
  at org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
  at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
  at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
  at org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
  at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
  at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
  at org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
  at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
  at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
  at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
  at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
  at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
  at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
  at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
  at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
  ... 47 elided
Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
  at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
  at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
  ... 79 more

Note, I usually verify by copying the spark-runtime jar to spark jars dir (can't usually get --packages flag to work as indicated on https://iceberg.apache.org/how-to-release/#verifying-with-spark, as version is not released yet), so let me know if I am using the wrong jar?

Thanks
Szehon

On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>> wrote:
+1 (non-binding)

  *   validated checksum and signature
  *   checked license docs & ran RAT checks
  *   ran build and tests with JDK11

Eduard

On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>> wrote:
+1 (non-binding)

  *   Verified the Spark runtime jar contents.
  *   Checked license docs, ran RAT checks.
  *   Validated checksum and signature.

Thanks,
Ajantha

On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <pr...@gmail.com>> wrote:
Hello Everyone,

Wanted to know your thoughts on whether we should also include the following bug fixes in this release as well:

1. MERGE INTO nullability fix, leads to query failure otherwise:
Reported instances :
a. https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
b. https://github.com/apache/iceberg/issues/5739
c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298

PR's (Merged):
a. https://github.com/apache/iceberg/pull/5880
b. https://github.com/apache/iceberg/pull/5679

2.  QueryFailure when running RewriteManifestProcedure on Date / Timestamp partitioned table when
`spark.sql.datetime.java8API.enabled` is true.
Reported instances :
a. https://github.com/apache/iceberg/issues/5104
b. https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469

PR :
a. https://github.com/apache/iceberg/pull/5860

Regards,
Prashant Singh

On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org>> wrote:

+1 (binding)

  *   Checked license docs, ran RAT checks
  *   Validated checksum and signature
  *   Built and tested with Java 11
  *   Built binary artifacts with Java 8

On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org>> wrote:
Hi Everyone,

I propose that we release the following RC as the official Apache Iceberg 1.0.0 release.

The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
* This corresponds to the tag: apache-iceberg-1.0.0-rc0
* https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
* https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01

The release tarball, signature, and checksums are here:
* https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0

You can find the KEYS file here:
* https://dist.apache.org/repos/dist/dev/iceberg/KEYS

Convenience binary artifacts are staged on Nexus. The Maven repository URL is:
* https://repository.apache.org/content/repositories/orgapacheiceberg-1106/

Please download, verify, and test.

This release is based on the latest 0.14.1 release. It includes changes to remove deprecated APIs and the following additional bug fixes:
* Increase metrics limit to 100 columns
* Bump Spark patch versions for CVE-2022-33891
* Exclude Scala from Spark runtime Jars

Please vote in the next 72 hours.

[ ] +1 Release this as Apache Iceberg 1.0.0
[ ] +0
[ ] -1 Do not release this because...


--
Ryan Blue


--
Ryan Blue


--
Rushan Jiang

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Yufei Gu <fl...@gmail.com>.

+1 (non-binding)
- Verified Signature
- Verified Checksum
- Verified License
- Verified build and test with Java 11
- Test with Apache Spark 3.3

Best,

Yufei

`This is not a contribution`


On Mon, Oct 10, 2022 at 7:07 PM Rushan Jiang <ru...@andrew.cmu.edu> wrote:

> +1 (non-binding)
> - Verify Signature
> - Verify Checksums
> - Verify License Documentation
> - Verify Build and Test in Java 8 and 11
>
> On Mon, Oct 10, 2022 at 7:11 PM Daniel Weeks <dw...@apache.org> wrote:
>
>> +1 (binding)
>>
>> Verified sigs/sums/license/build/test (java 8)
>>
>> Everything checks out,
>> -Dan
>>
>> On Mon, Oct 10, 2022 at 3:25 PM Steven Wu <st...@gmail.com> wrote:
>>
>>> Never mind. Missed the information that this 1.0.0 is based on the
>>> latest 0.14.1 release, which doesn't contain the PR 5318. I thought it was
>>> based on the latest master branch.
>>>
>>> +1 (non-binding)
>>> - Verify signature
>>> - Verify checksum
>>> - Tried SQL insert and query with Flink 1.15
>>>
>>> On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <st...@gmail.com> wrote:
>>>
>>>> Ryan,
>>>>
>>>> It seems that this PR (merged on July 28) for the Flink FLIP-27 source
>>>> is not included. https://github.com/apache/iceberg/pull/5318
>>>>
>>>> The commit ID still contains the old file in the old location. The new
>>>> location should be "flink/source/IcebergTableSource.java" with FLIP-27
>>>> config support.
>>>>
>>>> https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
>>>>
>>>> Thanks,
>>>> Steven
>>>>
>>>> On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com>
>>>> wrote:
>>>>
>>>>> Whoops, sorry for the noise, I made a typo and was using the wrong
>>>>> scala version of the iceberg-spark-runtime jar, this works.
>>>>>
>>>>> +1 (non-binding)
>>>>> - Verify signature
>>>>> - Verify checksum
>>>>> - Verify license documentation
>>>>> - Tried with Spark 3.3
>>>>> - Ran unit tests
>>>>>
>>>>> Thanks
>>>>> Szehon
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I get a NoClassDefFoundError from IcebergSparkExtensions when running
>>>>>> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
>>>>>> jar doesn't contain scala classes, unlike previous jars
>>>>>> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>>>>>>
>>>>>> scala> spark.sql("show databases").show
>>>>>> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>>>>>>   at
>>>>>> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>>>>>>   at
>>>>>> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>>>>>>   at
>>>>>> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>>>>>>   at
>>>>>> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>>>>>>   at
>>>>>> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>>>>>>   at
>>>>>> scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>>>>>>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>>>>>>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>>>>>>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>>>>>>   at
>>>>>> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>>>>>>   at
>>>>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>>>>>>   at
>>>>>> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>>>>>>   at
>>>>>> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>>>>>>   at
>>>>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>>>>>>   at
>>>>>> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>>>>>>   at
>>>>>> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>>>>>>   at
>>>>>> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>>>>>>   at
>>>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>>>>>>   at
>>>>>> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>>>>>>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>>>>>>   at
>>>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>>>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>>>>>>   at
>>>>>> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>>>>>>   at
>>>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>>>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>>>>>>   ... 47 elided
>>>>>> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>>>>>>   at
>>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>>>>>   at
>>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>>>>>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>>>>>   ... 79 more
>>>>>>
>>>>>> Note, I usually verify by copying the spark-runtime jar to spark jars
>>>>>> dir (can't usually get --packages flag to work as indicated on
>>>>>> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
>>>>>> version is not released yet), so let me know if I am using the wrong jar?
>>>>>>
>>>>>> Thanks
>>>>>> Szehon
>>>>>>
>>>>>> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <
>>>>>> eduard@tabular.io> wrote:
>>>>>>
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>>    - validated checksum and signature
>>>>>>>    - checked license docs & ran RAT checks
>>>>>>>    - ran build and tests with JDK11
>>>>>>>
>>>>>>>
>>>>>>> Eduard
>>>>>>>
>>>>>>> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 (non-binding)
>>>>>>>>
>>>>>>>>
>>>>>>>>    - Verified the Spark runtime jar contents.
>>>>>>>>    - Checked license docs, ran RAT checks.
>>>>>>>>    - Validated checksum and signature.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ajantha
>>>>>>>>
>>>>>>>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <
>>>>>>>> prashant010696@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Everyone,
>>>>>>>>>
>>>>>>>>> Wanted to know your thoughts on whether we should also include the
>>>>>>>>> following bug fixes in this release as well:
>>>>>>>>>
>>>>>>>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>>>>>>>> *Reported instances :*
>>>>>>>>> a.
>>>>>>>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>>>>>>>> b. https://github.com/apache/iceberg/issues/5739
>>>>>>>>> c.
>>>>>>>>> https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>>>>>>>
>>>>>>>>> *PR's (Merged):*
>>>>>>>>> a. https://github.com/apache/iceberg/pull/5880
>>>>>>>>> b. https://github.com/apache/iceberg/pull/5679
>>>>>>>>>
>>>>>>>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>>>>>>>> Timestamp partitioned table when
>>>>>>>>> `spark.sql.datetime.java8API.enabled` is true.
>>>>>>>>> *Reported instances :*
>>>>>>>>> a. https://github.com/apache/iceberg/issues/5104
>>>>>>>>> b.
>>>>>>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>>>>>>>
>>>>>>>>> *PR* :
>>>>>>>>> a. https://github.com/apache/iceberg/pull/5860
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Prashant Singh
>>>>>>>>>
>>>>>>>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> +1 (binding)
>>>>>>>>>>
>>>>>>>>>>    - Checked license docs, ran RAT checks
>>>>>>>>>>    - Validated checksum and signature
>>>>>>>>>>    - Built and tested with Java 11
>>>>>>>>>>    - Built binary artifacts with Java 8
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>
>>>>>>>>>>> I propose that we release the following RC as the official
>>>>>>>>>>> Apache Iceberg 1.0.0 release.
>>>>>>>>>>>
>>>>>>>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>>>>>>>> *
>>>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>>>>>>>> *
>>>>>>>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>>>>
>>>>>>>>>>> The release tarball, signature, and checksums are here:
>>>>>>>>>>> *
>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>>>>>>>
>>>>>>>>>>> You can find the KEYS file here:
>>>>>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>>>>>>
>>>>>>>>>>> Convenience binary artifacts are staged on Nexus. The Maven
>>>>>>>>>>> repository URL is:
>>>>>>>>>>> *
>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>>>>>>>
>>>>>>>>>>> Please download, verify, and test.
>>>>>>>>>>>
>>>>>>>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>>>>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>>>>>>>> * Increase metrics limit to 100 columns
>>>>>>>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>>>>>>>> * Exclude Scala from Spark runtime Jars
>>>>>>>>>>>
>>>>>>>>>>> Please vote in the next 72 hours.
>>>>>>>>>>>
>>>>>>>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>>>>>>>> [ ] +0
>>>>>>>>>>> [ ] -1 Do not release this because...
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ryan Blue
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ryan Blue
>>>>>>>>>>
>>>>>>>>>
>
> --
> *Rushan Jiang*
>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Rushan Jiang <ru...@andrew.cmu.edu>.

+1 (non-binding)
- Verify Signature
- Verify Checksums
- Verify License Documentation
- Verify Build and Test in Java 8 and 11

On Mon, Oct 10, 2022 at 7:11 PM Daniel Weeks <dw...@apache.org> wrote:

> +1 (binding)
>
> Verified sigs/sums/license/build/test (java 8)
>
> Everything checks out,
> -Dan
>
> On Mon, Oct 10, 2022 at 3:25 PM Steven Wu <st...@gmail.com> wrote:
>
>> Never mind. Missed the information that this 1.0.0 is based on the latest
>> 0.14.1 release, which doesn't contain the PR 5318. I thought it was based
>> on the latest master branch.
>>
>> +1 (non-binding)
>> - Verify signature
>> - Verify checksum
>> - Tried SQL insert and query with Flink 1.15
>>
>> On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <st...@gmail.com> wrote:
>>
>>> Ryan,
>>>
>>> It seems that this PR (merged on July 28) for the Flink FLIP-27 source
>>> is not included. https://github.com/apache/iceberg/pull/5318
>>>
>>> The commit ID still contains the old file in the old location. The new
>>> location should be "flink/source/IcebergTableSource.java" with FLIP-27
>>> config support.
>>>
>>> https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
>>>
>>> Thanks,
>>> Steven
>>>
>>> On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com>
>>> wrote:
>>>
>>>> Whoops, sorry for the noise, I made a typo and was using the wrong
>>>> scala version of the iceberg-spark-runtime jar, this works.
>>>>
>>>> +1 (non-binding)
>>>> - Verify signature
>>>> - Verify checksum
>>>> - Verify license documentation
>>>> - Tried with Spark 3.3
>>>> - Ran unit tests
>>>>
>>>> Thanks
>>>> Szehon
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I get a NoClassDefFoundError from IcebergSparkExtensions when running
>>>>> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
>>>>> jar doesn't contain scala classes, unlike previous jars
>>>>> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>>>>>
>>>>> scala> spark.sql("show databases").show
>>>>> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>>>>>   at
>>>>> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>>>>>   at
>>>>> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>>>>>   at
>>>>> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>>>>>   at
>>>>> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>>>>>   at
>>>>> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>>>>>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>>>>>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>>>>>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>>>>>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>>>>>   at
>>>>> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>>>>>   at
>>>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>>>>>   at
>>>>> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>>>>>   at
>>>>> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>>>>>   at
>>>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>>>>>   at
>>>>> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>>>>>   at
>>>>> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>>>>>   at
>>>>> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>>>>>   at
>>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>>>>>   at
>>>>> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>>>>>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>>>>>   at
>>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>>>>>   at
>>>>> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>>>>>   at
>>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>>>>>   ... 47 elided
>>>>> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>>>>>   at
>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>>>>   at
>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>>>>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>>>>   ... 79 more
>>>>>
>>>>> Note, I usually verify by copying the spark-runtime jar to spark jars
>>>>> dir (can't usually get --packages flag to work as indicated on
>>>>> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
>>>>> version is not released yet), so let me know if I am using the wrong jar?
>>>>>
>>>>> Thanks
>>>>> Szehon
>>>>>
>>>>> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
>>>>> wrote:
>>>>>
>>>>>> +1 (non-binding)
>>>>>>
>>>>>>    - validated checksum and signature
>>>>>>    - checked license docs & ran RAT checks
>>>>>>    - ran build and tests with JDK11
>>>>>>
>>>>>>
>>>>>> Eduard
>>>>>>
>>>>>> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>>
>>>>>>>    - Verified the Spark runtime jar contents.
>>>>>>>    - Checked license docs, ran RAT checks.
>>>>>>>    - Validated checksum and signature.
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ajantha
>>>>>>>
>>>>>>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <
>>>>>>> prashant010696@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello Everyone,
>>>>>>>>
>>>>>>>> Wanted to know your thoughts on whether we should also include the
>>>>>>>> following bug fixes in this release as well:
>>>>>>>>
>>>>>>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>>>>>>> *Reported instances :*
>>>>>>>> a.
>>>>>>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>>>>>>> b. https://github.com/apache/iceberg/issues/5739
>>>>>>>> c.
>>>>>>>> https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>>>>>>
>>>>>>>> *PR's (Merged):*
>>>>>>>> a. https://github.com/apache/iceberg/pull/5880
>>>>>>>> b. https://github.com/apache/iceberg/pull/5679
>>>>>>>>
>>>>>>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>>>>>>> Timestamp partitioned table when
>>>>>>>> `spark.sql.datetime.java8API.enabled` is true.
>>>>>>>> *Reported instances :*
>>>>>>>> a. https://github.com/apache/iceberg/issues/5104
>>>>>>>> b.
>>>>>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>>>>>>
>>>>>>>> *PR* :
>>>>>>>> a. https://github.com/apache/iceberg/pull/5860
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Prashant Singh
>>>>>>>>
>>>>>>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>>
>>>>>>>>> +1 (binding)
>>>>>>>>>
>>>>>>>>>    - Checked license docs, ran RAT checks
>>>>>>>>>    - Validated checksum and signature
>>>>>>>>>    - Built and tested with Java 11
>>>>>>>>>    - Built binary artifacts with Java 8
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Everyone,
>>>>>>>>>>
>>>>>>>>>> I propose that we release the following RC as the official Apache
>>>>>>>>>> Iceberg 1.0.0 release.
>>>>>>>>>>
>>>>>>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>>>>>>> *
>>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>>>>>>> *
>>>>>>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>>>
>>>>>>>>>> The release tarball, signature, and checksums are here:
>>>>>>>>>> *
>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>>>>>>
>>>>>>>>>> You can find the KEYS file here:
>>>>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>>>>>
>>>>>>>>>> Convenience binary artifacts are staged on Nexus. The Maven
>>>>>>>>>> repository URL is:
>>>>>>>>>> *
>>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>>>>>>
>>>>>>>>>> Please download, verify, and test.
>>>>>>>>>>
>>>>>>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>>>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>>>>>>> * Increase metrics limit to 100 columns
>>>>>>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>>>>>>> * Exclude Scala from Spark runtime Jars
>>>>>>>>>>
>>>>>>>>>> Please vote in the next 72 hours.
>>>>>>>>>>
>>>>>>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>>>>>>> [ ] +0
>>>>>>>>>> [ ] -1 Do not release this because...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ryan Blue
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ryan Blue
>>>>>>>>>
>>>>>>>>

-- 
*Rushan Jiang*

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Daniel Weeks <dw...@apache.org>.

+1 (binding)

Verified sigs/sums/license/build/test (java 8)

Everything checks out,
-Dan

On Mon, Oct 10, 2022 at 3:25 PM Steven Wu <st...@gmail.com> wrote:

> Never mind. Missed the information that this 1.0.0 is based on the latest
> 0.14.1 release, which doesn't contain the PR 5318. I thought it was based
> on the latest master branch.
>
> +1 (non-binding)
> - Verify signature
> - Verify checksum
> - Tried SQL insert and query with Flink 1.15
>
> On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <st...@gmail.com> wrote:
>
>> Ryan,
>>
>> It seems that this PR (merged on July 28) for the Flink FLIP-27 source is
>> not included. https://github.com/apache/iceberg/pull/5318
>>
>> The commit ID still contains the old file in the old location. The new
>> location should be "flink/source/IcebergTableSource.java" with FLIP-27
>> config support.
>>
>> https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
>>
>> Thanks,
>> Steven
>>
>> On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com>
>> wrote:
>>
>>> Whoops, sorry for the noise, I made a typo and was using the wrong scala
>>> version of the iceberg-spark-runtime jar, this works.
>>>
>>> +1 (non-binding)
>>> - Verify signature
>>> - Verify checksum
>>> - Verify license documentation
>>> - Tried with Spark 3.3
>>> - Ran unit tests
>>>
>>> Thanks
>>> Szehon
>>>
>>>
>>>
>>>
>>> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I get a NoClassDefFoundError from IcebergSparkExtensions when running
>>>> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
>>>> jar doesn't contain scala classes, unlike previous jars
>>>> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>>>>
>>>> scala> spark.sql("show databases").show
>>>> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>>>>   at
>>>> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>>>>   at
>>>> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>>>>   at
>>>> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>>>>   at
>>>> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>>>>   at
>>>> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>>>>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>>>>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>>>>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>>>>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>>>>   at
>>>> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>>>>   at
>>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>>>>   at
>>>> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>>>>   at
>>>> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>>>>   at
>>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>>>>   at
>>>> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>>>>   at
>>>> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>>>>   at
>>>> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>>>>   at
>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>>>>   at
>>>> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>>>>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>>>>   at
>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>>>>   at
>>>> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>>>>   at
>>>> org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>>>>   ... 47 elided
>>>> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>>>>   at
>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>>>   at
>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>>>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>>>   ... 79 more
>>>>
>>>> Note, I usually verify by copying the spark-runtime jar to spark jars
>>>> dir (can't usually get --packages flag to work as indicated on
>>>> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
>>>> version is not released yet), so let me know if I am using the wrong jar?
>>>>
>>>> Thanks
>>>> Szehon
>>>>
>>>> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
>>>> wrote:
>>>>
>>>>> +1 (non-binding)
>>>>>
>>>>>    - validated checksum and signature
>>>>>    - checked license docs & ran RAT checks
>>>>>    - ran build and tests with JDK11
>>>>>
>>>>>
>>>>> Eduard
>>>>>
>>>>> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1 (non-binding)
>>>>>>
>>>>>>
>>>>>>    - Verified the Spark runtime jar contents.
>>>>>>    - Checked license docs, ran RAT checks.
>>>>>>    - Validated checksum and signature.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Ajantha
>>>>>>
>>>>>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <
>>>>>> prashant010696@gmail.com> wrote:
>>>>>>
>>>>>>> Hello Everyone,
>>>>>>>
>>>>>>> Wanted to know your thoughts on whether we should also include the
>>>>>>> following bug fixes in this release as well:
>>>>>>>
>>>>>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>>>>>> *Reported instances :*
>>>>>>> a.
>>>>>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>>>>>> b. https://github.com/apache/iceberg/issues/5739
>>>>>>> c.
>>>>>>> https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>>>>>
>>>>>>> *PR's (Merged):*
>>>>>>> a. https://github.com/apache/iceberg/pull/5880
>>>>>>> b. https://github.com/apache/iceberg/pull/5679
>>>>>>>
>>>>>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>>>>>> Timestamp partitioned table when
>>>>>>> `spark.sql.datetime.java8API.enabled` is true.
>>>>>>> *Reported instances :*
>>>>>>> a. https://github.com/apache/iceberg/issues/5104
>>>>>>> b.
>>>>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>>>>>
>>>>>>> *PR* :
>>>>>>> a. https://github.com/apache/iceberg/pull/5860
>>>>>>>
>>>>>>> Regards,
>>>>>>> Prashant Singh
>>>>>>>
>>>>>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>
>>>>>>>> +1 (binding)
>>>>>>>>
>>>>>>>>    - Checked license docs, ran RAT checks
>>>>>>>>    - Validated checksum and signature
>>>>>>>>    - Built and tested with Java 11
>>>>>>>>    - Built binary artifacts with Java 8
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>> I propose that we release the following RC as the official Apache
>>>>>>>>> Iceberg 1.0.0 release.
>>>>>>>>>
>>>>>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>>>>>> *
>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>>>>>> *
>>>>>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>>
>>>>>>>>> The release tarball, signature, and checksums are here:
>>>>>>>>> *
>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>>>>>
>>>>>>>>> You can find the KEYS file here:
>>>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>>>>
>>>>>>>>> Convenience binary artifacts are staged on Nexus. The Maven
>>>>>>>>> repository URL is:
>>>>>>>>> *
>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>>>>>
>>>>>>>>> Please download, verify, and test.
>>>>>>>>>
>>>>>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>>>>>> * Increase metrics limit to 100 columns
>>>>>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>>>>>> * Exclude Scala from Spark runtime Jars
>>>>>>>>>
>>>>>>>>> Please vote in the next 72 hours.
>>>>>>>>>
>>>>>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>>>>>> [ ] +0
>>>>>>>>> [ ] -1 Do not release this because...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ryan Blue
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ryan Blue
>>>>>>>>
>>>>>>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Steven Wu <st...@gmail.com>.

Never mind. Missed the information that this 1.0.0 is based on the latest
0.14.1 release, which doesn't contain the PR 5318. I thought it was based
on the latest master branch.

+1 (non-binding)
- Verify signature
- Verify checksum
- Tried SQL insert and query with Flink 1.15

On Mon, Oct 10, 2022 at 3:20 PM Steven Wu <st...@gmail.com> wrote:

> Ryan,
>
> It seems that this PR (merged on July 28) for the Flink FLIP-27 source is
> not included. https://github.com/apache/iceberg/pull/5318
>
> The commit ID still contains the old file in the old location. The new
> location should be "flink/source/IcebergTableSource.java" with FLIP-27
> config support.
>
> https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
>
> Thanks,
> Steven
>
> On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com>
> wrote:
>
>> Whoops, sorry for the noise, I made a typo and was using the wrong scala
>> version of the iceberg-spark-runtime jar, this works.
>>
>> +1 (non-binding)
>> - Verify signature
>> - Verify checksum
>> - Verify license documentation
>> - Tried with Spark 3.3
>> - Ran unit tests
>>
>> Thanks
>> Szehon
>>
>>
>>
>>
>> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I get a NoClassDefFoundError from IcebergSparkExtensions when running
>>> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
>>> jar doesn't contain scala classes, unlike previous jars
>>> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>>>
>>> scala> spark.sql("show databases").show
>>> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>>>   at
>>> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>>>   at
>>> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>>>   at
>>> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>>>   at
>>> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>>>   at
>>> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>>>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>>>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>>>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>>>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>>>   at
>>> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>>>   at
>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>>>   at
>>> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>>>   at
>>> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>>>   at
>>> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>>>   at
>>> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>>>   at
>>> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>>>   at
>>> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>>>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>>>   at
>>> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>>>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>>>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>>>   at
>>> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>>>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>>>   ... 47 elided
>>> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>>>   at
>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>>   at
>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>>   ... 79 more
>>>
>>> Note, I usually verify by copying the spark-runtime jar to spark jars
>>> dir (can't usually get --packages flag to work as indicated on
>>> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
>>> version is not released yet), so let me know if I am using the wrong jar?
>>>
>>> Thanks
>>> Szehon
>>>
>>> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
>>> wrote:
>>>
>>>> +1 (non-binding)
>>>>
>>>>    - validated checksum and signature
>>>>    - checked license docs & ran RAT checks
>>>>    - ran build and tests with JDK11
>>>>
>>>>
>>>> Eduard
>>>>
>>>> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1 (non-binding)
>>>>>
>>>>>
>>>>>    - Verified the Spark runtime jar contents.
>>>>>    - Checked license docs, ran RAT checks.
>>>>>    - Validated checksum and signature.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Ajantha
>>>>>
>>>>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <
>>>>> prashant010696@gmail.com> wrote:
>>>>>
>>>>>> Hello Everyone,
>>>>>>
>>>>>> Wanted to know your thoughts on whether we should also include the
>>>>>> following bug fixes in this release as well:
>>>>>>
>>>>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>>>>> *Reported instances :*
>>>>>> a.
>>>>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>>>>> b. https://github.com/apache/iceberg/issues/5739
>>>>>> c.
>>>>>> https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>>>>
>>>>>> *PR's (Merged):*
>>>>>> a. https://github.com/apache/iceberg/pull/5880
>>>>>> b. https://github.com/apache/iceberg/pull/5679
>>>>>>
>>>>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>>>>> Timestamp partitioned table when
>>>>>> `spark.sql.datetime.java8API.enabled` is true.
>>>>>> *Reported instances :*
>>>>>> a. https://github.com/apache/iceberg/issues/5104
>>>>>> b.
>>>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>>>>
>>>>>> *PR* :
>>>>>> a. https://github.com/apache/iceberg/pull/5860
>>>>>>
>>>>>> Regards,
>>>>>> Prashant Singh
>>>>>>
>>>>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>>>>
>>>>>>> +1 (binding)
>>>>>>>
>>>>>>>    - Checked license docs, ran RAT checks
>>>>>>>    - Validated checksum and signature
>>>>>>>    - Built and tested with Java 11
>>>>>>>    - Built binary artifacts with Java 8
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>>>>
>>>>>>>> Hi Everyone,
>>>>>>>>
>>>>>>>> I propose that we release the following RC as the official Apache
>>>>>>>> Iceberg 1.0.0 release.
>>>>>>>>
>>>>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>>>>> *
>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>>>>> *
>>>>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>>
>>>>>>>> The release tarball, signature, and checksums are here:
>>>>>>>> *
>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>>>>
>>>>>>>> You can find the KEYS file here:
>>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>>>
>>>>>>>> Convenience binary artifacts are staged on Nexus. The Maven
>>>>>>>> repository URL is:
>>>>>>>> *
>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>>>>
>>>>>>>> Please download, verify, and test.
>>>>>>>>
>>>>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>>>>> * Increase metrics limit to 100 columns
>>>>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>>>>> * Exclude Scala from Spark runtime Jars
>>>>>>>>
>>>>>>>> Please vote in the next 72 hours.
>>>>>>>>
>>>>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>>>>> [ ] +0
>>>>>>>> [ ] -1 Do not release this because...
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ryan Blue
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Ryan Blue
>>>>>>>
>>>>>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Steven Wu <st...@gmail.com>.

Ryan,

It seems that this PR (merged on July 28) for the Flink FLIP-27 source is
not included. https://github.com/apache/iceberg/pull/5318

The commit ID still contains the old file in the old location. The new
location should be "flink/source/IcebergTableSource.java" with FLIP-27
config support.
https://github.com/apache/iceberg/blob/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01/flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java

Thanks,
Steven

On Mon, Oct 10, 2022 at 11:37 AM Szehon Ho <sz...@gmail.com> wrote:

> Whoops, sorry for the noise, I made a typo and was using the wrong scala
> version of the iceberg-spark-runtime jar, this works.
>
> +1 (non-binding)
> - Verify signature
> - Verify checksum
> - Verify license documentation
> - Tried with Spark 3.3
> - Ran unit tests
>
> Thanks
> Szehon
>
>
>
>
> On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I get a NoClassDefFoundError from IcebergSparkExtensions when running
>> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
>> jar doesn't contain scala classes, unlike previous jars
>> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>>
>> scala> spark.sql("show databases").show
>> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>>   at
>> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>>   at
>> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>>   at
>> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>>   at
>> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>>   at
>> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>>   at
>> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>>   at
>> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>>   at
>> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>>   at
>> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>>   at
>> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>>   at
>> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>>   at
>> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>>   at
>> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>>   at
>> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>>   at
>> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>>   at
>> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>>   ... 47 elided
>> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>>   at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>   at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>   ... 79 more
>>
>> Note, I usually verify by copying the spark-runtime jar to spark jars dir
>> (can't usually get --packages flag to work as indicated on
>> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
>> version is not released yet), so let me know if I am using the wrong jar?
>>
>> Thanks
>> Szehon
>>
>> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
>> wrote:
>>
>>> +1 (non-binding)
>>>
>>>    - validated checksum and signature
>>>    - checked license docs & ran RAT checks
>>>    - ran build and tests with JDK11
>>>
>>>
>>> Eduard
>>>
>>> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
>>> wrote:
>>>
>>>> +1 (non-binding)
>>>>
>>>>
>>>>    - Verified the Spark runtime jar contents.
>>>>    - Checked license docs, ran RAT checks.
>>>>    - Validated checksum and signature.
>>>>
>>>>
>>>> Thanks,
>>>> Ajantha
>>>>
>>>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <
>>>> prashant010696@gmail.com> wrote:
>>>>
>>>>> Hello Everyone,
>>>>>
>>>>> Wanted to know your thoughts on whether we should also include the
>>>>> following bug fixes in this release as well:
>>>>>
>>>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>>>> *Reported instances :*
>>>>> a.
>>>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>>>> b. https://github.com/apache/iceberg/issues/5739
>>>>> c.
>>>>> https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>>>
>>>>> *PR's (Merged):*
>>>>> a. https://github.com/apache/iceberg/pull/5880
>>>>> b. https://github.com/apache/iceberg/pull/5679
>>>>>
>>>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>>>> Timestamp partitioned table when
>>>>> `spark.sql.datetime.java8API.enabled` is true.
>>>>> *Reported instances :*
>>>>> a. https://github.com/apache/iceberg/issues/5104
>>>>> b.
>>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>>>
>>>>> *PR* :
>>>>> a. https://github.com/apache/iceberg/pull/5860
>>>>>
>>>>> Regards,
>>>>> Prashant Singh
>>>>>
>>>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>>>
>>>>>> +1 (binding)
>>>>>>
>>>>>>    - Checked license docs, ran RAT checks
>>>>>>    - Validated checksum and signature
>>>>>>    - Built and tested with Java 11
>>>>>>    - Built binary artifacts with Java 8
>>>>>>
>>>>>>
>>>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>>>
>>>>>>> Hi Everyone,
>>>>>>>
>>>>>>> I propose that we release the following RC as the official Apache
>>>>>>> Iceberg 1.0.0 release.
>>>>>>>
>>>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>>>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>>>> *
>>>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>>
>>>>>>> The release tarball, signature, and checksums are here:
>>>>>>> *
>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>>>
>>>>>>> You can find the KEYS file here:
>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>>
>>>>>>> Convenience binary artifacts are staged on Nexus. The Maven
>>>>>>> repository URL is:
>>>>>>> *
>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>>>
>>>>>>> Please download, verify, and test.
>>>>>>>
>>>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>>>> * Increase metrics limit to 100 columns
>>>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>>>> * Exclude Scala from Spark runtime Jars
>>>>>>>
>>>>>>> Please vote in the next 72 hours.
>>>>>>>
>>>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>>>> [ ] +0
>>>>>>> [ ] -1 Do not release this because...
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Ryan Blue
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ryan Blue
>>>>>>
>>>>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Szehon Ho <sz...@gmail.com>.

Whoops, sorry for the noise, I made a typo and was using the wrong scala
version of the iceberg-spark-runtime jar, this works.

+1 (non-binding)
- Verify signature
- Verify checksum
- Verify license documentation
- Tried with Spark 3.3
- Ran unit tests

Thanks
Szehon




On Mon, Oct 10, 2022 at 11:26 AM Szehon Ho <sz...@gmail.com> wrote:

> Hi,
>
> I get a NoClassDefFoundError from IcebergSparkExtensions when running
> Spark 3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this
> jar doesn't contain scala classes, unlike previous jars
> iceberg-spark-runtime-3.3_2.12-0.14.1.jar.
>
> scala> spark.sql("show databases").show
> java.lang.NoClassDefFoundError: scala/collection/SeqOps
>   at
> org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
>   at
> org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
>   at
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
>   at
> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
>   at
> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
>   at scala.collection.TraversableLike.map(TraversableLike.scala:286)
>   at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
>   at scala.collection.AbstractTraversable.map(Traversable.scala:108)
>   at
> org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
>   at
> org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
>   at
> org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
>   at
> org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
>   at
> org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
>   at
> org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
>   at
> org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
>   at
> org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
>   at
> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
>   at
> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
>   at
> org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
>   at
> org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at
> org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
>   at
> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
>   at
> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
>   at
> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
>   at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
>   at
> org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
>   at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
>   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
>   ... 47 elided
> Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
>   at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>   at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>   ... 79 more
>
> Note, I usually verify by copying the spark-runtime jar to spark jars dir
> (can't usually get --packages flag to work as indicated on
> https://iceberg.apache.org/how-to-release/#verifying-with-spark, as
> version is not released yet), so let me know if I am using the wrong jar?
>
> Thanks
> Szehon
>
> On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
> wrote:
>
>> +1 (non-binding)
>>
>>    - validated checksum and signature
>>    - checked license docs & ran RAT checks
>>    - ran build and tests with JDK11
>>
>>
>> Eduard
>>
>> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
>> wrote:
>>
>>> +1 (non-binding)
>>>
>>>
>>>    - Verified the Spark runtime jar contents.
>>>    - Checked license docs, ran RAT checks.
>>>    - Validated checksum and signature.
>>>
>>>
>>> Thanks,
>>> Ajantha
>>>
>>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <
>>> prashant010696@gmail.com> wrote:
>>>
>>>> Hello Everyone,
>>>>
>>>> Wanted to know your thoughts on whether we should also include the
>>>> following bug fixes in this release as well:
>>>>
>>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>>> *Reported instances :*
>>>> a.
>>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>>> b. https://github.com/apache/iceberg/issues/5739
>>>> c.
>>>> https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>>
>>>> *PR's (Merged):*
>>>> a. https://github.com/apache/iceberg/pull/5880
>>>> b. https://github.com/apache/iceberg/pull/5679
>>>>
>>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>>> Timestamp partitioned table when
>>>> `spark.sql.datetime.java8API.enabled` is true.
>>>> *Reported instances :*
>>>> a. https://github.com/apache/iceberg/issues/5104
>>>> b.
>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>>
>>>> *PR* :
>>>> a. https://github.com/apache/iceberg/pull/5860
>>>>
>>>> Regards,
>>>> Prashant Singh
>>>>
>>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>>
>>>>> +1 (binding)
>>>>>
>>>>>    - Checked license docs, ran RAT checks
>>>>>    - Validated checksum and signature
>>>>>    - Built and tested with Java 11
>>>>>    - Built binary artifacts with Java 8
>>>>>
>>>>>
>>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> I propose that we release the following RC as the official Apache
>>>>>> Iceberg 1.0.0 release.
>>>>>>
>>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>>> *
>>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>>
>>>>>> The release tarball, signature, and checksums are here:
>>>>>> *
>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>>
>>>>>> You can find the KEYS file here:
>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>
>>>>>> Convenience binary artifacts are staged on Nexus. The Maven
>>>>>> repository URL is:
>>>>>> *
>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>>
>>>>>> Please download, verify, and test.
>>>>>>
>>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>>> * Increase metrics limit to 100 columns
>>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>>> * Exclude Scala from Spark runtime Jars
>>>>>>
>>>>>> Please vote in the next 72 hours.
>>>>>>
>>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>>> [ ] +0
>>>>>> [ ] -1 Do not release this because...
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ryan Blue
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ryan Blue
>>>>>
>>>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Szehon Ho <sz...@gmail.com>.

Hi,

I get a NoClassDefFoundError from IcebergSparkExtensions when running Spark
3.3, with iceberg-spark-runtime-3.3_2.12-1.0.0.jar.  I noticed this jar
doesn't contain scala classes, unlike previous jars
iceberg-spark-runtime-3.3_2.12-0.14.1.jar.

scala> spark.sql("show databases").show
java.lang.NoClassDefFoundError: scala/collection/SeqOps
  at
org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$2(IcebergSparkSessionExtensions.scala:50)
  at
org.apache.spark.sql.SparkSessionExtensions.$anonfun$buildResolutionRules$1(SparkSessionExtensions.scala:152)
  at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
  at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
  at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
  at scala.collection.TraversableLike.map(TraversableLike.scala:286)
  at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
  at scala.collection.AbstractTraversable.map(Traversable.scala:108)
  at
org.apache.spark.sql.SparkSessionExtensions.buildResolutionRules(SparkSessionExtensions.scala:152)
  at
org.apache.spark.sql.internal.BaseSessionStateBuilder.customResolutionRules(BaseSessionStateBuilder.scala:216)
  at
org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:94)
  at
org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:85)
  at
org.apache.spark.sql.internal.BaseSessionStateBuilder.$anonfun$build$2(BaseSessionStateBuilder.scala:360)
  at
org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:87)
  at
org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:87)
  at
org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:76)
  at
org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
  at
org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:185)
  at
org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
  at
org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at
org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:184)
  at
org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
  at
org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
  at
org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
  at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
  at
org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:622)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:617)
  ... 47 elided
Caused by: java.lang.ClassNotFoundException: scala.collection.SeqOps
  at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
  at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
  ... 79 more

Note, I usually verify by copying the spark-runtime jar to spark jars dir
(can't usually get --packages flag to work as indicated on
https://iceberg.apache.org/how-to-release/#verifying-with-spark, as version
is not released yet), so let me know if I am using the wrong jar?

Thanks
Szehon

On Mon, Oct 10, 2022 at 9:22 AM Eduard Tudenhoefner <ed...@tabular.io>
wrote:

> +1 (non-binding)
>
>    - validated checksum and signature
>    - checked license docs & ran RAT checks
>    - ran build and tests with JDK11
>
>
> Eduard
>
> On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com>
> wrote:
>
>> +1 (non-binding)
>>
>>
>>    - Verified the Spark runtime jar contents.
>>    - Checked license docs, ran RAT checks.
>>    - Validated checksum and signature.
>>
>>
>> Thanks,
>> Ajantha
>>
>> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <pr...@gmail.com>
>> wrote:
>>
>>> Hello Everyone,
>>>
>>> Wanted to know your thoughts on whether we should also include the
>>> following bug fixes in this release as well:
>>>
>>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>>> *Reported instances :*
>>> a.
>>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>>> b. https://github.com/apache/iceberg/issues/5739
>>> c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>>
>>> *PR's (Merged):*
>>> a. https://github.com/apache/iceberg/pull/5880
>>> b. https://github.com/apache/iceberg/pull/5679
>>>
>>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>>> Timestamp partitioned table when
>>> `spark.sql.datetime.java8API.enabled` is true.
>>> *Reported instances :*
>>> a. https://github.com/apache/iceberg/issues/5104
>>> b.
>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>>
>>> *PR* :
>>> a. https://github.com/apache/iceberg/pull/5860
>>>
>>> Regards,
>>> Prashant Singh
>>>
>>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>>    - Checked license docs, ran RAT checks
>>>>    - Validated checksum and signature
>>>>    - Built and tested with Java 11
>>>>    - Built binary artifacts with Java 8
>>>>
>>>>
>>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>>
>>>>> Hi Everyone,
>>>>>
>>>>> I propose that we release the following RC as the official Apache
>>>>> Iceberg 1.0.0 release.
>>>>>
>>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>>> *
>>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>>
>>>>> The release tarball, signature, and checksums are here:
>>>>> *
>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>>
>>>>> You can find the KEYS file here:
>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>
>>>>> Convenience binary artifacts are staged on Nexus. The Maven repository
>>>>> URL is:
>>>>> *
>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>>
>>>>> Please download, verify, and test.
>>>>>
>>>>> This release is based on the latest 0.14.1 release. It includes
>>>>> changes to remove deprecated APIs and the following additional bug fixes:
>>>>> * Increase metrics limit to 100 columns
>>>>> * Bump Spark patch versions for CVE-2022-33891
>>>>> * Exclude Scala from Spark runtime Jars
>>>>>
>>>>> Please vote in the next 72 hours.
>>>>>
>>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>>> [ ] +0
>>>>> [ ] -1 Do not release this because...
>>>>>
>>>>>
>>>>> --
>>>>> Ryan Blue
>>>>>
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Eduard Tudenhoefner <ed...@tabular.io>.

+1 (non-binding)

   - validated checksum and signature
   - checked license docs & ran RAT checks
   - ran build and tests with JDK11


Eduard

On Mon, Oct 10, 2022 at 8:01 AM Ajantha Bhat <aj...@gmail.com> wrote:

> +1 (non-binding)
>
>
>    - Verified the Spark runtime jar contents.
>    - Checked license docs, ran RAT checks.
>    - Validated checksum and signature.
>
>
> Thanks,
> Ajantha
>
> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <pr...@gmail.com>
> wrote:
>
>> Hello Everyone,
>>
>> Wanted to know your thoughts on whether we should also include the
>> following bug fixes in this release as well:
>>
>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>> *Reported instances :*
>> a.
>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>> b. https://github.com/apache/iceberg/issues/5739
>> c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>
>> *PR's (Merged):*
>> a. https://github.com/apache/iceberg/pull/5880
>> b. https://github.com/apache/iceberg/pull/5679
>>
>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>> Timestamp partitioned table when
>> `spark.sql.datetime.java8API.enabled` is true.
>> *Reported instances :*
>> a. https://github.com/apache/iceberg/issues/5104
>> b.
>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>
>> *PR* :
>> a. https://github.com/apache/iceberg/pull/5860
>>
>> Regards,
>> Prashant Singh
>>
>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>
>>> +1 (binding)
>>>
>>>    - Checked license docs, ran RAT checks
>>>    - Validated checksum and signature
>>>    - Built and tested with Java 11
>>>    - Built binary artifacts with Java 8
>>>
>>>
>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> Hi Everyone,
>>>>
>>>> I propose that we release the following RC as the official Apache
>>>> Iceberg 1.0.0 release.
>>>>
>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>> *
>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>
>>>> The release tarball, signature, and checksums are here:
>>>> *
>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>
>>>> You can find the KEYS file here:
>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>
>>>> Convenience binary artifacts are staged on Nexus. The Maven repository
>>>> URL is:
>>>> *
>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>
>>>> Please download, verify, and test.
>>>>
>>>> This release is based on the latest 0.14.1 release. It includes changes
>>>> to remove deprecated APIs and the following additional bug fixes:
>>>> * Increase metrics limit to 100 columns
>>>> * Bump Spark patch versions for CVE-2022-33891
>>>> * Exclude Scala from Spark runtime Jars
>>>>
>>>> Please vote in the next 72 hours.
>>>>
>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>> [ ] +0
>>>> [ ] -1 Do not release this because...
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by "Driesprong, Fokko" <fo...@driesprong.frl>.

+1 (non-binding)

- Ran the license and checksum
- Ran test suite of Trino against 1.0.0 and didn't see any issues

Cheers, Fokko


Op ma 10 okt. 2022 om 08:00 schreef Ajantha Bhat <aj...@gmail.com>:

> +1 (non-binding)
>
>
>    - Verified the Spark runtime jar contents.
>    - Checked license docs, ran RAT checks.
>    - Validated checksum and signature.
>
>
> Thanks,
> Ajantha
>
> On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <pr...@gmail.com>
> wrote:
>
>> Hello Everyone,
>>
>> Wanted to know your thoughts on whether we should also include the
>> following bug fixes in this release as well:
>>
>> 1. MERGE INTO nullability fix, leads to query failure otherwise:
>> *Reported instances :*
>> a.
>> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
>> b. https://github.com/apache/iceberg/issues/5739
>> c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>>
>> *PR's (Merged):*
>> a. https://github.com/apache/iceberg/pull/5880
>> b. https://github.com/apache/iceberg/pull/5679
>>
>> 2.  QueryFailure when running RewriteManifestProcedure on Date /
>> Timestamp partitioned table when
>> `spark.sql.datetime.java8API.enabled` is true.
>> *Reported instances :*
>> a. https://github.com/apache/iceberg/issues/5104
>> b.
>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>>
>> *PR* :
>> a. https://github.com/apache/iceberg/pull/5860
>>
>> Regards,
>> Prashant Singh
>>
>> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>>
>>> +1 (binding)
>>>
>>>    - Checked license docs, ran RAT checks
>>>    - Validated checksum and signature
>>>    - Built and tested with Java 11
>>>    - Built binary artifacts with Java 8
>>>
>>>
>>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> Hi Everyone,
>>>>
>>>> I propose that we release the following RC as the official Apache
>>>> Iceberg 1.0.0 release.
>>>>
>>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>>> *
>>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>>
>>>> The release tarball, signature, and checksums are here:
>>>> *
>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>>
>>>> You can find the KEYS file here:
>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>
>>>> Convenience binary artifacts are staged on Nexus. The Maven repository
>>>> URL is:
>>>> *
>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>>
>>>> Please download, verify, and test.
>>>>
>>>> This release is based on the latest 0.14.1 release. It includes changes
>>>> to remove deprecated APIs and the following additional bug fixes:
>>>> * Increase metrics limit to 100 columns
>>>> * Bump Spark patch versions for CVE-2022-33891
>>>> * Exclude Scala from Spark runtime Jars
>>>>
>>>> Please vote in the next 72 hours.
>>>>
>>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>>> [ ] +0
>>>> [ ] -1 Do not release this because...
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Ajantha Bhat <aj...@gmail.com>.

+1 (non-binding)


   - Verified the Spark runtime jar contents.
   - Checked license docs, ran RAT checks.
   - Validated checksum and signature.


Thanks,
Ajantha

On Mon, Oct 10, 2022 at 10:45 AM Prashant Singh <pr...@gmail.com>
wrote:

> Hello Everyone,
>
> Wanted to know your thoughts on whether we should also include the
> following bug fixes in this release as well:
>
> 1. MERGE INTO nullability fix, leads to query failure otherwise:
> *Reported instances :*
> a.
> https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
> b. https://github.com/apache/iceberg/issues/5739
> c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298
>
> *PR's (Merged):*
> a. https://github.com/apache/iceberg/pull/5880
> b. https://github.com/apache/iceberg/pull/5679
>
> 2.  QueryFailure when running RewriteManifestProcedure on Date / Timestamp
> partitioned table when
> `spark.sql.datetime.java8API.enabled` is true.
> *Reported instances :*
> a. https://github.com/apache/iceberg/issues/5104
> b. https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469
>
> *PR* :
> a. https://github.com/apache/iceberg/pull/5860
>
> Regards,
> Prashant Singh
>
> On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:
>
>> +1 (binding)
>>
>>    - Checked license docs, ran RAT checks
>>    - Validated checksum and signature
>>    - Built and tested with Java 11
>>    - Built binary artifacts with Java 8
>>
>>
>> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>>
>>> Hi Everyone,
>>>
>>> I propose that we release the following RC as the official Apache
>>> Iceberg 1.0.0 release.
>>>
>>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>>> *
>>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>>
>>> The release tarball, signature, and checksums are here:
>>> *
>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>>
>>> You can find the KEYS file here:
>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>
>>> Convenience binary artifacts are staged on Nexus. The Maven repository
>>> URL is:
>>> *
>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>>
>>> Please download, verify, and test.
>>>
>>> This release is based on the latest 0.14.1 release. It includes changes
>>> to remove deprecated APIs and the following additional bug fixes:
>>> * Increase metrics limit to 100 columns
>>> * Bump Spark patch versions for CVE-2022-33891
>>> * Exclude Scala from Spark runtime Jars
>>>
>>> Please vote in the next 72 hours.
>>>
>>> [ ] +1 Release this as Apache Iceberg 1.0.0
>>> [ ] +0
>>> [ ] -1 Do not release this because...
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>
>>
>> --
>> Ryan Blue
>>
>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Prashant Singh <pr...@gmail.com>.

Hello Everyone,

Wanted to know your thoughts on whether we should also include the
following bug fixes in this release as well:

1. MERGE INTO nullability fix, leads to query failure otherwise:
*Reported instances :*
a.
https://stackoverflow.com/questions/73424454/spark-iceberg-merge-into-issue-caused-by-org-apache-spark-sql-analysisexcep
b. https://github.com/apache/iceberg/issues/5739
c. https://github.com/apache/iceberg/issues/5424#issuecomment-1220688298

*PR's (Merged):*
a. https://github.com/apache/iceberg/pull/5880
b. https://github.com/apache/iceberg/pull/5679

2.  QueryFailure when running RewriteManifestProcedure on Date / Timestamp
partitioned table when
`spark.sql.datetime.java8API.enabled` is true.
*Reported instances :*
a. https://github.com/apache/iceberg/issues/5104
b. https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1663982635731469

*PR* :
a. https://github.com/apache/iceberg/pull/5860

Regards,
Prashant Singh

On Mon, Oct 10, 2022 at 4:15 AM Ryan Blue <bl...@apache.org> wrote:

> +1 (binding)
>
>    - Checked license docs, ran RAT checks
>    - Validated checksum and signature
>    - Built and tested with Java 11
>    - Built binary artifacts with Java 8
>
>
> On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:
>
>> Hi Everyone,
>>
>> I propose that we release the following RC as the official Apache Iceberg
>> 1.0.0 release.
>>
>> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
>> *
>> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>>
>> The release tarball, signature, and checksums are here:
>> * https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>>
>> You can find the KEYS file here:
>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>
>> Convenience binary artifacts are staged on Nexus. The Maven repository
>> URL is:
>> *
>> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>>
>> Please download, verify, and test.
>>
>> This release is based on the latest 0.14.1 release. It includes changes
>> to remove deprecated APIs and the following additional bug fixes:
>> * Increase metrics limit to 100 columns
>> * Bump Spark patch versions for CVE-2022-33891
>> * Exclude Scala from Spark runtime Jars
>>
>> Please vote in the next 72 hours.
>>
>> [ ] +1 Release this as Apache Iceberg 1.0.0
>> [ ] +0
>> [ ] -1 Do not release this because...
>>
>>
>> --
>> Ryan Blue
>>
>
>
> --
> Ryan Blue
>

Re: [VOTE] Release Apache Iceberg 1.0.0 RC0

Posted by Ryan Blue <bl...@apache.org>.

+1 (binding)

   - Checked license docs, ran RAT checks
   - Validated checksum and signature
   - Built and tested with Java 11
   - Built binary artifacts with Java 8


On Sun, Oct 9, 2022 at 3:42 PM Ryan Blue <bl...@apache.org> wrote:

> Hi Everyone,
>
> I propose that we release the following RC as the official Apache Iceberg
> 1.0.0 release.
>
> The commit ID is e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
> * This corresponds to the tag: apache-iceberg-1.0.0-rc0
> * https://github.com/apache/iceberg/commits/apache-iceberg-1.0.0-rc0
> *
> https://github.com/apache/iceberg/tree/e2bb9ad7e792efca419fa7c4a1afde7c4c44fa01
>
> The release tarball, signature, and checksums are here:
> * https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.0.0-rc0
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>
> Convenience binary artifacts are staged on Nexus. The Maven repository URL
> is:
> *
> https://repository.apache.org/content/repositories/orgapacheiceberg-1106/
>
> Please download, verify, and test.
>
> This release is based on the latest 0.14.1 release. It includes changes to
> remove deprecated APIs and the following additional bug fixes:
> * Increase metrics limit to 100 columns
> * Bump Spark patch versions for CVE-2022-33891
> * Exclude Scala from Spark runtime Jars
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Iceberg 1.0.0
> [ ] +0
> [ ] -1 Do not release this because...
>
>
> --
> Ryan Blue
>


-- 
Ryan Blue