You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Ryan Blue <bl...@apache.org> on 2020/04/28 20:39:32 UTC
[VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Hi everyone,
I propose the following RC to be released as official Apache Iceberg
0.8.0-incubating release.
The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
* This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
*
https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
* https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
The release tarball, signature, and checksums are here:
*
https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
You can find the KEYS file here:
* https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
Convenience binary artifacts are staged in Nexus. The Maven repository URL
is:
* https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
This release contains many bug fixes and several new features:
* Actions to remove orphaned files and to optimize metadata for query
performance
* Support for ORC data files
* Snapshot cherry-picking
* Incremental scan planning based on table history
* In and notIn expressions
* An InputFormat for writing MR jobs
Please download, verify, and test.
Please vote in the next 72 hours.
[ ] +1 Release this as Apache Iceberg 0.8.0-incubating
[ ] +0
[ ] -1 Do not release this because...
--
Ryan Blue
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Ryan Murray <ry...@dremio.com>.
+1 (non-binding)
Validated checksums and signature, ran ./gradlew build, and ran license
check.
Loaded into a downstream project and it worked as expected.
Best,
Ryan
On Wed, Apr 29, 2020 at 5:26 AM tison <wa...@gmail.com> wrote:
> Hi Ryan,
>
> Thanks for starting the voting process.
>
> +1 (non-binding) I verified:
>
> √ RAT checks passed
> √ signature is correct
> √ checksum is correct
> √ build from source
> √ run tests locally
>
> I'd like to ask 2 questions:
>
> 1. The verify flow above is very clear, shall we have a page under
> https://iceberg.apache.org/ project tag document it?
> 2. Given it is a voting phase for the release, what if we have a pending
> PR for the updates of website(https://iceberg.apache.org/releases/) so
> that we can verify it in this phase also.
>
> Best,
> tison.
>
>
> Ryan Blue <bl...@apache.org> 于2020年4月29日周三 上午8:37写道：
>
>> +1 (binding)
>>
>> Validated checksums and signature, ran ./gradlew check, and ran license
>> check.
>>
>> I also downloaded Spark 2.4.5 and loaded iceberg-spark-runtime like this:
>>
>> spark-2.4.5-bin-hadoop2.7/bin/spark-shell \
>> --conf spark.jars.repositories=https://repository.apache.org/content/repositories/orgapacheiceberg-1005/ \
>> --packages org.apache.iceberg:iceberg-spark-runtime:0.8.0-incubating \
>> --conf spark.sql.warehouse.dir=/home/blue/tmp/spark-warehouse \
>> --conf spark.hadoop.hive.metastore.uris=thrift://localhost:45655
>>
>> I spot-checked Hive writes, overwrites, metadata tables, and
>> input_file_name().
>>
>> By the way, to run a metastore, I just started one of the Iceberg Hive
>> metastore tests with an infinite loop and added a print statement to get
>> the thrift URI. That’s an easy way to create one.
>>
>> On Tue, Apr 28, 2020 at 1:44 PM Ryan Blue <bl...@apache.org> wrote:
>>
>>> Here are the steps to verify the release that I sent out last time, for
>>> anyone that doesn’t want to look them up:
>>>
>>> 1. Download the source tarball, signature (.asc), and checksum
>>> (.sha512) from
>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>> 2. Import gpg keys: download KEYS and run gpg --import
>>> /path/to/downloaded/KEYS (optional if this hasn’t changed)
>>> 3. Verify the signature by running: gpg --verify
>>> apache-iceberg-0.8.0-incubating.tar.gz.asc
>>> 4. Verify the checksum by running: sha512sum -c
>>> apache-iceberg-0.8.0-incubating.tar.gz.sha512
>>> 5. Untar the archive and go into the source directory: tar xzf
>>> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
>>> 6. Run RAT checks to validate license headers: dev/check-license
>>> 7. Build and test the project: ./gradlew build (use Java 8)
>>>
>>> You can also validate the LICENSE and NOTICE documentation, which is
>>> included in the source tarball, as well as the staged binary artifacts. The
>>> latest update to the spark-runtime Jar’s was PR #966
>>> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
>>> review it.
>>>
>>> To validate the convenience binaries, add the Maven URL from the email
>>> above to a downstream project and update your Iceberg dependency to
>>> 0.8.0-incubating, like this:
>>>
>>> repositories {
>>> maven {
>>> name 'stagedIceberg'
>>> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
>>> }
>>> }
>>>
>>> ext {
>>> icebergVersion = '0.8.0-incubating'
>>> }
>>>
>>> Then run the downstream project’s tests.
>>>
>>> Thanks for reviewing and voting, everyone!
>>>
>>> rb
>>>
>>> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I propose the following RC to be released as official Apache Iceberg
>>>> 0.8.0-incubating release.
>>>>
>>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>>> *
>>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>>
>>>> The release tarball, signature, and checksums are here:
>>>> *
>>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>>
>>>> You can find the KEYS file here:
>>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>>
>>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>>> URL is:
>>>> *
>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>>
>>>> This release contains many bug fixes and several new features:
>>>> * Actions to remove orphaned files and to optimize metadata for query
>>>> performance
>>>> * Support for ORC data files
>>>> * Snapshot cherry-picking
>>>> * Incremental scan planning based on table history
>>>> * In and notIn expressions
>>>> * An InputFormat for writing MR jobs
>>>>
>>>> Please download, verify, and test.
>>>>
>>>> Please vote in the next 72 hours.
>>>>
>>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>>> [ ] +0
>>>> [ ] -1 Do not release this because...
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>
>>
>> --
>> Ryan Blue
>>
>
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by tison <wa...@gmail.com>.
Hi Ryan,
Thanks for starting the voting process.
+1 (non-binding) I verified:
√ RAT checks passed
√ signature is correct
√ checksum is correct
√ build from source
√ run tests locally
I'd like to ask 2 questions:
1. The verify flow above is very clear, shall we have a page under
https://iceberg.apache.org/ project tag document it?
2. Given it is a voting phase for the release, what if we have a pending PR
for the updates of website(https://iceberg.apache.org/releases/) so that we
can verify it in this phase also.
Best,
tison.
Ryan Blue <bl...@apache.org> 于2020年4月29日周三 上午8:37写道：
> +1 (binding)
>
> Validated checksums and signature, ran ./gradlew check, and ran license
> check.
>
> I also downloaded Spark 2.4.5 and loaded iceberg-spark-runtime like this:
>
> spark-2.4.5-bin-hadoop2.7/bin/spark-shell \
> --conf spark.jars.repositories=https://repository.apache.org/content/repositories/orgapacheiceberg-1005/ \
> --packages org.apache.iceberg:iceberg-spark-runtime:0.8.0-incubating \
> --conf spark.sql.warehouse.dir=/home/blue/tmp/spark-warehouse \
> --conf spark.hadoop.hive.metastore.uris=thrift://localhost:45655
>
> I spot-checked Hive writes, overwrites, metadata tables, and
> input_file_name().
>
> By the way, to run a metastore, I just started one of the Iceberg Hive
> metastore tests with an infinite loop and added a print statement to get
> the thrift URI. That’s an easy way to create one.
>
> On Tue, Apr 28, 2020 at 1:44 PM Ryan Blue <bl...@apache.org> wrote:
>
>> Here are the steps to verify the release that I sent out last time, for
>> anyone that doesn’t want to look them up:
>>
>> 1. Download the source tarball, signature (.asc), and checksum
>> (.sha512) from
>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>> 2. Import gpg keys: download KEYS and run gpg --import
>> /path/to/downloaded/KEYS (optional if this hasn’t changed)
>> 3. Verify the signature by running: gpg --verify
>> apache-iceberg-0.8.0-incubating.tar.gz.asc
>> 4. Verify the checksum by running: sha512sum -c
>> apache-iceberg-0.8.0-incubating.tar.gz.sha512
>> 5. Untar the archive and go into the source directory: tar xzf
>> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
>> 6. Run RAT checks to validate license headers: dev/check-license
>> 7. Build and test the project: ./gradlew build (use Java 8)
>>
>> You can also validate the LICENSE and NOTICE documentation, which is
>> included in the source tarball, as well as the staged binary artifacts. The
>> latest update to the spark-runtime Jar’s was PR #966
>> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
>> review it.
>>
>> To validate the convenience binaries, add the Maven URL from the email
>> above to a downstream project and update your Iceberg dependency to
>> 0.8.0-incubating, like this:
>>
>> repositories {
>> maven {
>> name 'stagedIceberg'
>> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
>> }
>> }
>>
>> ext {
>> icebergVersion = '0.8.0-incubating'
>> }
>>
>> Then run the downstream project’s tests.
>>
>> Thanks for reviewing and voting, everyone!
>>
>> rb
>>
>> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>>
>>> Hi everyone,
>>>
>>> I propose the following RC to be released as official Apache Iceberg
>>> 0.8.0-incubating release.
>>>
>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>> *
>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>
>>> The release tarball, signature, and checksums are here:
>>> *
>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>
>>> You can find the KEYS file here:
>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>
>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>> URL is:
>>> *
>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>
>>> This release contains many bug fixes and several new features:
>>> * Actions to remove orphaned files and to optimize metadata for query
>>> performance
>>> * Support for ORC data files
>>> * Snapshot cherry-picking
>>> * Incremental scan planning based on table history
>>> * In and notIn expressions
>>> * An InputFormat for writing MR jobs
>>>
>>> Please download, verify, and test.
>>>
>>> Please vote in the next 72 hours.
>>>
>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>> [ ] +0
>>> [ ] -1 Do not release this because...
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>
>>
>> --
>> Ryan Blue
>>
>
>
> --
> Ryan Blue
>
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Miao Wang <mi...@adobe.com.INVALID>.
+1, Verified locally following instructions
From: Ryan Blue <bl...@apache.org>
Reply-To: "dev@iceberg.apache.org" <de...@iceberg.apache.org>
Date: Tuesday, April 28, 2020 at 5:37 PM
To: Ryan Blue <bl...@apache.org>
Cc: Iceberg Dev List <de...@iceberg.apache.org>
Subject: Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
+1 (binding)
Validated checksums and signature, ran ./gradlew check, and ran license check.
I also downloaded Spark 2.4.5 and loaded iceberg-spark-runtime like this:
spark-2.4.5-bin-hadoop2.7/bin/spark-shell \
--conf spark.jars.repositories=https://repository.apache.org/content/repositories/orgapacheiceberg-1005/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapacheiceberg-1005%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357275264&sdata=%2F2j0GQdcJkJenSuCEFDdxNGCCOESL%2B8R8mjXsp0JO%2F8%3D&reserved=0> \
--packages org.apache.iceberg:iceberg-spark-runtime:0.8.0-incubating \
--conf spark.sql.warehouse.dir=/home/blue/tmp/spark-warehouse \
--conf spark.hadoop.hive.metastore.uris=thrift://localhost:45655
I spot-checked Hive writes, overwrites, metadata tables, and input_file_name().
By the way, to run a metastore, I just started one of the Iceberg Hive metastore tests with an infinite loop and added a print statement to get the thrift URI. That’s an easy way to create one.
On Tue, Apr 28, 2020 at 1:44 PM Ryan Blue <bl...@apache.org>> wrote:
Here are the steps to verify the release that I sent out last time, for anyone that doesn’t want to look them up:
1. Download the source tarball, signature (.asc), and checksum (.sha512) from https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Ficeberg%2Fapache-iceberg-0.8.0-incubating-rc1%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357275264&sdata=XUVa4AXB4sZ5Q0DHURfbZ2a4RoLlTBAUE7YBe3y6uAg%3D&reserved=0>
2. Import gpg keys: download KEYS and run gpg --import /path/to/downloaded/KEYS (optional if this hasn’t changed)
3. Verify the signature by running: gpg --verify apache-iceberg-0.8.0-incubating.tar.gz.asc
4. Verify the checksum by running: sha512sum -c apache-iceberg-0.8.0-incubating.tar.gz.sha512
5. Untar the archive and go into the source directory: tar xzf apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
6. Run RAT checks to validate license headers: dev/check-license
7. Build and test the project: ./gradlew build (use Java 8)
You can also validate the LICENSE and NOTICE documentation, which is included in the source tarball, as well as the staged binary artifacts. The latest update to the spark-runtime Jar’s was PR #966<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-iceberg%2Fpull%2F966&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357285257&sdata=FOGI9awfJsk2d71Nd1Rbk3imEwOS7vbTI7lEDWvk2QY%3D&reserved=0> if you’d like to review it.
To validate the convenience binaries, add the Maven URL from the email above to a downstream project and update your Iceberg dependency to 0.8.0-incubating, like this:
repositories {
maven {
name 'stagedIceberg'
url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapacheiceberg-1005%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357285257&sdata=8u5SE9a%2Fl2yPNxxxT7%2FiwX2aqPz%2BmFAvVp1Yj54hssA%3D&reserved=0>'
}
}
ext {
icebergVersion = '0.8.0-incubating'
}
Then run the downstream project’s tests.
Thanks for reviewing and voting, everyone!
rb
On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org>> wrote:
Hi everyone,
I propose the following RC to be released as official Apache Iceberg 0.8.0-incubating release.
The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
* This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
* https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-iceberg%2Fcommits%2Fapache-iceberg-0.8.0-incubating-rc1&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357295250&sdata=WuS1FljHsB%2BezfOKy%2F1i7L3Gc1LxBpIw6aPMjSBmisg%3D&reserved=0>
* https://github.com/apache/incubator-iceberg/tree/4c2dd0ac<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-iceberg%2Ftree%2F4c2dd0ac&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357295250&sdata=wb7sm%2FZ4xGkFjrgZZ0Eo5E8p03iuETWSSdzITnuLelk%3D&reserved=0>
The release tarball, signature, and checksums are here:
* https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Ficeberg%2Fapache-iceberg-0.8.0-incubating-rc1%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357295250&sdata=otK4CxUcuFgeV9Bj5Kxc7Z2w3ey6ytKX9QRJyzbVT5A%3D&reserved=0>
You can find the KEYS file here:
* https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Ficeberg%2FKEYS&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357305243&sdata=DbQHT2qQzxgLS9Sa9Yerfmw3oILHL0gAk%2FhHrwdx%2FC0%3D&reserved=0>
Convenience binary artifacts are staged in Nexus. The Maven repository URL is:
* https://repository.apache.org/content/repositories/orgapacheiceberg-1005/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapacheiceberg-1005%2F&data=02%7C01%7Cmiwang%40adobe.com%7Cc529012d0b104c984b3b08d7ebd5764c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637237174357305243&sdata=%2FyeX2KAvoam4BQosIhFJQLSzBvSzj1a3N3eIkdQoigw%3D&reserved=0>
This release contains many bug fixes and several new features:
* Actions to remove orphaned files and to optimize metadata for query performance
* Support for ORC data files
* Snapshot cherry-picking
* Incremental scan planning based on table history
* In and notIn expressions
* An InputFormat for writing MR jobs
Please download, verify, and test.
Please vote in the next 72 hours.
[ ] +1 Release this as Apache Iceberg 0.8.0-incubating
[ ] +0
[ ] -1 Do not release this because...
--
Ryan Blue
--
Ryan Blue
--
Ryan Blue
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Ryan Blue <bl...@apache.org>.
+1 (binding)
Validated checksums and signature, ran ./gradlew check, and ran license
check.
I also downloaded Spark 2.4.5 and loaded iceberg-spark-runtime like this:
spark-2.4.5-bin-hadoop2.7/bin/spark-shell \
--conf spark.jars.repositories=https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
\
--packages org.apache.iceberg:iceberg-spark-runtime:0.8.0-incubating \
--conf spark.sql.warehouse.dir=/home/blue/tmp/spark-warehouse \
--conf spark.hadoop.hive.metastore.uris=thrift://localhost:45655
I spot-checked Hive writes, overwrites, metadata tables, and
input_file_name().
By the way, to run a metastore, I just started one of the Iceberg Hive
metastore tests with an infinite loop and added a print statement to get
the thrift URI. That’s an easy way to create one.
On Tue, Apr 28, 2020 at 1:44 PM Ryan Blue <bl...@apache.org> wrote:
> Here are the steps to verify the release that I sent out last time, for
> anyone that doesn’t want to look them up:
>
> 1. Download the source tarball, signature (.asc), and checksum
> (.sha512) from
> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
> 2. Import gpg keys: download KEYS and run gpg --import
> /path/to/downloaded/KEYS (optional if this hasn’t changed)
> 3. Verify the signature by running: gpg --verify
> apache-iceberg-0.8.0-incubating.tar.gz.asc
> 4. Verify the checksum by running: sha512sum -c
> apache-iceberg-0.8.0-incubating.tar.gz.sha512
> 5. Untar the archive and go into the source directory: tar xzf
> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
> 6. Run RAT checks to validate license headers: dev/check-license
> 7. Build and test the project: ./gradlew build (use Java 8)
>
> You can also validate the LICENSE and NOTICE documentation, which is
> included in the source tarball, as well as the staged binary artifacts. The
> latest update to the spark-runtime Jar’s was PR #966
> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
> review it.
>
> To validate the convenience binaries, add the Maven URL from the email
> above to a downstream project and update your Iceberg dependency to
> 0.8.0-incubating, like this:
>
> repositories {
> maven {
> name 'stagedIceberg'
> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
> }
> }
>
> ext {
> icebergVersion = '0.8.0-incubating'
> }
>
> Then run the downstream project’s tests.
>
> Thanks for reviewing and voting, everyone!
>
> rb
>
> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>
>> Hi everyone,
>>
>> I propose the following RC to be released as official Apache Iceberg
>> 0.8.0-incubating release.
>>
>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>> *
>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>
>> The release tarball, signature, and checksums are here:
>> *
>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>
>> You can find the KEYS file here:
>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>
>> Convenience binary artifacts are staged in Nexus. The Maven repository
>> URL is:
>> *
>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>
>> This release contains many bug fixes and several new features:
>> * Actions to remove orphaned files and to optimize metadata for query
>> performance
>> * Support for ORC data files
>> * Snapshot cherry-picking
>> * Incremental scan planning based on table history
>> * In and notIn expressions
>> * An InputFormat for writing MR jobs
>>
>> Please download, verify, and test.
>>
>> Please vote in the next 72 hours.
>>
>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>> [ ] +0
>> [ ] -1 Do not release this because...
>>
>>
>> --
>> Ryan Blue
>>
>
>
> --
> Ryan Blue
>
--
Ryan Blue
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Anton Okolnychyi <ao...@apple.com.INVALID>.
There is one issue we may want to fix before the release.
We should suppress exceptions while unlocking in HiveTableOperations. Right now, we throw a runtime exception. This leads to cases when the commit is actually successful and we swap the pointer but the subsequent exception during unlock triggers the clean-up logic and we remove valid metadata files. This corrupts the table metadata as HMS will have references to files that are not there. If this happens, people will have to drop the table, find the correct metadata file and recreate the table from it.
I am preparing a fix right now. I’d rather include it in the release. What does everybody think?
- Anton
> On 29 Apr 2020, at 09:23, RD <rd...@gmail.com> wrote:
>
> +1 . Verified the outlined steps.
>
> -R
>
> On Wed, Apr 29, 2020 at 8:09 AM tison <wander4096@gmail.com <ma...@gmail.com>> wrote:
> Hi Junjie,
>
> It is kind of GPG scope issue that you should manually trust Ryan's KEYS.
>
> FYI https://www.gnupg.org/gph/en/manual/x334.html <https://www.gnupg.org/gph/en/manual/x334.html>
>
> Best,
> tison.
>
>
> Junjie Chen <chenjunjiedada@gmail.com <ma...@gmail.com>> 于2020年4月29日周三 下午9:47写道：
> Ran 1-7 steps
> found a warning:
> gpg: assuming signed data in 'apache-iceberg-0.8.0-incubating.tar.gz'
> gpg: Signature made Wed Apr 29 04:07:57 2020 CST
> gpg: using RSA key D21CFB9BDBC379681261C7A086781D4FA4B2E9B5
> gpg: Good signature from "Ryan Blue (CODE SIGNING KEY) <blue@apache.org <ma...@apache.org>>" [unknown]
> gpg: aka "Ryan Blue <blue@apache.org <ma...@apache.org>>" [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg: There is no indication that the signature belongs to the owner.
>
> +1 (non-binding), if the warning is ok.
>
>
>
>
> On Wed, Apr 29, 2020 at 7:37 PM Mass Dosage <massdosage@gmail.com <ma...@gmail.com>> wrote:
> +1 (non-binding) [I assume only Apache/Iceberg members have binding votes?)
>
> Similar to others I verified:
>
> √ RAT checks passed
> √ signature is correct
> √ checksum is correct
> √ build from source
> √ run tests locally
>
> Thanks,
>
> Adrian
>
> On Tue, 28 Apr 2020 at 21:45, Ryan Blue <blue@apache.org <ma...@apache.org>> wrote:
> Here are the steps to verify the release that I sent out last time, for anyone that doesn’t want to look them up:
>
> Download the source tarball, signature (.asc), and checksum (.sha512) from https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/ <https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/>
> Import gpg keys: download KEYS and run gpg --import /path/to/downloaded/KEYS (optional if this hasn’t changed)
> Verify the signature by running: gpg --verify apache-iceberg-0.8.0-incubating.tar.gz.asc
> Verify the checksum by running: sha512sum -c apache-iceberg-0.8.0-incubating.tar.gz.sha512
> Untar the archive and go into the source directory: tar xzf apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
> Run RAT checks to validate license headers: dev/check-license
> Build and test the project: ./gradlew build (use Java 8)
> You can also validate the LICENSE and NOTICE documentation, which is included in the source tarball, as well as the staged binary artifacts. The latest update to the spark-runtime Jar’s was PR #966 <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to review it.
>
> To validate the convenience binaries, add the Maven URL from the email above to a downstream project and update your Iceberg dependency to 0.8.0-incubating, like this:
>
> repositories {
> maven {
> name 'stagedIceberg'
> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/ <https://repository.apache.org/content/repositories/orgapacheiceberg-1005/>'
> }
> }
>
> ext {
> icebergVersion = '0.8.0-incubating'
> }
> Then run the downstream project’s tests.
>
> Thanks for reviewing and voting, everyone!
>
> rb
>
>
> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <blue@apache.org <ma...@apache.org>> wrote:
> Hi everyone,
>
> I propose the following RC to be released as official Apache Iceberg 0.8.0-incubating release.
>
> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
> * https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1 <https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1>
> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac <https://github.com/apache/incubator-iceberg/tree/4c2dd0ac>
>
> The release tarball, signature, and checksums are here:
> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/ <https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/>
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS <https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS>
>
> Convenience binary artifacts are staged in Nexus. The Maven repository URL is:
> * https://repository.apache.org/content/repositories/orgapacheiceberg-1005/ <https://repository.apache.org/content/repositories/orgapacheiceberg-1005/>
>
> This release contains many bug fixes and several new features:
> * Actions to remove orphaned files and to optimize metadata for query performance
> * Support for ORC data files
> * Snapshot cherry-picking
> * Incremental scan planning based on table history
> * In and notIn expressions
> * An InputFormat for writing MR jobs
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
> [ ] +0
> [ ] -1 Do not release this because...
>
>
> --
> Ryan Blue
>
>
> --
> Ryan Blue
>
>
> --
> Best Regards
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by RD <rd...@gmail.com>.
+1 . Verified the outlined steps.
-R
On Wed, Apr 29, 2020 at 8:09 AM tison <wa...@gmail.com> wrote:
> Hi Junjie,
>
> It is kind of GPG scope issue that you should manually trust Ryan's KEYS.
>
> FYI https://www.gnupg.org/gph/en/manual/x334.html
>
> Best,
> tison.
>
>
> Junjie Chen <ch...@gmail.com> 于2020年4月29日周三 下午9:47写道：
>
>> Ran 1-7 steps
>> found a warning:
>> gpg: assuming signed data in 'apache-iceberg-0.8.0-incubating.tar.gz'
>> gpg: Signature made Wed Apr 29 04:07:57 2020 CST
>> gpg: using RSA key D21CFB9BDBC379681261C7A086781D4FA4B2E9B5
>> gpg: Good signature from "Ryan Blue (CODE SIGNING KEY) <bl...@apache.org>"
>> [unknown]
>> gpg: aka "Ryan Blue <bl...@apache.org>" [unknown]
>> gpg: WARNING: This key is not certified with a trusted signature!
>> gpg: There is no indication that the signature belongs to the
>> owner.
>>
>> +1 (non-binding), if the warning is ok.
>>
>>
>>
>>
>> On Wed, Apr 29, 2020 at 7:37 PM Mass Dosage <ma...@gmail.com> wrote:
>>
>>> +1 (non-binding) [I assume only Apache/Iceberg members have binding
>>> votes?)
>>>
>>> Similar to others I verified:
>>>
>>> √ RAT checks passed
>>> √ signature is correct
>>> √ checksum is correct
>>> √ build from source
>>> √ run tests locally
>>>
>>> Thanks,
>>>
>>> Adrian
>>>
>>> On Tue, 28 Apr 2020 at 21:45, Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> Here are the steps to verify the release that I sent out last time, for
>>>> anyone that doesn’t want to look them up:
>>>>
>>>> 1. Download the source tarball, signature (.asc), and checksum
>>>> (.sha512) from
>>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>> 2. Import gpg keys: download KEYS and run gpg --import
>>>> /path/to/downloaded/KEYS (optional if this hasn’t changed)
>>>> 3. Verify the signature by running: gpg --verify
>>>> apache-iceberg-0.8.0-incubating.tar.gz.asc
>>>> 4. Verify the checksum by running: sha512sum -c
>>>> apache-iceberg-0.8.0-incubating.tar.gz.sha512
>>>> 5. Untar the archive and go into the source directory: tar xzf
>>>> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
>>>> 6. Run RAT checks to validate license headers: dev/check-license
>>>> 7. Build and test the project: ./gradlew build (use Java 8)
>>>>
>>>> You can also validate the LICENSE and NOTICE documentation, which is
>>>> included in the source tarball, as well as the staged binary artifacts. The
>>>> latest update to the spark-runtime Jar’s was PR #966
>>>> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like
>>>> to review it.
>>>>
>>>> To validate the convenience binaries, add the Maven URL from the email
>>>> above to a downstream project and update your Iceberg dependency to
>>>> 0.8.0-incubating, like this:
>>>>
>>>> repositories {
>>>> maven {
>>>> name 'stagedIceberg'
>>>> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
>>>> }
>>>> }
>>>>
>>>> ext {
>>>> icebergVersion = '0.8.0-incubating'
>>>> }
>>>>
>>>> Then run the downstream project’s tests.
>>>>
>>>> Thanks for reviewing and voting, everyone!
>>>>
>>>> rb
>>>>
>>>> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I propose the following RC to be released as official Apache Iceberg
>>>>> 0.8.0-incubating release.
>>>>>
>>>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>>>> *
>>>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>>>
>>>>> The release tarball, signature, and checksums are here:
>>>>> *
>>>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>>>
>>>>> You can find the KEYS file here:
>>>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>>>
>>>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>>>> URL is:
>>>>> *
>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>>>
>>>>> This release contains many bug fixes and several new features:
>>>>> * Actions to remove orphaned files and to optimize metadata for query
>>>>> performance
>>>>> * Support for ORC data files
>>>>> * Snapshot cherry-picking
>>>>> * Incremental scan planning based on table history
>>>>> * In and notIn expressions
>>>>> * An InputFormat for writing MR jobs
>>>>>
>>>>> Please download, verify, and test.
>>>>>
>>>>> Please vote in the next 72 hours.
>>>>>
>>>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>>>> [ ] +0
>>>>> [ ] -1 Do not release this because...
>>>>>
>>>>>
>>>>> --
>>>>> Ryan Blue
>>>>>
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>
>>
>> --
>> Best Regards
>>
>
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by tison <wa...@gmail.com>.
Hi Junjie,
It is kind of GPG scope issue that you should manually trust Ryan's KEYS.
FYI https://www.gnupg.org/gph/en/manual/x334.html
Best,
tison.
Junjie Chen <ch...@gmail.com> 于2020年4月29日周三 下午9:47写道：
> Ran 1-7 steps
> found a warning:
> gpg: assuming signed data in 'apache-iceberg-0.8.0-incubating.tar.gz'
> gpg: Signature made Wed Apr 29 04:07:57 2020 CST
> gpg: using RSA key D21CFB9BDBC379681261C7A086781D4FA4B2E9B5
> gpg: Good signature from "Ryan Blue (CODE SIGNING KEY) <bl...@apache.org>"
> [unknown]
> gpg: aka "Ryan Blue <bl...@apache.org>" [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg: There is no indication that the signature belongs to the
> owner.
>
> +1 (non-binding), if the warning is ok.
>
>
>
>
> On Wed, Apr 29, 2020 at 7:37 PM Mass Dosage <ma...@gmail.com> wrote:
>
>> +1 (non-binding) [I assume only Apache/Iceberg members have binding
>> votes?)
>>
>> Similar to others I verified:
>>
>> √ RAT checks passed
>> √ signature is correct
>> √ checksum is correct
>> √ build from source
>> √ run tests locally
>>
>> Thanks,
>>
>> Adrian
>>
>> On Tue, 28 Apr 2020 at 21:45, Ryan Blue <bl...@apache.org> wrote:
>>
>>> Here are the steps to verify the release that I sent out last time, for
>>> anyone that doesn’t want to look them up:
>>>
>>> 1. Download the source tarball, signature (.asc), and checksum
>>> (.sha512) from
>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>> 2. Import gpg keys: download KEYS and run gpg --import
>>> /path/to/downloaded/KEYS (optional if this hasn’t changed)
>>> 3. Verify the signature by running: gpg --verify
>>> apache-iceberg-0.8.0-incubating.tar.gz.asc
>>> 4. Verify the checksum by running: sha512sum -c
>>> apache-iceberg-0.8.0-incubating.tar.gz.sha512
>>> 5. Untar the archive and go into the source directory: tar xzf
>>> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
>>> 6. Run RAT checks to validate license headers: dev/check-license
>>> 7. Build and test the project: ./gradlew build (use Java 8)
>>>
>>> You can also validate the LICENSE and NOTICE documentation, which is
>>> included in the source tarball, as well as the staged binary artifacts. The
>>> latest update to the spark-runtime Jar’s was PR #966
>>> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
>>> review it.
>>>
>>> To validate the convenience binaries, add the Maven URL from the email
>>> above to a downstream project and update your Iceberg dependency to
>>> 0.8.0-incubating, like this:
>>>
>>> repositories {
>>> maven {
>>> name 'stagedIceberg'
>>> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
>>> }
>>> }
>>>
>>> ext {
>>> icebergVersion = '0.8.0-incubating'
>>> }
>>>
>>> Then run the downstream project’s tests.
>>>
>>> Thanks for reviewing and voting, everyone!
>>>
>>> rb
>>>
>>> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I propose the following RC to be released as official Apache Iceberg
>>>> 0.8.0-incubating release.
>>>>
>>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>>> *
>>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>>
>>>> The release tarball, signature, and checksums are here:
>>>> *
>>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>>
>>>> You can find the KEYS file here:
>>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>>
>>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>>> URL is:
>>>> *
>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>>
>>>> This release contains many bug fixes and several new features:
>>>> * Actions to remove orphaned files and to optimize metadata for query
>>>> performance
>>>> * Support for ORC data files
>>>> * Snapshot cherry-picking
>>>> * Incremental scan planning based on table history
>>>> * In and notIn expressions
>>>> * An InputFormat for writing MR jobs
>>>>
>>>> Please download, verify, and test.
>>>>
>>>> Please vote in the next 72 hours.
>>>>
>>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>>> [ ] +0
>>>> [ ] -1 Do not release this because...
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>
>
> --
> Best Regards
>
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Junjie Chen <ch...@gmail.com>.
Ran 1-7 steps
found a warning:
gpg: assuming signed data in 'apache-iceberg-0.8.0-incubating.tar.gz'
gpg: Signature made Wed Apr 29 04:07:57 2020 CST
gpg: using RSA key D21CFB9BDBC379681261C7A086781D4FA4B2E9B5
gpg: Good signature from "Ryan Blue (CODE SIGNING KEY) <bl...@apache.org>"
[unknown]
gpg: aka "Ryan Blue <bl...@apache.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the
owner.
+1 (non-binding), if the warning is ok.
On Wed, Apr 29, 2020 at 7:37 PM Mass Dosage <ma...@gmail.com> wrote:
> +1 (non-binding) [I assume only Apache/Iceberg members have binding votes?)
>
> Similar to others I verified:
>
> √ RAT checks passed
> √ signature is correct
> √ checksum is correct
> √ build from source
> √ run tests locally
>
> Thanks,
>
> Adrian
>
> On Tue, 28 Apr 2020 at 21:45, Ryan Blue <bl...@apache.org> wrote:
>
>> Here are the steps to verify the release that I sent out last time, for
>> anyone that doesn’t want to look them up:
>>
>> 1. Download the source tarball, signature (.asc), and checksum
>> (.sha512) from
>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>> 2. Import gpg keys: download KEYS and run gpg --import
>> /path/to/downloaded/KEYS (optional if this hasn’t changed)
>> 3. Verify the signature by running: gpg --verify
>> apache-iceberg-0.8.0-incubating.tar.gz.asc
>> 4. Verify the checksum by running: sha512sum -c
>> apache-iceberg-0.8.0-incubating.tar.gz.sha512
>> 5. Untar the archive and go into the source directory: tar xzf
>> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
>> 6. Run RAT checks to validate license headers: dev/check-license
>> 7. Build and test the project: ./gradlew build (use Java 8)
>>
>> You can also validate the LICENSE and NOTICE documentation, which is
>> included in the source tarball, as well as the staged binary artifacts. The
>> latest update to the spark-runtime Jar’s was PR #966
>> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
>> review it.
>>
>> To validate the convenience binaries, add the Maven URL from the email
>> above to a downstream project and update your Iceberg dependency to
>> 0.8.0-incubating, like this:
>>
>> repositories {
>> maven {
>> name 'stagedIceberg'
>> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
>> }
>> }
>>
>> ext {
>> icebergVersion = '0.8.0-incubating'
>> }
>>
>> Then run the downstream project’s tests.
>>
>> Thanks for reviewing and voting, everyone!
>>
>> rb
>>
>> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>>
>>> Hi everyone,
>>>
>>> I propose the following RC to be released as official Apache Iceberg
>>> 0.8.0-incubating release.
>>>
>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>> *
>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>
>>> The release tarball, signature, and checksums are here:
>>> *
>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>
>>> You can find the KEYS file here:
>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>
>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>> URL is:
>>> *
>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>
>>> This release contains many bug fixes and several new features:
>>> * Actions to remove orphaned files and to optimize metadata for query
>>> performance
>>> * Support for ORC data files
>>> * Snapshot cherry-picking
>>> * Incremental scan planning based on table history
>>> * In and notIn expressions
>>> * An InputFormat for writing MR jobs
>>>
>>> Please download, verify, and test.
>>>
>>> Please vote in the next 72 hours.
>>>
>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>> [ ] +0
>>> [ ] -1 Do not release this because...
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>
>>
>> --
>> Ryan Blue
>>
>
--
Best Regards
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Mass Dosage <ma...@gmail.com>.
+1 (non-binding) [I assume only Apache/Iceberg members have binding votes?)
Similar to others I verified:
√ RAT checks passed
√ signature is correct
√ checksum is correct
√ build from source
√ run tests locally
Thanks,
Adrian
On Tue, 28 Apr 2020 at 21:45, Ryan Blue <bl...@apache.org> wrote:
> Here are the steps to verify the release that I sent out last time, for
> anyone that doesn’t want to look them up:
>
> 1. Download the source tarball, signature (.asc), and checksum
> (.sha512) from
> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
> 2. Import gpg keys: download KEYS and run gpg --import
> /path/to/downloaded/KEYS (optional if this hasn’t changed)
> 3. Verify the signature by running: gpg --verify
> apache-iceberg-0.8.0-incubating.tar.gz.asc
> 4. Verify the checksum by running: sha512sum -c
> apache-iceberg-0.8.0-incubating.tar.gz.sha512
> 5. Untar the archive and go into the source directory: tar xzf
> apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
> 6. Run RAT checks to validate license headers: dev/check-license
> 7. Build and test the project: ./gradlew build (use Java 8)
>
> You can also validate the LICENSE and NOTICE documentation, which is
> included in the source tarball, as well as the staged binary artifacts. The
> latest update to the spark-runtime Jar’s was PR #966
> <https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
> review it.
>
> To validate the convenience binaries, add the Maven URL from the email
> above to a downstream project and update your Iceberg dependency to
> 0.8.0-incubating, like this:
>
> repositories {
> maven {
> name 'stagedIceberg'
> url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
> }
> }
>
> ext {
> icebergVersion = '0.8.0-incubating'
> }
>
> Then run the downstream project’s tests.
>
> Thanks for reviewing and voting, everyone!
>
> rb
>
> On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
>
>> Hi everyone,
>>
>> I propose the following RC to be released as official Apache Iceberg
>> 0.8.0-incubating release.
>>
>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>> *
>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>
>> The release tarball, signature, and checksums are here:
>> *
>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>
>> You can find the KEYS file here:
>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>
>> Convenience binary artifacts are staged in Nexus. The Maven repository
>> URL is:
>> *
>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>
>> This release contains many bug fixes and several new features:
>> * Actions to remove orphaned files and to optimize metadata for query
>> performance
>> * Support for ORC data files
>> * Snapshot cherry-picking
>> * Incremental scan planning based on table history
>> * In and notIn expressions
>> * An InputFormat for writing MR jobs
>>
>> Please download, verify, and test.
>>
>> Please vote in the next 72 hours.
>>
>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>> [ ] +0
>> [ ] -1 Do not release this because...
>>
>>
>> --
>> Ryan Blue
>>
>
>
> --
> Ryan Blue
>
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Ryan Blue <bl...@apache.org>.
Here are the steps to verify the release that I sent out last time, for
anyone that doesn’t want to look them up:
1. Download the source tarball, signature (.asc), and checksum (.sha512)
from
https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
2. Import gpg keys: download KEYS and run gpg --import
/path/to/downloaded/KEYS (optional if this hasn’t changed)
3. Verify the signature by running: gpg --verify
apache-iceberg-0.8.0-incubating.tar.gz.asc
4. Verify the checksum by running: sha512sum -c
apache-iceberg-0.8.0-incubating.tar.gz.sha512
5. Untar the archive and go into the source directory: tar xzf
apache-iceberg-0.8.0-incubating.tar.gz && cd apache-iceberg-0.8.0-incubating
6. Run RAT checks to validate license headers: dev/check-license
7. Build and test the project: ./gradlew build (use Java 8)
You can also validate the LICENSE and NOTICE documentation, which is
included in the source tarball, as well as the staged binary artifacts. The
latest update to the spark-runtime Jar’s was PR #966
<https://github.com/apache/incubator-iceberg/pull/966> if you’d like to
review it.
To validate the convenience binaries, add the Maven URL from the email
above to a downstream project and update your Iceberg dependency to
0.8.0-incubating, like this:
repositories {
maven {
name 'stagedIceberg'
url 'https://repository.apache.org/content/repositories/orgapacheiceberg-1005/'
}
}
ext {
icebergVersion = '0.8.0-incubating'
}
Then run the downstream project’s tests.
Thanks for reviewing and voting, everyone!
rb
On Tue, Apr 28, 2020 at 1:39 PM Ryan Blue <bl...@apache.org> wrote:
> Hi everyone,
>
> I propose the following RC to be released as official Apache Iceberg
> 0.8.0-incubating release.
>
> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
> *
> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>
> The release tarball, signature, and checksums are here:
> *
> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>
> Convenience binary artifacts are staged in Nexus. The Maven repository URL
> is:
> *
> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>
> This release contains many bug fixes and several new features:
> * Actions to remove orphaned files and to optimize metadata for query
> performance
> * Support for ORC data files
> * Snapshot cherry-picking
> * Incremental scan planning based on table history
> * In and notIn expressions
> * An InputFormat for writing MR jobs
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
> [ ] +0
> [ ] -1 Do not release this because...
>
>
> --
> Ryan Blue
>
--
Ryan Blue
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
We've committed a fix for the issue that James found and Anton has a bug
fix for table locking as well, so I'm going to start a new vote for RC2.
Thanks for voting on this, everyone!
On Wed, Apr 29, 2020 at 4:18 PM Ryan Blue <rb...@netflix.com> wrote:
> Thanks, Parth! I updated the PR. Could you review it?
>
> On Wed, Apr 29, 2020 at 4:11 PM parth brahmbhatt <
> brahmbhatt.parth@gmail.com> wrote:
>
>> +1 once we apply the patch.
>>
>> The patch works with presto but it breaks iceberg build. We need to add a
>> typecast to FileIO in TableMetadataParserTest here
>> <https://github.com/Parth-Brahmbhatt/incubator-iceberg/blob/presto-patch/core/src/test/java/org/apache/iceberg/TableMetadataParserTest.java#L78>
>> .
>>
>> Thanks
>> Parth
>>
>>
>>
>> On Wed, Apr 29, 2020 at 2:19 PM Ryan Blue <rb...@netflix.com.invalid>
>> wrote:
>>
>>> Thanks, James. Is it possible for you to test with this patch?
>>> https://patch-diff.githubusercontent.com/raw/apache/incubator-iceberg/pull/986.patch
>>>
>>> Those methods were removed, and it should be easy for Presto to update.
>>> That said, I don't like breaking downstream projects so it would be nice to
>>> fix it. I think this patch might.
>>>
>>> On Wed, Apr 29, 2020 at 1:12 PM James Taylor <ja...@apache.org>
>>> wrote:
>>>
>>>> Verified signature, checksum, rat checks, build, and unit tests. All
>>>> looked good. Tried building latest prestosql master with proposed new
>>>> iceberg release, but received these compilation errors:
>>>>
>>>> [ERROR] Failed to execute goal
>>>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile
>>>> (default-compile) on project presto-iceberg: Compilation failure:
>>>> Compilation failure:
>>>> [ERROR]
>>>> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/IcebergMetadata.java:[359,34]
>>>> method newTableMetadata in class org.apache.iceberg.TableMetadata cannot be
>>>> applied to given types;
>>>> [ERROR] required:
>>>> org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,java.util.Map<java.lang.String,java.lang.String>
>>>> [ERROR] found:
>>>> org.apache.iceberg.TableOperations,org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,com.google.common.collect.ImmutableMap<java.lang.String,java.lang.String>
>>>> [ERROR] reason: actual and formal argument lists differ in length
>>>> [ERROR]
>>>> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/HiveTableOperations.java:[301,44]
>>>> no suitable method found for
>>>> read(io.prestosql.plugin.iceberg.HiveTableOperations,org.apache.iceberg.io.InputFile)
>>>> [ERROR] method
>>>> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,java.lang.String)
>>>> is not applicable
>>>> [ERROR] (argument mismatch;
>>>> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
>>>> org.apache.iceberg.io.FileIO)
>>>> [ERROR] method
>>>> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,org.apache.iceberg.io.InputFile)
>>>> is not applicable
>>>> [ERROR] (argument mismatch;
>>>> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
>>>> org.apache.iceberg.io.FileIO)
>>>>
>>>> Is this a showstopper? Probably a simple PR in presto would fix it, but
>>>> unclear if b/w compatibility is a goal or not.
>>>>
>>>> Thanks,
>>>> James
>>>>
>>>> On Tue, Apr 28, 2020 at 1:40 PM Ryan Blue <bl...@apache.org> wrote:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I propose the following RC to be released as official Apache Iceberg
>>>>> 0.8.0-incubating release.
>>>>>
>>>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>>>> *
>>>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>>>
>>>>> The release tarball, signature, and checksums are here:
>>>>> *
>>>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>>>
>>>>> You can find the KEYS file here:
>>>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>>>
>>>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>>>> URL is:
>>>>> *
>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>>>
>>>>> This release contains many bug fixes and several new features:
>>>>> * Actions to remove orphaned files and to optimize metadata for query
>>>>> performance
>>>>> * Support for ORC data files
>>>>> * Snapshot cherry-picking
>>>>> * Incremental scan planning based on table history
>>>>> * In and notIn expressions
>>>>> * An InputFormat for writing MR jobs
>>>>>
>>>>> Please download, verify, and test.
>>>>>
>>>>> Please vote in the next 72 hours.
>>>>>
>>>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>>>> [ ] +0
>>>>> [ ] -1 Do not release this because...
>>>>>
>>>>>
>>>>> --
>>>>> Ryan Blue
>>>>>
>>>>
>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Netflix
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
--
Ryan Blue
Software Engineer
Netflix
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Thanks, Parth! I updated the PR. Could you review it?
On Wed, Apr 29, 2020 at 4:11 PM parth brahmbhatt <br...@gmail.com>
wrote:
> +1 once we apply the patch.
>
> The patch works with presto but it breaks iceberg build. We need to add a
> typecast to FileIO in TableMetadataParserTest here
> <https://github.com/Parth-Brahmbhatt/incubator-iceberg/blob/presto-patch/core/src/test/java/org/apache/iceberg/TableMetadataParserTest.java#L78>
> .
>
> Thanks
> Parth
>
>
>
> On Wed, Apr 29, 2020 at 2:19 PM Ryan Blue <rb...@netflix.com.invalid>
> wrote:
>
>> Thanks, James. Is it possible for you to test with this patch?
>> https://patch-diff.githubusercontent.com/raw/apache/incubator-iceberg/pull/986.patch
>>
>> Those methods were removed, and it should be easy for Presto to update.
>> That said, I don't like breaking downstream projects so it would be nice to
>> fix it. I think this patch might.
>>
>> On Wed, Apr 29, 2020 at 1:12 PM James Taylor <ja...@apache.org>
>> wrote:
>>
>>> Verified signature, checksum, rat checks, build, and unit tests. All
>>> looked good. Tried building latest prestosql master with proposed new
>>> iceberg release, but received these compilation errors:
>>>
>>> [ERROR] Failed to execute goal
>>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile
>>> (default-compile) on project presto-iceberg: Compilation failure:
>>> Compilation failure:
>>> [ERROR]
>>> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/IcebergMetadata.java:[359,34]
>>> method newTableMetadata in class org.apache.iceberg.TableMetadata cannot be
>>> applied to given types;
>>> [ERROR] required:
>>> org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,java.util.Map<java.lang.String,java.lang.String>
>>> [ERROR] found:
>>> org.apache.iceberg.TableOperations,org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,com.google.common.collect.ImmutableMap<java.lang.String,java.lang.String>
>>> [ERROR] reason: actual and formal argument lists differ in length
>>> [ERROR]
>>> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/HiveTableOperations.java:[301,44]
>>> no suitable method found for
>>> read(io.prestosql.plugin.iceberg.HiveTableOperations,org.apache.iceberg.io.InputFile)
>>> [ERROR] method
>>> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,java.lang.String)
>>> is not applicable
>>> [ERROR] (argument mismatch;
>>> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
>>> org.apache.iceberg.io.FileIO)
>>> [ERROR] method
>>> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,org.apache.iceberg.io.InputFile)
>>> is not applicable
>>> [ERROR] (argument mismatch;
>>> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
>>> org.apache.iceberg.io.FileIO)
>>>
>>> Is this a showstopper? Probably a simple PR in presto would fix it, but
>>> unclear if b/w compatibility is a goal or not.
>>>
>>> Thanks,
>>> James
>>>
>>> On Tue, Apr 28, 2020 at 1:40 PM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I propose the following RC to be released as official Apache Iceberg
>>>> 0.8.0-incubating release.
>>>>
>>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>>> *
>>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>>
>>>> The release tarball, signature, and checksums are here:
>>>> *
>>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>>
>>>> You can find the KEYS file here:
>>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>>
>>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>>> URL is:
>>>> *
>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>>
>>>> This release contains many bug fixes and several new features:
>>>> * Actions to remove orphaned files and to optimize metadata for query
>>>> performance
>>>> * Support for ORC data files
>>>> * Snapshot cherry-picking
>>>> * Incremental scan planning based on table history
>>>> * In and notIn expressions
>>>> * An InputFormat for writing MR jobs
>>>>
>>>> Please download, verify, and test.
>>>>
>>>> Please vote in the next 72 hours.
>>>>
>>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>>> [ ] +0
>>>> [ ] -1 Do not release this because...
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>>
>>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>
--
Ryan Blue
Software Engineer
Netflix
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by parth brahmbhatt <br...@gmail.com>.
+1 once we apply the patch.
The patch works with presto but it breaks iceberg build. We need to add a
typecast to FileIO in TableMetadataParserTest here
<https://github.com/Parth-Brahmbhatt/incubator-iceberg/blob/presto-patch/core/src/test/java/org/apache/iceberg/TableMetadataParserTest.java#L78>
.
Thanks
Parth
On Wed, Apr 29, 2020 at 2:19 PM Ryan Blue <rb...@netflix.com.invalid> wrote:
> Thanks, James. Is it possible for you to test with this patch?
> https://patch-diff.githubusercontent.com/raw/apache/incubator-iceberg/pull/986.patch
>
> Those methods were removed, and it should be easy for Presto to update.
> That said, I don't like breaking downstream projects so it would be nice to
> fix it. I think this patch might.
>
> On Wed, Apr 29, 2020 at 1:12 PM James Taylor <ja...@apache.org>
> wrote:
>
>> Verified signature, checksum, rat checks, build, and unit tests. All
>> looked good. Tried building latest prestosql master with proposed new
>> iceberg release, but received these compilation errors:
>>
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile
>> (default-compile) on project presto-iceberg: Compilation failure:
>> Compilation failure:
>> [ERROR]
>> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/IcebergMetadata.java:[359,34]
>> method newTableMetadata in class org.apache.iceberg.TableMetadata cannot be
>> applied to given types;
>> [ERROR] required:
>> org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,java.util.Map<java.lang.String,java.lang.String>
>> [ERROR] found:
>> org.apache.iceberg.TableOperations,org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,com.google.common.collect.ImmutableMap<java.lang.String,java.lang.String>
>> [ERROR] reason: actual and formal argument lists differ in length
>> [ERROR]
>> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/HiveTableOperations.java:[301,44]
>> no suitable method found for
>> read(io.prestosql.plugin.iceberg.HiveTableOperations,org.apache.iceberg.io.InputFile)
>> [ERROR] method
>> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,java.lang.String)
>> is not applicable
>> [ERROR] (argument mismatch;
>> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
>> org.apache.iceberg.io.FileIO)
>> [ERROR] method
>> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,org.apache.iceberg.io.InputFile)
>> is not applicable
>> [ERROR] (argument mismatch;
>> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
>> org.apache.iceberg.io.FileIO)
>>
>> Is this a showstopper? Probably a simple PR in presto would fix it, but
>> unclear if b/w compatibility is a goal or not.
>>
>> Thanks,
>> James
>>
>> On Tue, Apr 28, 2020 at 1:40 PM Ryan Blue <bl...@apache.org> wrote:
>>
>>> Hi everyone,
>>>
>>> I propose the following RC to be released as official Apache Iceberg
>>> 0.8.0-incubating release.
>>>
>>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>>> *
>>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>>
>>> The release tarball, signature, and checksums are here:
>>> *
>>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>>
>>> You can find the KEYS file here:
>>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>>
>>> Convenience binary artifacts are staged in Nexus. The Maven repository
>>> URL is:
>>> *
>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>>
>>> This release contains many bug fixes and several new features:
>>> * Actions to remove orphaned files and to optimize metadata for query
>>> performance
>>> * Support for ORC data files
>>> * Snapshot cherry-picking
>>> * Incremental scan planning based on table history
>>> * In and notIn expressions
>>> * An InputFormat for writing MR jobs
>>>
>>> Please download, verify, and test.
>>>
>>> Please vote in the next 72 hours.
>>>
>>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>>> [ ] +0
>>> [ ] -1 Do not release this because...
>>>
>>>
>>> --
>>> Ryan Blue
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Thanks, James. Is it possible for you to test with this patch?
https://patch-diff.githubusercontent.com/raw/apache/incubator-iceberg/pull/986.patch
Those methods were removed, and it should be easy for Presto to update.
That said, I don't like breaking downstream projects so it would be nice to
fix it. I think this patch might.
On Wed, Apr 29, 2020 at 1:12 PM James Taylor <ja...@apache.org> wrote:
> Verified signature, checksum, rat checks, build, and unit tests. All
> looked good. Tried building latest prestosql master with proposed new
> iceberg release, but received these compilation errors:
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile
> (default-compile) on project presto-iceberg: Compilation failure:
> Compilation failure:
> [ERROR]
> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/IcebergMetadata.java:[359,34]
> method newTableMetadata in class org.apache.iceberg.TableMetadata cannot be
> applied to given types;
> [ERROR] required:
> org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,java.util.Map<java.lang.String,java.lang.String>
> [ERROR] found:
> org.apache.iceberg.TableOperations,org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,com.google.common.collect.ImmutableMap<java.lang.String,java.lang.String>
> [ERROR] reason: actual and formal argument lists differ in length
> [ERROR]
> /Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/HiveTableOperations.java:[301,44]
> no suitable method found for
> read(io.prestosql.plugin.iceberg.HiveTableOperations,org.apache.iceberg.io.InputFile)
> [ERROR] method
> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,java.lang.String)
> is not applicable
> [ERROR] (argument mismatch;
> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
> org.apache.iceberg.io.FileIO)
> [ERROR] method
> org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,org.apache.iceberg.io.InputFile)
> is not applicable
> [ERROR] (argument mismatch;
> io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
> org.apache.iceberg.io.FileIO)
>
> Is this a showstopper? Probably a simple PR in presto would fix it, but
> unclear if b/w compatibility is a goal or not.
>
> Thanks,
> James
>
> On Tue, Apr 28, 2020 at 1:40 PM Ryan Blue <bl...@apache.org> wrote:
>
>> Hi everyone,
>>
>> I propose the following RC to be released as official Apache Iceberg
>> 0.8.0-incubating release.
>>
>> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
>> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
>> *
>> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
>> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>>
>> The release tarball, signature, and checksums are here:
>> *
>> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>>
>> You can find the KEYS file here:
>> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>>
>> Convenience binary artifacts are staged in Nexus. The Maven repository
>> URL is:
>> *
>> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>>
>> This release contains many bug fixes and several new features:
>> * Actions to remove orphaned files and to optimize metadata for query
>> performance
>> * Support for ORC data files
>> * Snapshot cherry-picking
>> * Incremental scan planning based on table history
>> * In and notIn expressions
>> * An InputFormat for writing MR jobs
>>
>> Please download, verify, and test.
>>
>> Please vote in the next 72 hours.
>>
>> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
>> [ ] +0
>> [ ] -1 Do not release this because...
>>
>>
>> --
>> Ryan Blue
>>
>
--
Ryan Blue
Software Engineer
Netflix
Re: [VOTE] Release Apache Iceberg 0.8.0-incubating RC1
Posted by James Taylor <ja...@apache.org>.
Verified signature, checksum, rat checks, build, and unit tests. All looked
good. Tried building latest prestosql master with proposed new iceberg
release, but received these compilation errors:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile
(default-compile) on project presto-iceberg: Compilation failure:
Compilation failure:
[ERROR]
/Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/IcebergMetadata.java:[359,34]
method newTableMetadata in class org.apache.iceberg.TableMetadata cannot be
applied to given types;
[ERROR] required:
org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,java.util.Map<java.lang.String,java.lang.String>
[ERROR] found:
org.apache.iceberg.TableOperations,org.apache.iceberg.Schema,org.apache.iceberg.PartitionSpec,java.lang.String,com.google.common.collect.ImmutableMap<java.lang.String,java.lang.String>
[ERROR] reason: actual and formal argument lists differ in length
[ERROR]
/Users/jamestaylor/dev/presto/presto-iceberg/src/main/java/io/prestosql/plugin/iceberg/HiveTableOperations.java:[301,44]
no suitable method found for
read(io.prestosql.plugin.iceberg.HiveTableOperations,org.apache.iceberg.io.InputFile)
[ERROR] method
org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,java.lang.String)
is not applicable
[ERROR] (argument mismatch;
io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
org.apache.iceberg.io.FileIO)
[ERROR] method
org.apache.iceberg.TableMetadataParser.read(org.apache.iceberg.io.FileIO,org.apache.iceberg.io.InputFile)
is not applicable
[ERROR] (argument mismatch;
io.prestosql.plugin.iceberg.HiveTableOperations cannot be converted to
org.apache.iceberg.io.FileIO)
Is this a showstopper? Probably a simple PR in presto would fix it, but
unclear if b/w compatibility is a goal or not.
Thanks,
James
On Tue, Apr 28, 2020 at 1:40 PM Ryan Blue <bl...@apache.org> wrote:
> Hi everyone,
>
> I propose the following RC to be released as official Apache Iceberg
> 0.8.0-incubating release.
>
> The commit id is 4c2dd0ac2c832cc425b33d3b578025fa4e295392
> * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc1
> *
> https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc1
> * https://github.com/apache/incubator-iceberg/tree/4c2dd0ac
>
> The release tarball, signature, and checksums are here:
> *
> https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc1/
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS
>
> Convenience binary artifacts are staged in Nexus. The Maven repository URL
> is:
> *
> https://repository.apache.org/content/repositories/orgapacheiceberg-1005/
>
> This release contains many bug fixes and several new features:
> * Actions to remove orphaned files and to optimize metadata for query
> performance
> * Support for ORC data files
> * Snapshot cherry-picking
> * Incremental scan planning based on table history
> * In and notIn expressions
> * An InputFormat for writing MR jobs
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Iceberg 0.8.0-incubating
> [ ] +0
> [ ] -1 Do not release this because...
>
>
> --
> Ryan Blue
>