You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Felix Cheung <fe...@hotmail.com> on 2017/11/08 23:57:43 UTC

Cutting the RC for Spark 2.2.1 release

Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1



Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
Thanks Dongjoon! I will track that.

________________________________
From: Dongjoon Hyun <do...@gmail.com>
Sent: Wednesday, November 8, 2017 7:41:20 PM
To: Holden Karau
Cc: Felix Cheung; dev@spark.apache.org
Subject: Re: Cutting the RC for Spark 2.2.1 release

It's great, Felix!

As of today, `branch-2.2` seems to be broken due to SPARK-22211 (Scala UT failure) and SPARK-22417 (Python UT failure).
I pinged you at both.

Bests,
Dongjoon.


On Wed, Nov 8, 2017 at 5:51 PM, Holden Karau <ho...@pigscanfly.ca>> wrote:
Thanks for stepping up and running the 2.2.1 release :)

On Wed, Nov 8, 2017 at 3:57 PM Felix Cheung <fe...@hotmail.com>> wrote:
Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1


--
Twitter: https://twitter.com/holdenkarau


Re: Cutting the RC for Spark 2.2.1 release

Posted by Dongjoon Hyun <do...@gmail.com>.
It's great, Felix!

As of today, `branch-2.2` seems to be broken due to SPARK-22211 (Scala UT
failure) and SPARK-22417 (Python UT failure).
I pinged you at both.

Bests,
Dongjoon.


On Wed, Nov 8, 2017 at 5:51 PM, Holden Karau <ho...@pigscanfly.ca> wrote:

> Thanks for stepping up and running the 2.2.1 release :)
>
> On Wed, Nov 8, 2017 at 3:57 PM Felix Cheung <fe...@hotmail.com>
> wrote:
>
>> Hi!
>>
>> As we are closing down on the few known issues I think we are ready to
>> tag and cut the 2.2.1 release.
>>
>> If you are aware of any issue that you think should go into this release
>> please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be
>> scrubbing JIRA in the next few days.
>>
>> So unless we hear otherwise, I’m going to tag and build the RC starting
>> Saturday EOD (PT). Please be patient since I’m going to be new at this :)
>> but will keep the dev@ posted for any update.
>>
>> Yours
>> RM for 2.2.1
>>
>>
>> --
> Twitter: https://twitter.com/holdenkarau
>

Re: Cutting the RC for Spark 2.2.1 release

Posted by Holden Karau <ho...@pigscanfly.ca>.
Thanks for stepping up and running the 2.2.1 release :)

On Wed, Nov 8, 2017 at 3:57 PM Felix Cheung <fe...@hotmail.com>
wrote:

> Hi!
>
> As we are closing down on the few known issues I think we are ready to tag
> and cut the 2.2.1 release.
>
> If you are aware of any issue that you think should go into this release
> please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be
> scrubbing JIRA in the next few days.
>
> So unless we hear otherwise, I’m going to tag and build the RC starting
> Saturday EOD (PT). Please be patient since I’m going to be new at this :)
> but will keep the dev@ posted for any update.
>
> Yours
> RM for 2.2.1
>
>
> --
Twitter: https://twitter.com/holdenkarau

Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
Now I’m seeing an error with Closing nexus staging repository.

staged_repo_id=orgapachespark-1254

< HTTP/1.1 401 Unauthorized
< Date: Tue, 14 Nov 2017 12:32:57 GMT
< Server: Nexus/2.13.0-01
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
* Authentication problem. Ignoring this.
< WWW-Authenticate: BASIC realm="Sonatype Nexus Repository Manager API"
< Content-Length: 0
< Via: 1.1 repository.apache.org<http://repository.apache.org>

Does working with Nexus require special permission in LDAP?
I couldn’t login to the web interface at repository.apache.org either.
________________________________
From: Felix Cheung <fe...@hotmail.com>
Sent: Monday, November 13, 2017 11:23:44 AM
To: Sean Owen
Cc: Holden Karau; dev@spark.apache.org
Subject: Re: Cutting the RC for Spark 2.2.1 release

Ouch ;) yes that works and RC1 is tagged.


________________________________
From: Sean Owen <so...@cloudera.com>
Sent: Monday, November 13, 2017 10:54:48 AM
To: Felix Cheung
Cc: Holden Karau; dev@spark.apache.org
Subject: Re: Cutting the RC for Spark 2.2.1 release

It's repo.maven.apache.org<http://repo.maven.apache.org> ?

On Mon, Nov 13, 2017 at 12:52 PM Felix Cheung <fe...@hotmail.com>> wrote:
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org<http://repo.maven.org>: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org<http://repo.maven.org>: Name or service not known -> [Help 2]



Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
Ouch ;) yes that works and RC1 is tagged.


________________________________
From: Sean Owen <so...@cloudera.com>
Sent: Monday, November 13, 2017 10:54:48 AM
To: Felix Cheung
Cc: Holden Karau; dev@spark.apache.org
Subject: Re: Cutting the RC for Spark 2.2.1 release

It's repo.maven.apache.org<http://repo.maven.apache.org> ?

On Mon, Nov 13, 2017 at 12:52 PM Felix Cheung <fe...@hotmail.com>> wrote:
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org<http://repo.maven.org>: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org<http://repo.maven.org>: Name or service not known -> [Help 2]



Re: Cutting the RC for Spark 2.2.1 release

Posted by Sean Owen <so...@cloudera.com>.
It's repo.maven.apache.org ?

On Mon, Nov 13, 2017 at 12:52 PM Felix Cheung <fe...@hotmail.com>
wrote:

> I did change it, but getting unknown host?
>
> [ERROR] Non-resolvable parent POM for
> org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer
> artifact org.apache:apache:pom:14 from/to central (
> https://repo.maven.org/maven2): repo.maven.org: Name or service not known
> and 'parent.relativePath' points at wrong local POM @ line 22, column 11:
> Unknown host repo.maven.org: Name or service not known -> [Help 2]
>
>
>

Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org<http://repo.maven.org>: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org<http://repo.maven.org>: Name or service not known -> [Help 2]

_____________________________
From: Sean Owen <so...@cloudera.com>>
Sent: Monday, November 13, 2017 10:48 AM
Subject: Re: Cutting the RC for Spark 2.2.1 release
To: Felix Cheung <fe...@hotmail.com>>
Cc: Holden Karau <ho...@pigscanfly.ca>>, <de...@spark.apache.org>>


I'm not seeing a problem building, myself. However we could change the location of the Maven Repository in our POM to https://repo.maven.apache.org/maven2/ without any consequence.

The only reason we overrode it was to force it to use HTTPS which still doesn't look like the default (!): https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM

On a related note, we could also update the POM to inherit from the latest Apache parent POM, while we're at it, to get the latest declarations relevant to the ASF. Doesn't need to happen in 2.2.x

On Mon, Nov 13, 2017 at 12:39 PM Felix Cheung <fe...@hotmail.com>> wrote:
Anything to build with maven on a clean machine.
It couldn’t connect to maven central repo.




Re: Cutting the RC for Spark 2.2.1 release

Posted by Sean Owen <so...@cloudera.com>.
I'm not seeing a problem building, myself. However we could change the
location of the Maven Repository in our POM to
https://repo.maven.apache.org/maven2/ without any consequence.

The only reason we overrode it was to force it to use HTTPS which still
doesn't look like the default (!):
https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM

On a related note, we could also update the POM to inherit from the latest
Apache parent POM, while we're at it, to get the latest declarations
relevant to the ASF. Doesn't need to happen in 2.2.x

On Mon, Nov 13, 2017 at 12:39 PM Felix Cheung <fe...@hotmail.com>
wrote:

> Anything to build with maven on a clean machine.
> It couldn’t connect to maven central repo.
>
>

Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
Anything to build with maven on a clean machine.
It couldn’t connect to maven central repo.

________________________________
From: Holden Karau <ho...@pigscanfly.ca>
Sent: Monday, November 13, 2017 10:38:03 AM
To: Felix Cheung
Cc: dev@spark.apache.org
Subject: Re: Cutting the RC for Spark 2.2.1 release

Which script is this from?

On Mon, Nov 13, 2017 at 10:37 AM Felix Cheung <fe...@hotmail.com>> wrote:
Build/test looks good but I’m hitting a new issue with sonatype when tagging

"Host name 'repo1.maven.org<http://repo1.maven.org>' does not match the certificate subject provided by the peer (CN=repo.maven.apache.org<http://repo.maven.apache.org>, O="Sonatype, Inc", L=Fulton, ST=MD, C=US)"

https://issues.sonatype.org/browse/MVNCENTRAL-1369

Stay tuned.

________________________________
From: Felix Cheung <fe...@hotmail.com>>
Sent: Monday, November 13, 2017 12:00:41 AM
To: dev@spark.apache.org<ma...@spark.apache.org>
Subject: Re: Cutting the RC for Spark 2.2.1 release

Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.

_____________________________
From: Felix Cheung <fe...@hotmail.com>>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <de...@spark.apache.org>>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1




--
Twitter: https://twitter.com/holdenkarau

Re: Cutting the RC for Spark 2.2.1 release

Posted by Holden Karau <ho...@pigscanfly.ca>.
Which script is this from?

On Mon, Nov 13, 2017 at 10:37 AM Felix Cheung <fe...@hotmail.com>
wrote:

> Build/test looks good but I’m hitting a new issue with sonatype when
> tagging
>
> "Host name 'repo1.maven.org' does not match the certificate subject
> provided by the peer (CN=repo.maven.apache.org, O="Sonatype, Inc",
> L=Fulton, ST=MD, C=US)"
>
> https://issues.sonatype.org/browse/MVNCENTRAL-1369
>
> Stay tuned.
>
> ------------------------------
> *From:* Felix Cheung <fe...@hotmail.com>
> *Sent:* Monday, November 13, 2017 12:00:41 AM
> *To:* dev@spark.apache.org
> *Subject:* Re: Cutting the RC for Spark 2.2.1 release
>
> Quick update:
>
> We merged 6 fixes Friday and 7 fixes today (thanks!), since some are
> hand-merged I’m waiting for clean builds from Jenkins and test passes. As
> of now it looks like we need to take one more fix for Scala 2.10.
>
> With any luck we should be tagging for build tomorrow morning (PT).
>
> There should not be any issue targetting 2.2.1 except for SPARK-22042. As
> it is not a regression and it seems it might take a while, we won’t be
> blocking the release.
>
> _____________________________
> From: Felix Cheung <fe...@hotmail.com>
> Sent: Wednesday, November 8, 2017 3:57 PM
> Subject: Cutting the RC for Spark 2.2.1 release
> To: <de...@spark.apache.org>
>
>
> Hi!
>
> As we are closing down on the few known issues I think we are ready to tag
> and cut the 2.2.1 release.
>
> If you are aware of any issue that you think should go into this release
> please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be
> scrubbing JIRA in the next few days.
>
> So unless we hear otherwise, I’m going to tag and build the RC starting
> Saturday EOD (PT). Please be patient since I’m going to be new at this :)
> but will keep the dev@ posted for any update.
>
> Yours
> RM for 2.2.1
>
>
>
>
> --
Twitter: https://twitter.com/holdenkarau

Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
Build/test looks good but I’m hitting a new issue with sonatype when tagging

"Host name 'repo1.maven.org' does not match the certificate subject provided by the peer (CN=repo.maven.apache.org, O="Sonatype, Inc", L=Fulton, ST=MD, C=US)"

https://issues.sonatype.org/browse/MVNCENTRAL-1369

Stay tuned.

________________________________
From: Felix Cheung <fe...@hotmail.com>
Sent: Monday, November 13, 2017 12:00:41 AM
To: dev@spark.apache.org
Subject: Re: Cutting the RC for Spark 2.2.1 release

Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.

_____________________________
From: Felix Cheung <fe...@hotmail.com>>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <de...@spark.apache.org>>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1





Re: Cutting the RC for Spark 2.2.1 release

Posted by Felix Cheung <fe...@hotmail.com>.
Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.

_____________________________
From: Felix Cheung <fe...@hotmail.com>>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <de...@spark.apache.org>>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1