You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Hyukjin Kwon <gu...@gmail.com> on 2021/04/14 04:42:08 UTC
[PSA] Please read: PR builder now runs test and build in your forked repository
Hi all,
After https://github.com/apache/spark/pull/32092 merged, now we run the
GitHub Actions
workflows in your forked repository.
In short, please see this example HyukjinKwon#34
<https://github.com/HyukjinKwon/spark/pull/34>
1. You create a PR and your repository triggers the workflow. Your PR
uses the resources allocated to you for testing.
2. Apache Spark repository finds your workflow, and links it in a
comment in your PR
Please let me know if you guys find any weird behaviour related to this.
*What does that mean to contributors?*
Please sync your branch to the latest master branch in Apache Spark in
order for the main repository to run the workflow and detect it.
*What does that mean to committers?*
Now, GitHub Actions will show a green even when GitHub Actions builds are
running (in contributor's forked repository). Please check the build
notified by github-actions bot before merging it.
There would be a followup work to reflect the status of the forked
repository's build to
the status of PR.
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
Hi all,
Currently only the PR author can retrigger the workflow run to test the PR.
It's impossible for other people, even committers, to re-run
the tests in PRs because the workflow runs in the forked repository. So we
should ask the authors to retrigger the tests.
Thanks Dongjoon for pointing this out.
I have two suggestions for that but I need some help:
1. One idea (thanks Gengliang) is for GitHub Actions workflows to check the
comments in PR, and automatically retrigger the runs in the forked
repository.
I managed to come up with how to implement this, and described it at
https://issues.apache.org/jira/browse/SPARK-35157.
2. I think it will take a while to implement 1, approach . For the time
being, we should probably at least add more words at here
<https://github.com/apache/spark/blob/master/.github/workflows/notify_test_workflow.yml#L110>
saying
the authors something
like they have to fix the changes and/or retrigger the workflow if it
fails, which will be shown here:
[image: Screen Shot 2021-04-21 at 10.04.05 AM.png]
I filed a JIRA for that here:
https://issues.apache.org/jira/browse/SPARK-35158
I will be unavailable for a few weeks because of a bunch of work being
delayed .. It would be great if someone who's interested in GitHub Actions
takes a stab for it.
Thank you guys!
2021년 4월 18일 (일) 오전 11:35, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> The PR is merged. The status is now supposed to be reported properly as a
> GitHub check instead of relying on a comment.
> Please let me know if there's any suspicious behavior
>
> 2021년 4월 16일 (금) 오전 10:38, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>
>> The PR is ready for a review now:
>> https://github.com/apache/spark/pull/32193.
>> I would appreciate any reviews on this ..
>>
>> 2021년 4월 16일 (금) 오전 1:31, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>
>>> FYI, I made a POC PR to integrate it with GitHub Actions status (instead
>>> of a comment): https://github.com/apache/spark/pull/32193.
>>> This is just a minimal working version. Please take a look when you guys
>>> find some time.
>>>
>>> 2021년 4월 15일 (목) 오후 11:12, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>
>>>> I think we should probably leverage "action required" that allows us
>>>> to link URLs with some guides (see
>>>> https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites)
>>>> to inform users to enable it.
>>>>
>>>> 2021년 4월 15일 (목) 오후 1:26, Kent Yao <ya...@gmail.com>님이 작성:
>>>>
>>>>>
>>>>>
>>>>> I quick tested it by fork another apache project which contains GA
>>>>> workflows. The above pic shows what the action page looks like at the very
>>>>> beginning.
>>>>>
>>>>> Need to hit the green button to activate it
>>>>>
>>>>>
>>>>>
>>>>> *Kent Yao *
>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>> *a spark enthusiast*
>>>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>>>> unified multi-tenant JDBC interface for large-scale data processing and
>>>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A
>>>>> Spark SQL extension which provides SQL Standard Authorization for **Apache
>>>>> Spark <http://spark.apache.org/>.*
>>>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>>>> for reading data from and transferring data to Postgres / Greenplum with
>>>>> Spark SQL and DataFrames, 10~100x faster.*
>>>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>>>>> brings useful functions from various modern database management systems to **Apache
>>>>> Spark <http://spark.apache.org/>.*
>>>>>
>>>>>
>>>>>
>>>>> On 04/15/2021 12:17，Hyukjin Kwon<gu...@gmail.com>
>>>>> <gu...@gmail.com> wrote：
>>>>>
>>>>> I remember it's turned on by default (?). If not, yeah we should
>>>>> document.
>>>>>
>>>>> 2021년 4월 15일 (목) 오후 1:14, Kent Yao <ya...@gmail.com>님이 작성:
>>>>>
>>>>>> Thanks Hyukjin and Yikun,
>>>>>>
>>>>>> > 2. New Forks have to turn on GitHub action by the fork owner
>>>>>> manually
>>>>>>
>>>>>> And we may still need a suitable place to make this note clearer to
>>>>>> new contributors or someone delete and re-fork their forked repo.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> *Kent Yao *
>>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>> *a spark enthusiast*
>>>>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>>>>> unified multi-tenant JDBC interface for large-scale data processing and
>>>>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>>>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A
>>>>>> Spark SQL extension which provides SQL Standard Authorization for **Apache
>>>>>> Spark <http://spark.apache.org/>.*
>>>>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A
>>>>>> library for reading data from and transferring data to Postgres / Greenplum
>>>>>> with Spark SQL and DataFrames, 10~100x faster.*
>>>>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A**
>>>>>> library t**hat brings useful functions from various modern database
>>>>>> management systems to **Apache Spark <http://spark.apache.org/>.*
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 04/15/2021 12:09，Hyukjin Kwon<gu...@gmail.com>
>>>>>> <gu...@gmail.com> wrote：
>>>>>>
>>>>>> The issue is fixed now. Please keep monitoring this. Thank you all!
>>>>>> The spark community is super active and cooperative!
>>>>>>
>>>>>> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>>
>>>>>>> The fix will be straightforward. We can either, in Github Actions
>>>>>>> workflow,:
>>>>>>> - remove fast forward option and see if ti works
>>>>>>> - or git rebase before merge the branch
>>>>>>>
>>>>>>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>>>
>>>>>>>> I think it works mostly correctly as Dongjoon investigated and
>>>>>>>> shared (Thanks a lot!).
>>>>>>>> One problem seems to be syncing to the master seems too strict (
>>>>>>>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>>>>>>>> Thanks Yikun.
>>>>>>>> I think we should make it less strict. I can create a PR right away
>>>>>>>> but would like to encourage Yikun or Kent to do it in order to keep the
>>>>>>>> credits of their investigation.
>>>>>>>>
>>>>>>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이
>>>>>>>> 작성:
>>>>>>>>
>>>>>>>>> Hi, Kent.
>>>>>>>>>
>>>>>>>>> I checked (1) in your PR, but those test result comments look
>>>>>>>>> correct to me.
>>>>>>>>> Please note that both Jenkins and GitHub Action leave the same
>>>>>>>>> number of comments on the same GitHash.
>>>>>>>>> Given that, there are not fake comments. It looks like a real
>>>>>>>>> result of your commits on that PR.
>>>>>>>>>
>>>>>>>>> GitHash: 23248c3
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>>> (GitHub Action)
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>>>>>>>> (Jenkins)
>>>>>>>>>
>>>>>>>>> GitHash: 8dbed7b
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>>>>>>>> (GitHub Action)
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>>>>>>>> (Jenkins)
>>>>>>>>>
>>>>>>>>> GitHash: a3a6c5e
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>>>>>>>> (GitHub Action)
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>>>>>>>> (Jenkins)
>>>>>>>>>
>>>>>>>>> GitHash: b6d26b7
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>>>>>>>> (GitHub Action)
>>>>>>>>>
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>>>>>>>> (Jenkins)
>>>>>>>>>
>>>>>>>>> Could you recheck it?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. Github-actions notification could be wrong when another PR
>>>>>>>>>> opened
>>>>>>>>>> with some same commits, and you will get a lot of fake comments
>>>>>>>>>> then.
>>>>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>>>>> chosen one.
>>>>>>>>>> 1.1
>>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi ALL, here is something I notice after this change:
>>>>>>>>>>
>>>>>>>>>> 1. Github-actions notification could be wrong when another PR
>>>>>>>>>> opened
>>>>>>>>>> with some same commits, and you will get a lot of fake comments
>>>>>>>>>> then.
>>>>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>>>>> chosen one.
>>>>>>>>>> 1.1
>>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>>>> 2. New Forks have to turn on GitHub action by the fork owner
>>>>>>>>>> manually
>>>>>>>>>> 3. `Notify test workflow` keeps waiting when the build flow
>>>>>>>>>> canceled
>>>>>>>>>> or the whole fork gone
>>>>>>>>>> 4. After refreshed master or even re-forked :(, I still got
>>>>>>>>>> failures
>>>>>>>>>> and seems not alone
>>>>>>>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after
>>>>>>>>>> sync)
>>>>>>>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after
>>>>>>>>>> re-forked)
>>>>>>>>>> 4.3.
>>>>>>>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>>>>>>>> (some other failures noticed)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Bests,
>>>>>>>>>>
>>>>>>>>>> Kent
>>>>>>>>>>
>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>>>>>>>> >
>>>>>>>>>> > Thank you again, Hyukjin.
>>>>>>>>>> >
>>>>>>>>>> > Bests,
>>>>>>>>>> > Dongjoon.
>>>>>>>>>> >
>>>>>>>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> >>
>>>>>>>>>> >> Cool, thanks!
>>>>>>>>>> >>
>>>>>>>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>>>>>>>> >>>
>>>>>>>>>> >>> Good point! I had to clarify.
>>>>>>>>>> >>> Once is enough. The sync is needed for your branch to include
>>>>>>>>>> the changes of https://github.com/apache/spark/pull/32092.
>>>>>>>>>> >>>
>>>>>>>>>> >>>
>>>>>>>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Hi Hyukjin,
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> > Please sync your branch to the latest master branch in
>>>>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>>>>> detect it.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Do we need to sync master for every PR or just one-time cost
>>>>>>>>>> to keep up with the current master branch？
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Kent Yao
>>>>>>>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase
>>>>>>>>>> Corp.
>>>>>>>>>> >>>> a spark enthusiast
>>>>>>>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for
>>>>>>>>>> large-scale data processing and analytics, built on top of Apache Spark.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL
>>>>>>>>>> Standard Authorization for Apache Spark.
>>>>>>>>>> >>>> spark-postgres A library for reading data from and
>>>>>>>>>> transferring data to Postgres / Greenplum with Spark SQL and DataFrames,
>>>>>>>>>> 10~100x faster.
>>>>>>>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>>>>>>>> functions from various modern database management systems to Apache Spark.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>>
>>>>>>>>>> >>>>
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Cool~Thanks, Hyukjin
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>>>>>>> >>>>>>
>>>>>>>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>>>>>>>> >>>>>>
>>>>>>>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <
>>>>>>>>>> ltnwgl@gmail.com> wrote:
>>>>>>>>>> >>>>>>>
>>>>>>>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>>>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>>>>>>>> https://github.com/apache/spark/pull/32158
>>>>>>>>>> >>>>>>>
>>>>>>>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> Hi all,
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092
>>>>>>>>>> merged, now we run the GitHub Actions
>>>>>>>>>> >>>>>>>> workflows in your forked repository.
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> You create a PR and your repository triggers the
>>>>>>>>>> workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>>>> >>>>>>>> Apache Spark repository finds your workflow, and links
>>>>>>>>>> it in a comment in your PR
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>>>>> related to this.
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> What does that mean to contributors?
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> Please sync your branch to the latest master branch in
>>>>>>>>>> Apache Spark in order for your forked repository to run the workflow, and
>>>>>>>>>> >>>>>>>> for the main repository to detect the workflow.
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> What does that mean to committers?
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>>>>> Actions builds are running (in contributor's forked repository).
>>>>>>>>>> >>>>>>>> Please check the build notified by github-actions bot
>>>>>>>>>> before merging it.
>>>>>>>>>> >>>>>>>> There would be a followup work to reflect the status of
>>>>>>>>>> the forked repository's build to the status of PR.
>>>>>>>>>> >>>>>>>>
>>>>>>>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <
>>>>>>>>>> gurwls223@gmail.com>님이 작성:
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> Hi all,
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092
>>>>>>>>>> merged, now we run the GitHub Actions
>>>>>>>>>> >>>>>>>>> workflows in your forked repository.
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> You create a PR and your repository triggers the
>>>>>>>>>> workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links
>>>>>>>>>> it in a comment in your PR
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>>>>> related to this.
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> What does that mean to contributors?
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> Please sync your branch to the latest master branch in
>>>>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>>>>> detect it.
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> What does that mean to committers?
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>>>>> Actions builds are running (in contributor's forked repository). Please
>>>>>>>>>> check the build notified by github-actions bot before merging it.
>>>>>>>>>> >>>>>>>>> There would be a followup work to reflect the status of
>>>>>>>>>> the forked repository's build to
>>>>>>>>>> >>>>>>>>> the status of PR.
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>>>>>>>>
>>>>>>>>>> >>
>>>>>>>>>> >>
>>>>>>>>>> >> --
>>>>>>>>>> >>
>>>>>>>>>> >>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Kent Yao
>>>>>>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>>>>>> a spark enthusiast
>>>>>>>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale
>>>>>>>>>> data
>>>>>>>>>> processing and analytics, built on top of Apache Spark.
>>>>>>>>>>
>>>>>>>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>>>>>>> Authorization for Apache Spark.
>>>>>>>>>> spark-postgres A library for reading data from and transferring
>>>>>>>>>> data
>>>>>>>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x
>>>>>>>>>> faster.
>>>>>>>>>> itatchiA library that brings useful functions from various modern
>>>>>>>>>> database management systems to Apache Spark.
>>>>>>>>>>
>>>>>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
The PR is merged. The status is now supposed to be reported properly as a
GitHub check instead of relying on a comment.
Please let me know if there's any suspicious behavior
2021년 4월 16일 (금) 오전 10:38, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> The PR is ready for a review now:
> https://github.com/apache/spark/pull/32193.
> I would appreciate any reviews on this ..
>
> 2021년 4월 16일 (금) 오전 1:31, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>
>> FYI, I made a POC PR to integrate it with GitHub Actions status (instead
>> of a comment): https://github.com/apache/spark/pull/32193.
>> This is just a minimal working version. Please take a look when you guys
>> find some time.
>>
>> 2021년 4월 15일 (목) 오후 11:12, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>
>>> I think we should probably leverage "action required" that allows us
>>> to link URLs with some guides (see
>>> https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites)
>>> to inform users to enable it.
>>>
>>> 2021년 4월 15일 (목) 오후 1:26, Kent Yao <ya...@gmail.com>님이 작성:
>>>
>>>>
>>>>
>>>> I quick tested it by fork another apache project which contains GA
>>>> workflows. The above pic shows what the action page looks like at the very
>>>> beginning.
>>>>
>>>> Need to hit the green button to activate it
>>>>
>>>>
>>>>
>>>> *Kent Yao *
>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>> *a spark enthusiast*
>>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>>> unified multi-tenant JDBC interface for large-scale data processing and
>>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>>>> SQL extension which provides SQL Standard Authorization for **Apache
>>>> Spark <http://spark.apache.org/>.*
>>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>>> for reading data from and transferring data to Postgres / Greenplum with
>>>> Spark SQL and DataFrames, 10~100x faster.*
>>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>>>> brings useful functions from various modern database management systems to **Apache
>>>> Spark <http://spark.apache.org/>.*
>>>>
>>>>
>>>>
>>>> On 04/15/2021 12:17，Hyukjin Kwon<gu...@gmail.com>
>>>> <gu...@gmail.com> wrote：
>>>>
>>>> I remember it's turned on by default (?). If not, yeah we should
>>>> document.
>>>>
>>>> 2021년 4월 15일 (목) 오후 1:14, Kent Yao <ya...@gmail.com>님이 작성:
>>>>
>>>>> Thanks Hyukjin and Yikun,
>>>>>
>>>>> > 2. New Forks have to turn on GitHub action by the fork owner manually
>>>>>
>>>>> And we may still need a suitable place to make this note clearer to
>>>>> new contributors or someone delete and re-fork their forked repo.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> *Kent Yao *
>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>> *a spark enthusiast*
>>>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>>>> unified multi-tenant JDBC interface for large-scale data processing and
>>>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A
>>>>> Spark SQL extension which provides SQL Standard Authorization for **Apache
>>>>> Spark <http://spark.apache.org/>.*
>>>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>>>> for reading data from and transferring data to Postgres / Greenplum with
>>>>> Spark SQL and DataFrames, 10~100x faster.*
>>>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>>>>> brings useful functions from various modern database management systems to **Apache
>>>>> Spark <http://spark.apache.org/>.*
>>>>>
>>>>>
>>>>>
>>>>> On 04/15/2021 12:09，Hyukjin Kwon<gu...@gmail.com>
>>>>> <gu...@gmail.com> wrote：
>>>>>
>>>>> The issue is fixed now. Please keep monitoring this. Thank you all!
>>>>> The spark community is super active and cooperative!
>>>>>
>>>>> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>
>>>>>> The fix will be straightforward. We can either, in Github Actions
>>>>>> workflow,:
>>>>>> - remove fast forward option and see if ti works
>>>>>> - or git rebase before merge the branch
>>>>>>
>>>>>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>>
>>>>>>> I think it works mostly correctly as Dongjoon investigated and
>>>>>>> shared (Thanks a lot!).
>>>>>>> One problem seems to be syncing to the master seems too strict (
>>>>>>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>>>>>>> Thanks Yikun.
>>>>>>> I think we should make it less strict. I can create a PR right away
>>>>>>> but would like to encourage Yikun or Kent to do it in order to keep the
>>>>>>> credits of their investigation.
>>>>>>>
>>>>>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이
>>>>>>> 작성:
>>>>>>>
>>>>>>>> Hi, Kent.
>>>>>>>>
>>>>>>>> I checked (1) in your PR, but those test result comments look
>>>>>>>> correct to me.
>>>>>>>> Please note that both Jenkins and GitHub Action leave the same
>>>>>>>> number of comments on the same GitHash.
>>>>>>>> Given that, there are not fake comments. It looks like a real
>>>>>>>> result of your commits on that PR.
>>>>>>>>
>>>>>>>> GitHash: 23248c3
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>> (GitHub Action)
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>>>>>>> (Jenkins)
>>>>>>>>
>>>>>>>> GitHash: 8dbed7b
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>>>>>>> (GitHub Action)
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>>>>>>> (Jenkins)
>>>>>>>>
>>>>>>>> GitHash: a3a6c5e
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>>>>>>> (GitHub Action)
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>>>>>>> (Jenkins)
>>>>>>>>
>>>>>>>> GitHash: b6d26b7
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>>>>>>> (GitHub Action)
>>>>>>>>
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>>>>>>> (Jenkins)
>>>>>>>>
>>>>>>>> Could you recheck it?
>>>>>>>>
>>>>>>>>
>>>>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>>>>> with some same commits, and you will get a lot of fake comments
>>>>>>>>> then.
>>>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>>>> chosen one.
>>>>>>>>> 1.1
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi ALL, here is something I notice after this change:
>>>>>>>>>
>>>>>>>>> 1. Github-actions notification could be wrong when another PR
>>>>>>>>> opened
>>>>>>>>> with some same commits, and you will get a lot of fake comments
>>>>>>>>> then.
>>>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>>>> chosen one.
>>>>>>>>> 1.1
>>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>>> 2. New Forks have to turn on GitHub action by the fork owner
>>>>>>>>> manually
>>>>>>>>> 3. `Notify test workflow` keeps waiting when the build flow
>>>>>>>>> canceled
>>>>>>>>> or the whole fork gone
>>>>>>>>> 4. After refreshed master or even re-forked :(, I still got
>>>>>>>>> failures
>>>>>>>>> and seems not alone
>>>>>>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>>>>>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after
>>>>>>>>> re-forked)
>>>>>>>>> 4.3.
>>>>>>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>>>>>>> (some other failures noticed)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Bests,
>>>>>>>>>
>>>>>>>>> Kent
>>>>>>>>>
>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>>>>>>> >
>>>>>>>>> > Thank you again, Hyukjin.
>>>>>>>>> >
>>>>>>>>> > Bests,
>>>>>>>>> > Dongjoon.
>>>>>>>>> >
>>>>>>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Cool, thanks!
>>>>>>>>> >>
>>>>>>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>>>>>>> >>>
>>>>>>>>> >>> Good point! I had to clarify.
>>>>>>>>> >>> Once is enough. The sync is needed for your branch to include
>>>>>>>>> the changes of https://github.com/apache/spark/pull/32092.
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>>>>>> >>>>
>>>>>>>>> >>>> Hi Hyukjin,
>>>>>>>>> >>>>
>>>>>>>>> >>>> > Please sync your branch to the latest master branch in
>>>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>>>> detect it.
>>>>>>>>> >>>>
>>>>>>>>> >>>> Do we need to sync master for every PR or just one-time cost
>>>>>>>>> to keep up with the current master branch？
>>>>>>>>> >>>>
>>>>>>>>> >>>> Kent Yao
>>>>>>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase
>>>>>>>>> Corp.
>>>>>>>>> >>>> a spark enthusiast
>>>>>>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for
>>>>>>>>> large-scale data processing and analytics, built on top of Apache Spark.
>>>>>>>>> >>>>
>>>>>>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL
>>>>>>>>> Standard Authorization for Apache Spark.
>>>>>>>>> >>>> spark-postgres A library for reading data from and
>>>>>>>>> transferring data to Postgres / Greenplum with Spark SQL and DataFrames,
>>>>>>>>> 10~100x faster.
>>>>>>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>>>>>>> functions from various modern database management systems to Apache Spark.
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>>>>>> >>>>
>>>>>>>>> >>>> Cool~Thanks, Hyukjin
>>>>>>>>> >>>>
>>>>>>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <
>>>>>>>>> ltnwgl@gmail.com> wrote:
>>>>>>>>> >>>>>>>
>>>>>>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>>>>>>> https://github.com/apache/spark/pull/32158
>>>>>>>>> >>>>>>>
>>>>>>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> Hi all,
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>>>>> now we run the GitHub Actions
>>>>>>>>> >>>>>>>> workflows in your forked repository.
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> You create a PR and your repository triggers the
>>>>>>>>> workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>>> >>>>>>>> Apache Spark repository finds your workflow, and links it
>>>>>>>>> in a comment in your PR
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>>>> related to this.
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> What does that mean to contributors?
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> Please sync your branch to the latest master branch in
>>>>>>>>> Apache Spark in order for your forked repository to run the workflow, and
>>>>>>>>> >>>>>>>> for the main repository to detect the workflow.
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> What does that mean to committers?
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>>>> Actions builds are running (in contributor's forked repository).
>>>>>>>>> >>>>>>>> Please check the build notified by github-actions bot
>>>>>>>>> before merging it.
>>>>>>>>> >>>>>>>> There would be a followup work to reflect the status of
>>>>>>>>> the forked repository's build to the status of PR.
>>>>>>>>> >>>>>>>>
>>>>>>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <
>>>>>>>>> gurwls223@gmail.com>님이 작성:
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> Hi all,
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092
>>>>>>>>> merged, now we run the GitHub Actions
>>>>>>>>> >>>>>>>>> workflows in your forked repository.
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> You create a PR and your repository triggers the
>>>>>>>>> workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links
>>>>>>>>> it in a comment in your PR
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>>>> related to this.
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> What does that mean to contributors?
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> Please sync your branch to the latest master branch in
>>>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>>>> detect it.
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> What does that mean to committers?
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>>>> Actions builds are running (in contributor's forked repository). Please
>>>>>>>>> check the build notified by github-actions bot before merging it.
>>>>>>>>> >>>>>>>>> There would be a followup work to reflect the status of
>>>>>>>>> the forked repository's build to
>>>>>>>>> >>>>>>>>> the status of PR.
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>>>>>>>>
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> --
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Kent Yao
>>>>>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>>>>> a spark enthusiast
>>>>>>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>>>>>>>> processing and analytics, built on top of Apache Spark.
>>>>>>>>>
>>>>>>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>>>>>> Authorization for Apache Spark.
>>>>>>>>> spark-postgres A library for reading data from and transferring
>>>>>>>>> data
>>>>>>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x
>>>>>>>>> faster.
>>>>>>>>> itatchiA library that brings useful functions from various modern
>>>>>>>>> database management systems to Apache Spark.
>>>>>>>>>
>>>>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
The PR is ready for a review now: https://github.com/apache/spark/pull/32193
.
I would appreciate any reviews on this ..
2021년 4월 16일 (금) 오전 1:31, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> FYI, I made a POC PR to integrate it with GitHub Actions status (instead
> of a comment): https://github.com/apache/spark/pull/32193.
> This is just a minimal working version. Please take a look when you guys
> find some time.
>
> 2021년 4월 15일 (목) 오후 11:12, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>
>> I think we should probably leverage "action required" that allows us
>> to link URLs with some guides (see
>> https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites)
>> to inform users to enable it.
>>
>> 2021년 4월 15일 (목) 오후 1:26, Kent Yao <ya...@gmail.com>님이 작성:
>>
>>>
>>>
>>> I quick tested it by fork another apache project which contains GA
>>> workflows. The above pic shows what the action page looks like at the very
>>> beginning.
>>>
>>> Need to hit the green button to activate it
>>>
>>>
>>>
>>> *Kent Yao *
>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>> *a spark enthusiast*
>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>> unified multi-tenant JDBC interface for large-scale data processing and
>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>>> SQL extension which provides SQL Standard Authorization for **Apache
>>> Spark <http://spark.apache.org/>.*
>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>> for reading data from and transferring data to Postgres / Greenplum with
>>> Spark SQL and DataFrames, 10~100x faster.*
>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>>> brings useful functions from various modern database management systems to **Apache
>>> Spark <http://spark.apache.org/>.*
>>>
>>>
>>>
>>> On 04/15/2021 12:17，Hyukjin Kwon<gu...@gmail.com>
>>> <gu...@gmail.com> wrote：
>>>
>>> I remember it's turned on by default (?). If not, yeah we should
>>> document.
>>>
>>> 2021년 4월 15일 (목) 오후 1:14, Kent Yao <ya...@gmail.com>님이 작성:
>>>
>>>> Thanks Hyukjin and Yikun,
>>>>
>>>> > 2. New Forks have to turn on GitHub action by the fork owner manually
>>>>
>>>> And we may still need a suitable place to make this note clearer to new
>>>> contributors or someone delete and re-fork their forked repo.
>>>>
>>>> Thanks
>>>>
>>>>
>>>> *Kent Yao *
>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>> *a spark enthusiast*
>>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>>> unified multi-tenant JDBC interface for large-scale data processing and
>>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>>>> SQL extension which provides SQL Standard Authorization for **Apache
>>>> Spark <http://spark.apache.org/>.*
>>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>>> for reading data from and transferring data to Postgres / Greenplum with
>>>> Spark SQL and DataFrames, 10~100x faster.*
>>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>>>> brings useful functions from various modern database management systems to **Apache
>>>> Spark <http://spark.apache.org/>.*
>>>>
>>>>
>>>>
>>>> On 04/15/2021 12:09，Hyukjin Kwon<gu...@gmail.com>
>>>> <gu...@gmail.com> wrote：
>>>>
>>>> The issue is fixed now. Please keep monitoring this. Thank you all! The
>>>> spark community is super active and cooperative!
>>>>
>>>> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>
>>>>> The fix will be straightforward. We can either, in Github Actions
>>>>> workflow,:
>>>>> - remove fast forward option and see if ti works
>>>>> - or git rebase before merge the branch
>>>>>
>>>>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>
>>>>>> I think it works mostly correctly as Dongjoon investigated and shared
>>>>>> (Thanks a lot!).
>>>>>> One problem seems to be syncing to the master seems too strict (
>>>>>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>>>>>> Thanks Yikun.
>>>>>> I think we should make it less strict. I can create a PR right away
>>>>>> but would like to encourage Yikun or Kent to do it in order to keep the
>>>>>> credits of their investigation.
>>>>>>
>>>>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이
>>>>>> 작성:
>>>>>>
>>>>>>> Hi, Kent.
>>>>>>>
>>>>>>> I checked (1) in your PR, but those test result comments look
>>>>>>> correct to me.
>>>>>>> Please note that both Jenkins and GitHub Action leave the same
>>>>>>> number of comments on the same GitHash.
>>>>>>> Given that, there are not fake comments. It looks like a real result
>>>>>>> of your commits on that PR.
>>>>>>>
>>>>>>> GitHash: 23248c3
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>> (GitHub Action)
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>>>>>> (Jenkins)
>>>>>>>
>>>>>>> GitHash: 8dbed7b
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>>>>>> (GitHub Action)
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>>>>>> (Jenkins)
>>>>>>>
>>>>>>> GitHash: a3a6c5e
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>>>>>> (GitHub Action)
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>>>>>> (Jenkins)
>>>>>>>
>>>>>>> GitHash: b6d26b7
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>>>>>> (GitHub Action)
>>>>>>>
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>>>>>> (Jenkins)
>>>>>>>
>>>>>>> Could you recheck it?
>>>>>>>
>>>>>>>
>>>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>>>> with some same commits, and you will get a lot of fake comments
>>>>>>>> then.
>>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>>> chosen one.
>>>>>>>> 1.1
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi ALL, here is something I notice after this change:
>>>>>>>>
>>>>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>>>> with some same commits, and you will get a lot of fake comments
>>>>>>>> then.
>>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>>> chosen one.
>>>>>>>> 1.1
>>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>> 2. New Forks have to turn on GitHub action by the fork owner
>>>>>>>> manually
>>>>>>>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>>>>>>>> or the whole fork gone
>>>>>>>> 4. After refreshed master or even re-forked :(, I still got failures
>>>>>>>> and seems not alone
>>>>>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>>>>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after
>>>>>>>> re-forked)
>>>>>>>> 4.3.
>>>>>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>>>>>> (some other failures noticed)
>>>>>>>>
>>>>>>>>
>>>>>>>> Bests,
>>>>>>>>
>>>>>>>> Kent
>>>>>>>>
>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>>>>>> >
>>>>>>>> > Thank you again, Hyukjin.
>>>>>>>> >
>>>>>>>> > Bests,
>>>>>>>> > Dongjoon.
>>>>>>>> >
>>>>>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> >>
>>>>>>>> >> Cool, thanks!
>>>>>>>> >>
>>>>>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>>>>>> >>>
>>>>>>>> >>> Good point! I had to clarify.
>>>>>>>> >>> Once is enough. The sync is needed for your branch to include
>>>>>>>> the changes of https://github.com/apache/spark/pull/32092.
>>>>>>>> >>>
>>>>>>>> >>>
>>>>>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>>>>> >>>>
>>>>>>>> >>>> Hi Hyukjin,
>>>>>>>> >>>>
>>>>>>>> >>>> > Please sync your branch to the latest master branch in
>>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>>> detect it.
>>>>>>>> >>>>
>>>>>>>> >>>> Do we need to sync master for every PR or just one-time cost
>>>>>>>> to keep up with the current master branch？
>>>>>>>> >>>>
>>>>>>>> >>>> Kent Yao
>>>>>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase
>>>>>>>> Corp.
>>>>>>>> >>>> a spark enthusiast
>>>>>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale
>>>>>>>> data processing and analytics, built on top of Apache Spark.
>>>>>>>> >>>>
>>>>>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL
>>>>>>>> Standard Authorization for Apache Spark.
>>>>>>>> >>>> spark-postgres A library for reading data from and
>>>>>>>> transferring data to Postgres / Greenplum with Spark SQL and DataFrames,
>>>>>>>> 10~100x faster.
>>>>>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>>>>>> functions from various modern database management systems to Apache Spark.
>>>>>>>> >>>>
>>>>>>>> >>>>
>>>>>>>> >>>>
>>>>>>>> >>>>
>>>>>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>>>>> >>>>
>>>>>>>> >>>> Cool~Thanks, Hyukjin
>>>>>>>> >>>>
>>>>>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>>>>> >>>>>
>>>>>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>>>> >>>>>
>>>>>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>>>>> >>>>>>
>>>>>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>>>>>> >>>>>>
>>>>>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <
>>>>>>>> ltnwgl@gmail.com> wrote:
>>>>>>>> >>>>>>>
>>>>>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>>>>>> https://github.com/apache/spark/pull/32158
>>>>>>>> >>>>>>>
>>>>>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> Hi all,
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>>>> now we run the GitHub Actions
>>>>>>>> >>>>>>>> workflows in your forked repository.
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> You create a PR and your repository triggers the workflow.
>>>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>>>> >>>>>>>> Apache Spark repository finds your workflow, and links it
>>>>>>>> in a comment in your PR
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>>> related to this.
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> What does that mean to contributors?
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> Please sync your branch to the latest master branch in
>>>>>>>> Apache Spark in order for your forked repository to run the workflow, and
>>>>>>>> >>>>>>>> for the main repository to detect the workflow.
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> What does that mean to committers?
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>>> Actions builds are running (in contributor's forked repository).
>>>>>>>> >>>>>>>> Please check the build notified by github-actions bot
>>>>>>>> before merging it.
>>>>>>>> >>>>>>>> There would be a followup work to reflect the status of
>>>>>>>> the forked repository's build to the status of PR.
>>>>>>>> >>>>>>>>
>>>>>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <
>>>>>>>> gurwls223@gmail.com>님이 작성:
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> Hi all,
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>>>> now we run the GitHub Actions
>>>>>>>> >>>>>>>>> workflows in your forked repository.
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> You create a PR and your repository triggers the
>>>>>>>> workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links it
>>>>>>>> in a comment in your PR
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>>> related to this.
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> What does that mean to contributors?
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> Please sync your branch to the latest master branch in
>>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>>> detect it.
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> What does that mean to committers?
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>>> Actions builds are running (in contributor's forked repository). Please
>>>>>>>> check the build notified by github-actions bot before merging it.
>>>>>>>> >>>>>>>>> There would be a followup work to reflect the status of
>>>>>>>> the forked repository's build to
>>>>>>>> >>>>>>>>> the status of PR.
>>>>>>>> >>>>>>>>>
>>>>>>>> >>>>>>>>>
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> --
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Kent Yao
>>>>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>>>> a spark enthusiast
>>>>>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>>>>>>> processing and analytics, built on top of Apache Spark.
>>>>>>>>
>>>>>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>>>>> Authorization for Apache Spark.
>>>>>>>> spark-postgres A library for reading data from and transferring data
>>>>>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x
>>>>>>>> faster.
>>>>>>>> itatchiA library that brings useful functions from various modern
>>>>>>>> database management systems to Apache Spark.
>>>>>>>>
>>>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
FYI, I made a POC PR to integrate it with GitHub Actions status (instead of
a comment): https://github.com/apache/spark/pull/32193.
This is just a minimal working version. Please take a look when you guys
find some time.
2021년 4월 15일 (목) 오후 11:12, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> I think we should probably leverage "action required" that allows us
> to link URLs with some guides (see
> https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites)
> to inform users to enable it.
>
> 2021년 4월 15일 (목) 오후 1:26, Kent Yao <ya...@gmail.com>님이 작성:
>
>>
>>
>> I quick tested it by fork another apache project which contains GA
>> workflows. The above pic shows what the action page looks like at the very
>> beginning.
>>
>> Need to hit the green button to activate it
>>
>>
>>
>> *Kent Yao *
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> *a spark enthusiast*
>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>> unified multi-tenant JDBC interface for large-scale data processing and
>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>> SQL extension which provides SQL Standard Authorization for **Apache
>> Spark <http://spark.apache.org/>.*
>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>> for reading data from and transferring data to Postgres / Greenplum with
>> Spark SQL and DataFrames, 10~100x faster.*
>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>> brings useful functions from various modern database management systems to **Apache
>> Spark <http://spark.apache.org/>.*
>>
>>
>>
>> On 04/15/2021 12:17，Hyukjin Kwon<gu...@gmail.com>
>> <gu...@gmail.com> wrote：
>>
>> I remember it's turned on by default (?). If not, yeah we should document.
>>
>> 2021년 4월 15일 (목) 오후 1:14, Kent Yao <ya...@gmail.com>님이 작성:
>>
>>> Thanks Hyukjin and Yikun,
>>>
>>> > 2. New Forks have to turn on GitHub action by the fork owner manually
>>>
>>> And we may still need a suitable place to make this note clearer to new
>>> contributors or someone delete and re-fork their forked repo.
>>>
>>> Thanks
>>>
>>>
>>> *Kent Yao *
>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>> *a spark enthusiast*
>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>> unified multi-tenant JDBC interface for large-scale data processing and
>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>>> SQL extension which provides SQL Standard Authorization for **Apache
>>> Spark <http://spark.apache.org/>.*
>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>> for reading data from and transferring data to Postgres / Greenplum with
>>> Spark SQL and DataFrames, 10~100x faster.*
>>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>>> brings useful functions from various modern database management systems to **Apache
>>> Spark <http://spark.apache.org/>.*
>>>
>>>
>>>
>>> On 04/15/2021 12:09，Hyukjin Kwon<gu...@gmail.com>
>>> <gu...@gmail.com> wrote：
>>>
>>> The issue is fixed now. Please keep monitoring this. Thank you all! The
>>> spark community is super active and cooperative!
>>>
>>> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>
>>>> The fix will be straightforward. We can either, in Github Actions
>>>> workflow,:
>>>> - remove fast forward option and see if ti works
>>>> - or git rebase before merge the branch
>>>>
>>>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>
>>>>> I think it works mostly correctly as Dongjoon investigated and shared
>>>>> (Thanks a lot!).
>>>>> One problem seems to be syncing to the master seems too strict (
>>>>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>>>>> Thanks Yikun.
>>>>> I think we should make it less strict. I can create a PR right away
>>>>> but would like to encourage Yikun or Kent to do it in order to keep the
>>>>> credits of their investigation.
>>>>>
>>>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이
>>>>> 작성:
>>>>>
>>>>>> Hi, Kent.
>>>>>>
>>>>>> I checked (1) in your PR, but those test result comments look correct
>>>>>> to me.
>>>>>> Please note that both Jenkins and GitHub Action leave the same number
>>>>>> of comments on the same GitHash.
>>>>>> Given that, there are not fake comments. It looks like a real result
>>>>>> of your commits on that PR.
>>>>>>
>>>>>> GitHash: 23248c3
>>>>>>
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>> (GitHub Action)
>>>>>>
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>>>>> (Jenkins)
>>>>>>
>>>>>> GitHash: 8dbed7b
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>>>>> (GitHub Action)
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>>>>> (Jenkins)
>>>>>>
>>>>>> GitHash: a3a6c5e
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>>>>> (GitHub Action)
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>>>>> (Jenkins)
>>>>>>
>>>>>> GitHash: b6d26b7
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>>>>> (GitHub Action)
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>>>>> (Jenkins)
>>>>>>
>>>>>> Could you recheck it?
>>>>>>
>>>>>>
>>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>>> with some same commits, and you will get a lot of fake comments then.
>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>> chosen one.
>>>>>>> 1.1
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi ALL, here is something I notice after this change:
>>>>>>>
>>>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>>> with some same commits, and you will get a lot of fake comments then.
>>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>>> chosen one.
>>>>>>> 1.1
>>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>> 2. New Forks have to turn on GitHub action by the fork owner manually
>>>>>>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>>>>>>> or the whole fork gone
>>>>>>> 4. After refreshed master or even re-forked :(, I still got failures
>>>>>>> and seems not alone
>>>>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>>>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after
>>>>>>> re-forked)
>>>>>>> 4.3.
>>>>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>>>>> (some other failures noticed)
>>>>>>>
>>>>>>>
>>>>>>> Bests,
>>>>>>>
>>>>>>> Kent
>>>>>>>
>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>>>>> >
>>>>>>> > Thank you again, Hyukjin.
>>>>>>> >
>>>>>>> > Bests,
>>>>>>> > Dongjoon.
>>>>>>> >
>>>>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> Cool, thanks!
>>>>>>> >>
>>>>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>>>>> >>>
>>>>>>> >>> Good point! I had to clarify.
>>>>>>> >>> Once is enough. The sync is needed for your branch to include
>>>>>>> the changes of https://github.com/apache/spark/pull/32092.
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>>>> >>>>
>>>>>>> >>>> Hi Hyukjin,
>>>>>>> >>>>
>>>>>>> >>>> > Please sync your branch to the latest master branch in Apache
>>>>>>> Spark in order for the main repository to run the workflow and detect it.
>>>>>>> >>>>
>>>>>>> >>>> Do we need to sync master for every PR or just one-time cost to
>>>>>>> keep up with the current master branch？
>>>>>>> >>>>
>>>>>>> >>>> Kent Yao
>>>>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase
>>>>>>> Corp.
>>>>>>> >>>> a spark enthusiast
>>>>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale
>>>>>>> data processing and analytics, built on top of Apache Spark.
>>>>>>> >>>>
>>>>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL
>>>>>>> Standard Authorization for Apache Spark.
>>>>>>> >>>> spark-postgres A library for reading data from and transferring
>>>>>>> data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>>>>> functions from various modern database management systems to Apache Spark.
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>>>> >>>>
>>>>>>> >>>> Cool~Thanks, Hyukjin
>>>>>>> >>>>
>>>>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>>>> >>>>>
>>>>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>>> >>>>>
>>>>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>>>> >>>>>>
>>>>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>>>>> >>>>>>
>>>>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <
>>>>>>> ltnwgl@gmail.com> wrote:
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>>>>> https://github.com/apache/spark/pull/32158
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Hi all,
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>>> now we run the GitHub Actions
>>>>>>> >>>>>>>> workflows in your forked repository.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> You create a PR and your repository triggers the workflow.
>>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>>> >>>>>>>> Apache Spark repository finds your workflow, and links it
>>>>>>> in a comment in your PR
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>> related to this.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> What does that mean to contributors?
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Please sync your branch to the latest master branch in
>>>>>>> Apache Spark in order for your forked repository to run the workflow, and
>>>>>>> >>>>>>>> for the main repository to detect the workflow.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> What does that mean to committers?
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>> Actions builds are running (in contributor's forked repository).
>>>>>>> >>>>>>>> Please check the build notified by github-actions bot
>>>>>>> before merging it.
>>>>>>> >>>>>>>> There would be a followup work to reflect the status of the
>>>>>>> forked repository's build to the status of PR.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
>>>>>>> 작성:
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> Hi all,
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>>> now we run the GitHub Actions
>>>>>>> >>>>>>>>> workflows in your forked repository.
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> You create a PR and your repository triggers the workflow.
>>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links it
>>>>>>> in a comment in your PR
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>>> related to this.
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> What does that mean to contributors?
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> Please sync your branch to the latest master branch in
>>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>>> detect it.
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> What does that mean to committers?
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>>> Actions builds are running (in contributor's forked repository). Please
>>>>>>> check the build notified by github-actions bot before merging it.
>>>>>>> >>>>>>>>> There would be a followup work to reflect the status of
>>>>>>> the forked repository's build to
>>>>>>> >>>>>>>>> the status of PR.
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> --
>>>>>>> >>
>>>>>>> >>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Kent Yao
>>>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>>> a spark enthusiast
>>>>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>>>>>> processing and analytics, built on top of Apache Spark.
>>>>>>>
>>>>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>>>> Authorization for Apache Spark.
>>>>>>> spark-postgres A library for reading data from and transferring data
>>>>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x
>>>>>>> faster.
>>>>>>> itatchiA library that brings useful functions from various modern
>>>>>>> database management systems to Apache Spark.
>>>>>>>
>>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
I think we should probably leverage "action required" that allows us
to link URLs with some guides (see
https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites)
to inform users to enable it.
2021년 4월 15일 (목) 오후 1:26, Kent Yao <ya...@gmail.com>님이 작성:
>
>
> I quick tested it by fork another apache project which contains GA
> workflows. The above pic shows what the action page looks like at the very
> beginning.
>
> Need to hit the green button to activate it
>
>
>
> *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> *a spark enthusiast*
> *kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top
> of Apache Spark <http://spark.apache.org/>.*
> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> SQL extension which provides SQL Standard Authorization for **Apache
> Spark <http://spark.apache.org/>.*
> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.*
> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
> brings useful functions from various modern database management systems to **Apache
> Spark <http://spark.apache.org/>.*
>
>
>
> On 04/15/2021 12:17，Hyukjin Kwon<gu...@gmail.com>
> <gu...@gmail.com> wrote：
>
> I remember it's turned on by default (?). If not, yeah we should document.
>
> 2021년 4월 15일 (목) 오후 1:14, Kent Yao <ya...@gmail.com>님이 작성:
>
>> Thanks Hyukjin and Yikun,
>>
>> > 2. New Forks have to turn on GitHub action by the fork owner manually
>>
>> And we may still need a suitable place to make this note clearer to new
>> contributors or someone delete and re-fork their forked repo.
>>
>> Thanks
>>
>>
>> *Kent Yao *
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> *a spark enthusiast*
>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>> unified multi-tenant JDBC interface for large-scale data processing and
>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>> SQL extension which provides SQL Standard Authorization for **Apache
>> Spark <http://spark.apache.org/>.*
>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>> for reading data from and transferring data to Postgres / Greenplum with
>> Spark SQL and DataFrames, 10~100x faster.*
>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
>> brings useful functions from various modern database management systems to **Apache
>> Spark <http://spark.apache.org/>.*
>>
>>
>>
>> On 04/15/2021 12:09，Hyukjin Kwon<gu...@gmail.com>
>> <gu...@gmail.com> wrote：
>>
>> The issue is fixed now. Please keep monitoring this. Thank you all! The
>> spark community is super active and cooperative!
>>
>> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>
>>> The fix will be straightforward. We can either, in Github Actions
>>> workflow,:
>>> - remove fast forward option and see if ti works
>>> - or git rebase before merge the branch
>>>
>>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>
>>>> I think it works mostly correctly as Dongjoon investigated and shared
>>>> (Thanks a lot!).
>>>> One problem seems to be syncing to the master seems too strict (
>>>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>>>> Thanks Yikun.
>>>> I think we should make it less strict. I can create a PR right away but
>>>> would like to encourage Yikun or Kent to do it in order to keep the credits
>>>> of their investigation.
>>>>
>>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이 작성:
>>>>
>>>>> Hi, Kent.
>>>>>
>>>>> I checked (1) in your PR, but those test result comments look correct
>>>>> to me.
>>>>> Please note that both Jenkins and GitHub Action leave the same number
>>>>> of comments on the same GitHash.
>>>>> Given that, there are not fake comments. It looks like a real result
>>>>> of your commits on that PR.
>>>>>
>>>>> GitHash: 23248c3
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>> (GitHub Action)
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>>>> (Jenkins)
>>>>>
>>>>> GitHash: 8dbed7b
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>>>> (GitHub Action)
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>>>> (Jenkins)
>>>>>
>>>>> GitHash: a3a6c5e
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>>>> (GitHub Action)
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>>>> (Jenkins)
>>>>>
>>>>> GitHash: b6d26b7
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>>>> (GitHub Action)
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>>>> (Jenkins)
>>>>>
>>>>> Could you recheck it?
>>>>>
>>>>>
>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>> with some same commits, and you will get a lot of fake comments then.
>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>> chosen one.
>>>>>> 1.1
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>>
>>>>>
>>>>>
>>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
>>>>>
>>>>>> Hi ALL, here is something I notice after this change:
>>>>>>
>>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>>> with some same commits, and you will get a lot of fake comments then.
>>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>>> chosen one.
>>>>>> 1.1
>>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>> 2. New Forks have to turn on GitHub action by the fork owner manually
>>>>>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>>>>>> or the whole fork gone
>>>>>> 4. After refreshed master or even re-forked :(, I still got failures
>>>>>> and seems not alone
>>>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after
>>>>>> re-forked)
>>>>>> 4.3.
>>>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>>>> (some other failures noticed)
>>>>>>
>>>>>>
>>>>>> Bests,
>>>>>>
>>>>>> Kent
>>>>>>
>>>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>>>> >
>>>>>> > Thank you again, Hyukjin.
>>>>>> >
>>>>>> > Bests,
>>>>>> > Dongjoon.
>>>>>> >
>>>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Cool, thanks!
>>>>>> >>
>>>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>>>> >>>
>>>>>> >>> Good point! I had to clarify.
>>>>>> >>> Once is enough. The sync is needed for your branch to include the
>>>>>> changes of https://github.com/apache/spark/pull/32092.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>>> >>>>
>>>>>> >>>> Hi Hyukjin,
>>>>>> >>>>
>>>>>> >>>> > Please sync your branch to the latest master branch in Apache
>>>>>> Spark in order for the main repository to run the workflow and detect it.
>>>>>> >>>>
>>>>>> >>>> Do we need to sync master for every PR or just one-time cost to
>>>>>> keep up with the current master branch？
>>>>>> >>>>
>>>>>> >>>> Kent Yao
>>>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>> >>>> a spark enthusiast
>>>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale
>>>>>> data processing and analytics, built on top of Apache Spark.
>>>>>> >>>>
>>>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL
>>>>>> Standard Authorization for Apache Spark.
>>>>>> >>>> spark-postgres A library for reading data from and transferring
>>>>>> data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>>>> functions from various modern database management systems to Apache Spark.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>>> >>>>
>>>>>> >>>> Cool~Thanks, Hyukjin
>>>>>> >>>>
>>>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>>> >>>>>
>>>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>> >>>>>
>>>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>>> >>>>>>
>>>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>>>> >>>>>>
>>>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <
>>>>>> ltnwgl@gmail.com> wrote:
>>>>>> >>>>>>>
>>>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>>>> https://github.com/apache/spark/pull/32158
>>>>>> >>>>>>>
>>>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>>>> gurwls223@gmail.com> wrote:
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Hi all,
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>> now we run the GitHub Actions
>>>>>> >>>>>>>> workflows in your forked repository.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> You create a PR and your repository triggers the workflow.
>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>> >>>>>>>> Apache Spark repository finds your workflow, and links it in
>>>>>> a comment in your PR
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>> related to this.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> What does that mean to contributors?
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Please sync your branch to the latest master branch in
>>>>>> Apache Spark in order for your forked repository to run the workflow, and
>>>>>> >>>>>>>> for the main repository to detect the workflow.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> What does that mean to committers?
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>> Actions builds are running (in contributor's forked repository).
>>>>>> >>>>>>>> Please check the build notified by github-actions bot before
>>>>>> merging it.
>>>>>> >>>>>>>> There would be a followup work to reflect the status of the
>>>>>> forked repository's build to the status of PR.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
>>>>>> 작성:
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Hi all,
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>>> now we run the GitHub Actions
>>>>>> >>>>>>>>> workflows in your forked repository.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> You create a PR and your repository triggers the workflow.
>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links it
>>>>>> in a comment in your PR
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>>>> related to this.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> What does that mean to contributors?
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Please sync your branch to the latest master branch in
>>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>>> detect it.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> What does that mean to committers?
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>>> Actions builds are running (in contributor's forked repository). Please
>>>>>> check the build notified by github-actions bot before merging it.
>>>>>> >>>>>>>>> There would be a followup work to reflect the status of the
>>>>>> forked repository's build to
>>>>>> >>>>>>>>> the status of PR.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>
>>>>>> >>
>>>>>> >> --
>>>>>> >>
>>>>>> >>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Kent Yao
>>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>>> a spark enthusiast
>>>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>>>>> processing and analytics, built on top of Apache Spark.
>>>>>>
>>>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>>> Authorization for Apache Spark.
>>>>>> spark-postgres A library for reading data from and transferring data
>>>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>>>> itatchiA library that brings useful functions from various modern
>>>>>> database management systems to Apache Spark.
>>>>>>
>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your forked repository
Posted by Kent Yao <ya...@gmail.com>.
I quick tested it by fork another apache project which contains GA workflows.
The above pic shows what the action page looks like at the very beginning.
Need to hit the green button to activate it
**Kent Yao **
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
** _a s_ _park enthusiast_**
_[kyuubi](https://github.com/yaooqinn/kyuubi) is a unified multi-tenant JDBC
interface for large-scale data processing and analytics, built on top of
[Apache Spark](http://spark.apache.org/)._
#
_[spark-authorizer](https://github.com/yaooqinn/spark-authorizer) A Spark SQL
extension which provides SQL Standard Authorization for _ _[Apache
Spark](http://spark.apache.org/)._
_ _[spark-postgres](https://github.com/yaooqinn/spark-postgres) A library
for reading data from and transferring data to Postgres / Greenplum with Spark
SQL and DataFrames, 10~100x faster.__
_ _[itatchi](https://github.com/yaooqinn/spark-func-extras) A__ _library t_
_hat brings useful functions from various modern database management systems
to _ _ _[Apache Spark](http://spark.apache.org/).__
#
On 04/15/2021 12:17，[Hyukjin
Kwon<gu...@gmail.com>](mailto:gurwls223@gmail.com) wrote：
> I remember it's turned on by default (?). If not, yeah we should document.
>
>
>
>
> 2021년 4월 15일 (목) 오후 1:14, Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)>님이 작성:
>
>
>> Thanks Hyukjin and Yikun,
>>
>>
>
>>
>> > 2\. New Forks have to turn on GitHub action by the fork owner manually
>
>>
>>
>
>>
>> And we may still need a suitable place to make this note clearer to new
contributors or someone delete and re-fork their forked repo.
>>
>>
>
>>
>> Thanks
>>
>>
>
>>
>>
>
>>
>> **Kent Yao **
>>
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>
>> ** _a s_ _park enthusiast_**
>>
>> _[kyuubi](https://github.com/yaooqinn/kyuubi) is a unified multi-tenant
JDBC interface for large-scale data processing and analytics, built on top of
[Apache Spark](http://spark.apache.org/)._
>>
>> #
>>
>> _[spark-authorizer](https://github.com/yaooqinn/spark-authorizer) A Spark
SQL extension which provides SQL Standard Authorization for _ _[Apache
Spark](http://spark.apache.org/)._
>>
>> _ _[spark-postgres](https://github.com/yaooqinn/spark-postgres) A library
for reading data from and transferring data to Postgres / Greenplum with Spark
SQL and DataFrames, 10~100x faster.__
>>
>> _ _[itatchi](https://github.com/yaooqinn/spark-func-extras) A__ _library
t_ _hat brings useful functions from various modern database management
systems to _ _ _[Apache Spark](http://spark.apache.org/).__
>>
>> #
>>
>>
>
>>
>>
>
>>
>>
>
>>
>> On 04/15/2021 12:09，[Hyukjin
Kwon<gu...@gmail.com>](mailto:gurwls223@gmail.com) wrote：
>>
>>> The issue is fixed now. Please keep monitoring this. Thank you all! The
spark community is super active and cooperative!
>>>
>>>
>
>>>
>>> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
>
>>>
>>>> The fix will be straightforward. We can either, in Github Actions
workflow,:
> \- remove fast forward option and see if ti works
> \- or git rebase before merge the branch
>>>>
>>>>
>
>>>>
>>>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
>
>>>>
>>>>> I think it works mostly correctly as Dongjoon investigated and shared
(Thanks a lot!).
> One problem seems to be syncing to the master seems too strict
(<https://github.com/apache/spark/pull/32168#issuecomment-819736508>). Thanks
Yikun.
> I think we should make it less strict. I can create a PR right away but
would like to encourage Yikun or Kent to do it in order to keep the credits of
their investigation.
>>>>>
>>>>>
>
>>>>>
>>>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun
<[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)>님이 작성:
>
>>>>>
>>>>>> Hi, Kent.
>>>>>>
>>>>>>
> I checked (1) in your PR, but those test result comments look correct to me.
>>>>>>
>>>>>> Please note that both Jenkins and GitHub Action leave the same number
of comments on the same GitHash.
>>>>>>
>>>>>> Given that, there are not fake comments. It looks like a real result of
your commits on that PR.
>>>>>>
>>>>>>
> GitHash: 23248c3
> <https://github.com/apache/spark/pull/32144#issuecomment-819679970>
(GitHub Action)
> <https://github.com/apache/spark/pull/32144#issuecomment-819647368>
(Jenkins)
>
> GitHash: 8dbed7b
> <https://github.com/apache/spark/pull/32144#issuecomment-819684782>
(GitHub Action)
> <https://github.com/apache/spark/pull/32144#issuecomment-819578976>
(Jenkins)
>
> GitHash: a3a6c5e
> <https://github.com/apache/spark/pull/32144#issuecomment-819690465>
(GitHub Action)
> <https://github.com/apache/spark/pull/32144#issuecomment-819793557>
(Jenkins)
>
> GitHash: b6d26b7
> <https://github.com/apache/spark/pull/32144#issuecomment-819691416>
(GitHub Action)
>>>>>>
>>>>>> <https://github.com/apache/spark/pull/32144#issuecomment-819791485>
(Jenkins)
>>>>>>
>>>>>>
>
>>>>>>
>>>>>> Could you recheck it?
>>>>>>
>>>>>>
>
>>>>>>
>>>>>>
>
>>>>>>
>>>>>>> 1\. Github-actions notification could be wrong when another PR opened
> with some same commits, and you will get a lot of fake comments then.
> Meanwhile, the new PR get no comments, even if it is actually the
> chosen one.
> 1.1 <https://github.com/apache/spark/pull/32144#issuecomment-819679970>
>
>>>>>>
>>>>>>
>
>>>>>>
>>>>>>
>
>>>>>>
>>>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)> wrote:
>
>>>>>>
>>>>>>> Hi ALL, here is something I notice after this change:
>
> 1\. Github-actions notification could be wrong when another PR opened
> with some same commits, and you will get a lot of fake comments then.
> Meanwhile, the new PR get no comments, even if it is actually the
> chosen one.
> 1.1 <https://github.com/apache/spark/pull/32144#issuecomment-819679970>
> 2\. New Forks have to turn on GitHub action by the fork owner manually
> 3\. `Notify test workflow` keeps waiting when the build flow canceled
> or the whole fork gone
> 4\. After refreshed master or even re-forked :(, I still got failures
> and seems not alone
> 4.1. <https://github.com/apache/spark/pull/32168> (PR after sync)
> 4.2. <https://github.com/apache/spark/pull/32172> (PR after re-forked)
> 4.3.
<https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true>
> (some other failures noticed)
>
>
> Bests,
>
> Kent
>
> Dongjoon Hyun <[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)>
于2021年4月14日周三 下午11:34写道：
> >
> > Thank you again, Hyukjin.
> >
> > Bests,
> > Dongjoon.
> >
> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)> wrote:
> >>
> >> Cool, thanks!
> >>
> >> Hyukjin Kwon <[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>
于2021年4月14日周三 下午8:19写道：
> >>>
> >>> Good point! I had to clarify.
> >>> Once is enough. The sync is needed for your branch to include the
changes of <https://github.com/apache/spark/pull/32092>.
> >>>
> >>>
> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)>님이 작성:
> >>>>
> >>>> Hi Hyukjin,
> >>>>
> >>>> > Please sync your branch to the latest master branch in Apache Spark
in order for the main repository to run the workflow and detect it.
> >>>>
> >>>> Do we need to sync master for every PR or just one-time cost to keep
up with the current master branch？
> >>>>
> >>>> Kent Yao
> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> >>>> a spark enthusiast
> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
processing and analytics, built on top of Apache Spark.
> >>>>
> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
Authorization for Apache Spark.
> >>>> spark-postgres A library for reading data from and transferring data
to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
> >>>> spark-func-extrasA library that brings excellent and useful functions
from various modern database management systems to Apache Spark.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On 04/14/2021 15:41，Kent Yao<[yao@apache.org](mailto:yao@apache.org)>
wrote：
> >>>>
> >>>> Cool~Thanks, Hyukjin
> >>>>
> >>>> Yuanjian Li <[xyliyuanjian@gmail.com](mailto:xyliyuanjian@gmail.com)>
于2021年4月14日周三 下午3:39写道：
> >>>>>
> >>>>> Awesome! Thanks for making this happen, Hyukjin!
> >>>>>
> >>>>> Yi Wu <[yi.wu@databricks.com](mailto:yi.wu@databricks.com)>
于2021年4月14日周三 下午2:51写道：
> >>>>>>
> >>>>>> Thanks for the great work, Hyukjin!
> >>>>>>
> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang
<[ltnwgl@gmail.com](mailto:ltnwgl@gmail.com)> wrote:
> >>>>>>>
> >>>>>>> Thanks for the amazing work, Hyukjin!
> >>>>>>> I created a PR for trial and it looks well so far:
<https://github.com/apache/spark/pull/32158>
> >>>>>>>
> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)> wrote:
> >>>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> After <https://github.com/apache/spark/pull/32092> merged, now we
run the GitHub Actions
> >>>>>>>> workflows in your forked repository.
> >>>>>>>>
> >>>>>>>> In short, please see this example HyukjinKwon#34
> >>>>>>>>
> >>>>>>>> You create a PR and your repository triggers the workflow. Your PR
uses the resources allocated to you for testing.
> >>>>>>>> Apache Spark repository finds your workflow, and links it in a
comment in your PR
> >>>>>>>>
> >>>>>>>> Please let me know if you guys find any weird behaviour related to
this.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What does that mean to contributors?
> >>>>>>>>
> >>>>>>>> Please sync your branch to the latest master branch in Apache
Spark in order for your forked repository to run the workflow, and
> >>>>>>>> for the main repository to detect the workflow.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What does that mean to committers?
> >>>>>>>>
> >>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
builds are running (in contributor's forked repository).
> >>>>>>>> Please check the build notified by github-actions bot before
merging it.
> >>>>>>>> There would be a followup work to reflect the status of the forked
repository's build to the status of PR.
> >>>>>>>>
> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
> >>>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> After <https://github.com/apache/spark/pull/32092> merged, now we
run the GitHub Actions
> >>>>>>>>> workflows in your forked repository.
> >>>>>>>>>
> >>>>>>>>> In short, please see this example HyukjinKwon#34
> >>>>>>>>>
> >>>>>>>>> You create a PR and your repository triggers the workflow. Your
PR uses the resources allocated to you for testing.
> >>>>>>>>> Apache Spark repository finds your workflow, and links it in a
comment in your PR
> >>>>>>>>>
> >>>>>>>>> Please let me know if you guys find any weird behaviour related
to this.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What does that mean to contributors?
> >>>>>>>>>
> >>>>>>>>> Please sync your branch to the latest master branch in Apache
Spark in order for the main repository to run the workflow and detect it.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What does that mean to committers?
> >>>>>>>>>
> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
builds are running (in contributor's forked repository). Please check the
build notified by github-actions bot before merging it.
> >>>>>>>>> There would be a followup work to reflect the status of the
forked repository's build to
> >>>>>>>>> the status of PR.
> >>>>>>>>>
> >>>>>>>>>
> >>
> >>
> >> \--
> >>
> >>
>
>
>
> \--
>
> Kent Yao
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> a spark enthusiast
> kyuubiis a unified multi-tenant JDBC interface for large-scale data
> processing and analytics, built on top of Apache Spark.
>
> spark-authorizerA Spark SQL extension which provides SQL Standard
> Authorization for Apache Spark.
> spark-postgres A library for reading data from and transferring data
> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
> itatchiA library that brings useful functions from various modern
> database management systems to Apache Spark.
>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
I remember it's turned on by default (?). If not, yeah we should document.
2021년 4월 15일 (목) 오후 1:14, Kent Yao <ya...@gmail.com>님이 작성:
> Thanks Hyukjin and Yikun,
>
> > 2. New Forks have to turn on GitHub action by the fork owner manually
>
> And we may still need a suitable place to make this note clearer to new
> contributors or someone delete and re-fork their forked repo.
>
> Thanks
>
>
> *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> *a spark enthusiast*
> *kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top
> of Apache Spark <http://spark.apache.org/>.*
> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> SQL extension which provides SQL Standard Authorization for **Apache
> Spark <http://spark.apache.org/>.*
> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.*
> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
> brings useful functions from various modern database management systems to **Apache
> Spark <http://spark.apache.org/>.*
>
>
>
> On 04/15/2021 12:09，Hyukjin Kwon<gu...@gmail.com>
> <gu...@gmail.com> wrote：
>
> The issue is fixed now. Please keep monitoring this. Thank you all! The
> spark community is super active and cooperative!
>
> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>
>> The fix will be straightforward. We can either, in Github Actions
>> workflow,:
>> - remove fast forward option and see if ti works
>> - or git rebase before merge the branch
>>
>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>
>>> I think it works mostly correctly as Dongjoon investigated and shared
>>> (Thanks a lot!).
>>> One problem seems to be syncing to the master seems too strict (
>>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>>> Thanks Yikun.
>>> I think we should make it less strict. I can create a PR right away but
>>> would like to encourage Yikun or Kent to do it in order to keep the credits
>>> of their investigation.
>>>
>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이 작성:
>>>
>>>> Hi, Kent.
>>>>
>>>> I checked (1) in your PR, but those test result comments look correct
>>>> to me.
>>>> Please note that both Jenkins and GitHub Action leave the same number
>>>> of comments on the same GitHash.
>>>> Given that, there are not fake comments. It looks like a real result of
>>>> your commits on that PR.
>>>>
>>>> GitHash: 23248c3
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>> (GitHub Action)
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>>> (Jenkins)
>>>>
>>>> GitHash: 8dbed7b
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>>> (GitHub Action)
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>>> (Jenkins)
>>>>
>>>> GitHash: a3a6c5e
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>>> (GitHub Action)
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>>> (Jenkins)
>>>>
>>>> GitHash: b6d26b7
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>>> (GitHub Action)
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>>> (Jenkins)
>>>>
>>>> Could you recheck it?
>>>>
>>>>
>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>> with some same commits, and you will get a lot of fake comments then.
>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>> chosen one.
>>>>> 1.1
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>>
>>>>
>>>>
>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
>>>>
>>>>> Hi ALL, here is something I notice after this change:
>>>>>
>>>>> 1. Github-actions notification could be wrong when another PR opened
>>>>> with some same commits, and you will get a lot of fake comments then.
>>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>>> chosen one.
>>>>> 1.1
>>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>> 2. New Forks have to turn on GitHub action by the fork owner manually
>>>>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>>>>> or the whole fork gone
>>>>> 4. After refreshed master or even re-forked :(, I still got failures
>>>>> and seems not alone
>>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after
>>>>> re-forked)
>>>>> 4.3.
>>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>>> (some other failures noticed)
>>>>>
>>>>>
>>>>> Bests,
>>>>>
>>>>> Kent
>>>>>
>>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>>> >
>>>>> > Thank you again, Hyukjin.
>>>>> >
>>>>> > Bests,
>>>>> > Dongjoon.
>>>>> >
>>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
>>>>> >>
>>>>> >> Cool, thanks!
>>>>> >>
>>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>>> >>>
>>>>> >>> Good point! I had to clarify.
>>>>> >>> Once is enough. The sync is needed for your branch to include the
>>>>> changes of https://github.com/apache/spark/pull/32092.
>>>>> >>>
>>>>> >>>
>>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>> >>>>
>>>>> >>>> Hi Hyukjin,
>>>>> >>>>
>>>>> >>>> > Please sync your branch to the latest master branch in Apache
>>>>> Spark in order for the main repository to run the workflow and detect it.
>>>>> >>>>
>>>>> >>>> Do we need to sync master for every PR or just one-time cost to
>>>>> keep up with the current master branch？
>>>>> >>>>
>>>>> >>>> Kent Yao
>>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>> >>>> a spark enthusiast
>>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale
>>>>> data processing and analytics, built on top of Apache Spark.
>>>>> >>>>
>>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>> Authorization for Apache Spark.
>>>>> >>>> spark-postgres A library for reading data from and transferring
>>>>> data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>>> functions from various modern database management systems to Apache Spark.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>> >>>>
>>>>> >>>> Cool~Thanks, Hyukjin
>>>>> >>>>
>>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>> >>>>>
>>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>> >>>>>
>>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>> >>>>>>
>>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>>> >>>>>>
>>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <
>>>>> ltnwgl@gmail.com> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>>> https://github.com/apache/spark/pull/32158
>>>>> >>>>>>>
>>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>>> gurwls223@gmail.com> wrote:
>>>>> >>>>>>>>
>>>>> >>>>>>>> Hi all,
>>>>> >>>>>>>>
>>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now
>>>>> we run the GitHub Actions
>>>>> >>>>>>>> workflows in your forked repository.
>>>>> >>>>>>>>
>>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>>> >>>>>>>>
>>>>> >>>>>>>> You create a PR and your repository triggers the workflow.
>>>>> Your PR uses the resources allocated to you for testing.
>>>>> >>>>>>>> Apache Spark repository finds your workflow, and links it in
>>>>> a comment in your PR
>>>>> >>>>>>>>
>>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>>> related to this.
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>> What does that mean to contributors?
>>>>> >>>>>>>>
>>>>> >>>>>>>> Please sync your branch to the latest master branch in Apache
>>>>> Spark in order for your forked repository to run the workflow, and
>>>>> >>>>>>>> for the main repository to detect the workflow.
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>> What does that mean to committers?
>>>>> >>>>>>>>
>>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>> Actions builds are running (in contributor's forked repository).
>>>>> >>>>>>>> Please check the build notified by github-actions bot before
>>>>> merging it.
>>>>> >>>>>>>> There would be a followup work to reflect the status of the
>>>>> forked repository's build to the status of PR.
>>>>> >>>>>>>>
>>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
>>>>> 작성:
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Hi all,
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged,
>>>>> now we run the GitHub Actions
>>>>> >>>>>>>>> workflows in your forked repository.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> You create a PR and your repository triggers the workflow.
>>>>> Your PR uses the resources allocated to you for testing.
>>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links it in
>>>>> a comment in your PR
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>>> related to this.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> What does that mean to contributors?
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Please sync your branch to the latest master branch in
>>>>> Apache Spark in order for the main repository to run the workflow and
>>>>> detect it.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> What does that mean to committers?
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>>> Actions builds are running (in contributor's forked repository). Please
>>>>> check the build notified by github-actions bot before merging it.
>>>>> >>>>>>>>> There would be a followup work to reflect the status of the
>>>>> forked repository's build to
>>>>> >>>>>>>>> the status of PR.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >>
>>>>> >>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Kent Yao
>>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>>> a spark enthusiast
>>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>>>> processing and analytics, built on top of Apache Spark.
>>>>>
>>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>>> Authorization for Apache Spark.
>>>>> spark-postgres A library for reading data from and transferring data
>>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>>> itatchiA library that brings useful functions from various modern
>>>>> database management systems to Apache Spark.
>>>>>
>>>>
Re: [PSA] Please read: PR builder now runs test and build in your forked repository
Posted by Kent Yao <ya...@gmail.com>.
Thanks Hyukjin and Yikun,
> 2\. New Forks have to turn on GitHub action by the fork owner manually
And we may still need a suitable place to make this note clearer to new
contributors or someone delete and re-fork their forked repo.
Thanks
**Kent Yao **
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
** _a s_ _park enthusiast_**
_[kyuubi](https://github.com/yaooqinn/kyuubi) is a unified multi-tenant JDBC
interface for large-scale data processing and analytics, built on top of
[Apache Spark](http://spark.apache.org/)._
#
_[spark-authorizer](https://github.com/yaooqinn/spark-authorizer) A Spark SQL
extension which provides SQL Standard Authorization for _ _[Apache
Spark](http://spark.apache.org/)._
_ _[spark-postgres](https://github.com/yaooqinn/spark-postgres) A library
for reading data from and transferring data to Postgres / Greenplum with Spark
SQL and DataFrames, 10~100x faster.__
_ _[itatchi](https://github.com/yaooqinn/spark-func-extras) A__ _library t_
_hat brings useful functions from various modern database management systems
to _ _ _[Apache Spark](http://spark.apache.org/).__
#
On 04/15/2021 12:09，[Hyukjin
Kwon<gu...@gmail.com>](mailto:gurwls223@gmail.com) wrote：
> The issue is fixed now. Please keep monitoring this. Thank you all! The
spark community is super active and cooperative!
>
>
>
>
> 2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
>
>
>> The fix will be straightforward. We can either, in Github Actions
workflow,:
> \- remove fast forward option and see if ti works
> \- or git rebase before merge the branch
>>
>>
>
>>
>> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
>
>>
>>> I think it works mostly correctly as Dongjoon investigated and shared
(Thanks a lot!).
> One problem seems to be syncing to the master seems too strict
(<https://github.com/apache/spark/pull/32168#issuecomment-819736508>). Thanks
Yikun.
> I think we should make it less strict. I can create a PR right away but
would like to encourage Yikun or Kent to do it in order to keep the credits of
their investigation.
>>>
>>>
>
>>>
>>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun
<[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)>님이 작성:
>
>>>
>>>> Hi, Kent.
>>>>
>>>>
> I checked (1) in your PR, but those test result comments look correct to me.
>>>>
>>>> Please note that both Jenkins and GitHub Action leave the same number of
comments on the same GitHash.
>>>>
>>>> Given that, there are not fake comments. It looks like a real result of
your commits on that PR.
>>>>
>>>>
> GitHash: 23248c3
> <https://github.com/apache/spark/pull/32144#issuecomment-819679970>
(GitHub Action)
> <https://github.com/apache/spark/pull/32144#issuecomment-819647368>
(Jenkins)
>
> GitHash: 8dbed7b
> <https://github.com/apache/spark/pull/32144#issuecomment-819684782>
(GitHub Action)
> <https://github.com/apache/spark/pull/32144#issuecomment-819578976>
(Jenkins)
>
> GitHash: a3a6c5e
> <https://github.com/apache/spark/pull/32144#issuecomment-819690465>
(GitHub Action)
> <https://github.com/apache/spark/pull/32144#issuecomment-819793557>
(Jenkins)
>
> GitHash: b6d26b7
> <https://github.com/apache/spark/pull/32144#issuecomment-819691416>
(GitHub Action)
>>>>
>>>> <https://github.com/apache/spark/pull/32144#issuecomment-819791485>
(Jenkins)
>>>>
>>>>
>
>>>>
>>>> Could you recheck it?
>>>>
>>>>
>
>>>>
>>>>
>
>>>>
>>>>> 1\. Github-actions notification could be wrong when another PR opened
> with some same commits, and you will get a lot of fake comments then.
> Meanwhile, the new PR get no comments, even if it is actually the
> chosen one.
> 1.1 <https://github.com/apache/spark/pull/32144#issuecomment-819679970>
>
>>>>
>>>>
>
>>>>
>>>>
>
>>>>
>>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)> wrote:
>
>>>>
>>>>> Hi ALL, here is something I notice after this change:
>
> 1\. Github-actions notification could be wrong when another PR opened
> with some same commits, and you will get a lot of fake comments then.
> Meanwhile, the new PR get no comments, even if it is actually the
> chosen one.
> 1.1 <https://github.com/apache/spark/pull/32144#issuecomment-819679970>
> 2\. New Forks have to turn on GitHub action by the fork owner manually
> 3\. `Notify test workflow` keeps waiting when the build flow canceled
> or the whole fork gone
> 4\. After refreshed master or even re-forked :(, I still got failures
> and seems not alone
> 4.1. <https://github.com/apache/spark/pull/32168> (PR after sync)
> 4.2. <https://github.com/apache/spark/pull/32172> (PR after re-forked)
> 4.3.
<https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true>
> (some other failures noticed)
>
>
> Bests,
>
> Kent
>
> Dongjoon Hyun <[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)>
于2021年4月14日周三 下午11:34写道：
> >
> > Thank you again, Hyukjin.
> >
> > Bests,
> > Dongjoon.
> >
> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)> wrote:
> >>
> >> Cool, thanks!
> >>
> >> Hyukjin Kwon <[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>
于2021年4月14日周三 下午8:19写道：
> >>>
> >>> Good point! I had to clarify.
> >>> Once is enough. The sync is needed for your branch to include the
changes of <https://github.com/apache/spark/pull/32092>.
> >>>
> >>>
> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao
<[yaooqinn@gmail.com](mailto:yaooqinn@gmail.com)>님이 작성:
> >>>>
> >>>> Hi Hyukjin,
> >>>>
> >>>> > Please sync your branch to the latest master branch in Apache Spark
in order for the main repository to run the workflow and detect it.
> >>>>
> >>>> Do we need to sync master for every PR or just one-time cost to keep
up with the current master branch？
> >>>>
> >>>> Kent Yao
> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> >>>> a spark enthusiast
> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
processing and analytics, built on top of Apache Spark.
> >>>>
> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
Authorization for Apache Spark.
> >>>> spark-postgres A library for reading data from and transferring data
to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
> >>>> spark-func-extrasA library that brings excellent and useful functions
from various modern database management systems to Apache Spark.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On 04/14/2021 15:41，Kent Yao<[yao@apache.org](mailto:yao@apache.org)>
wrote：
> >>>>
> >>>> Cool~Thanks, Hyukjin
> >>>>
> >>>> Yuanjian Li <[xyliyuanjian@gmail.com](mailto:xyliyuanjian@gmail.com)>
于2021年4月14日周三 下午3:39写道：
> >>>>>
> >>>>> Awesome! Thanks for making this happen, Hyukjin!
> >>>>>
> >>>>> Yi Wu <[yi.wu@databricks.com](mailto:yi.wu@databricks.com)>
于2021年4月14日周三 下午2:51写道：
> >>>>>>
> >>>>>> Thanks for the great work, Hyukjin!
> >>>>>>
> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang
<[ltnwgl@gmail.com](mailto:ltnwgl@gmail.com)> wrote:
> >>>>>>>
> >>>>>>> Thanks for the amazing work, Hyukjin!
> >>>>>>> I created a PR for trial and it looks well so far:
<https://github.com/apache/spark/pull/32158>
> >>>>>>>
> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)> wrote:
> >>>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> After <https://github.com/apache/spark/pull/32092> merged, now we
run the GitHub Actions
> >>>>>>>> workflows in your forked repository.
> >>>>>>>>
> >>>>>>>> In short, please see this example HyukjinKwon#34
> >>>>>>>>
> >>>>>>>> You create a PR and your repository triggers the workflow. Your PR
uses the resources allocated to you for testing.
> >>>>>>>> Apache Spark repository finds your workflow, and links it in a
comment in your PR
> >>>>>>>>
> >>>>>>>> Please let me know if you guys find any weird behaviour related to
this.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What does that mean to contributors?
> >>>>>>>>
> >>>>>>>> Please sync your branch to the latest master branch in Apache
Spark in order for your forked repository to run the workflow, and
> >>>>>>>> for the main repository to detect the workflow.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What does that mean to committers?
> >>>>>>>>
> >>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
builds are running (in contributor's forked repository).
> >>>>>>>> Please check the build notified by github-actions bot before
merging it.
> >>>>>>>> There would be a followup work to reflect the status of the forked
repository's build to the status of PR.
> >>>>>>>>
> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
> >>>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> After <https://github.com/apache/spark/pull/32092> merged, now we
run the GitHub Actions
> >>>>>>>>> workflows in your forked repository.
> >>>>>>>>>
> >>>>>>>>> In short, please see this example HyukjinKwon#34
> >>>>>>>>>
> >>>>>>>>> You create a PR and your repository triggers the workflow. Your
PR uses the resources allocated to you for testing.
> >>>>>>>>> Apache Spark repository finds your workflow, and links it in a
comment in your PR
> >>>>>>>>>
> >>>>>>>>> Please let me know if you guys find any weird behaviour related
to this.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What does that mean to contributors?
> >>>>>>>>>
> >>>>>>>>> Please sync your branch to the latest master branch in Apache
Spark in order for the main repository to run the workflow and detect it.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What does that mean to committers?
> >>>>>>>>>
> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
builds are running (in contributor's forked repository). Please check the
build notified by github-actions bot before merging it.
> >>>>>>>>> There would be a followup work to reflect the status of the
forked repository's build to
> >>>>>>>>> the status of PR.
> >>>>>>>>>
> >>>>>>>>>
> >>
> >>
> >> \--
> >>
> >>
>
>
>
> \--
>
> Kent Yao
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> a spark enthusiast
> kyuubiis a unified multi-tenant JDBC interface for large-scale data
> processing and analytics, built on top of Apache Spark.
>
> spark-authorizerA Spark SQL extension which provides SQL Standard
> Authorization for Apache Spark.
> spark-postgres A library for reading data from and transferring data
> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
> itatchiA library that brings useful functions from various modern
> database management systems to Apache Spark.
>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
The issue is fixed now. Please keep monitoring this. Thank you all! The
spark community is super active and cooperative!
2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> The fix will be straightforward. We can either, in Github Actions
> workflow,:
> - remove fast forward option and see if ti works
> - or git rebase before merge the branch
>
> 2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>
>> I think it works mostly correctly as Dongjoon investigated and shared
>> (Thanks a lot!).
>> One problem seems to be syncing to the master seems too strict (
>> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
>> Thanks Yikun.
>> I think we should make it less strict. I can create a PR right away but
>> would like to encourage Yikun or Kent to do it in order to keep the credits
>> of their investigation.
>>
>> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이 작성:
>>
>>> Hi, Kent.
>>>
>>> I checked (1) in your PR, but those test result comments look correct to
>>> me.
>>> Please note that both Jenkins and GitHub Action leave the same number of
>>> comments on the same GitHash.
>>> Given that, there are not fake comments. It looks like a real result of
>>> your commits on that PR.
>>>
>>> GitHash: 23248c3
>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>> (GitHub Action)
>>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>>> (Jenkins)
>>>
>>> GitHash: 8dbed7b
>>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>>> (GitHub Action)
>>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>>> (Jenkins)
>>>
>>> GitHash: a3a6c5e
>>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>>> (GitHub Action)
>>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>>> (Jenkins)
>>>
>>> GitHash: b6d26b7
>>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>>> (GitHub Action)
>>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>>> (Jenkins)
>>>
>>> Could you recheck it?
>>>
>>>
>>> 1. Github-actions notification could be wrong when another PR opened
>>>> with some same commits, and you will get a lot of fake comments then.
>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>> chosen one.
>>>> 1.1
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>>
>>>
>>>
>>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
>>>
>>>> Hi ALL, here is something I notice after this change:
>>>>
>>>> 1. Github-actions notification could be wrong when another PR opened
>>>> with some same commits, and you will get a lot of fake comments then.
>>>> Meanwhile, the new PR get no comments, even if it is actually the
>>>> chosen one.
>>>> 1.1
>>>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>> 2. New Forks have to turn on GitHub action by the fork owner manually
>>>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>>>> or the whole fork gone
>>>> 4. After refreshed master or even re-forked :(, I still got failures
>>>> and seems not alone
>>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after re-forked)
>>>> 4.3.
>>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>>> (some other failures noticed)
>>>>
>>>>
>>>> Bests,
>>>>
>>>> Kent
>>>>
>>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>>> >
>>>> > Thank you again, Hyukjin.
>>>> >
>>>> > Bests,
>>>> > Dongjoon.
>>>> >
>>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
>>>> >>
>>>> >> Cool, thanks!
>>>> >>
>>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>> >>>
>>>> >>> Good point! I had to clarify.
>>>> >>> Once is enough. The sync is needed for your branch to include the
>>>> changes of https://github.com/apache/spark/pull/32092.
>>>> >>>
>>>> >>>
>>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>> >>>>
>>>> >>>> Hi Hyukjin,
>>>> >>>>
>>>> >>>> > Please sync your branch to the latest master branch in Apache
>>>> Spark in order for the main repository to run the workflow and detect it.
>>>> >>>>
>>>> >>>> Do we need to sync master for every PR or just one-time cost to
>>>> keep up with the current master branch？
>>>> >>>>
>>>> >>>> Kent Yao
>>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>> >>>> a spark enthusiast
>>>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale
>>>> data processing and analytics, built on top of Apache Spark.
>>>> >>>>
>>>> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>> Authorization for Apache Spark.
>>>> >>>> spark-postgres A library for reading data from and transferring
>>>> data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>> >>>> spark-func-extrasA library that brings excellent and useful
>>>> functions from various modern database management systems to Apache Spark.
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>> >>>>
>>>> >>>> Cool~Thanks, Hyukjin
>>>> >>>>
>>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>> >>>>>
>>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>> >>>>>
>>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>> >>>>>>
>>>> >>>>>> Thanks for the great work, Hyukjin!
>>>> >>>>>>
>>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com>
>>>> wrote:
>>>> >>>>>>>
>>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>>> >>>>>>> I created a PR for trial and it looks well so far:
>>>> https://github.com/apache/spark/pull/32158
>>>> >>>>>>>
>>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>>> gurwls223@gmail.com> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>> Hi all,
>>>> >>>>>>>>
>>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now
>>>> we run the GitHub Actions
>>>> >>>>>>>> workflows in your forked repository.
>>>> >>>>>>>>
>>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>>> >>>>>>>>
>>>> >>>>>>>> You create a PR and your repository triggers the workflow.
>>>> Your PR uses the resources allocated to you for testing.
>>>> >>>>>>>> Apache Spark repository finds your workflow, and links it in a
>>>> comment in your PR
>>>> >>>>>>>>
>>>> >>>>>>>> Please let me know if you guys find any weird behaviour
>>>> related to this.
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> What does that mean to contributors?
>>>> >>>>>>>>
>>>> >>>>>>>> Please sync your branch to the latest master branch in Apache
>>>> Spark in order for your forked repository to run the workflow, and
>>>> >>>>>>>> for the main repository to detect the workflow.
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> What does that mean to committers?
>>>> >>>>>>>>
>>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>>>> builds are running (in contributor's forked repository).
>>>> >>>>>>>> Please check the build notified by github-actions bot before
>>>> merging it.
>>>> >>>>>>>> There would be a followup work to reflect the status of the
>>>> forked repository's build to the status of PR.
>>>> >>>>>>>>
>>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
>>>> 작성:
>>>> >>>>>>>>>
>>>> >>>>>>>>> Hi all,
>>>> >>>>>>>>>
>>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now
>>>> we run the GitHub Actions
>>>> >>>>>>>>> workflows in your forked repository.
>>>> >>>>>>>>>
>>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>>> >>>>>>>>>
>>>> >>>>>>>>> You create a PR and your repository triggers the workflow.
>>>> Your PR uses the resources allocated to you for testing.
>>>> >>>>>>>>> Apache Spark repository finds your workflow, and links it in
>>>> a comment in your PR
>>>> >>>>>>>>>
>>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>>> related to this.
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> What does that mean to contributors?
>>>> >>>>>>>>>
>>>> >>>>>>>>> Please sync your branch to the latest master branch in Apache
>>>> Spark in order for the main repository to run the workflow and detect it.
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> What does that mean to committers?
>>>> >>>>>>>>>
>>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub
>>>> Actions builds are running (in contributor's forked repository). Please
>>>> check the build notified by github-actions bot before merging it.
>>>> >>>>>>>>> There would be a followup work to reflect the status of the
>>>> forked repository's build to
>>>> >>>>>>>>> the status of PR.
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>
>>>> >>
>>>> >> --
>>>> >>
>>>> >>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Kent Yao
>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>> a spark enthusiast
>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>>> processing and analytics, built on top of Apache Spark.
>>>>
>>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>>> Authorization for Apache Spark.
>>>> spark-postgres A library for reading data from and transferring data
>>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>> itatchiA library that brings useful functions from various modern
>>>> database management systems to Apache Spark.
>>>>
>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
The fix will be straightforward. We can either, in Github Actions workflow,:
- remove fast forward option and see if ti works
- or git rebase before merge the branch
2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> I think it works mostly correctly as Dongjoon investigated and shared
> (Thanks a lot!).
> One problem seems to be syncing to the master seems too strict (
> https://github.com/apache/spark/pull/32168#issuecomment-819736508).
> Thanks Yikun.
> I think we should make it less strict. I can create a PR right away but
> would like to encourage Yikun or Kent to do it in order to keep the credits
> of their investigation.
>
> 2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이 작성:
>
>> Hi, Kent.
>>
>> I checked (1) in your PR, but those test result comments look correct to
>> me.
>> Please note that both Jenkins and GitHub Action leave the same number of
>> comments on the same GitHash.
>> Given that, there are not fake comments. It looks like a real result of
>> your commits on that PR.
>>
>> GitHash: 23248c3
>> https://github.com/apache/spark/pull/32144#issuecomment-819679970
>> (GitHub Action)
>> https://github.com/apache/spark/pull/32144#issuecomment-819647368
>> (Jenkins)
>>
>> GitHash: 8dbed7b
>> https://github.com/apache/spark/pull/32144#issuecomment-819684782
>> (GitHub Action)
>> https://github.com/apache/spark/pull/32144#issuecomment-819578976
>> (Jenkins)
>>
>> GitHash: a3a6c5e
>> https://github.com/apache/spark/pull/32144#issuecomment-819690465
>> (GitHub Action)
>> https://github.com/apache/spark/pull/32144#issuecomment-819793557
>> (Jenkins)
>>
>> GitHash: b6d26b7
>> https://github.com/apache/spark/pull/32144#issuecomment-819691416
>> (GitHub Action)
>> https://github.com/apache/spark/pull/32144#issuecomment-819791485
>> (Jenkins)
>>
>> Could you recheck it?
>>
>>
>> 1. Github-actions notification could be wrong when another PR opened
>>> with some same commits, and you will get a lot of fake comments then.
>>> Meanwhile, the new PR get no comments, even if it is actually the
>>> chosen one.
>>> 1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>>
>>
>>
>> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
>>
>>> Hi ALL, here is something I notice after this change:
>>>
>>> 1. Github-actions notification could be wrong when another PR opened
>>> with some same commits, and you will get a lot of fake comments then.
>>> Meanwhile, the new PR get no comments, even if it is actually the
>>> chosen one.
>>> 1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>> 2. New Forks have to turn on GitHub action by the fork owner manually
>>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>>> or the whole fork gone
>>> 4. After refreshed master or even re-forked :(, I still got failures
>>> and seems not alone
>>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>>> 4.2. https://github.com/apache/spark/pull/32172 (PR after re-forked)
>>> 4.3.
>>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>>> (some other failures noticed)
>>>
>>>
>>> Bests,
>>>
>>> Kent
>>>
>>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>>> >
>>> > Thank you again, Hyukjin.
>>> >
>>> > Bests,
>>> > Dongjoon.
>>> >
>>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
>>> >>
>>> >> Cool, thanks!
>>> >>
>>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>> >>>
>>> >>> Good point! I had to clarify.
>>> >>> Once is enough. The sync is needed for your branch to include the
>>> changes of https://github.com/apache/spark/pull/32092.
>>> >>>
>>> >>>
>>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>> >>>>
>>> >>>> Hi Hyukjin,
>>> >>>>
>>> >>>> > Please sync your branch to the latest master branch in Apache
>>> Spark in order for the main repository to run the workflow and detect it.
>>> >>>>
>>> >>>> Do we need to sync master for every PR or just one-time cost to
>>> keep up with the current master branch？
>>> >>>>
>>> >>>> Kent Yao
>>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>> >>>> a spark enthusiast
>>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>> processing and analytics, built on top of Apache Spark.
>>> >>>>
>>> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>> Authorization for Apache Spark.
>>> >>>> spark-postgres A library for reading data from and transferring
>>> data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>> >>>> spark-func-extrasA library that brings excellent and useful
>>> functions from various modern database management systems to Apache Spark.
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>> >>>>
>>> >>>> Cool~Thanks, Hyukjin
>>> >>>>
>>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>> >>>>>
>>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>>> >>>>>
>>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>> >>>>>>
>>> >>>>>> Thanks for the great work, Hyukjin!
>>> >>>>>>
>>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com>
>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Thanks for the amazing work, Hyukjin!
>>> >>>>>>> I created a PR for trial and it looks well so far:
>>> https://github.com/apache/spark/pull/32158
>>> >>>>>>>
>>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>>> gurwls223@gmail.com> wrote:
>>> >>>>>>>>
>>> >>>>>>>> Hi all,
>>> >>>>>>>>
>>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now
>>> we run the GitHub Actions
>>> >>>>>>>> workflows in your forked repository.
>>> >>>>>>>>
>>> >>>>>>>> In short, please see this example HyukjinKwon#34
>>> >>>>>>>>
>>> >>>>>>>> You create a PR and your repository triggers the workflow. Your
>>> PR uses the resources allocated to you for testing.
>>> >>>>>>>> Apache Spark repository finds your workflow, and links it in a
>>> comment in your PR
>>> >>>>>>>>
>>> >>>>>>>> Please let me know if you guys find any weird behaviour related
>>> to this.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> What does that mean to contributors?
>>> >>>>>>>>
>>> >>>>>>>> Please sync your branch to the latest master branch in Apache
>>> Spark in order for your forked repository to run the workflow, and
>>> >>>>>>>> for the main repository to detect the workflow.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> What does that mean to committers?
>>> >>>>>>>>
>>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>>> builds are running (in contributor's forked repository).
>>> >>>>>>>> Please check the build notified by github-actions bot before
>>> merging it.
>>> >>>>>>>> There would be a followup work to reflect the status of the
>>> forked repository's build to the status of PR.
>>> >>>>>>>>
>>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
>>> 작성:
>>> >>>>>>>>>
>>> >>>>>>>>> Hi all,
>>> >>>>>>>>>
>>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now
>>> we run the GitHub Actions
>>> >>>>>>>>> workflows in your forked repository.
>>> >>>>>>>>>
>>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>>> >>>>>>>>>
>>> >>>>>>>>> You create a PR and your repository triggers the workflow.
>>> Your PR uses the resources allocated to you for testing.
>>> >>>>>>>>> Apache Spark repository finds your workflow, and links it in a
>>> comment in your PR
>>> >>>>>>>>>
>>> >>>>>>>>> Please let me know if you guys find any weird behaviour
>>> related to this.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> What does that mean to contributors?
>>> >>>>>>>>>
>>> >>>>>>>>> Please sync your branch to the latest master branch in Apache
>>> Spark in order for the main repository to run the workflow and detect it.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> What does that mean to committers?
>>> >>>>>>>>>
>>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>>> builds are running (in contributor's forked repository). Please check the
>>> build notified by github-actions bot before merging it.
>>> >>>>>>>>> There would be a followup work to reflect the status of the
>>> forked repository's build to
>>> >>>>>>>>> the status of PR.
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >>
>>>
>>>
>>>
>>> --
>>>
>>> Kent Yao
>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>> a spark enthusiast
>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>>> processing and analytics, built on top of Apache Spark.
>>>
>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>>> Authorization for Apache Spark.
>>> spark-postgres A library for reading data from and transferring data
>>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>> itatchiA library that brings useful functions from various modern
>>> database management systems to Apache Spark.
>>>
>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
I think it works mostly correctly as Dongjoon investigated and shared
(Thanks a lot!).
One problem seems to be syncing to the master seems too strict (
https://github.com/apache/spark/pull/32168#issuecomment-819736508). Thanks
Yikun.
I think we should make it less strict. I can create a PR right away but
would like to encourage Yikun or Kent to do it in order to keep the credits
of their investigation.
2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <do...@gmail.com>님이 작성:
> Hi, Kent.
>
> I checked (1) in your PR, but those test result comments look correct to
> me.
> Please note that both Jenkins and GitHub Action leave the same number of
> comments on the same GitHash.
> Given that, there are not fake comments. It looks like a real result of
> your commits on that PR.
>
> GitHash: 23248c3
> https://github.com/apache/spark/pull/32144#issuecomment-819679970
> (GitHub Action)
> https://github.com/apache/spark/pull/32144#issuecomment-819647368
> (Jenkins)
>
> GitHash: 8dbed7b
> https://github.com/apache/spark/pull/32144#issuecomment-819684782
> (GitHub Action)
> https://github.com/apache/spark/pull/32144#issuecomment-819578976
> (Jenkins)
>
> GitHash: a3a6c5e
> https://github.com/apache/spark/pull/32144#issuecomment-819690465
> (GitHub Action)
> https://github.com/apache/spark/pull/32144#issuecomment-819793557
> (Jenkins)
>
> GitHash: b6d26b7
> https://github.com/apache/spark/pull/32144#issuecomment-819691416
> (GitHub Action)
> https://github.com/apache/spark/pull/32144#issuecomment-819791485
> (Jenkins)
>
> Could you recheck it?
>
>
> 1. Github-actions notification could be wrong when another PR opened
>> with some same commits, and you will get a lot of fake comments then.
>> Meanwhile, the new PR get no comments, even if it is actually the
>> chosen one.
>> 1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
>>
>
>
> On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
>
>> Hi ALL, here is something I notice after this change:
>>
>> 1. Github-actions notification could be wrong when another PR opened
>> with some same commits, and you will get a lot of fake comments then.
>> Meanwhile, the new PR get no comments, even if it is actually the
>> chosen one.
>> 1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
>> 2. New Forks have to turn on GitHub action by the fork owner manually
>> 3. `Notify test workflow` keeps waiting when the build flow canceled
>> or the whole fork gone
>> 4. After refreshed master or even re-forked :(, I still got failures
>> and seems not alone
>> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
>> 4.2. https://github.com/apache/spark/pull/32172 (PR after re-forked)
>> 4.3.
>> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
>> (some other failures noticed)
>>
>>
>> Bests,
>>
>> Kent
>>
>> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>> >
>> > Thank you again, Hyukjin.
>> >
>> > Bests,
>> > Dongjoon.
>> >
>> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
>> >>
>> >> Cool, thanks!
>> >>
>> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>> >>>
>> >>> Good point! I had to clarify.
>> >>> Once is enough. The sync is needed for your branch to include the
>> changes of https://github.com/apache/spark/pull/32092.
>> >>>
>> >>>
>> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>> >>>>
>> >>>> Hi Hyukjin,
>> >>>>
>> >>>> > Please sync your branch to the latest master branch in Apache
>> Spark in order for the main repository to run the workflow and detect it.
>> >>>>
>> >>>> Do we need to sync master for every PR or just one-time cost to keep
>> up with the current master branch？
>> >>>>
>> >>>> Kent Yao
>> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> >>>> a spark enthusiast
>> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>> processing and analytics, built on top of Apache Spark.
>> >>>>
>> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
>> Authorization for Apache Spark.
>> >>>> spark-postgres A library for reading data from and transferring data
>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>> >>>> spark-func-extrasA library that brings excellent and useful
>> functions from various modern database management systems to Apache Spark.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>> >>>>
>> >>>> Cool~Thanks, Hyukjin
>> >>>>
>> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>> >>>>>
>> >>>>> Awesome! Thanks for making this happen, Hyukjin!
>> >>>>>
>> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>> >>>>>>
>> >>>>>> Thanks for the great work, Hyukjin!
>> >>>>>>
>> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com>
>> wrote:
>> >>>>>>>
>> >>>>>>> Thanks for the amazing work, Hyukjin!
>> >>>>>>> I created a PR for trial and it looks well so far:
>> https://github.com/apache/spark/pull/32158
>> >>>>>>>
>> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <
>> gurwls223@gmail.com> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi all,
>> >>>>>>>>
>> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we
>> run the GitHub Actions
>> >>>>>>>> workflows in your forked repository.
>> >>>>>>>>
>> >>>>>>>> In short, please see this example HyukjinKwon#34
>> >>>>>>>>
>> >>>>>>>> You create a PR and your repository triggers the workflow. Your
>> PR uses the resources allocated to you for testing.
>> >>>>>>>> Apache Spark repository finds your workflow, and links it in a
>> comment in your PR
>> >>>>>>>>
>> >>>>>>>> Please let me know if you guys find any weird behaviour related
>> to this.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> What does that mean to contributors?
>> >>>>>>>>
>> >>>>>>>> Please sync your branch to the latest master branch in Apache
>> Spark in order for your forked repository to run the workflow, and
>> >>>>>>>> for the main repository to detect the workflow.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> What does that mean to committers?
>> >>>>>>>>
>> >>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>> builds are running (in contributor's forked repository).
>> >>>>>>>> Please check the build notified by github-actions bot before
>> merging it.
>> >>>>>>>> There would be a followup work to reflect the status of the
>> forked repository's build to the status of PR.
>> >>>>>>>>
>> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
>> 작성:
>> >>>>>>>>>
>> >>>>>>>>> Hi all,
>> >>>>>>>>>
>> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now
>> we run the GitHub Actions
>> >>>>>>>>> workflows in your forked repository.
>> >>>>>>>>>
>> >>>>>>>>> In short, please see this example HyukjinKwon#34
>> >>>>>>>>>
>> >>>>>>>>> You create a PR and your repository triggers the workflow. Your
>> PR uses the resources allocated to you for testing.
>> >>>>>>>>> Apache Spark repository finds your workflow, and links it in a
>> comment in your PR
>> >>>>>>>>>
>> >>>>>>>>> Please let me know if you guys find any weird behaviour related
>> to this.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> What does that mean to contributors?
>> >>>>>>>>>
>> >>>>>>>>> Please sync your branch to the latest master branch in Apache
>> Spark in order for the main repository to run the workflow and detect it.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> What does that mean to committers?
>> >>>>>>>>>
>> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>> builds are running (in contributor's forked repository). Please check the
>> build notified by github-actions bot before merging it.
>> >>>>>>>>> There would be a followup work to reflect the status of the
>> forked repository's build to
>> >>>>>>>>> the status of PR.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>
>> >>
>> >> --
>> >>
>> >>
>>
>>
>>
>> --
>>
>> Kent Yao
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> a spark enthusiast
>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
>> processing and analytics, built on top of Apache Spark.
>>
>> spark-authorizerA Spark SQL extension which provides SQL Standard
>> Authorization for Apache Spark.
>> spark-postgres A library for reading data from and transferring data
>> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>> itatchiA library that brings useful functions from various modern
>> database management systems to Apache Spark.
>>
>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Dongjoon Hyun <do...@gmail.com>.
Hi, Kent.
I checked (1) in your PR, but those test result comments look correct to me.
Please note that both Jenkins and GitHub Action leave the same number of
comments on the same GitHash.
Given that, there are not fake comments. It looks like a real result of
your commits on that PR.
GitHash: 23248c3
https://github.com/apache/spark/pull/32144#issuecomment-819679970
(GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819647368
(Jenkins)
GitHash: 8dbed7b
https://github.com/apache/spark/pull/32144#issuecomment-819684782
(GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819578976
(Jenkins)
GitHash: a3a6c5e
https://github.com/apache/spark/pull/32144#issuecomment-819690465
(GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819793557
(Jenkins)
GitHash: b6d26b7
https://github.com/apache/spark/pull/32144#issuecomment-819691416
(GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819791485
(Jenkins)
Could you recheck it?
1. Github-actions notification could be wrong when another PR opened
> with some same commits, and you will get a lot of fake comments then.
> Meanwhile, the new PR get no comments, even if it is actually the
> chosen one.
> 1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
>
On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <ya...@gmail.com> wrote:
> Hi ALL, here is something I notice after this change:
>
> 1. Github-actions notification could be wrong when another PR opened
> with some same commits, and you will get a lot of fake comments then.
> Meanwhile, the new PR get no comments, even if it is actually the
> chosen one.
> 1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
> 2. New Forks have to turn on GitHub action by the fork owner manually
> 3. `Notify test workflow` keeps waiting when the build flow canceled
> or the whole fork gone
> 4. After refreshed master or even re-forked :(, I still got failures
> and seems not alone
> 4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
> 4.2. https://github.com/apache/spark/pull/32172 (PR after re-forked)
> 4.3.
> https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
> (some other failures noticed)
>
>
> Bests,
>
> Kent
>
> Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
> >
> > Thank you again, Hyukjin.
> >
> > Bests,
> > Dongjoon.
> >
> > On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
> >>
> >> Cool, thanks!
> >>
> >> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
> >>>
> >>> Good point! I had to clarify.
> >>> Once is enough. The sync is needed for your branch to include the
> changes of https://github.com/apache/spark/pull/32092.
> >>>
> >>>
> >>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
> >>>>
> >>>> Hi Hyukjin,
> >>>>
> >>>> > Please sync your branch to the latest master branch in Apache Spark
> in order for the main repository to run the workflow and detect it.
> >>>>
> >>>> Do we need to sync master for every PR or just one-time cost to keep
> up with the current master branch？
> >>>>
> >>>> Kent Yao
> >>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> >>>> a spark enthusiast
> >>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data
> processing and analytics, built on top of Apache Spark.
> >>>>
> >>>> spark-authorizerA Spark SQL extension which provides SQL Standard
> Authorization for Apache Spark.
> >>>> spark-postgres A library for reading data from and transferring data
> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
> >>>> spark-func-extrasA library that brings excellent and useful functions
> from various modern database management systems to Apache Spark.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
> >>>>
> >>>> Cool~Thanks, Hyukjin
> >>>>
> >>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
> >>>>>
> >>>>> Awesome! Thanks for making this happen, Hyukjin!
> >>>>>
> >>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
> >>>>>>
> >>>>>> Thanks for the great work, Hyukjin!
> >>>>>>
> >>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com>
> wrote:
> >>>>>>>
> >>>>>>> Thanks for the amazing work, Hyukjin!
> >>>>>>> I created a PR for trial and it looks well so far:
> https://github.com/apache/spark/pull/32158
> >>>>>>>
> >>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com>
> wrote:
> >>>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we
> run the GitHub Actions
> >>>>>>>> workflows in your forked repository.
> >>>>>>>>
> >>>>>>>> In short, please see this example HyukjinKwon#34
> >>>>>>>>
> >>>>>>>> You create a PR and your repository triggers the workflow. Your
> PR uses the resources allocated to you for testing.
> >>>>>>>> Apache Spark repository finds your workflow, and links it in a
> comment in your PR
> >>>>>>>>
> >>>>>>>> Please let me know if you guys find any weird behaviour related
> to this.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What does that mean to contributors?
> >>>>>>>>
> >>>>>>>> Please sync your branch to the latest master branch in Apache
> Spark in order for your forked repository to run the workflow, and
> >>>>>>>> for the main repository to detect the workflow.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What does that mean to committers?
> >>>>>>>>
> >>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
> builds are running (in contributor's forked repository).
> >>>>>>>> Please check the build notified by github-actions bot before
> merging it.
> >>>>>>>> There would be a followup work to reflect the status of the
> forked repository's build to the status of PR.
> >>>>>>>>
> >>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이
> 작성:
> >>>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we
> run the GitHub Actions
> >>>>>>>>> workflows in your forked repository.
> >>>>>>>>>
> >>>>>>>>> In short, please see this example HyukjinKwon#34
> >>>>>>>>>
> >>>>>>>>> You create a PR and your repository triggers the workflow. Your
> PR uses the resources allocated to you for testing.
> >>>>>>>>> Apache Spark repository finds your workflow, and links it in a
> comment in your PR
> >>>>>>>>>
> >>>>>>>>> Please let me know if you guys find any weird behaviour related
> to this.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What does that mean to contributors?
> >>>>>>>>>
> >>>>>>>>> Please sync your branch to the latest master branch in Apache
> Spark in order for the main repository to run the workflow and detect it.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What does that mean to committers?
> >>>>>>>>>
> >>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
> builds are running (in contributor's forked repository). Please check the
> build notified by github-actions bot before merging it.
> >>>>>>>>> There would be a followup work to reflect the status of the
> forked repository's build to
> >>>>>>>>> the status of PR.
> >>>>>>>>>
> >>>>>>>>>
> >>
> >>
> >> --
> >>
> >>
>
>
>
> --
>
> Kent Yao
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> a spark enthusiast
> kyuubiis a unified multi-tenant JDBC interface for large-scale data
> processing and analytics, built on top of Apache Spark.
>
> spark-authorizerA Spark SQL extension which provides SQL Standard
> Authorization for Apache Spark.
> spark-postgres A library for reading data from and transferring data
> to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
> itatchiA library that brings useful functions from various modern
> database management systems to Apache Spark.
>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Kent Yao <ya...@gmail.com>.
Hi ALL, here is something I notice after this change:
1. Github-actions notification could be wrong when another PR opened
with some same commits, and you will get a lot of fake comments then.
Meanwhile, the new PR get no comments, even if it is actually the
chosen one.
1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
2. New Forks have to turn on GitHub action by the fork owner manually
3. `Notify test workflow` keeps waiting when the build flow canceled
or the whole fork gone
4. After refreshed master or even re-forked :(, I still got failures
and seems not alone
4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
4.2. https://github.com/apache/spark/pull/32172 (PR after re-forked)
4.3. https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
(some other failures noticed)
Bests,
Kent
Dongjoon Hyun <do...@gmail.com> 于2021年4月14日周三 下午11:34写道：
>
> Thank you again, Hyukjin.
>
> Bests,
> Dongjoon.
>
> On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
>>
>> Cool, thanks!
>>
>> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>>>
>>> Good point! I had to clarify.
>>> Once is enough. The sync is needed for your branch to include the changes of https://github.com/apache/spark/pull/32092.
>>>
>>>
>>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>>>
>>>> Hi Hyukjin,
>>>>
>>>> > Please sync your branch to the latest master branch in Apache Spark in order for the main repository to run the workflow and detect it.
>>>>
>>>> Do we need to sync master for every PR or just one-time cost to keep up with the current master branch？
>>>>
>>>> Kent Yao
>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>> a spark enthusiast
>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark.
>>>>
>>>> spark-authorizerA Spark SQL extension which provides SQL Standard Authorization for Apache Spark.
>>>> spark-postgres A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>> spark-func-extrasA library that brings excellent and useful functions from various modern database management systems to Apache Spark.
>>>>
>>>>
>>>>
>>>>
>>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> wrote：
>>>>
>>>> Cool~Thanks, Hyukjin
>>>>
>>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>>>
>>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>
>>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>>>
>>>>>> Thanks for the great work, Hyukjin!
>>>>>>
>>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com> wrote:
>>>>>>>
>>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>>> I created a PR for trial and it looks well so far: https://github.com/apache/spark/pull/32158
>>>>>>>
>>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run the GitHub Actions
>>>>>>>> workflows in your forked repository.
>>>>>>>>
>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>
>>>>>>>> You create a PR and your repository triggers the workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>> Apache Spark repository finds your workflow, and links it in a comment in your PR
>>>>>>>>
>>>>>>>> Please let me know if you guys find any weird behaviour related to this.
>>>>>>>>
>>>>>>>>
>>>>>>>> What does that mean to contributors?
>>>>>>>>
>>>>>>>> Please sync your branch to the latest master branch in Apache Spark in order for your forked repository to run the workflow, and
>>>>>>>> for the main repository to detect the workflow.
>>>>>>>>
>>>>>>>>
>>>>>>>> What does that mean to committers?
>>>>>>>>
>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds are running (in contributor's forked repository).
>>>>>>>> Please check the build notified by github-actions bot before merging it.
>>>>>>>> There would be a followup work to reflect the status of the forked repository's build to the status of PR.
>>>>>>>>
>>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run the GitHub Actions
>>>>>>>>> workflows in your forked repository.
>>>>>>>>>
>>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>>
>>>>>>>>> You create a PR and your repository triggers the workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>>> Apache Spark repository finds your workflow, and links it in a comment in your PR
>>>>>>>>>
>>>>>>>>> Please let me know if you guys find any weird behaviour related to this.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What does that mean to contributors?
>>>>>>>>>
>>>>>>>>> Please sync your branch to the latest master branch in Apache Spark in order for the main repository to run the workflow and detect it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What does that mean to committers?
>>>>>>>>>
>>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds are running (in contributor's forked repository). Please check the build notified by github-actions bot before merging it.
>>>>>>>>> There would be a followup work to reflect the status of the forked repository's build to
>>>>>>>>> the status of PR.
>>>>>>>>>
>>>>>>>>>
>>
>>
>> --
>>
>>
--
Kent Yao
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
a spark enthusiast
kyuubiis a unified multi-tenant JDBC interface for large-scale data
processing and analytics, built on top of Apache Spark.
spark-authorizerA Spark SQL extension which provides SQL Standard
Authorization for Apache Spark.
spark-postgres A library for reading data from and transferring data
to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
itatchiA library that brings useful functions from various modern
database management systems to Apache Spark.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you again, Hyukjin.
Bests,
Dongjoon.
On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <ya...@gmail.com> wrote:
> Cool, thanks!
>
> Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
>
>> Good point! I had to clarify.
>> Once is enough. The sync is needed for your branch to include the changes
>> of https://github.com/apache/spark/pull/32092.
>>
>>
>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>>
>>> Hi Hyukjin,
>>>
>>> > Please sync your branch to the latest master branch in Apache Spark in
>>> order for the main repository to run the workflow and detect it.
>>>
>>> Do we need to sync master for every PR or just one-time cost to keep up
>>> with the current master branch？
>>>
>>> *Kent Yao *
>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>> *a spark enthusiast*
>>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>>> unified multi-tenant JDBC interface for large-scale data processing and
>>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>>> SQL extension which provides SQL Standard Authorization for **Apache
>>> Spark <http://spark.apache.org/>.*
>>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>>> for reading data from and transferring data to Postgres / Greenplum with
>>> Spark SQL and DataFrames, 10~100x faster.*
>>> *spark-func-extras <https://github.com/yaooqinn/spark-func-extras>A
>>> library that brings excellent and useful functions from various modern
>>> database management systems to Apache Spark <http://spark.apache.org/>.*
>>>
>>>
>>>
>>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> <ya...@apache.org> wrote：
>>>
>>> Cool~Thanks, Hyukjin
>>>
>>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>>
>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>
>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>>
>>>>> Thanks for the great work, Hyukjin!
>>>>>
>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>> I created a PR for trial and it looks well so far:
>>>>>> https://github.com/apache/spark/pull/32158
>>>>>>
>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>>>>> the GitHub Actions
>>>>>>> workflows in your forked repository.
>>>>>>>
>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>>>
>>>>>>> 1. You create a PR and your repository triggers the workflow.
>>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>>> 2. Apache Spark repository finds your workflow, and links it in
>>>>>>> a comment in your PR
>>>>>>>
>>>>>>> Please let me know if you guys find any weird behaviour related to
>>>>>>> this.
>>>>>>>
>>>>>>>
>>>>>>> *What does that mean to contributors?*
>>>>>>>
>>>>>>> Please sync your branch to the latest master branch in Apache Spark
>>>>>>> in order for your forked repository to run the workflow, and
>>>>>>> for the main repository to detect the workflow.
>>>>>>>
>>>>>>>
>>>>>>> *What does that mean to committers?*
>>>>>>>
>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>>>>>>> builds are running (in contributor's forked repository).
>>>>>>> Please check the build notified by github-actions bot before merging
>>>>>>> it.
>>>>>>> There would be a followup work to reflect the status of the forked
>>>>>>> repository's build to the status of PR.
>>>>>>>
>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we
>>>>>>>> run the GitHub Actions
>>>>>>>> workflows in your forked repository.
>>>>>>>>
>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>>>>
>>>>>>>> 1. You create a PR and your repository triggers the workflow.
>>>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>>>> 2. Apache Spark repository finds your workflow, and links it in
>>>>>>>> a comment in your PR
>>>>>>>>
>>>>>>>> Please let me know if you guys find any weird behaviour related to
>>>>>>>> this.
>>>>>>>>
>>>>>>>>
>>>>>>>> *What does that mean to contributors?*
>>>>>>>>
>>>>>>>> Please sync your branch to the latest master branch in Apache Spark
>>>>>>>> in order for the main repository to run the workflow and detect it.
>>>>>>>>
>>>>>>>>
>>>>>>>> *What does that mean to committers?*
>>>>>>>>
>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>>>>>>>> builds are running (in contributor's forked repository). Please check the
>>>>>>>> build notified by github-actions bot before merging it.
>>>>>>>> There would be a followup work to reflect the status of the forked
>>>>>>>> repository's build to
>>>>>>>> the status of PR.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>
> --
>
>
>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Kent Yao <ya...@gmail.com>.
Cool, thanks!
Hyukjin Kwon <gu...@gmail.com> 于2021年4月14日周三 下午8:19写道：
> Good point! I had to clarify.
> Once is enough. The sync is needed for your branch to include the changes
> of https://github.com/apache/spark/pull/32092.
>
>
> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
>
>> Hi Hyukjin,
>>
>> > Please sync your branch to the latest master branch in Apache Spark in
>> order for the main repository to run the workflow and detect it.
>>
>> Do we need to sync master for every PR or just one-time cost to keep up
>> with the current master branch？
>>
>> *Kent Yao *
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> *a spark enthusiast*
>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a
>> unified multi-tenant JDBC interface for large-scale data processing and
>> analytics, built on top of Apache Spark <http://spark.apache.org/>.*
>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
>> SQL extension which provides SQL Standard Authorization for **Apache
>> Spark <http://spark.apache.org/>.*
>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
>> for reading data from and transferring data to Postgres / Greenplum with
>> Spark SQL and DataFrames, 10~100x faster.*
>> *spark-func-extras <https://github.com/yaooqinn/spark-func-extras>A
>> library that brings excellent and useful functions from various modern
>> database management systems to Apache Spark <http://spark.apache.org/>.*
>>
>>
>>
>> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> <ya...@apache.org> wrote：
>>
>> Cool~Thanks, Hyukjin
>>
>> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>>
>>> Awesome! Thanks for making this happen, Hyukjin!
>>>
>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>>
>>>> Thanks for the great work, Hyukjin!
>>>>
>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks for the amazing work, Hyukjin!
>>>>> I created a PR for trial and it looks well so far:
>>>>> https://github.com/apache/spark/pull/32158
>>>>>
>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>>>> the GitHub Actions
>>>>>> workflows in your forked repository.
>>>>>>
>>>>>> In short, please see this example HyukjinKwon#34
>>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>>
>>>>>> 1. You create a PR and your repository triggers the workflow.
>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>> 2. Apache Spark repository finds your workflow, and links it in a
>>>>>> comment in your PR
>>>>>>
>>>>>> Please let me know if you guys find any weird behaviour related to
>>>>>> this.
>>>>>>
>>>>>>
>>>>>> *What does that mean to contributors?*
>>>>>>
>>>>>> Please sync your branch to the latest master branch in Apache Spark
>>>>>> in order for your forked repository to run the workflow, and
>>>>>> for the main repository to detect the workflow.
>>>>>>
>>>>>>
>>>>>> *What does that mean to committers?*
>>>>>>
>>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>>>>> are running (in contributor's forked repository).
>>>>>> Please check the build notified by github-actions bot before merging
>>>>>> it.
>>>>>> There would be a followup work to reflect the status of the forked
>>>>>> repository's build to the status of PR.
>>>>>>
>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>>>>> the GitHub Actions
>>>>>>> workflows in your forked repository.
>>>>>>>
>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>>>
>>>>>>> 1. You create a PR and your repository triggers the workflow.
>>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>>> 2. Apache Spark repository finds your workflow, and links it in
>>>>>>> a comment in your PR
>>>>>>>
>>>>>>> Please let me know if you guys find any weird behaviour related to
>>>>>>> this.
>>>>>>>
>>>>>>>
>>>>>>> *What does that mean to contributors?*
>>>>>>>
>>>>>>> Please sync your branch to the latest master branch in Apache Spark
>>>>>>> in order for the main repository to run the workflow and detect it.
>>>>>>>
>>>>>>>
>>>>>>> *What does that mean to committers?*
>>>>>>>
>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions
>>>>>>> builds are running (in contributor's forked repository). Please check the
>>>>>>> build notified by github-actions bot before merging it.
>>>>>>> There would be a followup work to reflect the status of the forked
>>>>>>> repository's build to
>>>>>>> the status of PR.
>>>>>>>
>>>>>>>
>>>>>>>
--
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
Good point! I had to clarify.
Once is enough. The sync is needed for your branch to include the changes
of https://github.com/apache/spark/pull/32092.
2021년 4월 14일 (수) 오후 9:11, Kent Yao <ya...@gmail.com>님이 작성:
> Hi Hyukjin,
>
> > Please sync your branch to the latest master branch in Apache Spark in
> order for the main repository to run the workflow and detect it.
>
> Do we need to sync master for every PR or just one-time cost to keep up
> with the current master branch？
>
> *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> *a spark enthusiast*
> *kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top
> of Apache Spark <http://spark.apache.org/>.*
> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> SQL extension which provides SQL Standard Authorization for **Apache
> Spark <http://spark.apache.org/>.*
> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.*
> *spark-func-extras <https://github.com/yaooqinn/spark-func-extras>A
> library that brings excellent and useful functions from various modern
> database management systems to Apache Spark <http://spark.apache.org/>.*
>
>
>
> On 04/14/2021 15:41，Kent Yao<ya...@apache.org> <ya...@apache.org> wrote：
>
> Cool~Thanks, Hyukjin
>
> Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
>
>> Awesome! Thanks for making this happen, Hyukjin!
>>
>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>>
>>> Thanks for the great work, Hyukjin!
>>>
>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com> wrote:
>>>
>>>> Thanks for the amazing work, Hyukjin!
>>>> I created a PR for trial and it looks well so far:
>>>> https://github.com/apache/spark/pull/32158
>>>>
>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>>> the GitHub Actions
>>>>> workflows in your forked repository.
>>>>>
>>>>> In short, please see this example HyukjinKwon#34
>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>
>>>>> 1. You create a PR and your repository triggers the workflow. Your
>>>>> PR uses the resources allocated to you for testing.
>>>>> 2. Apache Spark repository finds your workflow, and links it in a
>>>>> comment in your PR
>>>>>
>>>>> Please let me know if you guys find any weird behaviour related to
>>>>> this.
>>>>>
>>>>>
>>>>> *What does that mean to contributors?*
>>>>>
>>>>> Please sync your branch to the latest master branch in Apache Spark in
>>>>> order for your forked repository to run the workflow, and
>>>>> for the main repository to detect the workflow.
>>>>>
>>>>>
>>>>> *What does that mean to committers?*
>>>>>
>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>>>> are running (in contributor's forked repository).
>>>>> Please check the build notified by github-actions bot before merging
>>>>> it.
>>>>> There would be a followup work to reflect the status of the forked
>>>>> repository's build to the status of PR.
>>>>>
>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>>>> the GitHub Actions
>>>>>> workflows in your forked repository.
>>>>>>
>>>>>> In short, please see this example HyukjinKwon#34
>>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>>
>>>>>> 1. You create a PR and your repository triggers the workflow.
>>>>>> Your PR uses the resources allocated to you for testing.
>>>>>> 2. Apache Spark repository finds your workflow, and links it in a
>>>>>> comment in your PR
>>>>>>
>>>>>> Please let me know if you guys find any weird behaviour related to
>>>>>> this.
>>>>>>
>>>>>>
>>>>>> *What does that mean to contributors?*
>>>>>>
>>>>>> Please sync your branch to the latest master branch in Apache Spark
>>>>>> in order for the main repository to run the workflow and detect it.
>>>>>>
>>>>>>
>>>>>> *What does that mean to committers?*
>>>>>>
>>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>>>>> are running (in contributor's forked repository). Please check the build
>>>>>> notified by github-actions bot before merging it.
>>>>>> There would be a followup work to reflect the status of the forked
>>>>>> repository's build to
>>>>>> the status of PR.
>>>>>>
>>>>>>
>>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your forked repository
Posted by Kent Yao <ya...@gmail.com>.
Hi Hyukjin,
> Please sync your branch to the latest master branch in Apache Spark in order
for the main repository to run the workflow and detect it.
Do we need to sync master for every PR or just one-time cost to keep up with
the current master branch？
**Kent Yao **
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
** _a s_ _park enthusiast_**
_[kyuubi](https://github.com/yaooqinn/kyuubi) is a unified multi-tenant JDBC
interface for large-scale data processing and analytics, built on top of
[Apache Spark](http://spark.apache.org/)._
#
_[spark-authorizer](https://github.com/yaooqinn/spark-authorizer) A Spark SQL
extension which provides SQL Standard Authorization for _ _[Apache
Spark](http://spark.apache.org/)._
_ _[spark-postgres](https://github.com/yaooqinn/spark-postgres) A library
for reading data from and transferring data to Postgres / Greenplum with Spark
SQL and DataFrames, 10~100x faster.__
_ _[spark-func-extras](https://github.com/yaooqinn/spark-func-extras) A
library that brings excellent and useful functions from various modern
database management systems to _ _[Apache Spark](http://spark.apache.org/).__
#
On 04/14/2021 15:41，[Kent Yao<ya...@apache.org>](mailto:yao@apache.org) wrote：
> Cool~Thanks, Hyukjin
>
>
>
>
> Yuanjian Li <[xyliyuanjian@gmail.com](mailto:xyliyuanjian@gmail.com)>
于2021年4月14日周三 下午3:39写道：
>
>
>> Awesome! Thanks for making this happen, Hyukjin!
>>
>>
>
>>
>> Yi Wu <[yi.wu@databricks.com](mailto:yi.wu@databricks.com)> 于2021年4月14日周三
下午2:51写道：
>
>>
>>> Thanks for the great work, Hyukjin!
>>>
>>>
>
>>>
>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang
<[ltnwgl@gmail.com](mailto:ltnwgl@gmail.com)> wrote:
>
>>>
>>>> Thanks for the amazing work, Hyukjin!
>>>>
>>>> I created a PR for trial and it looks well so far:
<https://github.com/apache/spark/pull/32158>
>>>>
>>>>
>
>>>>
>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)> wrote:
>
>>>>
>>>>> Hi all,
>
> After <https://github.com/apache/spark/pull/32092> merged, now we run the
GitHub Actions
> workflows in your forked repository.
>
>
>>>>>
>>>>> In short, please see this example
[HyukjinKwon#34](https://github.com/HyukjinKwon/spark/pull/34)
>>>>>
>>>>> 1. You create a PR and your repository triggers the workflow. Your PR
uses the resources allocated to you for testing.
>>>>> 2. Apache Spark repository finds your workflow, and links it in a
comment in your PR
>
>>>>>
>>>>>
>>>>> Please let me know if you guys find any weird behaviour related to this.
>
>
> **What does that mean to contributors?**
>>>>>
>>>>>
> Please sync your branch to the latest master branch in Apache Spark in
order for your forked repository to run the workflow, and
> for the main repository to detect the workflow.
>
>
>
>>>>>
>>>>> **What does that mean to committers?**
>
> Now, GitHub Actions will show a green even when GitHub Actions builds are
running (in contributor's forked repository).
> Please check the build notified by github-actions bot before merging it.
> There would be a followup work to reflect the status of the forked
repository's build to the status of PR.
>>>>>
>>>>>
>
>>>>>
>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>님이 작성:
>
>>>>>
>>>>>> Hi all,
>
> After <https://github.com/apache/spark/pull/32092> merged, now we run the
GitHub Actions
> workflows in your forked repository.
>
>
>>>>>>
>>>>>> In short, please see this example
[HyukjinKwon#34](https://github.com/HyukjinKwon/spark/pull/34)
>>>>>>
>>>>>> 1. You create a PR and your repository triggers the workflow. Your PR
uses the resources allocated to you for testing.
>>>>>> 2. Apache Spark repository finds your workflow, and links it in a
comment in your PR
>
>>>>>>
>>>>>>
>>>>>> Please let me know if you guys find any weird behaviour related to
this.
>
>
> **What does that mean to contributors?**
>>>>>>
>>>>>>
> Please sync your branch to the latest master branch in Apache Spark in
order for the main repository to run the workflow and detect it.
>
>
>
>>>>>>
>>>>>> **What does that mean to committers?**
>
> Now, GitHub Actions will show a green even when GitHub Actions builds are
running (in contributor's forked repository). Please check the build notified
by github-actions bot before merging it.
> There would be a followup work to reflect the status of the forked
repository's build to
> the status of PR.
>
>
>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Kent Yao <ya...@apache.org>.
Cool~Thanks, Hyukjin
Yuanjian Li <xy...@gmail.com> 于2021年4月14日周三 下午3:39写道：
> Awesome! Thanks for making this happen, Hyukjin!
>
> Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
>
>> Thanks for the great work, Hyukjin!
>>
>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com> wrote:
>>
>>> Thanks for the amazing work, Hyukjin!
>>> I created a PR for trial and it looks well so far:
>>> https://github.com/apache/spark/pull/32158
>>>
>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>> the GitHub Actions
>>>> workflows in your forked repository.
>>>>
>>>> In short, please see this example HyukjinKwon#34
>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>
>>>> 1. You create a PR and your repository triggers the workflow. Your
>>>> PR uses the resources allocated to you for testing.
>>>> 2. Apache Spark repository finds your workflow, and links it in a
>>>> comment in your PR
>>>>
>>>> Please let me know if you guys find any weird behaviour related to this.
>>>>
>>>>
>>>> *What does that mean to contributors?*
>>>>
>>>> Please sync your branch to the latest master branch in Apache Spark in
>>>> order for your forked repository to run the workflow, and
>>>> for the main repository to detect the workflow.
>>>>
>>>>
>>>> *What does that mean to committers?*
>>>>
>>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>>> are running (in contributor's forked repository).
>>>> Please check the build notified by github-actions bot before merging it.
>>>> There would be a followup work to reflect the status of the forked
>>>> repository's build to the status of PR.
>>>>
>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>>
>>>>> Hi all,
>>>>>
>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>>> the GitHub Actions
>>>>> workflows in your forked repository.
>>>>>
>>>>> In short, please see this example HyukjinKwon#34
>>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>>
>>>>> 1. You create a PR and your repository triggers the workflow. Your
>>>>> PR uses the resources allocated to you for testing.
>>>>> 2. Apache Spark repository finds your workflow, and links it in a
>>>>> comment in your PR
>>>>>
>>>>> Please let me know if you guys find any weird behaviour related to
>>>>> this.
>>>>>
>>>>>
>>>>> *What does that mean to contributors?*
>>>>>
>>>>> Please sync your branch to the latest master branch in Apache Spark in
>>>>> order for the main repository to run the workflow and detect it.
>>>>>
>>>>>
>>>>> *What does that mean to committers?*
>>>>>
>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>>>> are running (in contributor's forked repository). Please check the build
>>>>> notified by github-actions bot before merging it.
>>>>> There would be a followup work to reflect the status of the forked
>>>>> repository's build to
>>>>> the status of PR.
>>>>>
>>>>>
>>>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Yuanjian Li <xy...@gmail.com>.
Awesome! Thanks for making this happen, Hyukjin!
Yi Wu <yi...@databricks.com> 于2021年4月14日周三 下午2:51写道：
> Thanks for the great work, Hyukjin!
>
> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com> wrote:
>
>> Thanks for the amazing work, Hyukjin!
>> I created a PR for trial and it looks well so far:
>> https://github.com/apache/spark/pull/32158
>>
>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>>> GitHub Actions
>>> workflows in your forked repository.
>>>
>>> In short, please see this example HyukjinKwon#34
>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>
>>> 1. You create a PR and your repository triggers the workflow. Your
>>> PR uses the resources allocated to you for testing.
>>> 2. Apache Spark repository finds your workflow, and links it in a
>>> comment in your PR
>>>
>>> Please let me know if you guys find any weird behaviour related to this.
>>>
>>>
>>> *What does that mean to contributors?*
>>>
>>> Please sync your branch to the latest master branch in Apache Spark in
>>> order for your forked repository to run the workflow, and
>>> for the main repository to detect the workflow.
>>>
>>>
>>> *What does that mean to committers?*
>>>
>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>> are running (in contributor's forked repository).
>>> Please check the build notified by github-actions bot before merging it.
>>> There would be a followup work to reflect the status of the forked
>>> repository's build to the status of PR.
>>>
>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>>
>>>> Hi all,
>>>>
>>>> After https://github.com/apache/spark/pull/32092 merged, now we run
>>>> the GitHub Actions
>>>> workflows in your forked repository.
>>>>
>>>> In short, please see this example HyukjinKwon#34
>>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>>
>>>> 1. You create a PR and your repository triggers the workflow. Your
>>>> PR uses the resources allocated to you for testing.
>>>> 2. Apache Spark repository finds your workflow, and links it in a
>>>> comment in your PR
>>>>
>>>> Please let me know if you guys find any weird behaviour related to this.
>>>>
>>>>
>>>> *What does that mean to contributors?*
>>>>
>>>> Please sync your branch to the latest master branch in Apache Spark in
>>>> order for the main repository to run the workflow and detect it.
>>>>
>>>>
>>>> *What does that mean to committers?*
>>>>
>>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>>> are running (in contributor's forked repository). Please check the build
>>>> notified by github-actions bot before merging it.
>>>> There would be a followup work to reflect the status of the forked
>>>> repository's build to
>>>> the status of PR.
>>>>
>>>>
>>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Yi Wu <yi...@databricks.com>.
Thanks for the great work, Hyukjin!
On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <lt...@gmail.com> wrote:
> Thanks for the amazing work, Hyukjin!
> I created a PR for trial and it looks well so far:
> https://github.com/apache/spark/pull/32158
>
> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> Hi all,
>>
>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>> GitHub Actions
>> workflows in your forked repository.
>>
>> In short, please see this example HyukjinKwon#34
>> <https://github.com/HyukjinKwon/spark/pull/34>
>>
>> 1. You create a PR and your repository triggers the workflow. Your PR
>> uses the resources allocated to you for testing.
>> 2. Apache Spark repository finds your workflow, and links it in a
>> comment in your PR
>>
>> Please let me know if you guys find any weird behaviour related to this.
>>
>>
>> *What does that mean to contributors?*
>>
>> Please sync your branch to the latest master branch in Apache Spark in
>> order for your forked repository to run the workflow, and
>> for the main repository to detect the workflow.
>>
>>
>> *What does that mean to committers?*
>>
>> Now, GitHub Actions will show a green even when GitHub Actions builds are
>> running (in contributor's forked repository).
>> Please check the build notified by github-actions bot before merging it.
>> There would be a followup work to reflect the status of the forked
>> repository's build to the status of PR.
>>
>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>>
>>> Hi all,
>>>
>>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>>> GitHub Actions
>>> workflows in your forked repository.
>>>
>>> In short, please see this example HyukjinKwon#34
>>> <https://github.com/HyukjinKwon/spark/pull/34>
>>>
>>> 1. You create a PR and your repository triggers the workflow. Your
>>> PR uses the resources allocated to you for testing.
>>> 2. Apache Spark repository finds your workflow, and links it in a
>>> comment in your PR
>>>
>>> Please let me know if you guys find any weird behaviour related to this.
>>>
>>>
>>> *What does that mean to contributors?*
>>>
>>> Please sync your branch to the latest master branch in Apache Spark in
>>> order for the main repository to run the workflow and detect it.
>>>
>>>
>>> *What does that mean to committers?*
>>>
>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>> are running (in contributor's forked repository). Please check the build
>>> notified by github-actions bot before merging it.
>>> There would be a followup work to reflect the status of the forked
>>> repository's build to
>>> the status of PR.
>>>
>>>
>>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Gengliang Wang <lt...@gmail.com>.
Thanks for the amazing work, Hyukjin!
I created a PR for trial and it looks well so far:
https://github.com/apache/spark/pull/32158
On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gu...@gmail.com> wrote:
> Hi all,
>
> After https://github.com/apache/spark/pull/32092 merged, now we run the
> GitHub Actions
> workflows in your forked repository.
>
> In short, please see this example HyukjinKwon#34
> <https://github.com/HyukjinKwon/spark/pull/34>
>
> 1. You create a PR and your repository triggers the workflow. Your PR
> uses the resources allocated to you for testing.
> 2. Apache Spark repository finds your workflow, and links it in a
> comment in your PR
>
> Please let me know if you guys find any weird behaviour related to this.
>
>
> *What does that mean to contributors?*
>
> Please sync your branch to the latest master branch in Apache Spark in
> order for your forked repository to run the workflow, and
> for the main repository to detect the workflow.
>
>
> *What does that mean to committers?*
>
> Now, GitHub Actions will show a green even when GitHub Actions builds are
> running (in contributor's forked repository).
> Please check the build notified by github-actions bot before merging it.
> There would be a followup work to reflect the status of the forked
> repository's build to the status of PR.
>
> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
>
>> Hi all,
>>
>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>> GitHub Actions
>> workflows in your forked repository.
>>
>> In short, please see this example HyukjinKwon#34
>> <https://github.com/HyukjinKwon/spark/pull/34>
>>
>> 1. You create a PR and your repository triggers the workflow. Your PR
>> uses the resources allocated to you for testing.
>> 2. Apache Spark repository finds your workflow, and links it in a
>> comment in your PR
>>
>> Please let me know if you guys find any weird behaviour related to this.
>>
>>
>> *What does that mean to contributors?*
>>
>> Please sync your branch to the latest master branch in Apache Spark in
>> order for the main repository to run the workflow and detect it.
>>
>>
>> *What does that mean to committers?*
>>
>> Now, GitHub Actions will show a green even when GitHub Actions builds are
>> running (in contributor's forked repository). Please check the build
>> notified by github-actions bot before merging it.
>> There would be a followup work to reflect the status of the forked
>> repository's build to
>> the status of PR.
>>
>>
>>
Re: [PSA] Please read: PR builder now runs test and build in your
forked repository
Posted by Hyukjin Kwon <gu...@gmail.com>.
Hi all,
After https://github.com/apache/spark/pull/32092 merged, now we run the
GitHub Actions
workflows in your forked repository.
In short, please see this example HyukjinKwon#34
<https://github.com/HyukjinKwon/spark/pull/34>
1. You create a PR and your repository triggers the workflow. Your PR
uses the resources allocated to you for testing.
2. Apache Spark repository finds your workflow, and links it in a
comment in your PR
Please let me know if you guys find any weird behaviour related to this.
*What does that mean to contributors?*
Please sync your branch to the latest master branch in Apache Spark in
order for your forked repository to run the workflow, and
for the main repository to detect the workflow.
*What does that mean to committers?*
Now, GitHub Actions will show a green even when GitHub Actions builds are
running (in contributor's forked repository).
Please check the build notified by github-actions bot before merging it.
There would be a followup work to reflect the status of the forked
repository's build to the status of PR.
2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gu...@gmail.com>님이 작성:
> Hi all,
>
> After https://github.com/apache/spark/pull/32092 merged, now we run the
> GitHub Actions
> workflows in your forked repository.
>
> In short, please see this example HyukjinKwon#34
> <https://github.com/HyukjinKwon/spark/pull/34>
>
> 1. You create a PR and your repository triggers the workflow. Your PR
> uses the resources allocated to you for testing.
> 2. Apache Spark repository finds your workflow, and links it in a
> comment in your PR
>
> Please let me know if you guys find any weird behaviour related to this.
>
>
> *What does that mean to contributors?*
>
> Please sync your branch to the latest master branch in Apache Spark in
> order for the main repository to run the workflow and detect it.
>
>
> *What does that mean to committers?*
>
> Now, GitHub Actions will show a green even when GitHub Actions builds are
> running (in contributor's forked repository). Please check the build
> notified by github-actions bot before merging it.
> There would be a followup work to reflect the status of the forked
> repository's build to
> the status of PR.
>
>
>