Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.

> It's unavoidable if "users" prefer to use an alternative communication
mechanism rather than the user mailing list.

The following is the number of people in the official channels.

- user@spark.apache.org has 4519 subscribers.
- dev@spark.apache.org has 3149 subscribers.
- ASF Official Slack channel has 602 subscribers.

May I ask if the users prefer to use the ASF Official Slack channel
than the user mailing list?

Dongjoon.



On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <ka...@gmail.com>
wrote:

> I'm reading through the page "Briefing: The Apache Way", and in the
> section of "Open Communications", restriction of communication inside ASF
> INFRA (mailing list) is more about code and decision-making.
>
> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>
> It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list. Before Stack Overflow days,
> there had been a meaningful number of questions around user@. It's just
> impossible to let them go back and post to the user mailing list.
>
> We just need to make sure it is not the purpose of employing Slack to move
> all discussions about developments, direction of the project, etc which
> must happen in dev@/private@. The purpose of Slack thread here does not
> seem to aim to serve the purpose.
>
>
> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Good discussions and proposals.all around.
>>
>> I have used slack in anger on a customer site before. For small and
>> medium size groups it is good and affordable. Alternatives have been
>> suggested as well so those who like investigative search can agree and come
>> up with a freebie one.
>> I am inclined to agree with Bjorn that this slack has more social
>> dimensions than the mailing list. It is akin to a sports club using
>> WhatsApp groups for communication. Remember we were originally looking for
>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>> I think Slack and mailing groups can coexist happily. On a more serious
>> note, when I joined the user group back in 2015-2016, there was a lot of
>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>> a slack type medium may improve members participation.
>>
>> so +1 for me as well.
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>    view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>
>>> +1.
>>>
>>> To Shani’s point, there are multiple OSS projects that use the free
>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>> MLflow, etc.
>>>
>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>
>>>> Hey everyone,
>>>>
>>>> I think we should remain on a free program in slack.
>>>>
>>>> In my option the free program is more then enough, the only down side
>>>> is we could only see the last 90 days messages.
>>>>
>>>> From what I know the Airflow community (which has strong active
>>>> community in slack) also use the free program (You can tell by the 90 days
>>>> limit notice in their workspace).
>>>>
>>>> You can find the pricing and features comparison between the slack
>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>
>>>> Have a great day,
>>>> Shani
>>>>
>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com> wrote:
>>>>
>>>> 
>>>>
>>>>
>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>> discussion - but I can understand the concern.
>>>> Would be great if you or Matei could update the thread on the result of
>>>> deliberations, once it reaches a logical consensus: before we set up
>>>> official policy around it.
>>>>
>>>> Regards,
>>>> Mridul
>>>>
>>>>
>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>> bjornjorgensen@gmail.com> wrote:
>>>>
>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>> you would not have dared to ask about before on mailing lists.
>>>>> But then there is the price and what is the best for an open source
>>>>> project.
>>>>>
>>>>> The price for using slack is expensive.
>>>>> Right now for those that have join spark slack
>>>>> $8.75 USD
>>>>> 72 members
>>>>> 1 month
>>>>> $630 USD
>>>>>
>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>
>>>>> And they - slack does not have an option for open source projects.
>>>>>
>>>>> There seems to be some alternatives for open source software. I have
>>>>> not tried it.
>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>
>>>>> <image.png>
>>>>>
>>>>>
>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>
>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com>:
>>>>>
>>>>>> Hi Dongjoon
>>>>>>
>>>>>> to your points if I may
>>>>>>
>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>> channels?
>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>> corrected
>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>> list because we didn't set up any rule here yet.
>>>>>>    fair enough
>>>>>>
>>>>>> going back to your original point
>>>>>>
>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>> software everywhere. One way or other they do help getting community
>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>> happens to be one of them.
>>>>>> I am of the opinion that creating such silos is already a reality and
>>>>>> we ought to be pragmatic. Unless there is an overriding reason, we should
>>>>>> embrace it as slack can co-exist with the other mailing lists and channels
>>>>>> like linkedin etc.
>>>>>>
>>>>>> Hope this clarifies my position
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>    view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <do...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> To Mich.
>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>> channels?
>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>
>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>> Matei from your side.
>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>
>>>>>>> There is on-going discussion about the communication channels
>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>> Please hold on any official action for this topic. We will know how
>>>>>>> to support it seamlessly.
>>>>>>>
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi, Dongjoon,
>>>>>>>>
>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their own
>>>>>>>> Slack workspaces last year. I did not see an objection from the ASF board.
>>>>>>>> At the same time, Slack workspaces are very popular and useful in most
>>>>>>>> non-ASF open source communities. TBH, we are kind of late. I think we can
>>>>>>>> do the same in our community?
>>>>>>>>
>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>
>>>>>>>> Xiao
>>>>>>>>
>>>>>>>>
>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>
>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>
>>>>>>>>> (cc Matei)
>>>>>>>>>
>>>>>>>>> Please hold on the vote.
>>>>>>>>>
>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>
>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>> before starting anything official.
>>>>>>>>>
>>>>>>>>> Bests,
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1
>>>>>>>>>>
>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>
>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Xiao
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>> 07:07写道：
>>>>>>>>>>
>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <zg...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>
>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>
>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>> community.
>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>> users.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>
>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>
>>>>> --
>>>>> Bjørn Jørgensen
>>>>> Vestre Aspehaug 4
>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>> 6010 Ålesund
>>>>> Norge
>>>>>
>>>>> +47 480 94 297
>>>>>
>>>>

Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

Do you think there is a way to put it back to the official ASF-provided
Slack channel?

Dongjoon.

On Mon, Apr 3, 2023 at 2:18 PM Mich Talebzadeh <mi...@gmail.com>
wrote:

>
> I for myself prefer to use the newly formed slack.
>
> sparkcommunitytalk.slack.com
>
> In summary, it may be a good idea to take a tour of it and see for
> yourself. Topics are sectioned as per user requests.
>
> I trust this answers your question.
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>    view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 3 Apr 2023 at 20:59, Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - user@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>> there had been a meaningful number of questions around user@. It's just
>>> impossible to let them go back and post to the user mailing list.
>>>
>>> We just need to make sure it is not the purpose of employing Slack to
>>> move all discussions about developments, direction of the project, etc
>>> which must happen in dev@/private@. The purpose of Slack thread here
>>> does not seem to aim to serve the purpose.
>>>
>>>
>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Good discussions and proposals.all around.
>>>>
>>>> I have used slack in anger on a customer site before. For small and
>>>> medium size groups it is good and affordable. Alternatives have been
>>>> suggested as well so those who like investigative search can agree and come
>>>> up with a freebie one.
>>>> I am inclined to agree with Bjorn that this slack has more social
>>>> dimensions than the mailing list. It is akin to a sports club using
>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>> a slack type medium may improve members participation.
>>>>
>>>> so +1 for me as well.
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>    view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>>
>>>>> +1.
>>>>>
>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>> MLflow, etc.
>>>>>
>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>
>>>>>> Hey everyone,
>>>>>>
>>>>>> I think we should remain on a free program in slack.
>>>>>>
>>>>>> In my option the free program is more then enough, the only down side
>>>>>> is we could only see the last 90 days messages.
>>>>>>
>>>>>> From what I know the Airflow community (which has strong active
>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>> limit notice in their workspace).
>>>>>>
>>>>>> You can find the pricing and features comparison between the slack
>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>
>>>>>> Have a great day,
>>>>>> Shani
>>>>>>
>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> 
>>>>>>
>>>>>>
>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>> discussion - but I can understand the concern.
>>>>>> Would be great if you or Matei could update the thread on the result
>>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>>> official policy around it.
>>>>>>
>>>>>> Regards,
>>>>>> Mridul
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>
>>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>>> But then there is the price and what is the best for an open source
>>>>>>> project.
>>>>>>>
>>>>>>> The price for using slack is expensive.
>>>>>>> Right now for those that have join spark slack
>>>>>>> $8.75 USD
>>>>>>> 72 members
>>>>>>> 1 month
>>>>>>> $630 USD
>>>>>>>
>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>
>>>>>>> And they - slack does not have an option for open source projects.
>>>>>>>
>>>>>>> There seems to be some alternatives for open source software. I have
>>>>>>> not tried it.
>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>
>>>>>>> <image.png>
>>>>>>>
>>>>>>>
>>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>>
>>>>>>>
>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>
>>>>>>>> Hi Dongjoon
>>>>>>>>
>>>>>>>> to your points if I may
>>>>>>>>
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>> corrected
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>    fair enough
>>>>>>>>
>>>>>>>> going back to your original point
>>>>>>>>
>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>> happens to be one of them.
>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>> channels like linkedin etc.
>>>>>>>>
>>>>>>>> Hope this clarifies my position
>>>>>>>>
>>>>>>>> Mich Talebzadeh,
>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>> Palantir Technologies Limited
>>>>>>>>
>>>>>>>>
>>>>>>>>    view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> To Mich.
>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>> channels?
>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>
>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>>> Matei from your side.
>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>
>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>> how to support it seamlessly.
>>>>>>>>>
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>
>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>> can do the same in our community?
>>>>>>>>>>
>>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>>
>>>>>>>>>> Xiao
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>
>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>
>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>
>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>
>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>
>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>
>>>>>>>>>>> Bests,
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1
>>>>>>>>>>>>
>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Xiao
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>
>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Bjørn Jørgensen
>>>>>>> Vestre Aspehaug 4
>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>> 6010 Ålesund
>>>>>>> Norge
>>>>>>>
>>>>>>> +47 480 94 297
>>>>>>>
>>>>>>

Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

Do you think there is a way to put it back to the official ASF-provided
Slack channel?

Dongjoon.

On Mon, Apr 3, 2023 at 2:18 PM Mich Talebzadeh <mi...@gmail.com>
wrote:

>
> I for myself prefer to use the newly formed slack.
>
> sparkcommunitytalk.slack.com
>
> In summary, it may be a good idea to take a tour of it and see for
> yourself. Topics are sectioned as per user requests.
>
> I trust this answers your question.
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>    view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 3 Apr 2023 at 20:59, Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - user@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>> there had been a meaningful number of questions around user@. It's just
>>> impossible to let them go back and post to the user mailing list.
>>>
>>> We just need to make sure it is not the purpose of employing Slack to
>>> move all discussions about developments, direction of the project, etc
>>> which must happen in dev@/private@. The purpose of Slack thread here
>>> does not seem to aim to serve the purpose.
>>>
>>>
>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Good discussions and proposals.all around.
>>>>
>>>> I have used slack in anger on a customer site before. For small and
>>>> medium size groups it is good and affordable. Alternatives have been
>>>> suggested as well so those who like investigative search can agree and come
>>>> up with a freebie one.
>>>> I am inclined to agree with Bjorn that this slack has more social
>>>> dimensions than the mailing list. It is akin to a sports club using
>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>> a slack type medium may improve members participation.
>>>>
>>>> so +1 for me as well.
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>    view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>>
>>>>> +1.
>>>>>
>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>> MLflow, etc.
>>>>>
>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>
>>>>>> Hey everyone,
>>>>>>
>>>>>> I think we should remain on a free program in slack.
>>>>>>
>>>>>> In my option the free program is more then enough, the only down side
>>>>>> is we could only see the last 90 days messages.
>>>>>>
>>>>>> From what I know the Airflow community (which has strong active
>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>> limit notice in their workspace).
>>>>>>
>>>>>> You can find the pricing and features comparison between the slack
>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>
>>>>>> Have a great day,
>>>>>> Shani
>>>>>>
>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> 
>>>>>>
>>>>>>
>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>> discussion - but I can understand the concern.
>>>>>> Would be great if you or Matei could update the thread on the result
>>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>>> official policy around it.
>>>>>>
>>>>>> Regards,
>>>>>> Mridul
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>
>>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>>> But then there is the price and what is the best for an open source
>>>>>>> project.
>>>>>>>
>>>>>>> The price for using slack is expensive.
>>>>>>> Right now for those that have join spark slack
>>>>>>> $8.75 USD
>>>>>>> 72 members
>>>>>>> 1 month
>>>>>>> $630 USD
>>>>>>>
>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>
>>>>>>> And they - slack does not have an option for open source projects.
>>>>>>>
>>>>>>> There seems to be some alternatives for open source software. I have
>>>>>>> not tried it.
>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>
>>>>>>> <image.png>
>>>>>>>
>>>>>>>
>>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>>
>>>>>>>
>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>
>>>>>>>> Hi Dongjoon
>>>>>>>>
>>>>>>>> to your points if I may
>>>>>>>>
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>> corrected
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>    fair enough
>>>>>>>>
>>>>>>>> going back to your original point
>>>>>>>>
>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>> happens to be one of them.
>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>> channels like linkedin etc.
>>>>>>>>
>>>>>>>> Hope this clarifies my position
>>>>>>>>
>>>>>>>> Mich Talebzadeh,
>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>> Palantir Technologies Limited
>>>>>>>>
>>>>>>>>
>>>>>>>>    view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> To Mich.
>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>> channels?
>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>
>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>>> Matei from your side.
>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>
>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>> how to support it seamlessly.
>>>>>>>>>
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>
>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>> can do the same in our community?
>>>>>>>>>>
>>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>>
>>>>>>>>>> Xiao
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>
>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>
>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>
>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>
>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>
>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>
>>>>>>>>>>> Bests,
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1
>>>>>>>>>>>>
>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Xiao
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>
>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Bjørn Jørgensen
>>>>>>> Vestre Aspehaug 4
>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>> 6010 Ålesund
>>>>>>> Norge
>>>>>>>
>>>>>>> +47 480 94 297
>>>>>>>
>>>>>>

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

I for myself prefer to use the newly formed slack.

sparkcommunitytalk.slack.com

In summary, it may be a good idea to take a tour of it and see for
yourself. Topics are sectioned as per user requests.

I trust this answers your question.

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 3 Apr 2023 at 20:59, Dongjoon Hyun <do...@gmail.com> wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.
>
> > It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - user@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list. Before Stack Overflow days,
>> there had been a meaningful number of questions around user@. It's just
>> impossible to let them go back and post to the user mailing list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of Slack thread here
>> does not seem to aim to serve the purpose.
>>
>>
>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Good discussions and proposals.all around.
>>>
>>> I have used slack in anger on a customer site before. For small and
>>> medium size groups it is good and affordable. Alternatives have been
>>> suggested as well so those who like investigative search can agree and come
>>> up with a freebie one.
>>> I am inclined to agree with Bjorn that this slack has more social
>>> dimensions than the mailing list. It is akin to a sports club using
>>> WhatsApp groups for communication. Remember we were originally looking for
>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>> I think Slack and mailing groups can coexist happily. On a more serious
>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>> a slack type medium may improve members participation.
>>>
>>> so +1 for me as well.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>    view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>
>>>> +1.
>>>>
>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>> MLflow, etc.
>>>>
>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>
>>>>> Hey everyone,
>>>>>
>>>>> I think we should remain on a free program in slack.
>>>>>
>>>>> In my option the free program is more then enough, the only down side
>>>>> is we could only see the last 90 days messages.
>>>>>
>>>>> From what I know the Airflow community (which has strong active
>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>> limit notice in their workspace).
>>>>>
>>>>> You can find the pricing and features comparison between the slack
>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>
>>>>> Have a great day,
>>>>> Shani
>>>>>
>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> 
>>>>>
>>>>>
>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>> discussion - but I can understand the concern.
>>>>> Would be great if you or Matei could update the thread on the result
>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>> official policy around it.
>>>>>
>>>>> Regards,
>>>>> Mridul
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>
>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>> But then there is the price and what is the best for an open source
>>>>>> project.
>>>>>>
>>>>>> The price for using slack is expensive.
>>>>>> Right now for those that have join spark slack
>>>>>> $8.75 USD
>>>>>> 72 members
>>>>>> 1 month
>>>>>> $630 USD
>>>>>>
>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>
>>>>>> And they - slack does not have an option for open source projects.
>>>>>>
>>>>>> There seems to be some alternatives for open source software. I have
>>>>>> not tried it.
>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>
>>>>>> <image.png>
>>>>>>
>>>>>>
>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>
>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>
>>>>>>> Hi Dongjoon
>>>>>>>
>>>>>>> to your points if I may
>>>>>>>
>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>> channels?
>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>> corrected
>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>    fair enough
>>>>>>>
>>>>>>> going back to your original point
>>>>>>>
>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>> happens to be one of them.
>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>> channels like linkedin etc.
>>>>>>>
>>>>>>> Hope this clarifies my position
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>    view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <do...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> To Mich.
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>
>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>> Matei from your side.
>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>
>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>> Please hold on any official action for this topic. We will know how
>>>>>>>> to support it seamlessly.
>>>>>>>>
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>
>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>> can do the same in our community?
>>>>>>>>>
>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>
>>>>>>>>> Xiao
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>
>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>
>>>>>>>>>> (cc Matei)
>>>>>>>>>>
>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>
>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>
>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>> before starting anything official.
>>>>>>>>>>
>>>>>>>>>> Bests,
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Xiao
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>
>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <zg...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Bjørn Jørgensen
>>>>>> Vestre Aspehaug 4
>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>> 6010 Ålesund
>>>>>> Norge
>>>>>>
>>>>>> +47 480 94 297
>>>>>>
>>>>>

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

That 3 months retention is just a soft setting. For low volume traffic, it
can be negotiated to a year’s retention. Let me see what we can do about it.

HTH

On Tue, 4 Apr 2023 at 09:31, Bjørn Jørgensen <bj...@gmail.com>
wrote:

> One of the things that I don't like about this slack solution is that
> questions and answers disappear after 90 days. Today's maillist solution is
> indexed by search engines and when one day you wonder about something, you
> can find solutions with the help of just searching the web. Another
> question that I have is why has apache superset
> <https://superset.apache.org> taken away their slack channel. They have
> it linked to on the website but the link is giving errors
> <https://apache-superset.slack.com/join/shared_invite/zt-1pj34ugpe-23fJZ7DIH~F~ffIkHsRv1g#/shared-invite/error>.
> Have they had any experience that this was not the best solution or is it
> just that the link does not work.
>
> tir. 4. apr. 2023 kl. 09:06 skrev Mich Talebzadeh <
> mich.talebzadeh@gmail.com>:
>
>> Hi Shani,
>>
>> I believe I am an admin so that is fine by me.
>>
>> Hi Dongioon,
>>
>> With regard to summarising the discussion etc, no need, It is like
>> flogging the dead horse, we have already discussed it enough. I don't see
>> the point of it.
>>
>> HTH
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>    view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 4 Apr 2023 at 07:06, <sh...@gmail.com> wrote:
>>
>>> Hey Dongjoon, Denny and all,
>>>
>>> I’ve created the current slack.
>>> All users have the option to create channels for different topics.
>>>
>>> I don’t see a reason for creating a new one.
>>>
>>> If anyone want to be admin on the current slack channel you all are
>>> welcome to send me a msg and I’ll grand permission.
>>>
>>> Have a great week,
>>> Shani Alisar
>>>
>>>
>>>
>>>
>>>
>>> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:
>>>
>>> 
>>> Thank you, Denny.
>>>
>>> May I interpret your comment as a request to support multiple channels
>>> in ASF too?
>>>
>>> > because it would allow us to create multiple channels for different
>>> topics
>>>
>>> Any other reasons?
>>>
>>> Dongjoon.
>>>
>>>
>>> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:
>>>
>>>> I do think creating a new Slack channel would be helpful because it
>>>> would allow us to create multiple channels for different topics -
>>>> streaming, graph, ML, etc.
>>>>
>>>> We would need a volunteer core to maintain it so we can keep the spirit
>>>> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
>>>> active.
>>>>
>>>>
>>>>
>>>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> Shall we summarize the discussion so far?
>>>>>
>>>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>>>>> initiate this thread instead of "Slack" vs "Mailing list"?
>>>>>
>>>>> If ASF Slack provides what you need, is it better than creating a
>>>>> new Slack channel?
>>>>>
>>>>> Or, is there another reason for us to create a new Slack channel?
>>>>>
>>>>> Dongjoon.
>>>>>
>>>>>
>>>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> I agree, whatever individual sentiments are.
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>    view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> Just to be clear, if there is no strong volunteer to make the new
>>>>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>>>>> place where there are only questions and promos but zero answers. I see
>>>>>>> volunteers here demanding for another channel, so I want to see us go with
>>>>>>> the most preferred way for these volunteers.
>>>>>>>
>>>>>>> User mailing list does not go in a good shape. I hope we give
>>>>>>> another try with recent technology to see whether we can gain traction - if
>>>>>>> we fail, the user mailing list will still be there.
>>>>>>>
>>>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>>>>> look into the number of mails being sent to the list.
>>>>>>>>
>>>>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>>>>> in 2015 and earlier.
>>>>>>>> Let's face the fact. User mailing list is dying, even before we
>>>>>>>> start discussion about alternative communication methods.
>>>>>>>>
>>>>>>>> Users never go with the way if it's just because PMC members (or
>>>>>>>> ASF) have preference. They are going with the way they are convenient.
>>>>>>>>
>>>>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>>>>> but we are also talking about the cost as well.
>>>>>>>>
>>>>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>>>>> As long as we are being serious about the cost, I don't think we
>>>>>>>> are going to land in the way "users" are convenient.
>>>>>>>>
>>>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>>>>> channel.
>>>>>>>>>
>>>>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>>>>> communication mechanism rather than the user mailing list.
>>>>>>>>>
>>>>>>>>> The following is the number of people in the official channels.
>>>>>>>>>
>>>>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>>>>
>>>>>>>>> May I ask if the users prefer to use the ASF Official Slack
>>>>>>>>> channel than the user mailing list?
>>>>>>>>>
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in
>>>>>>>>>> the section of "Open Communications", restriction of communication inside
>>>>>>>>>> ASF INFRA (mailing list) is more about code and decision-making.
>>>>>>>>>>
>>>>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>>>>
>>>>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>>>>
>>>>>>>>>> We just need to make sure it is not the purpose of
>>>>>>>>>> employing Slack to move all discussions about developments, direction of
>>>>>>>>>> the project, etc which must happen in dev@/private@. The purpose
>>>>>>>>>> of Slack thread here does not seem to aim to serve the purpose.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>>>>
>>>>>>>>>>> I have used slack in anger on a customer site before. For small
>>>>>>>>>>> and medium size groups it is good and affordable. Alternatives have been
>>>>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>>>>> up with a freebie one.
>>>>>>>>>>> I am inclined to agree with Bjorn that this slack has more
>>>>>>>>>>> social dimensions than the mailing list. It is akin to a sports club using
>>>>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>>>>
>>>>>>>>>>> so +1 for me as well.
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1.
>>>>>>>>>>>>
>>>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In my option the free program is more then enough, the only
>>>>>>>>>>>>> down side is we could only see the last 90 days messages.
>>>>>>>>>>>>>
>>>>>>>>>>>>> From what I know the Airflow community (which has strong
>>>>>>>>>>>>> active community in slack) also use the free program (You can tell by the
>>>>>>>>>>>>> 90 days limit notice in their workspace).
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>>>>
>>>>>>>>>>>>> Have a great day,
>>>>>>>>>>>>> Shani
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <
>>>>>>>>>>>>> mridul@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of
>>>>>>>>>>>>> the discussion - but I can understand the concern.
>>>>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>>>>> up official policy around it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Mridul
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I like the idea of having a talk channel. It can make it
>>>>>>>>>>>>>> easier for everyone to say hello. Or to dare to ask about small or big
>>>>>>>>>>>>>> matters that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>>>>> source project.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>>>>> $8.75 USD
>>>>>>>>>>>>>> 72 members
>>>>>>>>>>>>>> 1 month
>>>>>>>>>>>>>> $630 USD
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>>>>> projects.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There seems to be some alternatives for open source software.
>>>>>>>>>>>>>> I have not tried it.
>>>>>>>>>>>>>> Like
>>>>>>>>>>>>>> https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <image.png>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>>>>> stand corrected
>>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>> archive...
>>>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why
>>>>>>>>>>>>>>>> I added Matei from your side.
>>>>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is on-going discussion about the communication
>>>>>>>>>>>>>>>> channels outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <
>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official
>>>>>>>>>>>>>>>>> process for ASF archiving. Since our PMC are the owner of the slack
>>>>>>>>>>>>>>>>> workspace, we can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF
>>>>>>>>>>>>>>>>>> archive before starting anything official.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com>
>>>>>>>>>>>>>>>>>>> 于2023年3月28日周二 07:07写道：
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd
>>>>>>>>>>>>>>>>>>>>>> like to join :)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>>>>> Norge
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>
> --
> Bjørn Jørgensen
> Vestre Aspehaug 4
> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
> 6010 Ålesund
> Norge
>
> +47 480 94 297
>
-- 
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

That 3 months retention is just a soft setting. For low volume traffic, it
can be negotiated to a year’s retention. Let me see what we can do about it.

HTH

On Tue, 4 Apr 2023 at 09:31, Bjørn Jørgensen <bj...@gmail.com>
wrote:

> One of the things that I don't like about this slack solution is that
> questions and answers disappear after 90 days. Today's maillist solution is
> indexed by search engines and when one day you wonder about something, you
> can find solutions with the help of just searching the web. Another
> question that I have is why has apache superset
> <https://superset.apache.org> taken away their slack channel. They have
> it linked to on the website but the link is giving errors
> <https://apache-superset.slack.com/join/shared_invite/zt-1pj34ugpe-23fJZ7DIH~F~ffIkHsRv1g#/shared-invite/error>.
> Have they had any experience that this was not the best solution or is it
> just that the link does not work.
>
> tir. 4. apr. 2023 kl. 09:06 skrev Mich Talebzadeh <
> mich.talebzadeh@gmail.com>:
>
>> Hi Shani,
>>
>> I believe I am an admin so that is fine by me.
>>
>> Hi Dongioon,
>>
>> With regard to summarising the discussion etc, no need, It is like
>> flogging the dead horse, we have already discussed it enough. I don't see
>> the point of it.
>>
>> HTH
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>    view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 4 Apr 2023 at 07:06, <sh...@gmail.com> wrote:
>>
>>> Hey Dongjoon, Denny and all,
>>>
>>> I’ve created the current slack.
>>> All users have the option to create channels for different topics.
>>>
>>> I don’t see a reason for creating a new one.
>>>
>>> If anyone want to be admin on the current slack channel you all are
>>> welcome to send me a msg and I’ll grand permission.
>>>
>>> Have a great week,
>>> Shani Alisar
>>>
>>>
>>>
>>>
>>>
>>> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:
>>>
>>> 
>>> Thank you, Denny.
>>>
>>> May I interpret your comment as a request to support multiple channels
>>> in ASF too?
>>>
>>> > because it would allow us to create multiple channels for different
>>> topics
>>>
>>> Any other reasons?
>>>
>>> Dongjoon.
>>>
>>>
>>> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:
>>>
>>>> I do think creating a new Slack channel would be helpful because it
>>>> would allow us to create multiple channels for different topics -
>>>> streaming, graph, ML, etc.
>>>>
>>>> We would need a volunteer core to maintain it so we can keep the spirit
>>>> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
>>>> active.
>>>>
>>>>
>>>>
>>>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> Shall we summarize the discussion so far?
>>>>>
>>>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>>>>> initiate this thread instead of "Slack" vs "Mailing list"?
>>>>>
>>>>> If ASF Slack provides what you need, is it better than creating a
>>>>> new Slack channel?
>>>>>
>>>>> Or, is there another reason for us to create a new Slack channel?
>>>>>
>>>>> Dongjoon.
>>>>>
>>>>>
>>>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> I agree, whatever individual sentiments are.
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>    view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> Just to be clear, if there is no strong volunteer to make the new
>>>>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>>>>> place where there are only questions and promos but zero answers. I see
>>>>>>> volunteers here demanding for another channel, so I want to see us go with
>>>>>>> the most preferred way for these volunteers.
>>>>>>>
>>>>>>> User mailing list does not go in a good shape. I hope we give
>>>>>>> another try with recent technology to see whether we can gain traction - if
>>>>>>> we fail, the user mailing list will still be there.
>>>>>>>
>>>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>>>>> look into the number of mails being sent to the list.
>>>>>>>>
>>>>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>>>>> in 2015 and earlier.
>>>>>>>> Let's face the fact. User mailing list is dying, even before we
>>>>>>>> start discussion about alternative communication methods.
>>>>>>>>
>>>>>>>> Users never go with the way if it's just because PMC members (or
>>>>>>>> ASF) have preference. They are going with the way they are convenient.
>>>>>>>>
>>>>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>>>>> but we are also talking about the cost as well.
>>>>>>>>
>>>>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>>>>> As long as we are being serious about the cost, I don't think we
>>>>>>>> are going to land in the way "users" are convenient.
>>>>>>>>
>>>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>>>>> channel.
>>>>>>>>>
>>>>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>>>>> communication mechanism rather than the user mailing list.
>>>>>>>>>
>>>>>>>>> The following is the number of people in the official channels.
>>>>>>>>>
>>>>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>>>>
>>>>>>>>> May I ask if the users prefer to use the ASF Official Slack
>>>>>>>>> channel than the user mailing list?
>>>>>>>>>
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in
>>>>>>>>>> the section of "Open Communications", restriction of communication inside
>>>>>>>>>> ASF INFRA (mailing list) is more about code and decision-making.
>>>>>>>>>>
>>>>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>>>>
>>>>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>>>>
>>>>>>>>>> We just need to make sure it is not the purpose of
>>>>>>>>>> employing Slack to move all discussions about developments, direction of
>>>>>>>>>> the project, etc which must happen in dev@/private@. The purpose
>>>>>>>>>> of Slack thread here does not seem to aim to serve the purpose.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>>>>
>>>>>>>>>>> I have used slack in anger on a customer site before. For small
>>>>>>>>>>> and medium size groups it is good and affordable. Alternatives have been
>>>>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>>>>> up with a freebie one.
>>>>>>>>>>> I am inclined to agree with Bjorn that this slack has more
>>>>>>>>>>> social dimensions than the mailing list. It is akin to a sports club using
>>>>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>>>>
>>>>>>>>>>> so +1 for me as well.
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1.
>>>>>>>>>>>>
>>>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In my option the free program is more then enough, the only
>>>>>>>>>>>>> down side is we could only see the last 90 days messages.
>>>>>>>>>>>>>
>>>>>>>>>>>>> From what I know the Airflow community (which has strong
>>>>>>>>>>>>> active community in slack) also use the free program (You can tell by the
>>>>>>>>>>>>> 90 days limit notice in their workspace).
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>>>>
>>>>>>>>>>>>> Have a great day,
>>>>>>>>>>>>> Shani
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <
>>>>>>>>>>>>> mridul@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of
>>>>>>>>>>>>> the discussion - but I can understand the concern.
>>>>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>>>>> up official policy around it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Mridul
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I like the idea of having a talk channel. It can make it
>>>>>>>>>>>>>> easier for everyone to say hello. Or to dare to ask about small or big
>>>>>>>>>>>>>> matters that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>>>>> source project.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>>>>> $8.75 USD
>>>>>>>>>>>>>> 72 members
>>>>>>>>>>>>>> 1 month
>>>>>>>>>>>>>> $630 USD
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>>>>> projects.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There seems to be some alternatives for open source software.
>>>>>>>>>>>>>> I have not tried it.
>>>>>>>>>>>>>> Like
>>>>>>>>>>>>>> https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <image.png>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>>>>> stand corrected
>>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>> archive...
>>>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why
>>>>>>>>>>>>>>>> I added Matei from your side.
>>>>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is on-going discussion about the communication
>>>>>>>>>>>>>>>> channels outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <
>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official
>>>>>>>>>>>>>>>>> process for ASF archiving. Since our PMC are the owner of the slack
>>>>>>>>>>>>>>>>> workspace, we can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF
>>>>>>>>>>>>>>>>>> archive before starting anything official.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com>
>>>>>>>>>>>>>>>>>>> 于2023年3月28日周二 07:07写道：
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd
>>>>>>>>>>>>>>>>>>>>>> like to join :)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>>>>> Norge
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>
> --
> Bjørn Jørgensen
> Vestre Aspehaug 4
> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
> 6010 Ålesund
> Norge
>
> +47 480 94 297
>
-- 
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.

Re: Slack for PySpark users

[Bjørn Jørgensen <bj...@gmail.com>]

One of the things that I don't like about this slack solution is that
questions and answers disappear after 90 days. Today's maillist solution is
indexed by search engines and when one day you wonder about something, you
can find solutions with the help of just searching the web. Another
question that I have is why has apache superset
<https://superset.apache.org> taken away their slack channel. They have it
linked to on the website but the link is giving errors
<https://apache-superset.slack.com/join/shared_invite/zt-1pj34ugpe-23fJZ7DIH~F~ffIkHsRv1g#/shared-invite/error>.
Have they had any experience that this was not the best solution or is it
just that the link does not work.

tir. 4. apr. 2023 kl. 09:06 skrev Mich Talebzadeh <mich.talebzadeh@gmail.com
>:

> Hi Shani,
>
> I believe I am an admin so that is fine by me.
>
> Hi Dongioon,
>
> With regard to summarising the discussion etc, no need, It is like
> flogging the dead horse, we have already discussed it enough. I don't see
> the point of it.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>    view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 4 Apr 2023 at 07:06, <sh...@gmail.com> wrote:
>
>> Hey Dongjoon, Denny and all,
>>
>> I’ve created the current slack.
>> All users have the option to create channels for different topics.
>>
>> I don’t see a reason for creating a new one.
>>
>> If anyone want to be admin on the current slack channel you all are
>> welcome to send me a msg and I’ll grand permission.
>>
>> Have a great week,
>> Shani Alisar
>>
>>
>>
>>
>>
>> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:
>>
>> 
>> Thank you, Denny.
>>
>> May I interpret your comment as a request to support multiple channels in
>> ASF too?
>>
>> > because it would allow us to create multiple channels for different
>> topics
>>
>> Any other reasons?
>>
>> Dongjoon.
>>
>>
>> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:
>>
>>> I do think creating a new Slack channel would be helpful because it
>>> would allow us to create multiple channels for different topics -
>>> streaming, graph, ML, etc.
>>>
>>> We would need a volunteer core to maintain it so we can keep the spirit
>>> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
>>> active.
>>>
>>>
>>>
>>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> Shall we summarize the discussion so far?
>>>>
>>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>>>> initiate this thread instead of "Slack" vs "Mailing list"?
>>>>
>>>> If ASF Slack provides what you need, is it better than creating a
>>>> new Slack channel?
>>>>
>>>> Or, is there another reason for us to create a new Slack channel?
>>>>
>>>> Dongjoon.
>>>>
>>>>
>>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> I agree, whatever individual sentiments are.
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>    view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> Just to be clear, if there is no strong volunteer to make the new
>>>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>>>> place where there are only questions and promos but zero answers. I see
>>>>>> volunteers here demanding for another channel, so I want to see us go with
>>>>>> the most preferred way for these volunteers.
>>>>>>
>>>>>> User mailing list does not go in a good shape. I hope we give another
>>>>>> try with recent technology to see whether we can gain traction - if we
>>>>>> fail, the user mailing list will still be there.
>>>>>>
>>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>>>> look into the number of mails being sent to the list.
>>>>>>>
>>>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>>>> in 2015 and earlier.
>>>>>>> Let's face the fact. User mailing list is dying, even before we
>>>>>>> start discussion about alternative communication methods.
>>>>>>>
>>>>>>> Users never go with the way if it's just because PMC members (or
>>>>>>> ASF) have preference. They are going with the way they are convenient.
>>>>>>>
>>>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>>>> but we are also talking about the cost as well.
>>>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>>>> As long as we are being serious about the cost, I don't think we are
>>>>>>> going to land in the way "users" are convenient.
>>>>>>>
>>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <
>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>
>>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>>>> channel.
>>>>>>>>
>>>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>>>> communication mechanism rather than the user mailing list.
>>>>>>>>
>>>>>>>> The following is the number of people in the official channels.
>>>>>>>>
>>>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>>>
>>>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>>>>> than the user mailing list?
>>>>>>>>
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in
>>>>>>>>> the section of "Open Communications", restriction of communication inside
>>>>>>>>> ASF INFRA (mailing list) is more about code and decision-making.
>>>>>>>>>
>>>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>>>
>>>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>>>
>>>>>>>>> We just need to make sure it is not the purpose of employing Slack
>>>>>>>>> to move all discussions about developments, direction of the project, etc
>>>>>>>>> which must happen in dev@/private@. The purpose of Slack thread
>>>>>>>>> here does not seem to aim to serve the purpose.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>>>
>>>>>>>>>> I have used slack in anger on a customer site before. For small
>>>>>>>>>> and medium size groups it is good and affordable. Alternatives have been
>>>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>>>> up with a freebie one.
>>>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>>>
>>>>>>>>>> so +1 for me as well.
>>>>>>>>>>
>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1.
>>>>>>>>>>>
>>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>>>
>>>>>>>>>>>> In my option the free program is more then enough, the only
>>>>>>>>>>>> down side is we could only see the last 90 days messages.
>>>>>>>>>>>>
>>>>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>>>>> limit notice in their workspace).
>>>>>>>>>>>>
>>>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>>>
>>>>>>>>>>>> Have a great day,
>>>>>>>>>>>> Shani
>>>>>>>>>>>>
>>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of
>>>>>>>>>>>> the discussion - but I can understand the concern.
>>>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>>>> up official policy around it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Mridul
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I like the idea of having a talk channel. It can make it
>>>>>>>>>>>>> easier for everyone to say hello. Or to dare to ask about small or big
>>>>>>>>>>>>> matters that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>>>> source project.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>>>> $8.75 USD
>>>>>>>>>>>>> 72 members
>>>>>>>>>>>>> 1 month
>>>>>>>>>>>>> $630 USD
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>>>
>>>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>>>> projects.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There seems to be some alternatives for open source software.
>>>>>>>>>>>>> I have not tried it.
>>>>>>>>>>>>> Like
>>>>>>>>>>>>> https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>>>
>>>>>>>>>>>>> <image.png>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>>>
>>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>>>> stand corrected
>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>> archive...
>>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is on-going discussion about the communication
>>>>>>>>>>>>>>> channels outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <
>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official
>>>>>>>>>>>>>>>> process for ASF archiving. Since our PMC are the owner of the slack
>>>>>>>>>>>>>>>> workspace, we can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF
>>>>>>>>>>>>>>>>> archive before starting anything official.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com>
>>>>>>>>>>>>>>>>>> 于2023年3月28日周二 07:07写道：
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd
>>>>>>>>>>>>>>>>>>>>> like to join :)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>>>> Norge
>>>>>>>>>>>>>
>>>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>>>
>>>>>>>>>>>>

-- 
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge

+47 480 94 297

Re: Slack for PySpark users

[Bjørn Jørgensen <bj...@gmail.com>]

One of the things that I don't like about this slack solution is that
questions and answers disappear after 90 days. Today's maillist solution is
indexed by search engines and when one day you wonder about something, you
can find solutions with the help of just searching the web. Another
question that I have is why has apache superset
<https://superset.apache.org> taken away their slack channel. They have it
linked to on the website but the link is giving errors
<https://apache-superset.slack.com/join/shared_invite/zt-1pj34ugpe-23fJZ7DIH~F~ffIkHsRv1g#/shared-invite/error>.
Have they had any experience that this was not the best solution or is it
just that the link does not work.

tir. 4. apr. 2023 kl. 09:06 skrev Mich Talebzadeh <mich.talebzadeh@gmail.com
>:

> Hi Shani,
>
> I believe I am an admin so that is fine by me.
>
> Hi Dongioon,
>
> With regard to summarising the discussion etc, no need, It is like
> flogging the dead horse, we have already discussed it enough. I don't see
> the point of it.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>    view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 4 Apr 2023 at 07:06, <sh...@gmail.com> wrote:
>
>> Hey Dongjoon, Denny and all,
>>
>> I’ve created the current slack.
>> All users have the option to create channels for different topics.
>>
>> I don’t see a reason for creating a new one.
>>
>> If anyone want to be admin on the current slack channel you all are
>> welcome to send me a msg and I’ll grand permission.
>>
>> Have a great week,
>> Shani Alisar
>>
>>
>>
>>
>>
>> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:
>>
>> 
>> Thank you, Denny.
>>
>> May I interpret your comment as a request to support multiple channels in
>> ASF too?
>>
>> > because it would allow us to create multiple channels for different
>> topics
>>
>> Any other reasons?
>>
>> Dongjoon.
>>
>>
>> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:
>>
>>> I do think creating a new Slack channel would be helpful because it
>>> would allow us to create multiple channels for different topics -
>>> streaming, graph, ML, etc.
>>>
>>> We would need a volunteer core to maintain it so we can keep the spirit
>>> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
>>> active.
>>>
>>>
>>>
>>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> Shall we summarize the discussion so far?
>>>>
>>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>>>> initiate this thread instead of "Slack" vs "Mailing list"?
>>>>
>>>> If ASF Slack provides what you need, is it better than creating a
>>>> new Slack channel?
>>>>
>>>> Or, is there another reason for us to create a new Slack channel?
>>>>
>>>> Dongjoon.
>>>>
>>>>
>>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> I agree, whatever individual sentiments are.
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>    view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> Just to be clear, if there is no strong volunteer to make the new
>>>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>>>> place where there are only questions and promos but zero answers. I see
>>>>>> volunteers here demanding for another channel, so I want to see us go with
>>>>>> the most preferred way for these volunteers.
>>>>>>
>>>>>> User mailing list does not go in a good shape. I hope we give another
>>>>>> try with recent technology to see whether we can gain traction - if we
>>>>>> fail, the user mailing list will still be there.
>>>>>>
>>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>>>> look into the number of mails being sent to the list.
>>>>>>>
>>>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>>>> in 2015 and earlier.
>>>>>>> Let's face the fact. User mailing list is dying, even before we
>>>>>>> start discussion about alternative communication methods.
>>>>>>>
>>>>>>> Users never go with the way if it's just because PMC members (or
>>>>>>> ASF) have preference. They are going with the way they are convenient.
>>>>>>>
>>>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>>>> but we are also talking about the cost as well.
>>>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>>>> As long as we are being serious about the cost, I don't think we are
>>>>>>> going to land in the way "users" are convenient.
>>>>>>>
>>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <
>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>
>>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>>>> channel.
>>>>>>>>
>>>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>>>> communication mechanism rather than the user mailing list.
>>>>>>>>
>>>>>>>> The following is the number of people in the official channels.
>>>>>>>>
>>>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>>>
>>>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>>>>> than the user mailing list?
>>>>>>>>
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in
>>>>>>>>> the section of "Open Communications", restriction of communication inside
>>>>>>>>> ASF INFRA (mailing list) is more about code and decision-making.
>>>>>>>>>
>>>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>>>
>>>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>>>
>>>>>>>>> We just need to make sure it is not the purpose of employing Slack
>>>>>>>>> to move all discussions about developments, direction of the project, etc
>>>>>>>>> which must happen in dev@/private@. The purpose of Slack thread
>>>>>>>>> here does not seem to aim to serve the purpose.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>>>
>>>>>>>>>> I have used slack in anger on a customer site before. For small
>>>>>>>>>> and medium size groups it is good and affordable. Alternatives have been
>>>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>>>> up with a freebie one.
>>>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>>>
>>>>>>>>>> so +1 for me as well.
>>>>>>>>>>
>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1.
>>>>>>>>>>>
>>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>>>
>>>>>>>>>>>> In my option the free program is more then enough, the only
>>>>>>>>>>>> down side is we could only see the last 90 days messages.
>>>>>>>>>>>>
>>>>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>>>>> limit notice in their workspace).
>>>>>>>>>>>>
>>>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>>>
>>>>>>>>>>>> Have a great day,
>>>>>>>>>>>> Shani
>>>>>>>>>>>>
>>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of
>>>>>>>>>>>> the discussion - but I can understand the concern.
>>>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>>>> up official policy around it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Mridul
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I like the idea of having a talk channel. It can make it
>>>>>>>>>>>>> easier for everyone to say hello. Or to dare to ask about small or big
>>>>>>>>>>>>> matters that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>>>> source project.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>>>> $8.75 USD
>>>>>>>>>>>>> 72 members
>>>>>>>>>>>>> 1 month
>>>>>>>>>>>>> $630 USD
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>>>
>>>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>>>> projects.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There seems to be some alternatives for open source software.
>>>>>>>>>>>>> I have not tried it.
>>>>>>>>>>>>> Like
>>>>>>>>>>>>> https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>>>
>>>>>>>>>>>>> <image.png>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>>>
>>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>>>> stand corrected
>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>> archive...
>>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is on-going discussion about the communication
>>>>>>>>>>>>>>> channels outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <
>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official
>>>>>>>>>>>>>>>> process for ASF archiving. Since our PMC are the owner of the slack
>>>>>>>>>>>>>>>> workspace, we can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF
>>>>>>>>>>>>>>>>> archive before starting anything official.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com>
>>>>>>>>>>>>>>>>>> 于2023年3月28日周二 07:07写道：
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd
>>>>>>>>>>>>>>>>>>>>> like to join :)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>>>> Norge
>>>>>>>>>>>>>
>>>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>>>
>>>>>>>>>>>>

-- 
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge

+47 480 94 297

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

Hi Shani,

I believe I am an admin so that is fine by me.

Hi Dongioon,

With regard to summarising the discussion etc, no need, It is like flogging
the dead horse, we have already discussed it enough. I don't see the point
of it.

HTH

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Tue, 4 Apr 2023 at 07:06, <sh...@gmail.com> wrote:

> Hey Dongjoon, Denny and all,
>
> I’ve created the current slack.
> All users have the option to create channels for different topics.
>
> I don’t see a reason for creating a new one.
>
> If anyone want to be admin on the current slack channel you all are
> welcome to send me a msg and I’ll grand permission.
>
> Have a great week,
> Shani Alisar
>
>
>
>
>
> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:
>
> 
> Thank you, Denny.
>
> May I interpret your comment as a request to support multiple channels in
> ASF too?
>
> > because it would allow us to create multiple channels for different
> topics
>
> Any other reasons?
>
> Dongjoon.
>
>
> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:
>
>> I do think creating a new Slack channel would be helpful because it would
>> allow us to create multiple channels for different topics - streaming,
>> graph, ML, etc.
>>
>> We would need a volunteer core to maintain it so we can keep the spirit
>> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
>> active.
>>
>>
>>
>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Shall we summarize the discussion so far?
>>>
>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>>> initiate this thread instead of "Slack" vs "Mailing list"?
>>>
>>> If ASF Slack provides what you need, is it better than creating a
>>> new Slack channel?
>>>
>>> Or, is there another reason for us to create a new Slack channel?
>>>
>>> Dongjoon.
>>>
>>>
>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> I agree, whatever individual sentiments are.
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>    view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
>>>> wrote:
>>>>
>>>>> Just to be clear, if there is no strong volunteer to make the new
>>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>>> place where there are only questions and promos but zero answers. I see
>>>>> volunteers here demanding for another channel, so I want to see us go with
>>>>> the most preferred way for these volunteers.
>>>>>
>>>>> User mailing list does not go in a good shape. I hope we give another
>>>>> try with recent technology to see whether we can gain traction - if we
>>>>> fail, the user mailing list will still be there.
>>>>>
>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>>> look into the number of mails being sent to the list.
>>>>>>
>>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>>> in 2015 and earlier.
>>>>>> Let's face the fact. User mailing list is dying, even before we start
>>>>>> discussion about alternative communication methods.
>>>>>>
>>>>>> Users never go with the way if it's just because PMC members (or ASF)
>>>>>> have preference. They are going with the way they are convenient.
>>>>>>
>>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>>> but we are also talking about the cost as well.
>>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>>> As long as we are being serious about the cost, I don't think we are
>>>>>> going to land in the way "users" are convenient.
>>>>>>
>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>>> channel.
>>>>>>>
>>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>>> communication mechanism rather than the user mailing list.
>>>>>>>
>>>>>>> The following is the number of people in the official channels.
>>>>>>>
>>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>>
>>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>>>> than the user mailing list?
>>>>>>>
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>>>>
>>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>>
>>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>>
>>>>>>>> We just need to make sure it is not the purpose of employing Slack
>>>>>>>> to move all discussions about developments, direction of the project, etc
>>>>>>>> which must happen in dev@/private@. The purpose of Slack thread
>>>>>>>> here does not seem to aim to serve the purpose.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>>
>>>>>>>>> I have used slack in anger on a customer site before. For small
>>>>>>>>> and medium size groups it is good and affordable. Alternatives have been
>>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>>> up with a freebie one.
>>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>>
>>>>>>>>> so +1 for me as well.
>>>>>>>>>
>>>>>>>>> Mich Talebzadeh,
>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    view my Linkedin profile
>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1.
>>>>>>>>>>
>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey everyone,
>>>>>>>>>>>
>>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>>
>>>>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>>>>
>>>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>>>> limit notice in their workspace).
>>>>>>>>>>>
>>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>>
>>>>>>>>>>> Have a great day,
>>>>>>>>>>> Shani
>>>>>>>>>>>
>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>>>>> discussion - but I can understand the concern.
>>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>>> up official policy around it.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Mridul
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>>> source project.
>>>>>>>>>>>>
>>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>>> $8.75 USD
>>>>>>>>>>>> 72 members
>>>>>>>>>>>> 1 month
>>>>>>>>>>>> $630 USD
>>>>>>>>>>>>
>>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>>
>>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>>> projects.
>>>>>>>>>>>>
>>>>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>>>>> have not tried it.
>>>>>>>>>>>> Like
>>>>>>>>>>>> https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>>
>>>>>>>>>>>> <image.png>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>>
>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>>
>>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>>> stand corrected
>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>>
>>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>>
>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>> archive...
>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF
>>>>>>>>>>>>>>>> archive before starting anything official.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd
>>>>>>>>>>>>>>>>>>>> like to join :)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>>> Norge
>>>>>>>>>>>>
>>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>>
>>>>>>>>>>>

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

Hi Shani,

I believe I am an admin so that is fine by me.

Hi Dongioon,

With regard to summarising the discussion etc, no need, It is like flogging
the dead horse, we have already discussed it enough. I don't see the point
of it.

HTH

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Tue, 4 Apr 2023 at 07:06, <sh...@gmail.com> wrote:

> Hey Dongjoon, Denny and all,
>
> I’ve created the current slack.
> All users have the option to create channels for different topics.
>
> I don’t see a reason for creating a new one.
>
> If anyone want to be admin on the current slack channel you all are
> welcome to send me a msg and I’ll grand permission.
>
> Have a great week,
> Shani Alisar
>
>
>
>
>
> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:
>
> 
> Thank you, Denny.
>
> May I interpret your comment as a request to support multiple channels in
> ASF too?
>
> > because it would allow us to create multiple channels for different
> topics
>
> Any other reasons?
>
> Dongjoon.
>
>
> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:
>
>> I do think creating a new Slack channel would be helpful because it would
>> allow us to create multiple channels for different topics - streaming,
>> graph, ML, etc.
>>
>> We would need a volunteer core to maintain it so we can keep the spirit
>> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
>> active.
>>
>>
>>
>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Shall we summarize the discussion so far?
>>>
>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>>> initiate this thread instead of "Slack" vs "Mailing list"?
>>>
>>> If ASF Slack provides what you need, is it better than creating a
>>> new Slack channel?
>>>
>>> Or, is there another reason for us to create a new Slack channel?
>>>
>>> Dongjoon.
>>>
>>>
>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> I agree, whatever individual sentiments are.
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>    view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
>>>> wrote:
>>>>
>>>>> Just to be clear, if there is no strong volunteer to make the new
>>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>>> place where there are only questions and promos but zero answers. I see
>>>>> volunteers here demanding for another channel, so I want to see us go with
>>>>> the most preferred way for these volunteers.
>>>>>
>>>>> User mailing list does not go in a good shape. I hope we give another
>>>>> try with recent technology to see whether we can gain traction - if we
>>>>> fail, the user mailing list will still be there.
>>>>>
>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>>> look into the number of mails being sent to the list.
>>>>>>
>>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>>> in 2015 and earlier.
>>>>>> Let's face the fact. User mailing list is dying, even before we start
>>>>>> discussion about alternative communication methods.
>>>>>>
>>>>>> Users never go with the way if it's just because PMC members (or ASF)
>>>>>> have preference. They are going with the way they are convenient.
>>>>>>
>>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>>> but we are also talking about the cost as well.
>>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>>> As long as we are being serious about the cost, I don't think we are
>>>>>> going to land in the way "users" are convenient.
>>>>>>
>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>>> channel.
>>>>>>>
>>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>>> communication mechanism rather than the user mailing list.
>>>>>>>
>>>>>>> The following is the number of people in the official channels.
>>>>>>>
>>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>>
>>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>>>> than the user mailing list?
>>>>>>>
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>>>>
>>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>>
>>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>>
>>>>>>>> We just need to make sure it is not the purpose of employing Slack
>>>>>>>> to move all discussions about developments, direction of the project, etc
>>>>>>>> which must happen in dev@/private@. The purpose of Slack thread
>>>>>>>> here does not seem to aim to serve the purpose.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>>
>>>>>>>>> I have used slack in anger on a customer site before. For small
>>>>>>>>> and medium size groups it is good and affordable. Alternatives have been
>>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>>> up with a freebie one.
>>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>>
>>>>>>>>> so +1 for me as well.
>>>>>>>>>
>>>>>>>>> Mich Talebzadeh,
>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    view my Linkedin profile
>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1.
>>>>>>>>>>
>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey everyone,
>>>>>>>>>>>
>>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>>
>>>>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>>>>
>>>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>>>> limit notice in their workspace).
>>>>>>>>>>>
>>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>>
>>>>>>>>>>> Have a great day,
>>>>>>>>>>> Shani
>>>>>>>>>>>
>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>>>>> discussion - but I can understand the concern.
>>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>>> up official policy around it.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Mridul
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>>> source project.
>>>>>>>>>>>>
>>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>>> $8.75 USD
>>>>>>>>>>>> 72 members
>>>>>>>>>>>> 1 month
>>>>>>>>>>>> $630 USD
>>>>>>>>>>>>
>>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>>
>>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>>> projects.
>>>>>>>>>>>>
>>>>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>>>>> have not tried it.
>>>>>>>>>>>> Like
>>>>>>>>>>>> https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>>
>>>>>>>>>>>> <image.png>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>>
>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>>
>>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>>> stand corrected
>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>>
>>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>>
>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>> archive...
>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF
>>>>>>>>>>>>>>>> archive before starting anything official.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd
>>>>>>>>>>>>>>>>>>>> like to join :)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>>> Norge
>>>>>>>>>>>>
>>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>>
>>>>>>>>>>>

Re: Slack for PySpark users

[sh...@gmail.com]

Hey Dongjoon, Denny and all,

  

I’ve created the current slack.

All users have the option to create channels for different topics.

  

I don’t see a reason for creating a new one.

  

If anyone want to be admin on the current slack channel you all are welcome to
send me a msg and I’ll grand permission.

  

Have a great week,

Shani Alisar

  

  

  

  

  

> On 4 Apr 2023, at 3:51, Dongjoon Hyun <do...@gmail.com> wrote:  
>  
>

> 
>
> Thank you, Denny.
>
>  
>
>
> May I interpret your comment as a request to support multiple channels in
> ASF too?
>
>  
>
>
> > because it would allow us to create multiple channels for different topics
>
>  
>
>
> Any other reasons?
>
>  
>
>
> Dongjoon.
>
>  
>
>
>  
>
>
> On Mon, Apr 3, 2023 at 5:31 PM Denny Lee
> <[denny.g.lee@gmail.com](mailto:denny.g.lee@gmail.com)> wrote:  
>
>

>> I do think creating a new Slack channel would be helpful because it would
allow us to create multiple channels for different topics - streaming, graph,
ML, etc.  
>>

>>  
>
>>

>> We would need a volunteer core to maintain it so we can keep the spirit and
letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
active.

>>

>>  
>
>>

>>  
>
>>

>>  
>
>>

>> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun
<[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)> wrote:  
>
>>

>>> Shall we summarize the discussion so far?

>>>

>>>  
>
>>>

>>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
initiate this thread instead of "Slack" vs "Mailing list"?

>>>

>>>  
>
>>>

>>> If ASF Slack provides what you need, is it better than creating a new
Slack channel?

>>>

>>>  
>
>>>

>>> Or, is there another reason for us to create a new Slack channel?

>>>

>>>  
>
>>>

>>> Dongjoon.  
>
>>>

>>>  
>
>>>

>>>  
>
>>>

>>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh
<[mich.talebzadeh@gmail.com](mailto:mich.talebzadeh@gmail.com)> wrote:  
>
>>>

>>>> I agree, whatever individual sentiments are.

>>>>

>>>>  
>
>>>>

>>>> Mich Talebzadeh,  
>
>>>>

>>>> Lead Solutions Architect/Engineering Lead

>>>>

>>>> Palantir Technologies Limited

>>>>

>>>>  
>
>>>>

>>>>  ![](https://ci3.googleusercontent.com/mail-
sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE)
**** [ view my Linkedin profile](https://www.linkedin.com/in/mich-talebzadeh-
ph-d-5205b2/)

>>>>

>>>>  
>
>>>>

>>>>  <https://en.everybodywiki.com/Mich_Talebzadeh>

>>>>

>>>>  
>>>>

>>>>  **Disclaimer:**  Use it at your own risk. Any and all responsibility for
any loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed. The
author will in no case be liable for any monetary damages arising from such
loss, damage or destruction.

>>>>

>>>>  
>>>>

>>>>  
>
>>>>

>>>>  
>
>>>>

>>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim
<[kabhwan.opensource@gmail.com](mailto:kabhwan.opensource@gmail.com)> wrote:  
>
>>>>

>>>>> Just to be clear, if there is no strong volunteer to make the new
community channel stay active, I'd probably be OK to not fork the channel. You
can see a strong counter example from #spark channel in ASF. It is the place
where there are only questions and promos but zero answers. I see volunteers
here demanding for another channel, so I want to see us go with the most
preferred way for these volunteers.

>>>>>

>>>>>  
>
>>>>>

>>>>> User mailing list does not go in a good shape. I hope we give another
try with recent technology to see whether we can gain traction - if we fail,
the user mailing list will still be there.

>>>>>

>>>>>  
>
>>>>>

>>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim
<[kabhwan.opensource@gmail.com](mailto:kabhwan.opensource@gmail.com)> wrote:  
>
>>>>>

>>>>>> The number of subscribers doesn't give any meaningful value. Please
look into the number of mails being sent to the list.

>>>>>>

>>>>>>  
>
>>>>>>

>>>>>> <ht...@spark.apache.org>  
>
>>>>>>

>>>>>> The latest month there were more than 200 emails being sent was Feb
2022, more than a year ago. It was more than 1k in 2016, and more than 2k in
2015 and earlier.

>>>>>>

>>>>>> Let's face the fact. User mailing list is dying, even before we start
discussion about alternative communication methods.

>>>>>>

>>>>>>  
>
>>>>>>

>>>>>> Users never go with the way if it's just because PMC members (or ASF)
have preference. They are going with the way they are convenient.

>>>>>>

>>>>>>  
>
>>>>>>

>>>>>> Same applies here - if ASF Slack requires a restricted invitation
mechanism then it won't work. Looks like there is a link for an invitation,
but we are also talking about the cost as well.

>>>>>>

>>>>>> <https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites>

>>>>>>

>>>>>> As long as we are being serious about the cost, I don't think we are
going to land in the way "users" are convenient.

>>>>>>

>>>>>>  
>
>>>>>>

>>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun
<[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)> wrote:  
>
>>>>>>

>>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
channel.

>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>> > It's unavoidable if "users" prefer to use an alternative
communication mechanism rather than the user mailing list.

>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>> The following is the number of people in the official channels.

>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>> \- [user@spark.apache.org](mailto:user@spark.apache.org) has 4519
subscribers.  
>
>>>>>>>

>>>>>>> - [dev@spark.apache.org](mailto:dev@spark.apache.org) has 3149
subscribers.

>>>>>>>

>>>>>>> \- ASF Official Slack channel has 602 subscribers.

>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
than the user mailing list?

>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>> Dongjoon.

>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>>  
>
>>>>>>>

>>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim
<[kabhwan.opensource@gmail.com](mailto:kabhwan.opensource@gmail.com)> wrote:  
>
>>>>>>>

>>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
section of "Open Communications", restriction of communication inside ASF
INFRA (mailing list) is more about code and decision-making.

>>>>>>>>

>>>>>>>> <https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-
hard-to-define>  
>
>>>>>>>>

>>>>>>>>  
>
>>>>>>>>

>>>>>>>> It's unavoidable if "users" prefer to use an alternative
communication mechanism rather than the user mailing list. Before Stack
Overflow days, there had been a meaningful number of questions around user@.
It's just impossible to let them go back and post to the user mailing list.

>>>>>>>>

>>>>>>>>  
>
>>>>>>>>

>>>>>>>> We just need to make sure it is not the purpose of employing Slack to
move all discussions about developments, direction of the project, etc which
must happen in dev@/private@. The purpose of Slack thread here does not seem
to aim to serve the purpose.

>>>>>>>>

>>>>>>>>  
>
>>>>>>>>

>>>>>>>>  
>
>>>>>>>>

>>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh
<[mich.talebzadeh@gmail.com](mailto:mich.talebzadeh@gmail.com)> wrote:  
>
>>>>>>>>

>>>>>>>>> Good discussions and proposals.all around.

>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>> I have used slack in anger on a customer site before. For small and
medium size groups it is good and affordable. Alternatives have been suggested
as well so those who like investigative search can agree and come up with a
freebie one.

>>>>>>>>>

>>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
dimensions than the mailing list. It is akin to a sports club using WhatsApp
groups for communication. Remember we were originally looking for space for
webinars, including Spark on Linkedin that Denney Lee suggested. I think Slack
and mailing groups can coexist happily. On a more serious note, when I joined
the user group back in 2015-2016, there was a lot of traffic. Currently we
hardly get many mails daily <> less than 5. So having a slack type medium may
improve members participation.

>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>> so +1 for me as well.

>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>> Mich Talebzadeh,

>>>>>>>>>

>>>>>>>>> Lead Solutions Architect/Engineering Lead

>>>>>>>>>

>>>>>>>>> Palantir Technologies Limited

>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>>  ![](https://ci3.googleusercontent.com/mail-
sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE)
**** [ view my Linkedin profile](https://www.linkedin.com/in/mich-talebzadeh-
ph-d-5205b2/)

>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>>  <https://en.everybodywiki.com/Mich_Talebzadeh>

>>>>>>>>>

>>>>>>>>>  
>>>>>>>>>

>>>>>>>>>  **Disclaimer:**  Use it at your own risk. Any and all
responsibility for any loss, damage or destruction of data or any other
property which may arise from relying on this email's technical content is
explicitly disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.

>>>>>>>>>

>>>>>>>>>  
>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>>  
>
>>>>>>>>>

>>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee
<[denny.g.lee@gmail.com](mailto:denny.g.lee@gmail.com)> wrote:  
>
>>>>>>>>>

>>>>>>>>>> +1.

>>>>>>>>>>

>>>>>>>>>>  
>
>>>>>>>>>>

>>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the free
Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
MLflow, etc.  
>>>>>>>>>>

>>>>>>>>>>  
>
>>>>>>>>>>

>>>>>>>>>> On Thu, Mar 30, 2023 at 14:15
<[shani.alishar@gmail.com](mailto:shani.alishar@gmail.com)> wrote:  
>
>>>>>>>>>>

>>>>>>>>>>> Hey everyone,

>>>>>>>>>>>

>>>>>>>>>>>  
>
>>>>>>>>>>>

>>>>>>>>>>> I think we should remain on a free program in slack.

>>>>>>>>>>>

>>>>>>>>>>>  
>
>>>>>>>>>>>

>>>>>>>>>>> In my option the free program is more then enough, the only down
side is we could only see the last 90 days messages.

>>>>>>>>>>>

>>>>>>>>>>>  
>
>>>>>>>>>>>

>>>>>>>>>>> From what I know the Airflow community (which has strong active
community in slack) also use the free program (You can tell by the 90 days
limit notice in their workspace).

>>>>>>>>>>>

>>>>>>>>>>>  
>
>>>>>>>>>>>

>>>>>>>>>>> You can find the pricing and features comparison between the slack
programs [here](https://slack.com/intl/en-gb/pricing) .

>>>>>>>>>>>

>>>>>>>>>>>  
>
>>>>>>>>>>>

>>>>>>>>>>> Have a great day,

>>>>>>>>>>>

>>>>>>>>>>> Shani

>>>>>>>>>>>

>>>>>>>>>>>  
>
>>>>>>>>>>>

>>>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan
<[mridul@gmail.com](mailto:mridul@gmail.com)> wrote:  
>  
>
>>>>>>>>>>>

>>>>>>>>>>>> 

>>>>>>>>>>>

>>>>>>>>>>>>  
>
>>>>>>>>>>>>

>>>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
discussion - but I can understand the concern.

>>>>>>>>>>>>

>>>>>>>>>>>> Would be great if you or Matei could update the thread on the
result of deliberations, once it reaches a logical consensus: before we set up
official policy around it.

>>>>>>>>>>>>

>>>>>>>>>>>>  
>
>>>>>>>>>>>>

>>>>>>>>>>>> Regards,

>>>>>>>>>>>>

>>>>>>>>>>>> Mridul

>>>>>>>>>>>>

>>>>>>>>>>>>  
>
>>>>>>>>>>>>

>>>>>>>>>>>>  
>
>>>>>>>>>>>>

>>>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen
<[bjornjorgensen@gmail.com](mailto:bjornjorgensen@gmail.com)> wrote:  
>
>>>>>>>>>>>>

>>>>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
for everyone to say hello. Or to dare to ask about small or big matters that
you would not have dared to ask about before on mailing lists.

>>>>>>>>>>>>>

>>>>>>>>>>>>> But then there is the price and what is the best for an open
source project.

>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> The price for using slack is expensive.

>>>>>>>>>>>>>

>>>>>>>>>>>>> Right now for those that have join spark slack

>>>>>>>>>>>>>

>>>>>>>>>>>>> $8.75 USD  
> 72 members  
> 1 month  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> $630 USD

>>>>>>>>>>>>>

>>>>>>>>>>>>>
[https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2](https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2)  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> And they - slack does not have an option for open source
projects.  
>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> There seems to be some alternatives for open source software. I
have not tried it.

>>>>>>>>>>>>>

>>>>>>>>>>>>> Like <https://www.rocket.chat/blog/slack-open-source-
alternatives>

>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> <image.png>

>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> rocket chat is open source
<https://github.com/RocketChat/Rocket.Chat>

>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh
<[mich.talebzadeh@gmail.com](mailto:mich.talebzadeh@gmail.com)>:  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>>> Hi Dongjoon

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> to your points if I may

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> \- Do you have any reference from other official ASF-related
Slack channels?

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>    No, I don't have any reference from other official ASF-
related Slack channels because I don't think that matters. However, I stand
corrected

>>>>>>>>>>>>>>

>>>>>>>>>>>>>> \- To be clear, I intentionally didn't refer to any specific
mailing list because we didn't set up any rule here yet.  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>    fair enough

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> going back to your original point

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
Slack activities created an isolated silo outside of ASF mailing list
archive...

>>>>>>>>>>>>>>

>>>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
source software everywhere. One way or other they do help getting community
(inside the user groups and other) to get interested and involved. Slack
happens to be one of them.  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> I am of the opinion that creating such silos is already a
reality and we ought to be pragmatic. Unless there is an overriding reason, we
should embrace it as slack can co-exist with the other mailing lists and
channels like linkedin etc.  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> Hope this clarifies my position

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> Mich Talebzadeh,

>>>>>>>>>>>>>>

>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead

>>>>>>>>>>>>>>

>>>>>>>>>>>>>> Palantir Technologies Limited

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  ![](https://ci3.googleusercontent.com/mail-
sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE)
**** [ view my Linkedin profile](https://www.linkedin.com/in/mich-talebzadeh-
ph-d-5205b2/)

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  <https://en.everybodywiki.com/Mich_Talebzadeh>

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  **Disclaimer:**  Use it at your own risk. Any and all
responsibility for any loss, damage or destruction of data or any other
property which may arise from relying on this email's technical content is
explicitly disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.

>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun
<[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)> wrote:  
>
>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> To Mich.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> \- Do you have any reference from other official ASF-related
Slack channels?

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> \- To be clear, I intentionally didn't refer to any specific
mailing list because we didn't set up any rule here yet.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
added Matei from your side.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> > I did not see an objection from the ASF board.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> There is on-going discussion about the communication channels
outside ASF email which is specifically concerning Slack.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> Please hold on any official action for this topic. We will
know how to support it seamlessly.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> Dongjoon.

>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li
<[gatorsmile@gmail.com](mailto:gatorsmile@gmail.com)> wrote:  
>
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>> Hi, Dongjoon,

>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
their own Slack workspaces last year. I did not see an objection from the ASF
board. At the same time, Slack workspaces are very popular and useful in most
non-ASF open source communities. TBH, we are kind of late. I think we can do
the same in our community?  
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
for ASF archiving. Since our PMC are the owner of the slack workspace, we can
make a change based on the policy. WDYT?

>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>> Xiao

>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>> Dongjoon Hyun
<[dongjoon.hyun@gmail.com](mailto:dongjoon.hyun@gmail.com)> 于2023年3月30日周四
09:03写道：  
>
>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> Hi, Xiao and all.

>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> (cc Matei)  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> Please hold on the vote.

>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
Slack activities created an isolated silo outside of ASF mailing list archive.

>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
before starting anything official.

>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> Bests,

>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> Dongjoon.

>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li
<[gatorsmile@gmail.com](mailto:gatorsmile@gmail.com)> wrote:  
>
>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>> +1

>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>> + [@dev@spark.apache.org](mailto:dev@spark.apache.org)  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
faster communication. We can do the same in Apache Spark. The Slack workspace
will be maintained by the Apache Spark PMC. I propose to initiate a vote for
the creation of a new Apache Spark Slack workspace. Does that sound good?

>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>> Cheers,

>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>> Xiao  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>> Mich Talebzadeh
<[mich.talebzadeh@gmail.com](mailto:mich.talebzadeh@gmail.com)> 于2023年3月28日周二
07:07写道：  
>
>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>> I created one at slack called pyspark

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>> Mich Talebzadeh,

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>> Palantir Technologies Limited

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  ![](https://ci3.googleusercontent.com/mail-
sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE)
**** [ view my Linkedin profile](https://www.linkedin.com/in/mich-talebzadeh-
ph-d-5205b2/)

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  <https://en.everybodywiki.com/Mich_Talebzadeh>

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  **Disclaimer:**  Use it at your own risk. Any and all
responsibility for any loss, damage or destruction of data or any other
property which may arise from relying on this email's technical content is
explicitly disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.

>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli
<[zgolliasma@gmail.com](mailto:zgolliasma@gmail.com)> wrote:  
>
>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai
<[weiruanlysy@gmail.com](mailto:weiruanlysy@gmail.com)> a écrit :  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
to join :)

>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> Thank You & Best Regards

>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> Winston Lai

>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> * * *

>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> **From:** Denny Lee
<[denny.g.lee@gmail.com](mailto:denny.g.lee@gmail.com)>  
>  **Sent:** Tuesday, March 28, 2023 9:43:08 AM  
>  **To:** Hyukjin Kwon <[gurwls223@gmail.com](mailto:gurwls223@gmail.com)>  
>  **Cc:** keen <[keen85@gmx.net](mailto:keen85@gmx.net)>;
> [user@spark.apache.org](mailto:user@spark.apache.org)
> <[user@spark.apache.org](mailto:user@spark.apache.org)>  
>  **Subject:** Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!  
>
>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)> wrote:  
>
>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
channel so we can easily discuss with users and developers.

>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen
<[keen85@gmx.net](mailto:keen85@gmx.net)> wrote:  
>
>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>> Hi all,  
>  I really like **Slack** as communication channel for a tech community.  
>  There is a Slack workspace for **delta lake users**
> (<https://go.delta.io/slack>) that I enjoy a lot.  
>  I was wondering if there is something similar for PySpark users.  
>  
>
>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
new Slack workspace for PySpark users? (when explicitly mentioning that this
is _not_ officially part of Apache Spark)?

>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>> Cheers

>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>> Martin

>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>> \--  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>> Asma ZGOLLI

>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning  
>
>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>>  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> \--  
>
>>>>>>>>>>>>>

>>>>>>>>>>>>> Bjørn Jørgensen  
> [Vestre Aspehaug
> 4](https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g),
> 6010 Ålesund  
> Norge  
>  
> +47 480 94 297

Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

Thank you, Denny.

May I interpret your comment as a request to support multiple channels in
ASF too?

> because it would allow us to create multiple channels for different topics


Any other reasons?

Dongjoon.


On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:

> I do think creating a new Slack channel would be helpful because it would
> allow us to create multiple channels for different topics - streaming,
> graph, ML, etc.
>
> We would need a volunteer core to maintain it so we can keep the spirit
> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
> active.
>
>
>
> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Shall we summarize the discussion so far?
>>
>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>> initiate this thread instead of "Slack" vs "Mailing list"?
>>
>> If ASF Slack provides what you need, is it better than creating a
>> new Slack channel?
>>
>> Or, is there another reason for us to create a new Slack channel?
>>
>> Dongjoon.
>>
>>
>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>>> I agree, whatever individual sentiments are.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>    view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
>>> wrote:
>>>
>>>> Just to be clear, if there is no strong volunteer to make the new
>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>> place where there are only questions and promos but zero answers. I see
>>>> volunteers here demanding for another channel, so I want to see us go with
>>>> the most preferred way for these volunteers.
>>>>
>>>> User mailing list does not go in a good shape. I hope we give another
>>>> try with recent technology to see whether we can gain traction - if we
>>>> fail, the user mailing list will still be there.
>>>>
>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>> look into the number of mails being sent to the list.
>>>>>
>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>> in 2015 and earlier.
>>>>> Let's face the fact. User mailing list is dying, even before we start
>>>>> discussion about alternative communication methods.
>>>>>
>>>>> Users never go with the way if it's just because PMC members (or ASF)
>>>>> have preference. They are going with the way they are convenient.
>>>>>
>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>> but we are also talking about the cost as well.
>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>> As long as we are being serious about the cost, I don't think we are
>>>>> going to land in the way "users" are convenient.
>>>>>
>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>> channel.
>>>>>>
>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>> communication mechanism rather than the user mailing list.
>>>>>>
>>>>>> The following is the number of people in the official channels.
>>>>>>
>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>
>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>>> than the user mailing list?
>>>>>>
>>>>>> Dongjoon.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>>>
>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>
>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>
>>>>>>> We just need to make sure it is not the purpose of employing Slack
>>>>>>> to move all discussions about developments, direction of the project, etc
>>>>>>> which must happen in dev@/private@. The purpose of Slack thread
>>>>>>> here does not seem to aim to serve the purpose.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>
>>>>>>>> I have used slack in anger on a customer site before. For small and
>>>>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>> up with a freebie one.
>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>
>>>>>>>> so +1 for me as well.
>>>>>>>>
>>>>>>>> Mich Talebzadeh,
>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>> Palantir Technologies Limited
>>>>>>>>
>>>>>>>>
>>>>>>>>    view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1.
>>>>>>>>>
>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey everyone,
>>>>>>>>>>
>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>
>>>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>>>
>>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>>> limit notice in their workspace).
>>>>>>>>>>
>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>
>>>>>>>>>> Have a great day,
>>>>>>>>>> Shani
>>>>>>>>>>
>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>>>> discussion - but I can understand the concern.
>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>> up official policy around it.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Mridul
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>> source project.
>>>>>>>>>>>
>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>> $8.75 USD
>>>>>>>>>>> 72 members
>>>>>>>>>>> 1 month
>>>>>>>>>>> $630 USD
>>>>>>>>>>>
>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>
>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>> projects.
>>>>>>>>>>>
>>>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>>>> have not tried it.
>>>>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>
>>>>>>>>>>> <image.png>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>
>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>
>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>
>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>> stand corrected
>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>
>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>
>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>> archive...
>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>
>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
>>>>>>>>>>>>>>>>>>> to join :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>> Norge
>>>>>>>>>>>
>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>
>>>>>>>>>>

Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

Thank you, Denny.

May I interpret your comment as a request to support multiple channels in
ASF too?

> because it would allow us to create multiple channels for different topics


Any other reasons?

Dongjoon.


On Mon, Apr 3, 2023 at 5:31 PM Denny Lee <de...@gmail.com> wrote:

> I do think creating a new Slack channel would be helpful because it would
> allow us to create multiple channels for different topics - streaming,
> graph, ML, etc.
>
> We would need a volunteer core to maintain it so we can keep the spirit
> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
> active.
>
>
>
> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Shall we summarize the discussion so far?
>>
>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>> initiate this thread instead of "Slack" vs "Mailing list"?
>>
>> If ASF Slack provides what you need, is it better than creating a
>> new Slack channel?
>>
>> Or, is there another reason for us to create a new Slack channel?
>>
>> Dongjoon.
>>
>>
>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>>> I agree, whatever individual sentiments are.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>    view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
>>> wrote:
>>>
>>>> Just to be clear, if there is no strong volunteer to make the new
>>>> community channel stay active, I'd probably be OK to not fork the channel.
>>>> You can see a strong counter example from #spark channel in ASF. It is the
>>>> place where there are only questions and promos but zero answers. I see
>>>> volunteers here demanding for another channel, so I want to see us go with
>>>> the most preferred way for these volunteers.
>>>>
>>>> User mailing list does not go in a good shape. I hope we give another
>>>> try with recent technology to see whether we can gain traction - if we
>>>> fail, the user mailing list will still be there.
>>>>
>>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> The number of subscribers doesn't give any meaningful value. Please
>>>>> look into the number of mails being sent to the list.
>>>>>
>>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>>> The latest month there were more than 200 emails being sent was Feb
>>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>>> in 2015 and earlier.
>>>>> Let's face the fact. User mailing list is dying, even before we start
>>>>> discussion about alternative communication methods.
>>>>>
>>>>> Users never go with the way if it's just because PMC members (or ASF)
>>>>> have preference. They are going with the way they are convenient.
>>>>>
>>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>>> but we are also talking about the cost as well.
>>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>>> As long as we are being serious about the cost, I don't think we are
>>>>> going to land in the way "users" are convenient.
>>>>>
>>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>>> channel.
>>>>>>
>>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>>> communication mechanism rather than the user mailing list.
>>>>>>
>>>>>> The following is the number of people in the official channels.
>>>>>>
>>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>>
>>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>>> than the user mailing list?
>>>>>>
>>>>>> Dongjoon.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>>>
>>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>>
>>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>>
>>>>>>> We just need to make sure it is not the purpose of employing Slack
>>>>>>> to move all discussions about developments, direction of the project, etc
>>>>>>> which must happen in dev@/private@. The purpose of Slack thread
>>>>>>> here does not seem to aim to serve the purpose.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Good discussions and proposals.all around.
>>>>>>>>
>>>>>>>> I have used slack in anger on a customer site before. For small and
>>>>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>>> up with a freebie one.
>>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>>> a slack type medium may improve members participation.
>>>>>>>>
>>>>>>>> so +1 for me as well.
>>>>>>>>
>>>>>>>> Mich Talebzadeh,
>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>> Palantir Technologies Limited
>>>>>>>>
>>>>>>>>
>>>>>>>>    view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1.
>>>>>>>>>
>>>>>>>>> To Shani’s point, there are multiple OSS projects that use the
>>>>>>>>> free Slack version - top of mind include Delta, Presto, Flink, Trino,
>>>>>>>>> Datahub, MLflow, etc.
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey everyone,
>>>>>>>>>>
>>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>>
>>>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>>>
>>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>>> limit notice in their workspace).
>>>>>>>>>>
>>>>>>>>>> You can find the pricing and features comparison between the
>>>>>>>>>> slack programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>>
>>>>>>>>>> Have a great day,
>>>>>>>>>> Shani
>>>>>>>>>>
>>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>>>> discussion - but I can understand the concern.
>>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>>> up official policy around it.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Mridul
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>>> source project.
>>>>>>>>>>>
>>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>>> $8.75 USD
>>>>>>>>>>> 72 members
>>>>>>>>>>> 1 month
>>>>>>>>>>> $630 USD
>>>>>>>>>>>
>>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>>
>>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>>> projects.
>>>>>>>>>>>
>>>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>>>> have not tried it.
>>>>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>>
>>>>>>>>>>> <image.png>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> rocket chat is open source
>>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>>
>>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>>
>>>>>>>>>>>> to your points if I may
>>>>>>>>>>>>
>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>>> stand corrected
>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>    fair enough
>>>>>>>>>>>>
>>>>>>>>>>>> going back to your original point
>>>>>>>>>>>>
>>>>>>>>>>>> ..There is a concern expressed by ASF board because recent
>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>> archive...
>>>>>>>>>>>> Well, there are activities on Spark and indeed other open
>>>>>>>>>>>> source software everywhere. One way or other they do help getting community
>>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>>
>>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> To Mich.
>>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>>> Please hold on any official action for this topic. We will
>>>>>>>>>>>>> know how to support it seamlessly.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四
>>>>>>>>>>>>>> 09:03写道：
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g.,
>>>>>>>>>>>>>>>> Pinot, Druid, Flink) have created their own dedicated Slack workspaces for
>>>>>>>>>>>>>>>> faster communication. We can do the same in Apache Spark. The Slack
>>>>>>>>>>>>>>>> workspace will be maintained by the Apache Spark PMC. I propose to initiate
>>>>>>>>>>>>>>>> a vote for the creation of a new Apache Spark Slack workspace. Does that
>>>>>>>>>>>>>>>> sound good?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
>>>>>>>>>>>>>>>>>>> to join :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for
>>>>>>>>>>>>>>>>>>> PySpark users.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a
>>>>>>>>>>>>>>>>>>> new Slack workspace for PySpark users? (when explicitly mentioning that
>>>>>>>>>>>>>>>>>>> this is *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>>> 6010 Ålesund
>>>>>>>>>>> Norge
>>>>>>>>>>>
>>>>>>>>>>> +47 480 94 297
>>>>>>>>>>>
>>>>>>>>>>

Re: Slack for PySpark users

[Denny Lee <de...@gmail.com>]

I do think creating a new Slack channel would be helpful because it would
allow us to create multiple channels for different topics - streaming,
graph, ML, etc.

We would need a volunteer core to maintain it so we can keep the spirit and
letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
active.



On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com> wrote:

> Shall we summarize the discussion so far?
>
> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
> initiate this thread instead of "Slack" vs "Mailing list"?
>
> If ASF Slack provides what you need, is it better than creating a
> new Slack channel?
>
> Or, is there another reason for us to create a new Slack channel?
>
> Dongjoon.
>
>
> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> I agree, whatever individual sentiments are.
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>    view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> Just to be clear, if there is no strong volunteer to make the new
>>> community channel stay active, I'd probably be OK to not fork the channel.
>>> You can see a strong counter example from #spark channel in ASF. It is the
>>> place where there are only questions and promos but zero answers. I see
>>> volunteers here demanding for another channel, so I want to see us go with
>>> the most preferred way for these volunteers.
>>>
>>> User mailing list does not go in a good shape. I hope we give another
>>> try with recent technology to see whether we can gain traction - if we
>>> fail, the user mailing list will still be there.
>>>
>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> The number of subscribers doesn't give any meaningful value. Please
>>>> look into the number of mails being sent to the list.
>>>>
>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>> The latest month there were more than 200 emails being sent was Feb
>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>> in 2015 and earlier.
>>>> Let's face the fact. User mailing list is dying, even before we start
>>>> discussion about alternative communication methods.
>>>>
>>>> Users never go with the way if it's just because PMC members (or ASF)
>>>> have preference. They are going with the way they are convenient.
>>>>
>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>> but we are also talking about the cost as well.
>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>> As long as we are being serious about the cost, I don't think we are
>>>> going to land in the way "users" are convenient.
>>>>
>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>> channel.
>>>>>
>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>> communication mechanism rather than the user mailing list.
>>>>>
>>>>> The following is the number of people in the official channels.
>>>>>
>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>
>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>> than the user mailing list?
>>>>>
>>>>> Dongjoon.
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>>
>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>
>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>
>>>>>> We just need to make sure it is not the purpose of employing Slack to
>>>>>> move all discussions about developments, direction of the project, etc
>>>>>> which must happen in dev@/private@. The purpose of Slack thread here
>>>>>> does not seem to aim to serve the purpose.
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>>> Good discussions and proposals.all around.
>>>>>>>
>>>>>>> I have used slack in anger on a customer site before. For small and
>>>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>> up with a freebie one.
>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>> a slack type medium may improve members participation.
>>>>>>>
>>>>>>> so +1 for me as well.
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>    view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1.
>>>>>>>>
>>>>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>>>>> MLflow, etc.
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hey everyone,
>>>>>>>>>
>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>
>>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>>
>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>> limit notice in their workspace).
>>>>>>>>>
>>>>>>>>> You can find the pricing and features comparison between the slack
>>>>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>
>>>>>>>>> Have a great day,
>>>>>>>>> Shani
>>>>>>>>>
>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>>> discussion - but I can understand the concern.
>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>> up official policy around it.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Mridul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>> source project.
>>>>>>>>>>
>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>> $8.75 USD
>>>>>>>>>> 72 members
>>>>>>>>>> 1 month
>>>>>>>>>> $630 USD
>>>>>>>>>>
>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>
>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>> projects.
>>>>>>>>>>
>>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>>> have not tried it.
>>>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>
>>>>>>>>>> <image.png>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> rocket chat is open source
>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>
>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>
>>>>>>>>>>> to your points if I may
>>>>>>>>>>>
>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>> Slack channels?
>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>> stand corrected
>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>    fair enough
>>>>>>>>>>>
>>>>>>>>>>> going back to your original point
>>>>>>>>>>>
>>>>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>
>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> To Mich.
>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>
>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>
>>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>>>>> how to support it seamlessly.
>>>>>>>>>>>>
>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>
>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
>>>>>>>>>>>>>>>>>> to join :)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>> 6010 Ålesund
>>>>>>>>>> Norge
>>>>>>>>>>
>>>>>>>>>> +47 480 94 297
>>>>>>>>>>
>>>>>>>>>

Re: Slack for PySpark users

[Denny Lee <de...@gmail.com>]

I do think creating a new Slack channel would be helpful because it would
allow us to create multiple channels for different topics - streaming,
graph, ML, etc.

We would need a volunteer core to maintain it so we can keep the spirit and
letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
active.



On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun <do...@gmail.com> wrote:

> Shall we summarize the discussion so far?
>
> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
> initiate this thread instead of "Slack" vs "Mailing list"?
>
> If ASF Slack provides what you need, is it better than creating a
> new Slack channel?
>
> Or, is there another reason for us to create a new Slack channel?
>
> Dongjoon.
>
>
> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> I agree, whatever individual sentiments are.
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>    view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> Just to be clear, if there is no strong volunteer to make the new
>>> community channel stay active, I'd probably be OK to not fork the channel.
>>> You can see a strong counter example from #spark channel in ASF. It is the
>>> place where there are only questions and promos but zero answers. I see
>>> volunteers here demanding for another channel, so I want to see us go with
>>> the most preferred way for these volunteers.
>>>
>>> User mailing list does not go in a good shape. I hope we give another
>>> try with recent technology to see whether we can gain traction - if we
>>> fail, the user mailing list will still be there.
>>>
>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> The number of subscribers doesn't give any meaningful value. Please
>>>> look into the number of mails being sent to the list.
>>>>
>>>> https://lists.apache.org/list.html?user@spark.apache.org
>>>> The latest month there were more than 200 emails being sent was Feb
>>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>>> in 2015 and earlier.
>>>> Let's face the fact. User mailing list is dying, even before we start
>>>> discussion about alternative communication methods.
>>>>
>>>> Users never go with the way if it's just because PMC members (or ASF)
>>>> have preference. They are going with the way they are convenient.
>>>>
>>>> Same applies here - if ASF Slack requires a restricted invitation
>>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>>> but we are also talking about the cost as well.
>>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>>> As long as we are being serious about the cost, I don't think we are
>>>> going to land in the way "users" are convenient.
>>>>
>>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>>> channel.
>>>>>
>>>>> > It's unavoidable if "users" prefer to use an alternative
>>>>> communication mechanism rather than the user mailing list.
>>>>>
>>>>> The following is the number of people in the official channels.
>>>>>
>>>>> - user@spark.apache.org has 4519 subscribers.
>>>>> - dev@spark.apache.org has 3149 subscribers.
>>>>> - ASF Official Slack channel has 602 subscribers.
>>>>>
>>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>>> than the user mailing list?
>>>>>
>>>>> Dongjoon.
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>>
>>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>>
>>>>>> It's unavoidable if "users" prefer to use an alternative
>>>>>> communication mechanism rather than the user mailing list. Before Stack
>>>>>> Overflow days, there had been a meaningful number of questions around user@.
>>>>>> It's just impossible to let them go back and post to the user mailing list.
>>>>>>
>>>>>> We just need to make sure it is not the purpose of employing Slack to
>>>>>> move all discussions about developments, direction of the project, etc
>>>>>> which must happen in dev@/private@. The purpose of Slack thread here
>>>>>> does not seem to aim to serve the purpose.
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>>> Good discussions and proposals.all around.
>>>>>>>
>>>>>>> I have used slack in anger on a customer site before. For small and
>>>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>>> up with a freebie one.
>>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>>> a slack type medium may improve members participation.
>>>>>>>
>>>>>>> so +1 for me as well.
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>    view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1.
>>>>>>>>
>>>>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>>>>> MLflow, etc.
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hey everyone,
>>>>>>>>>
>>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>>
>>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>>
>>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>>> limit notice in their workspace).
>>>>>>>>>
>>>>>>>>> You can find the pricing and features comparison between the slack
>>>>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>>
>>>>>>>>> Have a great day,
>>>>>>>>> Shani
>>>>>>>>>
>>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>>> discussion - but I can understand the concern.
>>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>>> up official policy around it.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Mridul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>>> source project.
>>>>>>>>>>
>>>>>>>>>> The price for using slack is expensive.
>>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>>> $8.75 USD
>>>>>>>>>> 72 members
>>>>>>>>>> 1 month
>>>>>>>>>> $630 USD
>>>>>>>>>>
>>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>>
>>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>>> projects.
>>>>>>>>>>
>>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>>> have not tried it.
>>>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>>
>>>>>>>>>> <image.png>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> rocket chat is open source
>>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>>
>>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>>
>>>>>>>>>>> to your points if I may
>>>>>>>>>>>
>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>> Slack channels?
>>>>>>>>>>>    No, I don't have any reference from other official
>>>>>>>>>>> ASF-related Slack channels because I don't think that matters. However, I
>>>>>>>>>>> stand corrected
>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>    fair enough
>>>>>>>>>>>
>>>>>>>>>>> going back to your original point
>>>>>>>>>>>
>>>>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>>> happens to be one of them.
>>>>>>>>>>> I am of the opinion that creating such silos is already a
>>>>>>>>>>> reality and we ought to be pragmatic. Unless there is an overriding reason,
>>>>>>>>>>> we should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>>
>>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> To Mich.
>>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>>> Slack channels?
>>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>>
>>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>>
>>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>>>>> how to support it seamlessly.
>>>>>>>>>>>>
>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created
>>>>>>>>>>>>> their own Slack workspaces last year. I did not see an objection from the
>>>>>>>>>>>>> ASF board. At the same time, Slack workspaces are very popular and useful
>>>>>>>>>>>>> in most non-ASF open source communities. TBH, we are kind of late. I think
>>>>>>>>>>>>> we can do the same in our community?
>>>>>>>>>>>>>
>>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is a concern expressed by ASF board because recent
>>>>>>>>>>>>>> Slack activities created an isolated silo outside of ASF mailing list
>>>>>>>>>>>>>> archive.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <
>>>>>>>>>>>>>> gatorsmile@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
>>>>>>>>>>>>>>>>>> to join :)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a
>>>>>>>>>>>>>>>>>> tech community.
>>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Bjørn Jørgensen
>>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>>> 6010 Ålesund
>>>>>>>>>> Norge
>>>>>>>>>>
>>>>>>>>>> +47 480 94 297
>>>>>>>>>>
>>>>>>>>>

Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

Shall we summarize the discussion so far?

To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
initiate this thread instead of "Slack" vs "Mailing list"?

If ASF Slack provides what you need, is it better than creating a new Slack
channel?

Or, is there another reason for us to create a new Slack channel?

Dongjoon.


On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <mi...@gmail.com>
wrote:

> I agree, whatever individual sentiments are.
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>    view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> Just to be clear, if there is no strong volunteer to make the new
>> community channel stay active, I'd probably be OK to not fork the channel.
>> You can see a strong counter example from #spark channel in ASF. It is the
>> place where there are only questions and promos but zero answers. I see
>> volunteers here demanding for another channel, so I want to see us go with
>> the most preferred way for these volunteers.
>>
>> User mailing list does not go in a good shape. I hope we give another try
>> with recent technology to see whether we can gain traction - if we fail,
>> the user mailing list will still be there.
>>
>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> The number of subscribers doesn't give any meaningful value. Please look
>>> into the number of mails being sent to the list.
>>>
>>> https://lists.apache.org/list.html?user@spark.apache.org
>>> The latest month there were more than 200 emails being sent was Feb
>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>> in 2015 and earlier.
>>> Let's face the fact. User mailing list is dying, even before we start
>>> discussion about alternative communication methods.
>>>
>>> Users never go with the way if it's just because PMC members (or ASF)
>>> have preference. They are going with the way they are convenient.
>>>
>>> Same applies here - if ASF Slack requires a restricted invitation
>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>> but we are also talking about the cost as well.
>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>> As long as we are being serious about the cost, I don't think we are
>>> going to land in the way "users" are convenient.
>>>
>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>> channel.
>>>>
>>>> > It's unavoidable if "users" prefer to use an alternative
>>>> communication mechanism rather than the user mailing list.
>>>>
>>>> The following is the number of people in the official channels.
>>>>
>>>> - user@spark.apache.org has 4519 subscribers.
>>>> - dev@spark.apache.org has 3149 subscribers.
>>>> - ASF Official Slack channel has 602 subscribers.
>>>>
>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>> than the user mailing list?
>>>>
>>>> Dongjoon.
>>>>
>>>>
>>>>
>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>
>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>
>>>>> It's unavoidable if "users" prefer to use an alternative communication
>>>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>>>> there had been a meaningful number of questions around user@. It's
>>>>> just impossible to let them go back and post to the user mailing list.
>>>>>
>>>>> We just need to make sure it is not the purpose of employing Slack to
>>>>> move all discussions about developments, direction of the project, etc
>>>>> which must happen in dev@/private@. The purpose of Slack thread here
>>>>> does not seem to aim to serve the purpose.
>>>>>
>>>>>
>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> Good discussions and proposals.all around.
>>>>>>
>>>>>> I have used slack in anger on a customer site before. For small and
>>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>> up with a freebie one.
>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>> a slack type medium may improve members participation.
>>>>>>
>>>>>> so +1 for me as well.
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>    view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1.
>>>>>>>
>>>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>>>> MLflow, etc.
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hey everyone,
>>>>>>>>
>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>
>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>
>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>> limit notice in their workspace).
>>>>>>>>
>>>>>>>> You can find the pricing and features comparison between the slack
>>>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>
>>>>>>>> Have a great day,
>>>>>>>> Shani
>>>>>>>>
>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> 
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>> discussion - but I can understand the concern.
>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>> up official policy around it.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Mridul
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>> source project.
>>>>>>>>>
>>>>>>>>> The price for using slack is expensive.
>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>> $8.75 USD
>>>>>>>>> 72 members
>>>>>>>>> 1 month
>>>>>>>>> $630 USD
>>>>>>>>>
>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>
>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>> projects.
>>>>>>>>>
>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>> have not tried it.
>>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>
>>>>>>>>> <image.png>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> rocket chat is open source
>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>
>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>
>>>>>>>>>> to your points if I may
>>>>>>>>>>
>>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>>> channels?
>>>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>>>> corrected
>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>    fair enough
>>>>>>>>>>
>>>>>>>>>> going back to your original point
>>>>>>>>>>
>>>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>> happens to be one of them.
>>>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>
>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>
>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> To Mich.
>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>> Slack channels?
>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>
>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>
>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>>>> how to support it seamlessly.
>>>>>>>>>>>
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>
>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>>>> can do the same in our community?
>>>>>>>>>>>>
>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>
>>>>>>>>>>>> Xiao
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>
>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
>>>>>>>>>>>>>>>>> to join :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Bjørn Jørgensen
>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>> 6010 Ålesund
>>>>>>>>> Norge
>>>>>>>>>
>>>>>>>>> +47 480 94 297
>>>>>>>>>
>>>>>>>>

Re: Slack for PySpark users

[Dongjoon Hyun <do...@gmail.com>]

Shall we summarize the discussion so far?

To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
initiate this thread instead of "Slack" vs "Mailing list"?

If ASF Slack provides what you need, is it better than creating a new Slack
channel?

Or, is there another reason for us to create a new Slack channel?

Dongjoon.


On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh <mi...@gmail.com>
wrote:

> I agree, whatever individual sentiments are.
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>    view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> Just to be clear, if there is no strong volunteer to make the new
>> community channel stay active, I'd probably be OK to not fork the channel.
>> You can see a strong counter example from #spark channel in ASF. It is the
>> place where there are only questions and promos but zero answers. I see
>> volunteers here demanding for another channel, so I want to see us go with
>> the most preferred way for these volunteers.
>>
>> User mailing list does not go in a good shape. I hope we give another try
>> with recent technology to see whether we can gain traction - if we fail,
>> the user mailing list will still be there.
>>
>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> The number of subscribers doesn't give any meaningful value. Please look
>>> into the number of mails being sent to the list.
>>>
>>> https://lists.apache.org/list.html?user@spark.apache.org
>>> The latest month there were more than 200 emails being sent was Feb
>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>> in 2015 and earlier.
>>> Let's face the fact. User mailing list is dying, even before we start
>>> discussion about alternative communication methods.
>>>
>>> Users never go with the way if it's just because PMC members (or ASF)
>>> have preference. They are going with the way they are convenient.
>>>
>>> Same applies here - if ASF Slack requires a restricted invitation
>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>> but we are also talking about the cost as well.
>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>> As long as we are being serious about the cost, I don't think we are
>>> going to land in the way "users" are convenient.
>>>
>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>>> channel.
>>>>
>>>> > It's unavoidable if "users" prefer to use an alternative
>>>> communication mechanism rather than the user mailing list.
>>>>
>>>> The following is the number of people in the official channels.
>>>>
>>>> - user@spark.apache.org has 4519 subscribers.
>>>> - dev@spark.apache.org has 3149 subscribers.
>>>> - ASF Official Slack channel has 602 subscribers.
>>>>
>>>> May I ask if the users prefer to use the ASF Official Slack channel
>>>> than the user mailing list?
>>>>
>>>> Dongjoon.
>>>>
>>>>
>>>>
>>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>>> section of "Open Communications", restriction of communication inside ASF
>>>>> INFRA (mailing list) is more about code and decision-making.
>>>>>
>>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>>
>>>>> It's unavoidable if "users" prefer to use an alternative communication
>>>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>>>> there had been a meaningful number of questions around user@. It's
>>>>> just impossible to let them go back and post to the user mailing list.
>>>>>
>>>>> We just need to make sure it is not the purpose of employing Slack to
>>>>> move all discussions about developments, direction of the project, etc
>>>>> which must happen in dev@/private@. The purpose of Slack thread here
>>>>> does not seem to aim to serve the purpose.
>>>>>
>>>>>
>>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> Good discussions and proposals.all around.
>>>>>>
>>>>>> I have used slack in anger on a customer site before. For small and
>>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>>> suggested as well so those who like investigative search can agree and come
>>>>>> up with a freebie one.
>>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>>> a slack type medium may improve members participation.
>>>>>>
>>>>>> so +1 for me as well.
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>    view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1.
>>>>>>>
>>>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>>>> MLflow, etc.
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hey everyone,
>>>>>>>>
>>>>>>>> I think we should remain on a free program in slack.
>>>>>>>>
>>>>>>>> In my option the free program is more then enough, the only down
>>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>>
>>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>>> limit notice in their workspace).
>>>>>>>>
>>>>>>>> You can find the pricing and features comparison between the slack
>>>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>>
>>>>>>>> Have a great day,
>>>>>>>> Shani
>>>>>>>>
>>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> 
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>>> discussion - but I can understand the concern.
>>>>>>>> Would be great if you or Matei could update the thread on the
>>>>>>>> result of deliberations, once it reaches a logical consensus: before we set
>>>>>>>> up official policy around it.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Mridul
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I like the idea of having a talk channel. It can make it easier
>>>>>>>>> for everyone to say hello. Or to dare to ask about small or big matters
>>>>>>>>> that you would not have dared to ask about before on mailing lists.
>>>>>>>>> But then there is the price and what is the best for an open
>>>>>>>>> source project.
>>>>>>>>>
>>>>>>>>> The price for using slack is expensive.
>>>>>>>>> Right now for those that have join spark slack
>>>>>>>>> $8.75 USD
>>>>>>>>> 72 members
>>>>>>>>> 1 month
>>>>>>>>> $630 USD
>>>>>>>>>
>>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>>
>>>>>>>>> And they - slack does not have an option for open source
>>>>>>>>> projects.
>>>>>>>>>
>>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>>> have not tried it.
>>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>>
>>>>>>>>> <image.png>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> rocket chat is open source
>>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>>
>>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi Dongjoon
>>>>>>>>>>
>>>>>>>>>> to your points if I may
>>>>>>>>>>
>>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>>> channels?
>>>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>>>> corrected
>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>    fair enough
>>>>>>>>>>
>>>>>>>>>> going back to your original point
>>>>>>>>>>
>>>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>>> happens to be one of them.
>>>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>>> channels like linkedin etc.
>>>>>>>>>>
>>>>>>>>>> Hope this clarifies my position
>>>>>>>>>>
>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> To Mich.
>>>>>>>>>>> - Do you have any reference from other official ASF-related
>>>>>>>>>>> Slack channels?
>>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>>
>>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>>> added Matei from your side.
>>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>>
>>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>>>> how to support it seamlessly.
>>>>>>>>>>>
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>>
>>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>>>> can do the same in our community?
>>>>>>>>>>>>
>>>>>>>>>>>> We can follow the guide when the ASF has an official process
>>>>>>>>>>>> for ASF archiving. Since our PMC are the owner of the slack workspace, we
>>>>>>>>>>>> can make a change based on the policy. WDYT?
>>>>>>>>>>>>
>>>>>>>>>>>> Xiao
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>>
>>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bests,
>>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like
>>>>>>>>>>>>>>>>> to join :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Bjørn Jørgensen
>>>>>>>>> Vestre Aspehaug 4
>>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>>> 6010 Ålesund
>>>>>>>>> Norge
>>>>>>>>>
>>>>>>>>> +47 480 94 297
>>>>>>>>>
>>>>>>>>

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

I agree, whatever individual sentiments are.

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
wrote:

> Just to be clear, if there is no strong volunteer to make the new
> community channel stay active, I'd probably be OK to not fork the channel.
> You can see a strong counter example from #spark channel in ASF. It is the
> place where there are only questions and promos but zero answers. I see
> volunteers here demanding for another channel, so I want to see us go with
> the most preferred way for these volunteers.
>
> User mailing list does not go in a good shape. I hope we give another try
> with recent technology to see whether we can gain traction - if we fail,
> the user mailing list will still be there.
>
> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> The number of subscribers doesn't give any meaningful value. Please look
>> into the number of mails being sent to the list.
>>
>> https://lists.apache.org/list.html?user@spark.apache.org
>> The latest month there were more than 200 emails being sent was Feb 2022,
>> more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
>> and earlier.
>> Let's face the fact. User mailing list is dying, even before we start
>> discussion about alternative communication methods.
>>
>> Users never go with the way if it's just because PMC members (or ASF)
>> have preference. They are going with the way they are convenient.
>>
>> Same applies here - if ASF Slack requires a restricted invitation
>> mechanism then it won't work. Looks like there is a link for an invitation,
>> but we are also talking about the cost as well.
>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>> As long as we are being serious about the cost, I don't think we are
>> going to land in the way "users" are convenient.
>>
>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>> channel.
>>>
>>> > It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list.
>>>
>>> The following is the number of people in the official channels.
>>>
>>> - user@spark.apache.org has 4519 subscribers.
>>> - dev@spark.apache.org has 3149 subscribers.
>>> - ASF Official Slack channel has 602 subscribers.
>>>
>>> May I ask if the users prefer to use the ASF Official Slack channel
>>> than the user mailing list?
>>>
>>> Dongjoon.
>>>
>>>
>>>
>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>> section of "Open Communications", restriction of communication inside ASF
>>>> INFRA (mailing list) is more about code and decision-making.
>>>>
>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>
>>>> It's unavoidable if "users" prefer to use an alternative communication
>>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>>> there had been a meaningful number of questions around user@. It's
>>>> just impossible to let them go back and post to the user mailing list.
>>>>
>>>> We just need to make sure it is not the purpose of employing Slack to
>>>> move all discussions about developments, direction of the project, etc
>>>> which must happen in dev@/private@. The purpose of Slack thread here
>>>> does not seem to aim to serve the purpose.
>>>>
>>>>
>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Good discussions and proposals.all around.
>>>>>
>>>>> I have used slack in anger on a customer site before. For small and
>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>> suggested as well so those who like investigative search can agree and come
>>>>> up with a freebie one.
>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>> a slack type medium may improve members participation.
>>>>>
>>>>> so +1 for me as well.
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>    view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>>>
>>>>>> +1.
>>>>>>
>>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>>> MLflow, etc.
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>
>>>>>>> Hey everyone,
>>>>>>>
>>>>>>> I think we should remain on a free program in slack.
>>>>>>>
>>>>>>> In my option the free program is more then enough, the only down
>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>
>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>> limit notice in their workspace).
>>>>>>>
>>>>>>> You can find the pricing and features comparison between the slack
>>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>
>>>>>>> Have a great day,
>>>>>>> Shani
>>>>>>>
>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> 
>>>>>>>
>>>>>>>
>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>> discussion - but I can understand the concern.
>>>>>>> Would be great if you or Matei could update the thread on the result
>>>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>>>> official policy around it.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mridul
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>
>>>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>>>> But then there is the price and what is the best for an open source
>>>>>>>> project.
>>>>>>>>
>>>>>>>> The price for using slack is expensive.
>>>>>>>> Right now for those that have join spark slack
>>>>>>>> $8.75 USD
>>>>>>>> 72 members
>>>>>>>> 1 month
>>>>>>>> $630 USD
>>>>>>>>
>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>
>>>>>>>> And they - slack does not have an option for open source projects.
>>>>>>>>
>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>> have not tried it.
>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>
>>>>>>>> <image.png>
>>>>>>>>
>>>>>>>>
>>>>>>>> rocket chat is open source
>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>
>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>
>>>>>>>>> Hi Dongjoon
>>>>>>>>>
>>>>>>>>> to your points if I may
>>>>>>>>>
>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>> channels?
>>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>>> corrected
>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>    fair enough
>>>>>>>>>
>>>>>>>>> going back to your original point
>>>>>>>>>
>>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>> happens to be one of them.
>>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>> channels like linkedin etc.
>>>>>>>>>
>>>>>>>>> Hope this clarifies my position
>>>>>>>>>
>>>>>>>>> Mich Talebzadeh,
>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    view my Linkedin profile
>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> To Mich.
>>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>>> channels?
>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>
>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>> added Matei from your side.
>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>
>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>>> how to support it seamlessly.
>>>>>>>>>>
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>
>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>>> can do the same in our community?
>>>>>>>>>>>
>>>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Xiao
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>
>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>
>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>
>>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>>
>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>
>>>>>>>>>>>> Bests,
>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1
>>>>>>>>>>>>>
>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Bjørn Jørgensen
>>>>>>>> Vestre Aspehaug 4
>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>> 6010 Ålesund
>>>>>>>> Norge
>>>>>>>>
>>>>>>>> +47 480 94 297
>>>>>>>>
>>>>>>>

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

I agree, whatever individual sentiments are.

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim <ka...@gmail.com>
wrote:

> Just to be clear, if there is no strong volunteer to make the new
> community channel stay active, I'd probably be OK to not fork the channel.
> You can see a strong counter example from #spark channel in ASF. It is the
> place where there are only questions and promos but zero answers. I see
> volunteers here demanding for another channel, so I want to see us go with
> the most preferred way for these volunteers.
>
> User mailing list does not go in a good shape. I hope we give another try
> with recent technology to see whether we can gain traction - if we fail,
> the user mailing list will still be there.
>
> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> The number of subscribers doesn't give any meaningful value. Please look
>> into the number of mails being sent to the list.
>>
>> https://lists.apache.org/list.html?user@spark.apache.org
>> The latest month there were more than 200 emails being sent was Feb 2022,
>> more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
>> and earlier.
>> Let's face the fact. User mailing list is dying, even before we start
>> discussion about alternative communication methods.
>>
>> Users never go with the way if it's just because PMC members (or ASF)
>> have preference. They are going with the way they are convenient.
>>
>> Same applies here - if ASF Slack requires a restricted invitation
>> mechanism then it won't work. Looks like there is a link for an invitation,
>> but we are also talking about the cost as well.
>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>> As long as we are being serious about the cost, I don't think we are
>> going to land in the way "users" are convenient.
>>
>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>> channel.
>>>
>>> > It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list.
>>>
>>> The following is the number of people in the official channels.
>>>
>>> - user@spark.apache.org has 4519 subscribers.
>>> - dev@spark.apache.org has 3149 subscribers.
>>> - ASF Official Slack channel has 602 subscribers.
>>>
>>> May I ask if the users prefer to use the ASF Official Slack channel
>>> than the user mailing list?
>>>
>>> Dongjoon.
>>>
>>>
>>>
>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>>> section of "Open Communications", restriction of communication inside ASF
>>>> INFRA (mailing list) is more about code and decision-making.
>>>>
>>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>>
>>>> It's unavoidable if "users" prefer to use an alternative communication
>>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>>> there had been a meaningful number of questions around user@. It's
>>>> just impossible to let them go back and post to the user mailing list.
>>>>
>>>> We just need to make sure it is not the purpose of employing Slack to
>>>> move all discussions about developments, direction of the project, etc
>>>> which must happen in dev@/private@. The purpose of Slack thread here
>>>> does not seem to aim to serve the purpose.
>>>>
>>>>
>>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Good discussions and proposals.all around.
>>>>>
>>>>> I have used slack in anger on a customer site before. For small and
>>>>> medium size groups it is good and affordable. Alternatives have been
>>>>> suggested as well so those who like investigative search can agree and come
>>>>> up with a freebie one.
>>>>> I am inclined to agree with Bjorn that this slack has more social
>>>>> dimensions than the mailing list. It is akin to a sports club using
>>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>>> a slack type medium may improve members participation.
>>>>>
>>>>> so +1 for me as well.
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>    view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>>>
>>>>>> +1.
>>>>>>
>>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>>> MLflow, etc.
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>>
>>>>>>> Hey everyone,
>>>>>>>
>>>>>>> I think we should remain on a free program in slack.
>>>>>>>
>>>>>>> In my option the free program is more then enough, the only down
>>>>>>> side is we could only see the last 90 days messages.
>>>>>>>
>>>>>>> From what I know the Airflow community (which has strong active
>>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>>> limit notice in their workspace).
>>>>>>>
>>>>>>> You can find the pricing and features comparison between the slack
>>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>>
>>>>>>> Have a great day,
>>>>>>> Shani
>>>>>>>
>>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> 
>>>>>>>
>>>>>>>
>>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>>> discussion - but I can understand the concern.
>>>>>>> Would be great if you or Matei could update the thread on the result
>>>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>>>> official policy around it.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mridul
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>
>>>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>>>> But then there is the price and what is the best for an open source
>>>>>>>> project.
>>>>>>>>
>>>>>>>> The price for using slack is expensive.
>>>>>>>> Right now for those that have join spark slack
>>>>>>>> $8.75 USD
>>>>>>>> 72 members
>>>>>>>> 1 month
>>>>>>>> $630 USD
>>>>>>>>
>>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>>
>>>>>>>> And they - slack does not have an option for open source projects.
>>>>>>>>
>>>>>>>> There seems to be some alternatives for open source software. I
>>>>>>>> have not tried it.
>>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>>
>>>>>>>> <image.png>
>>>>>>>>
>>>>>>>>
>>>>>>>> rocket chat is open source
>>>>>>>> https://github.com/RocketChat/Rocket.Chat
>>>>>>>>
>>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>>
>>>>>>>>> Hi Dongjoon
>>>>>>>>>
>>>>>>>>> to your points if I may
>>>>>>>>>
>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>> channels?
>>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>>> corrected
>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>    fair enough
>>>>>>>>>
>>>>>>>>> going back to your original point
>>>>>>>>>
>>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>>> happens to be one of them.
>>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>>> channels like linkedin etc.
>>>>>>>>>
>>>>>>>>> Hope this clarifies my position
>>>>>>>>>
>>>>>>>>> Mich Talebzadeh,
>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    view my Linkedin profile
>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> To Mich.
>>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>>> channels?
>>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>>
>>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I
>>>>>>>>>> added Matei from your side.
>>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>>
>>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>>> how to support it seamlessly.
>>>>>>>>>>
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>>
>>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>>> can do the same in our community?
>>>>>>>>>>>
>>>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Xiao
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>>
>>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>>
>>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>>
>>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>>
>>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>>
>>>>>>>>>>>> Bests,
>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1
>>>>>>>>>>>>>
>>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Xiao
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack
>>>>>>>>>>>>>>>> channel so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Bjørn Jørgensen
>>>>>>>> Vestre Aspehaug 4
>>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>>> 6010 Ålesund
>>>>>>>> Norge
>>>>>>>>
>>>>>>>> +47 480 94 297
>>>>>>>>
>>>>>>>

Re: Slack for PySpark users

[Jungtaek Lim <ka...@gmail.com>]

Just to be clear, if there is no strong volunteer to make the new community
channel stay active, I'd probably be OK to not fork the channel. You can
see a strong counter example from #spark channel in ASF. It is the place
where there are only questions and promos but zero answers. I see
volunteers here demanding for another channel, so I want to see us go with
the most preferred way for these volunteers.

User mailing list does not go in a good shape. I hope we give another try
with recent technology to see whether we can gain traction - if we fail,
the user mailing list will still be there.

On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <ka...@gmail.com>
wrote:

> The number of subscribers doesn't give any meaningful value. Please look
> into the number of mails being sent to the list.
>
> https://lists.apache.org/list.html?user@spark.apache.org
> The latest month there were more than 200 emails being sent was Feb 2022,
> more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
> and earlier.
> Let's face the fact. User mailing list is dying, even before we start
> discussion about alternative communication methods.
>
> Users never go with the way if it's just because PMC members (or ASF) have
> preference. They are going with the way they are convenient.
>
> Same applies here - if ASF Slack requires a restricted invitation
> mechanism then it won't work. Looks like there is a link for an invitation,
> but we are also talking about the cost as well.
> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
> As long as we are being serious about the cost, I don't think we are going
> to land in the way "users" are convenient.
>
> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - user@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>> there had been a meaningful number of questions around user@. It's just
>>> impossible to let them go back and post to the user mailing list.
>>>
>>> We just need to make sure it is not the purpose of employing Slack to
>>> move all discussions about developments, direction of the project, etc
>>> which must happen in dev@/private@. The purpose of Slack thread here
>>> does not seem to aim to serve the purpose.
>>>
>>>
>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Good discussions and proposals.all around.
>>>>
>>>> I have used slack in anger on a customer site before. For small and
>>>> medium size groups it is good and affordable. Alternatives have been
>>>> suggested as well so those who like investigative search can agree and come
>>>> up with a freebie one.
>>>> I am inclined to agree with Bjorn that this slack has more social
>>>> dimensions than the mailing list. It is akin to a sports club using
>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>> a slack type medium may improve members participation.
>>>>
>>>> so +1 for me as well.
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>    view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>>
>>>>> +1.
>>>>>
>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>> MLflow, etc.
>>>>>
>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>
>>>>>> Hey everyone,
>>>>>>
>>>>>> I think we should remain on a free program in slack.
>>>>>>
>>>>>> In my option the free program is more then enough, the only down side
>>>>>> is we could only see the last 90 days messages.
>>>>>>
>>>>>> From what I know the Airflow community (which has strong active
>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>> limit notice in their workspace).
>>>>>>
>>>>>> You can find the pricing and features comparison between the slack
>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>
>>>>>> Have a great day,
>>>>>> Shani
>>>>>>
>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> 
>>>>>>
>>>>>>
>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>> discussion - but I can understand the concern.
>>>>>> Would be great if you or Matei could update the thread on the result
>>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>>> official policy around it.
>>>>>>
>>>>>> Regards,
>>>>>> Mridul
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>
>>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>>> But then there is the price and what is the best for an open source
>>>>>>> project.
>>>>>>>
>>>>>>> The price for using slack is expensive.
>>>>>>> Right now for those that have join spark slack
>>>>>>> $8.75 USD
>>>>>>> 72 members
>>>>>>> 1 month
>>>>>>> $630 USD
>>>>>>>
>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>
>>>>>>> And they - slack does not have an option for open source projects.
>>>>>>>
>>>>>>> There seems to be some alternatives for open source software. I have
>>>>>>> not tried it.
>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>
>>>>>>> <image.png>
>>>>>>>
>>>>>>>
>>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>>
>>>>>>>
>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>
>>>>>>>> Hi Dongjoon
>>>>>>>>
>>>>>>>> to your points if I may
>>>>>>>>
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>> corrected
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>    fair enough
>>>>>>>>
>>>>>>>> going back to your original point
>>>>>>>>
>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>> happens to be one of them.
>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>> channels like linkedin etc.
>>>>>>>>
>>>>>>>> Hope this clarifies my position
>>>>>>>>
>>>>>>>> Mich Talebzadeh,
>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>> Palantir Technologies Limited
>>>>>>>>
>>>>>>>>
>>>>>>>>    view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> To Mich.
>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>> channels?
>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>
>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>>> Matei from your side.
>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>
>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>> how to support it seamlessly.
>>>>>>>>>
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>
>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>> can do the same in our community?
>>>>>>>>>>
>>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>>
>>>>>>>>>> Xiao
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>
>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>
>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>
>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>
>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>
>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>
>>>>>>>>>>> Bests,
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1
>>>>>>>>>>>>
>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Xiao
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>
>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Bjørn Jørgensen
>>>>>>> Vestre Aspehaug 4
>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>> 6010 Ålesund
>>>>>>> Norge
>>>>>>>
>>>>>>> +47 480 94 297
>>>>>>>
>>>>>>

Re: Slack for PySpark users

[Jungtaek Lim <ka...@gmail.com>]

Just to be clear, if there is no strong volunteer to make the new community
channel stay active, I'd probably be OK to not fork the channel. You can
see a strong counter example from #spark channel in ASF. It is the place
where there are only questions and promos but zero answers. I see
volunteers here demanding for another channel, so I want to see us go with
the most preferred way for these volunteers.

User mailing list does not go in a good shape. I hope we give another try
with recent technology to see whether we can gain traction - if we fail,
the user mailing list will still be there.

On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <ka...@gmail.com>
wrote:

> The number of subscribers doesn't give any meaningful value. Please look
> into the number of mails being sent to the list.
>
> https://lists.apache.org/list.html?user@spark.apache.org
> The latest month there were more than 200 emails being sent was Feb 2022,
> more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
> and earlier.
> Let's face the fact. User mailing list is dying, even before we start
> discussion about alternative communication methods.
>
> Users never go with the way if it's just because PMC members (or ASF) have
> preference. They are going with the way they are convenient.
>
> Same applies here - if ASF Slack requires a restricted invitation
> mechanism then it won't work. Looks like there is a link for an invitation,
> but we are also talking about the cost as well.
> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
> As long as we are being serious about the cost, I don't think we are going
> to land in the way "users" are convenient.
>
> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - user@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>> there had been a meaningful number of questions around user@. It's just
>>> impossible to let them go back and post to the user mailing list.
>>>
>>> We just need to make sure it is not the purpose of employing Slack to
>>> move all discussions about developments, direction of the project, etc
>>> which must happen in dev@/private@. The purpose of Slack thread here
>>> does not seem to aim to serve the purpose.
>>>
>>>
>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Good discussions and proposals.all around.
>>>>
>>>> I have used slack in anger on a customer site before. For small and
>>>> medium size groups it is good and affordable. Alternatives have been
>>>> suggested as well so those who like investigative search can agree and come
>>>> up with a freebie one.
>>>> I am inclined to agree with Bjorn that this slack has more social
>>>> dimensions than the mailing list. It is akin to a sports club using
>>>> WhatsApp groups for communication. Remember we were originally looking for
>>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>>> I think Slack and mailing groups can coexist happily. On a more serious
>>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>>> a slack type medium may improve members participation.
>>>>
>>>> so +1 for me as well.
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>    view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>>
>>>>> +1.
>>>>>
>>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>>> MLflow, etc.
>>>>>
>>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>>
>>>>>> Hey everyone,
>>>>>>
>>>>>> I think we should remain on a free program in slack.
>>>>>>
>>>>>> In my option the free program is more then enough, the only down side
>>>>>> is we could only see the last 90 days messages.
>>>>>>
>>>>>> From what I know the Airflow community (which has strong active
>>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>>> limit notice in their workspace).
>>>>>>
>>>>>> You can find the pricing and features comparison between the slack
>>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>>
>>>>>> Have a great day,
>>>>>> Shani
>>>>>>
>>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> 
>>>>>>
>>>>>>
>>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>>> discussion - but I can understand the concern.
>>>>>> Would be great if you or Matei could update the thread on the result
>>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>>> official policy around it.
>>>>>>
>>>>>> Regards,
>>>>>> Mridul
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>
>>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>>> But then there is the price and what is the best for an open source
>>>>>>> project.
>>>>>>>
>>>>>>> The price for using slack is expensive.
>>>>>>> Right now for those that have join spark slack
>>>>>>> $8.75 USD
>>>>>>> 72 members
>>>>>>> 1 month
>>>>>>> $630 USD
>>>>>>>
>>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>>
>>>>>>> And they - slack does not have an option for open source projects.
>>>>>>>
>>>>>>> There seems to be some alternatives for open source software. I have
>>>>>>> not tried it.
>>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>>
>>>>>>> <image.png>
>>>>>>>
>>>>>>>
>>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>>
>>>>>>>
>>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>>
>>>>>>>> Hi Dongjoon
>>>>>>>>
>>>>>>>> to your points if I may
>>>>>>>>
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>>> corrected
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>    fair enough
>>>>>>>>
>>>>>>>> going back to your original point
>>>>>>>>
>>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>>> happens to be one of them.
>>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>>> channels like linkedin etc.
>>>>>>>>
>>>>>>>> Hope this clarifies my position
>>>>>>>>
>>>>>>>> Mich Talebzadeh,
>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>> Palantir Technologies Limited
>>>>>>>>
>>>>>>>>
>>>>>>>>    view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> To Mich.
>>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>>> channels?
>>>>>>>>> - To be clear, I intentionally didn't refer to any specific
>>>>>>>>> mailing list because we didn't set up any rule here yet.
>>>>>>>>>
>>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>>> Matei from your side.
>>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>>
>>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>>> Please hold on any official action for this topic. We will know
>>>>>>>>> how to support it seamlessly.
>>>>>>>>>
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>>
>>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>>> can do the same in our community?
>>>>>>>>>>
>>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>>
>>>>>>>>>> Xiao
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>>
>>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>>
>>>>>>>>>>> (cc Matei)
>>>>>>>>>>>
>>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>>
>>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>>
>>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>>> before starting anything official.
>>>>>>>>>>>
>>>>>>>>>>> Bests,
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1
>>>>>>>>>>>>
>>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Xiao
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>>
>>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <
>>>>>>>>>>>>> zgolliasma@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Bjørn Jørgensen
>>>>>>> Vestre Aspehaug 4
>>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>>> 6010 Ålesund
>>>>>>> Norge
>>>>>>>
>>>>>>> +47 480 94 297
>>>>>>>
>>>>>>

Re: Slack for PySpark users

[Jungtaek Lim <ka...@gmail.com>]

The number of subscribers doesn't give any meaningful value. Please look
into the number of mails being sent to the list.

https://lists.apache.org/list.html?user@spark.apache.org
The latest month there were more than 200 emails being sent was Feb 2022,
more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
and earlier.
Let's face the fact. User mailing list is dying, even before we start
discussion about alternative communication methods.

Users never go with the way if it's just because PMC members (or ASF) have
preference. They are going with the way they are convenient.

Same applies here - if ASF Slack requires a restricted invitation mechanism
then it won't work. Looks like there is a link for an invitation, but we
are also talking about the cost as well.
https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
As long as we are being serious about the cost, I don't think we are going
to land in the way "users" are convenient.

On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.
>
> > It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - user@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list. Before Stack Overflow days,
>> there had been a meaningful number of questions around user@. It's just
>> impossible to let them go back and post to the user mailing list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of Slack thread here
>> does not seem to aim to serve the purpose.
>>
>>
>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Good discussions and proposals.all around.
>>>
>>> I have used slack in anger on a customer site before. For small and
>>> medium size groups it is good and affordable. Alternatives have been
>>> suggested as well so those who like investigative search can agree and come
>>> up with a freebie one.
>>> I am inclined to agree with Bjorn that this slack has more social
>>> dimensions than the mailing list. It is akin to a sports club using
>>> WhatsApp groups for communication. Remember we were originally looking for
>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>> I think Slack and mailing groups can coexist happily. On a more serious
>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>> a slack type medium may improve members participation.
>>>
>>> so +1 for me as well.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>    view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>
>>>> +1.
>>>>
>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>> MLflow, etc.
>>>>
>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>
>>>>> Hey everyone,
>>>>>
>>>>> I think we should remain on a free program in slack.
>>>>>
>>>>> In my option the free program is more then enough, the only down side
>>>>> is we could only see the last 90 days messages.
>>>>>
>>>>> From what I know the Airflow community (which has strong active
>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>> limit notice in their workspace).
>>>>>
>>>>> You can find the pricing and features comparison between the slack
>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>
>>>>> Have a great day,
>>>>> Shani
>>>>>
>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> 
>>>>>
>>>>>
>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>> discussion - but I can understand the concern.
>>>>> Would be great if you or Matei could update the thread on the result
>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>> official policy around it.
>>>>>
>>>>> Regards,
>>>>> Mridul
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>
>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>> But then there is the price and what is the best for an open source
>>>>>> project.
>>>>>>
>>>>>> The price for using slack is expensive.
>>>>>> Right now for those that have join spark slack
>>>>>> $8.75 USD
>>>>>> 72 members
>>>>>> 1 month
>>>>>> $630 USD
>>>>>>
>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>
>>>>>> And they - slack does not have an option for open source projects.
>>>>>>
>>>>>> There seems to be some alternatives for open source software. I have
>>>>>> not tried it.
>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>
>>>>>> <image.png>
>>>>>>
>>>>>>
>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>
>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>
>>>>>>> Hi Dongjoon
>>>>>>>
>>>>>>> to your points if I may
>>>>>>>
>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>> channels?
>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>> corrected
>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>    fair enough
>>>>>>>
>>>>>>> going back to your original point
>>>>>>>
>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>> happens to be one of them.
>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>> channels like linkedin etc.
>>>>>>>
>>>>>>> Hope this clarifies my position
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>    view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <do...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> To Mich.
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>
>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>> Matei from your side.
>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>
>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>> Please hold on any official action for this topic. We will know how
>>>>>>>> to support it seamlessly.
>>>>>>>>
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>
>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>> can do the same in our community?
>>>>>>>>>
>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>
>>>>>>>>> Xiao
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>
>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>
>>>>>>>>>> (cc Matei)
>>>>>>>>>>
>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>
>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>
>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>> before starting anything official.
>>>>>>>>>>
>>>>>>>>>> Bests,
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Xiao
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>
>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <zg...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Bjørn Jørgensen
>>>>>> Vestre Aspehaug 4
>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>> 6010 Ålesund
>>>>>> Norge
>>>>>>
>>>>>> +47 480 94 297
>>>>>>
>>>>>

Re: Slack for PySpark users

[Jungtaek Lim <ka...@gmail.com>]

The number of subscribers doesn't give any meaningful value. Please look
into the number of mails being sent to the list.

https://lists.apache.org/list.html?user@spark.apache.org
The latest month there were more than 200 emails being sent was Feb 2022,
more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
and earlier.
Let's face the fact. User mailing list is dying, even before we start
discussion about alternative communication methods.

Users never go with the way if it's just because PMC members (or ASF) have
preference. They are going with the way they are convenient.

Same applies here - if ASF Slack requires a restricted invitation mechanism
then it won't work. Looks like there is a link for an invitation, but we
are also talking about the cost as well.
https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
As long as we are being serious about the cost, I don't think we are going
to land in the way "users" are convenient.

On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun <do...@gmail.com>
wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.
>
> > It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - user@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list. Before Stack Overflow days,
>> there had been a meaningful number of questions around user@. It's just
>> impossible to let them go back and post to the user mailing list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of Slack thread here
>> does not seem to aim to serve the purpose.
>>
>>
>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Good discussions and proposals.all around.
>>>
>>> I have used slack in anger on a customer site before. For small and
>>> medium size groups it is good and affordable. Alternatives have been
>>> suggested as well so those who like investigative search can agree and come
>>> up with a freebie one.
>>> I am inclined to agree with Bjorn that this slack has more social
>>> dimensions than the mailing list. It is akin to a sports club using
>>> WhatsApp groups for communication. Remember we were originally looking for
>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>> I think Slack and mailing groups can coexist happily. On a more serious
>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>> a slack type medium may improve members participation.
>>>
>>> so +1 for me as well.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>    view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>
>>>> +1.
>>>>
>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>> MLflow, etc.
>>>>
>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>
>>>>> Hey everyone,
>>>>>
>>>>> I think we should remain on a free program in slack.
>>>>>
>>>>> In my option the free program is more then enough, the only down side
>>>>> is we could only see the last 90 days messages.
>>>>>
>>>>> From what I know the Airflow community (which has strong active
>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>> limit notice in their workspace).
>>>>>
>>>>> You can find the pricing and features comparison between the slack
>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>
>>>>> Have a great day,
>>>>> Shani
>>>>>
>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> 
>>>>>
>>>>>
>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>> discussion - but I can understand the concern.
>>>>> Would be great if you or Matei could update the thread on the result
>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>> official policy around it.
>>>>>
>>>>> Regards,
>>>>> Mridul
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>
>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>> But then there is the price and what is the best for an open source
>>>>>> project.
>>>>>>
>>>>>> The price for using slack is expensive.
>>>>>> Right now for those that have join spark slack
>>>>>> $8.75 USD
>>>>>> 72 members
>>>>>> 1 month
>>>>>> $630 USD
>>>>>>
>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>
>>>>>> And they - slack does not have an option for open source projects.
>>>>>>
>>>>>> There seems to be some alternatives for open source software. I have
>>>>>> not tried it.
>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>
>>>>>> <image.png>
>>>>>>
>>>>>>
>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>
>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>
>>>>>>> Hi Dongjoon
>>>>>>>
>>>>>>> to your points if I may
>>>>>>>
>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>> channels?
>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>> corrected
>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>    fair enough
>>>>>>>
>>>>>>> going back to your original point
>>>>>>>
>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>> happens to be one of them.
>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>> channels like linkedin etc.
>>>>>>>
>>>>>>> Hope this clarifies my position
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>    view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <do...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> To Mich.
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>
>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>> Matei from your side.
>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>
>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>> Please hold on any official action for this topic. We will know how
>>>>>>>> to support it seamlessly.
>>>>>>>>
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>
>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>> can do the same in our community?
>>>>>>>>>
>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>
>>>>>>>>> Xiao
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>
>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>
>>>>>>>>>> (cc Matei)
>>>>>>>>>>
>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>
>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>
>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>> before starting anything official.
>>>>>>>>>>
>>>>>>>>>> Bests,
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Xiao
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>
>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <zg...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Bjørn Jørgensen
>>>>>> Vestre Aspehaug 4
>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>> 6010 Ålesund
>>>>>> Norge
>>>>>>
>>>>>> +47 480 94 297
>>>>>>
>>>>>

Re: Slack for PySpark users

[Mich Talebzadeh <mi...@gmail.com>]

I for myself prefer to use the newly formed slack.

sparkcommunitytalk.slack.com

In summary, it may be a good idea to take a tour of it and see for
yourself. Topics are sectioned as per user requests.

I trust this answers your question.

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 3 Apr 2023 at 20:59, Dongjoon Hyun <do...@gmail.com> wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.
>
> > It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - user@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list. Before Stack Overflow days,
>> there had been a meaningful number of questions around user@. It's just
>> impossible to let them go back and post to the user mailing list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of Slack thread here
>> does not seem to aim to serve the purpose.
>>
>>
>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Good discussions and proposals.all around.
>>>
>>> I have used slack in anger on a customer site before. For small and
>>> medium size groups it is good and affordable. Alternatives have been
>>> suggested as well so those who like investigative search can agree and come
>>> up with a freebie one.
>>> I am inclined to agree with Bjorn that this slack has more social
>>> dimensions than the mailing list. It is akin to a sports club using
>>> WhatsApp groups for communication. Remember we were originally looking for
>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>> I think Slack and mailing groups can coexist happily. On a more serious
>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>> a slack type medium may improve members participation.
>>>
>>> so +1 for me as well.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>    view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee <de...@gmail.com> wrote:
>>>
>>>> +1.
>>>>
>>>> To Shani’s point, there are multiple OSS projects that use the free
>>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>>> MLflow, etc.
>>>>
>>>> On Thu, Mar 30, 2023 at 14:15 <sh...@gmail.com> wrote:
>>>>
>>>>> Hey everyone,
>>>>>
>>>>> I think we should remain on a free program in slack.
>>>>>
>>>>> In my option the free program is more then enough, the only down side
>>>>> is we could only see the last 90 days messages.
>>>>>
>>>>> From what I know the Airflow community (which has strong active
>>>>> community in slack) also use the free program (You can tell by the 90 days
>>>>> limit notice in their workspace).
>>>>>
>>>>> You can find the pricing and features comparison between the slack
>>>>> programs here <https://slack.com/intl/en-gb/pricing> .
>>>>>
>>>>> Have a great day,
>>>>> Shani
>>>>>
>>>>> On 30 Mar 2023, at 23:38, Mridul Muralidharan <mr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> 
>>>>>
>>>>>
>>>>> Thanks for flagging the concern Dongjoon, I was not aware of the
>>>>> discussion - but I can understand the concern.
>>>>> Would be great if you or Matei could update the thread on the result
>>>>> of deliberations, once it reaches a logical consensus: before we set up
>>>>> official policy around it.
>>>>>
>>>>> Regards,
>>>>> Mridul
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>
>>>>>> I like the idea of having a talk channel. It can make it easier for
>>>>>> everyone to say hello. Or to dare to ask about small or big matters that
>>>>>> you would not have dared to ask about before on mailing lists.
>>>>>> But then there is the price and what is the best for an open source
>>>>>> project.
>>>>>>
>>>>>> The price for using slack is expensive.
>>>>>> Right now for those that have join spark slack
>>>>>> $8.75 USD
>>>>>> 72 members
>>>>>> 1 month
>>>>>> $630 USD
>>>>>>
>>>>>> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta&s=2
>>>>>>
>>>>>> And they - slack does not have an option for open source projects.
>>>>>>
>>>>>> There seems to be some alternatives for open source software. I have
>>>>>> not tried it.
>>>>>> Like https://www.rocket.chat/blog/slack-open-source-alternatives
>>>>>>
>>>>>> <image.png>
>>>>>>
>>>>>>
>>>>>> rocket chat is open source https://github.com/RocketChat/Rocket.Chat
>>>>>>
>>>>>> tor. 30. mar. 2023 kl. 18:54 skrev Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com>:
>>>>>>
>>>>>>> Hi Dongjoon
>>>>>>>
>>>>>>> to your points if I may
>>>>>>>
>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>> channels?
>>>>>>>    No, I don't have any reference from other official ASF-related
>>>>>>> Slack channels because I don't think that matters. However, I stand
>>>>>>> corrected
>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>    fair enough
>>>>>>>
>>>>>>> going back to your original point
>>>>>>>
>>>>>>> ..There is a concern expressed by ASF board because recent Slack
>>>>>>> activities created an isolated silo outside of ASF mailing list archive...
>>>>>>> Well, there are activities on Spark and indeed other open source
>>>>>>> software everywhere. One way or other they do help getting community
>>>>>>> (inside the user groups and other) to get interested and involved. Slack
>>>>>>> happens to be one of them.
>>>>>>> I am of the opinion that creating such silos is already a reality
>>>>>>> and we ought to be pragmatic. Unless there is an overriding reason, we
>>>>>>> should embrace it as slack can co-exist with the other mailing lists and
>>>>>>> channels like linkedin etc.
>>>>>>>
>>>>>>> Hope this clarifies my position
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>    view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 30 Mar 2023 at 17:28, Dongjoon Hyun <do...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> To Mich.
>>>>>>>> - Do you have any reference from other official ASF-related Slack
>>>>>>>> channels?
>>>>>>>> - To be clear, I intentionally didn't refer to any specific mailing
>>>>>>>> list because we didn't set up any rule here yet.
>>>>>>>>
>>>>>>>> To Xiao. I understand what you mean. That's the reason why I added
>>>>>>>> Matei from your side.
>>>>>>>> > I did not see an objection from the ASF board.
>>>>>>>>
>>>>>>>> There is on-going discussion about the communication channels
>>>>>>>> outside ASF email which is specifically concerning Slack.
>>>>>>>> Please hold on any official action for this topic. We will know how
>>>>>>>> to support it seamlessly.
>>>>>>>>
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 30, 2023 at 9:21 AM Xiao Li <ga...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi, Dongjoon,
>>>>>>>>>
>>>>>>>>> The other communities (e.g., Pinot, Druid, Flink) created their
>>>>>>>>> own Slack workspaces last year. I did not see an objection from the ASF
>>>>>>>>> board. At the same time, Slack workspaces are very popular and useful in
>>>>>>>>> most non-ASF open source communities. TBH, we are kind of late. I think we
>>>>>>>>> can do the same in our community?
>>>>>>>>>
>>>>>>>>> We can follow the guide when the ASF has an official process for
>>>>>>>>> ASF archiving. Since our PMC are the owner of the slack workspace, we can
>>>>>>>>> make a change based on the policy. WDYT?
>>>>>>>>>
>>>>>>>>> Xiao
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dongjoon Hyun <do...@gmail.com> 于2023年3月30日周四 09:03写道：
>>>>>>>>>
>>>>>>>>>> Hi, Xiao and all.
>>>>>>>>>>
>>>>>>>>>> (cc Matei)
>>>>>>>>>>
>>>>>>>>>> Please hold on the vote.
>>>>>>>>>>
>>>>>>>>>> There is a concern expressed by ASF board because recent Slack
>>>>>>>>>> activities created an isolated silo outside of ASF mailing list archive.
>>>>>>>>>>
>>>>>>>>>> We need to establish a way to embrace it back to ASF archive
>>>>>>>>>> before starting anything official.
>>>>>>>>>>
>>>>>>>>>> Bests,
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 29, 2023 at 11:32 PM Xiao Li <ga...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> + @dev@spark.apache.org <de...@spark.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> This is a good idea. The other Apache projects (e.g., Pinot,
>>>>>>>>>>> Druid, Flink) have created their own dedicated Slack workspaces for faster
>>>>>>>>>>> communication. We can do the same in Apache Spark. The Slack workspace will
>>>>>>>>>>> be maintained by the Apache Spark PMC. I propose to initiate a vote for the
>>>>>>>>>>> creation of a new Apache Spark Slack workspace. Does that sound good?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Xiao
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Mich Talebzadeh <mi...@gmail.com> 于2023年3月28日周二
>>>>>>>>>>> 07:07写道：
>>>>>>>>>>>
>>>>>>>>>>>> I created one at slack called pyspark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mich Talebzadeh,
>>>>>>>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>>>>>>>> Palantir Technologies Limited
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    view my Linkedin profile
>>>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all
>>>>>>>>>>>> responsibility for any loss, damage or destruction of data or any other
>>>>>>>>>>>> property which may arise from relying on this email's technical content is
>>>>>>>>>>>> explicitly disclaimed. The author will in no case be liable for any
>>>>>>>>>>>> monetary damages arising from such loss, damage or destruction.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:52, asma zgolli <zg...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 good idea, I d like to join as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le mar. 28 mars 2023 à 04:09, Winston Lai <
>>>>>>>>>>>>> weiruanlysy@gmail.com> a écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let us know when the channel is created. I'd like to
>>>>>>>>>>>>>> join :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank You & Best Regards
>>>>>>>>>>>>>> Winston Lai
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>> *From:* Denny Lee <de...@gmail.com>
>>>>>>>>>>>>>> *Sent:* Tuesday, March 28, 2023 9:43:08 AM
>>>>>>>>>>>>>> *To:* Hyukjin Kwon <gu...@gmail.com>
>>>>>>>>>>>>>> *Cc:* keen <ke...@gmx.net>; user@spark.apache.org <
>>>>>>>>>>>>>> user@spark.apache.org>
>>>>>>>>>>>>>> *Subject:* Re: Slack for PySpark users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 I think this is a great idea!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 27, 2023 at 6:24 PM Hyukjin Kwon <
>>>>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, actually I think we should better have a slack channel
>>>>>>>>>>>>>> so we can easily discuss with users and developers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, 28 Mar 2023 at 03:08, keen <ke...@gmx.net> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>> I really like *Slack *as communication channel for a tech
>>>>>>>>>>>>>> community.
>>>>>>>>>>>>>> There is a Slack workspace for *delta lake users* (
>>>>>>>>>>>>>> https://go.delta.io/slack) that I enjoy a lot.
>>>>>>>>>>>>>> I was wondering if there is something similar for PySpark
>>>>>>>>>>>>>> users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If not, would there be anything wrong with creating a new
>>>>>>>>>>>>>> Slack workspace for PySpark users? (when explicitly mentioning that this is
>>>>>>>>>>>>>> *not* officially part of Apache Spark)?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>> Martin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Asma ZGOLLI
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Bjørn Jørgensen
>>>>>> Vestre Aspehaug 4
>>>>>> <https://www.google.com/maps/search/Vestre+Aspehaug+4?entry=gmail&source=g>,
>>>>>> 6010 Ålesund
>>>>>> Norge
>>>>>>
>>>>>> +47 480 94 297
>>>>>>
>>>>>