You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Mich Talebzadeh <mi...@gmail.com> on 2023/03/13 16:16:46 UTC
Topics for Spark online classes & webinars
Hi guys
To move forward I selected these topics from the thread "Online classes for
spark topics".
To take this further I propose a confluence page to be seup.
Opinions and how to is welcome
Cheers
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
Re: Topics for Spark online classes & webinars
Posted by Bjørn Jørgensen <bj...@gmail.com>.
Great.
A case that I hope can be better documented, especially now that we have
Pandas API on Spark and many potential new users coming from Pandas.
Is how to start Spark with full available memory and CPU.
I use this function to do this in a notebook.
import multiprocessing
import os
import sys
from pyspark import SparkConf, SparkContext
from pyspark import pandas as ps
from pyspark.sql import *
os.environ["PYARROW_IGNORE_TIMEZONE"] = "1"
number_cores = int(multiprocessing.cpu_count())
mem_bytes = os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES") #
e.g. 4015976448
memory_gb = int(mem_bytes / (1024.0**3)) # e.g. 3.74
def get_spark_session(app_name: str, conf: SparkConf):
conf.setMaster("local[{}]".format(number_cores))
conf.set("spark.driver.memory", "{}g".format(memory_gb)).set(
"spark.sql.adaptive.enabled", "True"
).set(
"spark.serializer", "org.apache.spark.serializer.KryoSerializer"
).set(
"spark.sql.repl.eagerEval.maxNumRows", "10000"
).set(
"sc.setLogLevel", "ERROR"
)
return
SparkSession.builder.appName(app_name).config(conf=conf).getOrCreate()
spark = get_spark_session("My app", SparkConf())
ons. 15. mar. 2023 kl. 19:27 skrev Denny Lee <de...@gmail.com>:
> Thanks Mich for tackling this! I encourage everyone to add to the list so
> we can have a comprehensive list of topics, eh?!
>
> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>
>> https://www.linkedin.com/company/apachespark/
>>
>> and contribution from @asma zgolli <zg...@gmail.com>
>>
>> You will see my post at the bottom. Please add anything else on topics to
>> the list as a comment.
>>
>> We will then put them together in an article perhaps. Comments and
>> contributions are welcome.
>>
>> HTH
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead,
>> Palantir Technologies Limited
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>>> Hi Denny,
>>>
>>> That Apache Spark Linkedin page
>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>> allows a wider audience to benefit from it.
>>>
>>> +1 for me
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>
>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>>> these type of events - if you're cool with this? Or we could go through
>>>> the process of submitting and updating the current
>>>> https://spark.apache.org or request to leverage the original Spark
>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>> WDYT?
>>>>
>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Well that needs to be created first for this purpose. The appropriate
>>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com>
>>>>> can facilitate this as he offered his help.
>>>>>
>>>>>
>>>>> cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Mich,
>>>>>>
>>>>>> Can you please provide the link for the confluence page?
>>>>>>
>>>>>> Many thanks
>>>>>> Asma
>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>
>>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>>
>>>>>>> Apologies I missed the list.
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>>
>>>>>>> 1. Spark UI
>>>>>>> 2. Dynamic allocation
>>>>>>> 3. Tuning of jobs
>>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>>> 8. Spark on notebooks
>>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>>> 10. Spark on k8s
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi guys
>>>>>>>>
>>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>>> classes for spark topics".
>>>>>>>>
>>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>>
>>>>>>>> Opinions and how to is welcome
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
--
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge
+47 480 94 297
Re: Topics for Spark online classes & webinars
Posted by Bjørn Jørgensen <bj...@gmail.com>.
Great.
A case that I hope can be better documented, especially now that we have
Pandas API on Spark and many potential new users coming from Pandas.
Is how to start Spark with full available memory and CPU.
I use this function to do this in a notebook.
import multiprocessing
import os
import sys
from pyspark import SparkConf, SparkContext
from pyspark import pandas as ps
from pyspark.sql import *
os.environ["PYARROW_IGNORE_TIMEZONE"] = "1"
number_cores = int(multiprocessing.cpu_count())
mem_bytes = os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES") #
e.g. 4015976448
memory_gb = int(mem_bytes / (1024.0**3)) # e.g. 3.74
def get_spark_session(app_name: str, conf: SparkConf):
conf.setMaster("local[{}]".format(number_cores))
conf.set("spark.driver.memory", "{}g".format(memory_gb)).set(
"spark.sql.adaptive.enabled", "True"
).set(
"spark.serializer", "org.apache.spark.serializer.KryoSerializer"
).set(
"spark.sql.repl.eagerEval.maxNumRows", "10000"
).set(
"sc.setLogLevel", "ERROR"
)
return
SparkSession.builder.appName(app_name).config(conf=conf).getOrCreate()
spark = get_spark_session("My app", SparkConf())
ons. 15. mar. 2023 kl. 19:27 skrev Denny Lee <de...@gmail.com>:
> Thanks Mich for tackling this! I encourage everyone to add to the list so
> we can have a comprehensive list of topics, eh?!
>
> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>
>> https://www.linkedin.com/company/apachespark/
>>
>> and contribution from @asma zgolli <zg...@gmail.com>
>>
>> You will see my post at the bottom. Please add anything else on topics to
>> the list as a comment.
>>
>> We will then put them together in an article perhaps. Comments and
>> contributions are welcome.
>>
>> HTH
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead,
>> Palantir Technologies Limited
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>>> Hi Denny,
>>>
>>> That Apache Spark Linkedin page
>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>> allows a wider audience to benefit from it.
>>>
>>> +1 for me
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>
>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>>> these type of events - if you're cool with this? Or we could go through
>>>> the process of submitting and updating the current
>>>> https://spark.apache.org or request to leverage the original Spark
>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>> WDYT?
>>>>
>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Well that needs to be created first for this purpose. The appropriate
>>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com>
>>>>> can facilitate this as he offered his help.
>>>>>
>>>>>
>>>>> cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Mich,
>>>>>>
>>>>>> Can you please provide the link for the confluence page?
>>>>>>
>>>>>> Many thanks
>>>>>> Asma
>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>
>>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>>
>>>>>>> Apologies I missed the list.
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>>
>>>>>>> 1. Spark UI
>>>>>>> 2. Dynamic allocation
>>>>>>> 3. Tuning of jobs
>>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>>> 8. Spark on notebooks
>>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>>> 10. Spark on k8s
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi guys
>>>>>>>>
>>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>>> classes for spark topics".
>>>>>>>>
>>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>>
>>>>>>>> Opinions and how to is welcome
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> view my Linkedin profile
>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>>> arising from such loss, damage or destruction.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
--
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge
+47 480 94 297
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
https://join.slack.com/t/sparkcommunitytalk/shared_invite/zt-1rk11diac-hzGbOEdBHgjXf02IZ1mvUA
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 28 Mar 2023 at 19:38, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi Bjorn,
>
> you just need to create an account on slack and join any topic I believe
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 28 Mar 2023 at 18:57, Bjørn Jørgensen <bj...@gmail.com>
> wrote:
>
>> Do I need to get an invite before joining?
>>
>>
>> tir. 28. mar. 2023 kl. 18:51 skrev Mich Talebzadeh <
>> mich.talebzadeh@gmail.com>:
>>
>>> Hi all,
>>>
>>> There is a section in slack called webinars
>>>
>>>
>>> https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
>>>
>>> Asma Zgolli, agreed to prepare materials for Spark internals and/or
>>> comparing spark 3 and 2.
>>>
>>> I like to contribute to "Spark Streaming & Spark Structured Streaming"
>>> plus "Spark on k8s for both GCP and EKS concepts and contrasts"
>>>
>>> Other topics are mentioned below.
>>>
>>> -- Spark UI
>>> -- Dynamic allocation
>>> -- Tuning of jobs
>>> -- Collecting spark metrics for monitoring and alerting
>>> -- For those who prefer to use Pandas API on Spark since the release of
>>> Spark 3.2, What are some important notes for those users? For example, what
>>> are the additional factors affecting the Spark -- Performance using Pandas
>>> API on Spark? How to tune them in addition to the conventional Spark tuning
>>> methods applied to Spark SQL users.
>>> -- Spark internals and/or comparing spark 3 and 2
>>> -- Spark Streaming & Spark Structured Streaming
>>> -- Spark on notebooks
>>> -- Spark on serverless (for example Spark on Google Cloud)
>>> -- Spark on k8s
>>>
>>>
>>> If you are willing to contribute to presentation materials, please
>>> register your interest in slack/webinars.
>>>
>>>
>>> Thanks
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>>
>>> Spark on k8s for both GCP and EKS concepts and contrasts
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I suggest using the slack for the spark community created recently to
>>>> collaborate and work together on these topics and use the LinkedIn page to
>>>> publish the events and the webinars.
>>>>
>>>> Cheers,
>>>> Asma
>>>>
>>>> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a
>>>> écrit :
>>>>
>>>>> What we can do is get into the habit of compiling the list on LinkedIn
>>>>> but making sure this list is shared and broadcast here, eh?!
>>>>>
>>>>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>>>>> riverside.fm as well as simulcasting this on LinkedIn. This way you
>>>>> can view directly on the former without ever logging in with a user ID.
>>>>>
>>>>> HTH!!
>>>>>
>>>>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> Understood Nitin It would be wrong to act against one's conviction. I
>>>>>> am sure we can find a way around providing the contents
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Mich,
>>>>>>>
>>>>>>> Thanks for your prompt response ... much appreciated. I know how to
>>>>>>> and can create login IDs on such sites but I had taken conscious decision
>>>>>>> some 20 years ago ( and i will be going against my principles) not to be on
>>>>>>> such sites. Hence I had asked for is there any other way I can join/view
>>>>>>> recording of webinar.
>>>>>>>
>>>>>>> Anyways not to worry.
>>>>>>>
>>>>>>> Thanks & Regards
>>>>>>>
>>>>>>> Nitin.
>>>>>>>
>>>>>>>
>>>>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi Nitin,
>>>>>>>
>>>>>>> Linkedin is more of a professional media. FYI, I am only a member
>>>>>>> of Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>>>>> profile for yourself in linkedin :)
>>>>>>>
>>>>>>>
>>>>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>>>>
>>>>>>> see you there as well.
>>>>>>>
>>>>>>> Best of luck.
>>>>>>>
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <
>>>>>>> nitinbhansali@yahoo.com> wrote:
>>>>>>>
>>>>>>> Hello Mich,
>>>>>>>
>>>>>>> My apologies ... but I am not on any of such social/professional
>>>>>>> sites?
>>>>>>> Any other way to access such webinars/classes?
>>>>>>>
>>>>>>> Thanks & Regards
>>>>>>> Nitin.
>>>>>>>
>>>>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>>>>> denny.g.lee@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Thanks Mich for tackling this! I encourage everyone to add to the
>>>>>>> list so we can have a comprehensive list of topics, eh?!
>>>>>>>
>>>>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>>>>
>>>>>>> https://www.linkedin.com/company/apachespark/
>>>>>>>
>>>>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>>>>
>>>>>>> You will see my post at the bottom. Please add anything else on
>>>>>>> topics to the list as a comment.
>>>>>>>
>>>>>>> We will then put them together in an article perhaps. Comments and
>>>>>>> contributions are welcome.
>>>>>>>
>>>>>>> HTH
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Denny,
>>>>>>>
>>>>>>> That Apache Spark Linkedin page
>>>>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>>>>> allows a wider audience to benefit from it.
>>>>>>>
>>>>>>> +1 for me
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>>>>> <https://www.linkedin.com/company/apachespark/> and group to
>>>>>>> broadcast these type of events - if you're cool with this? Or we could go
>>>>>>> through the process of submitting and updating the current
>>>>>>> https://spark.apache.org or request to leverage the original Spark
>>>>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>>>>> WDYT?
>>>>>>>
>>>>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Well that needs to be created first for this purpose. The
>>>>>>> appropriate name etc. to be decided. Maybe @Denny Lee
>>>>>>> <de...@gmail.com> can facilitate this as he offered his
>>>>>>> help.
>>>>>>>
>>>>>>>
>>>>>>> cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hello Mich,
>>>>>>>
>>>>>>> Can you please provide the link for the confluence page?
>>>>>>>
>>>>>>> Many thanks
>>>>>>> Asma
>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>
>>>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>>>
>>>>>>> Apologies I missed the list.
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>>
>>>>>>> 1. Spark UI
>>>>>>> 2. Dynamic allocation
>>>>>>> 3. Tuning of jobs
>>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>>> 8. Spark on notebooks
>>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>>> 10. Spark on k8s
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi guys
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Asma ZGOLLI
>>>>
>>>> PhD in Big Data - Applied Machine Learning
>>>> Email : zgolliasma@gmail.com
>>>> Tel : (+49) 015777685768
>>>> Skype : asma_zgolli
>>>>
>>>
>>
>> --
>> Bjørn Jørgensen
>> Vestre Aspehaug 4, 6010 Ålesund
>> Norge
>>
>> +47 480 94 297
>>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
https://join.slack.com/t/sparkcommunitytalk/shared_invite/zt-1rk11diac-hzGbOEdBHgjXf02IZ1mvUA
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 28 Mar 2023 at 19:38, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi Bjorn,
>
> you just need to create an account on slack and join any topic I believe
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 28 Mar 2023 at 18:57, Bjørn Jørgensen <bj...@gmail.com>
> wrote:
>
>> Do I need to get an invite before joining?
>>
>>
>> tir. 28. mar. 2023 kl. 18:51 skrev Mich Talebzadeh <
>> mich.talebzadeh@gmail.com>:
>>
>>> Hi all,
>>>
>>> There is a section in slack called webinars
>>>
>>>
>>> https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
>>>
>>> Asma Zgolli, agreed to prepare materials for Spark internals and/or
>>> comparing spark 3 and 2.
>>>
>>> I like to contribute to "Spark Streaming & Spark Structured Streaming"
>>> plus "Spark on k8s for both GCP and EKS concepts and contrasts"
>>>
>>> Other topics are mentioned below.
>>>
>>> -- Spark UI
>>> -- Dynamic allocation
>>> -- Tuning of jobs
>>> -- Collecting spark metrics for monitoring and alerting
>>> -- For those who prefer to use Pandas API on Spark since the release of
>>> Spark 3.2, What are some important notes for those users? For example, what
>>> are the additional factors affecting the Spark -- Performance using Pandas
>>> API on Spark? How to tune them in addition to the conventional Spark tuning
>>> methods applied to Spark SQL users.
>>> -- Spark internals and/or comparing spark 3 and 2
>>> -- Spark Streaming & Spark Structured Streaming
>>> -- Spark on notebooks
>>> -- Spark on serverless (for example Spark on Google Cloud)
>>> -- Spark on k8s
>>>
>>>
>>> If you are willing to contribute to presentation materials, please
>>> register your interest in slack/webinars.
>>>
>>>
>>> Thanks
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>>
>>> Spark on k8s for both GCP and EKS concepts and contrasts
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I suggest using the slack for the spark community created recently to
>>>> collaborate and work together on these topics and use the LinkedIn page to
>>>> publish the events and the webinars.
>>>>
>>>> Cheers,
>>>> Asma
>>>>
>>>> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a
>>>> écrit :
>>>>
>>>>> What we can do is get into the habit of compiling the list on LinkedIn
>>>>> but making sure this list is shared and broadcast here, eh?!
>>>>>
>>>>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>>>>> riverside.fm as well as simulcasting this on LinkedIn. This way you
>>>>> can view directly on the former without ever logging in with a user ID.
>>>>>
>>>>> HTH!!
>>>>>
>>>>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> Understood Nitin It would be wrong to act against one's conviction. I
>>>>>> am sure we can find a way around providing the contents
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Mich,
>>>>>>>
>>>>>>> Thanks for your prompt response ... much appreciated. I know how to
>>>>>>> and can create login IDs on such sites but I had taken conscious decision
>>>>>>> some 20 years ago ( and i will be going against my principles) not to be on
>>>>>>> such sites. Hence I had asked for is there any other way I can join/view
>>>>>>> recording of webinar.
>>>>>>>
>>>>>>> Anyways not to worry.
>>>>>>>
>>>>>>> Thanks & Regards
>>>>>>>
>>>>>>> Nitin.
>>>>>>>
>>>>>>>
>>>>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi Nitin,
>>>>>>>
>>>>>>> Linkedin is more of a professional media. FYI, I am only a member
>>>>>>> of Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>>>>> profile for yourself in linkedin :)
>>>>>>>
>>>>>>>
>>>>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>>>>
>>>>>>> see you there as well.
>>>>>>>
>>>>>>> Best of luck.
>>>>>>>
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <
>>>>>>> nitinbhansali@yahoo.com> wrote:
>>>>>>>
>>>>>>> Hello Mich,
>>>>>>>
>>>>>>> My apologies ... but I am not on any of such social/professional
>>>>>>> sites?
>>>>>>> Any other way to access such webinars/classes?
>>>>>>>
>>>>>>> Thanks & Regards
>>>>>>> Nitin.
>>>>>>>
>>>>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>>>>> denny.g.lee@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Thanks Mich for tackling this! I encourage everyone to add to the
>>>>>>> list so we can have a comprehensive list of topics, eh?!
>>>>>>>
>>>>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>>>>
>>>>>>> https://www.linkedin.com/company/apachespark/
>>>>>>>
>>>>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>>>>
>>>>>>> You will see my post at the bottom. Please add anything else on
>>>>>>> topics to the list as a comment.
>>>>>>>
>>>>>>> We will then put them together in an article perhaps. Comments and
>>>>>>> contributions are welcome.
>>>>>>>
>>>>>>> HTH
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>>> Palantir Technologies Limited
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Denny,
>>>>>>>
>>>>>>> That Apache Spark Linkedin page
>>>>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>>>>> allows a wider audience to benefit from it.
>>>>>>>
>>>>>>> +1 for me
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>>>>> <https://www.linkedin.com/company/apachespark/> and group to
>>>>>>> broadcast these type of events - if you're cool with this? Or we could go
>>>>>>> through the process of submitting and updating the current
>>>>>>> https://spark.apache.org or request to leverage the original Spark
>>>>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>>>>> WDYT?
>>>>>>>
>>>>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Well that needs to be created first for this purpose. The
>>>>>>> appropriate name etc. to be decided. Maybe @Denny Lee
>>>>>>> <de...@gmail.com> can facilitate this as he offered his
>>>>>>> help.
>>>>>>>
>>>>>>>
>>>>>>> cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hello Mich,
>>>>>>>
>>>>>>> Can you please provide the link for the confluence page?
>>>>>>>
>>>>>>> Many thanks
>>>>>>> Asma
>>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>>
>>>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>>>
>>>>>>> Apologies I missed the list.
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>>
>>>>>>> 1. Spark UI
>>>>>>> 2. Dynamic allocation
>>>>>>> 3. Tuning of jobs
>>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>>> 8. Spark on notebooks
>>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>>> 10. Spark on k8s
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi guys
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Asma ZGOLLI
>>>>
>>>> PhD in Big Data - Applied Machine Learning
>>>> Email : zgolliasma@gmail.com
>>>> Tel : (+49) 015777685768
>>>> Skype : asma_zgolli
>>>>
>>>
>>
>> --
>> Bjørn Jørgensen
>> Vestre Aspehaug 4, 6010 Ålesund
>> Norge
>>
>> +47 480 94 297
>>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi Bjorn,
you just need to create an account on slack and join any topic I believe
HTH
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 28 Mar 2023 at 18:57, Bjørn Jørgensen <bj...@gmail.com>
wrote:
> Do I need to get an invite before joining?
>
>
> tir. 28. mar. 2023 kl. 18:51 skrev Mich Talebzadeh <
> mich.talebzadeh@gmail.com>:
>
>> Hi all,
>>
>> There is a section in slack called webinars
>>
>>
>> https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
>>
>> Asma Zgolli, agreed to prepare materials for Spark internals and/or
>> comparing spark 3 and 2.
>>
>> I like to contribute to "Spark Streaming & Spark Structured Streaming"
>> plus "Spark on k8s for both GCP and EKS concepts and contrasts"
>>
>> Other topics are mentioned below.
>>
>> -- Spark UI
>> -- Dynamic allocation
>> -- Tuning of jobs
>> -- Collecting spark metrics for monitoring and alerting
>> -- For those who prefer to use Pandas API on Spark since the release of
>> Spark 3.2, What are some important notes for those users? For example, what
>> are the additional factors affecting the Spark -- Performance using Pandas
>> API on Spark? How to tune them in addition to the conventional Spark tuning
>> methods applied to Spark SQL users.
>> -- Spark internals and/or comparing spark 3 and 2
>> -- Spark Streaming & Spark Structured Streaming
>> -- Spark on notebooks
>> -- Spark on serverless (for example Spark on Google Cloud)
>> -- Spark on k8s
>>
>>
>> If you are willing to contribute to presentation materials, please
>> register your interest in slack/webinars.
>>
>>
>> Thanks
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>>
>> Spark on k8s for both GCP and EKS concepts and contrasts
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
>>
>>> Hello everyone,
>>>
>>> I suggest using the slack for the spark community created recently to
>>> collaborate and work together on these topics and use the LinkedIn page to
>>> publish the events and the webinars.
>>>
>>> Cheers,
>>> Asma
>>>
>>> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a
>>> écrit :
>>>
>>>> What we can do is get into the habit of compiling the list on LinkedIn
>>>> but making sure this list is shared and broadcast here, eh?!
>>>>
>>>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>>>> riverside.fm as well as simulcasting this on LinkedIn. This way you
>>>> can view directly on the former without ever logging in with a user ID.
>>>>
>>>> HTH!!
>>>>
>>>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Understood Nitin It would be wrong to act against one's conviction. I
>>>>> am sure we can find a way around providing the contents
>>>>>
>>>>> Regards
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Mich,
>>>>>>
>>>>>> Thanks for your prompt response ... much appreciated. I know how to
>>>>>> and can create login IDs on such sites but I had taken conscious decision
>>>>>> some 20 years ago ( and i will be going against my principles) not to be on
>>>>>> such sites. Hence I had asked for is there any other way I can join/view
>>>>>> recording of webinar.
>>>>>>
>>>>>> Anyways not to worry.
>>>>>>
>>>>>> Thanks & Regards
>>>>>>
>>>>>> Nitin.
>>>>>>
>>>>>>
>>>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> Hi Nitin,
>>>>>>
>>>>>> Linkedin is more of a professional media. FYI, I am only a member of
>>>>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>>>> profile for yourself in linkedin :)
>>>>>>
>>>>>>
>>>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>>>
>>>>>> see you there as well.
>>>>>>
>>>>>> Best of luck.
>>>>>>
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello Mich,
>>>>>>
>>>>>> My apologies ... but I am not on any of such social/professional
>>>>>> sites?
>>>>>> Any other way to access such webinars/classes?
>>>>>>
>>>>>> Thanks & Regards
>>>>>> Nitin.
>>>>>>
>>>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>>>> denny.g.lee@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> Thanks Mich for tackling this! I encourage everyone to add to the
>>>>>> list so we can have a comprehensive list of topics, eh?!
>>>>>>
>>>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>>>
>>>>>> https://www.linkedin.com/company/apachespark/
>>>>>>
>>>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>>>
>>>>>> You will see my post at the bottom. Please add anything else on
>>>>>> topics to the list as a comment.
>>>>>>
>>>>>> We will then put them together in an article perhaps. Comments and
>>>>>> contributions are welcome.
>>>>>>
>>>>>> HTH
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Hi Denny,
>>>>>>
>>>>>> That Apache Spark Linkedin page
>>>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>>>> allows a wider audience to benefit from it.
>>>>>>
>>>>>> +1 for me
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>>>> <https://www.linkedin.com/company/apachespark/> and group to
>>>>>> broadcast these type of events - if you're cool with this? Or we could go
>>>>>> through the process of submitting and updating the current
>>>>>> https://spark.apache.org or request to leverage the original Spark
>>>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>>>> WDYT?
>>>>>>
>>>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Well that needs to be created first for this purpose. The appropriate
>>>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com>
>>>>>> can facilitate this as he offered his help.
>>>>>>
>>>>>>
>>>>>> cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello Mich,
>>>>>>
>>>>>> Can you please provide the link for the confluence page?
>>>>>>
>>>>>> Many thanks
>>>>>> Asma
>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>
>>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>>
>>>>>> Apologies I missed the list.
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>>
>>>>>> 1. Spark UI
>>>>>> 2. Dynamic allocation
>>>>>> 3. Tuning of jobs
>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>> 8. Spark on notebooks
>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>> 10. Spark on k8s
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Hi guys
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>
>>> --
>>> Asma ZGOLLI
>>>
>>> PhD in Big Data - Applied Machine Learning
>>> Email : zgolliasma@gmail.com
>>> Tel : (+49) 015777685768
>>> Skype : asma_zgolli
>>>
>>
>
> --
> Bjørn Jørgensen
> Vestre Aspehaug 4, 6010 Ålesund
> Norge
>
> +47 480 94 297
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi Bjorn,
you just need to create an account on slack and join any topic I believe
HTH
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 28 Mar 2023 at 18:57, Bjørn Jørgensen <bj...@gmail.com>
wrote:
> Do I need to get an invite before joining?
>
>
> tir. 28. mar. 2023 kl. 18:51 skrev Mich Talebzadeh <
> mich.talebzadeh@gmail.com>:
>
>> Hi all,
>>
>> There is a section in slack called webinars
>>
>>
>> https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
>>
>> Asma Zgolli, agreed to prepare materials for Spark internals and/or
>> comparing spark 3 and 2.
>>
>> I like to contribute to "Spark Streaming & Spark Structured Streaming"
>> plus "Spark on k8s for both GCP and EKS concepts and contrasts"
>>
>> Other topics are mentioned below.
>>
>> -- Spark UI
>> -- Dynamic allocation
>> -- Tuning of jobs
>> -- Collecting spark metrics for monitoring and alerting
>> -- For those who prefer to use Pandas API on Spark since the release of
>> Spark 3.2, What are some important notes for those users? For example, what
>> are the additional factors affecting the Spark -- Performance using Pandas
>> API on Spark? How to tune them in addition to the conventional Spark tuning
>> methods applied to Spark SQL users.
>> -- Spark internals and/or comparing spark 3 and 2
>> -- Spark Streaming & Spark Structured Streaming
>> -- Spark on notebooks
>> -- Spark on serverless (for example Spark on Google Cloud)
>> -- Spark on k8s
>>
>>
>> If you are willing to contribute to presentation materials, please
>> register your interest in slack/webinars.
>>
>>
>> Thanks
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>>
>> Spark on k8s for both GCP and EKS concepts and contrasts
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
>>
>>> Hello everyone,
>>>
>>> I suggest using the slack for the spark community created recently to
>>> collaborate and work together on these topics and use the LinkedIn page to
>>> publish the events and the webinars.
>>>
>>> Cheers,
>>> Asma
>>>
>>> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a
>>> écrit :
>>>
>>>> What we can do is get into the habit of compiling the list on LinkedIn
>>>> but making sure this list is shared and broadcast here, eh?!
>>>>
>>>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>>>> riverside.fm as well as simulcasting this on LinkedIn. This way you
>>>> can view directly on the former without ever logging in with a user ID.
>>>>
>>>> HTH!!
>>>>
>>>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Understood Nitin It would be wrong to act against one's conviction. I
>>>>> am sure we can find a way around providing the contents
>>>>>
>>>>> Regards
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Mich,
>>>>>>
>>>>>> Thanks for your prompt response ... much appreciated. I know how to
>>>>>> and can create login IDs on such sites but I had taken conscious decision
>>>>>> some 20 years ago ( and i will be going against my principles) not to be on
>>>>>> such sites. Hence I had asked for is there any other way I can join/view
>>>>>> recording of webinar.
>>>>>>
>>>>>> Anyways not to worry.
>>>>>>
>>>>>> Thanks & Regards
>>>>>>
>>>>>> Nitin.
>>>>>>
>>>>>>
>>>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> Hi Nitin,
>>>>>>
>>>>>> Linkedin is more of a professional media. FYI, I am only a member of
>>>>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>>>> profile for yourself in linkedin :)
>>>>>>
>>>>>>
>>>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>>>
>>>>>> see you there as well.
>>>>>>
>>>>>> Best of luck.
>>>>>>
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello Mich,
>>>>>>
>>>>>> My apologies ... but I am not on any of such social/professional
>>>>>> sites?
>>>>>> Any other way to access such webinars/classes?
>>>>>>
>>>>>> Thanks & Regards
>>>>>> Nitin.
>>>>>>
>>>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>>>> denny.g.lee@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> Thanks Mich for tackling this! I encourage everyone to add to the
>>>>>> list so we can have a comprehensive list of topics, eh?!
>>>>>>
>>>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>>>
>>>>>> https://www.linkedin.com/company/apachespark/
>>>>>>
>>>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>>>
>>>>>> You will see my post at the bottom. Please add anything else on
>>>>>> topics to the list as a comment.
>>>>>>
>>>>>> We will then put them together in an article perhaps. Comments and
>>>>>> contributions are welcome.
>>>>>>
>>>>>> HTH
>>>>>>
>>>>>> Mich Talebzadeh,
>>>>>> Lead Solutions Architect/Engineering Lead,
>>>>>> Palantir Technologies Limited
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Hi Denny,
>>>>>>
>>>>>> That Apache Spark Linkedin page
>>>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>>>> allows a wider audience to benefit from it.
>>>>>>
>>>>>> +1 for me
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>>>> <https://www.linkedin.com/company/apachespark/> and group to
>>>>>> broadcast these type of events - if you're cool with this? Or we could go
>>>>>> through the process of submitting and updating the current
>>>>>> https://spark.apache.org or request to leverage the original Spark
>>>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>>>> WDYT?
>>>>>>
>>>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Well that needs to be created first for this purpose. The appropriate
>>>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com>
>>>>>> can facilitate this as he offered his help.
>>>>>>
>>>>>>
>>>>>> cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello Mich,
>>>>>>
>>>>>> Can you please provide the link for the confluence page?
>>>>>>
>>>>>> Many thanks
>>>>>> Asma
>>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>>
>>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>>
>>>>>> Apologies I missed the list.
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>>
>>>>>> 1. Spark UI
>>>>>> 2. Dynamic allocation
>>>>>> 3. Tuning of jobs
>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>> 8. Spark on notebooks
>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>> 10. Spark on k8s
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>> Hi guys
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>
>>> --
>>> Asma ZGOLLI
>>>
>>> PhD in Big Data - Applied Machine Learning
>>> Email : zgolliasma@gmail.com
>>> Tel : (+49) 015777685768
>>> Skype : asma_zgolli
>>>
>>
>
> --
> Bjørn Jørgensen
> Vestre Aspehaug 4, 6010 Ålesund
> Norge
>
> +47 480 94 297
>
Re: Topics for Spark online classes & webinars
Posted by Bjørn Jørgensen <bj...@gmail.com>.
Do I need to get an invite before joining?
tir. 28. mar. 2023 kl. 18:51 skrev Mich Talebzadeh <
mich.talebzadeh@gmail.com>:
> Hi all,
>
> There is a section in slack called webinars
>
>
> https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
>
> Asma Zgolli, agreed to prepare materials for Spark internals and/or
> comparing spark 3 and 2.
>
> I like to contribute to "Spark Streaming & Spark Structured Streaming"
> plus "Spark on k8s for both GCP and EKS concepts and contrasts"
>
> Other topics are mentioned below.
>
> -- Spark UI
> -- Dynamic allocation
> -- Tuning of jobs
> -- Collecting spark metrics for monitoring and alerting
> -- For those who prefer to use Pandas API on Spark since the release of
> Spark 3.2, What are some important notes for those users? For example, what
> are the additional factors affecting the Spark -- Performance using Pandas
> API on Spark? How to tune them in addition to the conventional Spark tuning
> methods applied to Spark SQL users.
> -- Spark internals and/or comparing spark 3 and 2
> -- Spark Streaming & Spark Structured Streaming
> -- Spark on notebooks
> -- Spark on serverless (for example Spark on Google Cloud)
> -- Spark on k8s
>
>
> If you are willing to contribute to presentation materials, please
> register your interest in slack/webinars.
>
>
> Thanks
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
>
> Spark on k8s for both GCP and EKS concepts and contrasts
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
>
>> Hello everyone,
>>
>> I suggest using the slack for the spark community created recently to
>> collaborate and work together on these topics and use the LinkedIn page to
>> publish the events and the webinars.
>>
>> Cheers,
>> Asma
>>
>> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a écrit :
>>
>>> What we can do is get into the habit of compiling the list on LinkedIn
>>> but making sure this list is shared and broadcast here, eh?!
>>>
>>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>>> riverside.fm as well as simulcasting this on LinkedIn. This way you can
>>> view directly on the former without ever logging in with a user ID.
>>>
>>> HTH!!
>>>
>>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Understood Nitin It would be wrong to act against one's conviction. I
>>>> am sure we can find a way around providing the contents
>>>>
>>>> Regards
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>>> wrote:
>>>>
>>>>> Hi Mich,
>>>>>
>>>>> Thanks for your prompt response ... much appreciated. I know how to
>>>>> and can create login IDs on such sites but I had taken conscious decision
>>>>> some 20 years ago ( and i will be going against my principles) not to be on
>>>>> such sites. Hence I had asked for is there any other way I can join/view
>>>>> recording of webinar.
>>>>>
>>>>> Anyways not to worry.
>>>>>
>>>>> Thanks & Regards
>>>>>
>>>>> Nitin.
>>>>>
>>>>>
>>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>
>>>>> Hi Nitin,
>>>>>
>>>>> Linkedin is more of a professional media. FYI, I am only a member of
>>>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>>> profile for yourself in linkedin :)
>>>>>
>>>>>
>>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>>
>>>>> see you there as well.
>>>>>
>>>>> Best of luck.
>>>>>
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead,
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>>>> wrote:
>>>>>
>>>>> Hello Mich,
>>>>>
>>>>> My apologies ... but I am not on any of such social/professional
>>>>> sites?
>>>>> Any other way to access such webinars/classes?
>>>>>
>>>>> Thanks & Regards
>>>>> Nitin.
>>>>>
>>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>>> denny.g.lee@gmail.com> wrote:
>>>>>
>>>>>
>>>>> Thanks Mich for tackling this! I encourage everyone to add to the
>>>>> list so we can have a comprehensive list of topics, eh?!
>>>>>
>>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>>
>>>>> https://www.linkedin.com/company/apachespark/
>>>>>
>>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>>
>>>>> You will see my post at the bottom. Please add anything else on topics
>>>>> to the list as a comment.
>>>>>
>>>>> We will then put them together in an article perhaps. Comments and
>>>>> contributions are welcome.
>>>>>
>>>>> HTH
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead,
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Hi Denny,
>>>>>
>>>>> That Apache Spark Linkedin page
>>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>>> allows a wider audience to benefit from it.
>>>>>
>>>>> +1 for me
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>>>
>>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>>> <https://www.linkedin.com/company/apachespark/> and group to
>>>>> broadcast these type of events - if you're cool with this? Or we could go
>>>>> through the process of submitting and updating the current
>>>>> https://spark.apache.org or request to leverage the original Spark
>>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>>> WDYT?
>>>>>
>>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Well that needs to be created first for this purpose. The appropriate
>>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com>
>>>>> can facilitate this as he offered his help.
>>>>>
>>>>>
>>>>> cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hello Mich,
>>>>>
>>>>> Can you please provide the link for the confluence page?
>>>>>
>>>>> Many thanks
>>>>> Asma
>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>
>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>
>>>>> Apologies I missed the list.
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>>
>>>>> 1. Spark UI
>>>>> 2. Dynamic allocation
>>>>> 3. Tuning of jobs
>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>> example, what are the additional factors affecting the Spark performance
>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>> Spark tuning methods applied to Spark SQL users.
>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>> 8. Spark on notebooks
>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>> 10. Spark on k8s
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Hi guys
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>
>> --
>> Asma ZGOLLI
>>
>> PhD in Big Data - Applied Machine Learning
>> Email : zgolliasma@gmail.com
>> Tel : (+49) 015777685768
>> Skype : asma_zgolli
>>
>
--
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge
+47 480 94 297
Re: Topics for Spark online classes & webinars
Posted by Bjørn Jørgensen <bj...@gmail.com>.
Do I need to get an invite before joining?
tir. 28. mar. 2023 kl. 18:51 skrev Mich Talebzadeh <
mich.talebzadeh@gmail.com>:
> Hi all,
>
> There is a section in slack called webinars
>
>
> https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
>
> Asma Zgolli, agreed to prepare materials for Spark internals and/or
> comparing spark 3 and 2.
>
> I like to contribute to "Spark Streaming & Spark Structured Streaming"
> plus "Spark on k8s for both GCP and EKS concepts and contrasts"
>
> Other topics are mentioned below.
>
> -- Spark UI
> -- Dynamic allocation
> -- Tuning of jobs
> -- Collecting spark metrics for monitoring and alerting
> -- For those who prefer to use Pandas API on Spark since the release of
> Spark 3.2, What are some important notes for those users? For example, what
> are the additional factors affecting the Spark -- Performance using Pandas
> API on Spark? How to tune them in addition to the conventional Spark tuning
> methods applied to Spark SQL users.
> -- Spark internals and/or comparing spark 3 and 2
> -- Spark Streaming & Spark Structured Streaming
> -- Spark on notebooks
> -- Spark on serverless (for example Spark on Google Cloud)
> -- Spark on k8s
>
>
> If you are willing to contribute to presentation materials, please
> register your interest in slack/webinars.
>
>
> Thanks
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
>
> Spark on k8s for both GCP and EKS concepts and contrasts
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
>
>> Hello everyone,
>>
>> I suggest using the slack for the spark community created recently to
>> collaborate and work together on these topics and use the LinkedIn page to
>> publish the events and the webinars.
>>
>> Cheers,
>> Asma
>>
>> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a écrit :
>>
>>> What we can do is get into the habit of compiling the list on LinkedIn
>>> but making sure this list is shared and broadcast here, eh?!
>>>
>>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>>> riverside.fm as well as simulcasting this on LinkedIn. This way you can
>>> view directly on the former without ever logging in with a user ID.
>>>
>>> HTH!!
>>>
>>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Understood Nitin It would be wrong to act against one's conviction. I
>>>> am sure we can find a way around providing the contents
>>>>
>>>> Regards
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>>> wrote:
>>>>
>>>>> Hi Mich,
>>>>>
>>>>> Thanks for your prompt response ... much appreciated. I know how to
>>>>> and can create login IDs on such sites but I had taken conscious decision
>>>>> some 20 years ago ( and i will be going against my principles) not to be on
>>>>> such sites. Hence I had asked for is there any other way I can join/view
>>>>> recording of webinar.
>>>>>
>>>>> Anyways not to worry.
>>>>>
>>>>> Thanks & Regards
>>>>>
>>>>> Nitin.
>>>>>
>>>>>
>>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>
>>>>> Hi Nitin,
>>>>>
>>>>> Linkedin is more of a professional media. FYI, I am only a member of
>>>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>>> profile for yourself in linkedin :)
>>>>>
>>>>>
>>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>>
>>>>> see you there as well.
>>>>>
>>>>> Best of luck.
>>>>>
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead,
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>>>> wrote:
>>>>>
>>>>> Hello Mich,
>>>>>
>>>>> My apologies ... but I am not on any of such social/professional
>>>>> sites?
>>>>> Any other way to access such webinars/classes?
>>>>>
>>>>> Thanks & Regards
>>>>> Nitin.
>>>>>
>>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>>> denny.g.lee@gmail.com> wrote:
>>>>>
>>>>>
>>>>> Thanks Mich for tackling this! I encourage everyone to add to the
>>>>> list so we can have a comprehensive list of topics, eh?!
>>>>>
>>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>>
>>>>> https://www.linkedin.com/company/apachespark/
>>>>>
>>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>>
>>>>> You will see my post at the bottom. Please add anything else on topics
>>>>> to the list as a comment.
>>>>>
>>>>> We will then put them together in an article perhaps. Comments and
>>>>> contributions are welcome.
>>>>>
>>>>> HTH
>>>>>
>>>>> Mich Talebzadeh,
>>>>> Lead Solutions Architect/Engineering Lead,
>>>>> Palantir Technologies Limited
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Hi Denny,
>>>>>
>>>>> That Apache Spark Linkedin page
>>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>>> allows a wider audience to benefit from it.
>>>>>
>>>>> +1 for me
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>>>
>>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>>> <https://www.linkedin.com/company/apachespark/> and group to
>>>>> broadcast these type of events - if you're cool with this? Or we could go
>>>>> through the process of submitting and updating the current
>>>>> https://spark.apache.org or request to leverage the original Spark
>>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>>> WDYT?
>>>>>
>>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Well that needs to be created first for this purpose. The appropriate
>>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com>
>>>>> can facilitate this as he offered his help.
>>>>>
>>>>>
>>>>> cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hello Mich,
>>>>>
>>>>> Can you please provide the link for the confluence page?
>>>>>
>>>>> Many thanks
>>>>> Asma
>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>
>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>
>>>>> Apologies I missed the list.
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>>
>>>>> 1. Spark UI
>>>>> 2. Dynamic allocation
>>>>> 3. Tuning of jobs
>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>> example, what are the additional factors affecting the Spark performance
>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>> Spark tuning methods applied to Spark SQL users.
>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>> 8. Spark on notebooks
>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>> 10. Spark on k8s
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>> Hi guys
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>
>> --
>> Asma ZGOLLI
>>
>> PhD in Big Data - Applied Machine Learning
>> Email : zgolliasma@gmail.com
>> Tel : (+49) 015777685768
>> Skype : asma_zgolli
>>
>
--
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge
+47 480 94 297
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi all,
There is a section in slack called webinars
https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
Asma Zgolli, agreed to prepare materials for Spark internals and/or
comparing spark 3 and 2.
I like to contribute to "Spark Streaming & Spark Structured Streaming" plus
"Spark on k8s for both GCP and EKS concepts and contrasts"
Other topics are mentioned below.
-- Spark UI
-- Dynamic allocation
-- Tuning of jobs
-- Collecting spark metrics for monitoring and alerting
-- For those who prefer to use Pandas API on Spark since the release of
Spark 3.2, What are some important notes for those users? For example, what
are the additional factors affecting the Spark -- Performance using Pandas
API on Spark? How to tune them in addition to the conventional Spark tuning
methods applied to Spark SQL users.
-- Spark internals and/or comparing spark 3 and 2
-- Spark Streaming & Spark Structured Streaming
-- Spark on notebooks
-- Spark on serverless (for example Spark on Google Cloud)
-- Spark on k8s
If you are willing to contribute to presentation materials, please register
your interest in slack/webinars.
Thanks
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
Spark on k8s for both GCP and EKS concepts and contrasts
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
> Hello everyone,
>
> I suggest using the slack for the spark community created recently to
> collaborate and work together on these topics and use the LinkedIn page to
> publish the events and the webinars.
>
> Cheers,
> Asma
>
> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a écrit :
>
>> What we can do is get into the habit of compiling the list on LinkedIn
>> but making sure this list is shared and broadcast here, eh?!
>>
>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>> riverside.fm as well as simulcasting this on LinkedIn. This way you can
>> view directly on the former without ever logging in with a user ID.
>>
>> HTH!!
>>
>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Understood Nitin It would be wrong to act against one's conviction. I am
>>> sure we can find a way around providing the contents
>>>
>>> Regards
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>> wrote:
>>>
>>>> Hi Mich,
>>>>
>>>> Thanks for your prompt response ... much appreciated. I know how to and
>>>> can create login IDs on such sites but I had taken conscious decision some
>>>> 20 years ago ( and i will be going against my principles) not to be on such
>>>> sites. Hence I had asked for is there any other way I can join/view
>>>> recording of webinar.
>>>>
>>>> Anyways not to worry.
>>>>
>>>> Thanks & Regards
>>>>
>>>> Nitin.
>>>>
>>>>
>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>
>>>> Hi Nitin,
>>>>
>>>> Linkedin is more of a professional media. FYI, I am only a member of
>>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>> profile for yourself in linkedin :)
>>>>
>>>>
>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>
>>>> see you there as well.
>>>>
>>>> Best of luck.
>>>>
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead,
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>>> wrote:
>>>>
>>>> Hello Mich,
>>>>
>>>> My apologies ... but I am not on any of such social/professional
>>>> sites?
>>>> Any other way to access such webinars/classes?
>>>>
>>>> Thanks & Regards
>>>> Nitin.
>>>>
>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>> denny.g.lee@gmail.com> wrote:
>>>>
>>>>
>>>> Thanks Mich for tackling this! I encourage everyone to add to the list
>>>> so we can have a comprehensive list of topics, eh?!
>>>>
>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>
>>>> https://www.linkedin.com/company/apachespark/
>>>>
>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>
>>>> You will see my post at the bottom. Please add anything else on topics
>>>> to the list as a comment.
>>>>
>>>> We will then put them together in an article perhaps. Comments and
>>>> contributions are welcome.
>>>>
>>>> HTH
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead,
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Hi Denny,
>>>>
>>>> That Apache Spark Linkedin page
>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>> allows a wider audience to benefit from it.
>>>>
>>>> +1 for me
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>>
>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>>> these type of events - if you're cool with this? Or we could go through
>>>> the process of submitting and updating the current
>>>> https://spark.apache.org or request to leverage the original Spark
>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>> WDYT?
>>>>
>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Well that needs to be created first for this purpose. The appropriate
>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>>> facilitate this as he offered his help.
>>>>
>>>>
>>>> cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>>
>>>> Hello Mich,
>>>>
>>>> Can you please provide the link for the confluence page?
>>>>
>>>> Many thanks
>>>> Asma
>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>
>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>
>>>> Apologies I missed the list.
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>>
>>>> 1. Spark UI
>>>> 2. Dynamic allocation
>>>> 3. Tuning of jobs
>>>> 4. Collecting spark metrics for monitoring and alerting
>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>> release of Spark 3.2, What are some important notes for those users? For
>>>> example, what are the additional factors affecting the Spark performance
>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>> Spark tuning methods applied to Spark SQL users.
>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>> 7. Spark Streaming & Spark Structured Streaming
>>>> 8. Spark on notebooks
>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>> 10. Spark on k8s
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Hi guys
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>
> --
> Asma ZGOLLI
>
> PhD in Big Data - Applied Machine Learning
> Email : zgolliasma@gmail.com
> Tel : (+49) 015777685768
> Skype : asma_zgolli
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi all,
There is a section in slack called webinars
https://sparkcommunitytalk.slack.com/x-p4977943407059-5006939220983-5006939446887/messages/C0501NBTNQG
Asma Zgolli, agreed to prepare materials for Spark internals and/or
comparing spark 3 and 2.
I like to contribute to "Spark Streaming & Spark Structured Streaming" plus
"Spark on k8s for both GCP and EKS concepts and contrasts"
Other topics are mentioned below.
-- Spark UI
-- Dynamic allocation
-- Tuning of jobs
-- Collecting spark metrics for monitoring and alerting
-- For those who prefer to use Pandas API on Spark since the release of
Spark 3.2, What are some important notes for those users? For example, what
are the additional factors affecting the Spark -- Performance using Pandas
API on Spark? How to tune them in addition to the conventional Spark tuning
methods applied to Spark SQL users.
-- Spark internals and/or comparing spark 3 and 2
-- Spark Streaming & Spark Structured Streaming
-- Spark on notebooks
-- Spark on serverless (for example Spark on Google Cloud)
-- Spark on k8s
If you are willing to contribute to presentation materials, please register
your interest in slack/webinars.
Thanks
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
Spark on k8s for both GCP and EKS concepts and contrasts
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 28 Mar 2023 at 13:55, asma zgolli <zg...@gmail.com> wrote:
> Hello everyone,
>
> I suggest using the slack for the spark community created recently to
> collaborate and work together on these topics and use the LinkedIn page to
> publish the events and the webinars.
>
> Cheers,
> Asma
>
> Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a écrit :
>
>> What we can do is get into the habit of compiling the list on LinkedIn
>> but making sure this list is shared and broadcast here, eh?!
>>
>> As well, when we broadcast the videos, we can do this using zoom/jitsi/
>> riverside.fm as well as simulcasting this on LinkedIn. This way you can
>> view directly on the former without ever logging in with a user ID.
>>
>> HTH!!
>>
>> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Understood Nitin It would be wrong to act against one's conviction. I am
>>> sure we can find a way around providing the contents
>>>
>>> Regards
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>>> wrote:
>>>
>>>> Hi Mich,
>>>>
>>>> Thanks for your prompt response ... much appreciated. I know how to and
>>>> can create login IDs on such sites but I had taken conscious decision some
>>>> 20 years ago ( and i will be going against my principles) not to be on such
>>>> sites. Hence I had asked for is there any other way I can join/view
>>>> recording of webinar.
>>>>
>>>> Anyways not to worry.
>>>>
>>>> Thanks & Regards
>>>>
>>>> Nitin.
>>>>
>>>>
>>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>
>>>> Hi Nitin,
>>>>
>>>> Linkedin is more of a professional media. FYI, I am only a member of
>>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>>> profile for yourself in linkedin :)
>>>>
>>>>
>>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>>
>>>> see you there as well.
>>>>
>>>> Best of luck.
>>>>
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead,
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>>> wrote:
>>>>
>>>> Hello Mich,
>>>>
>>>> My apologies ... but I am not on any of such social/professional
>>>> sites?
>>>> Any other way to access such webinars/classes?
>>>>
>>>> Thanks & Regards
>>>> Nitin.
>>>>
>>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>>> denny.g.lee@gmail.com> wrote:
>>>>
>>>>
>>>> Thanks Mich for tackling this! I encourage everyone to add to the list
>>>> so we can have a comprehensive list of topics, eh?!
>>>>
>>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>>
>>>> https://www.linkedin.com/company/apachespark/
>>>>
>>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>>
>>>> You will see my post at the bottom. Please add anything else on topics
>>>> to the list as a comment.
>>>>
>>>> We will then put them together in an article perhaps. Comments and
>>>> contributions are welcome.
>>>>
>>>> HTH
>>>>
>>>> Mich Talebzadeh,
>>>> Lead Solutions Architect/Engineering Lead,
>>>> Palantir Technologies Limited
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Hi Denny,
>>>>
>>>> That Apache Spark Linkedin page
>>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>>> allows a wider audience to benefit from it.
>>>>
>>>> +1 for me
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>>
>>>> In the past, we've been using the Apache Spark LinkedIn page
>>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>>> these type of events - if you're cool with this? Or we could go through
>>>> the process of submitting and updating the current
>>>> https://spark.apache.org or request to leverage the original Spark
>>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>>> WDYT?
>>>>
>>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Well that needs to be created first for this purpose. The appropriate
>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>>> facilitate this as he offered his help.
>>>>
>>>>
>>>> cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>>
>>>> Hello Mich,
>>>>
>>>> Can you please provide the link for the confluence page?
>>>>
>>>> Many thanks
>>>> Asma
>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>
>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>
>>>> Apologies I missed the list.
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>>
>>>> 1. Spark UI
>>>> 2. Dynamic allocation
>>>> 3. Tuning of jobs
>>>> 4. Collecting spark metrics for monitoring and alerting
>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>> release of Spark 3.2, What are some important notes for those users? For
>>>> example, what are the additional factors affecting the Spark performance
>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>> Spark tuning methods applied to Spark SQL users.
>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>> 7. Spark Streaming & Spark Structured Streaming
>>>> 8. Spark on notebooks
>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>> 10. Spark on k8s
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>> Hi guys
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>
> --
> Asma ZGOLLI
>
> PhD in Big Data - Applied Machine Learning
> Email : zgolliasma@gmail.com
> Tel : (+49) 015777685768
> Skype : asma_zgolli
>
Re: Topics for Spark online classes & webinars
Posted by asma zgolli <zg...@gmail.com>.
Hello everyone,
I suggest using the slack for the spark community created recently to
collaborate and work together on these topics and use the LinkedIn page to
publish the events and the webinars.
Cheers,
Asma
Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a écrit :
> What we can do is get into the habit of compiling the list on LinkedIn but
> making sure this list is shared and broadcast here, eh?!
>
> As well, when we broadcast the videos, we can do this using zoom/jitsi/
> riverside.fm as well as simulcasting this on LinkedIn. This way you can
> view directly on the former without ever logging in with a user ID.
>
> HTH!!
>
> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Understood Nitin It would be wrong to act against one's conviction. I am
>> sure we can find a way around providing the contents
>>
>> Regards
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>> wrote:
>>
>>> Hi Mich,
>>>
>>> Thanks for your prompt response ... much appreciated. I know how to and
>>> can create login IDs on such sites but I had taken conscious decision some
>>> 20 years ago ( and i will be going against my principles) not to be on such
>>> sites. Hence I had asked for is there any other way I can join/view
>>> recording of webinar.
>>>
>>> Anyways not to worry.
>>>
>>> Thanks & Regards
>>>
>>> Nitin.
>>>
>>>
>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>
>>> Hi Nitin,
>>>
>>> Linkedin is more of a professional media. FYI, I am only a member of
>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>> profile for yourself in linkedin :)
>>>
>>>
>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>
>>> see you there as well.
>>>
>>> Best of luck.
>>>
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead,
>>> Palantir Technologies Limited
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>> wrote:
>>>
>>> Hello Mich,
>>>
>>> My apologies ... but I am not on any of such social/professional
>>> sites?
>>> Any other way to access such webinars/classes?
>>>
>>> Thanks & Regards
>>> Nitin.
>>>
>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>> denny.g.lee@gmail.com> wrote:
>>>
>>>
>>> Thanks Mich for tackling this! I encourage everyone to add to the list
>>> so we can have a comprehensive list of topics, eh?!
>>>
>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi all,
>>>
>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>
>>> https://www.linkedin.com/company/apachespark/
>>>
>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>
>>> You will see my post at the bottom. Please add anything else on topics
>>> to the list as a comment.
>>>
>>> We will then put them together in an article perhaps. Comments and
>>> contributions are welcome.
>>>
>>> HTH
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead,
>>> Palantir Technologies Limited
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi Denny,
>>>
>>> That Apache Spark Linkedin page
>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>> allows a wider audience to benefit from it.
>>>
>>> +1 for me
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>
>>> In the past, we've been using the Apache Spark LinkedIn page
>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>> these type of events - if you're cool with this? Or we could go through
>>> the process of submitting and updating the current
>>> https://spark.apache.org or request to leverage the original Spark
>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>> WDYT?
>>>
>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>> Well that needs to be created first for this purpose. The appropriate
>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>> facilitate this as he offered his help.
>>>
>>>
>>> cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>
>>> Hello Mich,
>>>
>>> Can you please provide the link for the confluence page?
>>>
>>> Many thanks
>>> Asma
>>> Ph.D. in Big Data - Applied Machine Learning
>>>
>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>>> a écrit :
>>>
>>> Apologies I missed the list.
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>>
>>> 1. Spark UI
>>> 2. Dynamic allocation
>>> 3. Tuning of jobs
>>> 4. Collecting spark metrics for monitoring and alerting
>>> 5. For those who prefer to use Pandas API on Spark since the
>>> release of Spark 3.2, What are some important notes for those users? For
>>> example, what are the additional factors affecting the Spark performance
>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>> Spark tuning methods applied to Spark SQL users.
>>> 6. Spark internals and/or comparing spark 3 and 2
>>> 7. Spark Streaming & Spark Structured Streaming
>>> 8. Spark on notebooks
>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>> 10. Spark on k8s
>>>
>>> Opinions and how to is welcome
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi guys
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>> Opinions and how to is welcome
>>>
>>> Cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
--
Asma ZGOLLI
PhD in Big Data - Applied Machine Learning
Email : zgolliasma@gmail.com
Tel : (+49) 015777685768
Skype : asma_zgolli
Re: Topics for Spark online classes & webinars
Posted by asma zgolli <zg...@gmail.com>.
Hello everyone,
I suggest using the slack for the spark community created recently to
collaborate and work together on these topics and use the LinkedIn page to
publish the events and the webinars.
Cheers,
Asma
Le jeu. 16 mars 2023 à 01:39, Denny Lee <de...@gmail.com> a écrit :
> What we can do is get into the habit of compiling the list on LinkedIn but
> making sure this list is shared and broadcast here, eh?!
>
> As well, when we broadcast the videos, we can do this using zoom/jitsi/
> riverside.fm as well as simulcasting this on LinkedIn. This way you can
> view directly on the former without ever logging in with a user ID.
>
> HTH!!
>
> On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Understood Nitin It would be wrong to act against one's conviction. I am
>> sure we can find a way around providing the contents
>>
>> Regards
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
>> wrote:
>>
>>> Hi Mich,
>>>
>>> Thanks for your prompt response ... much appreciated. I know how to and
>>> can create login IDs on such sites but I had taken conscious decision some
>>> 20 years ago ( and i will be going against my principles) not to be on such
>>> sites. Hence I had asked for is there any other way I can join/view
>>> recording of webinar.
>>>
>>> Anyways not to worry.
>>>
>>> Thanks & Regards
>>>
>>> Nitin.
>>>
>>>
>>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>
>>> Hi Nitin,
>>>
>>> Linkedin is more of a professional media. FYI, I am only a member of
>>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>>> profile for yourself in linkedin :)
>>>
>>>
>>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>>
>>> see you there as well.
>>>
>>> Best of luck.
>>>
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead,
>>> Palantir Technologies Limited
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>>> wrote:
>>>
>>> Hello Mich,
>>>
>>> My apologies ... but I am not on any of such social/professional
>>> sites?
>>> Any other way to access such webinars/classes?
>>>
>>> Thanks & Regards
>>> Nitin.
>>>
>>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>>> denny.g.lee@gmail.com> wrote:
>>>
>>>
>>> Thanks Mich for tackling this! I encourage everyone to add to the list
>>> so we can have a comprehensive list of topics, eh?!
>>>
>>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi all,
>>>
>>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>>
>>> https://www.linkedin.com/company/apachespark/
>>>
>>> and contribution from @asma zgolli <zg...@gmail.com>
>>>
>>> You will see my post at the bottom. Please add anything else on topics
>>> to the list as a comment.
>>>
>>> We will then put them together in an article perhaps. Comments and
>>> contributions are welcome.
>>>
>>> HTH
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead,
>>> Palantir Technologies Limited
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi Denny,
>>>
>>> That Apache Spark Linkedin page
>>> https://www.linkedin.com/company/apachespark/ looks fine. It also
>>> allows a wider audience to benefit from it.
>>>
>>> +1 for me
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>>
>>> In the past, we've been using the Apache Spark LinkedIn page
>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>> these type of events - if you're cool with this? Or we could go through
>>> the process of submitting and updating the current
>>> https://spark.apache.org or request to leverage the original Spark
>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>> WDYT?
>>>
>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>> Well that needs to be created first for this purpose. The appropriate
>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>> facilitate this as he offered his help.
>>>
>>>
>>> cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>
>>> Hello Mich,
>>>
>>> Can you please provide the link for the confluence page?
>>>
>>> Many thanks
>>> Asma
>>> Ph.D. in Big Data - Applied Machine Learning
>>>
>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>>> a écrit :
>>>
>>> Apologies I missed the list.
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>>
>>> 1. Spark UI
>>> 2. Dynamic allocation
>>> 3. Tuning of jobs
>>> 4. Collecting spark metrics for monitoring and alerting
>>> 5. For those who prefer to use Pandas API on Spark since the
>>> release of Spark 3.2, What are some important notes for those users? For
>>> example, what are the additional factors affecting the Spark performance
>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>> Spark tuning methods applied to Spark SQL users.
>>> 6. Spark internals and/or comparing spark 3 and 2
>>> 7. Spark Streaming & Spark Structured Streaming
>>> 8. Spark on notebooks
>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>> 10. Spark on k8s
>>>
>>> Opinions and how to is welcome
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi guys
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>> Opinions and how to is welcome
>>>
>>> Cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
--
Asma ZGOLLI
PhD in Big Data - Applied Machine Learning
Email : zgolliasma@gmail.com
Tel : (+49) 015777685768
Skype : asma_zgolli
Re: Topics for Spark online classes & webinars
Posted by Denny Lee <de...@gmail.com>.
What we can do is get into the habit of compiling the list on LinkedIn but
making sure this list is shared and broadcast here, eh?!
As well, when we broadcast the videos, we can do this using zoom/jitsi/
riverside.fm as well as simulcasting this on LinkedIn. This way you can
view directly on the former without ever logging in with a user ID.
HTH!!
On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Understood Nitin It would be wrong to act against one's conviction. I am
> sure we can find a way around providing the contents
>
> Regards
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
> wrote:
>
>> Hi Mich,
>>
>> Thanks for your prompt response ... much appreciated. I know how to and
>> can create login IDs on such sites but I had taken conscious decision some
>> 20 years ago ( and i will be going against my principles) not to be on such
>> sites. Hence I had asked for is there any other way I can join/view
>> recording of webinar.
>>
>> Anyways not to worry.
>>
>> Thanks & Regards
>>
>> Nitin.
>>
>>
>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>
>> Hi Nitin,
>>
>> Linkedin is more of a professional media. FYI, I am only a member of
>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>> profile for yourself in linkedin :)
>>
>>
>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>
>> see you there as well.
>>
>> Best of luck.
>>
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead,
>> Palantir Technologies Limited
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>> wrote:
>>
>> Hello Mich,
>>
>> My apologies ... but I am not on any of such social/professional sites?
>> Any other way to access such webinars/classes?
>>
>> Thanks & Regards
>> Nitin.
>>
>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>> denny.g.lee@gmail.com> wrote:
>>
>>
>> Thanks Mich for tackling this! I encourage everyone to add to the list
>> so we can have a comprehensive list of topics, eh?!
>>
>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi all,
>>
>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>
>> https://www.linkedin.com/company/apachespark/
>>
>> and contribution from @asma zgolli <zg...@gmail.com>
>>
>> You will see my post at the bottom. Please add anything else on topics to
>> the list as a comment.
>>
>> We will then put them together in an article perhaps. Comments and
>> contributions are welcome.
>>
>> HTH
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead,
>> Palantir Technologies Limited
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi Denny,
>>
>> That Apache Spark Linkedin page
>> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
>> a wider audience to benefit from it.
>>
>> +1 for me
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>
>> In the past, we've been using the Apache Spark LinkedIn page
>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>> these type of events - if you're cool with this? Or we could go through
>> the process of submitting and updating the current
>> https://spark.apache.org or request to leverage the original Spark
>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>> WDYT?
>>
>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>> Well that needs to be created first for this purpose. The appropriate
>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>> facilitate this as he offered his help.
>>
>>
>> cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>
>> Hello Mich,
>>
>> Can you please provide the link for the confluence page?
>>
>> Many thanks
>> Asma
>> Ph.D. in Big Data - Applied Machine Learning
>>
>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>> a écrit :
>>
>> Apologies I missed the list.
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>>
>> 1. Spark UI
>> 2. Dynamic allocation
>> 3. Tuning of jobs
>> 4. Collecting spark metrics for monitoring and alerting
>> 5. For those who prefer to use Pandas API on Spark since the release
>> of Spark 3.2, What are some important notes for those users? For example,
>> what are the additional factors affecting the Spark performance using
>> Pandas API on Spark? How to tune them in addition to the conventional Spark
>> tuning methods applied to Spark SQL users.
>> 6. Spark internals and/or comparing spark 3 and 2
>> 7. Spark Streaming & Spark Structured Streaming
>> 8. Spark on notebooks
>> 9. Spark on serverless (for example Spark on Google Cloud)
>> 10. Spark on k8s
>>
>> Opinions and how to is welcome
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi guys
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>> Opinions and how to is welcome
>>
>> Cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>>
>>
>>
Re: Topics for Spark online classes & webinars
Posted by Denny Lee <de...@gmail.com>.
What we can do is get into the habit of compiling the list on LinkedIn but
making sure this list is shared and broadcast here, eh?!
As well, when we broadcast the videos, we can do this using zoom/jitsi/
riverside.fm as well as simulcasting this on LinkedIn. This way you can
view directly on the former without ever logging in with a user ID.
HTH!!
On Wed, Mar 15, 2023 at 4:30 PM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Understood Nitin It would be wrong to act against one's conviction. I am
> sure we can find a way around providing the contents
>
> Regards
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
> wrote:
>
>> Hi Mich,
>>
>> Thanks for your prompt response ... much appreciated. I know how to and
>> can create login IDs on such sites but I had taken conscious decision some
>> 20 years ago ( and i will be going against my principles) not to be on such
>> sites. Hence I had asked for is there any other way I can join/view
>> recording of webinar.
>>
>> Anyways not to worry.
>>
>> Thanks & Regards
>>
>> Nitin.
>>
>>
>> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>
>> Hi Nitin,
>>
>> Linkedin is more of a professional media. FYI, I am only a member of
>> Linkedin, no facebook, etc.There is no reason for you NOT to create a
>> profile for yourself in linkedin :)
>>
>>
>> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>>
>> see you there as well.
>>
>> Best of luck.
>>
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead,
>> Palantir Technologies Limited
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
>> wrote:
>>
>> Hello Mich,
>>
>> My apologies ... but I am not on any of such social/professional sites?
>> Any other way to access such webinars/classes?
>>
>> Thanks & Regards
>> Nitin.
>>
>> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
>> denny.g.lee@gmail.com> wrote:
>>
>>
>> Thanks Mich for tackling this! I encourage everyone to add to the list
>> so we can have a comprehensive list of topics, eh?!
>>
>> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi all,
>>
>> Thanks to @Denny Lee <de...@gmail.com> to give access to
>>
>> https://www.linkedin.com/company/apachespark/
>>
>> and contribution from @asma zgolli <zg...@gmail.com>
>>
>> You will see my post at the bottom. Please add anything else on topics to
>> the list as a comment.
>>
>> We will then put them together in an article perhaps. Comments and
>> contributions are welcome.
>>
>> HTH
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead,
>> Palantir Technologies Limited
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi Denny,
>>
>> That Apache Spark Linkedin page
>> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
>> a wider audience to benefit from it.
>>
>> +1 for me
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>
>> In the past, we've been using the Apache Spark LinkedIn page
>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>> these type of events - if you're cool with this? Or we could go through
>> the process of submitting and updating the current
>> https://spark.apache.org or request to leverage the original Spark
>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>> WDYT?
>>
>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>> Well that needs to be created first for this purpose. The appropriate
>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>> facilitate this as he offered his help.
>>
>>
>> cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>
>> Hello Mich,
>>
>> Can you please provide the link for the confluence page?
>>
>> Many thanks
>> Asma
>> Ph.D. in Big Data - Applied Machine Learning
>>
>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>> a écrit :
>>
>> Apologies I missed the list.
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>>
>> 1. Spark UI
>> 2. Dynamic allocation
>> 3. Tuning of jobs
>> 4. Collecting spark metrics for monitoring and alerting
>> 5. For those who prefer to use Pandas API on Spark since the release
>> of Spark 3.2, What are some important notes for those users? For example,
>> what are the additional factors affecting the Spark performance using
>> Pandas API on Spark? How to tune them in addition to the conventional Spark
>> tuning methods applied to Spark SQL users.
>> 6. Spark internals and/or comparing spark 3 and 2
>> 7. Spark Streaming & Spark Structured Streaming
>> 8. Spark on notebooks
>> 9. Spark on serverless (for example Spark on Google Cloud)
>> 10. Spark on k8s
>>
>> Opinions and how to is welcome
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi guys
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>> Opinions and how to is welcome
>>
>> Cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>>
>>
>>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Understood Nitin It would be wrong to act against one's conviction. I am
sure we can find a way around providing the contents
Regards
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
wrote:
> Hi Mich,
>
> Thanks for your prompt response ... much appreciated. I know how to and
> can create login IDs on such sites but I had taken conscious decision some
> 20 years ago ( and i will be going against my principles) not to be on such
> sites. Hence I had asked for is there any other way I can join/view
> recording of webinar.
>
> Anyways not to worry.
>
> Thanks & Regards
>
> Nitin.
>
>
> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
> mich.talebzadeh@gmail.com> wrote:
>
>
> Hi Nitin,
>
> Linkedin is more of a professional media. FYI, I am only a member of
> Linkedin, no facebook, etc.There is no reason for you NOT to create a
> profile for yourself in linkedin :)
>
>
> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>
> see you there as well.
>
> Best of luck.
>
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
> wrote:
>
> Hello Mich,
>
> My apologies ... but I am not on any of such social/professional sites?
> Any other way to access such webinars/classes?
>
> Thanks & Regards
> Nitin.
>
> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
> denny.g.lee@gmail.com> wrote:
>
>
> Thanks Mich for tackling this! I encourage everyone to add to the list so
> we can have a comprehensive list of topics, eh?!
>
> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi all,
>
> Thanks to @Denny Lee <de...@gmail.com> to give access to
>
> https://www.linkedin.com/company/apachespark/
>
> and contribution from @asma zgolli <zg...@gmail.com>
>
> You will see my post at the bottom. Please add anything else on topics to
> the list as a comment.
>
> We will then put them together in an article perhaps. Comments and
> contributions are welcome.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi Denny,
>
> That Apache Spark Linkedin page
> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
> a wider audience to benefit from it.
>
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>
> In the past, we've been using the Apache Spark LinkedIn page
> <https://www.linkedin.com/company/apachespark/> and group to broadcast
> these type of events - if you're cool with this? Or we could go through
> the process of submitting and updating the current
> https://spark.apache.org or request to leverage the original Spark
> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
> WDYT?
>
> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Well that needs to be created first for this purpose. The appropriate name
> etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
> facilitate this as he offered his help.
>
>
> cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>
> Hello Mich,
>
> Can you please provide the link for the confluence page?
>
> Many thanks
> Asma
> Ph.D. in Big Data - Applied Machine Learning
>
> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
> a écrit :
>
> Apologies I missed the list.
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
>
> 1. Spark UI
> 2. Dynamic allocation
> 3. Tuning of jobs
> 4. Collecting spark metrics for monitoring and alerting
> 5. For those who prefer to use Pandas API on Spark since the release
> of Spark 3.2, What are some important notes for those users? For example,
> what are the additional factors affecting the Spark performance using
> Pandas API on Spark? How to tune them in addition to the conventional Spark
> tuning methods applied to Spark SQL users.
> 6. Spark internals and/or comparing spark 3 and 2
> 7. Spark Streaming & Spark Structured Streaming
> 8. Spark on notebooks
> 9. Spark on serverless (for example Spark on Google Cloud)
> 10. Spark on k8s
>
> Opinions and how to is welcome
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi guys
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
> Opinions and how to is welcome
>
> Cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
>
>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Understood Nitin It would be wrong to act against one's conviction. I am
sure we can find a way around providing the contents
Regards
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Wed, 15 Mar 2023 at 22:34, Nitin Bhansali <ni...@yahoo.com>
wrote:
> Hi Mich,
>
> Thanks for your prompt response ... much appreciated. I know how to and
> can create login IDs on such sites but I had taken conscious decision some
> 20 years ago ( and i will be going against my principles) not to be on such
> sites. Hence I had asked for is there any other way I can join/view
> recording of webinar.
>
> Anyways not to worry.
>
> Thanks & Regards
>
> Nitin.
>
>
> On Wednesday, 15 March 2023 at 20:37:55 GMT, Mich Talebzadeh <
> mich.talebzadeh@gmail.com> wrote:
>
>
> Hi Nitin,
>
> Linkedin is more of a professional media. FYI, I am only a member of
> Linkedin, no facebook, etc.There is no reason for you NOT to create a
> profile for yourself in linkedin :)
>
>
> https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
>
> see you there as well.
>
> Best of luck.
>
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
> wrote:
>
> Hello Mich,
>
> My apologies ... but I am not on any of such social/professional sites?
> Any other way to access such webinars/classes?
>
> Thanks & Regards
> Nitin.
>
> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
> denny.g.lee@gmail.com> wrote:
>
>
> Thanks Mich for tackling this! I encourage everyone to add to the list so
> we can have a comprehensive list of topics, eh?!
>
> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi all,
>
> Thanks to @Denny Lee <de...@gmail.com> to give access to
>
> https://www.linkedin.com/company/apachespark/
>
> and contribution from @asma zgolli <zg...@gmail.com>
>
> You will see my post at the bottom. Please add anything else on topics to
> the list as a comment.
>
> We will then put them together in an article perhaps. Comments and
> contributions are welcome.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi Denny,
>
> That Apache Spark Linkedin page
> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
> a wider audience to benefit from it.
>
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>
> In the past, we've been using the Apache Spark LinkedIn page
> <https://www.linkedin.com/company/apachespark/> and group to broadcast
> these type of events - if you're cool with this? Or we could go through
> the process of submitting and updating the current
> https://spark.apache.org or request to leverage the original Spark
> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
> WDYT?
>
> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Well that needs to be created first for this purpose. The appropriate name
> etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
> facilitate this as he offered his help.
>
>
> cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>
> Hello Mich,
>
> Can you please provide the link for the confluence page?
>
> Many thanks
> Asma
> Ph.D. in Big Data - Applied Machine Learning
>
> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
> a écrit :
>
> Apologies I missed the list.
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
>
> 1. Spark UI
> 2. Dynamic allocation
> 3. Tuning of jobs
> 4. Collecting spark metrics for monitoring and alerting
> 5. For those who prefer to use Pandas API on Spark since the release
> of Spark 3.2, What are some important notes for those users? For example,
> what are the additional factors affecting the Spark performance using
> Pandas API on Spark? How to tune them in addition to the conventional Spark
> tuning methods applied to Spark SQL users.
> 6. Spark internals and/or comparing spark 3 and 2
> 7. Spark Streaming & Spark Structured Streaming
> 8. Spark on notebooks
> 9. Spark on serverless (for example Spark on Google Cloud)
> 10. Spark on k8s
>
> Opinions and how to is welcome
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi guys
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
> Opinions and how to is welcome
>
> Cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
>
>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi Nitin,
Linkedin is more of a professional media. FYI, I am only a member of
Linkedin, no facebook, etc.There is no reason for you NOT to create a
profile for yourself in linkedin :)
https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
see you there as well.
Best of luck.
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead,
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
wrote:
> Hello Mich,
>
> My apologies ... but I am not on any of such social/professional sites?
> Any other way to access such webinars/classes?
>
> Thanks & Regards
> Nitin.
>
> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
> denny.g.lee@gmail.com> wrote:
>
>
> Thanks Mich for tackling this! I encourage everyone to add to the list so
> we can have a comprehensive list of topics, eh?!
>
> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi all,
>
> Thanks to @Denny Lee <de...@gmail.com> to give access to
>
> https://www.linkedin.com/company/apachespark/
>
> and contribution from @asma zgolli <zg...@gmail.com>
>
> You will see my post at the bottom. Please add anything else on topics to
> the list as a comment.
>
> We will then put them together in an article perhaps. Comments and
> contributions are welcome.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi Denny,
>
> That Apache Spark Linkedin page
> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
> a wider audience to benefit from it.
>
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>
> In the past, we've been using the Apache Spark LinkedIn page
> <https://www.linkedin.com/company/apachespark/> and group to broadcast
> these type of events - if you're cool with this? Or we could go through
> the process of submitting and updating the current
> https://spark.apache.org or request to leverage the original Spark
> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
> WDYT?
>
> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Well that needs to be created first for this purpose. The appropriate name
> etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
> facilitate this as he offered his help.
>
>
> cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>
> Hello Mich,
>
> Can you please provide the link for the confluence page?
>
> Many thanks
> Asma
> Ph.D. in Big Data - Applied Machine Learning
>
> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
> a écrit :
>
> Apologies I missed the list.
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
>
> 1. Spark UI
> 2. Dynamic allocation
> 3. Tuning of jobs
> 4. Collecting spark metrics for monitoring and alerting
> 5. For those who prefer to use Pandas API on Spark since the release
> of Spark 3.2, What are some important notes for those users? For example,
> what are the additional factors affecting the Spark performance using
> Pandas API on Spark? How to tune them in addition to the conventional Spark
> tuning methods applied to Spark SQL users.
> 6. Spark internals and/or comparing spark 3 and 2
> 7. Spark Streaming & Spark Structured Streaming
> 8. Spark on notebooks
> 9. Spark on serverless (for example Spark on Google Cloud)
> 10. Spark on k8s
>
> Opinions and how to is welcome
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi guys
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
> Opinions and how to is welcome
>
> Cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
>
>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi Nitin,
Linkedin is more of a professional media. FYI, I am only a member of
Linkedin, no facebook, etc.There is no reason for you NOT to create a
profile for yourself in linkedin :)
https://www.linkedin.com/help/linkedin/answer/a1338223/sign-up-to-join-linkedin?lang=en
see you there as well.
Best of luck.
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead,
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Wed, 15 Mar 2023 at 18:31, Nitin Bhansali <ni...@yahoo.com>
wrote:
> Hello Mich,
>
> My apologies ... but I am not on any of such social/professional sites?
> Any other way to access such webinars/classes?
>
> Thanks & Regards
> Nitin.
>
> On Wednesday, 15 March 2023 at 18:26:51 GMT, Denny Lee <
> denny.g.lee@gmail.com> wrote:
>
>
> Thanks Mich for tackling this! I encourage everyone to add to the list so
> we can have a comprehensive list of topics, eh?!
>
> On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi all,
>
> Thanks to @Denny Lee <de...@gmail.com> to give access to
>
> https://www.linkedin.com/company/apachespark/
>
> and contribution from @asma zgolli <zg...@gmail.com>
>
> You will see my post at the bottom. Please add anything else on topics to
> the list as a comment.
>
> We will then put them together in an article perhaps. Comments and
> contributions are welcome.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi Denny,
>
> That Apache Spark Linkedin page
> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
> a wider audience to benefit from it.
>
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>
> In the past, we've been using the Apache Spark LinkedIn page
> <https://www.linkedin.com/company/apachespark/> and group to broadcast
> these type of events - if you're cool with this? Or we could go through
> the process of submitting and updating the current
> https://spark.apache.org or request to leverage the original Spark
> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
> WDYT?
>
> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Well that needs to be created first for this purpose. The appropriate name
> etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
> facilitate this as he offered his help.
>
>
> cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>
> Hello Mich,
>
> Can you please provide the link for the confluence page?
>
> Many thanks
> Asma
> Ph.D. in Big Data - Applied Machine Learning
>
> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
> a écrit :
>
> Apologies I missed the list.
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
>
> 1. Spark UI
> 2. Dynamic allocation
> 3. Tuning of jobs
> 4. Collecting spark metrics for monitoring and alerting
> 5. For those who prefer to use Pandas API on Spark since the release
> of Spark 3.2, What are some important notes for those users? For example,
> what are the additional factors affecting the Spark performance using
> Pandas API on Spark? How to tune them in addition to the conventional Spark
> tuning methods applied to Spark SQL users.
> 6. Spark internals and/or comparing spark 3 and 2
> 7. Spark Streaming & Spark Structured Streaming
> 8. Spark on notebooks
> 9. Spark on serverless (for example Spark on Google Cloud)
> 10. Spark on k8s
>
> Opinions and how to is welcome
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi guys
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
> Opinions and how to is welcome
>
> Cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
>
>
>
Re: Topics for Spark online classes & webinars
Posted by Denny Lee <de...@gmail.com>.
Thanks Mich for tackling this! I encourage everyone to add to the list so
we can have a comprehensive list of topics, eh?!
On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi all,
>
> Thanks to @Denny Lee <de...@gmail.com> to give access to
>
> https://www.linkedin.com/company/apachespark/
>
> and contribution from @asma zgolli <zg...@gmail.com>
>
> You will see my post at the bottom. Please add anything else on topics to
> the list as a comment.
>
> We will then put them together in an article perhaps. Comments and
> contributions are welcome.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Hi Denny,
>>
>> That Apache Spark Linkedin page
>> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
>> a wider audience to benefit from it.
>>
>> +1 for me
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>
>>> In the past, we've been using the Apache Spark LinkedIn page
>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>> these type of events - if you're cool with this? Or we could go through
>>> the process of submitting and updating the current
>>> https://spark.apache.org or request to leverage the original Spark
>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>> WDYT?
>>>
>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Well that needs to be created first for this purpose. The appropriate
>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>>> facilitate this as he offered his help.
>>>>
>>>>
>>>> cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>>
>>>>> Hello Mich,
>>>>>
>>>>> Can you please provide the link for the confluence page?
>>>>>
>>>>> Many thanks
>>>>> Asma
>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>
>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>
>>>>>> Apologies I missed the list.
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>>
>>>>>> 1. Spark UI
>>>>>> 2. Dynamic allocation
>>>>>> 3. Tuning of jobs
>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>> 8. Spark on notebooks
>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>> 10. Spark on k8s
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>>> Hi guys
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
Re: Topics for Spark online classes & webinars
Posted by Denny Lee <de...@gmail.com>.
Thanks Mich for tackling this! I encourage everyone to add to the list so
we can have a comprehensive list of topics, eh?!
On Wed, Mar 15, 2023 at 10:27 Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi all,
>
> Thanks to @Denny Lee <de...@gmail.com> to give access to
>
> https://www.linkedin.com/company/apachespark/
>
> and contribution from @asma zgolli <zg...@gmail.com>
>
> You will see my post at the bottom. Please add anything else on topics to
> the list as a comment.
>
> We will then put them together in an article perhaps. Comments and
> contributions are welcome.
>
> HTH
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead,
> Palantir Technologies Limited
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Hi Denny,
>>
>> That Apache Spark Linkedin page
>> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
>> a wider audience to benefit from it.
>>
>> +1 for me
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>>
>>> In the past, we've been using the Apache Spark LinkedIn page
>>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>>> these type of events - if you're cool with this? Or we could go through
>>> the process of submitting and updating the current
>>> https://spark.apache.org or request to leverage the original Spark
>>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>>> WDYT?
>>>
>>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Well that needs to be created first for this purpose. The appropriate
>>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>>> facilitate this as he offered his help.
>>>>
>>>>
>>>> cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>>
>>>>> Hello Mich,
>>>>>
>>>>> Can you please provide the link for the confluence page?
>>>>>
>>>>> Many thanks
>>>>> Asma
>>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>>
>>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>>
>>>>>> Apologies I missed the list.
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>>
>>>>>> 1. Spark UI
>>>>>> 2. Dynamic allocation
>>>>>> 3. Tuning of jobs
>>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>>> example, what are the additional factors affecting the Spark performance
>>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>>> Spark tuning methods applied to Spark SQL users.
>>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>>> 8. Spark on notebooks
>>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>>> 10. Spark on k8s
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>>
>>>>>>> Hi guys
>>>>>>>
>>>>>>> To move forward I selected these topics from the thread "Online
>>>>>>> classes for spark topics".
>>>>>>>
>>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>>
>>>>>>> Opinions and how to is welcome
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> view my Linkedin profile
>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>>
>>>>>>>
>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>>> arise from relying on this email's technical content is explicitly
>>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>>> arising from such loss, damage or destruction.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi all,
Thanks to @Denny Lee <de...@gmail.com> to give access to
https://www.linkedin.com/company/apachespark/
and contribution from @asma zgolli <zg...@gmail.com>
You will see my post at the bottom. Please add anything else on topics to
the list as a comment.
We will then put them together in an article perhaps. Comments and
contributions are welcome.
HTH
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead,
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi Denny,
>
> That Apache Spark Linkedin page
> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
> a wider audience to benefit from it.
>
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>
>> In the past, we've been using the Apache Spark LinkedIn page
>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>> these type of events - if you're cool with this? Or we could go through
>> the process of submitting and updating the current
>> https://spark.apache.org or request to leverage the original Spark
>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>> WDYT?
>>
>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Well that needs to be created first for this purpose. The appropriate
>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>> facilitate this as he offered his help.
>>>
>>>
>>> cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>
>>>> Hello Mich,
>>>>
>>>> Can you please provide the link for the confluence page?
>>>>
>>>> Many thanks
>>>> Asma
>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>
>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>
>>>>> Apologies I missed the list.
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>>
>>>>> 1. Spark UI
>>>>> 2. Dynamic allocation
>>>>> 3. Tuning of jobs
>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>> example, what are the additional factors affecting the Spark performance
>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>> Spark tuning methods applied to Spark SQL users.
>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>> 8. Spark on notebooks
>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>> 10. Spark on k8s
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> Hi guys
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi all,
Thanks to @Denny Lee <de...@gmail.com> to give access to
https://www.linkedin.com/company/apachespark/
and contribution from @asma zgolli <zg...@gmail.com>
You will see my post at the bottom. Please add anything else on topics to
the list as a comment.
We will then put them together in an article perhaps. Comments and
contributions are welcome.
HTH
Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead,
Palantir Technologies Limited
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 14 Mar 2023 at 15:09, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi Denny,
>
> That Apache Spark Linkedin page
> https://www.linkedin.com/company/apachespark/ looks fine. It also allows
> a wider audience to benefit from it.
>
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
>
>> In the past, we've been using the Apache Spark LinkedIn page
>> <https://www.linkedin.com/company/apachespark/> and group to broadcast
>> these type of events - if you're cool with this? Or we could go through
>> the process of submitting and updating the current
>> https://spark.apache.org or request to leverage the original Spark
>> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
>> WDYT?
>>
>> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Well that needs to be created first for this purpose. The appropriate
>>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>>> facilitate this as he offered his help.
>>>
>>>
>>> cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>>
>>>> Hello Mich,
>>>>
>>>> Can you please provide the link for the confluence page?
>>>>
>>>> Many thanks
>>>> Asma
>>>> Ph.D. in Big Data - Applied Machine Learning
>>>>
>>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> a écrit :
>>>>
>>>>> Apologies I missed the list.
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>>
>>>>> 1. Spark UI
>>>>> 2. Dynamic allocation
>>>>> 3. Tuning of jobs
>>>>> 4. Collecting spark metrics for monitoring and alerting
>>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>>> release of Spark 3.2, What are some important notes for those users? For
>>>>> example, what are the additional factors affecting the Spark performance
>>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>>> Spark tuning methods applied to Spark SQL users.
>>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>>> 7. Spark Streaming & Spark Structured Streaming
>>>>> 8. Spark on notebooks
>>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>>> 10. Spark on k8s
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>>> mich.talebzadeh@gmail.com> wrote:
>>>>>
>>>>>> Hi guys
>>>>>>
>>>>>> To move forward I selected these topics from the thread "Online
>>>>>> classes for spark topics".
>>>>>>
>>>>>> To take this further I propose a confluence page to be seup.
>>>>>>
>>>>>> Opinions and how to is welcome
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction of data or any other property which may
>>>>>> arise from relying on this email's technical content is explicitly
>>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>>> arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi Denny,
That Apache Spark Linkedin page
https://www.linkedin.com/company/apachespark/ looks fine. It also allows a
wider audience to benefit from it.
+1 for me
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
> In the past, we've been using the Apache Spark LinkedIn page
> <https://www.linkedin.com/company/apachespark/> and group to broadcast
> these type of events - if you're cool with this? Or we could go through
> the process of submitting and updating the current
> https://spark.apache.org or request to leverage the original Spark
> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
> WDYT?
>
> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Well that needs to be created first for this purpose. The appropriate
>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>> facilitate this as he offered his help.
>>
>>
>> cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>
>>> Hello Mich,
>>>
>>> Can you please provide the link for the confluence page?
>>>
>>> Many thanks
>>> Asma
>>> Ph.D. in Big Data - Applied Machine Learning
>>>
>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>>> a écrit :
>>>
>>>> Apologies I missed the list.
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>>
>>>> 1. Spark UI
>>>> 2. Dynamic allocation
>>>> 3. Tuning of jobs
>>>> 4. Collecting spark metrics for monitoring and alerting
>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>> release of Spark 3.2, What are some important notes for those users? For
>>>> example, what are the additional factors affecting the Spark performance
>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>> Spark tuning methods applied to Spark SQL users.
>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>> 7. Spark Streaming & Spark Structured Streaming
>>>> 8. Spark on notebooks
>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>> 10. Spark on k8s
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Hi guys
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi Denny,
That Apache Spark Linkedin page
https://www.linkedin.com/company/apachespark/ looks fine. It also allows a
wider audience to benefit from it.
+1 for me
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 14 Mar 2023 at 14:23, Denny Lee <de...@gmail.com> wrote:
> In the past, we've been using the Apache Spark LinkedIn page
> <https://www.linkedin.com/company/apachespark/> and group to broadcast
> these type of events - if you're cool with this? Or we could go through
> the process of submitting and updating the current
> https://spark.apache.org or request to leverage the original Spark
> confluence page <https://cwiki.apache.org/confluence/display/SPARK>.
> WDYT?
>
> On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Well that needs to be created first for this purpose. The appropriate
>> name etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
>> facilitate this as he offered his help.
>>
>>
>> cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>>
>>> Hello Mich,
>>>
>>> Can you please provide the link for the confluence page?
>>>
>>> Many thanks
>>> Asma
>>> Ph.D. in Big Data - Applied Machine Learning
>>>
>>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>>> a écrit :
>>>
>>>> Apologies I missed the list.
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>>
>>>> 1. Spark UI
>>>> 2. Dynamic allocation
>>>> 3. Tuning of jobs
>>>> 4. Collecting spark metrics for monitoring and alerting
>>>> 5. For those who prefer to use Pandas API on Spark since the
>>>> release of Spark 3.2, What are some important notes for those users? For
>>>> example, what are the additional factors affecting the Spark performance
>>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>>> Spark tuning methods applied to Spark SQL users.
>>>> 6. Spark internals and/or comparing spark 3 and 2
>>>> 7. Spark Streaming & Spark Structured Streaming
>>>> 8. Spark on notebooks
>>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>>> 10. Spark on k8s
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Hi guys
>>>>>
>>>>> To move forward I selected these topics from the thread "Online
>>>>> classes for spark topics".
>>>>>
>>>>> To take this further I propose a confluence page to be seup.
>>>>>
>>>>> Opinions and how to is welcome
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
Re: Topics for Spark online classes & webinars
Posted by Denny Lee <de...@gmail.com>.
In the past, we've been using the Apache Spark LinkedIn page
<https://www.linkedin.com/company/apachespark/> and group to broadcast
these type of events - if you're cool with this? Or we could go through
the process of submitting and updating the current https://spark.apache.org
or request to leverage the original Spark confluence page
<https://cwiki.apache.org/confluence/display/SPARK>. WDYT?
On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Well that needs to be created first for this purpose. The appropriate name
> etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
> facilitate this as he offered his help.
>
>
> cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>
>> Hello Mich,
>>
>> Can you please provide the link for the confluence page?
>>
>> Many thanks
>> Asma
>> Ph.D. in Big Data - Applied Machine Learning
>>
>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>> a écrit :
>>
>>> Apologies I missed the list.
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>>
>>> 1. Spark UI
>>> 2. Dynamic allocation
>>> 3. Tuning of jobs
>>> 4. Collecting spark metrics for monitoring and alerting
>>> 5. For those who prefer to use Pandas API on Spark since the
>>> release of Spark 3.2, What are some important notes for those users? For
>>> example, what are the additional factors affecting the Spark performance
>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>> Spark tuning methods applied to Spark SQL users.
>>> 6. Spark internals and/or comparing spark 3 and 2
>>> 7. Spark Streaming & Spark Structured Streaming
>>> 8. Spark on notebooks
>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>> 10. Spark on k8s
>>>
>>> Opinions and how to is welcome
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>>> Hi guys
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>
>>
>>
>>
Re: Topics for Spark online classes & webinars
Posted by Denny Lee <de...@gmail.com>.
In the past, we've been using the Apache Spark LinkedIn page
<https://www.linkedin.com/company/apachespark/> and group to broadcast
these type of events - if you're cool with this? Or we could go through
the process of submitting and updating the current https://spark.apache.org
or request to leverage the original Spark confluence page
<https://cwiki.apache.org/confluence/display/SPARK>. WDYT?
On Mon, Mar 13, 2023 at 9:34 AM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Well that needs to be created first for this purpose. The appropriate name
> etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
> facilitate this as he offered his help.
>
>
> cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
>
>> Hello Mich,
>>
>> Can you please provide the link for the confluence page?
>>
>> Many thanks
>> Asma
>> Ph.D. in Big Data - Applied Machine Learning
>>
>> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
>> a écrit :
>>
>>> Apologies I missed the list.
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>>
>>> 1. Spark UI
>>> 2. Dynamic allocation
>>> 3. Tuning of jobs
>>> 4. Collecting spark metrics for monitoring and alerting
>>> 5. For those who prefer to use Pandas API on Spark since the
>>> release of Spark 3.2, What are some important notes for those users? For
>>> example, what are the additional factors affecting the Spark performance
>>> using Pandas API on Spark? How to tune them in addition to the conventional
>>> Spark tuning methods applied to Spark SQL users.
>>> 6. Spark internals and/or comparing spark 3 and 2
>>> 7. Spark Streaming & Spark Structured Streaming
>>> 8. Spark on notebooks
>>> 9. Spark on serverless (for example Spark on Google Cloud)
>>> 10. Spark on k8s
>>>
>>> Opinions and how to is welcome
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>>> Hi guys
>>>>
>>>> To move forward I selected these topics from the thread "Online classes
>>>> for spark topics".
>>>>
>>>> To take this further I propose a confluence page to be seup.
>>>>
>>>> Opinions and how to is welcome
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>
>>
>>
>>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Well that needs to be created first for this purpose. The appropriate name
etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
facilitate this as he offered his help.
cheers
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
> Hello Mich,
>
> Can you please provide the link for the confluence page?
>
> Many thanks
> Asma
> Ph.D. in Big Data - Applied Machine Learning
>
> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
> a écrit :
>
>> Apologies I missed the list.
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>>
>> 1. Spark UI
>> 2. Dynamic allocation
>> 3. Tuning of jobs
>> 4. Collecting spark metrics for monitoring and alerting
>> 5. For those who prefer to use Pandas API on Spark since the release
>> of Spark 3.2, What are some important notes for those users? For example,
>> what are the additional factors affecting the Spark performance using
>> Pandas API on Spark? How to tune them in addition to the conventional Spark
>> tuning methods applied to Spark SQL users.
>> 6. Spark internals and/or comparing spark 3 and 2
>> 7. Spark Streaming & Spark Structured Streaming
>> 8. Spark on notebooks
>> 9. Spark on serverless (for example Spark on Google Cloud)
>> 10. Spark on k8s
>>
>> Opinions and how to is welcome
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>>> Hi guys
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>> Opinions and how to is welcome
>>>
>>> Cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>
>
>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Well that needs to be created first for this purpose. The appropriate name
etc. to be decided. Maybe @Denny Lee <de...@gmail.com> can
facilitate this as he offered his help.
cheers
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Mon, 13 Mar 2023 at 16:29, asma zgolli <zg...@gmail.com> wrote:
> Hello Mich,
>
> Can you please provide the link for the confluence page?
>
> Many thanks
> Asma
> Ph.D. in Big Data - Applied Machine Learning
>
> Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>
> a écrit :
>
>> Apologies I missed the list.
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>>
>> 1. Spark UI
>> 2. Dynamic allocation
>> 3. Tuning of jobs
>> 4. Collecting spark metrics for monitoring and alerting
>> 5. For those who prefer to use Pandas API on Spark since the release
>> of Spark 3.2, What are some important notes for those users? For example,
>> what are the additional factors affecting the Spark performance using
>> Pandas API on Spark? How to tune them in addition to the conventional Spark
>> tuning methods applied to Spark SQL users.
>> 6. Spark internals and/or comparing spark 3 and 2
>> 7. Spark Streaming & Spark Structured Streaming
>> 8. Spark on notebooks
>> 9. Spark on serverless (for example Spark on Google Cloud)
>> 10. Spark on k8s
>>
>> Opinions and how to is welcome
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>>> Hi guys
>>>
>>> To move forward I selected these topics from the thread "Online classes
>>> for spark topics".
>>>
>>> To take this further I propose a confluence page to be seup.
>>>
>>> Opinions and how to is welcome
>>>
>>> Cheers
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>
>
>
>
Re: Topics for Spark online classes & webinars
Posted by "Suttraway, Manisha" <su...@amazon.com.INVALID>.
+! For these classes,
Please share confluence page link.
-Manisha
From: asma zgolli <zg...@gmail.com>
Date: Monday 13 March 2023 at 16:30
To: Mich Talebzadeh <mi...@gmail.com>
Cc: "user @spark" <us...@spark.apache.org>, Spark dev list <de...@spark.apache.org>
Subject: RE: [EXTERNAL]Topics for Spark online classes & webinars
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hello Mich,
Can you please provide the link for the confluence page?
Many thanks
Asma
Ph.D. in Big Data - Applied Machine Learning
Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com>> a écrit :
Apologies I missed the list.
To move forward I selected these topics from the thread "Online classes for spark topics".
To take this further I propose a confluence page to be seup.
1. Spark UI
2. Dynamic allocation
3. Tuning of jobs
4. Collecting spark metrics for monitoring and alerting
5. For those who prefer to use Pandas API on Spark since the release of Spark 3.2, What are some important notes for those users? For example, what are the additional factors affecting the Spark performance using Pandas API on Spark? How to tune them in addition to the conventional Spark tuning methods applied to Spark SQL users.
6. Spark internals and/or comparing spark 3 and 2
7. Spark Streaming & Spark Structured Streaming
8. Spark on notebooks
9. Spark on serverless (for example Spark on Google Cloud)
10. Spark on k8s
Opinions and how to is welcome
[Image removed by sender.] view my Linkedin profile<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>> wrote:
Hi guys
To move forward I selected these topics from the thread "Online classes for spark topics".
To take this further I propose a confluence page to be seup.
Opinions and how to is welcome
Cheers
[Image removed by sender.] view my Linkedin profile<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
Re: Topics for Spark online classes & webinars
Posted by asma zgolli <zg...@gmail.com>.
Hello Mich,
Can you please provide the link for the confluence page?
Many thanks
Asma
Ph.D. in Big Data - Applied Machine Learning
Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com> a
écrit :
> Apologies I missed the list.
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
>
> 1. Spark UI
> 2. Dynamic allocation
> 3. Tuning of jobs
> 4. Collecting spark metrics for monitoring and alerting
> 5. For those who prefer to use Pandas API on Spark since the release
> of Spark 3.2, What are some important notes for those users? For example,
> what are the additional factors affecting the Spark performance using
> Pandas API on Spark? How to tune them in addition to the conventional Spark
> tuning methods applied to Spark SQL users.
> 6. Spark internals and/or comparing spark 3 and 2
> 7. Spark Streaming & Spark Structured Streaming
> 8. Spark on notebooks
> 9. Spark on serverless (for example Spark on Google Cloud)
> 10. Spark on k8s
>
> Opinions and how to is welcome
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Hi guys
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>> Opinions and how to is welcome
>>
>> Cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>
Re: Topics for Spark online classes & webinars
Posted by asma zgolli <zg...@gmail.com>.
Hello Mich,
Can you please provide the link for the confluence page?
Many thanks
Asma
Ph.D. in Big Data - Applied Machine Learning
Le lun. 13 mars 2023 à 17:21, Mich Talebzadeh <mi...@gmail.com> a
écrit :
> Apologies I missed the list.
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
>
> 1. Spark UI
> 2. Dynamic allocation
> 3. Tuning of jobs
> 4. Collecting spark metrics for monitoring and alerting
> 5. For those who prefer to use Pandas API on Spark since the release
> of Spark 3.2, What are some important notes for those users? For example,
> what are the additional factors affecting the Spark performance using
> Pandas API on Spark? How to tune them in addition to the conventional Spark
> tuning methods applied to Spark SQL users.
> 6. Spark internals and/or comparing spark 3 and 2
> 7. Spark Streaming & Spark Structured Streaming
> 8. Spark on notebooks
> 9. Spark on serverless (for example Spark on Google Cloud)
> 10. Spark on k8s
>
> Opinions and how to is welcome
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> Hi guys
>>
>> To move forward I selected these topics from the thread "Online classes
>> for spark topics".
>>
>> To take this further I propose a confluence page to be seup.
>>
>> Opinions and how to is welcome
>>
>> Cheers
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>
Re: Topics for Spark online classes & webinars
Posted by Joris Billen <jo...@bigindustries.be>.
This is a very good idea-would love to read such a confluence page.
Adding a section “common mistakes/misconceptions” might be useful for many of these sections. It would describe undesired behaviour/errors one would get in case of not following some best practices.
On 13 Mar 2023, at 17:20, Mich Talebzadeh <mi...@gmail.com> wrote:
Apologies I missed the list.
To move forward I selected these topics from the thread "Online classes for spark topics".
To take this further I propose a confluence page to be seup.
1. Spark UI
2. Dynamic allocation
3. Tuning of jobs
4. Collecting spark metrics for monitoring and alerting
5. For those who prefer to use Pandas API on Spark since the release of Spark 3.2, What are some important notes for those users? For example, what are the additional factors affecting the Spark performance using Pandas API on Spark? How to tune them in addition to the conventional Spark tuning methods applied to Spark SQL users.
6. Spark internals and/or comparing spark 3 and 2
7. Spark Streaming & Spark Structured Streaming
8. Spark on notebooks
9. Spark on serverless (for example Spark on Google Cloud)
10. Spark on k8s
Opinions and how to is welcome
[https://ci3.googleusercontent.com/mail-sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE] view my Linkedin profile<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmich-talebzadeh-ph-d-5205b2%2F&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181129126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZP0m80Q1NGAMFzD4FFIW8kC%2FUaInAZwRM9HVV1UoJtE%3D&reserved=0>
https://en.everybodywiki.com/Mich_Talebzadeh<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.everybodywiki.com%2FMich_Talebzadeh&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181129126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LOs7tXiyHRNXc0ViAFRu13ltXXjzob7lENq0MvDDOxs%3D&reserved=0>
Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>> wrote:
Hi guys
To move forward I selected these topics from the thread "Online classes for spark topics".
To take this further I propose a confluence page to be seup.
Opinions and how to is welcome
Cheers
[https://ci3.googleusercontent.com/mail-sig/AIorK4zholKucR2Q9yMrKbHNn-o1TuS4mYXyi2KO6Xmx6ikHPySa9MLaLZ8t2hrA6AUcxSxDgHIwmKE] view my Linkedin profile<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmich-talebzadeh-ph-d-5205b2%2F&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181285354%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2t5OqcDn7%2B8Tv%2FBtuRGmEzPyJwbVcKaIuKZ4dzsigHs%3D&reserved=0>
https://en.everybodywiki.com/Mich_Talebzadeh<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.everybodywiki.com%2FMich_Talebzadeh&data=05%7C01%7Cjoris.billen%40bigindustries.be%7C33df48307f1049c3932508db23df1022%7C49c3d703357947bfa8887c913fbdced9%7C0%7C0%7C638143213181285354%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Zk11t0gp%2Fl3iOgx3dvKWMl5OniuErrrsxCPCVM6L9JM%3D&reserved=0>
Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Apologies I missed the list.
To move forward I selected these topics from the thread "Online classes for
spark topics".
To take this further I propose a confluence page to be seup.
1. Spark UI
2. Dynamic allocation
3. Tuning of jobs
4. Collecting spark metrics for monitoring and alerting
5. For those who prefer to use Pandas API on Spark since the release of
Spark 3.2, What are some important notes for those users? For example, what
are the additional factors affecting the Spark performance using Pandas API
on Spark? How to tune them in addition to the conventional Spark tuning
methods applied to Spark SQL users.
6. Spark internals and/or comparing spark 3 and 2
7. Spark Streaming & Spark Structured Streaming
8. Spark on notebooks
9. Spark on serverless (for example Spark on Google Cloud)
10. Spark on k8s
Opinions and how to is welcome
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi guys
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
> Opinions and how to is welcome
>
> Cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
Re: Topics for Spark online classes & webinars
Posted by Mich Talebzadeh <mi...@gmail.com>.
Apologies I missed the list.
To move forward I selected these topics from the thread "Online classes for
spark topics".
To take this further I propose a confluence page to be seup.
1. Spark UI
2. Dynamic allocation
3. Tuning of jobs
4. Collecting spark metrics for monitoring and alerting
5. For those who prefer to use Pandas API on Spark since the release of
Spark 3.2, What are some important notes for those users? For example, what
are the additional factors affecting the Spark performance using Pandas API
on Spark? How to tune them in addition to the conventional Spark tuning
methods applied to Spark SQL users.
6. Spark internals and/or comparing spark 3 and 2
7. Spark Streaming & Spark Structured Streaming
8. Spark on notebooks
9. Spark on serverless (for example Spark on Google Cloud)
10. Spark on k8s
Opinions and how to is welcome
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Mon, 13 Mar 2023 at 16:16, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Hi guys
>
> To move forward I selected these topics from the thread "Online classes
> for spark topics".
>
> To take this further I propose a confluence page to be seup.
>
> Opinions and how to is welcome
>
> Cheers
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>