You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Holden Karau <ho...@gmail.com> on 2024/12/07 06:19:08 UTC

Increasing Shading & Relocating for 4.0

Hi Y'all,

As we're getting closer to 4.0 I was thinking now is a good time for us to
try and reduce the class path we expose for JVM users. Are there any common
classes/packages folks would like to see shaded?

Cheers,

Holden :)

-- 
Twitter: https://twitter.com/holdenkarau
Fight Health Insurance: https://www.fighthealthinsurance.com/
<https://www.fighthealthinsurance.com/?q=hk_email>
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Pronouns: she/her

Re: Increasing Shading & Relocating for 4.0

Posted by Mich Talebzadeh <mi...@gmail.com>.
General comment without specifics. I think shading should be used* on a
case by case basis* when the benefits outweigh the drawbacks. How about
exploring alternatives such as modularization, dependency management, or
careful dependency selection, before resorting to shading? My point is that
shading will introduce more debugging and testing as packages will be
renamed impacting flexibility. Case in point, things like unit and
integration tests may need adjustments to account for the renamed packages.

HTH

Mich Talebzadeh,

Architect | Data Science | Financial Crime | GDPR & Compliance Specialist
PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial College
London <https://en.wikipedia.org/wiki/Imperial_College_London>
London, United Kingdom


   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>


 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* The information provided is correct to the best of my
knowledge but of course cannot be guaranteed . It is essential to note
that, as with any advice, quote "one test result is worth one-thousand
expert opinions (Werner  <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von
Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)".


On Sat, 7 Dec 2024 at 06:21, Holden Karau <ho...@gmail.com> wrote:

> Hi Y'all,
>
> As we're getting closer to 4.0 I was thinking now is a good time for us to
> try and reduce the class path we expose for JVM users. Are there any common
> classes/packages folks would like to see shaded?
>
> Cheers,
>
> Holden :)
>
> --
> Twitter: https://twitter.com/holdenkarau
> Fight Health Insurance: https://www.fighthealthinsurance.com/
> <https://www.fighthealthinsurance.com/?q=hk_email>
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
> Pronouns: she/her
>