[GitHub] HyukjinKwon edited a comment on issue #23715: [SPARK-26803][PYTHON] Add sbin subdirectory to pyspark

[GitBox <gi...@apache.org>]

HyukjinKwon edited a comment on issue #23715: [SPARK-26803][PYTHON] Add sbin subdirectory to pyspark
URL: https://github.com/apache/spark/pull/23715#issuecomment-459719068
 
 
   > It's valid if you submit an alternative PR that takes out everything you feel doesn't belong in the pyspark package, and argue that that supersedes this request. Otherwise your objection is impossible to argue against.
   
   It doesn't necessarily fix it right away. That's going to break the previous usecases even if it is something to get rid of non-PySpark stuff and this factor should also be considered. I am saying we should stop doing this and hold off if we see this way isn't the right way.
   
   For clarification, it doesn't mean we should fix something only because the existing way does that.
   
   > If you look at @holdenk's pr that created the pip package, you can see that the intention was to include all of spark that's required to run spark locally. That is the principle that should guide the decision whether to add the history server script.
   
   I am saying the intention should be re-discussed. I couldn't foresee such request like this. PyPi is for Python package. CRAN is for R package.
   
   > The history server is more than just 'useful'. It's essential for everyone new to spark who hasn't a clue how their job is actually executed and whether they wrote their script right.
   
   The history server is optional. People can still use PySpark, no? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org