You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/30 02:18:29 UTC
[GitHub] [spark] dgd-contributor opened a new pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
dgd-contributor opened a new pull request #34151:
URL: https://github.com/apache/spark/pull/34151
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
8. If you want to add or modify an error type or message, please read the guideline first in
'core/src/main/resources/error/README.md'.
-->
### What changes were proposed in this pull request?
Inline type hints for python/pyspark/sql/context.py from Inline type hints for python/pyspark/sql/context.pyi.
### Why are the changes needed?
Currently, there is type hint stub files python/pyspark/sql/context.pyi to show the expected types for functions, but we can also take advantage of static type checking within the functions by inlining the type hints.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existed test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931848600
**[Test build #143779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143779/testReport)** for PR 34151 at commit [`f258e10`](https://github.com/apache/spark/commit/f258e106a1954a916d456b21bf27577f16cba9f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720408924
##########
File path: python/pyspark/sql/context.py
##########
@@ -72,20 +91,25 @@ class SQLContext(object):
_instantiatedContext = None
- def __init__(self, sparkContext, sparkSession=None, jsqlContext=None):
+ def __init__(
+ self,
+ sparkContext: SparkContext,
+ sparkSession: Optional[SparkSession] = None,
+ jsqlContext: Optional[JavaObject] = None
+ ) -> None:
if sparkSession is None:
warnings.warn(
"Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.",
FutureWarning
)
self._sc = sparkContext
- self._jsc = self._sc._jsc
- self._jvm = self._sc._jvm
+ self._jsc = self._sc._jsc # type: ignore[attr-defined]
+ self._jvm = self._sc._jvm # type: ignore[attr-defined]
Review comment:
Should we add a note, that these could be removed, once `SparkContext` is inlined?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931223391
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48278/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930801385
**[Test build #143752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143752/testReport)** for PR 34151 at commit [`4039efc`](https://github.com/apache/spark/commit/4039efc8f07f5d962a2e063d0145a4aef0b219c3).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720413754
##########
File path: python/pyspark/sql/context.py
##########
@@ -220,7 +250,12 @@ def range(self, start, end=None, step=1, numPartitions=None):
"""
return self.sparkSession.range(start, end, step, numPartitions)
- def registerFunction(self, name, f, returnType=None):
+ def registerFunction(
+ self,
+ name: str,
+ f: Callable[..., Any],
+ returnType: DataType = None # type: ignore[assignment]
Review comment:
This can be converted to `Optional[DataType]`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931120390
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48270/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931909706
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143781/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930915330
**[Test build #143759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143759/testReport)** for PR 34151 at commit [`d9f63b2`](https://github.com/apache/spark/commit/d9f63b2e46534929f70a94951a5fc915cec2ea32).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930935711
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48263/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930935600
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48263/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931017567
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143764/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931120390
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48270/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720408118
##########
File path: python/pyspark/sql/context.py
##########
@@ -95,7 +119,7 @@ def __init__(self, sparkContext, sparkSession=None, jsqlContext=None):
SQLContext._instantiatedContext = self
@property
- def _ssql_ctx(self):
+ def _ssql_ctx(self): # type: ignore[no-untyped-def]
Review comment:
If the purpose is validation of our own codebase, we should avoid `type: ignoring` internal methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931006408
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48270/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931909477
**[Test build #143781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143781/testReport)** for PR 34151 at commit [`f258e10`](https://github.com/apache/spark/commit/f258e106a1954a916d456b21bf27577f16cba9f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931187427
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48274/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931317136
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931271086
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48278/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931848804
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143779/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930915330
**[Test build #143759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143759/testReport)** for PR 34151 at commit [`d9f63b2`](https://github.com/apache/spark/commit/d9f63b2e46534929f70a94951a5fc915cec2ea32).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931884121
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48291/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dgd-contributor commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
dgd-contributor commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-933111613
Thank you for your review, @zero323. As we and @HyukjinKwon discussed contributing by sharing accounts, we will close this PR, and recreate this one with an individual account. We will also update the code with your comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dgd-contributor closed pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
dgd-contributor closed pull request #34151:
URL: https://github.com/apache/spark/pull/34151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931090887
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48270/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930798942
ok to test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931976732
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48293/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720411372
##########
File path: python/pyspark/sql/context.py
##########
@@ -104,12 +128,12 @@ def _ssql_ctx(self):
return self._jsqlContext
@property
- def _conf(self):
+ def _conf(self): # type: ignore[no-untyped-def]
"""Accessor for the JVM SQL-specific configurations"""
return self.sparkSession._jsparkSession.sessionState().conf()
@classmethod
- def getOrCreate(cls, sc):
+ def getOrCreate(cls: type, sc: SparkContext) -> SQLContext:
Review comment:
I don't think we have to annotate `cls` (same way, as won't annotate `self`, with a few exceptions). However, if we're going to do that, we should probably use specific type (i.e. `"Type[Foo]"`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931894945
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931966884
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48293/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931848792
**[Test build #143779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143779/testReport)** for PR 34151 at commit [`f258e10`](https://github.com/apache/spark/commit/f258e106a1954a916d456b21bf27577f16cba9f7).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931937641
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48293/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931262163
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48278/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931015731
**[Test build #143764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143764/testReport)** for PR 34151 at commit [`c2203b6`](https://github.com/apache/spark/commit/c2203b6aeb0eb2b8b402bd2460d263f1fa1bf46d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720413754
##########
File path: python/pyspark/sql/context.py
##########
@@ -220,7 +250,12 @@ def range(self, start, end=None, step=1, numPartitions=None):
"""
return self.sparkSession.range(start, end, step, numPartitions)
- def registerFunction(self, name, f, returnType=None):
+ def registerFunction(
+ self,
+ name: str,
+ f: Callable[..., Any],
+ returnType: DataType = None # type: ignore[assignment]
Review comment:
This can be converted to `Optional[DataType]` and ignore removed:
https://github.com/apache/spark/blob/0494dc90af48ce7da0625485a4dc6917a244d580/python/pyspark/sql/udf.py#L357
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931848600
**[Test build #143779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143779/testReport)** for PR 34151 at commit [`f258e10`](https://github.com/apache/spark/commit/f258e106a1954a916d456b21bf27577f16cba9f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930801927
**[Test build #143752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143752/testReport)** for PR 34151 at commit [`4039efc`](https://github.com/apache/spark/commit/4039efc8f07f5d962a2e063d0145a4aef0b219c3).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930697030
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930801941
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143752/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931108630
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48274/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931194357
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143767/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931864968
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48291/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931271086
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48278/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931886652
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48291/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930916325
**[Test build #143759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143759/testReport)** for PR 34151 at commit [`d9f63b2`](https://github.com/apache/spark/commit/d9f63b2e46534929f70a94951a5fc915cec2ea32).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931909693
**[Test build #143781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143781/testReport)** for PR 34151 at commit [`f258e10`](https://github.com/apache/spark/commit/f258e106a1954a916d456b21bf27577f16cba9f7).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931193799
**[Test build #143767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143767/testReport)** for PR 34151 at commit [`3ff5c03`](https://github.com/apache/spark/commit/3ff5c03305de70707ae9d9bfe962eaec09bedb66).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720409548
##########
File path: python/pyspark/sql/context.py
##########
@@ -15,21 +15,40 @@
# limitations under the License.
#
+from __future__ import annotations
+
import sys
import warnings
+from typing import Optional, Union, Callable, Any, Iterable, List, Tuple, overload, TYPE_CHECKING
+
+from py4j.java_gateway import JavaObject # type: ignore[import]
-from pyspark import since, _NoValue
-from pyspark.sql.session import _monkey_patch_RDD, SparkSession
+from pyspark import since, _NoValue # type: ignore[attr-defined]
+from pyspark.sql.session import _monkey_patch_RDD, SparkSession # type: ignore[attr-defined]
from pyspark.sql.dataframe import DataFrame
from pyspark.sql.readwriter import DataFrameReader
from pyspark.sql.streaming import DataStreamReader
from pyspark.sql.udf import UDFRegistration # noqa: F401
from pyspark.sql.utils import install_exception_handler
+from pyspark.context import SparkContext
+from pyspark.rdd import RDD
+
+if TYPE_CHECKING:
+ from pyspark.sql._typing import (
+ UserDefinedFunctionLike,
+ RowLike,
+ DateTimeLiteral,
+ LiteralType,
+ DecimalLiteral
+ )
+ from pyspark.sql.types import DataType, AtomicType, StructType
+ from pyspark.sql.pandas._typing import DataFrameLike
+ from pyspark.sql.streaming import StreamingQueryManager
__all__ = ["SQLContext", "HiveContext"]
-class SQLContext(object):
+class SQLContext(object): # type: ignore[no-redef]
Review comment:
Why is this `ignore` added?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931191521
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-932183861
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931909477
**[Test build #143781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143781/testReport)** for PR 34151 at commit [`f258e10`](https://github.com/apache/spark/commit/f258e106a1954a916d456b21bf27577f16cba9f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930916420
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930801941
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143752/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930697030
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720462221
##########
File path: python/pyspark/sql/context.py
##########
@@ -415,10 +522,10 @@ def createExternalTable(self, tableName, path=None, source=None, schema=None, **
-------
:class:`DataFrame`
"""
- return self.sparkSession.catalog.createExternalTable(
+ return self.sparkSession.catalog.createExternalTable( # type: ignore[attr-defined]
Review comment:
__Note__ Created SPARK-36909 to track this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931017404
**[Test build #143764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143764/testReport)** for PR 34151 at commit [`c2203b6`](https://github.com/apache/spark/commit/c2203b6aeb0eb2b8b402bd2460d263f1fa1bf46d).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931194324
**[Test build #143767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143767/testReport)** for PR 34151 at commit [`3ff5c03`](https://github.com/apache/spark/commit/3ff5c03305de70707ae9d9bfe962eaec09bedb66).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931895021
cc @ueshin @xinrong-databricks @itholic @zero323 fyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931886652
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48291/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931017567
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143764/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931317136
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930801385
**[Test build #143752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143752/testReport)** for PR 34151 at commit [`4039efc`](https://github.com/apache/spark/commit/4039efc8f07f5d962a2e063d0145a4aef0b219c3).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930839370
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48263/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931193799
**[Test build #143767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143767/testReport)** for PR 34151 at commit [`3ff5c03`](https://github.com/apache/spark/commit/3ff5c03305de70707ae9d9bfe962eaec09bedb66).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931909706
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143781/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931848804
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143779/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #34151:
URL: https://github.com/apache/spark/pull/34151#discussion_r720422501
##########
File path: python/pyspark/sql/context.py
##########
@@ -415,10 +522,10 @@ def createExternalTable(self, tableName, path=None, source=None, schema=None, **
-------
:class:`DataFrame`
"""
- return self.sparkSession.catalog.createExternalTable(
+ return self.sparkSession.catalog.createExternalTable( # type: ignore[attr-defined]
Review comment:
If `createExternalTable` is missing from stubs it would be better to add it there (we should probably make a fix for the sake of 3.1 and 3.2 anyway) or wait for SPARK-36881 to be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931191521
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48274/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-930916420
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143759/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931976732
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48293/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34151: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34151:
URL: https://github.com/apache/spark/pull/34151#issuecomment-931015731
**[Test build #143764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143764/testReport)** for PR 34151 at commit [`c2203b6`](https://github.com/apache/spark/commit/c2203b6aeb0eb2b8b402bd2460d263f1fa1bf46d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org