You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/17 00:33:54 UTC
[GitHub] [spark] xinrong-databricks commented on a change in pull request #34238: [SPARK-36969][PYTHON] Inline type hints for SparkContext
xinrong-databricks commented on a change in pull request #34238:
URL: https://github.com/apache/spark/pull/34238#discussion_r730017148
##########
File path: python/pyspark/sql/functions.py
##########
@@ -71,7 +71,7 @@ def _get_get_jvm_function(name: str, sc: SparkContext) -> Callable:
Retrieves JVM function identified by name from
Java gateway associated with sc.
"""
- return getattr(sc._jvm.functions, name) # type: ignore[attr-defined]
+ return getattr(sc._jvm.functions, name) # type: ignore[attr-defined, union-attr]
Review comment:
Thanks!
##########
File path: python/pyspark/conf.py
##########
@@ -118,43 +126,56 @@ def __init__(self, loadDefaults=True, _jvm=None, _jconf=None):
if _jvm is not None:
# JVM is created, so create self._jconf directly through JVM
self._jconf = _jvm.SparkConf(loadDefaults)
- self._conf = None
+ self._conf: Optional[Dict[str, str]] = None
else:
# JVM is not created, so store data in self._conf first
self._jconf = None
self._conf = {}
- def set(self, key, value):
+ def set(self, key: str, value: Optional[str]) -> "SparkConf":
Review comment:
Is it possible for `value` to be `Any`?
##########
File path: python/pyspark/conf.py
##########
@@ -118,43 +126,56 @@ def __init__(self, loadDefaults=True, _jvm=None, _jconf=None):
if _jvm is not None:
# JVM is created, so create self._jconf directly through JVM
self._jconf = _jvm.SparkConf(loadDefaults)
- self._conf = None
+ self._conf: Optional[Dict[str, str]] = None
else:
# JVM is not created, so store data in self._conf first
self._jconf = None
self._conf = {}
- def set(self, key, value):
+ def set(self, key: str, value: Optional[str]) -> "SparkConf":
Review comment:
Is it possible for `value` to be `Optional[Any]`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org