You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/06/25 03:00:37 UTC

[GitHub] [spark] zhengruifeng opened a new pull request, #36986: [SPARK-39235][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

zhengruifeng opened a new pull request, #36986:
URL: https://github.com/apache/spark/pull/36986

   ### What changes were proposed in this pull request?
   corresponding changes in pyspark to [CacheTable, isCached, UncacheTable, setCurrentCatalog, currentCatalog, listCatalogs](https://issues.apache.org/jira/browse/SPARK-39506)
   
   1, add method `currentCatalog`, `setCurrentCatalog`, `listCatalogs`;
   2, update the document of `CacheTable`, `isCached`, `UncacheTable`, and add a UT
   
   ### Why are the changes needed?
   to support 3-layer-namespace
   
   
   ### Does this PR introduce _any_ user-facing change?
   yes, new api added
   
   
   ### How was this patch tested?
   added ut
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #36986: [SPARK-39235][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on PR #36986:
URL: https://github.com/apache/spark/pull/36986#issuecomment-1166206382

   All test passed except document build failure which also appears in other PRs.
   
   cc @HyukjinKwon @cloud-fan @amaliujia 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #36986: [SPARK-39598][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on code in PR #36986:
URL: https://github.com/apache/spark/pull/36986#discussion_r913377166


##########
python/pyspark/sql/catalog.py:
##########
@@ -79,6 +84,42 @@ def __init__(self, sparkSession: SparkSession) -> None:
         self._jsparkSession = sparkSession._jsparkSession
         self._jcatalog = sparkSession._jsparkSession.catalog()
 
+    def currentCatalog(self) -> str:

Review Comment:
   Just dropping a comment .. let's make sure these are seen in PySpark documentation properly. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #36986: [SPARK-39235][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #36986:
URL: https://github.com/apache/spark/pull/36986#issuecomment-1166211832

   I made a fix here to recover master branch, @zhengruifeng . 
   - https://github.com/apache/spark/pull/36987


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng closed pull request #36986: [SPARK-39598][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

Posted by GitBox <gi...@apache.org>.
zhengruifeng closed pull request #36986: [SPARK-39598][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace
URL: https://github.com/apache/spark/pull/36986


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #36986: [SPARK-39598][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on PR #36986:
URL: https://github.com/apache/spark/pull/36986#issuecomment-1171220378

   Merged to master, thanks @cloud-fan for reviewing!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on a diff in pull request #36986: [SPARK-39598][PYTHON] Make *cache*, *catalog* in the python side support 3-layer-namespace

Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on code in PR #36986:
URL: https://github.com/apache/spark/pull/36986#discussion_r913419410


##########
python/pyspark/sql/catalog.py:
##########
@@ -79,6 +84,42 @@ def __init__(self, sparkSession: SparkSession) -> None:
         self._jsparkSession = sparkSession._jsparkSession
         self._jcatalog = sparkSession._jsparkSession.catalog()
 
+    def currentCatalog(self) -> str:

Review Comment:
   let me check it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org