You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/05/25 09:42:51 UTC
[GitHub] [spark] Yikun commented on a diff in pull request #36353: [SPARK-38946][PYTHON][PS] Generates a new dataframe instead of operating inplace in df.eval/update/fillna/setitem
Yikun commented on code in PR #36353:
URL: https://github.com/apache/spark/pull/36353#discussion_r881448119
##########
python/pyspark/pandas/frame.py:
##########
@@ -490,20 +491,30 @@ def _internal(self) -> InternalFrame:
return cast(InternalFrame, self._internal_frame) # type: ignore[has-type]
def _update_internal_frame(
- self, internal: InternalFrame, requires_same_anchor: bool = True
+ self,
+ internal: InternalFrame,
+ requires_same_anchor: bool = True,
+ force_disconnect: bool = False,
) -> None:
"""
Update InternalFrame with the given one.
- If the column_label is changed or the new InternalFrame is not the same `anchor`,
- disconnect the link to the Series and create a new one.
+ If the column_label is changed or the new InternalFrame is not the same `anchor` or the
+ `force_disconnect` flag is set to True, disconnect the link to the Series and create a new
+ one.
Review Comment:
Thanks for review, not sure get your idea completely.
It's A or B or C, Create new dataframe if one of three coditions statified, so `if requires_same_anchor and force_disconnect are both True` also create the new dataframe.
##########
python/pyspark/pandas/frame.py:
##########
@@ -490,20 +491,30 @@ def _internal(self) -> InternalFrame:
return cast(InternalFrame, self._internal_frame) # type: ignore[has-type]
def _update_internal_frame(
- self, internal: InternalFrame, requires_same_anchor: bool = True
+ self,
+ internal: InternalFrame,
+ requires_same_anchor: bool = True,
+ force_disconnect: bool = False,
) -> None:
"""
Update InternalFrame with the given one.
- If the column_label is changed or the new InternalFrame is not the same `anchor`,
- disconnect the link to the Series and create a new one.
+ If the column_label is changed or the new InternalFrame is not the same `anchor` or the
+ `force_disconnect` flag is set to True, disconnect the link to the Series and create a new
+ one.
Review Comment:
Thanks for review, not sure got your idea completely.
It's A or B or C, Create new dataframe if one of three coditions statified, so `if requires_same_anchor and force_disconnect are both True` also create the new dataframe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org