[GitHub] [spark] dgd-contributor commented on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

[GitBox <gi...@apache.org>]

dgd-contributor commented on pull request #33954:
URL: https://github.com/apache/spark/pull/33954#issuecomment-917655939


   Just curious, with your example
   ``` python
   >>> def transform(pdf) -> pd.DataFrame[int, int]:
   ...     pdf['A'] = pdf.id + 1
   ...     return pdf
   ...
   >>> ks.range(5).koalas.apply_batch(transform)
   ```
   Could you please explain how the index type hint improve performance? I read  [default index](https://koalas.readthedocs.io/en/latest/user_guide/options.html#default-index-type) and [Specify the index column in conversion from Spark DataFrame to Koalas DataFrame](https://koalas.readthedocs.io/en/latest/user_guide/best_practices.html#specify-the-index-column-in-conversion-from-spark-dataframe-to-koalas-dataframe) but I have not understood. Thank you so much.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org