[GitHub] [spark] cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`
URL: https://github.com/apache/spark/pull/25419#discussion_r312925637
 
 

 ##########
 File path: python/pyspark/sql/avro/functions.py
 ##########
 @@ -69,26 +69,35 @@ def from_avro(data, jsonFormatSchema, options={}):
 
 @ignore_unicode_prefix
 @since(3.0)
-def to_avro(data):
+def to_avro(data, jsonFormatSchema=""):
     """
     Converts a column into binary of avro format.
 
     Note: Avro is built-in but external data source module since Spark 2.4. Please deploy the
     application as per the deployment section of "Apache Avro Data Source Guide".
 
     :param data: the data column.
+    :param jsonFormatSchema: user-specified output avro schema in JSON string format.
 
     >>> from pyspark.sql import Row
     >>> from pyspark.sql.avro.functions import to_avro
-    >>> data = [(1, Row(name='Alice', age=2))]
-    >>> df = spark.createDataFrame(data, ("key", "value"))
-    >>> df.select(to_avro(df.value).alias("avro")).collect()
 
 Review comment:
   can we keep the original example? Users don't have to specify the json schema string.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org