You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/09/15 07:00:58 UTC
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #42938: [SPARK-44788][CONNECT][PYTHON][SQL] Add from_xml and schema_of_xml to pyspark, spark connect and sql function
HyukjinKwon commented on code in PR #42938:
URL: https://github.com/apache/spark/pull/42938#discussion_r1326881986
##########
sql/core/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -7367,15 +7367,83 @@ object functions {
* "https://spark.apache.org/docs/latest/sql-data-sources-xml.html#data-source-option">
* Data Source Option</a> in the version you use.
* @group collection_funcs
- * @since
+ * @since 4.0.0
*/
// scalastyle:on line.size.limit
def from_xml(e: Column, schema: StructType, options: Map[String, String]): Column = withExpr {
XmlToStructs(CharVarcharUtils.failIfHasCharVarchar(schema), options, e.expr)
}
+ // scalastyle:off line.size.limit
+ /**
+ * (Java-specific) Parses a column containing a XML string into a `StructType`
+ * with the specified schema.
+ * Returns `null`, in the case of an unparseable string.
+ *
+ * @param e a string column containing XML data.
+ * @param schema the schema as a DDL-formatted string.
+ * @param options options to control how the XML is parsed. accepts the same options and the
+ * xml data source.
+ * See
+ * <a href=
+ * "https://spark.apache.org/docs/latest/sql-data-sources-xml.html#data-source-option">
+ * Data Source Option</a> in the version you use.
+ * @group collection_funcs
+ * @since 4.0.0
+ */
+ // scalastyle:on line.size.limit
+ def from_xml(e: Column, schema: String, options: java.util.Map[String, String]): Column = {
+ from_xml(e, schema, options.asScala.toMap)
+ }
+
+ // scalastyle:off line.size.limit
+
+ /**
+ * (Scala-specific) Parses a column containing a XML string into a `StructType`
+ * with the specified schema.
+ * Returns `null`, in the case of an unparseable string.
+ *
+ * @param e a string column containing XML data.
+ * @param schema the schema as a DDL-formatted string.
+ * @param options options to control how the XML is parsed. accepts the same options and the
+ * Xml data source.
+ * See
+ * <a href=
+ * "https://spark.apache.org/docs/latest/sql-data-sources-xml.html#data-source-option">
+ * Data Source Option</a> in the version you use.
+ * @group collection_funcs
+ * @since 4.0.0
+ */
+ // scalastyle:on line.size.limit
+ def from_xml(e: Column, schema: String, options: Map[String, String]): Column = {
Review Comment:
Let's probably remove the Scala specific `Map`, and only have one with Java signature. Scala can easily use them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org