You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by GitBox <gi...@apache.org> on 2022/04/15 06:13:47 UTC

[GitHub] [incubator-sedona] ngaur9 opened a new pull request, #608: Sedona 106 add st line from text

ngaur9 opened a new pull request, #608:
URL: https://github.com/apache/incubator-sedona/pull/608

   ## Is this PR related to a proposed Issue?
   [SEDONA-106](https://issues.apache.org/jira/browse/SEDONA-106)
   
   ## What changes were proposed in this PR?
   Added ST_LineFromText
   
   ## How was this patch tested?
   Added unit tests for Scala, Python and Java. Tested the website using mkdocs.
   
   ## Did this PR include necessary documentation updates?
   Yes. Added descriptions in the source code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851817126


##########
docs/archive/api/sql/GeoSparkSQL-Constructor.md:
##########
@@ -155,6 +155,24 @@ FROM polygontable
 SELECT ST_PolygonFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794,-74.0428197,40.6867969', ',') AS polygonshape
 ```
 
+## ST_LineFromText

Review Comment:
   I have moved it below ST_GeomFromWKT. Please let me know if it has to be moved to some other place or file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu merged pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu merged PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851817082


##########
docs/api/sql/Constructor.md:
##########
@@ -172,6 +172,24 @@ FROM linestringtable
 SELECT ST_LineStringFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794', ',') AS linestringshape
 ```
 
+## ST_LineFromText
+
+Introduction: Construct a Line from Text, delimited by Delimiter
+
+Format: `ST_LineFromText (Text:string, Delimiter:char)`
+
+Since: `v1.0.0`

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851817148


##########
sql/src/test/scala/org/apache/sedona/sql/predicateTestScala.scala:
##########
@@ -140,6 +140,37 @@ class predicateTestScala extends TestBaseScala {
       assert(equaldf.count() == 0, s"Expected 0 value but got ${equaldf.count()}")
 
     }
+
+    it("Passed ST_Equals for ST_LineFromText and ST_Polygon") {

Review Comment:
   Removed this test case and added a simple test case in constructorTestScala.scala file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851817126


##########
docs/archive/api/sql/GeoSparkSQL-Constructor.md:
##########
@@ -155,6 +155,24 @@ FROM polygontable
 SELECT ST_PolygonFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794,-74.0428197,40.6867969', ',') AS polygonshape
 ```
 
+## ST_LineFromText

Review Comment:
   I have moved below ST_GeomFromWKT. Let me know if it has to be moved to some other place or file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r854438357


##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,40 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])
+  extends Expression with CodegenFallback with UserDataGeneratator {
+  // This is an expression which takes one input expressions.
+  assert(inputExpressions.length == 1)
+
+  override def nullable: Boolean = false

Review Comment:
   nullable should be true



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851817126


##########
docs/archive/api/sql/GeoSparkSQL-Constructor.md:
##########
@@ -155,6 +155,24 @@ FROM polygontable
 SELECT ST_PolygonFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794,-74.0428197,40.6867969', ',') AS polygonshape
 ```
 
+## ST_LineFromText

Review Comment:
   I have moved it below ST_GeomFromWKT. Let me know if it has to be moved to some other place or file.



##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,37 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])

Review Comment:
   I referred ST_GeomFromWKT  function call and added a LINE filter. Let me know if it needs modification or refactoring.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r852520310


##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,36 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])
+  extends Expression with CodegenFallback with UserDataGeneratator {
+  // This is an expression which takes one input expressions.
+  assert(inputExpressions.length == 1)
+
+  override def nullable: Boolean = false
+
+  override def eval(inputRow: InternalRow): Any = {
+    val lineString = inputExpressions(0).eval(inputRow).asInstanceOf[UTF8String].toString
+
+    var fileDataSplitter = FileDataSplitter.WKT
+    var formatMapper = new FormatMapper(fileDataSplitter, false, GeometryType.LINESTRING)

Review Comment:
   This should be `var formatMapper = new FormatMapper(fileDataSplitter, false)` as shown in https://github.com/apache/incubator-sedona/blob/master/sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala#L145
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r852519668


##########
docs/archive/api/sql/GeoSparkSQL-Constructor.md:
##########
@@ -19,6 +19,19 @@ FROM polygontable
 SELECT ST_GeomFromWKT('POINT(40.7128,-74.0060)') AS geometry
 ```
 
+## ST_LineFromText
+
+Introduction: Construct a Line from Wkt. Unlimited UUID strings can be appended.
+

Review Comment:
   Again, do NOT change this doc please.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r854446352


##########
docs/api/sql/Constructor.md:
##########
@@ -172,6 +172,24 @@ FROM linestringtable
 SELECT ST_LineStringFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794', ',') AS linestringshape
 ```
 
+## ST_LineFromText
+
+Introduction: Construct a Line from Text, delimited by Delimiter
+
+Format: `ST_LineFromText (Text:string, Delimiter:char)`
+
+Since: `v1.2.1`
+
+Spark SQL example:
+```SQL
+SELECT ST_LineFromText(linetable._c0,',') AS lineshape
+FROM linetable
+```
+
+```SQL

Review Comment:
   This example should use WKT string as input. Please fix the doc accordingly



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851817138


##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,37 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])

Review Comment:
   I referred ST_GeomFromWKT  function call and added a LINE filter. Please, let me know if it needs modification or refactoring.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r852591258


##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,36 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])
+  extends Expression with CodegenFallback with UserDataGeneratator {
+  // This is an expression which takes one input expressions.
+  assert(inputExpressions.length == 1)
+
+  override def nullable: Boolean = false
+
+  override def eval(inputRow: InternalRow): Any = {
+    val lineString = inputExpressions(0).eval(inputRow).asInstanceOf[UTF8String].toString
+
+    var fileDataSplitter = FileDataSplitter.WKT
+    var formatMapper = new FormatMapper(fileDataSplitter, false, GeometryType.LINESTRING)

Review Comment:
   After you get the geomtetry, use geometry.getGeometryType to check if it is LineString. If not, return null. see https://locationtech.github.io/jts/javadoc/org/locationtech/jts/geom/Geometry.html#getGeometryType--



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on pull request #608: Sedona 106 add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#issuecomment-1100753889

   I have incorporated the changes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #608: Sedona 106 add st line from text

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r851718619


##########
docs/api/sql/Constructor.md:
##########
@@ -172,6 +172,24 @@ FROM linestringtable
 SELECT ST_LineStringFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794', ',') AS linestringshape
 ```
 
+## ST_LineFromText
+
+Introduction: Construct a Line from Text, delimited by Delimiter
+
+Format: `ST_LineFromText (Text:string, Delimiter:char)`
+
+Since: `v1.0.0`

Review Comment:
   v1.2.1



##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,37 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])

Review Comment:
   This implementation is not consistent with PostGIS: https://postgis.net/docs/ST_LineFromText.html
   
   The input string must be WKT. The logic should simply re-use the ST_GeomFromWKT



##########
sql/src/test/scala/org/apache/sedona/sql/predicateTestScala.scala:
##########
@@ -140,6 +140,37 @@ class predicateTestScala extends TestBaseScala {
       assert(equaldf.count() == 0, s"Expected 0 value but got ${equaldf.count()}")
 
     }
+
+    it("Passed ST_Equals for ST_LineFromText and ST_Polygon") {

Review Comment:
   Please create a simple test case. Not sure why this has anything to do with polygons.



##########
docs/archive/api/sql/GeoSparkSQL-Constructor.md:
##########
@@ -155,6 +155,24 @@ FROM polygontable
 SELECT ST_PolygonFromText('-74.0428197,40.6867969,-74.0421975,40.6921336,-74.0508020,40.6912794,-74.0428197,40.6867969', ',') AS polygonshape
 ```
 
+## ST_LineFromText

Review Comment:
   This doc is put in wrong place



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r852579065


##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,36 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])
+  extends Expression with CodegenFallback with UserDataGeneratator {
+  // This is an expression which takes one input expressions.
+  assert(inputExpressions.length == 1)
+
+  override def nullable: Boolean = false
+
+  override def eval(inputRow: InternalRow): Any = {
+    val lineString = inputExpressions(0).eval(inputRow).asInstanceOf[UTF8String].toString
+
+    var fileDataSplitter = FileDataSplitter.WKT
+    var formatMapper = new FormatMapper(fileDataSplitter, false, GeometryType.LINESTRING)

Review Comment:
   I just wanted to ask, if we remove the LineString, how will we identify if it is a LIne or some other geometrical figure?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-sedona] ngaur9 commented on a diff in pull request #608: [SEDONA-106] add st line from text

Posted by GitBox <gi...@apache.org>.
ngaur9 commented on code in PR #608:
URL: https://github.com/apache/incubator-sedona/pull/608#discussion_r853574134


##########
sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Constructors.scala:
##########
@@ -94,6 +94,36 @@ case class ST_PolygonFromText(inputExpressions: Seq[Expression])
   }
 }
 
+/**
+  * Return a line from a string. The string must be plain string and each coordinate must be separated by a delimiter.
+  *
+  * @param inputExpressions
+  */
+case class ST_LineFromText(inputExpressions: Seq[Expression])
+  extends Expression with CodegenFallback with UserDataGeneratator {
+  // This is an expression which takes one input expressions.
+  assert(inputExpressions.length == 1)
+
+  override def nullable: Boolean = false
+
+  override def eval(inputRow: InternalRow): Any = {
+    val lineString = inputExpressions(0).eval(inputRow).asInstanceOf[UTF8String].toString
+
+    var fileDataSplitter = FileDataSplitter.WKT
+    var formatMapper = new FormatMapper(fileDataSplitter, false, GeometryType.LINESTRING)

Review Comment:
   Added the Linestring filter to it. Please, let me know if anything else needs to be done. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org