You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by GitBox <gi...@apache.org> on 2021/10/04 20:33:58 UTC
[GitHub] [incubator-sedona] Imbruced opened a new pull request #552: St make polygon
Imbruced opened a new pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552
## Is this PR related to a proposed Issue?
SEDONA-45
## What changes were proposed in this PR?
New SQL function
## How was this patch tested?
Scala and Python unit tests
## Did this PR include necessary documentation updates?
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on a change in pull request #552: [SEDONA-45] St make polygon
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on a change in pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552#discussion_r722892657
##########
File path: sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Functions.scala
##########
@@ -1167,3 +1167,46 @@ case class ST_SubDivideExplode(children: Seq[Expression]) extends Generator {
override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = ev
}
+
+
+case class ST_MakePolygon(inputExpressions: Seq[Expression])
+ extends Expression with CodegenFallback {
+ override def nullable: Boolean = true
+ private val geometryFactory = new GeometryFactory()
+
+ override def eval(input: InternalRow): Any = {
+ inputExpressions.betweenLength(1, 2)
+ val exteriorRing = inputExpressions.head
+ val possibleHolesRaw = inputExpressions.tail.headOption.map(_.eval(input).asInstanceOf[ArrayData])
+ val numOfElements = possibleHolesRaw.map(_.numElements()).getOrElse(0)
+
+ val holes = (0 until numOfElements).map(el => possibleHolesRaw match {
+ case Some(value) => Some(value.getArray(el))
+ case None => None
+ }).filter(_.nonEmpty)
+ .map(el => el.map(_.toGeometry))
+ .flatMap{
+ case maybeLine: Option[LineString] =>
+ maybeLine.map(line => geometryFactory.createLinearRing(line.getCoordinates))
+ case _ => None
+ }
+
+ exteriorRing.toGeometry(input) match {
+ case geom: LineString =>
+ try {
+ val poly = new Polygon(geometryFactory.createLinearRing(geom.getCoordinates), holes.toArray, geometryFactory)
+ poly.toGenericArrayData
+ }
+ catch {
+ case e: Exception => null
Review comment:
Sounds good to me!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu merged pull request #552: [SEDONA-45] St make polygon
Posted by GitBox <gi...@apache.org>.
jiayuasu merged pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on a change in pull request #552: St make polygon
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on a change in pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552#discussion_r721912906
##########
File path: sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Functions.scala
##########
@@ -1167,3 +1167,46 @@ case class ST_SubDivideExplode(children: Seq[Expression]) extends Generator {
override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = ev
}
+
+
+case class ST_MakePolygon(inputExpressions: Seq[Expression])
+ extends Expression with CodegenFallback {
+ override def nullable: Boolean = true
+ private val geometryFactory = new GeometryFactory()
+
+ override def eval(input: InternalRow): Any = {
+ inputExpressions.betweenLength(1, 2)
+ val exteriorRing = inputExpressions.head
+ val possibleHolesRaw = inputExpressions.tail.headOption.map(_.eval(input).asInstanceOf[ArrayData])
+ val numOfElements = possibleHolesRaw.map(_.numElements()).getOrElse(0)
+
+ val holes = (0 until numOfElements).map(el => possibleHolesRaw match {
+ case Some(value) => Some(value.getArray(el))
+ case None => None
+ }).filter(_.nonEmpty)
+ .map(el => el.map(_.toGeometry))
+ .flatMap{
+ case maybeLine: Option[LineString] =>
+ maybeLine.map(line => geometryFactory.createLinearRing(line.getCoordinates))
+ case _ => None
+ }
+
+ exteriorRing.toGeometry(input) match {
+ case geom: LineString =>
+ try {
+ val poly = new Polygon(geometryFactory.createLinearRing(geom.getCoordinates), holes.toArray, geometryFactory)
+ poly.toGenericArrayData
+ }
+ catch {
+ case e: Exception => null
Review comment:
Will it be better if you throw an exception here instead of giving a null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on pull request #552: [SEDONA-45] St make polygon
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552#issuecomment-935480821
I will merge this later once the 1.1.0 vote passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced commented on pull request #552: St make polygon
Posted by GitBox <gi...@apache.org>.
Imbruced commented on pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552#issuecomment-933835554
I am adding new sql functions based on jira tasks. I will implement them one after another.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced commented on a change in pull request #552: [SEDONA-45] St make polygon
Posted by GitBox <gi...@apache.org>.
Imbruced commented on a change in pull request #552:
URL: https://github.com/apache/incubator-sedona/pull/552#discussion_r722737615
##########
File path: sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/Functions.scala
##########
@@ -1167,3 +1167,46 @@ case class ST_SubDivideExplode(children: Seq[Expression]) extends Generator {
override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = ev
}
+
+
+case class ST_MakePolygon(inputExpressions: Seq[Expression])
+ extends Expression with CodegenFallback {
+ override def nullable: Boolean = true
+ private val geometryFactory = new GeometryFactory()
+
+ override def eval(input: InternalRow): Any = {
+ inputExpressions.betweenLength(1, 2)
+ val exteriorRing = inputExpressions.head
+ val possibleHolesRaw = inputExpressions.tail.headOption.map(_.eval(input).asInstanceOf[ArrayData])
+ val numOfElements = possibleHolesRaw.map(_.numElements()).getOrElse(0)
+
+ val holes = (0 until numOfElements).map(el => possibleHolesRaw match {
+ case Some(value) => Some(value.getArray(el))
+ case None => None
+ }).filter(_.nonEmpty)
+ .map(el => el.map(_.toGeometry))
+ .flatMap{
+ case maybeLine: Option[LineString] =>
+ maybeLine.map(line => geometryFactory.createLinearRing(line.getCoordinates))
+ case _ => None
+ }
+
+ exteriorRing.toGeometry(input) match {
+ case geom: LineString =>
+ try {
+ val poly = new Polygon(geometryFactory.createLinearRing(geom.getCoordinates), holes.toArray, geometryFactory)
+ poly.toGenericArrayData
+ }
+ catch {
+ case e: Exception => null
Review comment:
The reason why I changed to null instead of raising exception was to make that fault tollerant. I can change that, but I heard some complaints about that in some cases application is running for several minutes and then fail due to data quality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org