[I] Raster Data Types in dataframe columns [sedona]

["normanb (via GitHub)" <gi...@apache.org>]

normanb opened a new issue, #2347:
URL: https://github.com/apache/sedona/issues/2347

   I have an interesting problem in that for some functions the dataframe column type is being set as `udt` when saved to a delta table and for other functions it is set as `binary`. I would prefer when the column is a geotiff that the column type is set as `udt` so that it is a usable `GridCoverage2d`.
   
   I am running on Databricks and Sedona is configured and working well. I am using DBT to define the models.
   
   `RS_UnionAgg` results in a column type `udt` but `RS_ReprojectMatch` does not.
   
   I am using `RS_AsGeotiff` as follows;
   
   ```
   RS_AsGeotiff(RS_ReprojectMatch(vals.raster, ref)) as values
   ```
   
   Which results in a column of type `binary` which is to be expected but I want `RS_ReprojectMatch` to set the column type as `udt`.
   
   If I don't wrap `RS_ReprojectMatch` is `RS_AsGeotiff` then I get the following error;
   
   ```
   Database Error org.apache.spark.sql.sedona_sql.UDT.RasterUDT$.<init>()
   ```
   
   Which is probably close to what I want without the error :) (as per `RS_UnionAgg` that gives a `udt` column type).
   
   I am using `RS_UnionAgg` after calling `RS_TileExplode` and that defines the `udt` column type appropriately for a column.
   
   My workaround is to convert the column immediately after loading;
   
   ```
   df= df_tmp.withColumn("values_raster", F.expr("RS_FromGeoTiff(values)")).drop("values")
   ```
   
   I would like to avoid the workaround but I could add it as a post hook I guess to convert the column type.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Raster Data Types in dataframe columns [sedona]

["normanb (via GitHub)" <gi...@apache.org>]

normanb commented on issue #2347:
URL: https://github.com/apache/sedona/issues/2347#issuecomment-3343841642

   @jiayuasu sorry for the delay, yes `RS_UnionAgg` is from https://sedona.apache.org/latest/api/sql/Raster-aggregate-function/#rs_union_aggr


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Raster Data Types in dataframe columns [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu commented on issue #2347:
URL: https://github.com/apache/sedona/issues/2347#issuecomment-3294279588

   @normanb Just want to make sure we are on the same page: Is `RS_UnionAgg` the function listed here? https://sedona.apache.org/latest/api/sql/Raster-aggregate-function/#rs_union_aggr


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Raster Data Types in dataframe columns [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu closed issue #2347: Raster Data Types in dataframe columns
URL: https://github.com/apache/sedona/issues/2347


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org