You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Indhumathi (Jira)" <ji...@apache.org> on 2022/03/04 09:57:00 UTC
[jira] [Resolved] (CARBONDATA-4325) Documentation Issue in Github Link: https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md and fix partition table creation with df issue
[ https://issues.apache.org/jira/browse/CARBONDATA-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Indhumathi resolved CARBONDATA-4325.
------------------------------------
Fix Version/s: 2.3.0
Resolution: Fixed
> Documentation Issue in Github Link: https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md and fix partition table creation with df issue
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-4325
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4325
> Project: CarbonData
> Issue Type: Bug
> Components: docs
> Reporter: PURUJIT CHAUGULE
> Priority: Minor
> Fix For: 2.3.0
>
> Attachments: Partition_Table_Creation_Fail_With_Spatial_Index_Property.png
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> *Scenario 1:*
> [https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md] :
> * Under _*SUPPORTED Options,*_ mention all supported Table Properties. Following are list of supported Table Properties not mentioned in the document:
> *
> ** bucketNumber
> ** bucketColumns
> ** streaming
> ** timestampformat
> ** dateformat
> ** SPATIAL_INDEX
> ** SPATIAL_INDEX_type
> ** SPATIAL_INDEX_sourcecolumns
> ** SPATIAL_INDEX_originLatitude
> ** SPATIAL_INDEX_gridSize
> ** SPATIAL_INDEX_conversionRatio
> ** SPATIAL_INDEX_class
> *Scenario 2:*
> _Partition Table Creation Using Spark Dataframe Fails with Spatial Index Property._
> Queries:
> val geoSchema = StructType(Seq(StructField("timevalue", LongType, nullable = true),
> StructField("longitude", LongType, nullable = false),
> StructField("latitude", LongType, nullable = false)))
> val geoDf = sqlContext.read.option("delimiter", ",").option("header", "true").schema(geoSchema).csv("hdfs://hacluster/geodata/geodata.csv")
> sql("drop table if exists source_index_df").show()
> geoDf.write
> .format("carbondata")
> .option("tableName", "source_index_df")
> .option("partitionColumns", "timevalue")
> .option("SPATIAL_INDEX", "mygeohash")
> .option("SPATIAL_INDEX.mygeohash.type", "geohash")
> .option("spatial_index.MyGeoHash.sourcecolumns", "longitude, latitude")
> .option("SPATIAL_INDEX.MyGeoHash.originLatitude", "39.832277")
> .option("SPATIAL_INDEX.mygeohash.gridSize", "50")
> .option("spatial_index.mygeohash.conversionRatio", "1000000")
> .option("spatial_index.mygeohash.CLASS", "org.apache.carbondata.geo.GeoHashIndex")
> .mode(SaveMode.Overwrite)
> .save()
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)