[GitHub] [iceberg] stayrascal commented on issue #1766: Reading iceberg table using Amazon Athena

[GitBox <gi...@apache.org>]

stayrascal commented on issue #1766:
URL: https://github.com/apache/iceberg/issues/1766#issuecomment-803775621


   Hi, just given an example here follow the [document](https://iceberg.apache.org/aws/), and meet a 'unknown' classification issue in Glue data catalog.
   
   1. Start Spark SQL
   ```
   spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:0.11.0,software.amazon.awssdk:bundle:2.15.40,software.amazon.awssdk:url-connection-client:2.15.40 \
       --conf spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCatalog \
       --conf spark.sql.catalog.my_catalog.warehouse=s3://xxxx/lake_house/iceberg/spark \
       --conf spark.sql.catalog.my_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \
       --conf spark.sql.catalog.my_catalog.lock-impl=org.apache.iceberg.aws.glue.DynamoLockManager \
       --conf spark.sql.catalog.my_catalog.lock.table=myGlueLockTable
   ```
   
   2. Create table and insert data:
   ```
   CREATE TABLE my_catalog.exist_db.table_demo (id bigint, data string) USING iceberg;
   INSERT INTO my_catalog.exist_db.table_demo VALUES (1, 'a'), (2, 'b'), (3, 'c');
   ```
   
   3. When Query table from AWS glue data catalog console, it look like missed the `storageDescriptor` filed about the table schema. didn't see where the [GuleCatalog](https://github.com/apache/iceberg/blob/ad78cc6cf2/aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java#L109) set the storage descriptor before create table by `Glue` client
   ```
   {
      "actionResponses":[
         {
            "action":"com.amazonaws.console.glue.awssdk.shared.context.AWSGlueContext.getTable",
            "data":{
               xxxxx
               },
               "sdkHttpMetadata":{
                  xxxx
                  },
                  "httpHeaders":{
                    xxxx
                  },
                  "httpStatusCode":200
               },
               "table":{
                  "name":"table_demo",
                  "databaseName":"exist_db",
                  "createTime":1616384916000,
                  "updateTime":1616384939000,
                  "retention":0,
                  "tableType":"EXTERNAL_TABLE",
                  "parameters":{
                     "previous_metadata_location":"s3://xxxxxxx/lake_house/iceberg/spark/exist_db.db/table_demo/metadata/00000-d7d65077-55a9-4b3b-b812-7ae34a4687be.metadata.json",
                     "metadata_location":"s3://xxxxxxx/lake_house/iceberg/spark/exist_db.db/table_demo/metadata/00001-3e4ed2c5-fe12-4d04-ab45-74f208afce71.metadata.json",
                     "table_type":"ICEBERG"
                  },
                  "createdBy":"arn:aws:sts::xxxxx:assumed-role/xxxxxxx",
                  "isRegisteredWithLakeFormation":false,
                  "catalogId":"xxxxxxx",
                  "isRowFilteringEnabled":false
               }
            }
         }
      ]
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org