You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "oguzerdogmus (via GitHub)" <gi...@apache.org> on 2024/10/16 01:17:12 UTC

[PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

oguzerdogmus opened a new pull request, #173:
URL: https://github.com/apache/iceberg-go/pull/173

   Continuation from [the previous PR](https://github.com/apache/iceberg-go/pull/169) to support Glue catalog. Add support for more namespace and table operations. In the next PR, remaining operations will be added to complete Glue catalog for tables (e.g. create table).
   
   I mostly followed the same semantics as [the pyiceberg Glue catalog](https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/glue.py) for the implementation. This means that Glue database fields (description and location URI) are stored as props. Error handling are also done in a similar way.
   
   ### Testing
   
   - Unit tests are added. I am not sure if the project has AWS Glue set up (or some sort of realistic mock) for integration testing but I did not see it run as part of the CI actions.
   
   Updated `cmd/iceberg/main.go` and manually tested using my personal AWS account:
   
   ```
   > go run ./cmd/iceberg create --catalog glue --description "my new catalog" --location-uri "s3://test-namespace/" namespace apache-iceberg-test-00006
   
   > go run ./cmd/iceberg describe --catalog glue namespace apache-iceberg-test-00006
   ┌──────────────────────────────────────┐
   | Key           | Value                |
   | ------------------------------------ |
   | database_type | ICEBERG              |
   | Description   | my new catalog       |
   | Location      | s3://test-namespace/ |
   └──────────────────────────────────────┘
   
   > go run ./cmd/iceberg list --catalog glue
   ┌───────────────────────────┐
   | IDs                       |
   | ------------------------- |
   | apache-iceberg-test-00001 |
   | apache-iceberg-test-00002 |
   | apache-iceberg-test-00003 |
   | apache-iceberg-test-00004 |
   | apache-iceberg-test-00005 |
   | apache-iceberg-test-00006 |
   └───────────────────────────┘
   
   > go run ./cmd/iceberg properties --catalog glue get namespace apache-iceberg-test-00006 Description
   my new catalog
   
   > go run ./cmd/iceberg properties --catalog glue set namespace apache-iceberg-test-00006 Description "updated description"
   updated Description on apache-iceberg-test-00006
   
   > go run ./cmd/iceberg properties --catalog glue get namespace apache-iceberg-test-00006 Description
   updated description
   
   > go run ./cmd/iceberg describe --catalog glue namespace apache-iceberg-test-00006
   ┌──────────────────────────────────────┐
   | Key           | Value                |
   | ------------------------------------ |
   | Description   | updated description  |
   | Location      | s3://test-namespace/ |
   | database_type | ICEBERG              |
   └──────────────────────────────────────┘
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "natusioe (via GitHub)" <gi...@apache.org>.
natusioe commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1811534102


##########
catalog/glue.go:
##########
@@ -180,8 +267,155 @@ func (c *GlueCatalog) ListNamespaces(ctx context.Context, parent table.Identifie
 	return icebergNamespaces, nil
 }
 
+// CreateNamespace creates a new Iceberg namespace in the Glue catalog.
+func (c *GlueCatalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
+	database, err := identifierToGlueDatabase(namespace)
+	if err != nil {
+		return err
+	}
+
+	databaseParameters := map[string]string{
+		databaseTypePropsKey: glueTypeIceberg,
+	}
+
+	description := props[descriptionPropsKey]
+	locationURI := props[locationPropsKey]
+
+	if description != "" {
+		databaseParameters[descriptionPropsKey] = description
+	}
+	if locationURI != "" {
+		databaseParameters[locationPropsKey] = locationURI
+	}
+
+	databaseInput := &types.DatabaseInput{
+		Name:       aws.String(database),
+		Parameters: databaseParameters,
+	}
+
+	params := &glue.CreateDatabaseInput{DatabaseInput: databaseInput}
+	_, err = c.glueSvc.CreateDatabase(ctx, params)

Review Comment:
   Added.



##########
catalog/glue.go:
##########
@@ -122,33 +149,93 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{
+		DatabaseName: aws.String(database),
+		Name:         aws.String(tableName),
+	}
+	_, err = c.glueSvc.DeleteTable(ctx, params)
+	if err != nil {
+		return fmt.Errorf("failed to drop table %s.%s: %w", database, tableName, err)
+	}
+
+	return nil
 }
 
+// RenameTable renames an Iceberg table in the Glue catalog.
 func (c *GlueCatalog) RenameTable(ctx context.Context, from, to table.Identifier) (*table.Table, error) {
-	return nil, fmt.Errorf("%w: [Glue Catalog] rename table", iceberg.ErrNotImplemented)
-}
+	fromDatabase, fromTable, err := identifierToGlueTable(from)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
-	return fmt.Errorf("%w: [Glue Catalog] create namespace", iceberg.ErrNotImplemented)
-}
+	toDatabase, toTable, err := identifierToGlueTable(to)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) DropNamespace(ctx context.Context, namespace table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop namespace", iceberg.ErrNotImplemented)
-}
+	if fromDatabase != toDatabase {
+		return nil, fmt.Errorf("cannot rename table across namespaces: %s -> %s", fromDatabase, toDatabase)
+	}
 
-func (c *GlueCatalog) LoadNamespaceProperties(ctx context.Context, namespace table.Identifier) (iceberg.Properties, error) {
-	return nil, fmt.Errorf("%w: [Glue Catalog] load namespace properties", iceberg.ErrNotImplemented)
-}
+	// Fetch the existing Glue table to copy the metadata into the new table.
+	fromGlueTable, err := c.getTable(ctx, fromDatabase, fromTable)
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch the table %s.%s: %w", fromDatabase, fromTable, err)
+	}
 
-func (c *GlueCatalog) UpdateNamespaceProperties(ctx context.Context, namespace table.Identifier,
-	removals []string, updates iceberg.Properties) (PropertiesUpdateSummary, error) {
-	return PropertiesUpdateSummary{}, fmt.Errorf("%w: [Glue Catalog] update namespace properties", iceberg.ErrNotImplemented)
+	// Create the new table.
+	_, err = c.glueSvc.CreateTable(ctx, &glue.CreateTableInput{

Review Comment:
   Yes, good catch, added.



##########
catalog/glue.go:
##########
@@ -30,22 +30,40 @@ import (
 	"github.com/aws/aws-sdk-go-v2/service/glue/types"
 )
 
-const glueTypeIceberg = "ICEBERG"
+const (
+	glueTypeIceberg      = "ICEBERG"
+	databaseTypePropsKey = "database_type"
+	tableTypePropsKey    = "table_type"
+	descriptionPropsKey  = "Description"
+
+	// Database location.
+	locationPropsKey = "Location"
+
+	// Table metadata location pointer.
+	metadataLocationPropsKey = "metadata_location"

Review Comment:
   Added the documentation to the pyiceberg.
   
   @nastra I will add `previous_metadata_location` in the next PR with the remaining ops (e.g. register_table) as it is used in them.



##########
catalog/glue.go:
##########
@@ -122,33 +149,93 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{

Review Comment:
   Added. Followed the Java pattern where we add catalog id in every request input. Event listener based approach from Python Iceberg might be ugly in Golang.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1804175030


##########
catalog/glue.go:
##########
@@ -122,33 +149,93 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{

Review Comment:
   does this not need a glue catalog id?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra merged PR #173:
URL: https://github.com/apache/iceberg-go/pull/173


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "natusioe (via GitHub)" <gi...@apache.org>.
natusioe commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1813036157


##########
catalog/glue.go:
##########
@@ -122,38 +165,104 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{
+		CatalogId:    c.catalogId,
+		DatabaseName: aws.String(database),
+		Name:         aws.String(tableName),
+	}
+	_, err = c.glueSvc.DeleteTable(ctx, params)
+	if err != nil {
+		return fmt.Errorf("failed to drop table %s.%s: %w", database, tableName, err)
+	}
+
+	return nil
 }
 
+// RenameTable renames an Iceberg table in the Glue catalog.
 func (c *GlueCatalog) RenameTable(ctx context.Context, from, to table.Identifier) (*table.Table, error) {
-	return nil, fmt.Errorf("%w: [Glue Catalog] rename table", iceberg.ErrNotImplemented)
-}
+	fromDatabase, fromTable, err := identifierToGlueTable(from)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
-	return fmt.Errorf("%w: [Glue Catalog] create namespace", iceberg.ErrNotImplemented)
-}
+	toDatabase, toTable, err := identifierToGlueTable(to)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) DropNamespace(ctx context.Context, namespace table.Identifier) error {

Review Comment:
   Updated to same order now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1804180423


##########
catalog/glue.go:
##########
@@ -122,33 +149,93 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{
+		DatabaseName: aws.String(database),
+		Name:         aws.String(tableName),
+	}
+	_, err = c.glueSvc.DeleteTable(ctx, params)
+	if err != nil {
+		return fmt.Errorf("failed to drop table %s.%s: %w", database, tableName, err)
+	}
+
+	return nil
 }
 
+// RenameTable renames an Iceberg table in the Glue catalog.
 func (c *GlueCatalog) RenameTable(ctx context.Context, from, to table.Identifier) (*table.Table, error) {
-	return nil, fmt.Errorf("%w: [Glue Catalog] rename table", iceberg.ErrNotImplemented)
-}
+	fromDatabase, fromTable, err := identifierToGlueTable(from)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
-	return fmt.Errorf("%w: [Glue Catalog] create namespace", iceberg.ErrNotImplemented)
-}
+	toDatabase, toTable, err := identifierToGlueTable(to)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) DropNamespace(ctx context.Context, namespace table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop namespace", iceberg.ErrNotImplemented)
-}
+	if fromDatabase != toDatabase {
+		return nil, fmt.Errorf("cannot rename table across namespaces: %s -> %s", fromDatabase, toDatabase)
+	}
 
-func (c *GlueCatalog) LoadNamespaceProperties(ctx context.Context, namespace table.Identifier) (iceberg.Properties, error) {
-	return nil, fmt.Errorf("%w: [Glue Catalog] load namespace properties", iceberg.ErrNotImplemented)
-}
+	// Fetch the existing Glue table to copy the metadata into the new table.
+	fromGlueTable, err := c.getTable(ctx, fromDatabase, fromTable)
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch the table %s.%s: %w", fromDatabase, fromTable, err)
+	}
 
-func (c *GlueCatalog) UpdateNamespaceProperties(ctx context.Context, namespace table.Identifier,
-	removals []string, updates iceberg.Properties) (PropertiesUpdateSummary, error) {
-	return PropertiesUpdateSummary{}, fmt.Errorf("%w: [Glue Catalog] update namespace properties", iceberg.ErrNotImplemented)
+	// Create the new table.
+	_, err = c.glueSvc.CreateTable(ctx, &glue.CreateTableInput{

Review Comment:
   I believe this is also missing the glue catalog id?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1804182610


##########
catalog/glue.go:
##########
@@ -180,8 +267,155 @@ func (c *GlueCatalog) ListNamespaces(ctx context.Context, parent table.Identifie
 	return icebergNamespaces, nil
 }
 
+// CreateNamespace creates a new Iceberg namespace in the Glue catalog.
+func (c *GlueCatalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
+	database, err := identifierToGlueDatabase(namespace)
+	if err != nil {
+		return err
+	}
+
+	databaseParameters := map[string]string{
+		databaseTypePropsKey: glueTypeIceberg,
+	}
+
+	description := props[descriptionPropsKey]
+	locationURI := props[locationPropsKey]
+
+	if description != "" {
+		databaseParameters[descriptionPropsKey] = description
+	}
+	if locationURI != "" {
+		databaseParameters[locationPropsKey] = locationURI
+	}
+
+	databaseInput := &types.DatabaseInput{
+		Name:       aws.String(database),
+		Parameters: databaseParameters,
+	}
+
+	params := &glue.CreateDatabaseInput{DatabaseInput: databaseInput}
+	_, err = c.glueSvc.CreateDatabase(ctx, params)

Review Comment:
   this is also missing a glue catalog id



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1804185400


##########
catalog/glue.go:
##########
@@ -30,22 +30,40 @@ import (
 	"github.com/aws/aws-sdk-go-v2/service/glue/types"
 )
 
-const glueTypeIceberg = "ICEBERG"
+const (
+	glueTypeIceberg      = "ICEBERG"
+	databaseTypePropsKey = "database_type"
+	tableTypePropsKey    = "table_type"
+	descriptionPropsKey  = "Description"
+
+	// Database location.
+	locationPropsKey = "Location"
+
+	// Table metadata location pointer.
+	metadataLocationPropsKey = "metadata_location"

Review Comment:
   I think you also need to set `previous_metadata_location` when committing: See https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/glue.py#L142-L145



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1803611627


##########
catalog/glue.go:
##########
@@ -30,22 +30,40 @@ import (
 	"github.com/aws/aws-sdk-go-v2/service/glue/types"
 )
 
-const glueTypeIceberg = "ICEBERG"
+const (
+	glueTypeIceberg      = "ICEBERG"
+	databaseTypePropsKey = "database_type"
+	tableTypePropsKey    = "table_type"
+	descriptionPropsKey  = "Description"
+
+	// Database location.
+	locationPropsKey = "Location"
+
+	// Table metadata location pointer.
+	metadataLocationPropsKey = "metadata_location"

Review Comment:
   is there documentation that these came from? Can we link to the source documentation that these came from?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1812088531


##########
catalog/glue.go:
##########
@@ -122,38 +165,104 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{
+		CatalogId:    c.catalogId,
+		DatabaseName: aws.String(database),
+		Name:         aws.String(tableName),
+	}
+	_, err = c.glueSvc.DeleteTable(ctx, params)
+	if err != nil {
+		return fmt.Errorf("failed to drop table %s.%s: %w", database, tableName, err)
+	}
+
+	return nil
 }
 
+// RenameTable renames an Iceberg table in the Glue catalog.
 func (c *GlueCatalog) RenameTable(ctx context.Context, from, to table.Identifier) (*table.Table, error) {
-	return nil, fmt.Errorf("%w: [Glue Catalog] rename table", iceberg.ErrNotImplemented)
-}
+	fromDatabase, fromTable, err := identifierToGlueTable(from)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
-	return fmt.Errorf("%w: [Glue Catalog] create namespace", iceberg.ErrNotImplemented)
-}
+	toDatabase, toTable, err := identifierToGlueTable(to)
+	if err != nil {
+		return nil, err
+	}
 
-func (c *GlueCatalog) DropNamespace(ctx context.Context, namespace table.Identifier) error {

Review Comment:
   moving methods around makes this quite difficult to review. I'd suggest to just add table-related methods after the namespace-related methods



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1804177804


##########
catalog/glue.go:
##########
@@ -122,33 +149,93 @@ func (c *GlueCatalog) LoadTable(ctx context.Context, identifier table.Identifier
 	return icebergTable, nil
 }
 
-func (c *GlueCatalog) CatalogType() CatalogType {
-	return Glue
-}
-
+// DropTable deletes an Iceberg table from the Glue catalog.
 func (c *GlueCatalog) DropTable(ctx context.Context, identifier table.Identifier) error {
-	return fmt.Errorf("%w: [Glue Catalog] drop table", iceberg.ErrNotImplemented)
+	database, tableName, err := identifierToGlueTable(identifier)
+	if err != nil {
+		return err
+	}
+
+	// Check if the table exists and is an Iceberg table.
+	_, err = c.getTable(ctx, database, tableName)
+	if err != nil {
+		return err
+	}
+
+	params := &glue.DeleteTableInput{

Review Comment:
   a glue catalog id is used in pyiceberg and in the java implementation, so I would have assumed that we're using it for the go client too



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(catalog/glue): add support for glue catalog namespace operations [iceberg-go]

Posted by "natusioe (via GitHub)" <gi...@apache.org>.
natusioe commented on code in PR #173:
URL: https://github.com/apache/iceberg-go/pull/173#discussion_r1803623754


##########
catalog/glue.go:
##########
@@ -30,22 +30,40 @@ import (
 	"github.com/aws/aws-sdk-go-v2/service/glue/types"
 )
 
-const glueTypeIceberg = "ICEBERG"
+const (
+	glueTypeIceberg      = "ICEBERG"
+	databaseTypePropsKey = "database_type"
+	tableTypePropsKey    = "table_type"
+	descriptionPropsKey  = "Description"
+
+	// Database location.
+	locationPropsKey = "Location"
+
+	// Table metadata location pointer.
+	metadataLocationPropsKey = "metadata_location"

Review Comment:
   I could not find the official spec but followed the same convention used by the PyIceberg [here](https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/__init__.py#L88). Let me know if this works.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org