You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "goutamadwant (via GitHub)" <gi...@apache.org> on 2026/08/20 04:18:11 UTC
[PR] [Feature][Connector-V2][Hive] Support AWS Glue Data Catalog metastore [seatunnel]
goutamadwant opened a new pull request, #11895:
URL: https://github.com/apache/seatunnel/pull/11895
### Purpose of this pull request
Closes #9856.
- Allow Hive sink configuration without `metastore_uri` when `hive.metastore.client.factory.class` is configured.
- Create the metastore client through the standard Hive client factory contract.
- Load factory and catalog properties from `hive_site_path`, `hive.hadoop.conf-path`, or `hive.hadoop.conf`.
- Keep the existing Thrift metastore path unchanged.
- Fail during catalog initialization when neither route is configured or the factory classes are unavailable.
- Document AWS Glue Data Catalog setup in English and Chinese.
### Does this PR introduce _any_ user-facing change?
Yes.
Hive sink can now use AWS Glue Data Catalog, or another compatible Hive metastore client factory, without a Thrift `metastore_uri`.
The configured client factory and its compatible Hive runtime classes must be available on the SeaTunnel runtime classpath. Authentication remains with the client implementation. The AWS Glue client uses the AWS SDK default credential provider chain.
Existing `metastore_uri` configurations continue to use the current Thrift client path.
### How was this patch tested?
Added coverage for inline factory configuration, `hive-site.xml` configuration, factory precedence over a Thrift URI, missing factory classes, missing metastore configuration, and Hive sink option exposure.
Focused tests:
```shell
./mvnw -pl seatunnel-connectors-v2/connector-hive -Dtest=HiveMetaStoreCatalogClientFactoryTest,HiveSinkFactoryTest test
```
The full Hive connector package was also verified on Java 11. All 98 tests passed.
### Check list
* [x] No new Jar binary package is added.
* [x] English and Chinese Hive sink documentation is updated.
* [x] No incompatible change is introduced.
* [x] This extends the existing Hive connector, so connector registration and distribution files do not need changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][Connector-V2][Hive] Support AWS Glue Data Catalog metastore [seatunnel]
Posted by "goutamadwant (via GitHub)" <gi...@apache.org>.
goutamadwant commented on PR #11895:
URL: https://github.com/apache/seatunnel/pull/11895#issuecomment-5474318831
@DanielLeens Rechecked the factory signature against the AWS Glue Hive 3 client source createMetaStoreClient uses primitive boolean so the current reflection contract is correct
added Javadocs for the factory contract and classloader lookup in e038d31d9 spotless and the 5 focused factory tests pass
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][Connector-V2][Hive] Support AWS Glue Data Catalog metastore [seatunnel]
Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11895:
URL: https://github.com/apache/seatunnel/pull/11895
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][Connector-V2][Hive] Support AWS Glue Data Catalog metastore [seatunnel]
Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11895:
URL: https://github.com/apache/seatunnel/pull/11895#issuecomment-5488587191
Quick CI follow-up on my review above: at the time, CI on this head was still queued/pending and I asked to confirm it came back green before merge.
Re-checked live: `Build` (run `99392999719`) now shows `pass`, along with `Notify test workflow` and `labeler`. No functional concerns remain from my side — a write-capable maintainer can take the final approve/merge pass whenever convenient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature][Connector-V2][Hive] Support AWS Glue Data Catalog metastore [seatunnel]
Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11895:
URL: https://github.com/apache/seatunnel/pull/11895#issuecomment-5480621137
CI update: at the time of my last review (2026-08-31T10:23:51Z) the `Build` check on head `e038d31d9` was still queued/pending, so I asked to make sure it came back green before merge. It has since completed: `Build` finished with conclusion `success` at 2026-08-31T13:50:14Z (https://github.com/apache/seatunnel/runs/99392999719). I have no other open functional concerns on this head — my prior blocker (the reflection-signature Issue 1) was already independently verified against the real AWS Glue source and resolved in that same review. I have comment-only review rights on this repo, so a write-capable maintainer can go ahead and finalize the approve/merge now that CI is green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org