You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "adutra (via GitHub)" <gi...@apache.org> on 2025/01/22 13:42:12 UTC
[PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
adutra opened a new pull request, #841:
URL: https://github.com/apache/polaris/pull/841
<!--
Possible security vulnerabilities: STOP here and contact security@apache.org instead!
Please update the title of the PR with a meaningful message - do not leave it "empty" or "generated"
Please update this summary field:
The summary should cover these topics, if applicable:
* the motivation for the change
* a description of the status quo, for example the current behavior
* the desired behavior
* etc
PR checklist:
- Do a self-review of your code before opening a pull request
- Make sure that there's good test coverage for the changes included in this PR
- Run tests locally before pushing a PR (./gradlew check)
- Code should have comments where applicable. Particularly hard-to-understand
areas deserve good in-line documentation.
- Include changes and enhancements to the documentation (in site/content/in-dev/unreleased)
- For Work In Progress Pull Requests, please use the Draft PR feature.
Make sure to add the information BELOW this comment.
Everything in this comment will NOT be added to the PR description.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926733864
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
Review Comment:
Yes I think it is probably safer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927614990
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,180 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Configuring Polaris for Production
-## Security
-
-### Configurations
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
Notable configuration used to secure a Polaris deployment are outlined below.
-#### oauth2
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+### OAuth2
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two implementations are
+supported out of the box:
-#### authenticator.tokenBroker
+- [rsa-key-pair] uses a pair of public and private keys;
+- [symmetric-key] uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+[rsa-key-pair]: https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTRSAKeyPairFactory.java
+[symmetric-key]: https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTSymmetricKeyFactory.java
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-## Metastore Management
+> [!IMPORTANT]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+Alternatively, you can use a symmetric key by setting the following properties:
-### Configuring EclipseLink
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
Review Comment:
Yes that's possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925743884
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
Review Comment:
This section is a great idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926194134
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
Review Comment:
Does it mean the bootstrap will fail if the realm exists already? If that's so, we may not mention it or just say you cannot bootstrap an existing realm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926194134
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
Review Comment:
Does it mean the bootstrap will fail if the realm exists already? If that's so, we may not mention it as the bootstrap tool will return an error or just say you cannot bootstrap an existing realm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on PR #841:
URL: https://github.com/apache/polaris/pull/841#issuecomment-2610966823
@flyrain this should be good to go now, let me know if I messed anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926713664
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
Review Comment:
I just checked and indeed it currently fails, and quite inelegantly. I will raise a PR shortly to improve the user experience.
In any case, this paragraph will disappear from this file since we agreed above to only mention option 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925740977
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
Review Comment:
It's not true that two types are supported; just that two implementations are provided out of the box.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925737581
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
Review Comment:
Is this intended to be the top-level header? If so, why `Tuning` instead of `Configuring`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926725101
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
Review Comment:
Could you please be more specific about what's wrong with the style? Not being a native English speaker, I'm struggling to figure out the issue here. I asked ChatGPT to improve it, I hope the result sounds better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926153747
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
+appropriate request header to the `curl` command, for example:
Review Comment:
Missing of realm header will be resolved to the first realm in the configuration `polaris.realm-context.realms`, right?
```suggestion
appropriate request header to the `curl` command, otherwise the Polaris will resolve the first realm in the configuration `polaris.realm-context.realms`, here is an example to set realm header:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925743008
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
Review Comment:
nit: style
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926197487
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
Review Comment:
```suggestion
must be performed **only once** for each realm in order to prepare the metastore manager to integrate with Polaris.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927567594
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
Review Comment:
Of course, sorry for being unclear! Sometimes it's hard to articulate exactly what comes across as unnatural. I am a native speaker but not a trained copywriter or editor :)
In this specific case I think the comma after `name` was unnecessary, and also the clause `a realm name that is not the default realm name` struck me as unwieldy.
The new version looks good -- and in either case the important thing is that all the information is there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "sfc-gh-emaynard (via GitHub)" <gi...@apache.org>.
sfc-gh-emaynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927560773
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
Review Comment:
Yeah absolutely. Something like `Sensitive catalog metadata and user information` maybe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925745714
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
Review Comment:
nit: style
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926260321
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
Review Comment:
Given we've already link to the doc `metastores.md`, I'd recommend to provide an example like the following, instead of descriptions here. But it isn't blocker for me. Let's update the doc ASAP. We can fix it in another PR.
```
./gradlew clean :polaris-quarkus-server:assemble -PeclipseLinkDeps=org.postgresql:postgresql:42.7.4
java -Dpolaris.persistence.type=eclipse-link \
-Dpolaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml \
-Dpolaris.persistence.eclipselink.persistence-unit=polaris \
-jar quarkus/server/build/quarkus-app/quarkus-run.jar
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926199376
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
Review Comment:
Does it make sense to ask production users to always provide the credentials instead of providing two options(auto generated or provided)? This isn't a blocker though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925741816
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
Review Comment:
This is a great callout.
I do fear that the website doesn't render `[!WARNING]` blocks correctly though
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925740501
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
Review Comment:
If we want readers to identify an implementation of an interface, linking to it (or its docs) seems prudent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926196616
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
Review Comment:
Does it make sense to always ask users to use option 1? We don't have to mention option 2 in that case. I think this will simplify usage, esp. for the first time users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926699001
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
Review Comment:
It still store the hashes, right? How about "it will be storing sensitive data and catalog metadata"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927543724
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,180 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Configuring Polaris for Production
-## Security
-
-### Configurations
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
Notable configuration used to secure a Polaris deployment are outlined below.
-#### oauth2
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+### OAuth2
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two implementations are
+supported out of the box:
-#### authenticator.tokenBroker
+- [rsa-key-pair] uses a pair of public and private keys;
+- [symmetric-key] uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+[rsa-key-pair]: https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTRSAKeyPairFactory.java
+[symmetric-key]: https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTSymmetricKeyFactory.java
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-## Metastore Management
+> [!IMPORTANT]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
Review Comment:
Can we mention how a user gets these key files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on PR #841:
URL: https://github.com/apache/polaris/pull/841#issuecomment-2610984241
Thanks a lot for the fix. @adutra. Hi @eric-maynard, can we merge it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925738058
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
Review Comment:
`configuration options` seems redundant
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "sfc-gh-ygu (via GitHub)" <gi...@apache.org>.
sfc-gh-ygu commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926080505
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
Review Comment:
I also think `Configuring` is more suitable here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925344446
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the [Configuration]({{% ref "configuration.md" %}}) page.
Review Comment:
Note: this page doesn't exist yet, I'm working on a PR for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926738958
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
Review Comment:
I will put the links back, although I'm generally not a fan of linking to source code. Users should know everything they need to know by reading the docs only. The latent issue with links to source code is that you either link to a blob, but then the blob will become outdated as time passes; or you link to the tip of a branch like `main`, but you risk having your link return 404 if the class is moved or deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927454653
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
Review Comment:
That would definitely be better imho. But do we publish javadocs somewhere? I don't recall.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "sfc-gh-emaynard (via GitHub)" <gi...@apache.org>.
sfc-gh-emaynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927560773
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
Review Comment:
Yeah absolutely. Something like `Sensitive catalog metadata and user information` maybe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927516129
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
Review Comment:
Page now exists, I will update this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926691956
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
Review Comment:
I went for "metastore", hope that's OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926696839
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
Review Comment:
Hmm actually it seems that `[!IMPORTANT]` is supported, I will use that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926675368
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
Review Comment:
Sorry, I don't remember why I changed this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925743552
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
Review Comment:
nit: Are we using `metastore` or `Metastore Manager`? Let's pick one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926153747
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
+appropriate request header to the `curl` command, for example:
Review Comment:
Missing of realm header will be resolved to the first realm in the configuration `polaris.realm-context.realms`, right?
```suggestion
appropriate request header to the `curl` command, otherwise Polaris will resolve to the first realm in the configuration `polaris.realm-context.realms`. Here is an example to set realm header:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925753433
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
Review Comment:
nit: style
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925739338
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
Review Comment:
I feel that this change implies other sections are _not_ related to security, which could be misleading. For example, bootstrap credentials are at least tangentially related to security.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927517604
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
Review Comment:
True. I guess we'll have to use blob links for now and revisit this later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927439566
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
Review Comment:
We could link to the javadocs if that feels better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927544825
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,180 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Configuring Polaris for Production
-## Security
-
-### Configurations
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
Notable configuration used to secure a Polaris deployment are outlined below.
-#### oauth2
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+### OAuth2
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two implementations are
+supported out of the box:
-#### authenticator.tokenBroker
+- [rsa-key-pair] uses a pair of public and private keys;
+- [symmetric-key] uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+[rsa-key-pair]: https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTRSAKeyPairFactory.java
+[symmetric-key]: https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTSymmetricKeyFactory.java
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-## Metastore Management
+> [!IMPORTANT]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+Alternatively, you can use a symmetric key by setting the following properties:
-### Configuring EclipseLink
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
Review Comment:
Is it possible to get the secrets from env variables?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927625709
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the [Configuration]({{% ref "configuration.md" %}}) page.
Review Comment:
The link is finally available, FYI. I changed the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra merged PR #841:
URL: https://github.com/apache/polaris/pull/841
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925385220
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
Review Comment:
Page doesn't exist yet , so I cannot include a link for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926153747
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
+appropriate request header to the `curl` command, for example:
Review Comment:
Missing of realm header will be resolved to the first realm in the configuration `polaris.realm-context.realms`, right?
```suggestion
appropriate request header to the `curl` command, otherwise the Polaris will resolve to the first realm in the configuration `polaris.realm-context.realms`. Here is an example to set realm header:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926153747
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
+
+The first approach is recommended as it is more flexible (you can bootstrap new realms at any time
+without incurring in a Polaris downtime). The second approach is useful for testing and development,
+but can be used in production as well.
+
+In order to bootstrap root credentials for a realm named `my-realm` when deploying Polaris, set the
+following environment variables:
+
+```
+export POLARIS_BOOTSTRAP_CREDENTIALS=my-realm,my-client-id,my-client-secret
```
+If the realm hasn't been bootstrapped yet, Polaris will create the realm and the `root` principal
+with the provided credentials upon first usage of that realm. If the realm already exists, Polaris
+will not attempt to update the `root` principal credentials.
+
You can verify the setup by attempting a token issue for the `root` principal:
```bash
-curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens -d "grant_type=client_credentials&client_id=my-client-id&client_secret=my-client-secret&scope=PRINCIPAL_ROLE:ALL"
+curl -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
+ -d "grant_type=client_credentials" \
+ -d "client_id=my-client-id" \
+ -d "client_secret=my-client-secret" \
+ -d "scope=PRINCIPAL_ROLE:ALL"
```
-which should return:
+Which should return an access token:
```json
-{"access_token":"...","token_type":"bearer","issued_token_type":"urn:ietf:params:oauth:token-type:access_token","expires_in":3600}
+{
+ "access_token": "...",
+ "token_type": "bearer",
+ "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
+ "expires_in": 3600
+}
```
-Note that if you used non-default realm name, for example, `iceberg` instead of `default-realm` in your `polaris-server.yml`, then you should add an appropriate request header:
+Note that if you used a realm name that is not the default realm name, then you should add an
+appropriate request header to the `curl` command, for example:
Review Comment:
Missing of realm header will be resolved to the first realm in the configuration `polaris.realm-context.realms`, right?
```suggestion
appropriate request header to the `curl` command, otherwise Polaris will resolve the realm to the first one in the configuration `polaris.realm-context.realms`. Here is an example to set realm header:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1925744935
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
Review Comment:
One one hand, this is not strictly true as it won't store credentials (currently).
On the other hand, this is a very important callout and I worry we are burying the lede by sticking it at the end here. Maybe `[!IMPORTANT]` it or something like that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926685356
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
Review Comment:
Indeed I don't know where I took this from 😅 Let me replace by bold characters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927556956
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
Review Comment:
Let's circulate this idea and make the change later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927460674
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
Review Comment:
I thought we had it on the webpage, but it looks like we don't. Bummer.
To solve the issue with a 404, we could link to either a commit or a release branch.
But also, if main changes to a degree that the interface changes, then the docs should be updated at that time as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1926704780
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+
+### Bootstrapping
+
+Before using Polaris, you must **bootstrap** the metastore manager. This is a manual operation that
+must be performed **only once** in order to prepare the metastore manager to integrate with Polaris.
+
+By default, when bootstrapping a new realm, Polaris will create randomised `CLIENT_ID` and
+`CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend.
+
+Depending on your database, this may not be convenient as the generated credentials are not stored
+in clear text in the database.
+
+In order to provide your own credentials for `root` principal (so you can request tokens via
+`api/catalog/v1/oauth/tokens`), there are two approaches:
+
+1. Use the Polaris Admin Tool to bootstrap the realm and set its `root`
+ principal credentials.
+2. Set the `root` principal credentials when deploying Polaris for the first time.
Review Comment:
I tend to agree, although I remember that the Helm chart currently has a heavy machinery that allows users to go with option 2. But let's remove option 1 from docs and figure out the Helm chart later (big PR coming...).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs: adapt configuring-polaris-for-production.md to Quarkus [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #841:
URL: https://github.com/apache/polaris/pull/841#discussion_r1927561483
##########
site/content/in-dev/unreleased/configuring-polaris-for-production.md:
##########
@@ -23,114 +23,179 @@ type: docs
weight: 600
---
-The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+## Tuning Polaris for Production
-## Security
+The default server configuration is intended for development and testing. When deploying Polaris in
+production, there are several best practices to keep in mind.
-### Configurations
+Notable configuration options used to secure a Polaris deployment are outlined below.
-Notable configuration used to secure a Polaris deployment are outlined below.
+For more information on how to configure Polaris and what configuration options are available,
+refer to the Configuration Reference page.
-#### oauth2
+### Security
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
+Notable configuration options used to secure a Polaris deployment are outlined below.
-* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
-* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/apache/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
+Polaris authentication requires specifying a token broker factory type. Two types are supported:
-#### authenticator.tokenBroker
+- `rsa-key-pair` uses a pair of public and private keys;
+- `symmetric-key` uses a shared secret.
-> [!WARNING]
-> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
+By default, Polaris uses `rsa-key-pair`, with randomly generated keys.
-#### callContextResolver & realmIdResolver
-* Use these configurations to specify a service that can resolve a realm from bearer tokens.
-* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/apache/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/apache/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
+> [!WARNING]
+> The default `rsa-key-pair` configuration is not suitable when deploying many replicas of Polaris,
+> as each replica will have its own set of keys. This will cause token validation to fail when a
+> request is routed to a different replica than the one that issued the token.
-## Metastore Management
+It is highly recommended to configure Polaris with previously-generated RSA keys. This can be done
+by setting the following properties:
-> [!IMPORTANT]
-> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
+```properties
+polaris.authentication.token-broker.type=rsa-key-pair
+polaris.authentication.token-broker.rsa-key-pair.public-key-file=/tmp/public.key
+polaris.authentication.token-broker.rsa-key-pair.private-key-file=/tmp/private.key
+```
-A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/apache/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
+Alternatively, you can use a symmetric key by setting the following properties:
-Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
+```properties
+polaris.authentication.token-broker.type=symmetric-key
+polaris.authentication.token-broker.symmetric-key.file=/tmp/symmetric.key
+```
-### Configuring EclipseLink
+Note: it is also possible to set the symmetric key secret directly in the configuration file, but
+that is not recommended for production use, as the secret is stored in plain text:
-To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}).
+```properties
+polaris.authentication.token-broker.symmetric-key.secret=my-secret
+```
-> [!IMPORTANT]
-> EclipseLink requires
-> 1. Building the JAR for the EclipseLink extension
-> 2. Setting the `eclipseLink` gradle property to `true`.
->
-> This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble`
+Finally, you can also configure the token broker to use a maximum lifespan by setting the following
+property:
-### Bootstrapping
+```properties
+polaris.authentication.token-broker.max-token-generation=PT1H
+```
-Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
+### Realm Id Resolver
-By default, Polaris will create randomised `CLIENT_ID` and `CLIENT_SECRET` for the `root` principal and store their hashes in the metastore backend. In order to provide your own credentials for `root` principal (so you can request tokens via `api/catalog/v1/oauth/tokens`), set the `POLARIS_BOOTSTRAP_CREDENTIALS` environment variable as follows:
+By default, Polaris resolves realms based on incoming request headers. You can configure the realm
+context resolver by setting the following properties in `application.properties`:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret
+```properties
+polaris.realm-context.realms=POLARIS,MY-REALM
+polaris.realm-context.header-name=Polaris-Realm
```
-The format of the environment variable is `realm,client_id,client_secret`. You can provide multiple credentials separated by `;`. For example, to provide credentials for two realms `my_realm` and `my_realm2`:
+Where:
-```
-export POLARIS_BOOTSTRAP_CREDENTIALS=my_realm,my-client-id,my-client-secret;my_realm2,my-client-id2,my-client-secret2
-```
+- `realms` is a comma-separated list of allowed realms. This setting _must_ be correctly configured.
+ At least one realm must be specified.
+- `header-name` is the name of the header used to resolve the realm; by default, it is
+ `Polaris-Realm`.
-It is also possible to use system properties to provide the credentials:
+If a request does not contain the specified header, Polaris will use the first realm in the list as
+the default realm. In the above example, `POLARIS` is the default realm.
-```
-java -Dpolaris.bootstrap.credentials=my_realm,my-client-id,my-client-secret -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+### Metastore Configuration
-Now, to bootstrap Polaris, run:
+A Metastore Manager should be configured with an implementation that durably persists Polaris
+entities. By default, Polaris uses an in-memory metastore.
-```bash
-java -jar /path/to/jar/polaris-service-all.jar bootstrap polaris-server.yml
-```
+> [!WARNING]
+> The default in-memory metastore is not suitable for production use, as it will lose all data
+> when the server is restarted; it is also unusable when multiple Polaris replicas are used.
-or in a container:
+To use a durable Metastore Manager, you need to switch to the EclipseLink metastore, and provide
+your own `persistence.xml` file. This file contains details of the database used for metastore
+management and the connection settings. For more information, refer to the [metastore
+documentation]({{% ref "metastores" %}}).
-```bash
-bin/polaris-service bootstrap config/polaris-server.yml
+Then, configure Polaris to use your metastore by setting the following properties:
+
+```properties
+polaris.persistence.type=eclipse-link
+polaris.persistence.eclipselink.configuration-file=/path/to/persistence.xml
+polaris.persistence.eclipselink.persistence-unit=polaris
```
-Afterward, Polaris can be launched normally:
+Where:
-```bash
-java -jar /path/to/jar/polaris-service-all.jar server polaris-server.yml
+- `polaris.persistence.type` indicates that we are using the EclipseLink metastore.
+- `polaris.persistence.eclipselink.configuration-file` is the path to the `persistence.xml` file.
+- `polaris.persistence.eclipselink.persistence-unit` is the name of the persistence unit to use (in
+ case the configuration file has many persistence units).
+
+Typically, in Kubernetes, you would define the `persistence.xml` file as a `ConfigMap` and set the
+`polaris.persistence.eclipselink.configuration-file` property to the path of the mounted file in
+the container.
+
+Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
Review Comment:
Yeah absolutely. Something like `...storing catalog metadata and user information` maybe. "Sensitive" is a good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org