You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "pingtimeout (via GitHub)" <gi...@apache.org> on 2025/04/01 15:34:24 UTC
[PR] Add Polaris benchmarks [polaris-tools]
pingtimeout opened a new pull request, #2:
URL: https://github.com/apache/polaris-tools/pull/2
This pull request introduces a comprehensive set of benchmarks to Polaris. The current set includes:
* A benchmark that populates an empty Polaris server with a dataset that have predefined attributes
* A benchmark that issues only read queries over that dataset
* A benchmark that issues read and write queries (entity updates) over that dataset, with a configurable read/write ratio
* A benchmark that creates commits against existing tables
A documentation if provided in the README.md file, including examples of the different datasets that can be generated. The datasets are procedural, which means that given the same input parameters, the same datasets will be generated, thus enabling reproducible benchmarks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023374728
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
I mean specifically the Authentication Actions, I didn't understand where that fit in the Write/Read benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023437257
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/util/CircularIterator.scala:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.util
+
+import scala.util.Random
+
+class CircularIterator[T](builder: () => Iterator[T]) extends Iterator[T] {
+ private var currentIterator: Iterator[T] = builder()
+
+ override def hasNext: Boolean = true
+
+ override def next(): T = synchronized {
+ if (!currentIterator.hasNext) {
+ currentIterator = builder()
+ }
+ currentIterator.next()
+ }
+}
+
+class BufferedRandomIterator[T](underlying: CircularIterator[T], bufferSize: Int)
+ extends Iterator[T] {
+ private var buffer: Iterator[T] = populateAndShuffle()
+
+ private def populateAndShuffle(): Iterator[T] =
+ Random.shuffle((1 to bufferSize).map(_ => underlying.next()).toList).iterator
Review Comment:
Do we need to seed this? not sure if we were doing determinism for the iterators here as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023373433
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
+ max-views = -1 # Maximum views (-1 for unlimited)
+}
+```
+
+### Connection Parameters
+
+Connection settings are configured under `http` and `auth`:
+
+```hocon
+http {
+ base-url = "http://localhost:8181" # Service URL
+}
+
+auth {
+ client-id = null # Required: OAuth2 client ID
+ client-secret = null # Required: OAuth2 client secret
+}
+```
+
+### Workload Parameters
+
+Workload settings are configured under `workload`:
+
+```hocon
+workload {
+ read-write-ratio = 0.8 # Ratio of reads (0.0-1.0)
+}
+```
+
+## Running the Benchmarks
+
+The benchmark uses [typesafe-config](https://github.com/lightbend/config) for configuration management. Default settings are in `src/gatling/resources/benchmark-defaults.conf`. This file should not be modified directly.
+
+To customize the benchmark settings, create your own `application.conf` file and specify it using the `-Dconfig.file` parameter. Your settings will override the default values.
+
+Example `application.conf`:
+```hocon
+auth {
+ client-id = "your-client-id"
+ client-secret = "your-client-secret"
+}
+
+http {
+ base-url = "http://your-polaris-instance:8181"
+}
+
+workload {
+ read-write-ratio = 0.8
+}
+```
+
+Run benchmarks with your configuration:
+
+```bash
+# Sequential dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential \
+ -Dconfig.file=./application.conf
+
+# Concurrent dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent \
+ -Dconfig.file=./application.conf
+```
+
+A message will show the location of the Gatling report:
+```
+Reports generated in: ./benchmarks/build/reports/gatling/<simulation-name>/index.html
+```
+
+### Example Polaris server startup
+
+For repeated testing and benchmarking purposes it's convenient to have fixed client-ID + client-secret combinations. **The following example is ONLY for testing and benchmarking against an airgapped Polaris instance**
+
+```bash
+# Start Polaris with the fixed client-ID/secret admin/admin
+# DO NEVER EVER USE THE FOLLOWING FOR ANY NON-AIRGAPPED POLARIS INSTANCE !!
+./gradlew :polaris-quarkus-server:quarkusBuild && java \
+ -Dpolaris.bootstrap.credentials=POLARIS,admin,admin \
+ -Djava.security.manager=allow \
+ -jar quarkus/server/build/quarkus-app/quarkus-run.jar
+```
+
+With the above you can run the benchmarks using a configuration file with `client-id = "admin"` and `client-secret = "admin"` - meant only for convenience in a fully airgapped system.
+
+# Test Dataset
+
+The benchmarks use synthetic procedural datasets that are generated deterministically at runtime. This means that given the same input parameters, the exact same dataset structure will always be generated. This approach allows generating large volumes of test data without having to store it, while ensuring reproducible benchmark results across different runs.
+
+The diagrams below describe the data sets that are used in benchmarks. Note that the benchmark dataset may not cover all Polaris features.
+
+## Generation rules
+
+The dataset has a tree shape. At the root of the tree is a Polaris realm that must exist before the dataset is created.
+
+An arbitrary number of catalogs can be created under the realm. However, only the first catalog (`C_0`) is used for the rest of the dataset.
Review Comment:
Ah nvm, I missed the previous sentence and was like where did realm come from :) My bad
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023137956
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
+ max-views = -1 # Maximum views (-1 for unlimited)
+}
+```
+
+### Connection Parameters
+
+Connection settings are configured under `http` and `auth`:
+
+```hocon
+http {
+ base-url = "http://localhost:8181" # Service URL
+}
+
+auth {
+ client-id = null # Required: OAuth2 client ID
+ client-secret = null # Required: OAuth2 client secret
+}
+```
+
+### Workload Parameters
+
+Workload settings are configured under `workload`:
+
+```hocon
+workload {
+ read-write-ratio = 0.8 # Ratio of reads (0.0-1.0)
+}
+```
+
+## Running the Benchmarks
+
+The benchmark uses [typesafe-config](https://github.com/lightbend/config) for configuration management. Default settings are in `src/gatling/resources/benchmark-defaults.conf`. This file should not be modified directly.
+
+To customize the benchmark settings, create your own `application.conf` file and specify it using the `-Dconfig.file` parameter. Your settings will override the default values.
+
+Example `application.conf`:
+```hocon
+auth {
+ client-id = "your-client-id"
+ client-secret = "your-client-secret"
+}
+
+http {
+ base-url = "http://your-polaris-instance:8181"
+}
+
+workload {
+ read-write-ratio = 0.8
+}
+```
+
+Run benchmarks with your configuration:
+
+```bash
+# Sequential dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential \
+ -Dconfig.file=./application.conf
+
+# Concurrent dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent \
+ -Dconfig.file=./application.conf
+```
+
+A message will show the location of the Gatling report:
+```
+Reports generated in: ./benchmarks/build/reports/gatling/<simulation-name>/index.html
+```
+
+### Example Polaris server startup
+
+For repeated testing and benchmarking purposes it's convenient to have fixed client-ID + client-secret combinations. **The following example is ONLY for testing and benchmarking against an airgapped Polaris instance**
+
+```bash
+# Start Polaris with the fixed client-ID/secret admin/admin
+# DO NEVER EVER USE THE FOLLOWING FOR ANY NON-AIRGAPPED POLARIS INSTANCE !!
+./gradlew :polaris-quarkus-server:quarkusBuild && java \
+ -Dpolaris.bootstrap.credentials=POLARIS,admin,admin \
+ -Djava.security.manager=allow \
+ -jar quarkus/server/build/quarkus-app/quarkus-run.jar
+```
+
+With the above you can run the benchmarks using a configuration file with `client-id = "admin"` and `client-secret = "admin"` - meant only for convenience in a fully airgapped system.
+
+# Test Dataset
+
+The benchmarks use synthetic procedural datasets that are generated deterministically at runtime. This means that given the same input parameters, the exact same dataset structure will always be generated. This approach allows generating large volumes of test data without having to store it, while ensuring reproducible benchmark results across different runs.
+
+The diagrams below describe the data sets that are used in benchmarks. Note that the benchmark dataset may not cover all Polaris features.
+
+## Generation rules
+
+The dataset has a tree shape. At the root of the tree is a Polaris realm that must exist before the dataset is created.
+
+An arbitrary number of catalogs can be created under the realm. However, only the first catalog (`C_0`) is used for the rest of the dataset.
+
+The namespaces part of the dataset is a complete `N`-ary tree. That is, it starts with a root namespace (`NS_0`) and then, each namespace contains exactly `0` or `N` children namespaces. The width as well as the depth of the namespaces tree are configurable. The total number of namespaces can easily be calculated with the following formulae, where `N` is the tree arity and `D` is the total tree depth, including the root:
Review Comment:
just a note, I believe you described N as width previously and not arity? Or am I mistaking this? Just wondering if we should change this to also say width for consistency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2024072834
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
+ }
+}
+
+dependencies {
+ gatling("com.typesafe.play:play-json_2.13:2.9.4")
Review Comment:
I have introduced `toml` in #1, once we merge it we need to rebase this PR and use it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023170019
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
Is this part of the benchmarks listed in the config? I don't remember seeing a doc in the readme for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#issuecomment-2771632609
I will wait for #1 to be merged as it contains the base build directives that could be leveraged in this PR. I might have to modify these base Gradle files though, in order to enable Scala projects. It looks like #1 assumes that all projects under this repository will be Java-based. To be continued.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023390669
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
Ok I see. The explanation is that before any request can be issued, the user must first authenticate against the OAuth endpoint. This class takes care of that. It can feel a little bit over-engineered as, at this time, authentication is performed only once at the very beginning of the benchmark. Example from the `CreateTreeDatasetConcurrent` simulation.
```
authenticate
.inject(atOnceUsers(1))
.andThen(createCatalogs.inject(atOnceUsers(50)))
.andThen(
..
)
```
But the OAuth token is only valid for 1h. Which means that any benchmark that lasts more than that will fail. I have separate modifications (not published yet) that renew this token periodically that make use of the `AuthenticationActions` more. Does that help clarify?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2025133667
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
+ }
+}
+
+dependencies {
+ gatling("com.typesafe.play:play-json_2.13:2.9.4")
Review Comment:
I mean `toml` for gradle dependencies, not for benchmark config... sorry about the confusion :sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023128949
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
Review Comment:
How do these two properites (max-tables/ max-views) interact with the specifications above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023483895
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
+ }
+}
+
+dependencies {
+ gatling("com.typesafe.play:play-json_2.13:2.9.4")
Review Comment:
nit: do we want to use `toml` files like the main Polaris repo?
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/NAryTreeBuilder.scala:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks
+
+/**
+ * Builds a complete N-ary tree structure for organizing namespaces in synthetic datasets.
+ *
+ * This builder is used to create a hierarchical namespace structure where each non-leaf namespace
+ * has exactly `nsWidth` child namespaces. The tree structure is used to organize tables and views
+ * in a deterministic way. Tables and views are placed in the leaf namespaces of the tree.
+ *
+ * @param nsWidth The number of children each non-leaf namespace will have (N)
+ * @param nsDepth The total depth of the tree, including the root namespace
+ */
+case class NAryTreeBuilder(nsWidth: Int, nsDepth: Int) {
+
+ /**
+ * Computes the path from the root node to the given ordinal.
+ *
+ * @param ordinal the ordinal of the node
+ * @param acc the accumulator for the path
+ * @return the path from the root node (included) to the given ordinal (included)
+ */
+ @scala.annotation.tailrec
+ final def pathToRoot(ordinal: Int, acc: List[Int] = Nil): List[Int] =
+ if (ordinal == 0) {
+ ordinal :: acc
+ } else {
+ val parent = (ordinal - 1) / nsWidth
+ pathToRoot(parent, ordinal :: acc)
+ }
+
+ /**
+ * Calculates the depth of a node in the n-ary tree based on its ordinal.
+ *
+ * @param ordinal The ordinal of the node.
+ * @return The depth of the node in the tree.
+ */
+ def depthOf(ordinal: Int): Int = {
+ if (ordinal == 0) return 0
+ if (nsWidth == 1) return ordinal
+
+ // Using the formula: floor(log_n((x * (n-1)) + 1))
+ val numerator = (ordinal * (nsWidth - 1)) + 1
+ (math.log(numerator) / math.log(nsWidth)).floor.toInt
+ }
+
+ /**
+ * Calculate the total number of nodes in a complete n-ary tree.
+ *
+ * @return The total number of nodes in the tree.
+ */
+ val numberOfNodes: Int = {
+ print(s"Computing number of nodes for n-ary tree with width $nsWidth and depth $nsDepth...")
Review Comment:
Why `print` here?
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
At some point Polaris will support external IdP (AFAIK), so auto-refresh hooks may not cover all cases.
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/util/CircularIterator.scala:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.util
+
+import scala.util.Random
+
+class CircularIterator[T](builder: () => Iterator[T]) extends Iterator[T] {
+ private var currentIterator: Iterator[T] = builder()
+
+ override def hasNext: Boolean = true
+
+ override def next(): T = synchronized {
+ if (!currentIterator.hasNext) {
+ currentIterator = builder()
+ }
+ currentIterator.next()
+ }
+}
+
+class BufferedRandomIterator[T](underlying: CircularIterator[T], bufferSize: Int)
+ extends Iterator[T] {
+ private var buffer: Iterator[T] = populateAndShuffle()
+
+ private def populateAndShuffle(): Iterator[T] =
+ Random.shuffle((1 to bufferSize).map(_ => underlying.next()).toList).iterator
Review Comment:
+1 to supporting seeds, but we ca probably add that later.
##########
benchmarks/src/gatling/resources/gatling.conf:
##########
@@ -0,0 +1,10 @@
+gatling {
+ charting {
+ indicators {
+ percentile1 = 25
+ percentile2 = 50
+ percentile3 = 75
Review Comment:
IQR :tada: ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023441729
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
We could. But I am not sure what the value would be for OSS Polaris. I need to research that more. AFAICT, the OAuth endpoint is deprecated for removal. Which means there could be a way to bypass authentication entirely later for internal testing purposes. Alternatively, if it only stays for development purposes, then we don't need to strictly follow the client credentials flow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2024266954
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/util/CircularIterator.scala:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.util
+
+import scala.util.Random
+
+class CircularIterator[T](builder: () => Iterator[T]) extends Iterator[T] {
+ private var currentIterator: Iterator[T] = builder()
+
+ override def hasNext: Boolean = true
+
+ override def next(): T = synchronized {
+ if (!currentIterator.hasNext) {
+ currentIterator = builder()
+ }
+ currentIterator.next()
+ }
+}
+
+class BufferedRandomIterator[T](underlying: CircularIterator[T], bufferSize: Int)
+ extends Iterator[T] {
+ private var buffer: Iterator[T] = populateAndShuffle()
+
+ private def populateAndShuffle(): Iterator[T] =
+ Random.shuffle((1 to bufferSize).map(_ => underlying.next()).toList).iterator
Review Comment:
Good point, I had overlooked that. Fixed with the last push.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#issuecomment-2769818510
@jbonofre just a heads up, I might need some help to get the license check and header check added to that project. I did not implement it as I thought, maybe we will have a parent project in this repository too, just like in `apache/polaris`. Let me know how you want to proceed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023141596
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
Review Comment:
:eyes: that's a big stack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023513356
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/util/CircularIterator.scala:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.util
+
+import scala.util.Random
+
+class CircularIterator[T](builder: () => Iterator[T]) extends Iterator[T] {
+ private var currentIterator: Iterator[T] = builder()
+
+ override def hasNext: Boolean = true
+
+ override def next(): T = synchronized {
+ if (!currentIterator.hasNext) {
+ currentIterator = builder()
+ }
+ currentIterator.next()
+ }
+}
+
+class BufferedRandomIterator[T](underlying: CircularIterator[T], bufferSize: Int)
+ extends Iterator[T] {
+ private var buffer: Iterator[T] = populateAndShuffle()
+
+ private def populateAndShuffle(): Iterator[T] =
+ Random.shuffle((1 to bufferSize).map(_ => underlying.next()).toList).iterator
Review Comment:
+1 to supporting seeds, but we can probably add that later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023165661
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/NAryTreeBuilder.scala:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks
+
+case class NAryTreeBuilder(nsWidth: Int, nsDepth: Int) {
Review Comment:
May be helpful to add in some details on the implementation of this class. Looks to me like this is essentially a class which maps a range of ordinal values into a tree structure. Is that correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023142049
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
Review Comment:
Ha, good catch. Those two properties cap the total number of tables that would otherwise be created/used.
Example for the dataset creation benchmark:
* with `width=2`, `depth=20`, `tables-per-namespace=4` and `max-tables=-1`
* 1048575 namespaces are created
* there are 524288 leaf namespaces
* and 4 tables in each leaf namespaces are created/used, so a total of 2097152 tables
* with `width=2`, `depth=20`, `tables-per-namespace=4` and `max-tables=1000000`
* 1048575 namespaces are created
* there are 524288 leaf namespaces
* and 4 tables in each leaf namespaces are created until 1 million tables have been created, then the benchmark stops
I should probably add that to the README...
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
Review Comment:
I don't think it is necessary, tbh, given that the initial PR did not have that and the benchmarks were compiling. But this comes from the canonical example of gatling-gradle plugin https://github.com/gatling/gatling-gradle-plugin-demo-scala/blob/main/build.gradle. So I kept it.
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
I assume you are talking about the `*Actions.scala` classes. These classes regroup the elements that are needed to perform certain actions based on either a feature (e.g. authenticating) or an entity (e.g. tables).
They are not listed in the README, indeed, as they are an implementation detail. This is also where the configuration is consumed, not defined.
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
+ max-views = -1 # Maximum views (-1 for unlimited)
+}
+```
+
+### Connection Parameters
+
+Connection settings are configured under `http` and `auth`:
+
+```hocon
+http {
+ base-url = "http://localhost:8181" # Service URL
+}
+
+auth {
+ client-id = null # Required: OAuth2 client ID
+ client-secret = null # Required: OAuth2 client secret
+}
+```
+
+### Workload Parameters
+
+Workload settings are configured under `workload`:
+
+```hocon
+workload {
+ read-write-ratio = 0.8 # Ratio of reads (0.0-1.0)
+}
+```
+
+## Running the Benchmarks
+
+The benchmark uses [typesafe-config](https://github.com/lightbend/config) for configuration management. Default settings are in `src/gatling/resources/benchmark-defaults.conf`. This file should not be modified directly.
+
+To customize the benchmark settings, create your own `application.conf` file and specify it using the `-Dconfig.file` parameter. Your settings will override the default values.
+
+Example `application.conf`:
+```hocon
+auth {
+ client-id = "your-client-id"
+ client-secret = "your-client-secret"
+}
+
+http {
+ base-url = "http://your-polaris-instance:8181"
+}
+
+workload {
+ read-write-ratio = 0.8
+}
+```
+
+Run benchmarks with your configuration:
+
+```bash
+# Sequential dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential \
+ -Dconfig.file=./application.conf
+
+# Concurrent dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent \
+ -Dconfig.file=./application.conf
+```
+
+A message will show the location of the Gatling report:
+```
+Reports generated in: ./benchmarks/build/reports/gatling/<simulation-name>/index.html
+```
+
+### Example Polaris server startup
+
+For repeated testing and benchmarking purposes it's convenient to have fixed client-ID + client-secret combinations. **The following example is ONLY for testing and benchmarking against an airgapped Polaris instance**
+
+```bash
+# Start Polaris with the fixed client-ID/secret admin/admin
+# DO NEVER EVER USE THE FOLLOWING FOR ANY NON-AIRGAPPED POLARIS INSTANCE !!
+./gradlew :polaris-quarkus-server:quarkusBuild && java \
+ -Dpolaris.bootstrap.credentials=POLARIS,admin,admin \
+ -Djava.security.manager=allow \
+ -jar quarkus/server/build/quarkus-app/quarkus-run.jar
+```
+
+With the above you can run the benchmarks using a configuration file with `client-id = "admin"` and `client-secret = "admin"` - meant only for convenience in a fully airgapped system.
+
+# Test Dataset
+
+The benchmarks use synthetic procedural datasets that are generated deterministically at runtime. This means that given the same input parameters, the exact same dataset structure will always be generated. This approach allows generating large volumes of test data without having to store it, while ensuring reproducible benchmark results across different runs.
+
+The diagrams below describe the data sets that are used in benchmarks. Note that the benchmark dataset may not cover all Polaris features.
+
+## Generation rules
+
+The dataset has a tree shape. At the root of the tree is a Polaris realm that must exist before the dataset is created.
+
+An arbitrary number of catalogs can be created under the realm. However, only the first catalog (`C_0`) is used for the rest of the dataset.
Review Comment:
I am not sure what you mean here?
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/NAryTreeBuilder.scala:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks
+
+case class NAryTreeBuilder(nsWidth: Int, nsDepth: Int) {
Review Comment:
You are absolutely correct. I can add some javadoc, yes. But it will be mostly a repeat of the readme without the charts that give visual examples.
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/CatalogActions.scala:
##########
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.DatasetParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing catalog operations in Apache Iceberg. This class provides methods
+ * to create and fetch catalogs.
+ *
+ * @param dp Dataset parameters controlling the dataset generation
+ * @param accessToken Reference to the authentication token for API requests
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class CatalogActions(
+ dp: DatasetParameters,
+ accessToken: AtomicReference[String],
+ maxRetries: Int = 10,
+ retryableHttpCodes: Set[Int] = Set(409, 500)
+) {
+ private val logger = LoggerFactory.getLogger(getClass)
+
+ /**
+ * Creates a Gatling Feeder that generates catalog names and their default storage locations. Each
+ * catalog will be named "C_n" where n is a sequential number, and will have a corresponding
+ * storage location under the configured base path.
+ *
+ * @return An iterator providing catalog names and their storage locations
+ */
+ def feeder(): Feeder[String] = Iterator
+ .from(0)
+ .map { i =>
+ val catalogName = s"C_$i"
+ Map(
+ "catalogName" -> catalogName,
+ "defaultBaseLocation" -> s"${dp.defaultBaseLocation}/$catalogName"
+ )
+ }
+ .take(dp.numCatalogs)
+
+ /**
+ * Creates a new Iceberg catalog with FILE storage type. The catalog is created as an INTERNAL
+ * type with a name and a default base location that are defined in the [[CatalogActions.feeder]].
+ * This represents the fundamental operation of establishing a new catalog in an Iceberg
+ * deployment.
+ *
+ * There is no limit to the number of users that can create catalogs concurrently.
+ */
+ val createCatalog: ChainBuilder =
+ retryOnHttpStatus(maxRetries, retryableHttpCodes, "Create catalog")(
Review Comment:
It is a good question. In my experience, Gatling benchmarks should have little, if any, external dependencies, so that they are easy to read. The risk is for Gatling benchmarks to become only readable/editable with a fully fledged IDE, instead of a simple text editor or even Github UI.
We _could_ extract those String constants into some sort of a Polaris client. But that would be it. The connection logic and all that would make a complete Polaris **client** would not be there, as they are Gatling specific. So I am not sure it would be net positive.
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
+ max-views = -1 # Maximum views (-1 for unlimited)
+}
+```
+
+### Connection Parameters
+
+Connection settings are configured under `http` and `auth`:
+
+```hocon
+http {
+ base-url = "http://localhost:8181" # Service URL
+}
+
+auth {
+ client-id = null # Required: OAuth2 client ID
+ client-secret = null # Required: OAuth2 client secret
+}
+```
+
+### Workload Parameters
+
+Workload settings are configured under `workload`:
+
+```hocon
+workload {
+ read-write-ratio = 0.8 # Ratio of reads (0.0-1.0)
+}
+```
+
+## Running the Benchmarks
+
+The benchmark uses [typesafe-config](https://github.com/lightbend/config) for configuration management. Default settings are in `src/gatling/resources/benchmark-defaults.conf`. This file should not be modified directly.
+
+To customize the benchmark settings, create your own `application.conf` file and specify it using the `-Dconfig.file` parameter. Your settings will override the default values.
+
+Example `application.conf`:
+```hocon
+auth {
+ client-id = "your-client-id"
+ client-secret = "your-client-secret"
+}
+
+http {
+ base-url = "http://your-polaris-instance:8181"
+}
+
+workload {
+ read-write-ratio = 0.8
+}
+```
+
+Run benchmarks with your configuration:
+
+```bash
+# Sequential dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential \
+ -Dconfig.file=./application.conf
+
+# Concurrent dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent \
+ -Dconfig.file=./application.conf
+```
+
+A message will show the location of the Gatling report:
+```
+Reports generated in: ./benchmarks/build/reports/gatling/<simulation-name>/index.html
+```
+
+### Example Polaris server startup
+
+For repeated testing and benchmarking purposes it's convenient to have fixed client-ID + client-secret combinations. **The following example is ONLY for testing and benchmarking against an airgapped Polaris instance**
+
+```bash
+# Start Polaris with the fixed client-ID/secret admin/admin
+# DO NEVER EVER USE THE FOLLOWING FOR ANY NON-AIRGAPPED POLARIS INSTANCE !!
+./gradlew :polaris-quarkus-server:quarkusBuild && java \
+ -Dpolaris.bootstrap.credentials=POLARIS,admin,admin \
+ -Djava.security.manager=allow \
+ -jar quarkus/server/build/quarkus-app/quarkus-run.jar
+```
+
+With the above you can run the benchmarks using a configuration file with `client-id = "admin"` and `client-secret = "admin"` - meant only for convenience in a fully airgapped system.
+
+# Test Dataset
+
+The benchmarks use synthetic procedural datasets that are generated deterministically at runtime. This means that given the same input parameters, the exact same dataset structure will always be generated. This approach allows generating large volumes of test data without having to store it, while ensuring reproducible benchmark results across different runs.
+
+The diagrams below describe the data sets that are used in benchmarks. Note that the benchmark dataset may not cover all Polaris features.
+
+## Generation rules
+
+The dataset has a tree shape. At the root of the tree is a Polaris realm that must exist before the dataset is created.
+
+An arbitrary number of catalogs can be created under the realm. However, only the first catalog (`C_0`) is used for the rest of the dataset.
+
+The namespaces part of the dataset is a complete `N`-ary tree. That is, it starts with a root namespace (`NS_0`) and then, each namespace contains exactly `0` or `N` children namespaces. The width as well as the depth of the namespaces tree are configurable. The total number of namespaces can easily be calculated with the following formulae, where `N` is the tree arity and `D` is the total tree depth, including the root:
Review Comment:
Good point, changing that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023407691
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/AuthenticationActions.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.ConnectionParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing authentication operations. This class provides methods to
+ * authenticate and manage access tokens for API requests.
+ *
+ * @param cp Connection parameters containing client credentials
+ * @param accessToken Reference to the authentication token shared across actions
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class AuthenticationActions(
Review Comment:
I believe we should be using the credential refresh hooks now? I haven't followed this enough in the OSS Iceberg a discussion but there should be auto-refresh hooks now but that may just be for the Iceberg client?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2024258341
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
+ }
+}
+
+dependencies {
+ gatling("com.typesafe.play:play-json_2.13:2.9.4")
Review Comment:
I believe this comment applies for the line just after `play-json` (the `typesafe-config` line). Play-json is used to parse the payloads returned by Polaris and ensure that maps (e.g. namespace properties) are equal. Given that there is no order guarantee between properties, a plain string comparison cannot be used. So play-json has to stay.
We _could_ move from typesafe-config to a `toml` file. But I would first like to double check that we are talking about the same thing. Typesafe config was initially preferred as it is already used to configure Gatling and offers the ability to have default parameter values for benchmarks that can then be overridden by users either from the CLI or a separate configuration (HOCON) file.
AFAICT in #1, toml files are used for the Gradle build. The equivalent of typesafe-config in #1 is picocli, not `toml` files. And I missing something @ajantha-bhat?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2024224403
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
Review Comment:
Added some more examples in the reference configuration file as well as clarified the 1-line description in the readme.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2036794807
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
Review Comment:
```suggestion
Benchmarks for the Polaris service using Gatling.
```
It's not its own repository
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#issuecomment-2794385853
Ok 👍, merging then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2024267700
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/NAryTreeBuilder.scala:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks
+
+/**
+ * Builds a complete N-ary tree structure for organizing namespaces in synthetic datasets.
+ *
+ * This builder is used to create a hierarchical namespace structure where each non-leaf namespace
+ * has exactly `nsWidth` child namespaces. The tree structure is used to organize tables and views
+ * in a deterministic way. Tables and views are placed in the leaf namespaces of the tree.
+ *
+ * @param nsWidth The number of children each non-leaf namespace will have (N)
+ * @param nsDepth The total depth of the tree, including the root namespace
+ */
+case class NAryTreeBuilder(nsWidth: Int, nsDepth: Int) {
+
+ /**
+ * Computes the path from the root node to the given ordinal.
+ *
+ * @param ordinal the ordinal of the node
+ * @param acc the accumulator for the path
+ * @return the path from the root node (included) to the given ordinal (included)
+ */
+ @scala.annotation.tailrec
+ final def pathToRoot(ordinal: Int, acc: List[Int] = Nil): List[Int] =
+ if (ordinal == 0) {
+ ordinal :: acc
+ } else {
+ val parent = (ordinal - 1) / nsWidth
+ pathToRoot(parent, ordinal :: acc)
+ }
+
+ /**
+ * Calculates the depth of a node in the n-ary tree based on its ordinal.
+ *
+ * @param ordinal The ordinal of the node.
+ * @return The depth of the node in the tree.
+ */
+ def depthOf(ordinal: Int): Int = {
+ if (ordinal == 0) return 0
+ if (nsWidth == 1) return ordinal
+
+ // Using the formula: floor(log_n((x * (n-1)) + 1))
+ val numerator = (ordinal * (nsWidth - 1)) + 1
+ (math.log(numerator) / math.log(nsWidth)).floor.toInt
+ }
+
+ /**
+ * Calculate the total number of nodes in a complete n-ary tree.
+ *
+ * @return The total number of nodes in the tree.
+ */
+ val numberOfNodes: Int = {
+ print(s"Computing number of nodes for n-ary tree with width $nsWidth and depth $nsDepth...")
Review Comment:
Removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "pingtimeout (via GitHub)" <gi...@apache.org>.
pingtimeout commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2024258864
##########
benchmarks/src/gatling/resources/gatling.conf:
##########
@@ -0,0 +1,10 @@
+gatling {
+ charting {
+ indicators {
+ percentile1 = 25
+ percentile2 = 50
+ percentile3 = 75
Review Comment:
Yes that one is specifically for your box plots :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre merged PR #2:
URL: https://github.com/apache/polaris-tools/pull/2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023134652
##########
benchmarks/README.md:
##########
@@ -0,0 +1,242 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Polaris Benchmarks
+
+This repository contains benchmarks for the Polaris service using Gatling.
+
+## Available Benchmarks
+
+### Dataset Creation Benchmark
+
+The CreateTreeDataset benchmark creates a test dataset with a specific structure. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential`: Creates entities one at a time
+- `org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent`: Creates up to 50 entities simultaneously
+
+These are write-only workloads designed to populate the system for subsequent benchmarks.
+
+### Read/Update Benchmark
+
+The ReadUpdateTreeDataset benchmark tests read and update operations on an existing dataset. It exists in two variants:
+
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetSequential`: Performs read/update operations one at a time
+- `org.apache.polaris.benchmarks.simulations.ReadUpdateTreeDatasetConcurrent`: Performs up to 20 read/update operations simultaneously
+
+These benchmarks can only be run after using CreateTreeDataset to populate the system.
+
+## Parameters
+
+All parameters are configured through the `application.conf` file located in `src/gatling/resources/`. The configuration uses the [Typesafe Config](https://github.com/lightbend/config) format.
+
+### Dataset Structure Parameters
+
+These parameters must be consistent across all benchmarks and are configured under `dataset.tree`:
+
+```hocon
+dataset.tree {
+ num-catalogs = 1 # Number of catalogs to create
+ namespace-width = 2 # Width of the namespace tree
+ namespace-depth = 4 # Depth of the namespace tree
+ tables-per-namespace = 5 # Tables per namespace
+ views-per-namespace = 3 # Views per namespace
+ columns-per-table = 10 # Columns per table
+ columns-per-view = 10 # Columns per view
+ default-base-location = "file:///tmp/polaris" # Base location for datasets
+ namespace-properties = 10 # Number of properties to add to each namespace
+ table-properties = 10 # Number of properties to add to each table
+ view-properties = 10 # Number of properties to add to each view
+ max-tables = -1 # Maximum tables (-1 for unlimited)
+ max-views = -1 # Maximum views (-1 for unlimited)
+}
+```
+
+### Connection Parameters
+
+Connection settings are configured under `http` and `auth`:
+
+```hocon
+http {
+ base-url = "http://localhost:8181" # Service URL
+}
+
+auth {
+ client-id = null # Required: OAuth2 client ID
+ client-secret = null # Required: OAuth2 client secret
+}
+```
+
+### Workload Parameters
+
+Workload settings are configured under `workload`:
+
+```hocon
+workload {
+ read-write-ratio = 0.8 # Ratio of reads (0.0-1.0)
+}
+```
+
+## Running the Benchmarks
+
+The benchmark uses [typesafe-config](https://github.com/lightbend/config) for configuration management. Default settings are in `src/gatling/resources/benchmark-defaults.conf`. This file should not be modified directly.
+
+To customize the benchmark settings, create your own `application.conf` file and specify it using the `-Dconfig.file` parameter. Your settings will override the default values.
+
+Example `application.conf`:
+```hocon
+auth {
+ client-id = "your-client-id"
+ client-secret = "your-client-secret"
+}
+
+http {
+ base-url = "http://your-polaris-instance:8181"
+}
+
+workload {
+ read-write-ratio = 0.8
+}
+```
+
+Run benchmarks with your configuration:
+
+```bash
+# Sequential dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetSequential \
+ -Dconfig.file=./application.conf
+
+# Concurrent dataset creation
+./gradlew gatlingRun --simulation org.apache.polaris.benchmarks.simulations.CreateTreeDatasetConcurrent \
+ -Dconfig.file=./application.conf
+```
+
+A message will show the location of the Gatling report:
+```
+Reports generated in: ./benchmarks/build/reports/gatling/<simulation-name>/index.html
+```
+
+### Example Polaris server startup
+
+For repeated testing and benchmarking purposes it's convenient to have fixed client-ID + client-secret combinations. **The following example is ONLY for testing and benchmarking against an airgapped Polaris instance**
+
+```bash
+# Start Polaris with the fixed client-ID/secret admin/admin
+# DO NEVER EVER USE THE FOLLOWING FOR ANY NON-AIRGAPPED POLARIS INSTANCE !!
+./gradlew :polaris-quarkus-server:quarkusBuild && java \
+ -Dpolaris.bootstrap.credentials=POLARIS,admin,admin \
+ -Djava.security.manager=allow \
+ -jar quarkus/server/build/quarkus-app/quarkus-run.jar
+```
+
+With the above you can run the benchmarks using a configuration file with `client-id = "admin"` and `client-secret = "admin"` - meant only for convenience in a fully airgapped system.
+
+# Test Dataset
+
+The benchmarks use synthetic procedural datasets that are generated deterministically at runtime. This means that given the same input parameters, the exact same dataset structure will always be generated. This approach allows generating large volumes of test data without having to store it, while ensuring reproducible benchmark results across different runs.
+
+The diagrams below describe the data sets that are used in benchmarks. Note that the benchmark dataset may not cover all Polaris features.
+
+## Generation rules
+
+The dataset has a tree shape. At the root of the tree is a Polaris realm that must exist before the dataset is created.
+
+An arbitrary number of catalogs can be created under the realm. However, only the first catalog (`C_0`) is used for the rest of the dataset.
Review Comment:
realm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023141596
##########
benchmarks/build.gradle.kts:
##########
@@ -0,0 +1,29 @@
+plugins {
+ scala
+ id("io.gatling.gradle") version "3.13.5.2"
+ id("com.diffplug.spotless") version "7.0.2"
+}
+
+description = "Polaris Iceberg REST API performance tests"
+
+tasks.withType<ScalaCompile> {
+ scalaCompileOptions.forkOptions.apply {
+ jvmArgs = listOf("-Xss100m") // Scala compiler may require a larger stack size when compiling Gatling simulations
Review Comment:
:eyes:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Polaris benchmarks [polaris-tools]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #2:
URL: https://github.com/apache/polaris-tools/pull/2#discussion_r2023178284
##########
benchmarks/src/gatling/scala/org/apache/polaris/benchmarks/actions/CatalogActions.scala:
##########
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.benchmarks.actions
+
+import io.gatling.core.Predef._
+import io.gatling.core.feeder.Feeder
+import io.gatling.core.structure.ChainBuilder
+import io.gatling.http.Predef._
+import org.apache.polaris.benchmarks.RetryOnHttpCodes.{
+ retryOnHttpStatus,
+ HttpRequestBuilderWithStatusSave
+}
+import org.apache.polaris.benchmarks.parameters.DatasetParameters
+import org.slf4j.LoggerFactory
+
+import java.util.concurrent.atomic.AtomicReference
+
+/**
+ * Actions for performance testing catalog operations in Apache Iceberg. This class provides methods
+ * to create and fetch catalogs.
+ *
+ * @param dp Dataset parameters controlling the dataset generation
+ * @param accessToken Reference to the authentication token for API requests
+ * @param maxRetries Maximum number of retry attempts for failed operations
+ * @param retryableHttpCodes HTTP status codes that should trigger a retry
+ */
+case class CatalogActions(
+ dp: DatasetParameters,
+ accessToken: AtomicReference[String],
+ maxRetries: Int = 10,
+ retryableHttpCodes: Set[Int] = Set(409, 500)
+) {
+ private val logger = LoggerFactory.getLogger(getClass)
+
+ /**
+ * Creates a Gatling Feeder that generates catalog names and their default storage locations. Each
+ * catalog will be named "C_n" where n is a sequential number, and will have a corresponding
+ * storage location under the configured base path.
+ *
+ * @return An iterator providing catalog names and their storage locations
+ */
+ def feeder(): Feeder[String] = Iterator
+ .from(0)
+ .map { i =>
+ val catalogName = s"C_$i"
+ Map(
+ "catalogName" -> catalogName,
+ "defaultBaseLocation" -> s"${dp.defaultBaseLocation}/$catalogName"
+ )
+ }
+ .take(dp.numCatalogs)
+
+ /**
+ * Creates a new Iceberg catalog with FILE storage type. The catalog is created as an INTERNAL
+ * type with a name and a default base location that are defined in the [[CatalogActions.feeder]].
+ * This represents the fundamental operation of establishing a new catalog in an Iceberg
+ * deployment.
+ *
+ * There is no limit to the number of users that can create catalogs concurrently.
+ */
+ val createCatalog: ChainBuilder =
+ retryOnHttpStatus(maxRetries, retryableHttpCodes, "Create catalog")(
Review Comment:
Looks good for now, but I was wondering if this should be using a Polaris client rather than doing String based querying? Probably not important because I assume we are going to keep these apis static but I generally am afraid of string constants.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org