You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "stevelorddremio (via GitHub)" <gi...@apache.org> on 2024/06/07 19:00:15 UTC

[PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

stevelorddremio opened a new pull request, #42035:
URL: https://github.com/apache/arrow/pull/42035

   <!--
   Thanks for opening a pull request!
   If this is your first pull request you can find detailed information on how 
   to contribute here:
     * [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
     * [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
   
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
   
   Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename the pull request title in the following format?
   
       GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   In the case of PARQUET issues on JIRA the title also supports:
   
       PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   -->
   
   ### Rationale for this change
   
   See Issue https://github.com/apache/arrow/issues/41947
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   ### What changes are included in this PR?
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   ### Are these changes tested?
   
   Yes
   <!--
   We typically require tests for all PRs in order to:
   1. Prevent the code from being accidentally broken by subsequent changes
   2. Serve as another way to document the expected behavior of the code
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   ### Are there any user-facing changes?
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   Introductiona of an optional catalog query parameter in the JDBC url string.
   
   <!--
   If there are any breaking changes to public APIs, please uncomment the line below and explain which changes are breaking.
   -->
   <!-- **This PR includes breaking changes to public APIs.** -->
   
   <!--
   Please uncomment the line below (and provide explanation) if the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld). We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
   -->
   <!-- **This PR contains a "Critical Fix".** -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1635805711


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -250,6 +273,31 @@ public interface PreparedStatement extends AutoCloseable {
     void close();
   }
 
+  /** A connection is created with catalog set as a session option. */
+  private void setSetCatalogInSessionIfPresent() {

Review Comment:
   I think this is good for now, but eventually we may want to reframe this as just "start a session" since I assume there'll be other parameters to set.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1635804618


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -761,6 +812,16 @@ public Builder withCallOptions(final Collection<CallOption> options) {
       return this;
     }
 
+    /**
+     * Sets the catalog for this handler.
+     * @param catalog the catalog
+     * @return this instance.
+     */
+    public Builder withCatalog(final String catalog) {
+      this.catalog = catalog;

Review Comment:
   Ah...what you had originally was fine then, but I was more questioning if empty string for catalog is valid (can't a system have an unnamed default catalog?) in which case it would be better to only differentiate between null/not-null. I proposed Optional to be explicit about the semantics but certainly `@Nullable String` is fine



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1636805796


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -250,6 +273,31 @@ public interface PreparedStatement extends AutoCloseable {
     void close();
   }
 
+  /** A connection is created with catalog set as a session option. */
+  private void setSetCatalogInSessionIfPresent() {

Review Comment:
   Agreed. This is rather explicit. I'll be looking to post an issue to cover more generalized session options at some point. This is more a MVP implementation for now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1638842952


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -761,6 +812,16 @@ public Builder withCallOptions(final Collection<CallOption> options) {
       return this;
     }
 
+    /**
+     * Sets the catalog for this handler.
+     * @param catalog the catalog
+     * @return this instance.
+     */
+    public Builder withCatalog(final String catalog) {
+      this.catalog = catalog;

Review Comment:
   Thinking about this again. If the client has set a catalog value that is not null then we should pass this through for validation by the server. The client shouldn't be doing that. 
   If the catalog is null then that should not be passed to the server. If the server supports a default, this is the case when that will be used.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1633946268


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -194,13 +209,20 @@ public FlightInfo getInfo(final String query) {
 
   @Override
   public void close() throws SQLException {
+    if (hasCatalog()) {
+      sqlClient.closeSession(new CloseSessionRequest(), getOptions());
+    }
     try {
       AutoCloseables.close(sqlClient);
     } catch (final Exception e) {
       throw new SQLException("Failed to clean up client resources.", e);
     }
   }
 
+  private boolean hasCatalog() {
+    return !Strings.isNullOrEmpty(catalog);

Review Comment:
   Also mostly wondering if an empty-string catalog would ever be valid



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1634416715


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -254,6 +281,26 @@ public interface PreparedStatement extends AutoCloseable {
    * @return a new prepared statement.
    */
   public PreparedStatement prepare(final String query) {
+    if (setCatalogInSession) {

Review Comment:
   Is there some way to do this at initial connection time? (Some things may not prepare a statement but we'd still want the catalog/session to exist right? e.g. if someone connects and queries the catalog/metadata)



##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -761,6 +812,16 @@ public Builder withCallOptions(final Collection<CallOption> options) {
       return this;
     }
 
+    /**
+     * Sets the catalog for this handler.
+     * @param catalog the catalog
+     * @return this instance.
+     */
+    public Builder withCatalog(final String catalog) {
+      this.catalog = catalog;

Review Comment:
   Why not store Optional<String> in the first place?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1635776382


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -194,13 +209,20 @@ public FlightInfo getInfo(final String query) {
 
   @Override
   public void close() throws SQLException {
+    if (hasCatalog()) {
+      sqlClient.closeSession(new CloseSessionRequest(), getOptions());
+    }
     try {
       AutoCloseables.close(sqlClient);
     } catch (final Exception e) {
       throw new SQLException("Failed to clean up client resources.", e);
     }
   }
 
+  private boolean hasCatalog() {
+    return !Strings.isNullOrEmpty(catalog);

Review Comment:
   I think an empty string catalog would definitely be odd. I updated the logic in Builder and added some tests to show the behaviour with various catalog strings. It behaves as I would expect. @lidavidm lets see if that matches your thinking also.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2233434371

   Please open a new issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2160092488

   @stevelorddremio thanks, do you mind rebasing/running the new formatter?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "scruz-denodo (via GitHub)" <gi...@apache.org>.
scruz-denodo commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2233307887

   Hi, I was testing this change and I am facing a problem when the `java.sql.Connection#close` method of the JDBC connection is called. 
   
   I am receiving the following error invoking the method.
   
   ```
   Exception in thread "main" java.sql.SQLException: UNKNOWN: Uncaught exception in the SynchronizationContext. Re-thrown.
   	at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
   	at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
   	at org.apache.arrow.driver.jdbc.ArrowFlightConnection.close(ArrowFlightConnection.java:187)
   	at com.arrowflight.test.Main.test7(Main.java:249)
   	at com.arrowflight.test.Main.main(Main.java:27)
   Caused by: io.grpc.StatusRuntimeException: UNKNOWN: Uncaught exception in the SynchronizationContext. Re-thrown.
   	at io.grpc.Status.asRuntimeException(Status.java:525)
   	at io.grpc.internal.RetriableStream$1.uncaughtException(RetriableStream.java:75)
   	at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:96)
   	at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:126)
   	at io.grpc.internal.RetriableStream.safeCloseMasterListener(RetriableStream.java:838)
   	at io.grpc.internal.RetriableStream.cancel(RetriableStream.java:531)
   	at io.grpc.internal.RetriableStream.start(RetriableStream.java:393)
   	at io.grpc.internal.ClientCallImpl.startInternal(ClientCallImpl.java:285)
   	at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:184)
   	at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
   	at org.apache.arrow.flight.grpc.ClientInterceptorAdapter$FlightClientCall.start(ClientInterceptorAdapter.java:142)
   	at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
   	at io.grpc.stub.MetadataUtils$HeaderAttachingClientInterceptor$HeaderAttachingClientCall.start(MetadataUtils.java:75)
   	at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:335)
   	at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:311)
   	at io.grpc.stub.ClientCalls.blockingServerStreamingCall(ClientCalls.java:210)
   	at org.apache.arrow.flight.impl.FlightServiceGrpc$FlightServiceBlockingStub.doAction(FlightServiceGrpc.java:874)
   	at org.apache.arrow.flight.FlightClient.doAction(FlightClient.java:168)
   	at org.apache.arrow.flight.FlightClient.closeSession(FlightClient.java:686)
   	at org.apache.arrow.flight.sql.FlightSqlClient.closeSession(FlightSqlClient.java:998)
   	at org.apache.arrow.driver.jdbc.client.ArrowFlightSqlClientHandler.close(ArrowFlightSqlClientHandler.java:224)
   	at org.apache.arrow.util.AutoCloseables.close(AutoCloseables.java:97)
   	at org.apache.arrow.util.AutoCloseables.close(AutoCloseables.java:75)
   	at org.apache.arrow.driver.jdbc.ArrowFlightConnection.close(ArrowFlightConnection.java:181)
   	... 2 more
   Caused by: java.util.concurrent.RejectedExecutionException: Task io.grpc.internal.SerializingExecutor@6ecdbab8 rejected from java.util.concurrent.ThreadPoolExecutor@3dd4a6fa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 25]
   	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
   	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355)
   	at io.grpc.internal.SerializingExecutor.schedule(SerializingExecutor.java:102)
   	at io.grpc.internal.SerializingExecutor.execute(SerializingExecutor.java:95)
   	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closedInternal(ClientCallImpl.java:736)
   	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closed(ClientCallImpl.java:680)
   	at io.grpc.internal.RetriableStream$4.run(RetriableStream.java:843)
   	at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:94)
   	... 23 more
   ```
   
   I think the reason is the following. Checking this code:
   
   https://github.com/apache/arrow/blob/6a2e19a852b367c72d7b12da4d104456491ed8b7/java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/ArrowFlightConnection.java#L174-L189
   
   - `close` method invokes `clientHandler.close();` at line 175. Here is where now is invoked the `closeSession` method at line 224. 
    https://github.com/apache/arrow/blob/6a2e19a852b367c72d7b12da4d104456491ed8b7/java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java#L222-L225
   - after that again on  `ArrowFlightConnection.java`, the `executorService` is closed at like 177
   - and the `clientHandler.close();` is called again at line 181.
     - with this call, the `closeSession` method is invoked again. And it is here when the exception is thrown. Because the executorService was already shutdown.
   
   https://github.com/apache/arrow/blob/6a2e19a852b367c72d7b12da4d104456491ed8b7/java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/ArrowFlightConnection.java#L174-L189
   
   I think that line line 181 `from org.apache.arrow.driver.jdbc.ArrowFlightConnection#close` should be remove for avoding the problem. The close is already done at 175.
   
   Also, I would remove the is if:
   https://github.com/apache/arrow/blob/6a2e19a852b367c72d7b12da4d104456491ed8b7/java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java#L222-L225
   why not invoking the `closeSession` when the catalog was not set?
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2155363855

   :warning: GitHub issue #41947 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1633544667


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -254,6 +276,21 @@ public interface PreparedStatement extends AutoCloseable {
    * @return a new prepared statement.
    */
   public PreparedStatement prepare(final String query) {
+    if (hasCatalog()) {

Review Comment:
   Good point. I'll set once.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1635509607


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -761,6 +812,16 @@ public Builder withCallOptions(final Collection<CallOption> options) {
       return this;
     }
 
+    /**
+     * Sets the catalog for this handler.
+     * @param catalog the catalog
+     * @return this instance.
+     */
+    public Builder withCatalog(final String catalog) {
+      this.catalog = catalog;

Review Comment:
   Hmm, I made Optional now, which means I pass Optional as a parameter into the constructor.  Not feeling that is the best practice, but have restricted the scope of createNewHandler() to package. Added extra unit tests to safe guard against possible NPE later.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1633598024


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -194,13 +209,20 @@ public FlightInfo getInfo(final String query) {
 
   @Override
   public void close() throws SQLException {
+    if (hasCatalog()) {
+      sqlClient.closeSession(new CloseSessionRequest(), getOptions());
+    }
     try {
       AutoCloseables.close(sqlClient);
     } catch (final Exception e) {
       throw new SQLException("Failed to clean up client resources.", e);
     }
   }
 
+  private boolean hasCatalog() {
+    return !Strings.isNullOrEmpty(catalog);

Review Comment:
   If I understand correctly, this is a readability issue?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1633598024


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -194,13 +209,20 @@ public FlightInfo getInfo(final String query) {
 
   @Override
   public void close() throws SQLException {
+    if (hasCatalog()) {
+      sqlClient.closeSession(new CloseSessionRequest(), getOptions());
+    }
     try {
       AutoCloseables.close(sqlClient);
     } catch (final Exception e) {
       throw new SQLException("Failed to clean up client resources.", e);
     }
   }
 
+  private boolean hasCatalog() {
+    return !Strings.isNullOrEmpty(catalog);

Review Comment:
   If I infer correctly, is this a readability issue?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2162051704

   I think AMD64 Windows Server 2022 Java JDK 11 will pass if re-triggered.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1633731201


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -254,6 +276,21 @@ public interface PreparedStatement extends AutoCloseable {
    * @return a new prepared statement.
    */
   public PreparedStatement prepare(final String query) {
+    if (hasCatalog()) {
+      final SetSessionOptionsRequest setSessionOptionRequest =
+              new SetSessionOptionsRequest(ImmutableMap.<String, SessionOptionValue>builder()
+                      .put(CATALOG, SessionOptionValueFactory.makeSessionOptionValue(catalog))
+                      .build());
+      final SetSessionOptionsResult result = sqlClient.setSessionOptions(setSessionOptionRequest, getOptions());
+      if (result.hasErrors()) {
+        Map<String, SetSessionOptionsResult.Error> errors = result.getErrors();
+        for (Map.Entry<String, SetSessionOptionsResult.Error> error : errors.entrySet()) {
+          LOGGER.warn(error.toString());
+        }
+        throw new RuntimeException(String.format("Cannot set session option for catalog = %s", catalog));

Review Comment:
   SQLException looks more appropriate but that can't be propagated back to the client as this ultimately is not defined by the Calcite interface.
   I've used a FlightRuntimeException instead. Let me know if this doesn't align with what you were thinking.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1636829869


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -761,6 +812,16 @@ public Builder withCallOptions(final Collection<CallOption> options) {
       return this;
     }
 
+    /**
+     * Sets the catalog for this handler.
+     * @param catalog the catalog
+     * @return this instance.
+     */
+    public Builder withCatalog(final String catalog) {
+      this.catalog = catalog;

Review Comment:
   Haha, yes. Hopefully the accepted values are clear now, but Optional might be overkill.
   Yes a system having an unnamed default catalog is a reasonable use-case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1637243596


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -761,6 +812,16 @@ public Builder withCallOptions(final Collection<CallOption> options) {
       return this;
     }
 
+    /**
+     * Sets the catalog for this handler.
+     * @param catalog the catalog
+     * @return this instance.
+     */
+    public Builder withCatalog(final String catalog) {
+      this.catalog = catalog;

Review Comment:
   Ah ok so then maybe this bit should be adjusted?
   
   ```
   this.catalog = Optional.ofNullable(catalog).map(String::trim).filter(str -> !str.isEmpty());
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2245029786

   Revision: 9a43834309bca1087b035ab776549ead9fd87fec
   
   Submitted crossbow builds: [ursacomputing/crossbow @ actions-f7d1d505f9](https://github.com/ursacomputing/crossbow/branches/all?query=actions-f7d1d505f9)
   
   |Task|Status|
   |----|------|
   |java-jars|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-java-jars)](https://github.com/ursacomputing/crossbow/actions/runs/10058383066/job/27801336985)|
   |test-conda-python-3.10-spark-v3.5.0|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-test-conda-python-3.10-spark-v3.5.0)](https://github.com/ursacomputing/crossbow/actions/runs/10058383046/job/27801335342)|
   |test-conda-python-3.11-spark-master|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-test-conda-python-3.11-spark-master)](https://github.com/ursacomputing/crossbow/actions/runs/10058383072/job/27801335541)|
   |test-conda-python-3.8-spark-v3.5.0|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-test-conda-python-3.8-spark-v3.5.0)](https://github.com/ursacomputing/crossbow/actions/runs/10058383184/job/27801335906)|
   |verify-rc-source-java-linux-almalinux-8-amd64|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-verify-rc-source-java-linux-almalinux-8-amd64)](https://github.com/ursacomputing/crossbow/actions/runs/10058383492/job/27801336280)|
   |verify-rc-source-java-linux-conda-latest-amd64|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-verify-rc-source-java-linux-conda-latest-amd64)](https://github.com/ursacomputing/crossbow/actions/runs/10058383176/job/27801335729)|
   |verify-rc-source-java-linux-ubuntu-20.04-amd64|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)](https://github.com/ursacomputing/crossbow/actions/runs/10058383065/job/27801335290)|
   |verify-rc-source-java-linux-ubuntu-22.04-amd64|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)](https://github.com/ursacomputing/crossbow/actions/runs/10058383603/job/27801336459)|
   |verify-rc-source-java-macos-amd64|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-f7d1d505f9-github-verify-rc-source-java-macos-amd64)](https://github.com/ursacomputing/crossbow/actions/runs/10058383048/job/27801335328)|


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1632453636


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -254,6 +276,21 @@ public interface PreparedStatement extends AutoCloseable {
    * @return a new prepared statement.
    */
   public PreparedStatement prepare(final String query) {
+    if (hasCatalog()) {
+      final SetSessionOptionsRequest setSessionOptionRequest =
+              new SetSessionOptionsRequest(ImmutableMap.<String, SessionOptionValue>builder()
+                      .put(CATALOG, SessionOptionValueFactory.makeSessionOptionValue(catalog))
+                      .build());
+      final SetSessionOptionsResult result = sqlClient.setSessionOptions(setSessionOptionRequest, getOptions());
+      if (result.hasErrors()) {
+        Map<String, SetSessionOptionsResult.Error> errors = result.getErrors();
+        for (Map.Entry<String, SetSessionOptionsResult.Error> error : errors.entrySet()) {
+          LOGGER.warn(error.toString());
+        }
+        throw new RuntimeException(String.format("Cannot set session option for catalog = %s", catalog));

Review Comment:
   Preferably throw a more specific exception?



##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -254,6 +276,21 @@ public interface PreparedStatement extends AutoCloseable {
    * @return a new prepared statement.
    */
   public PreparedStatement prepare(final String query) {
+    if (hasCatalog()) {

Review Comment:
   Would we really do this every time we execute a query? Shouldn't it be set once up front?



##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -194,13 +209,20 @@ public FlightInfo getInfo(final String query) {
 
   @Override
   public void close() throws SQLException {
+    if (hasCatalog()) {
+      sqlClient.closeSession(new CloseSessionRequest(), getOptions());
+    }
     try {
       AutoCloseables.close(sqlClient);
     } catch (final Exception e) {
       throw new SQLException("Failed to clean up client resources.", e);
     }
   }
 
+  private boolean hasCatalog() {
+    return !Strings.isNullOrEmpty(catalog);

Review Comment:
   Might prefer `Optional<String>` 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #42035:
URL: https://github.com/apache/arrow/pull/42035


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "vibhatha (via GitHub)" <gi...@apache.org>.
vibhatha commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2156913443

   Overall LGTM! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "scruz-denodo (via GitHub)" <gi...@apache.org>.
scruz-denodo commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2233311012

   Please, let me know if I have to open a new issue with this problem. 
   
   I put here as first step because this version 17 is being currently published.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1633888596


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -194,13 +209,20 @@ public FlightInfo getInfo(final String query) {
 
   @Override
   public void close() throws SQLException {
+    if (hasCatalog()) {
+      sqlClient.closeSession(new CloseSessionRequest(), getOptions());
+    }
     try {
       AutoCloseables.close(sqlClient);
     } catch (final Exception e) {
       throw new SQLException("Failed to clean up client resources.", e);
     }
   }
 
+  private boolean hasCatalog() {
+    return !Strings.isNullOrEmpty(catalog);

Review Comment:
   Updated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2159379012

   All updates have been incorporated.
   Ubuntu JDK 17 build fails. I think this will pass if re-triggered.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "stevelorddremio (via GitHub)" <gi...@apache.org>.
stevelorddremio commented on code in PR #42035:
URL: https://github.com/apache/arrow/pull/42035#discussion_r1635344693


##########
java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/client/ArrowFlightSqlClientHandler.java:
##########
@@ -254,6 +281,26 @@ public interface PreparedStatement extends AutoCloseable {
    * @return a new prepared statement.
    */
   public PreparedStatement prepare(final String query) {
+    if (setCatalogInSession) {

Review Comment:
   Yes, valid point. I think it makes sense to move this to where the connection is created as that is more appropriate.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "vibhatha (via GitHub)" <gi...@apache.org>.
vibhatha commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2245025925

   @github-actions crossbow submit -g java


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-41947: [Java] Support catalog in JDBC driver with session options [arrow]

Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #42035:
URL: https://github.com/apache/arrow/pull/42035#issuecomment-2169072287

   After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit cecd7710f31ae99758c58bcff42ec824b4291156.
   
   There was 1 benchmark result indicating a performance regression:
   
   - Commit Run on `test-mac-arm` at [2024-06-14 10:17:26Z](https://conbench.ursa.dev/compare/runs/56f9d691f7504d0b939fcfe45ffa3602...2f8aa62856e64e6ba0665f7a6f17ee32/)
     - [`tpch` (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-21, scale_factor=1](https://conbench.ursa.dev/compare/benchmarks/0666c051126f74198000890a8d22020e...0666c230dae47efd8000f79f74448b90)
   
   The [full Conbench report](https://github.com/apache/arrow/runs/26258615059) has more details. It also includes information about 16 possible false positives for unstable benchmarks that are known to sometimes produce them.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org