You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/12/09 09:34:42 UTC
[GitHub] [camel-quarkus] JiriOndrusek opened a new pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
JiriOndrusek opened a new pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370
…g issue if stored procedure is called #3260
fixes https://github.com/apache/camel-quarkus/issues/3327
Needs https://github.com/quarkusio/quarkus/issues/22058 to work correctly.
I introduced an option to start sql test with derby started by docker. - to simulate external database (and not dev service).
There are 2 things which may need some changes:
1. To run stored procedure on external derby db, there has to be a jar which is uploaded there. I created module `sql-derby` for that purpose and I put it on the same hierarchy level as `sql` test module. I'm not sure if this is the right approach, another option is to put both `sql` and `sql-derby` in one module.
2. I'm facing a problem with generated port number for docker container. If I let docker to generate its own random port, I'm not able propagate it back to datasource definition. Therefore there are 2 properties, which forces usage of docker derby db:
```
export SQL_USE_DERBY_DOCKER=true
export SQL_USE_DERBY_PORT="#port"
```
On the other hand in current moment, when there is no CI automation for derby (see https://github.com/apache/camel-quarkus/issues/3083) it could stay in that way.
WDYT? @aldettinger
<!-- Uncomment and fill this section if your PR is not trivial
[ ] An issue should be filed for the change unless this is a trivial change (fixing a typo or similar). One issue should ideally be fixed by not more than one commit and the other way round, each commit should fix just one issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful and properly spelled subject line and body. Copying the title of the associated issue is typically enough. Please include the issue number in the commit message prefixed by #.
[ ] The pull request description should explain what the pull request does, how, and why. If the info is available in the associated issue or some other external document, a link is enough.
[ ] Phrases like Fix #<issueNumber> or Fixes #<issueNumber> will auto-close the named issue upon merging the pull request. Using them is typically a good idea.
[ ] Please run mvn process-resources -Pformat (and amend the changes if necessary) before sending the pull request.
[ ] Contributor guide is your good friend: https://camel.apache.org/camel-quarkus/latest/contributor-guide.html
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-989996526
Ufortunately even if quarkus issue is fixed, test still fails in native. See this comment for a stacktrace - https://github.com/quarkusio/quarkus/issues/22058#issuecomment-989825112
I'll keep digging into this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1018369073
Derby dev service doesn't work because of quarkus error - https://github.com/quarkusio/quarkus/issues/23083
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1016158694
@zbendhiba your suggestion sounds good. I used this way and didn't think that it is against best practices in native. If it is possible to put a jar inside docker image, it would be solution.
Current fix uploads a jar into docker container
```
container = new FixedHostPortGenericContainer("az82/docker-derby")
.withFixedExposedPort(SqlHelper.getDerbyDockerPort(), 1527)
.withCopyFileToContainer(
MountableFile.forClasspathResource("derby/" + jars[0].getName()),
"/dbs/storedProcedure.jar")
.waitingFor(Wait.forListeningPort());
```
Do I understand correctly, that having jar with stored procedure among dependencies cause quarkus in native to inspect and process it,
Maybe I can use a current solution, where the jar won't be in dependency, but will be copied (or created) in this module via maven plugin. Would it help?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1017657284
I rebased this PR to `main`. I can locally run tests with external derby via docker in **JVM** mode Native mode fails with
```
ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): ERROR 08006: Insufficient data while reading from the network - expected a minimum of 6 bytes and received only 0 bytes. The connection has been terminated.
```
But native mode is not expected to be successful, see https://github.com/quarkusio/quarkus/issues/22058#issuecomment-989714178
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1018371355
@jamesnetherton The CI build fails with:
```
Run [[ -z $(git status --porcelain | grep -v antora.yml) ]] || { echo 'There are uncommitted changes'; git status; exit 1; }
There are uncommitted changes
HEAD detached at pull/3370/merge
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: pom.xml
```
But if I build the branch locally (even without `-Dquickly`) the is no modified file. Do you know what could be wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger edited a comment on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
aldettinger edited a comment on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-989980719
It looks a big part of the difficulty comes from the setup of a derby database using a stored proc from a jar.
I wonder whether such a setup could be already present in an already built docker image ? Maybe derby testers would already have such kind of setup present in a docker image ?
In any case, this is good to see this path being dug. Well done for that @JiriOndrusek :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] zbendhiba commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
zbendhiba commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1015517100
@JiriOndrusek I'm not sure if I understand this PR correctly. Why do we need this external jar for native mode ?
Is it for a Test Resource needs, or is it required for the code of the app ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] jamesnetherton commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
jamesnetherton commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1016556557
You'll want to rebase and change the target branch to `main` since Quarkus has been bumped to 2.7.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek edited a comment on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek edited a comment on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1017657284
I rebased this PR to `main`. I can locally run tests (successfully) with external derby via docker in **JVM** mode Native mode fails with
```
ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): ERROR 08006: Insufficient data while reading from the network - expected a minimum of 6 bytes and received only 0 bytes. The connection has been terminated.
```
But native mode is not expected to be successful, see https://github.com/quarkusio/quarkus/issues/22058#issuecomment-989714178
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
aldettinger commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-989980719
It looks a big part of the difficulty comes from the setup of a derby database using a stored proc from a jar.
I wonder whether such a setup could be already present in an already built docker image ? Maybe derby testers would already have such kind of setup present in a docker image ?
In case case, this is good to see this path being dug. Well done for that @JiriOndrusek :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1015526206
@zbendhiba I'll try to explain. From what I remember:
1. We want to add a test for derby against external db.
2. External db is started by the test via docker image.
3. To be able to execute stored procedure in the derby db, jar with the implementation of that procedure has to be uploaded into the db.
```
CALLLL sqlj.install_jar('/PATH_TO_JAR/camel-quarkus-integration-test-sql-derby-stored-procedure-*.jar', 'AddNumsProcedure' , 0)
CALL syscs_util.syscs_set_database_property('derby.database.classpath', 'APP.ADDNUMSPROCEDURE')
```
(In case of the dev service, derby runs with the same class loader and there is no need to upload this procedure. It is there because it is a dependecy of the module.)
My answer is that we need this jar for both JVM and native and it is used by the test resource to initialize derby (started via docker) to contain our stored procedure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] zbendhiba commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
zbendhiba commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1015547264
Maybe I'm wrong. Can you achieve this using it only at QuarkusTestResourceLifecycleManager level? Or is the problem come from the fact of the existence of a library in the pom.xml file ?
Or can we push the jar inside the Docker image using docker compose?
My point is that something needed just for TestResource should not be included in the native build. Unless I'm wrong about how QuarkusTestResourceLifecycleManager works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#discussion_r765938007
##########
File path: integration-tests/sql/README.adoc
##########
@@ -42,4 +42,32 @@ $Env:SQL_JDBC_USERNAME="#username"
$Env:SQL_JDBC_PASSWORD="#password"
```
-Oracle database could be used as external db. In that case use parameter `-Dcq.sqlJdbcKind=oracle`.
\ No newline at end of file
+Oracle database could be used as external db. In that case use parameter `-Dcq.sqlJdbcKind=oracle`.
+
+=== External Derby database
+
+To execute tests against external Derby database, stored procedure has to be uploaded into the database classpath.
+Jar with stored procedure for the derby database is creaed by module `sql-derby`.
+Jar could be uploaded via following commands through `ij`:
+```
+CALL sqlj.install_jar('/PATH_TO_JAR/camel-quarkus-integration-test-sql-derby-stored-procedure-*.jar', 'AddNumsProcedure' , 0)
+
+CALL syscs_util.syscs_set_database_property('derby.database.classpath', 'APP.ADDNUMSPROCEDURE')
+```
Review comment:
I agree that linkto some doc would help us to avoid modyfication od this readme. But on the other hand, the same code is used in docker initialization in this change. So we would need to keep it updated either way. (But of course in case that there is no better way of initialization of stored procedure in derby, it is still worth of investigation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#discussion_r765956231
##########
File path: integration-tests/sql/README.adoc
##########
@@ -42,4 +42,32 @@ $Env:SQL_JDBC_USERNAME="#username"
$Env:SQL_JDBC_PASSWORD="#password"
```
-Oracle database could be used as external db. In that case use parameter `-Dcq.sqlJdbcKind=oracle`.
\ No newline at end of file
+Oracle database could be used as external db. In that case use parameter `-Dcq.sqlJdbcKind=oracle`.
+
+=== External Derby database
+
+To execute tests against external Derby database, stored procedure has to be uploaded into the database classpath.
+Jar with stored procedure for the derby database is creaed by module `sql-derby`.
+Jar could be uploaded via following commands through `ij`:
+```
+CALL sqlj.install_jar('/PATH_TO_JAR/camel-quarkus-integration-test-sql-derby-stored-procedure-*.jar', 'AddNumsProcedure' , 0)
+
+CALL syscs_util.syscs_set_database_property('derby.database.classpath', 'APP.ADDNUMSPROCEDURE')
+```
Review comment:
Indeed, it's clear now that this is the itest doc and not the extension doc. So, yes I also think we need to keep this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] jamesnetherton commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
jamesnetherton commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1018420947
Do you need to have the target branch as `quarkus-main`? `main` has been updated to Quarkus 2.7.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1018423145
Thanks, this was the mistake, I rebase it on git repo but not on this PR. thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-989695079
I think that better approach (step 1) is to created sql-derby as a module in integration-tests-support, I'll try it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#discussion_r765904264
##########
File path: integration-tests/sql/README.adoc
##########
@@ -42,4 +42,32 @@ $Env:SQL_JDBC_USERNAME="#username"
$Env:SQL_JDBC_PASSWORD="#password"
```
-Oracle database could be used as external db. In that case use parameter `-Dcq.sqlJdbcKind=oracle`.
\ No newline at end of file
+Oracle database could be used as external db. In that case use parameter `-Dcq.sqlJdbcKind=oracle`.
+
+=== External Derby database
+
+To execute tests against external Derby database, stored procedure has to be uploaded into the database classpath.
+Jar with stored procedure for the derby database is creaed by module `sql-derby`.
+Jar could be uploaded via following commands through `ij`:
+```
+CALL sqlj.install_jar('/PATH_TO_JAR/camel-quarkus-integration-test-sql-derby-stored-procedure-*.jar', 'AddNumsProcedure' , 0)
+
+CALL syscs_util.syscs_set_database_property('derby.database.classpath', 'APP.ADDNUMSPROCEDURE')
+```
Review comment:
In such case, I wonder whether pointing to a documentation owned by derby would be good ? From time to time, I found us modifying docs for things that under camel-quarkus control.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] zbendhiba commented on pull request #3370: Sql test using derby doesn't start dev service and shows class loadin…
Posted by GitBox <gi...@apache.org>.
zbendhiba commented on pull request #3370:
URL: https://github.com/apache/camel-quarkus/pull/3370#issuecomment-1016167771
> @zbendhiba your suggestion sounds good. I used this way and didn't think that it is against best practices in native. If it is possible to put a jar inside docker image, it would be solution. Current fix uploads a jar into docker container
>
> ```
> container = new FixedHostPortGenericContainer("az82/docker-derby")
> .withFixedExposedPort(SqlHelper.getDerbyDockerPort(), 1527)
> .withCopyFileToContainer(
> MountableFile.forClasspathResource("derby/" + jars[0].getName()),
> "/dbs/storedProcedure.jar")
> .waitingFor(Wait.forListeningPort());
> ```
>
> Do I understand correctly, that having jar with stored procedure among dependencies cause quarkus in native to inspect and process it, Maybe I can use a current solution, where the jar won't be in dependency, but will be copied (or created) in this module via maven plugin. Would it help?
No I think it's fine. I was looking at related issues, and was thinking we have a native error because of this.
Thanks for your answer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org