You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/01/04 03:22:50 UTC
[GitHub] [druid] paul-rogers opened a new pull request #12115: Support extension services
paul-rogers opened a new pull request #12115:
URL: https://github.com/apache/druid/pull/12115
### Description
Several changes to better support extension services:
* Use Guice to publish the list of known node roles so an extension can add to it.
* Use the Guice list in for SQL servers system table so it will publish instances of the extension service.
* Split out the index task client into a base class to allow the client to be used for an extension service.
* Add a PrivateApi annotation to mark REST services primarily used internally to the server.
* Misc. code cleanup.
<hr>
##### Key changed/added classes in this PR
* `NodeRoles` - A Guice multi-binder to hold the list of Druid- and extension-defined node roles.
* `PrivateAPI` - Annotation to use to mark a REST service which is private to a server rather than being public. Such private APIs should not be documented.
* `TaskClient` - Holds most of what was in `IndexTaskClient`, but without the tight binding to the indexer. `IndexTaskClient` now extends `TaskClient` and adds the Indexer-specific assumptions.
<hr>
This PR has:
- [X] been self-reviewed.
- [X] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [X] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [X] been tested via the Maven build and the tests run in Github.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] paul-rogers commented on pull request #12115: Support extension services
Posted by GitBox <gi...@apache.org>.
paul-rogers commented on pull request #12115:
URL: https://github.com/apache/druid/pull/12115#issuecomment-1027509053
Closing as committers asked that this be broken into smaller PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] paul-rogers commented on pull request #12115: Support extension services
Posted by GitBox <gi...@apache.org>.
paul-rogers commented on pull request #12115:
URL: https://github.com/apache/druid/pull/12115#issuecomment-1008181878
The build mostly works except one test that seems to fail for a reason that seems totally unrelated to these changes:
```text
/home/travis/shared/logs/historical.log logtail ========================
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
Caused by: java.lang.IllegalArgumentException: Could not resolve type id 's3_zip' as a subtype of `org.apache.druid.segment.loading.LoadSpec`: known type ids = [local]
at [Source: UNKNOWN; line: -1, column: -1]
```
Anyone have any idea what causes this error?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] paul-rogers commented on pull request #12115: Support extension services
Posted by GitBox <gi...@apache.org>.
paul-rogers commented on pull request #12115:
URL: https://github.com/apache/druid/pull/12115#issuecomment-1012617113
Looks like the tests are flaky. Whacking the tests to run them again made the problem go away, so this is now ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] paul-rogers closed pull request #12115: Support extension services
Posted by GitBox <gi...@apache.org>.
paul-rogers closed pull request #12115:
URL: https://github.com/apache/druid/pull/12115
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] paul-rogers commented on pull request #12115: Support extension services
Posted by GitBox <gi...@apache.org>.
paul-rogers commented on pull request #12115:
URL: https://github.com/apache/druid/pull/12115#issuecomment-1008471046
There are failures in the build, but the same tests pass just fine on my Mac. From the build:
```text
[ERROR] testMoveSegment(org.apache.druid.server.coordinator.CuratorDruidCoordinatorTest) Time elapsed: 60.016 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 60 seconds
...
[ERROR] testMoveSegment(org.apache.druid.server.coordinator.CuratorDruidCoordinatorTest) Time elapsed: 60.016 s <<< ERROR!
java.lang.Exception: Appears to be stuck in thread NIOServerCxnFactory.SelectorThread-0
...
```
When run on the Mac:
```text
[INFO] Running org.apache.druid.server.coordinator.CuratorDruidCoordinatorTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.402 s - in org.apache.druid.server.coordinator.CuratorDruidCoordinatorTest
```
And, from the build:
```text
[ERROR] testWithoutCache(org.apache.druid.segment.realtime.firehose.SqlFirehoseFactoryTest) Time elapsed: 0.602 s <<< ERROR!
java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "org.apache.druid.data.input.impl.prefetch.ObjectOpenFunction.open(Object, java.io.File)" because "this.openObjectFunction" is null
```
When run on the Mac:
```text
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.986 s - in org.apache.druid.segment.realtime.firehose.SqlFirehoseFactoryTest
```
Any suggestions about how to resolve a test that works locally but fails when run on GitHub?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org