You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by GitBox <gi...@apache.org> on 2022/10/26 08:12:06 UTC
[GitHub] [incubator-streampipes] RobertIndie opened a new pull request, #128: Improve the integration test framework
RobertIndie opened a new pull request, #128:
URL: https://github.com/apache/incubator-streampipes/pull/128
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->
<!--
Thanks for contributing! Here are some tips you can follow to help us incorporate your contribution quickly and easily:
1. If this is your first time, please read our contributor guidelines:
- https://streampipes.apache.org/getinvolved.html
- https://cwiki.apache.org/confluence/display/STREAMPIPES/Getting+Started
2. Make sure the PR title is formatted like: `[STREAMPIPES-<Jira issue #>] PR title ...`
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., `[WIP][STREAMPIPES-<Jira issue #>] PR title ...`.
4. Please write your PR title to summarize what this PR proposes/fixes.
5. Be sure to keep the PR description updated to reflect all changes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. Make sure tests pass via `mvn clean install`.
8. (Optional) If the contribution is large, please file an Apache ICLA
- http://apache.org/licenses/icla.pdf
-->
### Purpose
Improve the integration test framework
### Approach
* Refine the interface of `AdapterTesterBase`
* Use the low level approach to start and stop the adapter
* Add `DebugAdapterSInk` to collect the data for verification
* Improve the pulsar adapter integration tests
### Samples
<!--
Provide high-level details about the samples related to this feature.
-->
### Remarks
<!--
List related issues/PRs, link to discussions in the mailing list, todo items, or any other notes related to the PR.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] RobertIndie commented on pull request #128: Improve the integration test framework
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on PR #128:
URL: https://github.com/apache/incubator-streampipes/pull/128#issuecomment-1295803058
Hi @tenthe . Thanks for your testing.
> I ran it without the env variable `TEST_MODE` starting a new docker service with Pulsar. Do you have any ideas what the problem might be?
I find that there is a race condition issue in the PulsarProtocol. When the `PulsarProtocol.run` returns, the pulsar consumer may not be set up completely. We need to wait for the subscribe operation to complete before `PulsarProtocol.run` returns. I think the best way for it is to do some refactor for the PulsarProtocol by using the CompletableFuture. I will do it in another PR. This PR only lets the consumer subscribe to the earliest position to avoid this problem. This is just a temporary solution because it changes the default configuration. Also, I will submit a new PR to make the subscription position configurable.
> Therefore, we can use the builder class `SpecificDataStreamAdapterBuilder`.
Great! Thanks for your information and your help. Do you want to submit the patch to this PR or create a new PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] dominikriemer merged pull request #128: Improve the integration test framework
Posted by GitBox <gi...@apache.org>.
dominikriemer merged PR #128:
URL: https://github.com/apache/incubator-streampipes/pull/128
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] tenthe commented on pull request #128: Improve the integration test framework
Posted by GitBox <gi...@apache.org>.
tenthe commented on PR #128:
URL: https://github.com/apache/incubator-streampipes/pull/128#issuecomment-1292152036
Thanks for providing the PR. I really like the new version for the test framework, especially the `DebugAdapterSink` that can be used to get the resulting event via a `BlockingQueue`.
I tried to run the test locally but got a NullPointer exception in PulsarAdapterTester in line 111. The method takeEvent returns null.
I ran it without the env variable `TEST_MODE` starting a new docker service with Pulsar. Do you have any ideas what the problem might be?
Once I have a running local version I can also help you to refactor the part in the prepareAdapter method. Therefore, we can use the builder class `SpecificDataStreamAdapterBuilder`. We probably have to extend it a little bit to also directly add the values for the configurations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org