You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/08/31 08:33:37 UTC

[GitHub] [kafka] yashmayya opened a new pull request, #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest

yashmayya opened a new pull request, #12575:
URL: https://github.com/apache/kafka/pull/12575

   - [ConnectRestApiTest](https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/connect/connect_rest_test.py) is currently failing on `trunk` and `3.3` with the following assertion error:
   
   ```
   AssertionError()
   Traceback (most recent call last):
     File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 183, in _do_run
       data = self.run_test()
     File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 243, in run_test
       return self.test_context.function(self.test)
     File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 433, in wrapper
       return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
     File "/opt/kafka-dev/tests/kafkatest/tests/connect/connect_rest_test.py", line 106, in test_rest_api
       self.verify_config(self.FILE_SOURCE_CONNECTOR, self.FILE_SOURCE_CONFIGS, configs)
     File "/opt/kafka-dev/tests/kafkatest/tests/connect/connect_rest_test.py", line 219, in verify_config
       assert config_def == set(config_names)
   ```
   
   - On closer inspection, this is because of the new source connector EOS related configs added in https://github.com/apache/kafka/pull/11775. 
   
   - Adding the following new configs - `offsets.storage.topic, transaction.boundary, exactly.once.support, transaction.boundary.interval.ms` in the expected config defs [here](https://github.com/apache/kafka/blob/6f4778301b1fcac1e2750cc697043d674eaa230d/tests/kafkatest/tests/connect/connect_rest_test.py#L35) fixes the tests on the 3.3 branch. However, the tests still fail on trunk due to the changes from https://github.com/apache/kafka/pull/12450.
   
   - The plan to fix this is to raise two PRs against trunk patching `connect_rest_test.py` - the first one fixing the EOS configs related issue which can be backported to 3.3 and the second one fixing the issue related to propagation of full connector configs to tasks which shouldn't be backported to 3.3 (because the commit from https://github.com/apache/kafka/pull/12450 is only on trunk and not on 3.3)
   
   - This is the first of the two PRs and should be backported to `3.3`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [kafka] yashmayya commented on pull request #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest

Posted by GitBox <gi...@apache.org>.
yashmayya commented on PR #12575:
URL: https://github.com/apache/kafka/pull/12575#issuecomment-1232638498

   @C0urante @jsancio could you please take a look at this whenever possible?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [kafka] C0urante commented on pull request #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest

Posted by GitBox <gi...@apache.org>.
C0urante commented on PR #12575:
URL: https://github.com/apache/kafka/pull/12575#issuecomment-1241138425

   Now that we've merged and backported https://github.com/apache/kafka/pull/11783, this should hopefully no longer be necessary. @yashmayya let me know if anything still fails; otherwise, we may want to close this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [kafka] jsancio commented on pull request #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest

Posted by GitBox <gi...@apache.org>.
jsancio commented on PR #12575:
URL: https://github.com/apache/kafka/pull/12575#issuecomment-1241267992

   I am going to close the PR. @yashmayya feel free to open the PR if you disagree.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [kafka] C0urante commented on pull request #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest

Posted by GitBox <gi...@apache.org>.
C0urante commented on PR #12575:
URL: https://github.com/apache/kafka/pull/12575#issuecomment-1232935220

   This is already addressed in https://github.com/apache/kafka/pull/11783. I'm not sure we can backport that to the 3.3 release branch but I'd like to try given that it has a few other low-risk quality-of-life improvements for Connect and was used to verify KIP-618 for the 3.3 release. If absolutely necessary, we can backport just this change, but I'd like to wait for a bit before making that call.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [kafka] jsancio closed pull request #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest

Posted by GitBox <gi...@apache.org>.
jsancio closed pull request #12575: KAFKA-14193: Add EOS related source connector configs to expected config defs in ConnectRestApiTest
URL: https://github.com/apache/kafka/pull/12575


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org