You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "allthingssecurity (via GitHub)" <gi...@apache.org> on 2026/10/12 00:05:59 UTC

[PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again [camel]

allthingssecurity opened a new pull request, #27695:
URL: https://github.com/apache/camel/pull/27695

   # Description
   
   [CAMEL-25548](https://issues.apache.org/jira/browse/CAMEL-25548)
   
   Since Twitter4J 4.1 a `twitter4j.v1.Query` is immutable: `sinceId`, `maxId`, `lang`, `count` and `geoCode` return a new `Query`. When Camel 4.0.0 moved to Twitter4J 4.1.2 the setters in `SearchConsumerHandler` and `SearchProducer` were renamed, but the returned `Query` was not used, so the query sent to Twitter had only the keywords:
   
   - the `twitter-search` consumer ignored `filterOld` (default `true`) and routed the most recent tweets of the search again at every poll (default every 30 seconds); it also ignored `lang`, `count` and the geolocation options;
   - the producer ignored `sinceId`, `lang`, `count` and the headers `CamelTwitterSinceId`, `CamelTwitterMaxId`, `CamelTwitterSearchLanguage` and `CamelTwitterCount`.
   
   This change keeps the `Query` that each call returns, in both classes. It also corrects the blank check of `SearchConsumerHandler.directConsume`, which was inverted in a later cleanup, so that a consumer with `type=direct` returned nothing when keywords were given.
   
   The options now work as documented (and as in Camel 3.x). Users can notice it: a poll routes only the tweets newer than the last one consumed, and a page has `count` tweets (default 5, where Twitter's default of 15 applied). The 4.23 upgrade guide has a note that says so and suggests raising `count` and `numberOfPages` for a search that matches more tweets between two polls.
   
   This PR also fixes the double request of the timeline consumers, in the same module: `AbstractStatusConsumerHandler.pollConsume` (home, mentions, user and list timelines) called `doPoll()`, which is the request to Twitter, twice in every poll. The tweets of the first response were routed, and the second response was only used for the id of the last tweet (the `since_id` of the next poll). So every poll sent two requests where one is enough, and a tweet posted between the two requests was only in the second response and was skipped. Now a poll sends one request and takes the last id from the tweets it routes. There is no upgrade guide note for this part: the consumer routes the same tweets with half the requests.
   
   Tests:
   - New `TwitterSearchQueryOptionsTest`, without network: a mocked `Twitter` client is set on the endpoint (`TwitterConfiguration.setTwitter`); its search records the query and returns the tweets with an id above `since_id` and up to `max_id`. The consumer is created with `startScheduler=false` and polled directly.
   - On main five of its six tests fail, for example `The second poll must not route the same tweets again ==> expected: <[10, 11]> but was: <[10, 11, 10, 11]>` and `expected: <[11, 12]> but was: <[10, 11, 12, 13]>` for the producer headers. The sixth (`filterOld=false` polls the tweets again) passes on main too.
   - New `TwitterTimelinePollTest`, same setup (mocked client whose home timeline returns the tweets above `since_id`): one test counts the requests of a poll, the other posts a tweet right after the first request has been answered and expects the next poll to route it. On main both fail: `A poll must send one request to the timeline ==> expected: <1> but was: <2>` and `expected: <[10, 11]> but was: <[10]>`.
   - With the change: camel-twitter 14 unit tests, 0 failures. The integration tests need Twitter credentials and were not run.
   - `mockito-junit-jupiter` is added as a test dependency of camel-twitter; both new tests use it.
   
   Not changed here: in the producer `filterOld` is still overridden by `sinceId` (default 1), as in Camel 3.x; and `numberOfPages` of the timeline endpoints is passed to `Paging.ofPage`, so a value above 1 requests that page only (the default of 1 is not affected).
   
   Found with a search for calls whose result is dropped on immutable types and a small Lean 4 model of the poll: on main a second poll with no new tweet routes all the tweets again, for every list of tweets; with the fix it routes nothing, and a poll routes exactly the tweets above the last id. I then reproduced it with the consumer and the producer.
   
   # Target
   
   - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch)
   
   # Tracking
   - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
   
   # Apache Camel coding standards and style
   
   - [x] I checked that each commit in the pull request has a meaningful subject line and body.
   - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes.
     (I built and tested camel-twitter, including the formatter and import-sort plugins. I did not run the full root build.)
   
   # AI-assisted contributions
   
   - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used.
     This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer.
   
   _Claude Code on behalf of allthingssecurity_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6129562796

   Conflict resolved (1e634e2a6a0c, rebased on `main`): the only conflicting file was `camel-4x-upgrade-guide-4_24.adoc`, where other PRs had added sections; they are all kept and this PR's section follows them. The code is unchanged from the approved commits.
   
   _Claude Code on behalf of allthingssecurity_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6120919399

   Thanks. Moved the upgrade-guide note out of `camel-4x-upgrade-guide-4_23.adoc` into a new `camel-4x-upgrade-guide-4_24.adoc` under `== Upgrading Camel 4.23 to 4.24` (created with the same header as the 4_23 guide), and added `- xref:camel-4x-upgrade-guide-4_24.adoc[Upgrade guide 4.23 -> 4.24]` to `camel-4x-upgrade-guide.adoc`.
   
   _Claude Code on behalf of allthingssecurity_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #27695:
URL: https://github.com/apache/camel/pull/27695


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6120343634

   <!-- ci-tested-modules -->
   :test_tube: **CI tested the following changed modules:**
   
   - `components/camel-twitter`
   - `docs`
   
   ---
   
   <details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 704 tested, 3 compile-only — current: 9 all tested</summary>
   
   [Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 9).
   
   **Skip-tests mode would test 9 modules** (2 direct + 0 downstream), **skip tests for 3** (generated code, meta-modules)
   
   <details><summary>Modules Scalpel would test (9)</summary>
   
   - `camel-jbang-mcp` ← depends on affected reactor module org.apache.camel:camel-yaml-dsl-validator
   - `camel-jbang-plugin-mcp` ← depends on affected reactor module org.apache.camel:camel-jbang-core
   - `camel-jbang-plugin-route-parser` ← depends on affected reactor module org.apache.camel:camel-route-parser
   - `camel-jbang-plugin-tui` ← depends on affected reactor module org.apache.camel:camel-yaml-dsl-validator
   - `camel-jbang-plugin-validate` ← depends on affected reactor module org.apache.camel:camel-yaml-dsl-validator
   - `camel-launcher-container` ← depends on affected reactor module org.apache.camel:camel-launcher
   - `camel-twitter` ← components/camel-twitter/src/main/java/org/apache/camel/component/twitter/search/SearchConsumerHandler.java, components/camel-twitter/src/main/java/org/apache/camel/component/twitter/search/SearchProducer.java, components/camel-twitter/src/main/java/org/apache/camel/component/twitter/timeline/AbstractStatusConsumerHandler.java, components/camel-twitter/src/test/java/org/apache/camel/component/twitter/TwitterSearchQueryOptionsTest.java, components/camel-twitter/src/test/java/org/apache/camel/component/twitter/TwitterTimelinePollTest.java, own pom components/camel-twitter/pom.xml changed
   - `camel-yaml-dsl-validator` ← depends on affected reactor module org.apache.camel:camel-catalog
   - `camel-yaml-dsl-validator-maven-plugin` ← depends on affected reactor module org.apache.camel:camel-yaml-dsl-validator
   
   </details>
   
   <details><summary>Modules with tests skipped (3)</summary>
   
   - `camel-itest`
   - `camel-yaml-dsl`
   - `camel-yaml-dsl-deserializers`
   
   </details>
   
   > :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
   
   </details>
   
   <details><summary>All tested modules (36 modules, 5m 16s total)</summary>
   
   **Total reactor time:** 5m 16s
   
   | Module | Duration | Status |
   | --- | --- | --- |
   | Camel :: Launcher | 47.5s | SUCCESS |
   | Camel :: JBang :: MCP | 39.4s | SUCCESS |
   | Camel :: Component DSL | 24.4s | SUCCESS |
   | Camel :: Catalog :: Camel Catalog | 22.8s | SUCCESS |
   | Camel :: YAML DSL :: Validator | 20.5s | SUCCESS |
   | Camel :: JBang :: Plugin :: Kubernetes | 19.6s | SUCCESS |
   | Camel :: YAML DSL | 19.4s | SUCCESS |
   | Camel :: JBang :: Plugin :: Validate | 16.4s | SUCCESS |
   | Camel :: Docs | 15.1s | SUCCESS |
   | Camel :: Twitter | 12.1s | SUCCESS |
   | Camel :: Kamelet Main | 11.2s | SUCCESS |
   | Camel :: JBang :: Plugin :: Testing | 10.9s | SUCCESS |
   | Camel :: YAML DSL :: Deserializers | 8.9s | SUCCESS |
   | Camel :: Catalog :: Camel Route Parser | 8.5s | SUCCESS |
   | Camel :: Catalog :: Camel Report Maven Plugin | 6.5s | SUCCESS |
   | Camel :: All Components Sync point | 5.2s | SUCCESS |
   | Camel :: YAML DSL :: Validator Maven Plugin | 4.5s | SUCCESS |
   | Camel :: YAML DSL :: Maven Plugins | 3.4s | SUCCESS |
   | Camel :: Catalog :: Maven | 2.8s | SUCCESS |
   | Camel :: Catalog :: Suggest (deprecated) | 2.2s | SUCCESS |
   | Camel :: JBang :: Integration tests | 1.9s | SUCCESS |
   | Camel :: Assembly | 1.7s | SUCCESS |
   | Camel :: JBang :: Plugin :: MCP | 1.6s | SUCCESS |
   | Camel :: JBang :: Plugin :: Edit | 1.4s | SUCCESS |
   | Camel :: Coverage | 1.2s | SUCCESS |
   | Camel :: Endpoint DSL :: Support | 1.1s | SUCCESS |
   | Camel :: JBang :: Plugin :: Generate | 1.1s | SUCCESS |
   | Camel :: Catalog :: Console | 1.1s | SUCCESS |
   | Camel :: Catalog :: Dummy Component | 1.1s | SUCCESS |
   | Camel :: JBang :: Main | 0.8s | SUCCESS |
   | Camel :: Launcher :: Container | 0.8s | SUCCESS |
   | Camel :: JBang :: Plugin :: Route Parser | 0.5s | SUCCESS |
   | Camel :: Endpoint DSL | n/a |  |
   | Camel :: Integration Tests | n/a |  |
   | Camel :: JBang :: Core | n/a |  |
   | Camel :: JBang :: Plugin :: TUI | n/a |  |
   
   **Top 20 slowest modules:**
   - `Camel :: Launcher` (47.5s)
   - `Camel :: JBang :: MCP` (39.4s)
   - `Camel :: Component DSL` (24.4s)
   - `Camel :: Catalog :: Camel Catalog` (22.8s)
   - `Camel :: YAML DSL :: Validator` (20.5s)
   - `Camel :: JBang :: Plugin :: Kubernetes` (19.6s)
   - `Camel :: YAML DSL` (19.4s)
   - `Camel :: JBang :: Plugin :: Validate` (16.4s)
   - `Camel :: Docs` (15.1s)
   - `Camel :: Twitter` (12.1s)
   - `Camel :: Kamelet Main` (11.2s)
   - `Camel :: JBang :: Plugin :: Testing` (10.9s)
   - `Camel :: YAML DSL :: Deserializers` (8.9s)
   - `Camel :: Catalog :: Camel Route Parser` (8.5s)
   - `Camel :: Catalog :: Camel Report Maven Plugin` (6.5s)
   - `Camel :: All Components Sync point` (5.2s)
   - `Camel :: YAML DSL :: Validator Maven Plugin` (4.5s)
   - `Camel :: YAML DSL :: Maven Plugins` (3.4s)
   - `Camel :: Catalog :: Maven` (2.8s)
   - `Camel :: Catalog :: Suggest (deprecated)` (2.2s)
   
   </details>
   
   ---
   :gear: [View full build and test results](https://github.com/apache/camel/actions/runs/38186721123)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6125048829

   Rebased on `main` (da779d90ef1d): the 4.24 upgrade guide is on main now, so the only conflict was that file, and this PR's section is kept after the existing one. The code is unchanged from the reviewed commits.
   
   _Claude Code on behalf of allthingssecurity_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #27695:
URL: https://github.com/apache/camel/pull/27695#discussion_r4246115841


##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -1050,6 +1050,16 @@ in the charset that the `Content-Type` declares, so the bytes match the header.
 the declared charset and read such a response as UTF-8 must now use the declared charset, and characters that the
 declared charset cannot represent are written as `?`.
 
+=== camel-twitter - twitter-search applies its query options again

Review Comment:
   This section belongs in `camel-4x-upgrade-guide-4_24.adoc` (see the review summary).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6131638692

   Conflict in the 4.24 upgrade guide resolved again (3becd179b52d); the code is unchanged.
   
   _Claude Code on behalf of allthingssecurity_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6130149734

   @allthingssecurity merge conflict


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6120112648

   The title now names both fixes: "CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a
   timeline poll sends one request". The description already had a paragraph and the tests for the double request in
   `AbstractStatusConsumerHandler`. No code change. Noted that this stays on main.
   
   _Claude Code on behalf of allthingssecurity_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again, and a timeline poll sends one request [camel]

Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6130239584

   @davsclaus resolved again (e0873698086f): only `camel-4x-upgrade-guide-4_24.adoc` conflicted, all sections of main are kept and this PR's section follows them; the code is unchanged. Each merge of one of these PRs makes the others conflict in that one file again; I re-resolve them within minutes of every merge.
   
   _Claude Code on behalf of allthingssecurity_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-25548: camel-twitter - twitter-search sends the query options again [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27695:
URL: https://github.com/apache/camel/pull/27695#issuecomment-6119652356

   :star2: Thank you for your contribution to the Apache Camel project! :star2:
     :robot: CI automation will test this PR automatically.
   
     :camel: Apache Camel Committers, please review the following items:
   
     * First-time contributors **require MANUAL approval** for the GitHub Actions to run
     * You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
     * You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
     * Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
     
     :warning: Be careful when sharing logs. Review their contents before sharing them publicly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org