You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Ernst Bunders (JIRA)" <ji...@apache.org> on 2010/09/14 16:02:40 UTC
[jira] Created: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
polling of feeds in FeedEntryPollingConsumer is broken.
-------------------------------------------------------
Key: CAMEL-3124
URL: https://issues.apache.org/activemq/browse/CAMEL-3124
Project: Apache Camel
Issue Type: Bug
Components: camel-rss
Affects Versions: 2.4.0
Environment: ubuntu 10.4
java 6
Reporter: Ernst Bunders
The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken. You can think of two ways that polling feeds could work:
1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
But the way it presently works:
- A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
This is easy to fix. In the below code:
public void poll() throws Exception {
Object feed = createFeed();
populateList(feed);
while (hasNextEntry()) {
Object entry = list.get(entryIndex--);
boolean valid = true;
if (entryFilter != null) {
valid = entryFilter.isValidEntry(endpoint, feed, entry);
}
if (valid) {
Exchange exchange = endpoint.createExchange(feed, entry);
getProcessor().process(exchange);
// return and wait for the next poll to continue from last time (this consumer is stateful)
return;
}
}
The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-3124:
-------------------------------
Fix Version/s: 2.5.0
(was: Future)
Lets get this improved for Camel 2.5 as we got time for this, waiting for Karaf 2.1 to be released.
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Priority: Minor
> Fix For: 2.5.0
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-3124.
--------------------------------
Resolution: Fixed
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.5.0
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Ernst Bunders (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61849#action_61849 ]
Ernst Bunders commented on CAMEL-3124:
--------------------------------------
Well, not strictly a bug. But it seems very strange behavior to me, If you should choose to process one item at each poll, here seems to be no reason to discard the list, in stead of keeping it over polls until depleted. Unless there is a reason I didn't think of...
Is there?
Also I think it would be nice to make the behavior configurable. We now chose to process all feed items with one poll, because it suites our needs, perhaps other use cases are best served with the one-item-per-poll strategy.
regards,
Ernst
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Priority: Minor
> Fix For: Future
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-3124:
-------------------------------
Original Estimate: (was: 5 minutes)
Remaining Estimate: (was: 5 minutes)
Issue Type: Improvement (was: Bug)
Priority: Minor (was: Major)
Description:
The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
You can think of two ways that polling feeds could work:
1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
But the way it presently works:
A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
This is easy to fix. In the below code:
{code}
public void poll() throws Exception {
Object feed = createFeed();
populateList(feed);
while (hasNextEntry()) {
Object entry = list.get(entryIndex--);
boolean valid = true;
if (entryFilter != null) {
valid = entryFilter.isValidEntry(endpoint, feed, entry);
}
if (valid) {
Exchange exchange = endpoint.createExchange(feed, entry);
getProcessor().process(exchange);
// return and wait for the next poll to continue from last time (this consumer is stateful)
return;
}
}
{code}
The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
was:
The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken. You can think of two ways that polling feeds could work:
1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
But the way it presently works:
- A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
This is easy to fix. In the below code:
public void poll() throws Exception {
Object feed = createFeed();
populateList(feed);
while (hasNextEntry()) {
Object entry = list.get(entryIndex--);
boolean valid = true;
if (entryFilter != null) {
valid = entryFilter.isValidEntry(endpoint, feed, entry);
}
if (valid) {
Exchange exchange = endpoint.createExchange(feed, entry);
getProcessor().process(exchange);
// return and wait for the next poll to continue from last time (this consumer is stateful)
return;
}
}
The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
This is how its designed to work form the very start, its not a bug.
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Priority: Minor
> Fix For: Future
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-3124:
----------------------------------
Assignee: Claus Ibsen
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.5.0
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61893#action_61893 ]
Claus Ibsen commented on CAMEL-3124:
------------------------------------
And you can set {{throttleEntries=false}} to let it parse the entire feed at once, and hence it doesn't keep state in memory between polls.
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.5.0
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-3124:
-------------------------------
Issue Type: Bug (was: Improvement)
After looking into it some more I agree this is a minor bug as it should keep the feed/list in memory to avoid re-loading the feed when it wasn't necessary.
trunk: 998075.
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.5.0
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-3124:
-------------------------------
Fix Version/s: Future
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Priority: Minor
> Fix For: Future
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3124) polling of feeds in
FeedEntryPollingConsumer is broken.
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61851#action_61851 ]
Claus Ibsen commented on CAMEL-3124:
------------------------------------
I agree we can improve this behavior, but however it was designed how it works now.
> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
> Key: CAMEL-3124
> URL: https://issues.apache.org/activemq/browse/CAMEL-3124
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-rss
> Affects Versions: 2.4.0
> Environment: ubuntu 10.4
> java 6
> Reporter: Ernst Bunders
> Priority: Minor
> Fix For: Future
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries' mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item to process is stored over polls but the list isn't. Also this creates a big network overhead when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
> Object feed = createFeed();
> populateList(feed);
> while (hasNextEntry()) {
> Object entry = list.get(entryIndex--);
> boolean valid = true;
> if (entryFilter != null) {
> valid = entryFilter.isValidEntry(endpoint, feed, entry);
> }
> if (valid) {
> Exchange exchange = endpoint.createExchange(feed, entry);
> getProcessor().process(exchange);
> // return and wait for the next poll to continue from last time (this consumer is stateful)
> return;
> }
> }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should be deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.