[jira] [Commented] (CAMEL-8539) S3 marker does not work

["Luca Burgazzoli (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121378#comment-15121378 ] 

Luca Burgazzoli commented on CAMEL-8539:
----------------------------------------

Looks like this has been fixed via [CAMEL-8431|https://issues.apache.org/jira/browse/CAMEL-8431]

{code:java}
if (!getConfiguration().isDeleteAfterRead()) {
    // if the marker is truncated, the nextMarker should not be null
    if (listObjects.getNextMarker() != null) {
        marker = listObjects.getNextMarker();
    } else {
        // if there is no marker, the files are consumed, we should not pull it again
        filesConsumed = true;
    }
}
{code}

isn't it ?



> S3 marker does not work
> -----------------------
>
>                 Key: CAMEL-8539
>                 URL: https://issues.apache.org/jira/browse/CAMEL-8539
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-aws
>    Affects Versions: 2.15.0
>            Reporter: Alexey Hanin
>         Attachments: 8539.patch
>
>
> S3Consumer doesn't set marker correctly, when {{deleteAfterRead}} is set to {{false}}. Therefore, on every subsequent poll, it starts from the beginning of the list.
> The reason for that is the following code in S3Consumer:
> {code:lang=java|title=S3Consumer.java}
>             if (!getConfiguration().isDeleteAfterRead()) {
>                 // where marker is track
>                 marker = listObjects.getMarker();
>             }
> {code}
> Accordingly to S3 client documentation, {{getMarker()}} return a marker previously set in request. Instead, {{getNextMarker()}} shall be used. Although, simply changing it to use {{getNextMarker()}} introduces an endless loop iterating over the same set of keys under the prefix, because when reached the end of the key set, {{getNextMarker()}} returns {{null}}, and on the next poll, the head of the list will be returned, and so on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)