You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@storm.apache.org by "Ryan Persaud (JIRA)" <ji...@apache.org> on 2017/05/11 15:26:04 UTC

[jira] [Updated] (STORM-2509) Writers in AbstractHDFSBolt are not closed/rotated when evicted from WritersMap

     [ https://issues.apache.org/jira/browse/STORM-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan Persaud updated STORM-2509:
--------------------------------
    Description: 
When the eldest entry in the WritersMap in the AbstractHDFSBolt gets removed due to the number of writers exceeding maxWriters (see below), the writer is not closed and rotation actions are not executed (doRotationAndRemoveWriter is not called).

{code}
static class WritersMap extends LinkedHashMap<String, AbstractHDFSWriter> {
    final long maxWriters;

    public WritersMap(long maxWriters) {
        super((int)maxWriters, 0.75f, true);
        this.maxWriters = maxWriters;
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry<String, AbstractHDFSWriter> eldest) {
        return this.size() > this.maxWriters;
    }
}
{code}

  was:
When the eldest entry in the WritersMap in the AbstractHDFSBolt gets removed due to the number of writers exceeding maxWriters (see below), the writer is not closed and rotation actions are not executed (doRotationAndRemoveWriter is not called).

static class WritersMap extends LinkedHashMap<String, AbstractHDFSWriter> {
    final long maxWriters;

    public WritersMap(long maxWriters) {
        super((int)maxWriters, 0.75f, true);
        this.maxWriters = maxWriters;
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry<String, AbstractHDFSWriter> eldest) {
        return this.size() > this.maxWriters;
    }
}


> Writers in AbstractHDFSBolt are not closed/rotated when evicted from WritersMap
> -------------------------------------------------------------------------------
>
>                 Key: STORM-2509
>                 URL: https://issues.apache.org/jira/browse/STORM-2509
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-hdfs
>    Affects Versions: 1.1.0
>            Reporter: Ryan Persaud
>            Priority: Minor
>
> When the eldest entry in the WritersMap in the AbstractHDFSBolt gets removed due to the number of writers exceeding maxWriters (see below), the writer is not closed and rotation actions are not executed (doRotationAndRemoveWriter is not called).
> {code}
> static class WritersMap extends LinkedHashMap<String, AbstractHDFSWriter> {
>     final long maxWriters;
>     public WritersMap(long maxWriters) {
>         super((int)maxWriters, 0.75f, true);
>         this.maxWriters = maxWriters;
>     }
>     @Override
>     protected boolean removeEldestEntry(Map.Entry<String, AbstractHDFSWriter> eldest) {
>         return this.size() > this.maxWriters;
>     }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)