[jira] [Comment Edited] (STORM-2358) Update storm hdfs spout to remove specific implementation handlings

["Sachin Pasalkar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864882#comment-15864882 ] 

Sachin Pasalkar edited comment on STORM-2358 at 2/14/17 2:17 AM:
-----------------------------------------------------------------

For point 1:

Its specific use case just to support why it needs to be public.

For point 2:

We are limiting code to be very specific to these 2 implementations we should have generic implementation. I see there is another check-in happening for ZippedTextFileReader (https://github.com/apache/storm/pull/1928#pullrequestreview-21597154). I have attached my code changes in JIRA, please take a look, where you need to provide class. 

For point 3:

Lets assume I have multiple topologies with different readers. So I defined the a base topology class with HDFSSpout in it. Now I always needs to pass the outputFields as separate array. This actually can be part of every reader class as its very specific to it.



was (Author: sachin):
For point 1:

Its specific use case just to support why it needs to be public.

For point 2:

We are limiting code to be very specific to these 2 implementations we should have generic implementation. I see there is another check-in happening for ZippedTextFileReader.I have attached my code changes in JIRA, please take a look, where you need to provide class. 

For point 3:

Lets assume I have multiple topologies with different readers. So I defined the a base topology class with HDFSSpout in it. Now I always needs to pass the outputFields as separate array. This actually can be part of every reader class as its very specific to it.


> Update storm hdfs spout to remove specific implementation handlings
> -------------------------------------------------------------------
>
>                 Key: STORM-2358
>                 URL: https://issues.apache.org/jira/browse/STORM-2358
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-hdfs
>    Affects Versions: 1.x
>            Reporter: Sachin Pasalkar
>            Assignee: Sachin Pasalkar
>              Labels: newbie
>         Attachments: AbstractFileReader.java, FileReader.java, HDFSSpout.java, SequenceFileReader.java, TextFileReader.java
>
>
> I was looking at storm hdfs spout code in 1.x branch, I found below
> improvements can be made in below code.
>   1.  Make org.apache.storm.hdfs.spout.AbstractFileReader as public so
> that it can be used in generics.
>   2.  org.apache.storm.hdfs.spout.HdfsSpout requires readerType as
> String. It will be great to have class<? extends AbstractFileReader>
> readerType; So we will not use Class.forName at multiple places also it
> will help in below point.
>   3.  HdfsSpout also needs to provide outFields which are declared as
> constants in each reader(e.g.SequenceFileReader). We can have abstract
> API AbstractFileReader in which return them to user to make it generic.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)