[jira] [Commented] (SLIDER-1090) Speed up SliderClient and AM by reading metainfo from accompany file

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLIDER-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163375#comment-15163375 ] 

Steve Loughran commented on SLIDER-1090:
----------------------------------------

Quick overview
* I don't think "acompany" is the term we'd use for the file. Associated File?
* If there is both an XML and a JSON, should the user be warned? 
* {{TestAgentUtils.}} is CRLF'd
* asserts (in the java tests) should use {{Assert.assertEquals(}} for their checks, with the static value first. This gives us a lot more diagnostics when things fail, and is especially useful on jenkins runs.

As an example, that final assert should be come
{code}
assertNotNull(metainfo.getApplication())
assertEquals("MYTESTAPPLICATION", metainfo.getApplication().getName());
{code}

you could also use try-with-resources to get a better close of your output streams:
{code}

try (BufferedWriter writer = new BufferedWriter(new FileWriter(f))) {
  writer.write(metainfo_str)
}

{code}

Finally, we'll need some docs somewhere. That's in svn so not something that can be covered in the same patch. (ASF site publishing is all svn based). I can help there. Actually, I should document how to contribute doc patches, shouldn't i?


> Speed up SliderClient and AM by reading metainfo from accompany file
> --------------------------------------------------------------------
>
>                 Key: SLIDER-1090
>                 URL: https://issues.apache.org/jira/browse/SLIDER-1090
>             Project: Slider
>          Issue Type: Bug
>    Affects Versions: Slider 0.81, Slider 0.90.2
>            Reporter: shanyu zhao
>         Attachments: SLIDER-1090.patch
>
>
> Today both SliderClient and SliderAM read package metainfo from HDFS. This requires downloading the whole package from HDFS. If package size is big (~200MB), this operation wastes tens of seconds in starting up the application.
> To speed up the operation of reading metainfo, we can read from an accompany xml/json file if it exists in the same HDFS folder as the zip package. If the accompany file does not exist, it falls back to the old way of downloading the zip package.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)