You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/07/16 22:32:04 UTC
[jira] [Commented] (STORM-942) Add FluxParser method
parseInputStream() to eliminate disk usage.
[ https://issues.apache.org/jira/browse/STORM-942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630281#comment-14630281 ]
ASF GitHub Bot commented on STORM-942:
--------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/637#discussion_r34834409
--- Diff: external/flux/flux-core/src/main/java/org/apache/storm/flux/parser/FluxParser.java ---
@@ -44,40 +44,47 @@ private FluxParser(){}
// TODO refactor input stream processing (see parseResource() method).
public static TopologyDef parseFile(String inputFile, boolean dumpYaml, boolean processIncludes,
String propertiesFile, boolean envSub) throws IOException {
- Yaml yaml = yaml();
+
FileInputStream in = new FileInputStream(inputFile);
- // TODO process properties, etc.
- TopologyDef topology = loadYaml(yaml, in, propertiesFile, envSub);
+ TopologyDef topology = parseInputStream(in, dumpYaml, processIncludes, propertiesFile, envSub);
in.close();
- if(dumpYaml){
- dumpYaml(topology, yaml);
- }
- if(processIncludes) {
- return processIncludes(yaml, topology, propertiesFile, envSub);
- } else {
- return topology;
- }
+
+ return topology;
}
public static TopologyDef parseResource(String resource, boolean dumpYaml, boolean processIncludes,
String propertiesFile, boolean envSub) throws IOException {
- Yaml yaml = yaml();
+
InputStream in = FluxParser.class.getResourceAsStream(resource);
- if(in == null){
- LOG.error("Unable to load classpath resource: " + resource);
- System.exit(1);
- }
- TopologyDef topology = loadYaml(yaml, in, propertiesFile, envSub);
+ TopologyDef topology = parseInputStream(in, dumpYaml, processIncludes, propertiesFile, envSub);
in.close();
- if(dumpYaml){
- dumpYaml(topology, yaml);
- }
- if(processIncludes) {
- return processIncludes(yaml, topology, propertiesFile, envSub);
- } else {
- return topology;
- }
+
+ return topology;
}
+
+ public static TopologyDef parseInputStream(InputStream inputStream, boolean dumpYaml, boolean processIncludes,
+ String propertiesFile, boolean envSub) throws IOException {
+
+ Yaml yaml = yaml();
+
+ if (inputStream == null) {
--- End diff --
@lyonbrw Indentation looks broken.
> Add FluxParser method parseInputStream() to eliminate disk usage.
> -----------------------------------------------------------------
>
> Key: STORM-942
> URL: https://issues.apache.org/jira/browse/STORM-942
> Project: Apache Storm
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Brendan W Lyon
> Fix For: 0.11.0
>
>
> Included reusable parseInputStream for use by parseFile and parseResource. Also allows a path for programatically creating topologies without the need to write to / read from disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)