You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by iyovcheva <gi...@git.apache.org> on 2016/01/27 10:25:30 UTC

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

GitHub user iyovcheva opened a pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176

     Generate config/effector descriptions for yaml-only blueprints

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/iyovcheva/incubator-brooklyn generate-config-from-yaml

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-brooklyn/pull/1176.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1176
    
----
commit 590a34bacbe95738bcc88734b3efc7feb2520e45
Author: Ivana Yovcheva <iv...@gmail.com>
Date:   2016-01-26T16:57:08Z

     Generate config/effector descriptions for yaml-only blueprints

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50986452
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    +                            config.add(entityConfigSummary);
    +                        }
    +                        itemDescriptor.put("config", config);
    +                        if (item.getCatalogItemType().name().equals("ENTITY")) {
    +                            entities.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("POLICY")) {
    +                            policies.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("LOCATION")) {
    +                            locations.add(itemDescriptor);
    +                        }
    +                    }
    +                    result = ImmutableMap.<String, Object>builder()
    +                            .put("entities", entities)
    +                            .put("policies", policies)
    +                            .put("enrichers", enrichers)
    +                            .put("locations", locations)
    +                            .put("locationResolvers", locationResolvers)
    +                            .build();
    +
    +                    jsonList.add(toJson(result));
                     }
    --- End diff --
    
    Remove the catalog items after processing them, so the next `addItems` call works with a clean catalog - avoiding conflicts.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
Github user iyovcheva commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r51100940
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -198,17 +277,36 @@ private void mkdir(String rootDir, String dirName) {
     
             protected List<URL> getUrls() throws MalformedURLException {
                 List<URL> urls = Lists.newArrayList();
    -            if (jars.isEmpty()) {
    -                String classpath = System.getenv("INITIAL_CLASSPATH");
    -                if (Strings.isNonBlank(classpath)) {
    -                    List<String> entries = Splitter.on(":").omitEmptyStrings().trimResults().splitToList(classpath);
    -                    for (String entry : entries) {
    -                        if (entry.endsWith(".jar") || entry.endsWith("/*")) {
    -                            urls.addAll(ClassFinder.toJarUrls(entry.replace("/*", "")));
    +            if (jars.isEmpty()) { //TODO: where is `jars` value write - AbstractMain.execCli(Cli<BrooklynCommand> parser, String ...args):line 250 BrooklynCommand command = parser.parse(args)
    --- End diff --
    
    It is not allowed now to parse jars and yaml at the same time. The idea was to prevent it and `jars` to have higher priority. Do you think it's better to throw exception after parsing the arguments, instead of making it impossible from the code?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
Github user iyovcheva commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r51270639
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    +                            config.add(entityConfigSummary);
    +                        }
    +                        itemDescriptor.put("config", config);
    +                        if (item.getCatalogItemType().name().equals("ENTITY")) {
    +                            entities.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("POLICY")) {
    +                            policies.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("LOCATION")) {
    +                            locations.add(itemDescriptor);
    +                        }
    +                    }
    +                    result = ImmutableMap.<String, Object>builder()
    +                            .put("entities", entities)
    +                            .put("policies", policies)
    +                            .put("enrichers", enrichers)
    +                            .put("locations", locations)
    +                            .put("locationResolvers", locationResolvers)
    +                            .build();
    +
    +                    jsonList.add(toJson(result));
                     }
    --- End diff --
    
    Fixed that. It's on another row and guess that's why it stays unchanged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by bostko <gi...@git.apache.org>.
Github user bostko commented on the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#issuecomment-176651040
  
    @tbouron @neykov if you had noticed docs were loading from a json file for a while.
    However I noticed the experience wasn't so smooth on production and reverted it back.
    When it is js file in the html it is automatically cached by the browser with no extra code.
    https://github.com/apache/incubator-brooklyn/pull/622


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50986073
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    +                            config.add(entityConfigSummary);
    +                        }
    +                        itemDescriptor.put("config", config);
    +                        if (item.getCatalogItemType().name().equals("ENTITY")) {
    +                            entities.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("POLICY")) {
    +                            policies.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("LOCATION")) {
    +                            locations.add(itemDescriptor);
    +                        }
    +                    }
    +                    result = ImmutableMap.<String, Object>builder()
    +                            .put("entities", entities)
    +                            .put("policies", policies)
    +                            .put("enrichers", enrichers)
    +                            .put("locations", locations)
    +                            .put("locationResolvers", locationResolvers)
    +                            .build();
    +
    +                    jsonList.add(toJson(result));
                     }
    -                
    -                List<Map<String, Object>> locations = (List<Map<String, Object>>) result.get("locations");
    -                String locationTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "location.html"));
    -                for (Map<String, Object> location : locations) {
    -                    String type = (String) location.get("type");
    -                    String locationHtml = TemplateProcessor.processTemplateContents(locationTemplateHtml, ImmutableMap.of("type", type));
    -                    Files.write(locationHtml, new File(Os.mergePaths(outputFolder, "locations", type + ".html")), Charsets.UTF_8);
    +            }
    +
    +            for (String json: jsonList) {
    +
    +                if (outputFolder == null) {
    +                    System.out.println(json);
    +                } else {
    +                    LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    +                    String outputPath = Os.mergePaths(outputFolder, "index.html");
    +                    String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    +                    mkdir(parentDir, "entities");
    +                    mkdir(parentDir, "policies");
    +                    mkdir(parentDir, "enrichers");
    +                    mkdir(parentDir, "locations");
    +                    mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    +
    +                    mkdir(parentDir, "style");
    +                    mkdir(Os.mergePaths(parentDir, "style"), "js");
    +                    mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    +
    +                    Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    --- End diff --
    
    Wouldn't this overwrite the file for each catalog entry? Also no need to re-generate the whole doc structure for each item as it's identical. 
    
    Maybe better to have a separate case for generating the output if coming from yaml, creating just the .js files? @tbouron wdyt?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
GitHub user iyovcheva reopened a pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176

     Generate config/effector descriptions for yaml-only blueprints

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/iyovcheva/incubator-brooklyn generate-config-from-yaml

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-brooklyn/pull/1176.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1176
    
----
commit 7774e46d81a18a17f315de599360888fbba50c10
Author: Ivana Yovcheva <iv...@gmail.com>
Date:   2016-01-26T16:57:08Z

     Generate config/effector descriptions for yaml-only blueprints

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50976497
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    --- End diff --
    
    Apart for ordering `priority` can be used to decide on whether an item should be shown to the user.
    Create a counter here, passing its value when `param.isPinned()`, `null` otherwise.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
Github user iyovcheva commented on the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#issuecomment-176244738
  
    Jenkins fails due to `DynamicSequentialTaskTest.testCancelled` which I think is not related to the issue.  The test is successful locally.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r51106768
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -198,17 +277,36 @@ private void mkdir(String rootDir, String dirName) {
     
             protected List<URL> getUrls() throws MalformedURLException {
                 List<URL> urls = Lists.newArrayList();
    -            if (jars.isEmpty()) {
    -                String classpath = System.getenv("INITIAL_CLASSPATH");
    -                if (Strings.isNonBlank(classpath)) {
    -                    List<String> entries = Splitter.on(":").omitEmptyStrings().trimResults().splitToList(classpath);
    -                    for (String entry : entries) {
    -                        if (entry.endsWith(".jar") || entry.endsWith("/*")) {
    -                            urls.addAll(ClassFinder.toJarUrls(entry.replace("/*", "")));
    +            if (jars.isEmpty()) { //TODO: where is `jars` value write - AbstractMain.execCli(Cli<BrooklynCommand> parser, String ...args):line 250 BrooklynCommand command = parser.parse(args)
    --- End diff --
    
    Yes, could print an error and exit, but I'm fine with current behaviour. What I was referring to though is the "TODO" note on the right.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by tbouron <gi...@git.apache.org>.
Github user tbouron commented on the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#issuecomment-175512415
  
    LGTM.
    
    One comment though. This generate a JS file which encapsulates the JSON data into a JS var. While this is ok for the documentation as it imports the file directly, it's a bit tedious to share the info with other services (i.e. community catalog) as we will need to `eval` this file to get the data.
    I know it was not introduced by this PR but would it be better to get this opportunity to change the format from JS to JSON?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50976043
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    --- End diff --
    
    You could treat `url` as a generic URL, so the app can fetch remote catalogs as well. Changing it to something like `Streams.readFullyString(url.openStream)` will be independent of the location.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#issuecomment-176947214
  
    test failure is clearly unrelated, maybe close and reopen to run again.
    
    meanwhile we should look at this /cc @aledsage :
    
    ```
    org.apache.brooklyn.entity.software.base.test.autoscaling.AutoScalerPolicyNoMoreMachinesTest.testResizeDirectly (from TestSuite)
    
    java.lang.AssertionError: removed=[EmptySoftwareProcessImpl{id=NhzxGta6}] expected [2] but found [1]
    	at org.testng.Assert.fail(Assert.java:94)
    	at org.testng.Assert.failNotEquals(Assert.java:494)
    	at org.testng.Assert.assertEquals(Assert.java:123)
    	at org.testng.Assert.assertEquals(Assert.java:370)
    	at org.apache.brooklyn.entity.software.base.test.autoscaling.AutoScalerPolicyNoMoreMachinesTest.assertSize(AutoScalerPolicyNoMoreMachinesTest.java:182)
    	at org.apache.brooklyn.entity.software.base.test.autoscaling.AutoScalerPolicyNoMoreMachinesTest.testResizeDirectly(AutoScalerPolicyNoMoreMachinesTest.java:105)
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50975207
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    --- End diff --
    
    This will read config from the user's `brooklyn.properties`. Better initialize it with an empty state using `new LocalManagementContext(BrooklynProperties.Factory.newEmpty());`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50975594
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -198,17 +277,36 @@ private void mkdir(String rootDir, String dirName) {
     
             protected List<URL> getUrls() throws MalformedURLException {
                 List<URL> urls = Lists.newArrayList();
    -            if (jars.isEmpty()) {
    -                String classpath = System.getenv("INITIAL_CLASSPATH");
    -                if (Strings.isNonBlank(classpath)) {
    -                    List<String> entries = Splitter.on(":").omitEmptyStrings().trimResults().splitToList(classpath);
    -                    for (String entry : entries) {
    -                        if (entry.endsWith(".jar") || entry.endsWith("/*")) {
    -                            urls.addAll(ClassFinder.toJarUrls(entry.replace("/*", "")));
    +            if (jars.isEmpty()) { //TODO: where is `jars` value write - AbstractMain.execCli(Cli<BrooklynCommand> parser, String ...args):line 250 BrooklynCommand command = parser.parse(args)
    --- End diff --
    
    TODO probably not needed any more?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by tbouron <gi...@git.apache.org>.
Github user tbouron commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50992238
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    +                            config.add(entityConfigSummary);
    +                        }
    +                        itemDescriptor.put("config", config);
    +                        if (item.getCatalogItemType().name().equals("ENTITY")) {
    +                            entities.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("POLICY")) {
    +                            policies.add(itemDescriptor);
    +                        } else if (item.getCatalogItemType().name().equals("LOCATION")) {
    +                            locations.add(itemDescriptor);
    +                        }
    +                    }
    +                    result = ImmutableMap.<String, Object>builder()
    +                            .put("entities", entities)
    +                            .put("policies", policies)
    +                            .put("enrichers", enrichers)
    +                            .put("locations", locations)
    +                            .put("locationResolvers", locationResolvers)
    +                            .build();
    +
    +                    jsonList.add(toJson(result));
                     }
    -                
    -                List<Map<String, Object>> locations = (List<Map<String, Object>>) result.get("locations");
    -                String locationTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "location.html"));
    -                for (Map<String, Object> location : locations) {
    -                    String type = (String) location.get("type");
    -                    String locationHtml = TemplateProcessor.processTemplateContents(locationTemplateHtml, ImmutableMap.of("type", type));
    -                    Files.write(locationHtml, new File(Os.mergePaths(outputFolder, "locations", type + ".html")), Charsets.UTF_8);
    +            }
    +
    +            for (String json: jsonList) {
    +
    +                if (outputFolder == null) {
    +                    System.out.println(json);
    +                } else {
    +                    LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    +                    String outputPath = Os.mergePaths(outputFolder, "index.html");
    +                    String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    +                    mkdir(parentDir, "entities");
    +                    mkdir(parentDir, "policies");
    +                    mkdir(parentDir, "enrichers");
    +                    mkdir(parentDir, "locations");
    +                    mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    +
    +                    mkdir(parentDir, "style");
    +                    mkdir(Os.mergePaths(parentDir, "style"), "js");
    +                    mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    +
    +                    Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    --- End diff --
    
    I don't think it matters in this case @neykov as a blueprint author will use the `br list-objects` command to generate this file for his YAML blueprint then copy it onto his blueprint git repository


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#issuecomment-177017397
  
    (easily fixed - the test was doing an immediate assert on something happening in another thread; PR to follow; @iyovcheva give it an hour or so for the other fix to land then please close+reopen this to run the jenkins tests again)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
Github user iyovcheva closed the pull request at:

    https://github.com/apache/incubator-brooklyn/pull/1176


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
GitHub user iyovcheva reopened a pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176

     Generate config/effector descriptions for yaml-only blueprints

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/iyovcheva/incubator-brooklyn generate-config-from-yaml

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-brooklyn/pull/1176.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1176
    
----
commit aab2fc9df56e4f333ff7a284384c080347a843ac
Author: Ivana Yovcheva <iv...@gmail.com>
Date:   2016-01-26T16:57:08Z

     Generate config/effector descriptions for yaml-only blueprints

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50976582
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    +                            config.add(entityConfigSummary);
    +                        }
    +                        itemDescriptor.put("config", config);
    +                        if (item.getCatalogItemType().name().equals("ENTITY")) {
    --- End diff --
    
    `getCatalogItemType()` returns an enum - compare against it instead of using strings.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50976207
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    --- End diff --
    
    The catalog item info should override the Java info, reverse the logic here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50976671
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -98,88 +115,150 @@ public Void call() throws Exception {
                 List<URL> urls = getUrls();
                 LOG.info("Retrieving objects from "+urls);
     
    -            // TODO Remove duplication from separate ListPolicyCommand etc
    -            List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    -            List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    -            List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    -            List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
    +            Map<String, Object> result = MutableMap.of();
    +            List<String> jsonList = new ArrayList<>();
     
    -            Map<String, Object> result = ImmutableMap.<String, Object>builder()
    -                    .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    -                    .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    -                    .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    -                    .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    -                    .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    -                    .build();
    +            if (!jars.isEmpty()) {
    +                // TODO Remove duplication from separate ListPolicyCommand etc
    +                List<Class<? extends Entity>> entityTypes = getTypes(urls, Entity.class);
    +                List<Class<? extends Policy>> policyTypes = getTypes(urls, Policy.class);
    +                List<Class<? extends Enricher>> enricherTypes = getTypes(urls, Enricher.class);
    +                List<Class<? extends Location>> locationTypes = getTypes(urls, Location.class, Boolean.FALSE);
     
    -            String json = toJson(result);
    +                result = ImmutableMap.<String, Object>builder()
    +                        .put("entities", ItemDescriptors.toItemDescriptors(entityTypes, headingsOnly, "name"))
    +                        .put("policies", ItemDescriptors.toItemDescriptors(policyTypes, headingsOnly, "name"))
    +                        .put("enrichers", ItemDescriptors.toItemDescriptors(enricherTypes, headingsOnly, "name"))
    +                        .put("locations", ItemDescriptors.toItemDescriptors(locationTypes, headingsOnly, "type"))
    +                        .put("locationResolvers", ItemDescriptors.toItemDescriptors(ImmutableList.copyOf(ServiceLoader.load(LocationResolver.class)), true))
    +                        .build();
    +                jsonList.add(toJson(result));
    +            } else if (!yaml.isEmpty()) {
    +                LocalManagementContext lmgmt = new LocalManagementContext();
    +                BrooklynCampPlatform platform = new BrooklynCampPlatform(
    +                        PlatformRootSummary.builder().name("Brooklyn CAMP Platform").build(),lmgmt)
    +                        .setConfigKeyAtManagmentContext();
    +                BrooklynCatalog catalog = lmgmt.getCatalog();
     
    -            if (outputFolder == null) {
    -                System.out.println(json);
    -            } else {
    -                LOG.info("Outputting item list (size "+itemCount+") to " + outputFolder);
    -                String outputPath = Os.mergePaths(outputFolder, "index.html");
    -                String parentDir = (new File(outputPath).getParentFile()).getAbsolutePath();
    -                mkdir(parentDir, "entities");
    -                mkdir(parentDir, "policies");
    -                mkdir(parentDir, "enrichers");
    -                mkdir(parentDir, "locations");
    -                mkdir(parentDir, "locationResolvers"); //TODO nothing written here yet...
    -                
    -                mkdir(parentDir, "style");
    -                mkdir(Os.mergePaths(parentDir, "style"), "js");
    -                mkdir(Os.mergePaths(parentDir, "style", "js"), "catalog");
    -                
    -                Files.write("var items = " + json, new File(Os.mergePaths(outputFolder, "items.js")), Charsets.UTF_8);
    -                ResourceUtils resourceUtils = ResourceUtils.create(this);
    -                
    -                // root - just loads the above JSON
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "brooklyn-object-list.html", "index.html");
    -                
    -                // statics - structure mirrors docs (not for any real reason however... the json is usually enough for our docs)
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "common.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "items.css");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.js");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/underscore-min.map");
    -                copyFromItemListerClasspathBaseStaticsToOutputDir(resourceUtils, "style/js/catalog/typeahead.js");
    -
    -                // now make pages for each item
    -                
    -                List<Map<String, Object>> entities = (List<Map<String, Object>>) result.get("entities");
    -                String entityTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "entity.html"));
    -                for (Map<String, Object> entity : entities) {
    -                    String type = (String) entity.get("type");
    -                    String name = (String) entity.get("name");
    -                    String entityHtml = TemplateProcessor.processTemplateContents(entityTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(entityHtml, new File(Os.mergePaths(outputFolder, "entities", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> policies = (List<Map<String, Object>>) result.get("policies");
    -                String policyTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "policy.html"));
    -                for (Map<String, Object> policy : policies) {
    -                    String type = (String) policy.get("type");
    -                    String name = (String) policy.get("name");
    -                    String policyHtml = TemplateProcessor.processTemplateContents(policyTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(policyHtml, new File(Os.mergePaths(outputFolder, "policies", type + ".html")), Charsets.UTF_8);
    -                }
    -                
    -                List<Map<String, Object>> enrichers = (List<Map<String, Object>>) result.get("enrichers");
    -                String enricherTemplateHtml = resourceUtils.getResourceAsString(Urls.mergePaths(BASE_TEMPLATES, "enricher.html"));
    -                for (Map<String, Object> enricher : enrichers) {
    -                    String type = (String) enricher.get("type");
    -                    String name = (String) enricher.get("name");
    -                    String enricherHtml = TemplateProcessor.processTemplateContents(enricherTemplateHtml, ImmutableMap.of("type", type, "name", name));
    -                    Files.write(enricherHtml, new File(Os.mergePaths(outputFolder, "enrichers", type + ".html")), Charsets.UTF_8);
    +                for (URL url: urls) {
    +                    List<Map<?,?>> entities = new ArrayList<>();
    +                    List<Map<?,?>> policies = new ArrayList<>();
    +                    List<Map<?,?>> enrichers = new ArrayList<>();
    +                    List<Map<?,?>> locations = new ArrayList<>();
    +                    List<Map<?,?>> locationResolvers = new ArrayList<>();
    +
    +                    String yamlContent = Files.toString(new File(url.getFile()), Charset.forName("UTF-8"));
    +
    +                    catalog.addItems(yamlContent);
    +                    for (CatalogItem item: catalog.getCatalogItems()) {
    +                        AbstractBrooklynObjectSpec<?,?> spec = catalog.createSpec(item);
    +                        Map<String,Object> itemDescriptor = ItemDescriptors.toItemDescriptor((Class<? extends BrooklynObject>) spec.getType(), false);
    +                        List<EntityConfigSummary> config = new ArrayList<>();
    +
    +                        if (!itemDescriptor.containsKey("name")) {
    +                            itemDescriptor.put("name", item.getDisplayName());
    +                        }
    +                        if (!itemDescriptor.containsKey("description")) {
    +                            itemDescriptor.put("description", item.getDescription());
    +                        }
    +                        if (!itemDescriptor.containsKey("iconUrl")) {
    +                            itemDescriptor.put("iconUrl", item.getIconUrl());
    +                        }
    +                        for (SpecParameter<?> param: spec.getParameters()) {
    +                            EntityConfigSummary entityConfigSummary = EntityTransformer.entityConfigSummary(param.getType(),
    +                                    param.getLabel(), null, MutableMap.<String,URI>of());
    +                            config.add(entityConfigSummary);
    +                        }
    +                        itemDescriptor.put("config", config);
    +                        if (item.getCatalogItemType().name().equals("ENTITY")) {
    --- End diff --
    
    `TEMPLATE`s also fall in this case (they are entities).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by iyovcheva <gi...@git.apache.org>.
Github user iyovcheva closed the pull request at:

    https://github.com/apache/incubator-brooklyn/pull/1176


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by neykov <gi...@git.apache.org>.
Github user neykov commented on a diff in the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#discussion_r50975501
  
    --- Diff: brooklyn-server/server-cli/src/main/java/org/apache/brooklyn/cli/ItemLister.java ---
    @@ -198,17 +277,36 @@ private void mkdir(String rootDir, String dirName) {
     
             protected List<URL> getUrls() throws MalformedURLException {
                 List<URL> urls = Lists.newArrayList();
    -            if (jars.isEmpty()) {
    -                String classpath = System.getenv("INITIAL_CLASSPATH");
    -                if (Strings.isNonBlank(classpath)) {
    -                    List<String> entries = Splitter.on(":").omitEmptyStrings().trimResults().splitToList(classpath);
    -                    for (String entry : entries) {
    -                        if (entry.endsWith(".jar") || entry.endsWith("/*")) {
    -                            urls.addAll(ClassFinder.toJarUrls(entry.replace("/*", "")));
    +            if (jars.isEmpty()) { //TODO: where is `jars` value write - AbstractMain.execCli(Cli<BrooklynCommand> parser, String ...args):line 250 BrooklynCommand command = parser.parse(args)
    +                if (!yaml.isEmpty()) {
    +                    try {
    +                        File yamlFolder = new File(yaml);
    +                        if (yamlFolder.isDirectory()) {
    +                            File[] fileList = yamlFolder.listFiles();
    +
    +                            for (File file : fileList) {
    +                                if (file.isFile() && (file.getName().contains(".yaml") || file.getName().contains(".bom"))) {
    +                                    urls.add(file.toURI().toURL());
    +                                }
    +                            }
    +                        } else {
    +                            urls.add(new File(yaml).toURI().toURL());
                             }
    +                    } catch (IOException e) {
    +                        e.printStackTrace();
    --- End diff --
    
    Better not to catch it at all, add it to the method signature instead.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-brooklyn pull request: Generate config/effector descript...

Posted by tbouron <gi...@git.apache.org>.
Github user tbouron commented on the pull request:

    https://github.com/apache/incubator-brooklyn/pull/1176#issuecomment-176809380
  
    @bostko Didn't notice that. Let's not reintroduce a slow process then.
    
    However, I quite like the @iyovcheva's idea of generate the `JS` **and** ` JSON` file. That will be much more safer and easier for the community catalog :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---