You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by GitBox <gi...@apache.org> on 2021/05/09 19:16:21 UTC
[GitHub] [daffodil-site] tuxji opened a new pull request #64: Update CLI documentation for Daffodil 3.1.0
tuxji opened a new pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64
Document the new "daffodil generate" subcommand that will be released
in Daffodil 3.1.0. Also mention the CC environment variable and
update some of the other CLI usage and options for clarity and
consistency. Wait until daffodil 3.1.0 is released before merging.
Please review and suggest fewer or more changes that you may want made. For example, I compared the CLI documentation against the --help output the CLI prints and noticed --nostream was printed, but it may not be necessary to actually document that option. I also made some other changes that I think makes some options clearer, but if we make these changes we might want to update the CLI's --help output to match these changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] jw3 commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
jw3 commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629409709
##########
File path: site/cli.md
##########
@@ -192,19 +200,24 @@ Unparse an infoset file, using either a DFDL schema or a saved parser.
: Split the input data at NUL characters and unparse each chunk separately to the same output file.
+``--nostream``
+
+ : Treat the entire input data as one infoset. This is the default behavior.
+
``-T TUNABLE=VALUE``
- : Modify Daffodil configuration options to change parsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
+ : Modify Daffodil configuration options to change unparsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
``-V, --validate [MODE]``
- : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
+ : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
Review comment:
Added some text. The validator api never got proper documentation, would it make sense to add a `validators.md` under `site/`?
##########
File path: site/cli.md
##########
@@ -192,19 +200,24 @@ Unparse an infoset file, using either a DFDL schema or a saved parser.
: Split the input data at NUL characters and unparse each chunk separately to the same output file.
+``--nostream``
+
+ : Treat the entire input data as one infoset. This is the default behavior.
+
``-T TUNABLE=VALUE``
- : Modify Daffodil configuration options to change parsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
+ : Modify Daffodil configuration options to change unparsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
``-V, --validate [MODE]``
- : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
+ : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
Review comment:
Added some text. The validator api never got proper documentation, would it make sense to add a `validators.md` under `site/` in a separate PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] stevedlawrence commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629283586
##########
File path: site/cli.md
##########
@@ -44,6 +44,10 @@ Setting environment variables may be necessary to allow for importing/includes t
: If you need to specify java options specific to Daffodil, you can set the ``DAFFODIL_JAVA_OPTS`` environment variable. If not specified, the ``JAVA_OPTS`` environment variable will be used. If that is not specified, reasonable defaults for Daffodil will be used.
+``CC``
+
+ : If you need to specify which C compiler Daffodil's runtime2 backend should call when running TDML files, you can set the ``CC`` environment variable. If that environment variable is not specified, then `cc` will be used as the default driver command for compiling C source files from TDML files.
Review comment:
Might mention the "test" subcommand somewhere just to make it clear this doesn't change anything with parse/unparse/generate. Maybe something as simple as
> should call when running TDML files with the "test" subcommand, ...
I don't think the generate subcommand has an option to compile after generating code, right?
##########
File path: site/cli.md
##########
@@ -192,19 +200,24 @@ Unparse an infoset file, using either a DFDL schema or a saved parser.
: Split the input data at NUL characters and unparse each chunk separately to the same output file.
+``--nostream``
+
+ : Treat the entire input data as one infoset. This is the default behavior.
+
``-T TUNABLE=VALUE``
- : Modify Daffodil configuration options to change parsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
+ : Modify Daffodil configuration options to change unparsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
``-V, --validate [MODE]``
- : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
+ : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
Review comment:
This has also been changed. You also provide an validator plugin name. I'm not sure we have any documentation about what that syntax is. @jw3, does that exist anywhere were we can reference it?
##########
File path: site/cli.md
##########
@@ -26,7 +26,7 @@ The binary Daffodil [releases](/releases) contain a ``/bin`` directory that cont
daffodil [GLOBAL_OPTIONS] <subcommand> [SUBCOMMAND_OPTIONS]
-The available subcommands are: [parse](#parse-subcommand), [unparse](#unparse-subcommand), [save-parser](#save-parser-subcommand), [test](#test-subcommand), and [performance](#performance-subcommand).
+The available subcommands are: [parse](#parse-subcommand), [unparse](#unparse-subcommand), [save-parser](#save-parser-subcommand), [test](#test-subcommand), [performance](#performance-subcommand), and [generate][#generate-subcommand].
Review comment:
The #generate-subcommand should be surrounded with parenthesis instead of square brackets.
Also, thoughts on making these all bullets instead of a comma separated list? It's starting to bet pretty long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629415412
##########
File path: .github/workflows/main.yml
##########
@@ -54,7 +55,7 @@ jobs:
jekyll build --source site
- name: Publish
- if: github.repository == 'apache/daffodil-site'
+ if: github.repository == 'apache/daffodil-site' && github.ref == 'refs/heads/master'
Review comment:
Whoever updates this branch next, please add the `${{ }}` to fix the conflict with the main branch. We can update the entire workflow in its own PR later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#issuecomment-836680186
Meanwhile, I've told GitHub to rerun Steve's last job again so the site will get updated correctly with the previous CLI page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] stevedlawrence merged pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
stevedlawrence merged pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] stevedlawrence commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629339577
##########
File path: site/cli.md
##########
@@ -44,6 +44,10 @@ Setting environment variables may be necessary to allow for importing/includes t
: If you need to specify java options specific to Daffodil, you can set the ``DAFFODIL_JAVA_OPTS`` environment variable. If not specified, the ``JAVA_OPTS`` environment variable will be used. If that is not specified, reasonable defaults for Daffodil will be used.
+``CC``
+
+ : If you need to specify which C compiler Daffodil's runtime2 backend should call when running TDML files, you can set the ``CC`` environment variable. If that environment variable is not specified, then `cc` will be used as the default driver command for compiling C source files from TDML files.
Review comment:
I must have seen double backticks somewhere and just assumed you always needed two. Good to know.
I wonder if it would make things cleaner if we went through and converted everything to single backticks? I doubt there are any places where we actually need the escaping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] stevedlawrence commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629392984
##########
File path: .github/workflows/main.yml
##########
@@ -25,6 +25,7 @@ jobs:
fail-fast: false
steps:
+ steps:
Review comment:
Is this an extra `steps:`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] stevedlawrence commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629405244
##########
File path: .github/workflows/main.yml
##########
@@ -54,7 +55,7 @@ jobs:
jekyll build --source site
- name: Publish
- if: github.repository == 'apache/daffodil-site'
+ if: github.repository == 'apache/daffodil-site' && github.ref == 'refs/heads/master'
Review comment:
Yeah, wouldn't surprise me if they actually support operators without curly braces. It's not clear why that limitation would be in place anyways.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] stevedlawrence commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629392749
##########
File path: .github/workflows/main.yml
##########
@@ -54,7 +55,7 @@ jobs:
jekyll build --source site
- name: Publish
- if: github.repository == 'apache/daffodil-site'
+ if: github.repository == 'apache/daffodil-site' && github.ref == 'refs/heads/master'
Review comment:
Just an FYI, github documentation says that your if-condition contains any operators (e.g. &&) then you have to wrap it in `${{ }}`.
From https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions
> When you use expressions in an if conditional, you may omit the expression syntax (${{ }}) because GitHub automatically evaluates the if conditional as an expression, unless the expression contains any operators. If the expression contains any operators, the expression must be contained within ${{ }} to explicitly mark it for evaluation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] jw3 commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
jw3 commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629363134
##########
File path: site/cli.md
##########
@@ -192,19 +200,24 @@ Unparse an infoset file, using either a DFDL schema or a saved parser.
: Split the input data at NUL characters and unparse each chunk separately to the same output file.
+``--nostream``
+
+ : Treat the entire input data as one infoset. This is the default behavior.
+
``-T TUNABLE=VALUE``
- : Modify Daffodil configuration options to change parsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
+ : Modify Daffodil configuration options to change unparsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
``-V, --validate [MODE]``
- : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
+ : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
Review comment:
No, there is nothing to point to, other than the updated cli arg string
`the validation mode. 'on', 'limited', 'off', or a validator plugin name.`
I can provide an update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629408199
##########
File path: .github/workflows/main.yml
##########
@@ -25,6 +25,7 @@ jobs:
fail-fast: false
steps:
+ steps:
Review comment:
Yeah, I'd become too ambitious, started to update the rest of the workflow since I noticed some actions should use full / explicit version numbers, and then taken these changes out but left the extra steps: by accident. This whole workflow needs to be updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629403557
##########
File path: .github/workflows/main.yml
##########
@@ -54,7 +55,7 @@ jobs:
jekyll build --source site
- name: Publish
- if: github.repository == 'apache/daffodil-site'
+ if: github.repository == 'apache/daffodil-site' && github.ref == 'refs/heads/master'
Review comment:
Huh, the && had worked for me in another workflow file I'd written, and the newest job just skipped the publish step in this workflow too. I guess GitHub would prefer developers use `if: ${{ x && y }}` but has heuristics just in case developers use `if: x && y` anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629412876
##########
File path: site/cli.md
##########
@@ -192,19 +200,24 @@ Unparse an infoset file, using either a DFDL schema or a saved parser.
: Split the input data at NUL characters and unparse each chunk separately to the same output file.
+``--nostream``
+
+ : Treat the entire input data as one infoset. This is the default behavior.
+
``-T TUNABLE=VALUE``
- : Modify Daffodil configuration options to change parsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
+ : Modify Daffodil configuration options to change unparsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
``-V, --validate [MODE]``
- : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
+ : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
Review comment:
I agree you should add a validators.md under site/ but please use this PR so we can merge all the changes at once after 3.1.0 gets released.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#issuecomment-836677323
My PR's branch was published again because it still has the old workflow file. I'll push another commit to update that workflow file as well as any other cleanups (remove a stray period, use one left quotation mark instead of two left quotation marks) that I notice or people suggest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-site] tuxji commented on a change in pull request #64: Update CLI documentation for Daffodil 3.1.0
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #64:
URL: https://github.com/apache/daffodil-site/pull/64#discussion_r629304939
##########
File path: site/cli.md
##########
@@ -26,7 +26,7 @@ The binary Daffodil [releases](/releases) contain a ``/bin`` directory that cont
daffodil [GLOBAL_OPTIONS] <subcommand> [SUBCOMMAND_OPTIONS]
-The available subcommands are: [parse](#parse-subcommand), [unparse](#unparse-subcommand), [save-parser](#save-parser-subcommand), [test](#test-subcommand), and [performance](#performance-subcommand).
+The available subcommands are: [parse](#parse-subcommand), [unparse](#unparse-subcommand), [save-parser](#save-parser-subcommand), [test](#test-subcommand), [performance](#performance-subcommand), and [generate][#generate-subcommand].
Review comment:
Thanks for the catch and I agree, let's convert to bullets.
##########
File path: site/cli.md
##########
@@ -44,6 +44,10 @@ Setting environment variables may be necessary to allow for importing/includes t
: If you need to specify java options specific to Daffodil, you can set the ``DAFFODIL_JAVA_OPTS`` environment variable. If not specified, the ``JAVA_OPTS`` environment variable will be used. If that is not specified, reasonable defaults for Daffodil will be used.
+``CC``
+
+ : If you need to specify which C compiler Daffodil's runtime2 backend should call when running TDML files, you can set the ``CC`` environment variable. If that environment variable is not specified, then `cc` will be used as the default driver command for compiling C source files from TDML files.
Review comment:
I like your suggestion and incorporated it. Correct, the generate subcommand doesn't compile after generating code. I also remembered that I should mention that Daffodil will pick the first driver command it finds within the PATH environment variable from a list (zig cc, cc, clang, gcc) and that zig cc comes first because it uses a caching system.
BTW, we don't need to use a pair of left quotation marks to begin inline code spans and end them with another pair of left quotation marks. A single left quotation mark will work too; the reason for allowing N left quotation marks is to allow consecutive N-1 left quotation marks to be used inside the text being quoted.
##########
File path: site/cli.md
##########
@@ -192,19 +200,24 @@ Unparse an infoset file, using either a DFDL schema or a saved parser.
: Split the input data at NUL characters and unparse each chunk separately to the same output file.
+``--nostream``
+
+ : Treat the entire input data as one infoset. This is the default behavior.
+
``-T TUNABLE=VALUE``
- : Modify Daffodil configuration options to change parsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
+ : Modify Daffodil configuration options to change unparsing behavior. See [Configuration](/configuration) for the list of tunable parameters.
``-V, --validate [MODE]``
- : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
+ : The validation mode. ``MODE`` must be one of ``on``, ``limited`` or ``off``. If ``MODE`` is not provided, defaults to ``on``. If ``--validate`` is not provided, defaults to ``off``. ``MODE`` cannot be ``on`` when used with the ``--parser`` option.
Review comment:
@jw3, you can take advantage of the branch I pushed (even though it wasn't standard operating procedure) and push your own change to it so we can review your change in the same PR too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org