[GitHub] [tvm-rfcs] gromero opened a new pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero opened a new pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33


   Hi, could the following RFC be reviewed please?
   
   It is about extending the current metadata associated with project options returned by the Project API.
   
   Thank you.
   
   Cheers,
   Gustavo


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch merged pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch merged pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] Mousius commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

Mousius commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r709242480



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.

Review comment:
       Could this be split into a markdown list rather than embedding it into the sentence?
   ```suggestion
   The metadata available for the project options, stored in instances of the ProjectOption class are limited as they do not:
   1. Provide a list of the API methods which support the options
   1. Allow determination if the options are required or optional
   1. Provide a default value if one is used by the Project API server.
   
   As a consequence it complicates the integration with command line interfaces that need to create command line arguments based on the project options available for a platform.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption

Review comment:
       I think this should just be the `Feature Name`.

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to

Review comment:
       ```suggestion
   discovered via the Project API `server_info_query` interface there is no way to
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.

Review comment:
       ```suggestion
   Both will contain a list of method names for which the option is available,
   either as a required option (if in `required` list), or as an optional option
   (if in `optional` list). At least one API method must be listed in `required` or
   in the `optional` list. A method name must be listed only in the `required` or in
   the `optional` list, i.e. an option can't be required and optional at the same
   time for given API method. An option can be required for a method and optional
   for another method.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.
+
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field 'default' in the metadata that the API can use

Review comment:
       ```suggestion
   Hence having an additional field `default` in the metadata that the API can use
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).

Review comment:
       ```suggestion
   know which options belong (or apply) to which API method (like the
   `generate_project`, `build`, `flash`, and `open_transport` methods).
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.

Review comment:
       Code snippets could make this clearer:
   ```suggestion
   Below it is explained in detail the need and properties of the four new members
   (`required`, `optional`, `type`, and `default`) proposed to be added to the
   `ProjectOption` class to extend the project option metadata returned by Project
   API `server_info_query` method. `required`, `optional`, and `type` are proposed
   as required fields, whilst `default` is proposed as an optional field.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.

Review comment:
       ```suggestion
   API method the following two new metadata are proposed: `required` and `optional`.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues

Review comment:
       Small typo and would be easier to read as a code snippet:
   ```suggestion
   `ProjectOption (`required`, `optional`, `type`, and `default`)` to address issues
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.
+
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field 'default' in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+'default' may be provided for an option, if applicable.
+
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is to 1) extend ProjectOption
+class by adding the new fields discussed above and 2) adjust the existing
+platform options to comply with the required fields. An example follows
+considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" or "optional" in kw, "'required' or 'optional' must be specified"

Review comment:
       I think this resolves to `required or (optional in kw)`
   ```suggestion
   +        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.
+
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field 'default' in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+'default' may be provided for an option, if applicable.
+
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is to 1) extend ProjectOption
+class by adding the new fields discussed above and 2) adjust the existing
+platform options to comply with the required fields. An example follows
+considering the Zephyr platform:

Review comment:
       ```suggestion
   All that's necessary to complish the proposal is:
   1. Extend the `ProjectOption` class by adding the new fields discussed above
   1. Adjust the existing platform options to comply with the required fields. 
   An example follows considering the Zephyr platform:
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.
+
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field 'default' in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+'default' may be provided for an option, if applicable.

Review comment:
       ```suggestion
   Hence having an additional field `default` in the metadata that the API can use
   to inform the user if the option has any default value is quite useful. It also
   avoids one to keep that information at the client / user side.
   
   `default` may be provided for an option, if applicable.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.

Review comment:
       ```suggestion
   The elements in the lists `required` and `optional` must be in the set of method
   names implemented by the ProjectAPIClient class and that have the parameter
   `options` defined. These methods are dispatched to the server, which implements
   the server counterparts to properly handle the client dispatches and
   ultimately defines the options available for each API method. The current method
   name that satisfy these criteria are `generate_project`, `build`, `flash`, and
   `open_transport`.
   
   `required` metadatum or `optional` metadatum (or both) must be provided for
   every option.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to

Review comment:
       ```suggestion
   Thus adding a `type` metadatum allows a much simpler way for the API users to
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.
+
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field 'default' in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+'default' may be provided for an option, if applicable.
+
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is to 1) extend ProjectOption
+class by adding the new fields discussed above and 2) adjust the existing
+platform options to comply with the required fields. An example follows
+considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+The impact on existing code is low and the current project options will only
+need to be adjusted to define the new mandatory fields, i.e. 'required',
+'optional', and 'type', all are straightforward to be provided for the existing
+options on the current supported platforms.

Review comment:
       ```suggestion
   The impact on existing code is low and the current project options will only
   need to be adjusted to define the new mandatory fields, i.e. `required`,
   `optional`, and `type`, all are straightforward to be provided for the existing
   options on the current supported platforms.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.

Review comment:
       ```suggestion
   `type` metadatum must be provided for every option.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".

Review comment:
       ```suggestion
   The following types, passed as strings, are proposed for the `type` metadatum:
   `"bool"`, `"str"`, `"int"`, and `"float"`.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a 'type' metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the 'type' metadatum:
+"bool", "str", "int", and "float".
+
+'type' metadatum must be provided for every option.
+
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field 'default' in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+'default' may be provided for an option, if applicable.
+
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is to 1) extend ProjectOption
+class by adding the new fields discussed above and 2) adjust the existing
+platform options to comply with the required fields. An example follows
+considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+The impact on existing code is low and the current project options will only
+need to be adjusted to define the new mandatory fields, i.e. 'required',
+'optional', and 'type', all are straightforward to be provided for the existing
+options on the current supported platforms.
+
+Adding the four new mumbers to ProjectOption will increase the size of
+ProjectOption class and consequently the amount of data return by
+'server_info_query', however since Project API client and serve run on the same
+host that is negligible.

Review comment:
       ```suggestion
   Adding the four new mumbers to `ProjectOption` will increase the size of
   `ProjectOption` class and consequently the amount of data return by
   `server_info_query`, however since Project API client and server run on the same
   host that is negligible.
   ```

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: project_api_extend_metadata
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+[RFC][Project API] Extend metadata in ProjectOption
+
+# Summary
+[summary]: #summary
+
+Extend metadata associated with project options provided by the Project API.
+
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class, A) do not provide a list of the API methods which support
+the options, B) do not allow to determine if the options are required or
+optional, C) and do not provide a default value if one is used by the Project
+API server. As a consequence it complicates the integration with command line
+interfaces that need to create command line arguments based on the project
+options available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+ProjecOption ('required', 'optional', 'type', and 'default') to address issues
+A, B, and C and ease the integration of Project API with CLI tools.
+
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+('required', 'optional', 'type', and 'default') proposed to be added to the
+ProjectOption class to extend the project option metadata returned by Project
+API 'server_info_query' method. 'required', 'optional', and 'type' are proposed
+as required fields, whilst 'default' is proposed as an optional field.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API 'server_info_query' interface there is no way to
+known which options belong (or apply) to which API method (like the
+generate_project, build, flash, and open_transport methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: 'required' and 'optional'.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in 'required' list), or as an optional option
+(if in 'optional' list). At least one API method must be listed in 'required' or
+in the 'optional list. A method name must be listed only in the 'required' or in
+the 'optional' list, i.e. an option can't be require and optional at the same
+time for given API method. An option can be required for a method and optional
+for another method.
+
+The elements in the lists 'required' and 'optional' must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+'options' defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are 'generate_project', 'build', 'flash', and
+'open_transport'.
+
+'required' metadatum or 'optional' metadatum (or both) must be provided for
+every option.
+
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum 'choices', but this not ideal. For example, for option 'verbose' it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum 'choices' is a couple of True and

Review comment:
       ```suggestion
   in metadatum `choices`, but this not ideal. For example, for option `verbose` it
   would be possible to infer it is a boolean option and therefore it can be
   converted to a command line flag if metadatum `choices` is a couple of True and
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r715071778



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       I think I know what you mean, you meant enforce that an option like `zephyr_model` (which was actually just removed) ever exists right? (i.e. an option which is not really associated to any of the methods provided by the API right?) 
   e.g.: https://github.com/apache/tvm/commit/73c28455d30df8167f3a596326fddeb294e7d11f#diff-e9070632c64e33062450a5c887a7a6b683d7294064584407aeff7aef24031dc1L293 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r714034324



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:

Review comment:
       Sure. Do you think it's correct to use **packed** types to refer to dict, tuple, and list instead of complex  types (the non-complex types would be the non-packed types so)?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r716956895



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:

Review comment:
       do you mean `struct.pack` or `PackedFunc`? I don't think `PackedFunc` has a place in this RPC, as it's using JSON-RPC not TVM RPC. I would think we would use JS objects to represent more complex values if we did go that direction, but honestly i prefer to not if possible to avoid over-complicating this.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-923185803


   > I apologise for the wall of requested changes @gromero, I've been very nit picky over the clarity of the markdown which I think a find/replace will almost immediately fix the majority of.
   >
   > Overall this all sounds sensible and makes sense to me as a change.
   
   @Mousius Actually, thanks a lot for the thorough review. I really appreciate it. I'm glad you've spotted the `assert()` bug and the typos. I've addressed all your comments in https://github.com/apache/tvm-rfcs/pull/33/commits/e6ad692b06d1eeedf4d780d97ad75a9b22122ec6  I hope it's all right now. Cheers.
    
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero edited a comment on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero edited a comment on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-936978446






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-918581638


   cc @areusch @manupa-arm 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r717062734



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:

Review comment:
       @areusch I didn't mean neither `struct.pack` nor `PackedFunc`. :) I'm actually acknowledging that we've settled on only non-complex JSON-serialized types. My last question is almost a nit, and I just would like to know if it would be ok to use the term 'packed' types instead of 'non-complex' types because I think Python uses that term to refer to built-in types like dict, tuple, list etc -- which we would like to not use here as we have settled.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r716956121



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       i mean more like--what if server.py receives a call to e.g. `generate_project` but someone passes `"verbose": True`. `server.py` should remove that option as, on consulting ProjectOption for "verbose" it is e.g. not required or optional to `generate_project`. we could even error here. similarly we could add enforcement in `client.py`, but the `server.py` is the one i care about.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r715071778



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       @areusch I think I know what you mean, you meant enforce that an option like `zephyr_model` (which was actually just removed) ever exists right? (i.e. an option which is not really associated to any of the methods provided by the API right?) 
   e.g.: https://github.com/apache/tvm/commit/73c28455d30df8167f3a596326fddeb294e7d11f#diff-e9070632c64e33062450a5c887a7a6b683d7294064584407aeff7aef24031dc1L293 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-936978446


   > @gromero please take a look at the comments here and lmk when it's ready for another review
   
   @areusch Hi. Please take a look. I've also added the text stressing that every option must be associated at least to one API method, as per our discussion last week in the Meetup. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch commented on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch commented on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-944382441


   thanks @gromero ! Please open an RFC tracking issue and reference this doc from there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-920051153


   Thanks a lot for the review @leandron 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero edited a comment on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero edited a comment on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-918581638


   cc @areusch @manupa-arm @Mousius @leandron


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r716956121



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       i mean more like--what if server.py receives a call to e.g. `generate_project` but someone passes `"verbose": True`. `server.py` should remove that option as, on consulting ProjectOption for "verbose" it is e.g. not required or optional to `generate_project`. we could even error here. similarly we could add enforcement in `client.py`, but the `server.py` is the one i care about.

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:

Review comment:
       do you mean `struct.pack` or `PackedFunc`? I don't think `PackedFunc` has a place in this RPC, as it's using JSON-RPC not TVM RPC. I would think we would use JS objects to represent more complex values if we did go that direction, but honestly i prefer to not if possible to avoid over-complicating this.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r717062734



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:

Review comment:
       @areusch I didn't mean neither `struct.pack` nor `PackedFunc`. :) I'm actually acknowledging that we've settled on only non-complex JSON-serialized types. My last question is almost a nit, and I just would like to know if it would be ok to use the term 'packed' types instead of 'non-complex' types because I think Python uses that term to refer to built-in types like dict, tuple, list etc -- which we would like to not use here as we have settled.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch commented on pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch commented on pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#issuecomment-932994088


   @gromero please take a look at the comments here and lmk when it's ready for another review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r715071778



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       @areusch I think I know what you mean, you mean enforce that an option like `zephyr_model` (which was actually just removed) ever exists right? (i.e. an option which is not really associated to any of the methods provided by the API right?) 
   e.g.: https://github.com/apache/tvm/commit/73c28455d30df8167f3a596326fddeb294e7d11f#diff-e9070632c64e33062450a5c887a7a6b683d7294064584407aeff7aef24031dc1L293 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] gromero commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

gromero commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r714028366



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       @areusch hmr I'm not sure I got it. I was thinking of it more along the lines of having all the provided options as having a `required` or `optional` list (or both), so never having an option which has neither a `required` list nor a `optional` list. Also an option provided is not *in*  a `required` or `optional` list... hmm but wait, did you mean "remove *methods*" (instead "remove options") provided actually?  :)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm-rfcs] areusch commented on a change in pull request #33: [RFC][Project API] Extend metadata in ProjectOption

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #33:
URL: https://github.com/apache/tvm-rfcs/pull/33#discussion_r713497721



##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically

Review comment:
       nit: s/that's/it's/

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of

Review comment:
       nit: knows

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:

Review comment:
       can you state the rationale behind these limits to capture it here? IIRC we settled on only non-complex JSON-serializable types.

##########
File path: rfcs/0020-project_api_extend_metadata.md
##########
@@ -0,0 +1,252 @@
+- Feature Name: extend_metadata_in_projectoption
+- Start Date: 2021-09-09
+- RFC PR: [apache/tvm-rfcs#0020](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0020](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to extend the current metadata associated with project options
+provided by the Project API to allow a better integration with command line
+interface tools, like TVMC.
+
+# Motivation
+[motivation]: #motivation
+
+Currently metadata associated with project options provided by the Project API
+is insufficent to allow building easily and automatically command line parsers
+used by CLI tool like TVMC.
+
+The metadata available for the project options, stored in instances of the
+ProjectOption class are limited as they do not:
+
+1. Provide a list of the API methods which support the options;
+2. Allow determination if the options are required or optional;
+3. Provide a default value if one is used by the Project API server.
+
+As a consequence it complicates the integration with command line interfaces
+that need to create command line arguments based on the project options
+available for a platform.
+
+This RFC proposes to extend the existing metadata with four new members in
+`ProjectOption` (`required`, `optional`, `type`, and `default`) to address
+issues **1.**, **2.**, and **3.** and ease the integration of Project API with
+CLI tools.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Below it is explained in detail the need and properties of the four new members
+(`required`, `optional`, `type`, and `default`) proposed to be added to the
+`ProjectOption` class to extend the project option metadata returned by Project
+API `server_info_query` method. `required`, `optional`, and `type` are proposed
+as **required fields**, whilst `default` is proposed as an **optional field**.
+
+Modals like "must", "may" and similar ones are interpreted in this RFC
+accordingly to the semantics defined by the IETF RFC-2119, 1997.
+
+## On "required" and "optional" metadata
+
+Currently even though all options available for a given project can be
+discovered via the Project API `server_info_query` interface there is no way to
+know which options belong (or apply) to which API method (like the
+`generate_project`, `build`, `flash`, and `open_transport` methods).
+
+This is fine when the user knowns beforehand which method accepts a set of
+options, so it's possible to manually select which options will be passed to a
+given API method, like when using the API in a Python script.
+
+However that's a problem when the API user (e.g. TVMC) needs to automatically
+determine the options available for the API methods, like when automatically
+building a command line parser with subcommand domains that closely mapped to
+the API methods (e.g. subcommands to create, build, and flash a project).
+
+Moreover, currently it's impossible to determine which option is required and
+which one is optional, so it would be at least necessary for the API user to
+build a static ad hoc table with all options available on a given project
+stating which option is required and which one is optional for the project. This
+is impractical to maintain and would result in the API user having to update the
+the static table every time an option is added, removed, or modified in the
+Project API server.
+
+Hence to ease the automatic detection of the options available on each Project
+API method the following two new metadata are proposed: `required` and
+`optional`.
+
+Both will contain a list of method names for which the option is available,
+either as a required option (if in `required` list), or as an optional option
+(if in `optional` list). At least one API method must be listed in `required` or
+in the `optional` list. A method name must be listed only in the `required` or
+in the `optional` list, i.e. an option can't be required and optional at the
+same time for given API method. An option can be required for a method and
+optional for another method.
+
+The elements in the lists `required` and `optional` must be in the set of method
+names implemented by the ProjectAPIClient class and that have the parameter
+`options` defined. These methods are dispatched to the server, which implements
+the server counterparts to properly handle the client dispatches and
+ultimately defines the options available for each API method. The current method
+name that satisfy these criteria are `generate_project`, `build`, `flash`, and
+`open_transport`.
+
+`required` metadatum or `optional` metadatum (or both) must be provided for
+every option.
+
+## On "type" metadatum
+
+The option type can sometimes be determined implicitly by what's return
+in metadatum `choices`, but this not ideal. For example, for option `verbose` it
+would be possible to infer it is a boolean option and therefore it can be
+converted to a command line flag if metadatum `choices` is a couple of True and
+False. Nonetheless that would lead to cumbersome logic at API user side (e.g.
+TVMC) to infer the option type, like iterating over the tuple elements to search
+for True or False. This can be solved directly if the option type is returned
+explicitly with the option.
+
+Thus adding a `type` metadatum allows a much simpler way for the API users to
+determine the type of an option when that is necessary for various reasons, like
+when building a command line parser based on the available project options.
+
+The following types, passed as strings, are proposed for the `type` metadatum:
+`"bool"`, `"str"`, `"int"`, and `"float"`.
+
+`type` metadatum must be provided for every option.
+
+## On "default" metadatum
+
+Sometimes Project API uses a default value if an option is not specified, but
+currently there is no way to determine it using the option metadata.
+
+However it's important for CLI tools to inform users what's the default value
+for a given option, if applicable, so the user can decide if it's necessary to
+provide a different value.
+
+The default values for the options on a project could be defined at the user
+side (e.g. TVMC) but that's not ideal.
+
+Hence having an additional field `default` in the metadata that the API can use
+to inform the user if the option has any default value is quite useful. It also
+avoids one to keep that information at the client / user side.
+
+`default` may be provided for an option, if applicable.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+All that's necessary to complish the proposal is:
+
+1. Extend `ProjectOption` class by adding the new fields discussed above;
+2. Adjust the existing platform options to comply with the required fields.
+
+An example follows considering the Zephyr platform:
+
+```
+diff --git a/python/tvm/micro/project_api/server.py b/python/tvm/micro/project_api/server.py
+index 07d328597..323bf418a 100644
+--- a/python/tvm/micro/project_api/server.py
++++ b/python/tvm/micro/project_api/server.py
+@@ -50,7 +50,8 @@ class ProjectOption(_ProjectOption):
+     def __new__(cls, name, **kw):
+         """Override __new__ to force all options except name to be specified as kwargs."""
+         assert "name" not in kw
+-        assert "type" in kw, "'type' parameter must be specified"
++        assert "required" in kw or "optional" in kw, "'required' or 'optional' must be specified"
++        assert "type" in kw, "'type' field must be specified"
+
+         kw["name"] = name
+         for param in ["choices", "default", "required", "optional"]:
+
+
+
+diff --git a/apps/microtvm/zephyr/template_project/microtvm_api_server.py b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+index 4e62739d5..8d0b1722c 100644
+--- a/apps/microtvm/zephyr/template_project/microtvm_api_server.py
++++ b/apps/microtvm/zephyr/template_project/microtvm_api_server.py
+@@ -216,40 +216,58 @@ if IS_TEMPLATE:
+ PROJECT_OPTIONS = [
+     server.ProjectOption(
+         "extra_files_tar",
+-        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
++        optional=["generate_project"],
+         type="str",
++        help="If given, during generate_project, uncompress the tarball at this path into the project dir.",
+     ),
+     server.ProjectOption(
+         "gdbserver_port", help=("If given, port number to use when running the local gdbserver."),
++        optional=["open_transport"],
+         type="int",
+     ),
+     server.ProjectOption(
+         "nrfjprog_snr",
+-        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with nRF targets, serial # of the attached board to use, from nrfjprog."),
+     ),
+     server.ProjectOption(
+         "openocd_serial",
+-        help=("When used with OpenOCD targets, serial # of the attached board to use."),
++        optional=["open_transport"],
+         type="int",
++        help=("When used with OpenOCD targets, serial # of the attached board to use."),
+     ),
+     server.ProjectOption(
+         "project_type",
+-        help="Type of project to generate.",
+         choices=tuple(PROJECT_TYPES),
++        required=["generate_project"],
+         type="str",
++        help="Type of project to generate.",
+     ),
+     server.ProjectOption("verbose", help="Run build with verbose output.", type="bool"),
+     server.ProjectOption(
+         "west_cmd",
++        optional=["generate_project"],
++        default="python3 -m west",
++        type="str",
+         help=(
+             "Path to the west tool. If given, supersedes both the zephyr_base "
+             "option and ZEPHYR_BASE environment variable."
+         ),
++    ),
++    server.ProjectOption(
++        "zephyr_base",
++        optional=["build", "open_transport"],
++        default="ZEPHYR_BASE",
++        type="str",
++        help="Path to the zephyr base directory.",
++    ),
++    server.ProjectOption(
++        "zephyr_board",
++        required=["generate_project", "build", "flash", "open_transport"],
++        help="Name of the Zephyr board to build for.",
+         type="str",
+     ),
+-    server.ProjectOption("zephyr_base", help="Path to the zephyr base directory.", type="str"),
+-    server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for.", type="str"),
+ ]
+```
+

Review comment:
       it would be great to spell out a brief explanation of how we plan to enforce these things (e.g. modify `server.py` and `client.py` to remove options provided which aren't in `required` or `optional` for the given call).




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org