You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by "Aaron Markham (JIRA)" <ji...@apache.org> on 2018/10/16 16:02:00 UTC
[jira] [Updated] (MXNET-1078) Installation instructions should be
validated automatically
[ https://issues.apache.org/jira/browse/MXNET-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Markham updated MXNET-1078:
---------------------------------
Description:
Some installation instructions can be tested automatically. However, when the instructions are coupled with text for the user, updates can easily break automation scripts. Also, some installation instructions follow a pattern of "if–>then" with multiple steps offering multiple options. This creates a difficult situation for an automation process.
For example, a contributor has worked on getting MXNet to work a Mac with one of those plugin GPUs. They want to add new instructions for this. Most other contributors don't have access to this setup and cannot verify the steps. We should not expect the contributor to update the instructions, accommodate CI, and also fix the installation instructions nightly test.
Another example would be that new users have no idea what math library they could or should use, and probably don't care if even presented the option. Other more advanced users may have preferences for what math library to use. Both sets of users find these options in the dependencies sections, and the setup guide offers several different math library approaches. Which pathway is more important to test? Can a script to test the installation instructions parse all the options and test them all?
I suggest the following approach.
# Create a template with a common flow for the installation instructions for any platform or language binding.
# The flow should consist of the steps the user should follow and code block containing each command that should be executed. If there is more than one code block, the page should end with a complete code block that has all of the steps in sequence; one that can be executed directly if it were copied and pasted.
# A nightly CI script can then be invoked to execute the consolidated installation code block that the instructions use.
# PRs for installation instructions should be validated against these requirements.
Optional approaches / enhancements
# The code blocks can be sourced by an include directive. This will pull the contents of a text file that contains the commands.
# Alternatively, multiple CI scripts can be created - one for each platform and binding combination, so it is clear what fails when everything else is passing. You can also disable a broken test much easier, and not disable everything or have constant failures when one thing is broken.
Timing on implementation is subject to getting the include feature to properly work with Sphinx, or when Sphinx is replaced with Jekyll for the website builds. Both platforms support includes, however the Sphinx implementation we use today seems partially broken and doesn't render includes as described in the Sphinx documentation.
was:
Some installation instructions can be tested automatically. However, when the instructions are coupled with text for the user, updates can easily break automation scripts.
I suggest the following approach.
# Create a common flow for the installation instructions for any platform or language binding.
# The flow should consist of the steps the user should follow and code block containing each command that should be executed. If there is more than one code block, the page should end with a complete code block that has all of the steps in sequence; one that can be executed directly if it were copied and pasted.
# The code blocks can be sourced by an include directive. This will pull the contents of a text file that contains the commands.
# A nightly CI script can then be invoked to execute the same text file that the instructions use. Alternatively, multiple CI scripts can be created - one for each platform and binding combination, so it is clear what fails when everything else is passing. You can also disable a broken test much easier, and not disable everything or have constant failures when one thing is broken.
# PRs for installation instructions should be validated against these requirements.
Timing on implementation is subject to getting the include feature to properly work with Sphinx, or when Sphinx is replaced with Jekyll for the website builds. Both platforms support includes, however the Sphinx implementation we use today seems partially broken and doesn't render includes as described in the Sphinx documentation.
> Installation instructions should be validated automatically
> -----------------------------------------------------------
>
> Key: MXNET-1078
> URL: https://issues.apache.org/jira/browse/MXNET-1078
> Project: Apache MXNet
> Issue Type: Story
> Reporter: Aaron Markham
> Priority: Major
>
> Some installation instructions can be tested automatically. However, when the instructions are coupled with text for the user, updates can easily break automation scripts. Also, some installation instructions follow a pattern of "if–>then" with multiple steps offering multiple options. This creates a difficult situation for an automation process.
> For example, a contributor has worked on getting MXNet to work a Mac with one of those plugin GPUs. They want to add new instructions for this. Most other contributors don't have access to this setup and cannot verify the steps. We should not expect the contributor to update the instructions, accommodate CI, and also fix the installation instructions nightly test.
> Another example would be that new users have no idea what math library they could or should use, and probably don't care if even presented the option. Other more advanced users may have preferences for what math library to use. Both sets of users find these options in the dependencies sections, and the setup guide offers several different math library approaches. Which pathway is more important to test? Can a script to test the installation instructions parse all the options and test them all?
>
> I suggest the following approach.
> # Create a template with a common flow for the installation instructions for any platform or language binding.
> # The flow should consist of the steps the user should follow and code block containing each command that should be executed. If there is more than one code block, the page should end with a complete code block that has all of the steps in sequence; one that can be executed directly if it were copied and pasted.
> # A nightly CI script can then be invoked to execute the consolidated installation code block that the instructions use.
> # PRs for installation instructions should be validated against these requirements.
> Optional approaches / enhancements
> # The code blocks can be sourced by an include directive. This will pull the contents of a text file that contains the commands.
> # Alternatively, multiple CI scripts can be created - one for each platform and binding combination, so it is clear what fails when everything else is passing. You can also disable a broken test much easier, and not disable everything or have constant failures when one thing is broken.
>
> Timing on implementation is subject to getting the include feature to properly work with Sphinx, or when Sphinx is replaced with Jekyll for the website builds. Both platforms support includes, however the Sphinx implementation we use today seems partially broken and doesn't render includes as described in the Sphinx documentation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org