You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airavata.apache.org by ma...@apache.org on 2020/06/22 15:03:49 UTC
[airavata-django-portal] 02/02: AIRAVATA-3285 Interactive parameter
documentation
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a commit to branch AIRAVATA-3285--Interactive-output-view-providers
in repository https://gitbox.apache.org/repos/asf/airavata-django-portal.git
commit c8fe65b81eb20ff49ed6a6b85db0f35420fadc1b
Author: Marcus Christie <ma...@apache.org>
AuthorDate: Mon Jun 22 11:02:24 2020 -0400
AIRAVATA-3285 Interactive parameter documentation
---
docs/dev/custom_output_view_provider.md | 317 +++++++++++++++++++++
docs/tutorial/gateways2019_tutorial.md | 114 ++++++--
.../gateways19/gaussian-eigenvalues-show_grid.png | Bin 0 -> 149262 bytes
mkdocs.yml | 7 +-
4 files changed, 406 insertions(+), 32 deletions(-)
diff --git a/docs/dev/custom_output_view_provider.md b/docs/dev/custom_output_view_provider.md
new file mode 100644
index 0000000..a34f60a
--- /dev/null
+++ b/docs/dev/custom_output_view_provider.md
@@ -0,0 +1,317 @@
+# Custom Output View Provider
+
+A custom _output view provider_ generates visualizations of experiment outputs.
+Output view providers are implemented as a Python function and packaged as a
+Python package, with the requisite metadata (more on this below). An output view
+provider is associated with the output of an application in the Application
+Catalog.
+
+There are several different output view display types, such as: image, link, or
+html. If configured an output view will be displayed for an output file and the
+Airavata Django Portal will invoke the custom output view provider to get the
+data to display. For example, if the output view display type is image, then the
+output view provider will be invoked and it should return image data.
+
+## Getting started
+
+See the
+[Gateways 2019 tutorial](../tutorial/gateways2019_tutorial.md#tutorial-exercise-create-a-custom-output-viewer-for-an-output-file)
+for help on setting up a development environment and implementing a simple
+output view provider.
+
+You can use this as a starting point to create your own custom output view
+provider. Here is what you would need to change:
+
+1. First add your custom output view provider implementation to
+ `output_views.py`.
+2. Rename the Python package name in `setup.py`.
+3. Update the `install_requires` list of dependencies based on what your custom
+ output view provider requires.
+4. Rename the Python module folder from `./gateways2019_tutorial` to whatever
+ you want to call it.
+5. Rename the output view provider in the `entry_points` metadata in `setup.py`.
+ For example, if you wanted to name your output view provider
+ `earthquake-sites-visualization` and you renamed your Python module folder
+ from `./gateways2019_tutorial` to `./earthquake_gateway`, then you could have
+ the following in the `entry_points`:
+
+```
+...
+ entry_points="""
+[airavata.output_view_providers]
+earthquake-sites-visualization = earthquake_gateway.output_views:EarthquakeSitesViewProvider
+""",
+```
+
+6. If you don't need a Django app, you can remove the `[airavata.djangoapp]`
+ section from `entry_points`.
+
+Please note, if you update `setup.py` and you're doing local development, you'll
+need to reinstall the package into your local Django instance's virtual
+environment using:
+
+```bash
+python setup.py develop
+```
+
+## Reference
+
+### Output View Provider interface
+
+Output view providers should be defined as a Python class. They should define
+the following attributes:
+
+- `display_type`: this should be one of _link_, _image_ or _html_.
+- `name`: this is the name of the output view provider displayed to the user.
+- `test_output_file`: (optional) the path to a file to use for testing
+ purposes. This file will be passed to the `generate_data` function as the
+ `output_file` parameter when the output file isn't available and the Django
+ server is running in DEBUG mode. This is helpful when developing a custom
+ output view provider in a local Django instance that doesn't have access to
+ the output files.
+
+The output view provider class should define the following method:
+
+```python
+def generate_data(self, request, experiment_output, experiment, output_file=None, **kwargs):
+
+ # Return a dictionary
+ return {
+ #...
+ }
+```
+
+The required contents of the dictionary varies based on the _display type_.
+
+#### Display type link
+
+The returned dictionary should include the following entries:
+
+- url
+- label
+
+The _label_ is the text of the link. Generally speaking this will be rendered
+as:
+
+```html
+<a href="{{ url }}">{{ label }}</a>
+```
+
+**Examples**
+
+- [SimCCS Maptool - SolutionLinkProvider](https://github.com/SciGaP/simccs-maptool/blob/master/simccs_maptool/output_views.py#L5)
+
+#### Display type image
+
+The returned dictionary should include the following entries:
+
+- image: a stream of bytes, i.e., either the result of `open(file, 'rb')` or
+ something equivalent like `io.BytesIO`.
+- mime-type: the mime-type of the image, for example, `image/png`.
+
+**Examples**
+
+- [AMP Gateway - TRexXPlotViewProvider](https://github.com/SciGaP/amp-gateway-django-app/blob/master/amp_gateway/plot.py#L115)
+
+#### Display type html
+
+The returned dictionary should include the following entries:
+
+- output: a raw HTML string
+- js: a static URL to a JavaScript file, for example,
+ `/static/earthquake_gateway/custom-leaflet-script.js`.
+
+**Examples**
+
+- [dREG - DregGenomeBrowserViewProvider](https://github.com/SciGaP/dreg-djangoapp/blob/master/dreg_djangoapp/output_views.py#L4)
+
+### Entry Point registration
+
+Custom output view providers are packaged as Python packages in order to be
+deployed into an instance of the Airavata Django Portal. The Python package must
+have metadata that indicates that it contains a custom output view provider.
+This metadata is specified as an _entry point_ in the package's `setup.py` file
+under the named parameter `entry_points`.
+
+The entry point must be added to an entry point group called
+`[airavata.output_view_providers]`. The entry point format is:
+
+```
+label = module:class
+```
+
+The _label_ is the identifier you will use when associating an output view
+provider with an output file in the Application Catalog. As such, you can name
+it whatever you want. The _module_ must be the Python module in which exists
+your output view provider. The _class_ must be the name of your output view
+provider class.
+
+See the **Getting Started** section for an example of how to format the entry
+point in `setup.py`.
+
+### Associating an output view provider with an output file
+
+In the Application Catalog, you can add JSON metadata to associate an output
+view provider with an output file.
+
+1. In the top navigation menu in the Airavata Django Portal, go to **Settings**.
+2. If not already selected, select the **Application Catalog** from the left
+ hand side navigation.
+3. Click on the application.
+4. Click on the **Interface** tab.
+5. Scroll down to the _Output Fields_ and find the output file with which you
+ want to associate the output view provider.
+6. In the _Metadata_ field, add or update the `output-view-providers` key. The
+ value should be an array (beginning and ending with square brackets). The
+ name of your output view provider is the label you gave it when you created
+ the entry point.
+
+The _Metadata_ field will have a value like this:
+
+```json
+{
+ "output-view-providers": ["gaussian-eigenvalues-plot"]
+}
+```
+
+Where instead of `gaussian-eigenvalues-plot` you would put or add the label of
+your custom output view provider.
+
+There's a special `default` output view provider that provides the default
+interface for output files, namely by providing a download link for the output
+file. This `default` output view provider will be shown initially to the user
+and the user can then select a custom output view provider from a drop down
+menu. If, instead, you would like your custom output view provider to be
+displayed initially, you can add the `default` view provider in the list of
+output-view-providers and place it second. For example:
+
+```json
+{
+ "output-view-providers": ["gaussian-eigenvalues-plot", "default"]
+}
+```
+
+would make the `gaussian-eigenvalues-plot` the initial output view provider. The
+user can access the default output view provider from the drop down menu.
+
+### Interactive parameters
+
+You can add some interactivity to your custom output view provider by adding one
+or more interactive parameters. An interactive parameter is a parameter that
+your custom output view provider declares, with a name and current value. The
+Airavata Django Portal will display all interactive parameters in a form and
+allow the user to manipulate them. When an interactive parameter is updated by
+the user, your custom output view provider will be again invoked with the new
+value of the parameter.
+
+To add an interactive parameter, you first need to add a keyword parameter to
+your `generate_data` function. For example, let's say you want to add a boolean
+`show_grid` parameter that the user can toggle on and off. You would change the
+signature of the `generate_data` function to:
+
+```python
+def generate_data(self, request, experiment_output, experiment, output_file=None, show_grid=False, **kwargs):
+
+ # Return a dictionary
+ return {
+ #...
+ }
+```
+
+In this example, the default value of `show_grid` is `False`, but you can make
+it `True` instead. The default value of the interactive parameter will be its
+value when it is initially invoked. It's recommended that you supply a default
+value but the default value can be `None` if there is no appropriate default
+value.
+
+Next, you need to declare the interactive parameter in the returned dictionary
+along with its current value in a special key called `interactive`. For example:
+
+```python
+def generate_data(self, request, experiment_output, experiment, output_file=None, show_grid=False, **kwargs):
+
+ # Return a dictionary
+ return {
+ #...
+ 'interactive': [
+ {'name': 'show_grid', 'value': show_grid}
+ ]
+ }
+```
+
+declares the interactive parameter named `show_grid` and its current value.
+
+The output view display will render a form showing the value of `show_grid` (in
+this case, since it is boolean, as a checkbox).
+
+#### Supported parameter types
+
+Besides boolean, the following additional parameter types are supported:
+
+| Type | UI Control | Additional options |
+| ------- | ----------------------- | ----------------------------------------------------------------------------------------------------------- |
+| Boolean | Checkbox | |
+| String | Text input | |
+| Integer | Stepper or Range slider | `min`, `max` and `step` - if `min` and `max` are supplied, renders as a range slider. `step` defaults to 1. |
+| Float | Stepper or Range slider | `min`, `max` and `step` - if `min` and `max` are supplied, renders as a range slider. |
+
+Further, if the interactive parameter defines an `options` list, this will
+render as a drop-down select. The `options` list can either be a list of
+strings, for example:
+
+```python
+def generate_data(self, request, experiment_output, experiment, output_file=None, color='red', **kwargs):
+
+ # Return a dictionary
+ return {
+ #...
+ 'interactive': [
+ {'name': 'color', 'value': color, 'options': ['red', 'green', 'blue']}
+ ]
+ }
+```
+
+Or, the `options` list can be a list of `(text, value)` tuples:
+
+```python
+def generate_data(self, request, experiment_output, experiment, output_file=None, color='red', **kwargs):
+
+ # Return a dictionary
+ return {
+ #...
+ 'interactive': [
+ {'name': 'color', 'value': color, 'options': [('Red', 'red'), ('Blue', 'blue'), ('Green', 'green')]}
+ ]
+ }
+```
+
+The `text` is what is displayed to the user as the value's label in the
+drop-down. The `value` is what will be passed to the output view provider when
+selected by the user.
+
+#### Additional configuration
+
+The following additional properties are supported:
+
+- **label** - by default the name of the interactive parameter is its label in
+ the interactive form. You can customize the label with the `label` property.
+- **help** - you can also display help text below the parameter in the
+ interactive form with the `help` property.
+
+For example:
+
+```python
+def generate_data(self, request, experiment_output, experiment, output_file=None, color='red', **kwargs):
+
+ # Return a dictionary
+ return {
+ #...
+ 'interactive': [
+ {'name': 'color',
+ 'value': color,
+ 'options': [('Red', 'red'), ('Blue', 'blue'), ('Green', 'green')],
+ 'label': 'Bar chart color',
+ 'help': 'Change the primary color of the bar chart.'}
+ ]
+ }
+```
diff --git a/docs/tutorial/gateways2019_tutorial.md b/docs/tutorial/gateways2019_tutorial.md
index d065031..1d7c0ca 100644
--- a/docs/tutorial/gateways2019_tutorial.md
+++ b/docs/tutorial/gateways2019_tutorial.md
@@ -607,6 +607,62 @@ It should look something like this:
+### (Optional) Interactive parameter
+
+In additional to producing static visualizations, output view providers can
+declare interactive parameters that can be manipulated by the user. We can add a
+simple boolean interactive parameter to toggle the display of the matplotlib
+grid as an example.
+
+1. Change the `generate_data` function so that it has an additional `show_grid`
+ parameter with a default value of `False`:
+
+```python
+ def generate_data(self, request, experiment_output, experiment, output_file=None, show_grid=False):
+```
+
+2. Add the following `.show_grid()` lines to the matplotlib code:
+
+```python
+...
+ fig.suptitle("Eigenvalues")
+ ax = fig.subplots(2, 1)
+ ax[0].plot(range(1, 10), homo_eigenvalues, label='Homo')
+ ax[0].set_ylabel('eV')
+ ax[0].legend()
+ ax[0].show_grid(show_grid)
+ ax[1].plot(range(1, 10), lumo_eigenvalues, label='Lumo')
+ ax[1].set_ylabel('eV')
+ ax[1].legend()
+ ax[1].show_grid(show_grid)
+...
+```
+
+3. Change the resulting dictionary to have the special `interactive` property
+ and declare the `show_grid` parameter:
+
+```python
+...
+ # return dictionary with image data
+ return {
+ 'image': image_bytes,
+ 'mime-type': 'image/png'
+ 'interactive': [
+ {'name': 'show_grid', 'value': show_grid}
+ ]
+ }
+```
+
+This will provider the user with a checkbox for manipulating the show_grid
+parameter. Every time the user changes it, the GaussianEigenvaluesViewProvider
+will be again invoked. It should look something like the following:
+
+
+
+There are several more interactive parameter types and additional options. You
+can learn more about them in the
+[custom output view provider documentation](../dev/custom_output_view_provider.md#interactive-parameters).
+
## Tutorial exercise: Create a custom Django app
In this tutorial exercise we'll create a fully custom user interface that lives
@@ -940,7 +996,7 @@ Now we'll use `AiravataAPI` to submit an Echo job.
value:
```javascript
-$("#run-button").click(e => {
+$("#run-button").click((e) => {
const greeting = $("#greeting-select").val();
});
```
@@ -952,7 +1008,7 @@ $("#run-button").click(e => {
```javascript
const loadAppInterface = services.ApplicationInterfaceService.retrieve({
- lookup: appInterfaceId
+ lookup: appInterfaceId,
});
```
@@ -967,7 +1023,7 @@ const loadAppInterface = services.ApplicationInterfaceService.retrieve({
const appDeploymentId =
"js-156-93.jetstream-cloud.org_Echo_37eb38ac-74c8-4aa4-a037-c656ab5bc6b8";
const loadQueues = services.ApplicationDeploymentService.getQueues({
- lookup: appDeploymentId
+ lookup: appDeploymentId,
});
```
@@ -988,15 +1044,15 @@ const loadWorkspacePrefs = services.WorkspacePreferencesService.get();
object then _save_ and _launch_ it. Here's the complete click handler:
```javascript
-$("#run-button").click(e => {
+$("#run-button").click((e) => {
const greeting = $("#greeting-select").val();
const loadAppInterface = services.ApplicationInterfaceService.retrieve({
- lookup: appInterfaceId
+ lookup: appInterfaceId,
});
const appDeploymentId =
"js-156-93.jetstream-cloud.org_Echo_37eb38ac-74c8-4aa4-a037-c656ab5bc6b8";
const loadQueues = services.ApplicationDeploymentService.getQueues({
- lookup: appDeploymentId
+ lookup: appDeploymentId,
});
const resourceHostId =
"js-156-93.jetstream-cloud.org_33019860-54c2-449b-96d3-988d4f5a501e";
@@ -1008,7 +1064,7 @@ $("#run-button").click(e => {
const experiment = appInterface.createExperiment();
experiment.experimentName = "Echo " + greeting;
experiment.projectId = workspacePrefs.most_recent_project_id;
- const cloudQueue = queues.find(q => q.queueName === queueName);
+ const cloudQueue = queues.find((q) => q.queueName === queueName);
experiment.userConfigurationData.groupResourceProfileId = groupResourceProfileId;
experiment.userConfigurationData.computationalResourceScheduling.resourceHostId = resourceHostId;
experiment.userConfigurationData.computationalResourceScheduling.totalCPUCount =
@@ -1023,9 +1079,9 @@ $("#run-button").click(e => {
return services.ExperimentService.create({ data: experiment });
})
- .then(exp => {
+ .then((exp) => {
return services.ExperimentService.launch({
- lookup: exp.experimentId
+ lookup: exp.experimentId,
});
});
});
@@ -1063,17 +1119,17 @@ We'll read the STDOUT file and display that in our experiment listing table.
```javascript
if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
services.FullExperimentService.retrieve({ lookup: exp.experimentId }).then(
- fullDetails => {
+ (fullDetails) => {
const stdoutDataProductId = fullDetails.experiment.experimentOutputs.find(
- o => o.name === "Echo-STDOUT"
+ (o) => o.name === "Echo-STDOUT"
).value;
const stdoutDataProduct = fullDetails.outputDataProducts.find(
- dp => dp.productUri === stdoutDataProductId
+ (dp) => dp.productUri === stdoutDataProductId
);
if (stdoutDataProduct && stdoutDataProduct.downloadURL) {
return fetch(stdoutDataProduct.downloadURL, {
- credentials: "same-origin"
- }).then(result => result.text());
+ credentials: "same-origin",
+ }).then((result) => result.text());
}
}
);
@@ -1085,20 +1141,20 @@ if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
```javascript
if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
services.FullExperimentService.retrieve({ lookup: exp.experimentId })
- .then(fullDetails => {
+ .then((fullDetails) => {
const stdoutDataProductId = fullDetails.experiment.experimentOutputs.find(
- o => o.name === "Echo-STDOUT"
+ (o) => o.name === "Echo-STDOUT"
).value;
const stdoutDataProduct = fullDetails.outputDataProducts.find(
- dp => dp.productUri === stdoutDataProductId
+ (dp) => dp.productUri === stdoutDataProductId
);
if (stdoutDataProduct && stdoutDataProduct.downloadURL) {
return fetch(stdoutDataProduct.downloadURL, {
- credentials: "same-origin"
- }).then(result => result.text());
+ credentials: "same-origin",
+ }).then((result) => result.text());
}
})
- .then(text => {
+ .then((text) => {
$(`#output_${index}`).text(text);
});
}
@@ -1121,8 +1177,8 @@ function loadExperiments() {
limit: 5,
[models.ExperimentSearchFields.USER_NAME.name]:
session.Session.username,
- [models.ExperimentSearchFields.APPLICATION_ID.name]: appInterfaceId
- }).then(data => {
+ [models.ExperimentSearchFields.APPLICATION_ID.name]: appInterfaceId,
+ }).then((data) => {
$("#experiment-list").empty();
data.results.forEach((exp, index) => {
$("#experiment-list").append(
@@ -1137,28 +1193,28 @@ function loadExperiments() {
// If experiment has finished, load full details, then parse the stdout file
if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
services.FullExperimentService.retrieve({
- lookup: exp.experimentId
+ lookup: exp.experimentId,
})
- .then(fullDetails => {
+ .then((fullDetails) => {
const stdoutDataProductId = fullDetails.experiment.experimentOutputs.find(
- o => o.name === "Echo-STDOUT"
+ (o) => o.name === "Echo-STDOUT"
).value;
const stdoutDataProduct = fullDetails.outputDataProducts.find(
- dp => dp.productUri === stdoutDataProductId
+ (dp) => dp.productUri === stdoutDataProductId
);
if (
stdoutDataProduct &&
stdoutDataProduct.downloadURL
) {
return fetch(stdoutDataProduct.downloadURL, {
- credentials: "same-origin"
- }).then(result => result.text());
+ credentials: "same-origin",
+ }).then((result) => result.text());
} else {
// If we can't download it, fake it
return FAKE_STDOUT;
}
})
- .then(text => {
+ .then((text) => {
$(`#output_${index}`).text(text);
});
}
diff --git a/docs/tutorial/screenshots/gateways19/gaussian-eigenvalues-show_grid.png b/docs/tutorial/screenshots/gateways19/gaussian-eigenvalues-show_grid.png
new file mode 100644
index 0000000..07bb66d
Binary files /dev/null and b/docs/tutorial/screenshots/gateways19/gaussian-eigenvalues-show_grid.png differ
diff --git a/mkdocs.yml b/mkdocs.yml
index 7740248..6bf42ea 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -8,14 +8,15 @@ nav:
- Using the integrated CMS: cms.md
- Developer Guide:
- Adding a Django App: dev/new_django_app.md
- - Adding a Custom Django App: dev/custom_django_app.md
- Developing the Frontend: dev/developing_frontend.md
- Developing the Backend: dev/developing_backend.md
- Developing a Wagtail Export (theme): dev/wagtail_export.md
+ - Customization Guide:
+ - Gateways 2019 Tutorial: tutorial/gateways2019_tutorial.md
+ - Adding a Custom Django App: dev/custom_django_app.md
+ - Adding a Custom Output View Provider: dev/custom_output_view_provider.md
- Administrator Guide:
- Tusd Installation: admin/tusd.md
- - Tutorials:
- - Gateways 2019: tutorial/gateways2019_tutorial.md
theme: readthedocs