You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airavata.apache.org by ma...@apache.org on 2021/06/16 18:39:17 UTC
[airavata-django-portal] 01/10: Updated output view provider
portion of tutorial to use cookiecutter
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-django-portal.git
commit c722afab543e99084d359293ac2ffe4126764da6
Author: Marcus Christie <ma...@apache.org>
AuthorDate: Tue Jun 8 11:50:30 2021 -0400
Updated output view provider portion of tutorial to use cookiecutter
---
Dockerfile | 2 +-
docs/tutorial/gateways_tutorial.md | 219 ++++++++++++++++++++++++++-----------
2 files changed, 156 insertions(+), 65 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 8e4fd4e..988ee3c 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -65,7 +65,7 @@ WORKDIR /code
COPY requirements.txt requirements-mysql.txt ./
COPY setup.* ./
COPY README.md .
-RUN pip install --upgrade pip
+RUN pip install --upgrade pip setuptools wheel
RUN pip install -r requirements.txt
RUN pip install -r requirements-mysql.txt
diff --git a/docs/tutorial/gateways_tutorial.md b/docs/tutorial/gateways_tutorial.md
index 9a6ef1b..1ebef32 100644
--- a/docs/tutorial/gateways_tutorial.md
+++ b/docs/tutorial/gateways_tutorial.md
@@ -347,8 +347,39 @@ possible to provide additional custom views for output files. Examples include:
- parameterized notebook
To be able to create a custom output viewer we'll need to write some Python
-code. First, we'll get a local version of the Django portal running which we'll
-use as a development environment.
+code. First we'll generate the custom Django app code.
+
+### Generate a custom Django app
+
+We have a local develop environment created. Now we can start adding custom
+code. First, we'll create a custom Django app, which is the standard way to
+package a Django extension.
+
+1. Install the latest version of cookiecutter. Cookiecutter is a tool for
+ generating project source code from a template.
+
+```sh
+pip install --user -U cookiecutter
+```
+
+2. Use cookiecutter to run the Airavata Django app template.
+
+```sh
+cd $HOME
+cookiecutter https://github.com/machristie/cookiecutter-airavata-django-app.git
+```
+
+You'll need to answer some questions. You can name it whatever you want, but to
+follow along with the tutorial, for `project_name` give **Gateways Tutorial
+App**. For the rest of the questions, you can simply accept the defaults:
+
+```
+project_name [My Custom Django App]: Gateways Tutorial App
+project_slug [gateways_tutorial_app]:
+project_short_description [Custom Django app with everything needed to be installed in the airavata-django-portal]:
+app_config_class_name [GatewaysTutorialAppConfig]:
+version [0.1.0]:
+```
### Setup local Django portal development environment
@@ -365,9 +396,9 @@ if you are interested).
```
cd $HOME
-git clone https://github.com/machristie/gateways19-tutorial.git
-cd gateways19-tutorial
-docker run -d --name gateways19-tutorial -p 8000:8000 -v "$PWD:/extensions" -v "$PWD/settings_local.py:/code/django_airavata/settings_local.py" machristie/airavata-django-portal
+git clone https://github.com/machristie/gateways19-tutorial.git gateways19-tutorial-final
+cd gateways_tutorial_app
+docker run -d --name gateways19-tutorial -p 8000:8000 -v "$PWD:/extensions" -v "$PWD/../gateways19-tutorial-final/settings_local.py:/code/django_airavata/settings_local.py" machristie/airavata-django-portal
```
!!! note "For remote Docker host users"
@@ -408,36 +439,74 @@ Go to [http://localhost:8000](http://localhost:8000), click on **Login in**,
enter your username and password. On the dashboard you should see the your
experiments listed on the right hand side.
-### Setup the custom output viewer package
+### Create the custom output viewer
-1. We've defined a custom output view provider, called
- GaussianEigenvaluesViewProvider, in `output_views.py`. Open
- `$HOME/gateways19-tutorial/gateways19_tutorial/output_views.py` in your
- editor and we'll look at how it is implemented. First we add some imports
+Now we'll also generate and implement a **Gaussian Eigenvalues View** provider.
-```python
-import io
-import os
+1. We'll run another cookiecutter template to generate the output view provider
+ code. First, change into the `gateways_tutorial_app` generated in the
+ previous step:
-import numpy as np
-from matplotlib.figure import Figure
+```sh
+cd $HOME/gateways_tutorial_app
+```
-from cclib.parser import ccopen
+2. Run the following cookiecutter template:
+
+```sh
+cookiecutter https://github.com/machristie/cookiecutter-airavata-django-output-view.git -f
+```
+
+You'll need to answer some questions again. For `project_name` give **Gaussian
+Eigenvalues View**. For `custom_django_app_module_name`, you need to provide the
+name of the Python module that was generated by
+cookiecutter-airavata-django-app, which for this tutorial is
+**gateways_tutorial_app**. For all of the other questions you can accept the
+default.
-BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+```
+project_name [My Custom Output View]: Gaussian Eigenvalues View
+project_slug [gaussian_eigenvalues_view]:
+project_short_description [Gaussian Eigenvalues View generates data for an output view in the Airavata Django Portal]:
+output_view_provider_class_name [GaussianEigenvaluesViewProvider]:
+custom_django_app_module_name []: gateways_tutorial_app
+output_views_directory_name [output_views]:
+Select output_view_display_type:
+1 - image
+2 - link
+3 - html
+Choose from 1, 2, 3 [1]:
+Select number_of_output_files:
+1 - single (URI)
+2 - multiple (URI_COLLECTION)
+Choose from 1, 2 [1]:
```
-2. Next we define the GaussianEigenvaluesViewProvider class, set some metadata
- attributes on the class. We set it's `display_type` to _image_ and give it a
- _name_:
+3. This creates a custom output view provider, called
+ GaussianEigenvaluesViewProvider, in `gateways-tutorialoutput_views/`. Open
+ `$HOME/gateways19-tutorial/gateways_tutorial_app/gateways_tutorial_app/output_views/gaussian_eigenvalues_view.py`
+ in your editor and we'll look at the generated code. The cookiecutter
+ template has generated a GaussianEigenvaluesViewProvider class with a method
+ called `generate_data`. The generate_data method has some commented out code
+ samples and links to further information. There is also guidance at the end
+ for how to prepare the values expected in the returned dictionary. Let's
+ start filling in the implementation.
+
+4. First we'll add some imports at the top. Replace the existing imports with these:
```python
-class GaussianEigenvaluesViewProvider:
- display_type = 'image'
- name = "Gaussian Eigenvalues"
+import io
+import os
+
+import numpy as np
+from cclib.parser import ccopen
+from django.conf import settings
+from matplotlib.figure import Figure
+
+from airavata_django_portal_sdk import user_storage
```
-3. Now we implement the
+5. Now we implement the
[`generate_data` function](../dev/custom_output_view_provider.md#output-view-provider-interface).
This function should return a dictionary with values that are expected for
this `display_type`. For a display type of _image_, the required return
@@ -497,18 +566,20 @@ molecular orbital energies. Then `matplotlib` is used to create two plots of
these values. Finally, the plots are exported as a PNG image that is returns as
a buffer of bytes.
-4. Altogether, the output_views.py file should have the following contents:
+7. Altogether, the output_views/gaussian_eigenvalues_view.py file should have
+ the following contents:
```python
import io
import os
import numpy as np
+from cclib.parser import ccopen
+from django.conf import settings
from matplotlib.figure import Figure
-from cclib.parser import ccopen
+from airavata_django_portal_sdk import user_storage
-BASE_DIR = os.path.dirname(os.path.abspath(__file__))
class GaussianEigenvaluesViewProvider:
display_type = 'image'
@@ -561,31 +632,44 @@ class GaussianEigenvaluesViewProvider:
```
-5. Now we need to register our _output view provider_ with the package metadata
- so that the Django Portal will be able to discover it. We add the following
- lines to the `entry_points` parameter in the
- `$HOME/gateways19-tutorial/setup.py` file:
-
-```python
-setuptools.setup(
-# ...
- entry_points="""
-[airavata.output_view_providers]
-gaussian-eigenvalues-plot = gateways19_tutorial.output_views:GaussianEigenvaluesViewProvider
-""",
-)
+8. Now we need to register our _output view provider_ with the package metadata
+ so that the Django Portal will be able to discover it. The cookiecutter
+ template already created this when it generated the
+ gaussian_eigenvalues_view.py code. We can take a look and make sure it added
+ an `airavata.output_view_providers` entry to the `[options.entry_points]`
+ section in the `$HOME/gateways19-tutorial/gateways_tutorial_app/setup.cfg`
+ file:
+
+```ini
+[options.entry_points]
+airavata.djangoapp =
+ gateways_tutorial_app = gateways_tutorial_app.apps:GatewaysTutorialAppConfig
+airavata.output_view_providers =
+ gaussian_eigenvalues_view = gateways_tutorial_app.output_views.gaussian_eigenvalues_view:GaussianEigenvaluesViewProvider
```
-`gaussian-eigenvalues-plot` is the output view provider id.
-`gateways19_tutorial.output_views` is the module in which the
-`GaussianEigenvaluesViewProvider` output view provider class is found.
+`gaussian_eigenvalues_view` is the output view provider id.
+`gateways_tutorial_app.output_views.gaussian_eigenvalues_view` is the module in
+which the `GaussianEigenvaluesViewProvider` output view provider class is found.
+
+9. While we're looking at setup.cfg, let's add our output view providers Python
+ dependencies. Under `install_requires` add _cclib_, _numpy_ and _matplotlib_,
+ so that it looks like:
+
+```ini
+install_requires =
+ django >= 2.2
+ airavata-django-portal-sdk
+ cclib
+ numpy
+ matplotlib
+```
-6. Now we need to install the _gateways19-tutorial_ package into the Django
- portal's virtual environment.
+10. Now we need to install the _gateways_tutorial_app_ package into the Django
+ portal's virtual environment.
```bash
-docker exec -w /extensions gateways19-tutorial pip install -r requirements.txt
-docker exec -w /extensions gateways19-tutorial python setup.py develop
+docker exec -w /extensions gateways19-tutorial pip install -e .
docker exec gateways19-tutorial touch /code/django_airavata/wsgi.py
```
@@ -610,7 +694,7 @@ additional output view of the file.
```json
{
- "output-view-providers": ["gaussian-eigenvalues-plot"]
+ "output-view-providers": ["gaussian_eigenvalues_view"]
}
```
@@ -633,7 +717,7 @@ declare interactive parameters that can be manipulated by the user. We can add a
simple boolean interactive parameter to toggle the display of the matplotlib
grid as an example.
-1. Open `$HOME/gateways19-tutorial/gateways19_tutorial/output_views.py`. Change
+1. Open `$HOME/gateways_tutorial_app/gateways_tutorial_app/output_views/gaussian_eigenvalues_view.py`. Change
the `generate_data` function so that it has an additional `show_grid`
parameter with a default value of `False`:
@@ -1102,15 +1186,18 @@ $("#run-button").click((e) => {
experiment.experimentName = "Echo " + greeting;
experiment.projectId = workspacePrefs.most_recent_project_id;
const cloudQueue = queues.find((q) => q.queueName === queueName);
- experiment.userConfigurationData.groupResourceProfileId = groupResourceProfileId;
- experiment.userConfigurationData.computationalResourceScheduling.resourceHostId = resourceHostId;
+ experiment.userConfigurationData.groupResourceProfileId =
+ groupResourceProfileId;
+ experiment.userConfigurationData.computationalResourceScheduling.resourceHostId =
+ resourceHostId;
experiment.userConfigurationData.computationalResourceScheduling.totalCPUCount =
cloudQueue.defaultCPUCount;
experiment.userConfigurationData.computationalResourceScheduling.nodeCount =
cloudQueue.defaultNodeCount;
experiment.userConfigurationData.computationalResourceScheduling.wallTimeLimit =
cloudQueue.defaultWalltime;
- experiment.userConfigurationData.computationalResourceScheduling.queueName = queueName;
+ experiment.userConfigurationData.computationalResourceScheduling.queueName =
+ queueName;
// Copy the selected greeting to the value of the first input
experiment.experimentInputs[0].value = greeting;
@@ -1151,9 +1238,10 @@ We'll read the STDOUT file and display that in our experiment listing table.
if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
services.FullExperimentService.retrieve({ lookup: exp.experimentId }).then(
(fullDetails) => {
- const stdoutDataProductId = fullDetails.experiment.experimentOutputs.find(
- (o) => o.name === "Echo-STDOUT"
- ).value;
+ const stdoutDataProductId =
+ fullDetails.experiment.experimentOutputs.find(
+ (o) => o.name === "Echo-STDOUT"
+ ).value;
const stdoutDataProduct = fullDetails.outputDataProducts.find(
(dp) => dp.productUri === stdoutDataProductId
);
@@ -1173,9 +1261,10 @@ if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
if (exp.experimentStatus === models.ExperimentState.COMPLETED) {
services.FullExperimentService.retrieve({ lookup: exp.experimentId })
.then((fullDetails) => {
- const stdoutDataProductId = fullDetails.experiment.experimentOutputs.find(
- (o) => o.name === "Echo-STDOUT"
- ).value;
+ const stdoutDataProductId =
+ fullDetails.experiment.experimentOutputs.find(
+ (o) => o.name === "Echo-STDOUT"
+ ).value;
const stdoutDataProduct = fullDetails.outputDataProducts.find(
(dp) => dp.productUri === stdoutDataProductId
);
@@ -1220,12 +1309,14 @@ function loadExperiments() {
lookup: exp.experimentId,
})
.then((fullDetails) => {
- const stdoutDataProductId = fullDetails.experiment.experimentOutputs.find(
- (o) => o.name === "Echo-STDOUT"
- ).value;
- const stdoutDataProduct = fullDetails.outputDataProducts.find(
- (dp) => dp.productUri === stdoutDataProductId
- );
+ const stdoutDataProductId =
+ fullDetails.experiment.experimentOutputs.find(
+ (o) => o.name === "Echo-STDOUT"
+ ).value;
+ const stdoutDataProduct =
+ fullDetails.outputDataProducts.find(
+ (dp) => dp.productUri === stdoutDataProductId
+ );
if (
stdoutDataProduct &&
stdoutDataProduct.downloadURL