You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airavata.apache.org by ma...@apache.org on 2023/04/27 20:23:27 UTC
[airavata] branch develop updated (d153e225ab -> 8bf8ee0069)
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a change to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
from d153e225ab Merge pull request #419 from isururanawaka/metaschedular
add 5d194bfd52 Ansible: adding GDAL dependency for global floods
add 2dd8fc5a73 Ansible: disabling new simvascular host for now
add 40a7a53781 Commented gateways to discontinue. WVSU, UAB, TopPIC, V4I
add aad3d5fa62 Adding AI-confluence Cybershuttle gateway
add de810dcfc8 Adding new AI-Confluence gateway
add 54bf27b827 Adding new AI-Confluence gateway CILogon and key
add 03175bd669 Adding Citation Info
add 2b0ee9d1a6 Deploying Molecular Dynamics Cybershuttle gateway
add 21d3001191 Adding gateway generated key to storage resource.
add 6a47c07f47 Enabling CILogon for the gateway
add 21a85b6229 Adding NASA VEDA gateway
add 7cca325bce Adding CILogon and gateway key for storage resource.
add aa2812e6e9 Adding the new gateway hosting VM ip address to host file for dREG gateway
add 61bce69104 Merge branch 'archive-user-data'
add 7876cea94f AIRAVATA-3694 configured testdrive for user data archives
new f4cc483734 AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess
new 538745201b Merge branch 'AIRAVATA-3697' into develop
new 33300327fc AIRAVATA-3697 Db migrations and changing PRODUCT_NAME, REPLICA_NAME to clobs
new 8bf8ee0069 Merge branch 'AIRAVATA-3697' into develop
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CITATION.cff | 56 ++++++++++++++++++++++
.../scigap/production/group_vars/django/vars.yml | 2 +
.../vars.yml | 16 +++----
.../production/host_vars/ai-confluence/vault.yml | 12 +++++
.../production/host_vars/global-flood/vars.yml | 2 +
.../vars.yml | 16 +++----
.../host_vars/molecular-dynamics/vault.yml | 12 +++++
.../vars.yml | 16 +++----
.../production/host_vars/nasa-veda/vault.yml | 12 +++++
.../scigap/production/host_vars/testdrive/vars.yml | 1 +
.../ansible/inventories/scigap/production/hosts | 17 ++++---
.../django/templates/django-ssl-vhost.conf.j2 | 12 ++++-
modules/ide-integration/README.md | 2 +-
.../init/05-replica-catalog-migrations.sql | 35 ++++++++++++++
.../entities/replicacatalog/DataProductEntity.java | 1 +
.../replicacatalog/DataReplicaLocationEntity.java | 1 +
.../src/main/resources/replicacatalog-mysql.sql | 35 +++++++-------
.../DeltaScripts/replicaCatalog_schema_delta.sql | 34 ++++++++++++-
18 files changed, 228 insertions(+), 54 deletions(-)
create mode 100644 CITATION.cff
copy dev-tools/ansible/inventories/scigap/production/host_vars/{neuroscience-cybershuttle => ai-confluence}/vars.yml (70%)
create mode 100644 dev-tools/ansible/inventories/scigap/production/host_vars/ai-confluence/vault.yml
copy dev-tools/ansible/inventories/scigap/production/host_vars/{neuroscience-cybershuttle => molecular-dynamics}/vars.yml (70%)
create mode 100644 dev-tools/ansible/inventories/scigap/production/host_vars/molecular-dynamics/vault.yml
copy dev-tools/ansible/inventories/scigap/production/host_vars/{neuroscience-cybershuttle => nasa-veda}/vars.yml (70%)
create mode 100644 dev-tools/ansible/inventories/scigap/production/host_vars/nasa-veda/vault.yml
create mode 100644 modules/ide-integration/src/main/containers/database_scripts/init/05-replica-catalog-migrations.sql
[airavata] 01/04: AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
commit f4cc483734c6f9c674c26e5f6b9627c4d7e9fc96
Author: Marcus Christie <ma...@apache.org>
AuthorDate: Thu Apr 27 15:42:44 2023 -0400
AIRAVATA-3697 Set lang and locale on WSGIDaemonProcess
This is so that the Django portal code can handle filenames with UTF8 characters.
---
.../ansible/roles/django/templates/django-ssl-vhost.conf.j2 | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/dev-tools/ansible/roles/django/templates/django-ssl-vhost.conf.j2 b/dev-tools/ansible/roles/django/templates/django-ssl-vhost.conf.j2
index 2c2f73f3b2..0764c34bd2 100644
--- a/dev-tools/ansible/roles/django/templates/django-ssl-vhost.conf.j2
+++ b/dev-tools/ansible/roles/django/templates/django-ssl-vhost.conf.j2
@@ -43,7 +43,7 @@
<VirtualHost *:{{ httpd_default_http_port }}>
ServerName {{ vhost_servername }}
-
+
## Redirect all http traffic to https
RewriteEngine On
RewriteCond %{HTTPS} off
@@ -99,7 +99,15 @@
{% endif %}
{% endfor %}
- WSGIDaemonProcess {{ vhost_servername }} display-name=%{GROUP} python-home={{ doc_root_dir }}/venv python-path={{ doc_root_dir }}/airavata-django-portal processes={{ django_wsgi_processes }} user={{ user }} group={{ group }}
+ WSGIDaemonProcess {{ vhost_servername }} \
+ display-name=%{GROUP} \
+ python-home={{ doc_root_dir }}/venv \
+ python-path={{ doc_root_dir }}/airavata-django-portal \
+ processes={{ django_wsgi_processes }} \
+ user={{ user }} \
+ group={{ group }} \
+ lang=en_US.UTF-8 \
+ locale=en_US.UTF-8
WSGIProcessGroup {{ vhost_servername }}
WSGIScriptAlias / {{ doc_root_dir }}/airavata-django-portal/django_airavata/wsgi.py
[airavata] 03/04: AIRAVATA-3697 Db migrations and changing PRODUCT_NAME, REPLICA_NAME to clobs
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
commit 33300327fc98d34ba5a62dbe75aea2678120a065
Author: Marcus Christie <ma...@apache.org>
AuthorDate: Thu Apr 27 16:22:34 2023 -0400
AIRAVATA-3697 Db migrations and changing PRODUCT_NAME, REPLICA_NAME to clobs
---
modules/ide-integration/README.md | 2 +-
.../init/05-replica-catalog-migrations.sql | 35 ++++++++++++++++++++++
.../entities/replicacatalog/DataProductEntity.java | 1 +
.../replicacatalog/DataReplicaLocationEntity.java | 1 +
.../src/main/resources/replicacatalog-mysql.sql | 35 ++++++++++------------
.../DeltaScripts/replicaCatalog_schema_delta.sql | 34 ++++++++++++++++++++-
6 files changed, 87 insertions(+), 21 deletions(-)
diff --git a/modules/ide-integration/README.md b/modules/ide-integration/README.md
index e661c4d83c..6ac08fa260 100644
--- a/modules/ide-integration/README.md
+++ b/modules/ide-integration/README.md
@@ -60,7 +60,7 @@ Using this module, you can setup a full Airavata installation inside Intelij IDE
* Apply any database migrations. Go to src/main/containers directory and run
```
- cat ./database_scripts/init/*-migrations.sql | docker exec -i resources_db_1 mysql -p123456
+ cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456
```
* Wait until all the services come up. This will initialize all utilities required to start Airavata server
diff --git a/modules/ide-integration/src/main/containers/database_scripts/init/05-replica-catalog-migrations.sql b/modules/ide-integration/src/main/containers/database_scripts/init/05-replica-catalog-migrations.sql
new file mode 100644
index 0000000000..db5e878fb9
--- /dev/null
+++ b/modules/ide-integration/src/main/containers/database_scripts/init/05-replica-catalog-migrations.sql
@@ -0,0 +1,35 @@
+USE replica_catalog;
+
+-- AIRAVATA-3697: Support file names that have UTF8 characters and that are long
+SET
+ FOREIGN_KEY_CHECKS = 0;
+
+ALTER DATABASE replica_catalog CHARACTER SET = utf8 COLLATE = utf8_bin;
+
+ALTER TABLE
+ DATA_PRODUCT CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ DATA_REPLICA_LOCATION CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ DATA_PRODUCT_METADATA CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ DATA_REPLICA_METADATA CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ CONFIGURATION CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+SET
+ FOREIGN_KEY_CHECKS = 1;
+
+ALTER TABLE
+ DATA_REPLICA_LOCATION
+MODIFY
+ REPLICA_NAME text;
+
+ALTER TABLE
+ DATA_PRODUCT
+MODIFY
+ PRODUCT_NAME text;
diff --git a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataProductEntity.java b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataProductEntity.java
index ba10d59620..e5c2eeac0f 100644
--- a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataProductEntity.java
+++ b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataProductEntity.java
@@ -43,6 +43,7 @@ public class DataProductEntity implements Serializable {
@Column(name = "GATEWAY_ID")
private String gatewayId;
+ @Lob
@Column(name = "PRODUCT_NAME")
private String productName;
diff --git a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataReplicaLocationEntity.java b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataReplicaLocationEntity.java
index ca0360b306..d943163eab 100644
--- a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataReplicaLocationEntity.java
+++ b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/entities/replicacatalog/DataReplicaLocationEntity.java
@@ -43,6 +43,7 @@ public class DataReplicaLocationEntity implements Serializable {
@Column(name = "PRODUCT_URI")
private String productUri;
+ @Lob
@Column(name = "REPLICA_NAME")
private String replicaName;
diff --git a/modules/registry/registry-core/src/main/resources/replicacatalog-mysql.sql b/modules/registry/registry-core/src/main/resources/replicacatalog-mysql.sql
index c3890702ca..7c8a0ee11d 100644
--- a/modules/registry/registry-core/src/main/resources/replicacatalog-mysql.sql
+++ b/modules/registry/registry-core/src/main/resources/replicacatalog-mysql.sql
@@ -18,12 +18,10 @@
* under the License.
*
*/
-
-CREATE TABLE DATA_PRODUCT
-(
+CREATE TABLE DATA_PRODUCT (
PRODUCT_URI VARCHAR (255),
GATEWAY_ID VARCHAR (255),
- PRODUCT_NAME VARCHAR (255),
+ PRODUCT_NAME TEXT,
PRODUCT_DESCRIPTION VARCHAR (255),
OWNER_NAME VARCHAR (255),
PARENT_PRODUCT_URI VARCHAR (255),
@@ -33,13 +31,12 @@ CREATE TABLE DATA_PRODUCT
PRODUCT_TYPE VARCHAR(10),
PRIMARY KEY (PRODUCT_URI),
FOREIGN KEY (PARENT_PRODUCT_URI) REFERENCES DATA_PRODUCT (PRODUCT_URI) ON DELETE CASCADE
-)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+) ENGINE = InnoDB DEFAULT CHARSET = utf8 COLLATE = utf8_bin;
-CREATE TABLE DATA_REPLICA_LOCATION
-(
+CREATE TABLE DATA_REPLICA_LOCATION (
REPLICA_ID VARCHAR (255),
PRODUCT_URI VARCHAR (255) NOT NULL,
- REPLICA_NAME VARCHAR (255),
+ REPLICA_NAME TEXT,
REPLICA_DESCRIPTION VARCHAR (255),
STORAGE_RESOURCE_ID VARCHAR (255),
FILE_PATH VARCHAR (1024),
@@ -50,31 +47,31 @@ CREATE TABLE DATA_REPLICA_LOCATION
REPLICA_PERSISTENT_TYPE VARCHAR(10),
PRIMARY KEY (REPLICA_ID),
FOREIGN KEY (PRODUCT_URI) REFERENCES DATA_PRODUCT(PRODUCT_URI) ON DELETE CASCADE
-)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+) ENGINE = InnoDB DEFAULT CHARSET = utf8 COLLATE = utf8_bin;
-CREATE TABLE DATA_PRODUCT_METADATA
-(
+CREATE TABLE DATA_PRODUCT_METADATA (
PRODUCT_URI VARCHAR(255) NOT NULL,
METADATA_KEY VARCHAR(255),
METADATA_VALUE VARCHAR(255),
PRIMARY KEY(PRODUCT_URI, METADATA_KEY),
FOREIGN KEY (PRODUCT_URI) REFERENCES DATA_PRODUCT(PRODUCT_URI) ON DELETE CASCADE
-)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+) ENGINE = InnoDB DEFAULT CHARSET = utf8 COLLATE = utf8_bin;
-CREATE TABLE DATA_REPLICA_METADATA
-(
+CREATE TABLE DATA_REPLICA_METADATA (
REPLICA_ID VARCHAR(255) NOT NULL,
METADATA_KEY VARCHAR(255),
METADATA_VALUE VARCHAR(255),
PRIMARY KEY(REPLICA_ID, METADATA_KEY),
FOREIGN KEY (REPLICA_ID) REFERENCES DATA_REPLICA_LOCATION(REPLICA_ID) ON DELETE CASCADE
-)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+) ENGINE = InnoDB DEFAULT CHARSET = utf8 COLLATE = utf8_bin;
-CREATE TABLE CONFIGURATION
-(
+CREATE TABLE CONFIGURATION (
CONFIG_KEY VARCHAR(255),
CONFIG_VAL VARCHAR(255),
PRIMARY KEY(CONFIG_KEY, CONFIG_VAL)
-)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+) ENGINE = InnoDB DEFAULT CHARSET = utf8 COLLATE = utf8_bin;
-INSERT INTO CONFIGURATION (CONFIG_KEY, CONFIG_VAL) VALUES('data_catalog_version', '0.16');
+INSERT INTO
+ CONFIGURATION (CONFIG_KEY, CONFIG_VAL)
+VALUES
+('data_catalog_version', '0.16');
diff --git a/modules/registry/release-migration-scripts/next/DeltaScripts/replicaCatalog_schema_delta.sql b/modules/registry/release-migration-scripts/next/DeltaScripts/replicaCatalog_schema_delta.sql
index 942ff54c85..3b4645208f 100644
--- a/modules/registry/release-migration-scripts/next/DeltaScripts/replicaCatalog_schema_delta.sql
+++ b/modules/registry/release-migration-scripts/next/DeltaScripts/replicaCatalog_schema_delta.sql
@@ -17,5 +17,37 @@
-- specific language governing permissions and limitations
-- under the License.
--
+USE replica_catalog;
-use replica_catalog;
+SET
+ FOREIGN_KEY_CHECKS = 0;
+
+ALTER DATABASE replica_catalog CHARACTER SET = utf8 COLLATE = utf8_bin;
+
+ALTER TABLE
+ DATA_PRODUCT CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ DATA_REPLICA_LOCATION CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ DATA_PRODUCT_METADATA CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ DATA_REPLICA_METADATA CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+ALTER TABLE
+ CONFIGURATION CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
+
+SET
+ FOREIGN_KEY_CHECKS = 1;
+
+ALTER TABLE
+ DATA_REPLICA_LOCATION
+MODIFY
+ REPLICA_NAME text;
+
+ALTER TABLE
+ DATA_PRODUCT
+MODIFY
+ PRODUCT_NAME text;
[airavata] 02/04: Merge branch 'AIRAVATA-3697' into develop
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
commit 538745201bd07a88ae3696aa59fa6d8ae4e81ceb
Merge: d153e225ab f4cc483734
Author: Marcus Christie <ma...@apache.org>
AuthorDate: Thu Apr 27 15:43:14 2023 -0400
Merge branch 'AIRAVATA-3697' into develop
CITATION.cff | 56 ++++++++++++++++++++++
.../scigap/production/group_vars/django/vars.yml | 2 +
.../production/host_vars/ai-confluence/vars.yml | 56 ++++++++++++++++++++++
.../production/host_vars/ai-confluence/vault.yml | 12 +++++
.../production/host_vars/global-flood/vars.yml | 2 +
.../host_vars/molecular-dynamics/vars.yml | 56 ++++++++++++++++++++++
.../host_vars/molecular-dynamics/vault.yml | 12 +++++
.../scigap/production/host_vars/nasa-veda/vars.yml | 56 ++++++++++++++++++++++
.../production/host_vars/nasa-veda/vault.yml | 12 +++++
.../scigap/production/host_vars/testdrive/vars.yml | 1 +
.../ansible/inventories/scigap/production/hosts | 17 ++++---
.../django/templates/django-ssl-vhost.conf.j2 | 12 ++++-
12 files changed, 285 insertions(+), 9 deletions(-)
[airavata] 04/04: Merge branch 'AIRAVATA-3697' into develop
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
machristie pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
commit 8bf8ee00691c016d7357aba094fd43b093aebfe0
Merge: 538745201b 33300327fc
Author: Marcus Christie <ma...@apache.org>
AuthorDate: Thu Apr 27 16:23:02 2023 -0400
Merge branch 'AIRAVATA-3697' into develop
modules/ide-integration/README.md | 2 +-
.../init/05-replica-catalog-migrations.sql | 35 ++++++++++++++++++++++
.../entities/replicacatalog/DataProductEntity.java | 1 +
.../replicacatalog/DataReplicaLocationEntity.java | 1 +
.../src/main/resources/replicacatalog-mysql.sql | 35 ++++++++++------------
.../DeltaScripts/replicaCatalog_schema_delta.sql | 34 ++++++++++++++++++++-
6 files changed, 87 insertions(+), 21 deletions(-)