You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2020/09/02 21:32:07 UTC
[GitHub] [incubator-superset] ejianu opened a new pull request #10771: [Helm] Automate datasource import
ejianu opened a new pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771
### SUMMARY
Helm chart change: Added 2 possible properties which will essentially automate the creation of database connections in superset.
Changes in the values file:
`extraConfigs` to which we can pass a yaml file that we exported previously, that defines a datasource.
`extraConfigMountPath` location at which this file will be
Changes in the chart:
Added a configMap template file that only creates the ConfigMap if the `extraConfigs` is set.
In the init-job file, the new "extra-config" volume is mounted only if `extraConfigs` is set.
In this way, the original approach should not be affected.
### TEST PLAN
1. Make sure you create a datasource in the interface and export it to yaml, then delete it and check it was deleted.
2. In the values file set the yaml you exported, this is an example, but any datasource exported yaml should do:
```
extraConfigs:
datasources.yaml: |
databases:
- allow_csv_upload: true
allow_ctas: true
allow_cvas: true
database_name: presto
extra: "{\r\n \"metadata_params\": {},\r\n \"engine_params\": {},\r\n \"\
metadata_cache_timeout\": {},\r\n \"schemas_allowed_for_csv_upload\": []\r\n\
}"
sqlalchemy_uri: presto://presto.whatever
tables: []
```
3. set the following property, the datasource file will be mounted at some location:
`extraConfigMountPath: "/app/configs"`
4. under the Init job configuration in the init script add a command that imports the file we defined previously:
echo "Initializing connection.... "
superset import_datasources -p {{ .Values.extraConfigMountPath }}/datasources.yaml
5. Check if the datasource was imported correctly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] nytai commented on a change in pull request #10771: [Helm] Automate datasource import
Posted by GitBox <gi...@apache.org>.
nytai commented on a change in pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771#discussion_r502222450
##########
File path: helm/superset/values.yaml
##########
@@ -238,4 +253,4 @@ nodeSelector: {}
tolerations: []
-affinity: {}
+affinity: {}
Review comment:
```suggestion
affinity: {}
```
##########
File path: helm/superset/templates/configmap-superset.yaml
##########
@@ -0,0 +1,32 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+{{- if .Values.extraConfigs }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ template "superset.fullname" . }}-extra-config
+ labels:
+ app: {{ template "superset.name" . }}
+ chart: {{ template "superset.chart" . }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+data:
+{{- range $path, $config := .Values.extraConfigs }}
+ {{ $path }}: |
+{{ tpl $config . | indent 4 -}}
+{{- end -}}
+{{- end -}}
Review comment:
```suggestion
{{- end -}}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] ejianu commented on pull request #10771: [Helm] Automate datasource import
Posted by GitBox <gi...@apache.org>.
ejianu commented on pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771#issuecomment-686433046
Changes made :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] craig-rueda commented on a change in pull request #10771: [Helm] Automate datasource import
Posted by GitBox <gi...@apache.org>.
craig-rueda commented on a change in pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771#discussion_r482677951
##########
File path: helm/superset/templates/configmap-superset.yaml
##########
@@ -0,0 +1,32 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+{{- if .Values.extraConfigs }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ template "superset.fullname" . }}-extra-config
+ labels:
+ app: {{ template "superset.name" . }}
+ chart: {{ template "superset.chart" . }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+data:
+{{- range $path, $config := .Values.extraConfigs }}
+ {{ $path }}: |
+{{ $config | indent 4 -}}
Review comment:
Can you pipe the val to tpl?
So, {{ tpl $config . | indent 4 -}}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] stale[bot] commented on pull request #10771: [Helm] Automate datasource import
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771#issuecomment-751238754
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue `.pinned` to prevent stale bot from closing the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] willbarrett commented on pull request #10771: [Helm] Automate datasource import
Posted by GitBox <gi...@apache.org>.
willbarrett commented on pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771#issuecomment-686644581
Please trim trailing whitespace to fix the precommit hook, after that should be good to merge.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] nytai commented on a change in pull request #10771: [Helm] Automate datasource import
Posted by GitBox <gi...@apache.org>.
nytai commented on a change in pull request #10771:
URL: https://github.com/apache/incubator-superset/pull/10771#discussion_r502222450
##########
File path: helm/superset/values.yaml
##########
@@ -238,4 +253,4 @@ nodeSelector: {}
tolerations: []
-affinity: {}
+affinity: {}
Review comment:
```suggestion
affinity: {}
```
##########
File path: helm/superset/templates/configmap-superset.yaml
##########
@@ -0,0 +1,32 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+{{- if .Values.extraConfigs }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ template "superset.fullname" . }}-extra-config
+ labels:
+ app: {{ template "superset.name" . }}
+ chart: {{ template "superset.chart" . }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+data:
+{{- range $path, $config := .Values.extraConfigs }}
+ {{ $path }}: |
+{{ tpl $config . | indent 4 -}}
+{{- end -}}
+{{- end -}}
Review comment:
```suggestion
{{- end -}}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org