You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/02/08 09:10:13 UTC
[GitHub] [superset] 1c7 opened a new issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
1c7 opened a new issue #18616:
URL: https://github.com/apache/superset/issues/18616
### Motivation
I am having difficult time figure out how to properly deploy Superset to a production enviroment.
I am planing to use AWS ECS.
### Proposed Change
1. I am gonna figure out how to deploy Superset in next couple days
2. I am gonna send a Pull Request with a new document title "Deploy" in `docs/` folder (Is this the correct location?)
3. main contributer gonna review my writing and write comment about it
4. everything seem fine, the PR got merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 commented on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 commented on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032382576
## Current progress:
- [x] write a Dockerfile that base on Dockerhub `apache/superset`
Dockerfile
```
FROM apache/superset
# ..etc
```
- [x] Install ClickHouse driver and use `superset_config.py` to config superset
```
FROM apache/superset
USER root
RUN pip install clickhouse-driver==0.2.0 && pip install clickhouse-sqlalchemy==0.1.6
USER superset
COPY superset_config.py /app/pythonpath/superset_config.py
```
- [x] Figure out how to set PostgreSQL as Superset metadata database (instead of using default SQlite)
- [ ] Figure out what other component I need to config/setup
- [ ] Caching with `CACHE_CONFIG` and `DATA_CACHE_CONFIG`
- [ ] Celery for long running queries
- [ ] Is there other component I am missing ?
- [ ] Deploy Superset on AWS ECS and make sure it work
- [ ] Write a document about my experiences & how to deploy
- [ ] Figure out where should I put the document inside Superset Github repository
- [ ] Send a Pull Request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 edited a comment on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 edited a comment on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032382576
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 edited a comment on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 edited a comment on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032382576
# My Progress:
- [x] write a Dockerfile that base on Dockerhub `apache/superset`
Dockerfile
```
FROM apache/superset
# ..etc
```
- [x] Install ClickHouse driver and use `superset_config.py` to config superset
```
FROM apache/superset
USER root
RUN pip install clickhouse-driver==0.2.0 && pip install clickhouse-sqlalchemy==0.1.6
USER superset
COPY superset_config.py /app/pythonpath/superset_config.py
```
- [x] Figure out how to set PostgreSQL as Superset metadata database (instead of using default SQlite)
- [x] Figure out what other component I need to config/setup
- [ ] Caching with `CACHE_CONFIG` and `DATA_CACHE_CONFIG`
- [ ] Celery for long running queries
- [ ] Is there other component I am missing ?
- [ ] Deploy Superset on AWS ECS and make sure it work
- [ ] Write a document about my experiences & how to deploy
- [ ] Figure out where should I put the document inside Superset Github repository
- [ ] Send a Pull Request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 edited a comment on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 edited a comment on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032382576
# My Progress:
- [x] write a Dockerfile that base on Dockerhub `apache/superset`
Dockerfile
```
FROM apache/superset
# ..etc
```
- [x] Install ClickHouse driver and use `superset_config.py` to config superset
```
FROM apache/superset
USER root
RUN pip install clickhouse-driver==0.2.0 && pip install clickhouse-sqlalchemy==0.1.6
USER superset
COPY superset_config.py /app/pythonpath/superset_config.py
```
- [x] Figure out how to set PostgreSQL as Superset metadata database (instead of using default SQlite)
- [ ] Figure out what other component I need to config/setup
- [ ] Caching with `CACHE_CONFIG` and `DATA_CACHE_CONFIG`
- [ ] Celery for long running queries
- [ ] Is there other component I am missing ?
- [ ] Deploy Superset on AWS ECS and make sure it work
- [ ] Write a document about my experiences & how to deploy
- [ ] Figure out where should I put the document inside Superset Github repository
- [ ] Send a Pull Request
## Conclusion
This could take me a couple of weeks, I would update my progress regularly here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 edited a comment on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 edited a comment on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032382576
# My Progress:
- [x] write a Dockerfile that base on Dockerhub `apache/superset`
Dockerfile
```
FROM apache/superset
# ..etc
```
- [x] Install ClickHouse driver and use `superset_config.py` to config superset
```
FROM apache/superset
USER root
RUN pip install clickhouse-driver==0.2.0 && pip install clickhouse-sqlalchemy==0.1.6
USER superset
COPY superset_config.py /app/pythonpath/superset_config.py
```
- [x] Figure out how to set PostgreSQL as Superset metadata database (instead of using default SQlite)
- [ ] Figure out what other component I need to config/setup
- [ ] Caching with `CACHE_CONFIG` and `DATA_CACHE_CONFIG`
- [ ] Celery for long running queries
- [ ] Is there other component I am missing ?
- [ ] Deploy Superset on AWS ECS and make sure it works
- [ ] Write a document about my experiences & how to deploy
- [ ] Figure out where should I put the document inside the Superset Github repository
- [ ] Send a Pull Request
## Conclusion
This could take me a couple of weeks, I would update my progress here regularly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 commented on issue #18616: Documentation on how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 commented on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032554407
@srinify Got it, Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 edited a comment on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 edited a comment on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032382576
# My Progress:
- [x] write a Dockerfile that base on Dockerhub `apache/superset`
Dockerfile
```
FROM apache/superset
# ..etc
```
- [x] Install ClickHouse driver and use `superset_config.py` to config superset
```
FROM apache/superset
USER root
RUN pip install clickhouse-driver==0.2.0 && pip install clickhouse-sqlalchemy==0.1.6
USER superset
COPY superset_config.py /app/pythonpath/superset_config.py
```
- [x] Figure out how to set PostgreSQL as Superset metadata database (instead of using default SQlite)
- [ ] Figure out what other component I need to config/setup
- [ ] Caching with `CACHE_CONFIG` and `DATA_CACHE_CONFIG`
- [ ] Celery for long running queries
- [ ] Is there other component I am missing ?
- [ ] Deploy Superset on AWS ECS and make sure it works
- [ ] Write a document about my experiences & how to deploy
- [ ] Figure out where should I put the document inside the Superset Github repository
- [ ] Send a Pull Request
# Conclusion
This could take me a couple of weeks, I would update my progress here regularly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 commented on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 commented on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032511473
@ad-m That seems helpful, Let me check it out. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] ad-m commented on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
ad-m commented on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032388142
In the case of Celery, you have to consider transport.
You should be careful about volumes and analyze: https://github.com/apache/superset/blob/master/docker-compose-non-dev.yml
I suggest creating a draft PR, so feedback can be provided ongoing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinify commented on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
srinify commented on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032551008
Hi there @1c7 normally SIP's are for large, potentially controversial changes that require discussion & consensus building. But in this case, I would say there isn't much controversy :)
If you'd like to contribute this documentation then you can open a PR and the community can review it!
Also check out these other tutorials:
- https://towardsdatascience.com/running-apache-superset-at-scale-1539e3945093
- https://aws.amazon.com/quickstart/architecture/apache-superset/
- https://www.obytes.com/blog/apache-superset-on-aws-ecs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] ad-m commented on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
ad-m commented on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032389271
Are you aware about ECS task definition of test env ( https://github.com/apache/superset/blob/master/.github/workflows/ecs-task-definition.json )?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] 1c7 edited a comment on issue #18616: [SIP-81] Proposal for writing a detail document about how to properly deploy Superset for production environment
Posted by GitBox <gi...@apache.org>.
1c7 edited a comment on issue #18616:
URL: https://github.com/apache/superset/issues/18616#issuecomment-1032511473
@ad-m That seems helpful(ECS Task Definition), Let me check it out. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org