You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/19 15:04:35 UTC
[GitHub] [airflow-ci-infra] ashb opened a new pull request #2: Lambda function to scale ASG based on Github webhooks
ashb opened a new pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2
This is very-rough and ready right now (it has no tests, nor instructions for deployin/updating it/how it works) - I'll add those when I have time over the next few days/week
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-763183558
https://040code.github.io/2020/05/25/scaling-selfhosted-action-runners - it roughly follows that, but we had to make some changes due to ASF github permissions (we can't have admin tokens that can create/destroy runners)
Very roughly:
This Lambda is behind an AWS API Gateway.
There is a GitHub app (had to be created in my personal account) configured to get "check" callbacks sent to the API Gateway.
The GitHub app is installed in the `apache/airflow` (and currently also `apache/airflow-site`) repos
(The scale in is handled by a custom metric sent to CloudWatch.)
PR for automation of the other parts is coming - I'm focusing on getting it working mostly, automation of all the AWS and set up of CI workflows for this repo to follow in the future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-764025725
I have been deploying this with `chalice deploy --profile airflow --stage prod`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-781224993
This still needs way more tests, but that can be worked on over time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-762901219
/cc @hmike96
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] potiuk commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-763065429
I see how the lambda works, but I woiuld love to know who is hitting the endpoint ? Is the blog I've sent some time ago describing the architecture? I do not need fully fledged description just few words on how it work. We can capture it later of course in a more peristent way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb merged pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-763211592
I'm not sure the logic about when to create a new instance works.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-ci-infra] ashb commented on pull request #2: Lambda function to scale ASG based on Github webhooks
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #2:
URL: https://github.com/apache/airflow-ci-infra/pull/2#issuecomment-764025725
I have been deploying this with `chalice deploy --profile airflow --stage prod`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org