You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/14 15:03:50 UTC
[GitHub] [airflow] sgringwe opened a new issue #8300: No way to create a
Dataflow job with Private IP configuration
sgringwe opened a new issue #8300: No way to create a Dataflow job with Private IP configuration
URL: https://github.com/apache/airflow/issues/8300
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
**Description**
It does not appear that Airflow supports https://cloud.google.com/dataflow/docs/guides/specifying-networks#public_ip_parameter in it's GCP Dataflow operators.
Looking at the documentation and at https://github.com/apache/airflow/blob/16d93c9e45e14179c7822fed248743f0c3fd935c/airflow/contrib/hooks/gcp_dataflow_hook.py#L275 it seems that Airflow does not support passing in a private IP configuration for Dataflow jobs. In Terraform, they use IpConfiguration as seen at https://github.com/terraform-providers/terraform-provider-google/blob/master/google/resource_dataflow_job.go#L183. Ideally, Airflow could use the same parameter.
Unfortunately, there does not seem to be any documentation for this parameter that is being used https://cloud.google.com/dataflow/docs/reference/rest/v1b3/RuntimeEnvironment. Potentially this is a bug only in their docs?
**Use case / motivation**
I want to use https://cloud.google.com/dataflow/docs/guides/specifying-networks#public_ip_parameter feature so my VMs do not have public IPs
**Related Issues**
I don't believe so
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #8300: No way to
create a Dataflow job with Private IP configuration
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8300: No way to create a Dataflow job with Private IP configuration
URL: https://github.com/apache/airflow/issues/8300#issuecomment-613496367
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8300: No way to create a
Dataflow job with Private IP configuration
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8300: No way to create a Dataflow job with Private IP configuration
URL: https://github.com/apache/airflow/issues/8300#issuecomment-613499271
Hi.
It looks like a real bug. In the near future, I will try to address this problem.
Thanks for the report.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services