You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Yuen-Kuei Hsueh (Jira)" <ji...@apache.org> on 2019/12/27 16:10:00 UTC
[jira] [Updated] (AIRFLOW-6372) Align Azure Blob Storage remote
logging URI scheme with S3 and GCS
[ https://issues.apache.org/jira/browse/AIRFLOW-6372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuen-Kuei Hsueh updated AIRFLOW-6372:
-------------------------------------
Description:
The `remote_base_log_folder` for S3 and GCS are URI Schemes, but Azure Blob Storage is an ugly format `wasb-<whatever you want here>` and [requires additional settings|https://airflow.apache.org/docs/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage].
For Hadoop ecosystem, the [URI Scheme for Azure Blob Storage|https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Accessing_wasb_URLs] was `wasb[s]://<containername>@<accountname>.blob.core.windows.net/<path>`. So I think we can use Hadoop URI Scheme for Azure Blob Storage to align the configuration and omit some parameter, such as `container_name`.
The changes should be also compatible with legacy configuration.
was:
The `remote_base_log_folder` for S3 and GCS are URI Schemes, but Azure Blob Storage is an ugly format `wasb-<whatever you want here>` and [requires additional settings|https://airflow.apache.org/docs/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage]. For Hadoop ecosystem, the [URI Scheme for Azure Blob Storage|https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Accessing_wasb_URLs] was `wasb[s]://<containername>@<accountname>.blob.core.windows.net/<path>`. So I think we can use Hadoop URI Scheme for Azure Blob Storage to align the configuration and omit some parameter, such as `container_name`.
The changes should be also compatible with legacy configuration.
> Align Azure Blob Storage remote logging URI scheme with S3 and GCS
> ------------------------------------------------------------------
>
> Key: AIRFLOW-6372
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6372
> Project: Apache Airflow
> Issue Type: Improvement
> Components: logging
> Affects Versions: 1.10.6
> Reporter: Yuen-Kuei Hsueh
> Assignee: Yuen-Kuei Hsueh
> Priority: Minor
>
> The `remote_base_log_folder` for S3 and GCS are URI Schemes, but Azure Blob Storage is an ugly format `wasb-<whatever you want here>` and [requires additional settings|https://airflow.apache.org/docs/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage].
> For Hadoop ecosystem, the [URI Scheme for Azure Blob Storage|https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Accessing_wasb_URLs] was `wasb[s]://<containername>@<accountname>.blob.core.windows.net/<path>`. So I think we can use Hadoop URI Scheme for Azure Blob Storage to align the configuration and omit some parameter, such as `container_name`.
>
> The changes should be also compatible with legacy configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)