You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2018/09/02 18:03:03 UTC
[jira] [Commented] (AIRFLOW-2551) Encode binary data with base64
standard rather than base64 url
[ https://issues.apache.org/jira/browse/AIRFLOW-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601459#comment-16601459 ]
Apache Spark commented on AIRFLOW-2551:
---------------------------------------
User 'whynick1' has created a pull request for this issue:
https://github.com/apache/incubator-airflow/pull/3449
> Encode binary data with base64 standard rather than base64 url
> --------------------------------------------------------------
>
> Key: AIRFLOW-2551
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2551
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Hongyi Wang
> Assignee: Hongyi Wang
> Priority: Major
> Fix For: 2.0.0
>
>
> When we try to load mysql data into Google BigQuery (mysql -> gcs -> bq), there is a binary filed (uuid), which will cause BigQuery job failed, with message "_Could not decode base64 string to bytes. Field: uuid; Value: _gJbkmC1QTiS-zZ46uiHWg==_"
> This was caused by "_col_val = base64.urlsafe_b64encode(col_val)_" in mysql_to_gcs_operator.
> We should use "_standard_b64encode()_" instead.
> {quote}{{Base64url encoding is basically base64 encoding except they use non-reserved URL characters (e.g. – is used instead of + and _ is used instead of /) __ }}
> {quote}
> Related to [AIRFLOW-2169]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)