You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ephraimbuddy (via GitHub)" <gi...@apache.org> on 2023/08/28 20:36:58 UTC
[GitHub] [airflow] ephraimbuddy opened a new pull request, #33851: Truncate Wasb storage account name if it's more than 24 characters
ephraimbuddy opened a new pull request, #33851:
URL: https://github.com/apache/airflow/pull/33851
Storage account names must be between 3 and 24 characters in length but for some reasons that I can't explain, we saw a situation where the storage name is more than 24 characters and had to be truncated before it could work. Maybe it was possible in the past to have more than 24 characters or it could come from cluster but whichever way, the solution that worked was truncating the account name to 24 characters.
https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview#storage-account-name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #33851: Truncate Wasb storage account name if it's more than 24 characters
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on code in PR #33851:
URL: https://github.com/apache/airflow/pull/33851#discussion_r1307977768
##########
airflow/providers/microsoft/azure/hooks/wasb.py:
##########
@@ -163,10 +163,18 @@ def get_conn(self) -> BlobServiceClient:
account_url = conn.host if conn.host else f"https://{conn.login}.blob.core.windows.net/"
parsed_url = urlparse(account_url)
- if not parsed_url.netloc and "." not in parsed_url.path:
- # if there's no netloc and no dots in the path, then user only
- # provided the Active Directory ID, not the full URL or DNS name
- account_url = f"https://{conn.login}.blob.core.windows.net/"
+ if not parsed_url.netloc:
+ if "." not in parsed_url.path:
+ # if there's no netloc and no dots in the path, then user only
+ # provided the Active Directory ID, not the full URL or DNS name
+ account_url = f"https://{conn.login}.blob.core.windows.net/"
+ else:
+ # if there's no netloc but there are dots in the path, then user
+ # provided the DNS name without the https:// prefix.
+ # Azure storage account name can only be 3 to 24 characters in length
+ # https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview#storage-account-name
+ acc_name = parsed_url.path.split(".")[0][:24]
+ account_url = f"https://{acc_name}." + ".".join(parsed_url.path.split(".")[1:])
Review Comment:
This is a little more clear to me, but feel free to choose either approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #33851: Truncate Wasb storage account name if it's more than 24 characters
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on code in PR #33851:
URL: https://github.com/apache/airflow/pull/33851#discussion_r1307976832
##########
airflow/providers/microsoft/azure/hooks/wasb.py:
##########
@@ -163,10 +163,18 @@ def get_conn(self) -> BlobServiceClient:
account_url = conn.host if conn.host else f"https://{conn.login}.blob.core.windows.net/"
parsed_url = urlparse(account_url)
- if not parsed_url.netloc and "." not in parsed_url.path:
- # if there's no netloc and no dots in the path, then user only
- # provided the Active Directory ID, not the full URL or DNS name
- account_url = f"https://{conn.login}.blob.core.windows.net/"
+ if not parsed_url.netloc:
+ if "." not in parsed_url.path:
+ # if there's no netloc and no dots in the path, then user only
+ # provided the Active Directory ID, not the full URL or DNS name
+ account_url = f"https://{conn.login}.blob.core.windows.net/"
+ else:
+ # if there's no netloc but there are dots in the path, then user
+ # provided the DNS name without the https:// prefix.
+ # Azure storage account name can only be 3 to 24 characters in length
+ # https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview#storage-account-name
+ acc_name = parsed_url.path.split(".")[0][:24]
+ account_url = f"https://{acc_name}." + ".".join(parsed_url.path.split(".")[1:])
Review Comment:
```suggestion
acc_name = account_url.split(".")[0][:24]
account_url = f"https://{acc_name}." + ".".join(account_url.split(".")[1:])
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #33851: Truncate Wasb storage account name if it's more than 24 characters
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on code in PR #33851:
URL: https://github.com/apache/airflow/pull/33851#discussion_r1307990895
##########
airflow/providers/microsoft/azure/hooks/wasb.py:
##########
@@ -163,10 +163,18 @@ def get_conn(self) -> BlobServiceClient:
account_url = conn.host if conn.host else f"https://{conn.login}.blob.core.windows.net/"
parsed_url = urlparse(account_url)
- if not parsed_url.netloc and "." not in parsed_url.path:
- # if there's no netloc and no dots in the path, then user only
- # provided the Active Directory ID, not the full URL or DNS name
- account_url = f"https://{conn.login}.blob.core.windows.net/"
+ if not parsed_url.netloc:
+ if "." not in parsed_url.path:
+ # if there's no netloc and no dots in the path, then user only
+ # provided the Active Directory ID, not the full URL or DNS name
+ account_url = f"https://{conn.login}.blob.core.windows.net/"
+ else:
+ # if there's no netloc but there are dots in the path, then user
+ # provided the DNS name without the https:// prefix.
+ # Azure storage account name can only be 3 to 24 characters in length
+ # https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview#storage-account-name
+ acc_name = parsed_url.path.split(".")[0][:24]
+ account_url = f"https://{acc_name}." + ".".join(parsed_url.path.split(".")[1:])
Review Comment:
Makes sense!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal merged pull request #33851: Truncate Wasb storage account name if it's more than 24 characters
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #33851:
URL: https://github.com/apache/airflow/pull/33851
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org