You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "mbarugelCA (via GitHub)" <gi...@apache.org> on 2025/02/23 22:56:11 UTC
[I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
mbarugelCA opened a new issue, #47003:
URL: https://github.com/apache/airflow/issues/47003
### Apache Airflow Provider(s)
snowflake
### Versions of Apache Airflow Providers
apache-airflow-providers-snowflake==6.0.0
### Apache Airflow version
2.10.5
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Docker-Compose
### Deployment details
_No response_
### What happened
The "Private key (text)" field for Snowflake connections should be a multi-line text input box, but is a password field instead:
<img width="759" alt="Image" src="https://github.com/user-attachments/assets/051fdd90-b0ab-4980-a7d1-14f723356a4c" />
This seems to be a regression introduced in [this](https://github.com/apache/airflow/commit/8a5eb22df1ac94a3792f8eaf0760b4c0f8f316f2) commit. Pasting a private key in PEM format fails because the line breaks do not paste well in the box as is.
This used to work well in version `5.7.0` of the provider.
### What you think should happen instead
_No response_
### How to reproduce
Install Apache Airflow with `apache-airflow[snowflake]==2.10.5`, start the webserver and go to Connections. Open the `snowflake_default` connection to see the incorrect input type for private key content.
### Anything else
This prevents setting up a Snowflake connection with private key authentication.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2708651674
> Same for us
As a workaround, does it work if you just remove the new lines and pass it as a single line?
You can probably do the below and test it
```
with open("privatekey.pem", "r") as pem_file:
single_line_key = "".join(pem_file.read().splitlines())
print(single_line_key)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2726301325
Ok, so here's the root cause:
After converting the field to a `BS3PasswordFieldWidget` , when we paste PEM key with new lines, this field automatically removes new lines.
For example:
```
key_start
text
key_end
```
is now converted to a single line `key_starttextkey_end` .
And, when format the PEM key to replace new lines with "\n" , it is actually saved as "\\n".
For example:
`key_start\ntext\nkey_end` to become `key_start\\ntext\\nkey_end`
Here's what I'm thinking to do:
1. Update docs to inform users to pass PEM key by replacing new lines with `\n`'s
2. In the code, I'll just replace `\\n` with `\n`
CC: @potiuk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #47003: Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections
URL: https://github.com/apache/airflow/issues/47003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2708840555
> [@rawwar](https://github.com/rawwar) I haven't tried this but I don't think it would work, because the `-----BEGIN PRIVATE KEY-----` should be in its own line afaik.
I don't have a Snowflake account handy; if you can, please try it and confirm here. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2728883625
Can we maybe extend the BS3PasswordFieldWidget and have our own widget that will behave the same but **NOT** replace `\n` - I think that would be the cleanest way to handle it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "potypeformer (via GitHub)" <gi...@apache.org>.
potypeformer commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2765489367
Base64 sounds logical, but wouldn’t that create a breaking change for users upgrading from an older version of the operator?
They might need to resubmit their keys in the connections to prevent pipeline failures.
That said, versions newer than 5.7 are already breaking pipelines anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "gagandeepnum (via GitHub)" <gi...@apache.org>.
gagandeepnum commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2679431080
Same for us
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2726282848
I'm able to replicate the issue. I think I have an idea of how to fix this. Also, I need to update documentation to make it clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2708888897
Can you join them using a new line character("/n") instead of "".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "mbarugelCA (via GitHub)" <gi...@apache.org>.
mbarugelCA commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2708839305
@rawwar I haven't tried this but I don't think it would work, because the `-----BEGIN PRIVATE KEY-----` should be in its own line afaik.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2714413724
> > Can you join them using a new line character("/n") instead of "".
>
> I get the same error if joining with `\n`. Also tried `\r\n` but same error again.
I will look into this tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2761735400
@potiuk , So, Now, I am extending BS3PasswordFieldWidget. But, where do I keep this class?
Since, it can be reused across providers, I don't feel like including it in snowflake provider code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2730120862
> Can we maybe extend the BS3PasswordFieldWidget and have our own widget that will behave the same but **NOT** replace `\n` - I think that would be the cleanest way to handle it.
Thanks. I'll work on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2725335482
Just got hold of a snowflake account. I'm actively working on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2708652975
it is intentionally updated to be a sensitive field to make it hidden on the UI.
But I think I should also update docs to add some hints to reformat the private key to be in a single line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "mbarugelCA (via GitHub)" <gi...@apache.org>.
mbarugelCA commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2708846656
> > [@rawwar](https://github.com/rawwar) I haven't tried this but I don't think it would work, because the `-----BEGIN PRIVATE KEY-----` should be in its own line afaik.
>
> I don't have a Snowflake account handy; if you can, please try it and confirm here. Thanks!
It returns this error:
```Traceback (most recent call last):
File "/usr/local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/airflow/__main__.py", line 62, in main
args.func(args)
File "/usr/local/lib/python3.10/site-packages/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/cli.py", line 116, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/session.py", line 97, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py", line 680, in task_test
ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True, raise_on_defer=True)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/session.py", line 97, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3239, in run
self._run_raw_task(
File "/usr/local/lib/python3.10/site-packages/airflow/utils/session.py", line 94, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3006, in _run_raw_task
return _run_raw_task(
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 274, in _run_raw_task
TaskInstance._execute_task_with_callbacks(
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3161, in _execute_task_with_callbacks
result = self._execute_task(context, task_orig)
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3185, in _execute_task
return _execute_task(self, context, task_orig)
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 768, in _execute_task
result = _execute_callable(context=context, **execute_callable_kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 734, in _execute_callable
return ExecutionCallableRunner(
File "/usr/local/lib/python3.10/site-packages/airflow/utils/operator_helpers.py", line 252, in run
return self.func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 424, in wrapper
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/decorators/base.py", line 266, in execute
return_value = super().execute(context)
File "/usr/local/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 424, in wrapper
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/operators/python.py", line 238, in execute
return_value = self.execute_callable()
File "/usr/local/lib/python3.10/site-packages/airflow/operators/python.py", line 256, in execute_callable
return runner.run(*self.op_args, **self.op_kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/operator_helpers.py", line 252, in run
return self.func(*args, **kwargs)
...
File "/usr/local/lib/python3.10/site-packages/airflow/providers/common/sql/hooks/sql.py", line 336, in get_first
return self.run(sql=sql, parameters=parameters, handler=fetch_one_handler)
File "/usr/local/lib/python3.10/site-packages/airflow/providers/snowflake/hooks/snowflake.py", line 459, in run
with closing(self.get_conn()) as conn:
File "/usr/local/lib/python3.10/site-packages/airflow/providers/snowflake/hooks/snowflake.py", line 329, in get_conn
conn_config = self._get_conn_params
File "/usr/local/lib/python3.10/functools.py", line 981, in __get__
val = self.func(instance)
File "/usr/local/lib/python3.10/site-packages/airflow/providers/snowflake/hooks/snowflake.py", line 272, in _get_conn_params
p_key = serialization.load_pem_private_key(
File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 494, in _handle_key_loading_error
raise ValueError(
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [<OpenSSLError(code=503841036, lib=60, reason=524556, reason_text=unsupported)>])```
This happens even before an actual connection to Snowflake is attempted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Snowflake provider: The `private_key_content` field is no longer a multi-line text input in Connections [airflow]
Posted by "mbarugelCA (via GitHub)" <gi...@apache.org>.
mbarugelCA commented on issue #47003:
URL: https://github.com/apache/airflow/issues/47003#issuecomment-2711272689
> Can you join them using a new line character("/n") instead of "".
I get the same error if joining with `\n`. Also tried `\r\n` but same error again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org