You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Dominik Riemer <ri...@apache.org> on 2021/03/10 20:38:33 UTC
Service authentication
Hi all,
I have a question concerning the authentication of some of our services (pipeline element containers and adapter containers) that make use of features from the backend/core via REST interfaces.
For instance, the DashboardSink uses an API endpoint from the core to create a new visualization, the notification sink calls the core to save a new notification and some pipeline elements request uploaded files from the core. Until now, endpoints that are used by internal services are completely unauthenticated, which is obviously a bad choice. But now, as we have initial version of the StreamPipes Client & REST API and the concept of API tokens, we can improve that.
So I think in the future any pipeline element could request any information that is required from the core by solely using the StreamPipes client. The question is, how should a services such as a pipeline element container obtain an API key for API access? Currently, the only way to obtain an API token is by creating one from the StreamPipes UI manually (profile -> API keys).
I see the following options:
a. The API key is created manually and provided as an environment variable to the pipeline element container – drawback is that the installation process would be much more complicated as users would need to start the core, go to the UI, configure API keys, add them to the installation script (e.g., the docker-compose file) and then start all pipeline element containers, but gives users most control over the process.
b. We add some concept that allows a pipeline element container to request API access from StreamPipes – users would see some message on the home screen in the UI that allows them to permit API access for a specific pipeline element. The open question in this case is how the pipeline element container stores the API key permanently, as the core does only store the hashed key. For instance, the pipeline element container could store the plain API key in Consul, but I guess that’s bad security practice?
Maybe there are also more/better options, but I’m no authentication expert, so if there is anyone with a good idea on how to approach this, this would be great 😉
Dominik
Re: Service authentication
Posted by Philipp Zehnder <ze...@apache.org>.
Hi,
I agree, we need a solution for that.
Both of your approaches are feasible.
I was thinking about an option c) which is similar to a).
So the idea is to provide the API key via the docker-compose file, but without generating it in the UI first.
Therefore, we would need a way in the backend to provide new tokens via environment variables on startup.
With this approach we could generate those tokens with the CLI or provide some initial ones in the docker compose.
This would ease the installation because nothing has to be configured manually for the single host deployment.
Philipp
> On 10. Mar 2021, at 22:41, Patrick Wiener <wi...@apache.org> wrote:
>
> Hi,
>
> good points and a relevant topic.
>
> Do you still consider having an unauthenticated way in case getting these API keys
> would end up involving several additional steps to the installation process?
>
> I see both options as feasible - either create/manage keys via the UI or have a mechanism
> internally that ultimately pops a notification in the UI asking the user for permission but I
> would not store them in consul and rather keep them locally if needed.
>
> Maybe two questions:
>
> - do you consider one token per service?
> - can tokens expire?
>
> Patrick
>
>
>
>> Am 10.03.2021 um 21:38 schrieb Dominik Riemer <ri...@apache.org>:
>>
>> Hi all,
>>
>>
>>
>> I have a question concerning the authentication of some of our services (pipeline element containers and adapter containers) that make use of features from the backend/core via REST interfaces.
>>
>> For instance, the DashboardSink uses an API endpoint from the core to create a new visualization, the notification sink calls the core to save a new notification and some pipeline elements request uploaded files from the core. Until now, endpoints that are used by internal services are completely unauthenticated, which is obviously a bad choice. But now, as we have initial version of the StreamPipes Client & REST API and the concept of API tokens, we can improve that.
>>
>>
>>
>> So I think in the future any pipeline element could request any information that is required from the core by solely using the StreamPipes client. The question is, how should a services such as a pipeline element container obtain an API key for API access? Currently, the only way to obtain an API token is by creating one from the StreamPipes UI manually (profile -> API keys).
>>
>>
>>
>> I see the following options:
>>
>> a. The API key is created manually and provided as an environment variable to the pipeline element container – drawback is that the installation process would be much more complicated as users would need to start the core, go to the UI, configure API keys, add them to the installation script (e.g., the docker-compose file) and then start all pipeline element containers, but gives users most control over the process.
>> b. We add some concept that allows a pipeline element container to request API access from StreamPipes – users would see some message on the home screen in the UI that allows them to permit API access for a specific pipeline element. The open question in this case is how the pipeline element container stores the API key permanently, as the core does only store the hashed key. For instance, the pipeline element container could store the plain API key in Consul, but I guess that’s bad security practice?
>>
>>
>>
>> Maybe there are also more/better options, but I’m no authentication expert, so if there is anyone with a good idea on how to approach this, this would be great 😉
>>
>>
>>
>> Dominik
>>
>>
>>
>>
>>
>
Re: Service authentication
Posted by Patrick Wiener <wi...@apache.org>.
Hi,
good points and a relevant topic.
Do you still consider having an unauthenticated way in case getting these API keys
would end up involving several additional steps to the installation process?
I see both options as feasible - either create/manage keys via the UI or have a mechanism
internally that ultimately pops a notification in the UI asking the user for permission but I
would not store them in consul and rather keep them locally if needed.
Maybe two questions:
- do you consider one token per service?
- can tokens expire?
Patrick
> Am 10.03.2021 um 21:38 schrieb Dominik Riemer <ri...@apache.org>:
>
> Hi all,
>
>
>
> I have a question concerning the authentication of some of our services (pipeline element containers and adapter containers) that make use of features from the backend/core via REST interfaces.
>
> For instance, the DashboardSink uses an API endpoint from the core to create a new visualization, the notification sink calls the core to save a new notification and some pipeline elements request uploaded files from the core. Until now, endpoints that are used by internal services are completely unauthenticated, which is obviously a bad choice. But now, as we have initial version of the StreamPipes Client & REST API and the concept of API tokens, we can improve that.
>
>
>
> So I think in the future any pipeline element could request any information that is required from the core by solely using the StreamPipes client. The question is, how should a services such as a pipeline element container obtain an API key for API access? Currently, the only way to obtain an API token is by creating one from the StreamPipes UI manually (profile -> API keys).
>
>
>
> I see the following options:
>
> a. The API key is created manually and provided as an environment variable to the pipeline element container – drawback is that the installation process would be much more complicated as users would need to start the core, go to the UI, configure API keys, add them to the installation script (e.g., the docker-compose file) and then start all pipeline element containers, but gives users most control over the process.
> b. We add some concept that allows a pipeline element container to request API access from StreamPipes – users would see some message on the home screen in the UI that allows them to permit API access for a specific pipeline element. The open question in this case is how the pipeline element container stores the API key permanently, as the core does only store the hashed key. For instance, the pipeline element container could store the plain API key in Consul, but I guess that’s bad security practice?
>
>
>
> Maybe there are also more/better options, but I’m no authentication expert, so if there is anyone with a good idea on how to approach this, this would be great 😉
>
>
>
> Dominik
>
>
>
>
>