[GitHub] [airflow] potiuk commented on pull request #22253: Add SparkKubernetesOperator crd implementation

[GitBox <gi...@apache.org>]

potiuk commented on PR #22253:
URL: https://github.com/apache/airflow/pull/22253#issuecomment-1096643179

   @dstandish  - this is yet another example and contribution to the discussion we have here: https://lists.apache.org/thread/m40ljkgcqsnvbxz6vlgnwflygvr6vznz
   
   The change here spans acrross both "airflow/kubernetes" (part of core) and "cncf.kubernetes" provider. And by a quick look I have completely no idea how those two interact.
   
   Will the new provider work without upgrading Airflow after the change ? Will it require the change to be released? Will the core continue running with old `cncf.kubernetes` ? 
   
   No idea.
   
   This is far more important issue to solve than just deprecating config parameters. And the root cause of it is that unlike in all other providers we have functionality split betweeen "core" and "provider" and one uses the other and there is implicit dependency.
   
   There are two options we could approach to solve this problem 
   
   1) decouple `cncf.kubernetes` from core completely and remove all the implicit binding between the two
   2) move out all stuff to cncf.kubernetes to "core" and deprecate the provider
   
   The first one might be possible - but  we need to know if it really is possible, and whether it brings some backwards, compatibility issues. But ultimately - your proposal about moving core kubernetes settings is in-line with this change.
   
   The second case is "easier" (requires no real changes to the core except moving the provider code to the core) but it does not nicely fit our "modular" approach of core and separate providers (Kubernetes is essentially an optional  dependency). 
   
   I think - in order to decide on deprecating the kubernetes settings from core - we need to be sure that 1) is possible - and for that we need to have a plan how to achieve that.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org