[I] [Meta-EPIC] Organizing DataFusion's Growth [datafusion]

["alamb (via GitHub)" <gi...@apache.org>]

alamb opened a new issue, #21036:
URL: https://github.com/apache/datafusion/issues/21036

   I feel that DataFusion's community (users, contributors, stakeholders) has grown to a point where we may need to devote some more time ("resources") to more explicit organization to support that growth. 
   
   There are several factors that are driving growth
   - More users!
   - More systems "in critical production" in the sense that even small changes in behavior can be perceived as regressions for existing users
   -  More contributors and parallel work, which increases the need for explicit roadmap and coordination.
   
   This is my attempt to gather a meta summary of challenges related to growth in one place.
   
   ## Release Challenges
   
     - [ ] https://github.com/apache/datafusion/issues/21034
     - [ ] https://github.com/apache/datafusion/issues/5269
     - [ ] https://github.com/apache/datafusion/discussions/20230
   
   
   ## Communication channels and “where should this conversation happen?” are a source of friction.
   
   Possibly related to above,  As @neilconway recently told me, the current DataFusion communication system seems a bit "everything everywhere all the time" in the sense that there is discord, github discussions, github issues, github pull requests, slack the website, and the weekly sync call and it is not always clear how to make sure you see relevant conversations
   
   - [ ] https://github.com/apache/datafusion/issues/14389
   - [ ] https://github.com/apache/datafusion/discussions/2861
   - [ ] https://github.com/apache/datafusion/discussions/7320
   - [ ] https://github.com/apache/datafusion/issues/14389
   
   ## Not clear how to get consensus for large changes
   
   
   I think in general it is not clear 
   1. What "large changes" are currently being considered
   2. What criteria is used to decide what constitutes a large change
   3. Who "decides" if we will make a large change
   
   This is definitely related to roadmap discussions as well
   
   - [ ] https://github.com/apache/datafusion/discussions/15365
   - [ ] https://github.com/apache/datafusion/discussions/21017
   - [ ] https://github.com/apache/datafusion/issues/18494
   
   
   ## Hard to know how to help for new contributors
   
   I don't always think it is easy for people to find ways to help. For example, if I have some time to review PRs, which one would be the most helpful?
   
   (TODO file a ticket about this)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org