You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Xinbin Huang <bi...@gmail.com> on 2020/03/21 19:29:39 UTC

[DISCUSS] Tips & Tricks for navigating codebase

Hi all,

I have been contributing to the project on and off for around 3 months now.

One common difficulty I found is to navigate the codebase efficiently. A
lot of time we need to make changes to several files (5+) while also
keeping other files open for referencing and I will easily lose track of
the big file structure in the project. I would say approximately 10% ~ 15%
of my time is spent on finding the right files to open, and I want to
improve this. Does anyone feel the same or have some tips & tricks that can
make browsing the codebase more efficiently? It can be any extensions,
plugins, bash magic etc.. I am currently using VS Code, and I use `grep -r`
quite a lot.

Regards,
Bin

Re: [DISCUSS] Tips & Tricks for navigating codebase

Posted by Jarek Potiuk <Ja...@polidea.com>.
The structure is indeed big and there are some cyclic dependencies and
especially classes/modules that have multiple responsibilities.

First of - all let me say what a lot of committers and contributors do
recently to fight with that -  we are unentangling that slowly so
hopefully it will be easier to understand the code.  We recently
completed the move to move all the "providers" to the new "providers"
package" (separating it out from the core). We are also adding type
hinting wherever we can - that makes it easy to understand which
objects you are acting on and what they do. And I encourage you to do
the same whenever you find a piece of code that screams for
refactoring/pylinting/type hinting - any help here is great.

Also as of 2.0 we also agreed we will use and enforce one code
formatter (black most likely) that will likely make the code
consistent and more readable.

Back to your questions:

I do not use VSCode that often (I just started to learn all the
helpful features) but I know it has plugins for nearly anything and
lots of people use it.
I personally use IntelliJ/PyCharm and its indexing/code navigation
features are rather good. It has a number of navigation options
between classes/packages. The trick is to have a local virtualenv set
up with all the dependencies installed and configure it in IntelliJ as
python interpreter. Breeze has the initialize-local-virtualenv command
that works well for most Linux/macOS (it will provide some hints as to
what are the prerequisites). The Pylint/MyPy plugins provide good
hints and prevent from many mistakes. Shellcheck is fantastic for bash
scripts.

J.


On Sat, Mar 21, 2020 at 8:30 PM Xinbin Huang <bi...@gmail.com> wrote:
>
> Hi all,
>
> I have been contributing to the project on and off for around 3 months now.
>
> One common difficulty I found is to navigate the codebase efficiently. A
> lot of time we need to make changes to several files (5+) while also
> keeping other files open for referencing and I will easily lose track of
> the big file structure in the project. I would say approximately 10% ~ 15%
> of my time is spent on finding the right files to open, and I want to
> improve this. Does anyone feel the same or have some tips & tricks that can
> make browsing the codebase more efficiently? It can be any extensions,
> plugins, bash magic etc.. I am currently using VS Code, and I use `grep -r`
> quite a lot.
>
> Regards,
> Bin



-- 

Jarek Potiuk
Polidea | Principal Software Engineer

M: +48 660 796 129