[DISCUSS] Enabling Dependabot security updates for our Javascript dependencies ?

[Jarek Potiuk <ja...@potiuk.com>]

Hello Everyone,

TL;DR; I would like to ask if we want to enable Dependabot to make
automated PRs updating our Javascript dependencies for the UI ?

Context:

We've been discussing in the Apache Software Foundation whether it is
OK from a legal/infrastructure point of view to use Dependabot to
monitor and make Pull Requests automatically to update our
dependencies.

So far the policy of the ASF was that dependabot creating automated
PRs is against the policies.
Generally discussion was whether automated PR which creates branches
directly in Airflow Repo automatically (not in a fork) is OK from the
"contribution" perspective (according to ASF there should always be a
human in the loop of the code contributed).

After a long discussion and arguments
https://issues.apache.org/jira/browse/LEGAL-589 (I took active part
there advocating for a change) the conclusion is that using Dependabot
is OK as long as you have protected branches - which makes it required
for a human reviewer (commiter) to review such branch and merge it to
the "protected" branch..

I think we cannot really use Dependabot for Airflow dependencies (it's
not as sop[histicated as it comes to multiple versions of Python and
constraints mechanism and we would have far too many PRs to handle if
it is about our  ~600 python dependencies. But I think it would be
cool to enable it for our Javascript dependencies for the UI (we are
following a very standard approach there with the usual yarn.lock so
it should be easy to plug dependabot in.

What we can get:

Better supply-chain security in general, but we will get some traffic
from automated PRs sent by the dependabot that we will have to handle,
review, possibly test and approve.

The result of it will be that we will get PRs about updated (and
especially security related) dependencies as quickly as they happen
and we will be able to see all the details of the security updates.
Currently we (maintainers) only see alerts about those
vulnerabilities, but with Dependabot security updates those will
become automated PRs.

Unlike the Python dependencies (which are automatically updated by our
CI)  we update our javascript irregularly in "bulk" - i.e. from time
to time we will refresh the lock file and update to the latest
dependencies. That has an advantage that we can likely test it in
bulk.

However my point of view is that making such updates more frequently
is better because if we update dependency one-by-one, we will not even
have to test it too much - as we will quickly see that the UI is
broken during our regular development and then it will be easier to
pin-point a culprit.

As one of my favourite quotes goes "If an upgrade is painful, simply
do it more often rather than less often - that makes it far less
painful in general".

More info about dependabot:
https://docs.github.com/en/code-security/repository-security-advisories/about-github-security-advisories-for-repositories

Example PR generated by Dependabot:
https://github.com/PolideaInternal/airflow-gepard/pull/356

J.

Re: [DISCUSS] Enabling Dependabot security updates for our Javascript dependencies ?

[Jarek Potiuk <ja...@potiuk.com>]

OK. Discussion finished then ;)

On Tue, Apr 5, 2022 at 4:32 PM Brent Bovenzi
<br...@astronomer.io.invalid> wrote:
>
> Big +1 from me.
>
> Our current "process" is just me seeing an alert and remembering to check `yarn audit`.
> Dependabot will make it far easier to stay disciplined. Most minor and patch version changes should be fairly easy to approve even w/o extensive JS knowledge.
>
> On Tue, Apr 5, 2022 at 4:23 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> Hello Everyone,
>>
>> TL;DR; I would like to ask if we want to enable Dependabot to make
>> automated PRs updating our Javascript dependencies for the UI ?
>>
>> Context:
>>
>> We've been discussing in the Apache Software Foundation whether it is
>> OK from a legal/infrastructure point of view to use Dependabot to
>> monitor and make Pull Requests automatically to update our
>> dependencies.
>>
>> So far the policy of the ASF was that dependabot creating automated
>> PRs is against the policies.
>> Generally discussion was whether automated PR which creates branches
>> directly in Airflow Repo automatically (not in a fork) is OK from the
>> "contribution" perspective (according to ASF there should always be a
>> human in the loop of the code contributed).
>>
>> After a long discussion and arguments
>> https://issues.apache.org/jira/browse/LEGAL-589 (I took active part
>> there advocating for a change) the conclusion is that using Dependabot
>> is OK as long as you have protected branches - which makes it required
>> for a human reviewer (commiter) to review such branch and merge it to
>> the "protected" branch..
>>
>> I think we cannot really use Dependabot for Airflow dependencies (it's
>> not as sop[histicated as it comes to multiple versions of Python and
>> constraints mechanism and we would have far too many PRs to handle if
>> it is about our  ~600 python dependencies. But I think it would be
>> cool to enable it for our Javascript dependencies for the UI (we are
>> following a very standard approach there with the usual yarn.lock so
>> it should be easy to plug dependabot in.
>>
>> What we can get:
>>
>> Better supply-chain security in general, but we will get some traffic
>> from automated PRs sent by the dependabot that we will have to handle,
>> review, possibly test and approve.
>>
>> The result of it will be that we will get PRs about updated (and
>> especially security related) dependencies as quickly as they happen
>> and we will be able to see all the details of the security updates.
>> Currently we (maintainers) only see alerts about those
>> vulnerabilities, but with Dependabot security updates those will
>> become automated PRs.
>>
>> Unlike the Python dependencies (which are automatically updated by our
>> CI)  we update our javascript irregularly in "bulk" - i.e. from time
>> to time we will refresh the lock file and update to the latest
>> dependencies. That has an advantage that we can likely test it in
>> bulk.
>>
>> However my point of view is that making such updates more frequently
>> is better because if we update dependency one-by-one, we will not even
>> have to test it too much - as we will quickly see that the UI is
>> broken during our regular development and then it will be easier to
>> pin-point a culprit.
>>
>> As one of my favourite quotes goes "If an upgrade is painful, simply
>> do it more often rather than less often - that makes it far less
>> painful in general".
>>
>> More info about dependabot:
>> https://docs.github.com/en/code-security/repository-security-advisories/about-github-security-advisories-for-repositories
>>
>> Example PR generated by Dependabot:
>> https://github.com/PolideaInternal/airflow-gepard/pull/356
>>
>> J.

Re: [DISCUSS] Enabling Dependabot security updates for our Javascript dependencies ?

[Brent Bovenzi <br...@astronomer.io.INVALID>]

Big +1 from me.

Our current "process" is just me seeing an alert and remembering to check
`yarn audit`.
Dependabot will make it far easier to stay disciplined. Most minor and
patch version changes should be fairly easy to approve even w/o extensive
JS knowledge.

On Tue, Apr 5, 2022 at 4:23 AM Jarek Potiuk <ja...@potiuk.com> wrote:

> Hello Everyone,
>
> TL;DR; I would like to ask if we want to enable Dependabot to make
> automated PRs updating our Javascript dependencies for the UI ?
>
> Context:
>
> We've been discussing in the Apache Software Foundation whether it is
> OK from a legal/infrastructure point of view to use Dependabot to
> monitor and make Pull Requests automatically to update our
> dependencies.
>
> So far the policy of the ASF was that dependabot creating automated
> PRs is against the policies.
> Generally discussion was whether automated PR which creates branches
> directly in Airflow Repo automatically (not in a fork) is OK from the
> "contribution" perspective (according to ASF there should always be a
> human in the loop of the code contributed).
>
> After a long discussion and arguments
> https://issues.apache.org/jira/browse/LEGAL-589 (I took active part
> there advocating for a change) the conclusion is that using Dependabot
> is OK as long as you have protected branches - which makes it required
> for a human reviewer (commiter) to review such branch and merge it to
> the "protected" branch..
>
> I think we cannot really use Dependabot for Airflow dependencies (it's
> not as sop[histicated as it comes to multiple versions of Python and
> constraints mechanism and we would have far too many PRs to handle if
> it is about our  ~600 python dependencies. But I think it would be
> cool to enable it for our Javascript dependencies for the UI (we are
> following a very standard approach there with the usual yarn.lock so
> it should be easy to plug dependabot in.
>
> What we can get:
>
> Better supply-chain security in general, but we will get some traffic
> from automated PRs sent by the dependabot that we will have to handle,
> review, possibly test and approve.
>
> The result of it will be that we will get PRs about updated (and
> especially security related) dependencies as quickly as they happen
> and we will be able to see all the details of the security updates.
> Currently we (maintainers) only see alerts about those
> vulnerabilities, but with Dependabot security updates those will
> become automated PRs.
>
> Unlike the Python dependencies (which are automatically updated by our
> CI)  we update our javascript irregularly in "bulk" - i.e. from time
> to time we will refresh the lock file and update to the latest
> dependencies. That has an advantage that we can likely test it in
> bulk.
>
> However my point of view is that making such updates more frequently
> is better because if we update dependency one-by-one, we will not even
> have to test it too much - as we will quickly see that the UI is
> broken during our regular development and then it will be easier to
> pin-point a culprit.
>
> As one of my favourite quotes goes "If an upgrade is painful, simply
> do it more often rather than less often - that makes it far less
> painful in general".
>
> More info about dependabot:
>
> https://docs.github.com/en/code-security/repository-security-advisories/about-github-security-advisories-for-repositories
>
> Example PR generated by Dependabot:
> https://github.com/PolideaInternal/airflow-gepard/pull/356
>
> J.
>