You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ozone.apache.org by Mukul Kumar Singh <mk...@gmail.com> on 2021/03/25 02:32:53 UTC
Secrets committed as part of Ozone Commits
Hi,
Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
committed into Ozone's source code. Secrets, gpg passphrases, passwords,
ssh private files should not be committed into Ozone source code as they
leak credentials into the source code.
This issue will be solved via the following steps
a) The above commit will be removed from the Ozone commit history. We
will force push to the Ozone master branch with this commit removed.
b) A new commit hook/ CI check will be added to prevent this from
hapennening again.
Please hold off on merging any new changes into Ozone until the commit
is removed from Apache commit history.
Thanks,
Mukul
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by Lokesh Jain <lj...@cloudera.com.INVALID>.
Hey Mukul
Thanks for reporting the issue! I also see the commit in 1.1.0-RC0 branch.
Regards
Lokesh
> On 25-Mar-2021, at 10:30 AM, Mukul Kumar Singh <mk...@gmail.com> wrote:
>
> Thanks Arpit and Sammi for the responses.
>
> Note: Please block any merges to Apache master while this commit is being removed.
>
> I will send out another email once the process is done.
>
>
> Thanks,
> Mukul
>
> On 25/03/21 9:19 am, Sammi Chen wrote:
>> +1
>>
>> Thanks Mukul for raising the issue.
>>
>> On Thu, Mar 25, 2021 at 10:33 AM Mukul Kumar Singh <mk...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>>
>>> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
>>> committed into Ozone's source code. Secrets, gpg passphrases, passwords,
>>> ssh private files should not be committed into Ozone source code as they
>>> leak credentials into the source code.
>>>
>>> This issue will be solved via the following steps
>>>
>>> a) The above commit will be removed from the Ozone commit history. We
>>> will force push to the Ozone master branch with this commit removed.
>>>
>>> b) A new commit hook/ CI check will be added to prevent this from
>>> hapennening again.
>>>
>>> Please hold off on merging any new changes into Ozone until the commit
>>> is removed from Apache commit history.
>>>
>>> Thanks,
>>>
>>> Mukul
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
>>> For additional commands, e-mail: dev-help@ozone.apache.org
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by Siyao Meng <si...@apache.org>.
Thanks Mukul.
Note that all open PRs based on master branch after Tue Mar 2 11:11:58
2021 PST (commit time of HDDS-4864) may need to be rebased due to the
divergence.
-Siyao
On Mar 24, 2021 at 10:27:24 PM, Mukul Kumar Singh <mk...@gmail.com>
wrote:
> Thanks everyone.
>
> I have pushed the changes to the master branch.
>
> Please resume any merges to Apache master.
>
> Thanks,
>
> Mukul
>
> On 25/03/21 10:30 am, Mukul Kumar Singh wrote:
>
> Thanks Arpit and Sammi for the responses.
>
>
> Note: Please block any merges to Apache master while this commit is
>
> being removed.
>
>
> I will send out another email once the process is done.
>
>
>
> Thanks,
>
> Mukul
>
>
> On 25/03/21 9:19 am, Sammi Chen wrote:
>
> > +1
>
> >
>
> > Thanks Mukul for raising the issue.
>
> >
>
> > On Thu, Mar 25, 2021 at 10:33 AM Mukul Kumar Singh
>
> > <mk...@gmail.com>
>
> > wrote:
>
> >
>
> >> Hi,
>
> >>
>
> >>
>
> >> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key
>
> >> was
>
> >> committed into Ozone's source code. Secrets, gpg passphrases,
>
> >> passwords,
>
> >> ssh private files should not be committed into Ozone source code as
>
> >> they
>
> >> leak credentials into the source code.
>
> >>
>
> >> This issue will be solved via the following steps
>
> >>
>
> >> a) The above commit will be removed from the Ozone commit history. We
>
> >> will force push to the Ozone master branch with this commit removed.
>
> >>
>
> >> b) A new commit hook/ CI check will be added to prevent this from
>
> >> hapennening again.
>
> >>
>
> >> Please hold off on merging any new changes into Ozone until the commit
>
> >> is removed from Apache commit history.
>
> >>
>
> >> Thanks,
>
> >>
>
> >> Mukul
>
> >>
>
> >>
>
> >> ---------------------------------------------------------------------
>
> >> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
>
> >> For additional commands, e-mail: dev-help@ozone.apache.org
>
> >>
>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>
Re: Secrets committed as part of Ozone Commits
Posted by Mukul Kumar Singh <mk...@gmail.com>.
Thanks everyone.
I have pushed the changes to the master branch.
Please resume any merges to Apache master.
Thanks,
Mukul
On 25/03/21 10:30 am, Mukul Kumar Singh wrote:
> Thanks Arpit and Sammi for the responses.
>
> Note: Please block any merges to Apache master while this commit is
> being removed.
>
> I will send out another email once the process is done.
>
>
> Thanks,
> Mukul
>
> On 25/03/21 9:19 am, Sammi Chen wrote:
>> +1
>>
>> Thanks Mukul for raising the issue.
>>
>> On Thu, Mar 25, 2021 at 10:33 AM Mukul Kumar Singh
>> <mk...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>>
>>> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key
>>> was
>>> committed into Ozone's source code. Secrets, gpg passphrases,
>>> passwords,
>>> ssh private files should not be committed into Ozone source code as
>>> they
>>> leak credentials into the source code.
>>>
>>> This issue will be solved via the following steps
>>>
>>> a) The above commit will be removed from the Ozone commit history. We
>>> will force push to the Ozone master branch with this commit removed.
>>>
>>> b) A new commit hook/ CI check will be added to prevent this from
>>> hapennening again.
>>>
>>> Please hold off on merging any new changes into Ozone until the commit
>>> is removed from Apache commit history.
>>>
>>> Thanks,
>>>
>>> Mukul
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
>>> For additional commands, e-mail: dev-help@ozone.apache.org
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by Mukul Kumar Singh <mk...@gmail.com>.
Thanks Arpit and Sammi for the responses.
Note: Please block any merges to Apache master while this commit is
being removed.
I will send out another email once the process is done.
Thanks,
Mukul
On 25/03/21 9:19 am, Sammi Chen wrote:
> +1
>
> Thanks Mukul for raising the issue.
>
> On Thu, Mar 25, 2021 at 10:33 AM Mukul Kumar Singh <mk...@gmail.com>
> wrote:
>
>> Hi,
>>
>>
>> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
>> committed into Ozone's source code. Secrets, gpg passphrases, passwords,
>> ssh private files should not be committed into Ozone source code as they
>> leak credentials into the source code.
>>
>> This issue will be solved via the following steps
>>
>> a) The above commit will be removed from the Ozone commit history. We
>> will force push to the Ozone master branch with this commit removed.
>>
>> b) A new commit hook/ CI check will be added to prevent this from
>> hapennening again.
>>
>> Please hold off on merging any new changes into Ozone until the commit
>> is removed from Apache commit history.
>>
>> Thanks,
>>
>> Mukul
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
>> For additional commands, e-mail: dev-help@ozone.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by Sammi Chen <sa...@apache.org>.
+1
Thanks Mukul for raising the issue.
On Thu, Mar 25, 2021 at 10:33 AM Mukul Kumar Singh <mk...@gmail.com>
wrote:
> Hi,
>
>
> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
> committed into Ozone's source code. Secrets, gpg passphrases, passwords,
> ssh private files should not be committed into Ozone source code as they
> leak credentials into the source code.
>
> This issue will be solved via the following steps
>
> a) The above commit will be removed from the Ozone commit history. We
> will force push to the Ozone master branch with this commit removed.
>
> b) A new commit hook/ CI check will be added to prevent this from
> hapennening again.
>
> Please hold off on merging any new changes into Ozone until the commit
> is removed from Apache commit history.
>
> Thanks,
>
> Mukul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>
Re: Secrets committed as part of Ozone Commits
Posted by Bharat Viswanadham <bh...@apache.org>.
+1
Thanks for raising this issue and taking care of this.
Thanks,
Bharat
On Thu, Mar 25, 2021 at 8:03 AM Mukul Kumar Singh <mk...@gmail.com>
wrote:
> Hi,
>
>
> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
> committed into Ozone's source code. Secrets, gpg passphrases, passwords,
> ssh private files should not be committed into Ozone source code as they
> leak credentials into the source code.
>
> This issue will be solved via the following steps
>
> a) The above commit will be removed from the Ozone commit history. We
> will force push to the Ozone master branch with this commit removed.
>
> b) A new commit hook/ CI check will be added to prevent this from
> hapennening again.
>
> Please hold off on merging any new changes into Ozone until the commit
> is removed from Apache commit history.
>
> Thanks,
>
> Mukul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>
Re: Secrets committed as part of Ozone Commits
Posted by Vivek Ratnavel <vi...@gmail.com>.
I agree with Marton. +1 for Marton's proposal.
- Vivek Subramanian
On Wed, Mar 24, 2021 at 11:32 PM Elek, Marton <el...@apache.org> wrote:
>
>
>
> -1
>
>
> 1. If you are interested about the opinion of all the other
> contributors, please start a discussion which is inclusive for all the
> timezones and wait at least one day.
>
> 2. Force push is a very intrusive way, it causes new problems and it
> doesn't solve the original problem itself. If any real secret is leaked,
> it's already unsafe to use, independent if you remove it from the
> history or not.
>
> There are bots which scans new commits and there are archives of the
> github events (like https://www.gharchive.org/). Removing the commit
> doesn't solve the problem as the secret is already leaked.
>
> It's not safe to use that secret any more whether you force push or not.
>
> 3. The right approach IMHO is revoking and invalidating the secret
> itself and simply revert the commit.
>
> 4. Force-pushing invalidates all of our commit ids which are part of
> our development history: the pull requests. All the merge links on the
> reworked PRs no points to invalid commits which are not part of the master.
>
> 5. Force push is useless as you should force-push to all the forks
> which includes the commit (impossible).
>
> 6. Force-push would be required not only to the master but to all the
> existing feature-branch too (which would invalidate existing commits,
> there, too.) (For example see git log origin/HDDS-2823 --grep=HDDS-4864)
>
> But it also requires to rewrite all the branches one (to rewrite the
> merge commits only once)
>
> 7. Force-push is not safe, it's very easy to make a mistake by any
> other developers. Push the old branches from local to any other branches
> or forks where the secrets will remain be exposed.
>
> My proposal is:
>
> 1. Restore the master to the previous state.
> 2. Invalidate/revoke the leaked secret ASAP
> 3. Revert the problematic commit and recommit it without the problems
> 4. (IN the future) do discussions which includes all the time-zones.
>
> Thanks,
> Marton
>
>
>
> On 3/25/21 3:32 AM, Mukul Kumar Singh wrote:
> > Hi,
> >
> >
> > Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
> > committed into Ozone's source code. Secrets, gpg passphrases, passwords,
> > ssh private files should not be committed into Ozone source code as they
> > leak credentials into the source code.
> >
> > This issue will be solved via the following steps
> >
> > a) The above commit will be removed from the Ozone commit history. We
> > will force push to the Ozone master branch with this commit removed.
> >
> > b) A new commit hook/ CI check will be added to prevent this from
> > hapennening again.
> >
> > Please hold off on merging any new changes into Ozone until the commit
> > is removed from Apache commit history.
> >
> > Thanks,
> >
> > Mukul
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> > For additional commands, e-mail: dev-help@ozone.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
> --
Regards,
Vivek Subramanian
Re: Secrets committed as part of Ozone Commits
Posted by Stephen O'Donnell <so...@cloudera.com.INVALID>.
I agree with Marton. Force pushing to a public git repo is likely to be
very disruptive going forward, and the only safe approach to deal with the
committed secret is to revoke it and add a new commit to remove it. If it's
revoked, having it in the git history will do no harm.
+1 to what Maron suggested.
On Thu, Mar 25, 2021 at 7:49 AM Elek, Marton <el...@apache.org> wrote:
>
> >
> > 4. Force-pushing invalidates all of our commit ids which are part of
> > our development history: the pull requests. All the merge links on the
> > reworked PRs now points to invalid commits which are not part of the
> master.
>
>
> As far as I see all of our open PRs are also broken and/or conflicted:
>
>
> Like this one:
>
> https://github.com/apache/ozone/pull/2065
>
> It supposed to be a simple dependency change AFAIK.
>
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>
Re: Secrets committed as part of Ozone Commits
Posted by "Elek, Marton" <el...@apache.org>.
>
> 4. Force-pushing invalidates all of our commit ids which are part of
> our development history: the pull requests. All the merge links on the
> reworked PRs now points to invalid commits which are not part of the master.
As far as I see all of our open PRs are also broken and/or conflicted:
Like this one:
https://github.com/apache/ozone/pull/2065
It supposed to be a simple dependency change AFAIK.
Marton
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by anu engineer <an...@gmail.com>.
Completely agree with Marton's proposal, force pushing really does not
work with Git.
Invalidation of the secrets is the best defence and it would be nice to get
back to a consistent tree.
Thanks
Anu
On Thu, Mar 25, 2021 at 1:47 AM Attila Doroszlai
<ad...@cloudera.com.invalid> wrote:
> On Thu, Mar 25, 2021 at 7:32 AM Elek, Marton <el...@apache.org> wrote:
> > My proposal is:
> >
> > 1. Restore the master to the previous state.
> > 2. Invalidate/revoke the leaked secret ASAP
> > 3. Revert the problematic commit and recommit it without the problems
> > 4. (IN the future) do discussions which includes all the time-zones.
>
> +1 for Marton's proposal.
>
> Please restore master to the previous state:
>
> https://github.com/apache/ozone/commit/520ba00da07f82707b8e3c8e1e91ee324f71353a
>
> -Attila
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>
Re: Secrets committed as part of Ozone Commits
Posted by Mukul Kumar Singh <mk...@gmail.com>.
Thanks for the replies everyone.
Sorry that I force-pushed in haste.
I did not see all the problems with the force-push earlier, I have now
pushed the original master before this force push was done.
Sorry for all the confusion because of this.
Thanks,
Mukul
On 25/03/21 2:16 pm, Attila Doroszlai wrote:
> On Thu, Mar 25, 2021 at 7:32 AM Elek, Marton <el...@apache.org> wrote:
>> My proposal is:
>>
>> 1. Restore the master to the previous state.
>> 2. Invalidate/revoke the leaked secret ASAP
>> 3. Revert the problematic commit and recommit it without the problems
>> 4. (IN the future) do discussions which includes all the time-zones.
> +1 for Marton's proposal.
>
> Please restore master to the previous state:
> https://github.com/apache/ozone/commit/520ba00da07f82707b8e3c8e1e91ee324f71353a
>
> -Attila
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by Attila Doroszlai <ad...@cloudera.com.INVALID>.
On Thu, Mar 25, 2021 at 7:32 AM Elek, Marton <el...@apache.org> wrote:
> My proposal is:
>
> 1. Restore the master to the previous state.
> 2. Invalidate/revoke the leaked secret ASAP
> 3. Revert the problematic commit and recommit it without the problems
> 4. (IN the future) do discussions which includes all the time-zones.
+1 for Marton's proposal.
Please restore master to the previous state:
https://github.com/apache/ozone/commit/520ba00da07f82707b8e3c8e1e91ee324f71353a
-Attila
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by "Elek, Marton" <el...@apache.org>.
-1
1. If you are interested about the opinion of all the other
contributors, please start a discussion which is inclusive for all the
timezones and wait at least one day.
2. Force push is a very intrusive way, it causes new problems and it
doesn't solve the original problem itself. If any real secret is leaked,
it's already unsafe to use, independent if you remove it from the
history or not.
There are bots which scans new commits and there are archives of the
github events (like https://www.gharchive.org/). Removing the commit
doesn't solve the problem as the secret is already leaked.
It's not safe to use that secret any more whether you force push or not.
3. The right approach IMHO is revoking and invalidating the secret
itself and simply revert the commit.
4. Force-pushing invalidates all of our commit ids which are part of
our development history: the pull requests. All the merge links on the
reworked PRs no points to invalid commits which are not part of the master.
5. Force push is useless as you should force-push to all the forks
which includes the commit (impossible).
6. Force-push would be required not only to the master but to all the
existing feature-branch too (which would invalidate existing commits,
there, too.) (For example see git log origin/HDDS-2823 --grep=HDDS-4864)
But it also requires to rewrite all the branches one (to rewrite the
merge commits only once)
7. Force-push is not safe, it's very easy to make a mistake by any
other developers. Push the old branches from local to any other branches
or forks where the secrets will remain be exposed.
My proposal is:
1. Restore the master to the previous state.
2. Invalidate/revoke the leaked secret ASAP
3. Revert the problematic commit and recommit it without the problems
4. (IN the future) do discussions which includes all the time-zones.
Thanks,
Marton
On 3/25/21 3:32 AM, Mukul Kumar Singh wrote:
> Hi,
>
>
> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was
> committed into Ozone's source code. Secrets, gpg passphrases, passwords,
> ssh private files should not be committed into Ozone source code as they
> leak credentials into the source code.
>
> This issue will be solved via the following steps
>
> a) The above commit will be removed from the Ozone commit history. We
> will force push to the Ozone master branch with this commit removed.
>
> b) A new commit hook/ CI check will be added to prevent this from
> hapennening again.
>
> Please hold off on merging any new changes into Ozone until the commit
> is removed from Apache commit history.
>
> Thanks,
>
> Mukul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Secrets committed as part of Ozone Commits
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
+1
It is surprising this was not caught by GitHub.
https://docs.github.com/en/code-security/secret-security/about-secret-scanning <https://docs.github.com/en/code-security/secret-security/about-secret-scanning>
Thanks for raising this.
> On Mar 24, 2021, at 7:32 PM, Mukul Kumar Singh <mk...@gmail.com> wrote:
>
> Hi,
>
>
> Recently, through one of the jiras(HDDS-4864), aws_secret_access_key was committed into Ozone's source code. Secrets, gpg passphrases, passwords, ssh private files should not be committed into Ozone source code as they leak credentials into the source code.
>
> This issue will be solved via the following steps
>
> a) The above commit will be removed from the Ozone commit history. We will force push to the Ozone master branch with this commit removed.
>
> b) A new commit hook/ CI check will be added to prevent this from hapennening again.
>
> Please hold off on merging any new changes into Ozone until the commit is removed from Apache commit history.
>
> Thanks,
>
> Mukul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>