You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by "Gray, Jonathan" <Jo...@comcast.com.INVALID> on 2020/12/07 23:00:13 UTC
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
HashiCorp Vault and/or Consul is the only other primary contender I think we've had proposed, but I'm +1/+1 as well.
Jonathan G
On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
Yes, I agree with the plugin interface as well, but that is what I was
hoping to defer to a follow-up thread, preferably with a rough draft
of a blueprint in hand. First, I just want to get an official
consensus on PostgreSQL (in this case as the _main_ plugin
implementation).
- Rawlin
On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
>
> +1 and +1 to what @neuman said. I'd vote this be framed more like "change
> TO secret store to a Plugin interface, and ATC will provide a Postgres
> Plugin."
>
> I'd also like to note, I believe our company has a legal requirement to
> have a separate "secret" database, so the Postgres secret store needs to at
> least have the ability to be a separate DB URL+auth than the primary TO
> Postgres DB.
>
>
> On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org> wrote:
>
> > I am +1 for using Postgres, however we should consider implementing the
> > "secret store" functionality in such a way that people can choose to
> > implement whatever backend they want. I think it can be accomplished using
> > the TO plugin functionality but I am sure people more familiar with the
> > code these days would know better. This would also provide a built in way
> > to migrate from one to the other without forcing everyone to change.
> >
> >
> >
> > On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org> wrote:
> >
> > > Hey folks,
> > >
> > > I hope by now everyone can agree that we need to replace Riak (it's
> > > been unmaintained for quite some time now). However, we might not all
> > > agree yet on what it should be replaced with (at least not
> > > officially). We've discussed it in threads here and there, but I'd
> > > like to get some official consensus before we really hit the ground
> > > running.
> > >
> > > I would like to propose that we replace Riak with PostgreSQL.
> > >
> > > Here are some of the reasons that I can think of (and have been
> > > mentioned by others in the past) for us to use PostgreSQL:
> > > - we all have much experience running it in production (because we
> > > already run it for the Traffic Ops database)
> > > - it would simplify ATC deployments by removing one more component
> > > from the system
> > > - it would simplify development as ATC devs are already familiar with
> > > traditional SQL databases, and we could reuse a lot of the existing
> > > code
> > > - it has a healthy community of support and doesn't seem to be losing
> > > steam anytime soon (it still remains the 2nd most popular OSS
> > > relational database behind MySQL [1])
> > >
> > > I would like this thread to focus on the merits (or lack thereof) of
> > > using PostgreSQL as a replacement for Riak. We can discuss the
> > > low-level implementation details separately in the blueprint I will
> > > propose as a follow-up to this discussion. Unless someone is
> > > vehemently -1 on using PostgreSQL to replace Riak, I will take silence
> > > as assent and move forward with the blueprint process.
> > >
> > > - Rawlin
> > >
> > > [1] https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$
> > >
> >
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
Posted by Rawlin Peters <ra...@apache.org>.
I've never used the experimental Emulated Vault, but if I understand
it correctly, it is supposed to mimic the Riak interface. The plan is
to keep Riak support while also adding postgres support, and you would
choose which backend to use via a new config option (which will
default to Riak for backwards-compatibility). I'm not positive the
Emulated Vault works out of the box with the current version of
Traffic Ops, since we use the Riak gRPC library instead of the Riak
HTTP API (which is what the old Perl version of Traffic Ops used). So
someone would probably need to add support for the Emulated Vault as
another Traffic Vault backend once the plugin interface is
implemented.
To answer your 2nd question, yes, everything that is currently stored
in Riak would be stored in Postgres instead if you were to choose the
Postgres option. If interested, keep an eye out for the blueprint I'll
be proposing within the next month or so.
- Rawlin
On Wed, Mar 3, 2021 at 4:43 AM Amarnath B S <am...@gmail.com> wrote:
>
> Is the plugin interface planned around the "emulated vault" experimental feature already in ATC? https://github.com/apache/trafficcontrol/tree/master/experimental/emulated_vault. The emulated vault shim layer looks quite clean and extensible.
> Also, just want to confirm that both the use cases of storing DS certs/keys and DNSSEC keys are planned to be covered.
> Thanks,
> -Amar
>
> On 2020/12/16 18:03:53, Rawlin Peters <ra...@apache.org> wrote:
> > Thank you all for the feedback. It sounds like Postgres can do the
> > job, so I'll move forward with the blueprint for implementing it as a
> > type of plugin interface with Postgres and Riak as the two
> > implementations to begin with. If we wanted to add HashiCorp Vault (or
> > something else) as another plugin implementation later on, the plugin
> > interface should make that easier.
> >
> > - Rawlin
> >
> > On Mon, Dec 7, 2020 at 9:48 PM Chatterjee, Srijeet
> > <SR...@comcast.com.invalid> wrote:
> > >
> > > +1 on moving away from Riak and using postgres instead. Riak can be a pain(atleast for new users) to set up and debug.
> > >
> > > --Srijeet
> > >
> > > From: John Rushford <jj...@gmail.com>
> > > Date: Monday, December 7, 2020 at 8:07 PM
> > > To: dev@trafficcontrol.apache.org <de...@trafficcontrol.apache.org>
> > > Subject: Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
> > > +1 on using Postgres. I’ve played around with Postgres and pgcrypto. Keeping certs and sig keys encrypted is easily done in postgres. I used asymmetric public private key encryption with pgcrypto where I stored the public key to encrypt in a database table. The private key was stored apart from the database and used by an api that had secure access to the private key. This worked quite well and is mostly done with Postgres queries
> > >
> > > Sent from my iPhone
> > >
> > > > On Dec 7, 2020, at 4:00 PM, Gray, Jonathan <Jo...@comcast.com.invalid> wrote:
> > > >
> > > > HashiCorp Vault and/or Consul is the only other primary contender I think we've had proposed, but I'm +1/+1 as well.
> > > >
> > > > Jonathan G
> > > >
> > > > On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
> > > >
> > > > Yes, I agree with the plugin interface as well, but that is what I was
> > > > hoping to defer to a follow-up thread, preferably with a rough draft
> > > > of a blueprint in hand. First, I just want to get an official
> > > > consensus on PostgreSQL (in this case as the _main_ plugin
> > > > implementation).
> > > >
> > > > - Rawlin
> > > >
> > > >> On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
> > > >>
> > > >> +1 and +1 to what @neuman said. I'd vote this be framed more like "change
> > > >> TO secret store to a Plugin interface, and ATC will provide a Postgres
> > > >> Plugin."
> > > >>
> > > >> I'd also like to note, I believe our company has a legal requirement to
> > > >> have a separate "secret" database, so the Postgres secret store needs to at
> > > >> least have the ability to be a separate DB URL+auth than the primary TO
> > > >> Postgres DB.
> > > >>
> > > >>
> > > >>> On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org> wrote:
> > > >>>
> > > >>> I am +1 for using Postgres, however we should consider implementing the
> > > >>> "secret store" functionality in such a way that people can choose to
> > > >>> implement whatever backend they want. I think it can be accomplished using
> > > >>> the TO plugin functionality but I am sure people more familiar with the
> > > >>> code these days would know better. This would also provide a built in way
> > > >>> to migrate from one to the other without forcing everyone to change.
> > > >>>
> > > >>>
> > > >>>
> > > >>>> On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org> wrote:
> > > >>>
> > > >>>> Hey folks,
> > > >>>>
> > > >>>> I hope by now everyone can agree that we need to replace Riak (it's
> > > >>>> been unmaintained for quite some time now). However, we might not all
> > > >>>> agree yet on what it should be replaced with (at least not
> > > >>>> officially). We've discussed it in threads here and there, but I'd
> > > >>>> like to get some official consensus before we really hit the ground
> > > >>>> running.
> > > >>>>
> > > >>>> I would like to propose that we replace Riak with PostgreSQL.
> > > >>>>
> > > >>>> Here are some of the reasons that I can think of (and have been
> > > >>>> mentioned by others in the past) for us to use PostgreSQL:
> > > >>>> - we all have much experience running it in production (because we
> > > >>>> already run it for the Traffic Ops database)
> > > >>>> - it would simplify ATC deployments by removing one more component
> > > >>>> from the system
> > > >>>> - it would simplify development as ATC devs are already familiar with
> > > >>>> traditional SQL databases, and we could reuse a lot of the existing
> > > >>>> code
> > > >>>> - it has a healthy community of support and doesn't seem to be losing
> > > >>>> steam anytime soon (it still remains the 2nd most popular OSS
> > > >>>> relational database behind MySQL [1])
> > > >>>>
> > > >>>> I would like this thread to focus on the merits (or lack thereof) of
> > > >>>> using PostgreSQL as a replacement for Riak. We can discuss the
> > > >>>> low-level implementation details separately in the blueprint I will
> > > >>>> propose as a follow-up to this discussion. Unless someone is
> > > >>>> vehemently -1 on using PostgreSQL to replace Riak, I will take silence
> > > >>>> as assent and move forward with the blueprint process.
> > > >>>>
> > > >>>> - Rawlin
> > > >>>>
> > > >>>> [1] https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$<https://urldefense.com/v3/__https:/db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$>
> > > >>>>
> > > >>>
> > > >
> >
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
Posted by Amarnath B S <am...@gmail.com>.
Is the plugin interface planned around the "emulated vault" experimental feature already in ATC? https://github.com/apache/trafficcontrol/tree/master/experimental/emulated_vault. The emulated vault shim layer looks quite clean and extensible.
Also, just want to confirm that both the use cases of storing DS certs/keys and DNSSEC keys are planned to be covered.
Thanks,
-Amar
On 2020/12/16 18:03:53, Rawlin Peters <ra...@apache.org> wrote:
> Thank you all for the feedback. It sounds like Postgres can do the
> job, so I'll move forward with the blueprint for implementing it as a
> type of plugin interface with Postgres and Riak as the two
> implementations to begin with. If we wanted to add HashiCorp Vault (or
> something else) as another plugin implementation later on, the plugin
> interface should make that easier.
>
> - Rawlin
>
> On Mon, Dec 7, 2020 at 9:48 PM Chatterjee, Srijeet
> <SR...@comcast.com.invalid> wrote:
> >
> > +1 on moving away from Riak and using postgres instead. Riak can be a pain(atleast for new users) to set up and debug.
> >
> > --Srijeet
> >
> > From: John Rushford <jj...@gmail.com>
> > Date: Monday, December 7, 2020 at 8:07 PM
> > To: dev@trafficcontrol.apache.org <de...@trafficcontrol.apache.org>
> > Subject: Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
> > +1 on using Postgres. I’ve played around with Postgres and pgcrypto. Keeping certs and sig keys encrypted is easily done in postgres. I used asymmetric public private key encryption with pgcrypto where I stored the public key to encrypt in a database table. The private key was stored apart from the database and used by an api that had secure access to the private key. This worked quite well and is mostly done with Postgres queries
> >
> > Sent from my iPhone
> >
> > > On Dec 7, 2020, at 4:00 PM, Gray, Jonathan <Jo...@comcast.com.invalid> wrote:
> > >
> > > HashiCorp Vault and/or Consul is the only other primary contender I think we've had proposed, but I'm +1/+1 as well.
> > >
> > > Jonathan G
> > >
> > > On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
> > >
> > > Yes, I agree with the plugin interface as well, but that is what I was
> > > hoping to defer to a follow-up thread, preferably with a rough draft
> > > of a blueprint in hand. First, I just want to get an official
> > > consensus on PostgreSQL (in this case as the _main_ plugin
> > > implementation).
> > >
> > > - Rawlin
> > >
> > >> On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
> > >>
> > >> +1 and +1 to what @neuman said. I'd vote this be framed more like "change
> > >> TO secret store to a Plugin interface, and ATC will provide a Postgres
> > >> Plugin."
> > >>
> > >> I'd also like to note, I believe our company has a legal requirement to
> > >> have a separate "secret" database, so the Postgres secret store needs to at
> > >> least have the ability to be a separate DB URL+auth than the primary TO
> > >> Postgres DB.
> > >>
> > >>
> > >>> On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org> wrote:
> > >>>
> > >>> I am +1 for using Postgres, however we should consider implementing the
> > >>> "secret store" functionality in such a way that people can choose to
> > >>> implement whatever backend they want. I think it can be accomplished using
> > >>> the TO plugin functionality but I am sure people more familiar with the
> > >>> code these days would know better. This would also provide a built in way
> > >>> to migrate from one to the other without forcing everyone to change.
> > >>>
> > >>>
> > >>>
> > >>>> On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org> wrote:
> > >>>
> > >>>> Hey folks,
> > >>>>
> > >>>> I hope by now everyone can agree that we need to replace Riak (it's
> > >>>> been unmaintained for quite some time now). However, we might not all
> > >>>> agree yet on what it should be replaced with (at least not
> > >>>> officially). We've discussed it in threads here and there, but I'd
> > >>>> like to get some official consensus before we really hit the ground
> > >>>> running.
> > >>>>
> > >>>> I would like to propose that we replace Riak with PostgreSQL.
> > >>>>
> > >>>> Here are some of the reasons that I can think of (and have been
> > >>>> mentioned by others in the past) for us to use PostgreSQL:
> > >>>> - we all have much experience running it in production (because we
> > >>>> already run it for the Traffic Ops database)
> > >>>> - it would simplify ATC deployments by removing one more component
> > >>>> from the system
> > >>>> - it would simplify development as ATC devs are already familiar with
> > >>>> traditional SQL databases, and we could reuse a lot of the existing
> > >>>> code
> > >>>> - it has a healthy community of support and doesn't seem to be losing
> > >>>> steam anytime soon (it still remains the 2nd most popular OSS
> > >>>> relational database behind MySQL [1])
> > >>>>
> > >>>> I would like this thread to focus on the merits (or lack thereof) of
> > >>>> using PostgreSQL as a replacement for Riak. We can discuss the
> > >>>> low-level implementation details separately in the blueprint I will
> > >>>> propose as a follow-up to this discussion. Unless someone is
> > >>>> vehemently -1 on using PostgreSQL to replace Riak, I will take silence
> > >>>> as assent and move forward with the blueprint process.
> > >>>>
> > >>>> - Rawlin
> > >>>>
> > >>>> [1] https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$<https://urldefense.com/v3/__https:/db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$>
> > >>>>
> > >>>
> > >
>
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
Posted by Rawlin Peters <ra...@apache.org>.
Thank you all for the feedback. It sounds like Postgres can do the
job, so I'll move forward with the blueprint for implementing it as a
type of plugin interface with Postgres and Riak as the two
implementations to begin with. If we wanted to add HashiCorp Vault (or
something else) as another plugin implementation later on, the plugin
interface should make that easier.
- Rawlin
On Mon, Dec 7, 2020 at 9:48 PM Chatterjee, Srijeet
<SR...@comcast.com.invalid> wrote:
>
> +1 on moving away from Riak and using postgres instead. Riak can be a pain(atleast for new users) to set up and debug.
>
> --Srijeet
>
> From: John Rushford <jj...@gmail.com>
> Date: Monday, December 7, 2020 at 8:07 PM
> To: dev@trafficcontrol.apache.org <de...@trafficcontrol.apache.org>
> Subject: Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
> +1 on using Postgres. I’ve played around with Postgres and pgcrypto. Keeping certs and sig keys encrypted is easily done in postgres. I used asymmetric public private key encryption with pgcrypto where I stored the public key to encrypt in a database table. The private key was stored apart from the database and used by an api that had secure access to the private key. This worked quite well and is mostly done with Postgres queries
>
> Sent from my iPhone
>
> > On Dec 7, 2020, at 4:00 PM, Gray, Jonathan <Jo...@comcast.com.invalid> wrote:
> >
> > HashiCorp Vault and/or Consul is the only other primary contender I think we've had proposed, but I'm +1/+1 as well.
> >
> > Jonathan G
> >
> > On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
> >
> > Yes, I agree with the plugin interface as well, but that is what I was
> > hoping to defer to a follow-up thread, preferably with a rough draft
> > of a blueprint in hand. First, I just want to get an official
> > consensus on PostgreSQL (in this case as the _main_ plugin
> > implementation).
> >
> > - Rawlin
> >
> >> On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
> >>
> >> +1 and +1 to what @neuman said. I'd vote this be framed more like "change
> >> TO secret store to a Plugin interface, and ATC will provide a Postgres
> >> Plugin."
> >>
> >> I'd also like to note, I believe our company has a legal requirement to
> >> have a separate "secret" database, so the Postgres secret store needs to at
> >> least have the ability to be a separate DB URL+auth than the primary TO
> >> Postgres DB.
> >>
> >>
> >>> On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org> wrote:
> >>>
> >>> I am +1 for using Postgres, however we should consider implementing the
> >>> "secret store" functionality in such a way that people can choose to
> >>> implement whatever backend they want. I think it can be accomplished using
> >>> the TO plugin functionality but I am sure people more familiar with the
> >>> code these days would know better. This would also provide a built in way
> >>> to migrate from one to the other without forcing everyone to change.
> >>>
> >>>
> >>>
> >>>> On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org> wrote:
> >>>
> >>>> Hey folks,
> >>>>
> >>>> I hope by now everyone can agree that we need to replace Riak (it's
> >>>> been unmaintained for quite some time now). However, we might not all
> >>>> agree yet on what it should be replaced with (at least not
> >>>> officially). We've discussed it in threads here and there, but I'd
> >>>> like to get some official consensus before we really hit the ground
> >>>> running.
> >>>>
> >>>> I would like to propose that we replace Riak with PostgreSQL.
> >>>>
> >>>> Here are some of the reasons that I can think of (and have been
> >>>> mentioned by others in the past) for us to use PostgreSQL:
> >>>> - we all have much experience running it in production (because we
> >>>> already run it for the Traffic Ops database)
> >>>> - it would simplify ATC deployments by removing one more component
> >>>> from the system
> >>>> - it would simplify development as ATC devs are already familiar with
> >>>> traditional SQL databases, and we could reuse a lot of the existing
> >>>> code
> >>>> - it has a healthy community of support and doesn't seem to be losing
> >>>> steam anytime soon (it still remains the 2nd most popular OSS
> >>>> relational database behind MySQL [1])
> >>>>
> >>>> I would like this thread to focus on the merits (or lack thereof) of
> >>>> using PostgreSQL as a replacement for Riak. We can discuss the
> >>>> low-level implementation details separately in the blueprint I will
> >>>> propose as a follow-up to this discussion. Unless someone is
> >>>> vehemently -1 on using PostgreSQL to replace Riak, I will take silence
> >>>> as assent and move forward with the blueprint process.
> >>>>
> >>>> - Rawlin
> >>>>
> >>>> [1] https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$<https://urldefense.com/v3/__https:/db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$>
> >>>>
> >>>
> >
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
Posted by "Chatterjee, Srijeet" <SR...@comcast.com.INVALID>.
+1 on moving away from Riak and using postgres instead. Riak can be a pain(atleast for new users) to set up and debug.
--Srijeet
From: John Rushford <jj...@gmail.com>
Date: Monday, December 7, 2020 at 8:07 PM
To: dev@trafficcontrol.apache.org <de...@trafficcontrol.apache.org>
Subject: Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
+1 on using Postgres. I’ve played around with Postgres and pgcrypto. Keeping certs and sig keys encrypted is easily done in postgres. I used asymmetric public private key encryption with pgcrypto where I stored the public key to encrypt in a database table. The private key was stored apart from the database and used by an api that had secure access to the private key. This worked quite well and is mostly done with Postgres queries
Sent from my iPhone
> On Dec 7, 2020, at 4:00 PM, Gray, Jonathan <Jo...@comcast.com.invalid> wrote:
>
> HashiCorp Vault and/or Consul is the only other primary contender I think we've had proposed, but I'm +1/+1 as well.
>
> Jonathan G
>
> On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
>
> Yes, I agree with the plugin interface as well, but that is what I was
> hoping to defer to a follow-up thread, preferably with a rough draft
> of a blueprint in hand. First, I just want to get an official
> consensus on PostgreSQL (in this case as the _main_ plugin
> implementation).
>
> - Rawlin
>
>> On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
>>
>> +1 and +1 to what @neuman said. I'd vote this be framed more like "change
>> TO secret store to a Plugin interface, and ATC will provide a Postgres
>> Plugin."
>>
>> I'd also like to note, I believe our company has a legal requirement to
>> have a separate "secret" database, so the Postgres secret store needs to at
>> least have the ability to be a separate DB URL+auth than the primary TO
>> Postgres DB.
>>
>>
>>> On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org> wrote:
>>>
>>> I am +1 for using Postgres, however we should consider implementing the
>>> "secret store" functionality in such a way that people can choose to
>>> implement whatever backend they want. I think it can be accomplished using
>>> the TO plugin functionality but I am sure people more familiar with the
>>> code these days would know better. This would also provide a built in way
>>> to migrate from one to the other without forcing everyone to change.
>>>
>>>
>>>
>>>> On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org> wrote:
>>>
>>>> Hey folks,
>>>>
>>>> I hope by now everyone can agree that we need to replace Riak (it's
>>>> been unmaintained for quite some time now). However, we might not all
>>>> agree yet on what it should be replaced with (at least not
>>>> officially). We've discussed it in threads here and there, but I'd
>>>> like to get some official consensus before we really hit the ground
>>>> running.
>>>>
>>>> I would like to propose that we replace Riak with PostgreSQL.
>>>>
>>>> Here are some of the reasons that I can think of (and have been
>>>> mentioned by others in the past) for us to use PostgreSQL:
>>>> - we all have much experience running it in production (because we
>>>> already run it for the Traffic Ops database)
>>>> - it would simplify ATC deployments by removing one more component
>>>> from the system
>>>> - it would simplify development as ATC devs are already familiar with
>>>> traditional SQL databases, and we could reuse a lot of the existing
>>>> code
>>>> - it has a healthy community of support and doesn't seem to be losing
>>>> steam anytime soon (it still remains the 2nd most popular OSS
>>>> relational database behind MySQL [1])
>>>>
>>>> I would like this thread to focus on the merits (or lack thereof) of
>>>> using PostgreSQL as a replacement for Riak. We can discuss the
>>>> low-level implementation details separately in the blueprint I will
>>>> propose as a follow-up to this discussion. Unless someone is
>>>> vehemently -1 on using PostgreSQL to replace Riak, I will take silence
>>>> as assent and move forward with the blueprint process.
>>>>
>>>> - Rawlin
>>>>
>>>> [1] https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$<https://urldefense.com/v3/__https:/db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$>
>>>>
>>>
>
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
Posted by John Rushford <jj...@gmail.com>.
+1 on using Postgres. I’ve played around with Postgres and pgcrypto. Keeping certs and sig keys encrypted is easily done in postgres. I used asymmetric public private key encryption with pgcrypto where I stored the public key to encrypt in a database table. The private key was stored apart from the database and used by an api that had secure access to the private key. This worked quite well and is mostly done with Postgres queries
Sent from my iPhone
> On Dec 7, 2020, at 4:00 PM, Gray, Jonathan <Jo...@comcast.com.invalid> wrote:
>
> HashiCorp Vault and/or Consul is the only other primary contender I think we've had proposed, but I'm +1/+1 as well.
>
> Jonathan G
>
> On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
>
> Yes, I agree with the plugin interface as well, but that is what I was
> hoping to defer to a follow-up thread, preferably with a rough draft
> of a blueprint in hand. First, I just want to get an official
> consensus on PostgreSQL (in this case as the _main_ plugin
> implementation).
>
> - Rawlin
>
>> On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
>>
>> +1 and +1 to what @neuman said. I'd vote this be framed more like "change
>> TO secret store to a Plugin interface, and ATC will provide a Postgres
>> Plugin."
>>
>> I'd also like to note, I believe our company has a legal requirement to
>> have a separate "secret" database, so the Postgres secret store needs to at
>> least have the ability to be a separate DB URL+auth than the primary TO
>> Postgres DB.
>>
>>
>>> On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org> wrote:
>>>
>>> I am +1 for using Postgres, however we should consider implementing the
>>> "secret store" functionality in such a way that people can choose to
>>> implement whatever backend they want. I think it can be accomplished using
>>> the TO plugin functionality but I am sure people more familiar with the
>>> code these days would know better. This would also provide a built in way
>>> to migrate from one to the other without forcing everyone to change.
>>>
>>>
>>>
>>>> On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org> wrote:
>>>
>>>> Hey folks,
>>>>
>>>> I hope by now everyone can agree that we need to replace Riak (it's
>>>> been unmaintained for quite some time now). However, we might not all
>>>> agree yet on what it should be replaced with (at least not
>>>> officially). We've discussed it in threads here and there, but I'd
>>>> like to get some official consensus before we really hit the ground
>>>> running.
>>>>
>>>> I would like to propose that we replace Riak with PostgreSQL.
>>>>
>>>> Here are some of the reasons that I can think of (and have been
>>>> mentioned by others in the past) for us to use PostgreSQL:
>>>> - we all have much experience running it in production (because we
>>>> already run it for the Traffic Ops database)
>>>> - it would simplify ATC deployments by removing one more component
>>>> from the system
>>>> - it would simplify development as ATC devs are already familiar with
>>>> traditional SQL databases, and we could reuse a lot of the existing
>>>> code
>>>> - it has a healthy community of support and doesn't seem to be losing
>>>> steam anytime soon (it still remains the 2nd most popular OSS
>>>> relational database behind MySQL [1])
>>>>
>>>> I would like this thread to focus on the merits (or lack thereof) of
>>>> using PostgreSQL as a replacement for Riak. We can discuss the
>>>> low-level implementation details separately in the blueprint I will
>>>> propose as a follow-up to this discussion. Unless someone is
>>>> vehemently -1 on using PostgreSQL to replace Riak, I will take silence
>>>> as assent and move forward with the blueprint process.
>>>>
>>>> - Rawlin
>>>>
>>>> [1] https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$
>>>>
>>>
>
Re: [EXTERNAL] Re: Replace Riak w/ PostgreSQL
Posted by Zach Hoffman <ho...@gmail.com>.
+1 on PostgreSQL so we can stop coding around Riak bugs. We can work out
the logistics later, but there are definitely secure ways to do it.
-Zach
On Mon, Dec 7, 2020 at 4:00 PM Gray, Jonathan
<Jo...@comcast.com.invalid> wrote:
> HashiCorp Vault and/or Consul is the only other primary contender I think
> we've had proposed, but I'm +1/+1 as well.
>
> Jonathan G
>
> On 12/7/20, 3:58 PM, "Rawlin Peters" <ra...@apache.org> wrote:
>
> Yes, I agree with the plugin interface as well, but that is what I was
> hoping to defer to a follow-up thread, preferably with a rough draft
> of a blueprint in hand. First, I just want to get an official
> consensus on PostgreSQL (in this case as the _main_ plugin
> implementation).
>
> - Rawlin
>
> On Mon, Dec 7, 2020 at 2:24 PM Robert O Butts <ro...@apache.org> wrote:
> >
> > +1 and +1 to what @neuman said. I'd vote this be framed more like
> "change
> > TO secret store to a Plugin interface, and ATC will provide a
> Postgres
> > Plugin."
> >
> > I'd also like to note, I believe our company has a legal requirement
> to
> > have a separate "secret" database, so the Postgres secret store
> needs to at
> > least have the ability to be a separate DB URL+auth than the primary
> TO
> > Postgres DB.
> >
> >
> > On Mon, Dec 7, 2020 at 2:13 PM Dave Neuman <ne...@apache.org>
> wrote:
> >
> > > I am +1 for using Postgres, however we should consider
> implementing the
> > > "secret store" functionality in such a way that people can choose
> to
> > > implement whatever backend they want. I think it can be
> accomplished using
> > > the TO plugin functionality but I am sure people more familiar
> with the
> > > code these days would know better. This would also provide a
> built in way
> > > to migrate from one to the other without forcing everyone to
> change.
> > >
> > >
> > >
> > > On Mon, Dec 7, 2020 at 1:48 PM Rawlin Peters <ra...@apache.org>
> wrote:
> > >
> > > > Hey folks,
> > > >
> > > > I hope by now everyone can agree that we need to replace Riak
> (it's
> > > > been unmaintained for quite some time now). However, we might
> not all
> > > > agree yet on what it should be replaced with (at least not
> > > > officially). We've discussed it in threads here and there, but
> I'd
> > > > like to get some official consensus before we really hit the
> ground
> > > > running.
> > > >
> > > > I would like to propose that we replace Riak with PostgreSQL.
> > > >
> > > > Here are some of the reasons that I can think of (and have been
> > > > mentioned by others in the past) for us to use PostgreSQL:
> > > > - we all have much experience running it in production (because
> we
> > > > already run it for the Traffic Ops database)
> > > > - it would simplify ATC deployments by removing one more
> component
> > > > from the system
> > > > - it would simplify development as ATC devs are already familiar
> with
> > > > traditional SQL databases, and we could reuse a lot of the
> existing
> > > > code
> > > > - it has a healthy community of support and doesn't seem to be
> losing
> > > > steam anytime soon (it still remains the 2nd most popular OSS
> > > > relational database behind MySQL [1])
> > > >
> > > > I would like this thread to focus on the merits (or lack
> thereof) of
> > > > using PostgreSQL as a replacement for Riak. We can discuss the
> > > > low-level implementation details separately in the blueprint I
> will
> > > > propose as a follow-up to this discussion. Unless someone is
> > > > vehemently -1 on using PostgreSQL to replace Riak, I will take
> silence
> > > > as assent and move forward with the blueprint process.
> > > >
> > > > - Rawlin
> > > >
> > > > [1]
> https://urldefense.com/v3/__https://db-engines.com/en/ranking_osvsc__;!!CQl3mcHX2A!Xy7f_5hSaPy1VOI6G3s7dnOKEWWmrpYJK1noQ67A3gTlldrSM596Zx4YjjbMHFUITPk4$
> > > >
> > >
>
>