Removal of "Long Description 1" and "Long Description 2"

["Chatterjee, Srijeet" <SR...@comcast.com.INVALID>]

Hi all,
I’m proposing a change to the DeliveryService structure and database table, to remove the “Long Description 1” and “Long Description 2” fields. These fields are seldom used, and can be replaced by the already existing “Long Description” field. In my migration, I’m planning to copy over all the data in the existing LD1 and LD2 fields, and append it to the LD field, and drop the LD! And LD2 fields from the table. Pls let me know if you have concerns with this change, or if you have a better way of doing this.

Thanks,
Srijeet

Re: Removal of "Long Description 1" and "Long Description 2"

[Rawlin Peters <ra...@apache.org>]

> It might be better to make the upgrade fail if the field isn't empty, requiring users to handle and move or delete the data themselves before upgrading.

I don't really jump for joy at the thought of having to contact and
hunt down the owners of 550 delivery services and manually move or
delete their longDesc1 and longDesc2 fields (and I would honestly feel
sorry for anyone that would have to do that for so many DSes), but I
do agree that simply deleting them or concatenating them onto longDesc
would probably break a lot of them. I can't imagine anything
mission-critical in those fields though, so I think if we combined the
values in a way that made it easier to parse out and separate the
individual legacy fields, that would probably be fine for most people.
For the rare few who are putting mission-critical stuff into longDesc1
and longDesc2, well, they should have plenty of time to manually
combine them before running the upgrade.

- Rawlin

On Tue, Jun 29, 2021 at 3:03 PM ocket 8888 <oc...@gmail.com> wrote:
>
> +1
>
> I don't have an opinion on failing the upgrade vs concatenating data when
> the columns are eventually dropped (frankly I think it'd be fine to just
> drop the data there because you would have no way to access it through the
> API anyway, but that's probably not a popular idea), but I do agree that
> it's going to need to wait until APIv3 is removed.
>
> On Tue, Jun 29, 2021 at 9:20 AM Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > yeah, what rob said. let's kill them but follow the correct api deprecation
> > process to avoid any "surprises".
> >
> > jeremy
> >
> > On Mon, Jun 28, 2021 at 10:02 PM Robert O Butts <ro...@apache.org> wrote:
> >
> > > I'm +1 on getting rid of those fields, but unfortunately we need to
> > support
> > > them in older APIs. People have scripts that rely on them, and parse
> > magic
> > > data they've put in those fields.
> > >
> > > We can remove the field from the API in newer major versions. But we need
> > > to somehow still serve it in the existing API versions (which will need
> > > supported for at least 1 major ATC version after the deprecation is
> > > announced).
> > >
> > > I agree, they're terrible and need removed. But I know for a fact there
> > are
> > > people putting arbitrary data in them in Production systems, which they
> > > parse with custom scripts hitting the API. Even if we didn't know that
> > for
> > > sure, we still need to follow the Deprecation/removal process, to avoid
> > > breaking users who don't necessarily follow the mailing lists closely.
> > >
> > > So, ATC 6.0 is about to be released, which introduces API 4.0. If we
> > remove
> > > `longDesc2` and `longDesc3` from API 4.0 and declare API 3.x deprecated
> > in
> > > ATC 6.0, we can remove API 3.x and older in ATC 7.0. Which will allow us
> > to
> > > delete the database fields in ATC 7.0.
> > >
> > > If we actually delete the database fields, we need to stop serving API
> > 3.x
> > > and older. We historically haven't removed API versions as often as we
> > > could, mostly because it's painful to users. But serving a half-baked API
> > > will cause all kinds of subtle bugs and data loss. If we delete the DB
> > > field, we need to actually stop serving /api/3.x.
> > >
> > > > copy over all the data in the existing LD1 and LD2 fields, and append
> > it
> > > to the LD field, and drop the LD! And LD2 fields from the table.
> > >
> > > I'm not so sure about this, though. If someone has a specific format,
> > > key-value, JSON, etc, just joining the fields is going to break their
> > > scripts. It might be better to make the upgrade fail if the field isn't
> > > empty, requiring users to handle and move or delete the data themselves
> > > before upgrading.
> > >
> > >
> > > On Mon, Jun 28, 2021 at 3:56 PM Chatterjee, Srijeet
> > > <SR...@comcast.com.invalid> wrote:
> > >
> > > > Hi all,
> > > > I’m proposing a change to the DeliveryService structure and database
> > > > table, to remove the “Long Description 1” and “Long Description 2”
> > > fields.
> > > > These fields are seldom used, and can be replaced by the already
> > existing
> > > > “Long Description” field. In my migration, I’m planning to copy over
> > all
> > > > the data in the existing LD1 and LD2 fields, and append it to the LD
> > > field,
> > > > and drop the LD! And LD2 fields from the table. Pls let me know if you
> > > have
> > > > concerns with this change, or if you have a better way of doing this.
> > > >
> > > > Thanks,
> > > > Srijeet
> > > >
> > >
> >

Re: Removal of "Long Description 1" and "Long Description 2"

[ocket 8888 <oc...@gmail.com>]

+1

I don't have an opinion on failing the upgrade vs concatenating data when
the columns are eventually dropped (frankly I think it'd be fine to just
drop the data there because you would have no way to access it through the
API anyway, but that's probably not a popular idea), but I do agree that
it's going to need to wait until APIv3 is removed.

On Tue, Jun 29, 2021 at 9:20 AM Jeremy Mitchell <mi...@gmail.com>
wrote:

> yeah, what rob said. let's kill them but follow the correct api deprecation
> process to avoid any "surprises".
>
> jeremy
>
> On Mon, Jun 28, 2021 at 10:02 PM Robert O Butts <ro...@apache.org> wrote:
>
> > I'm +1 on getting rid of those fields, but unfortunately we need to
> support
> > them in older APIs. People have scripts that rely on them, and parse
> magic
> > data they've put in those fields.
> >
> > We can remove the field from the API in newer major versions. But we need
> > to somehow still serve it in the existing API versions (which will need
> > supported for at least 1 major ATC version after the deprecation is
> > announced).
> >
> > I agree, they're terrible and need removed. But I know for a fact there
> are
> > people putting arbitrary data in them in Production systems, which they
> > parse with custom scripts hitting the API. Even if we didn't know that
> for
> > sure, we still need to follow the Deprecation/removal process, to avoid
> > breaking users who don't necessarily follow the mailing lists closely.
> >
> > So, ATC 6.0 is about to be released, which introduces API 4.0. If we
> remove
> > `longDesc2` and `longDesc3` from API 4.0 and declare API 3.x deprecated
> in
> > ATC 6.0, we can remove API 3.x and older in ATC 7.0. Which will allow us
> to
> > delete the database fields in ATC 7.0.
> >
> > If we actually delete the database fields, we need to stop serving API
> 3.x
> > and older. We historically haven't removed API versions as often as we
> > could, mostly because it's painful to users. But serving a half-baked API
> > will cause all kinds of subtle bugs and data loss. If we delete the DB
> > field, we need to actually stop serving /api/3.x.
> >
> > > copy over all the data in the existing LD1 and LD2 fields, and append
> it
> > to the LD field, and drop the LD! And LD2 fields from the table.
> >
> > I'm not so sure about this, though. If someone has a specific format,
> > key-value, JSON, etc, just joining the fields is going to break their
> > scripts. It might be better to make the upgrade fail if the field isn't
> > empty, requiring users to handle and move or delete the data themselves
> > before upgrading.
> >
> >
> > On Mon, Jun 28, 2021 at 3:56 PM Chatterjee, Srijeet
> > <SR...@comcast.com.invalid> wrote:
> >
> > > Hi all,
> > > I’m proposing a change to the DeliveryService structure and database
> > > table, to remove the “Long Description 1” and “Long Description 2”
> > fields.
> > > These fields are seldom used, and can be replaced by the already
> existing
> > > “Long Description” field. In my migration, I’m planning to copy over
> all
> > > the data in the existing LD1 and LD2 fields, and append it to the LD
> > field,
> > > and drop the LD! And LD2 fields from the table. Pls let me know if you
> > have
> > > concerns with this change, or if you have a better way of doing this.
> > >
> > > Thanks,
> > > Srijeet
> > >
> >
>

Re: Removal of "Long Description 1" and "Long Description 2"

[Jeremy Mitchell <mi...@gmail.com>]

yeah, what rob said. let's kill them but follow the correct api deprecation
process to avoid any "surprises".

jeremy

On Mon, Jun 28, 2021 at 10:02 PM Robert O Butts <ro...@apache.org> wrote:

> I'm +1 on getting rid of those fields, but unfortunately we need to support
> them in older APIs. People have scripts that rely on them, and parse magic
> data they've put in those fields.
>
> We can remove the field from the API in newer major versions. But we need
> to somehow still serve it in the existing API versions (which will need
> supported for at least 1 major ATC version after the deprecation is
> announced).
>
> I agree, they're terrible and need removed. But I know for a fact there are
> people putting arbitrary data in them in Production systems, which they
> parse with custom scripts hitting the API. Even if we didn't know that for
> sure, we still need to follow the Deprecation/removal process, to avoid
> breaking users who don't necessarily follow the mailing lists closely.
>
> So, ATC 6.0 is about to be released, which introduces API 4.0. If we remove
> `longDesc2` and `longDesc3` from API 4.0 and declare API 3.x deprecated in
> ATC 6.0, we can remove API 3.x and older in ATC 7.0. Which will allow us to
> delete the database fields in ATC 7.0.
>
> If we actually delete the database fields, we need to stop serving API 3.x
> and older. We historically haven't removed API versions as often as we
> could, mostly because it's painful to users. But serving a half-baked API
> will cause all kinds of subtle bugs and data loss. If we delete the DB
> field, we need to actually stop serving /api/3.x.
>
> > copy over all the data in the existing LD1 and LD2 fields, and append it
> to the LD field, and drop the LD! And LD2 fields from the table.
>
> I'm not so sure about this, though. If someone has a specific format,
> key-value, JSON, etc, just joining the fields is going to break their
> scripts. It might be better to make the upgrade fail if the field isn't
> empty, requiring users to handle and move or delete the data themselves
> before upgrading.
>
>
> On Mon, Jun 28, 2021 at 3:56 PM Chatterjee, Srijeet
> <SR...@comcast.com.invalid> wrote:
>
> > Hi all,
> > I’m proposing a change to the DeliveryService structure and database
> > table, to remove the “Long Description 1” and “Long Description 2”
> fields.
> > These fields are seldom used, and can be replaced by the already existing
> > “Long Description” field. In my migration, I’m planning to copy over all
> > the data in the existing LD1 and LD2 fields, and append it to the LD
> field,
> > and drop the LD! And LD2 fields from the table. Pls let me know if you
> have
> > concerns with this change, or if you have a better way of doing this.
> >
> > Thanks,
> > Srijeet
> >
>

Re: Removal of "Long Description 1" and "Long Description 2"

[Robert O Butts <ro...@apache.org>]

I'm +1 on getting rid of those fields, but unfortunately we need to support
them in older APIs. People have scripts that rely on them, and parse magic
data they've put in those fields.

We can remove the field from the API in newer major versions. But we need
to somehow still serve it in the existing API versions (which will need
supported for at least 1 major ATC version after the deprecation is
announced).

I agree, they're terrible and need removed. But I know for a fact there are
people putting arbitrary data in them in Production systems, which they
parse with custom scripts hitting the API. Even if we didn't know that for
sure, we still need to follow the Deprecation/removal process, to avoid
breaking users who don't necessarily follow the mailing lists closely.

So, ATC 6.0 is about to be released, which introduces API 4.0. If we remove
`longDesc2` and `longDesc3` from API 4.0 and declare API 3.x deprecated in
ATC 6.0, we can remove API 3.x and older in ATC 7.0. Which will allow us to
delete the database fields in ATC 7.0.

If we actually delete the database fields, we need to stop serving API 3.x
and older. We historically haven't removed API versions as often as we
could, mostly because it's painful to users. But serving a half-baked API
will cause all kinds of subtle bugs and data loss. If we delete the DB
field, we need to actually stop serving /api/3.x.

> copy over all the data in the existing LD1 and LD2 fields, and append it
to the LD field, and drop the LD! And LD2 fields from the table.

I'm not so sure about this, though. If someone has a specific format,
key-value, JSON, etc, just joining the fields is going to break their
scripts. It might be better to make the upgrade fail if the field isn't
empty, requiring users to handle and move or delete the data themselves
before upgrading.


On Mon, Jun 28, 2021 at 3:56 PM Chatterjee, Srijeet
<SR...@comcast.com.invalid> wrote:

> Hi all,
> I’m proposing a change to the DeliveryService structure and database
> table, to remove the “Long Description 1” and “Long Description 2” fields.
> These fields are seldom used, and can be replaced by the already existing
> “Long Description” field. In my migration, I’m planning to copy over all
> the data in the existing LD1 and LD2 fields, and append it to the LD field,
> and drop the LD! And LD2 fields from the table. Pls let me know if you have
> concerns with this change, or if you have a better way of doing this.
>
> Thanks,
> Srijeet
>