You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by "Gray, Jonathan" <Jo...@comcast.com> on 2018/11/20 02:48:08 UTC
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
I'm -1 depending on what the intended use case is.
Generic text fields should only be useful to human operators. In the case where you intend anything to programmatically access that information and it's generally useful, you're better off with specific columns per point of data. This is how we ended up with unparsable, yet critical, data in the comment fields of physical location table when we should have added real columns. The example in the issue is delivery services. The description field that I think is being referenced is one of LongDesc, LongDesc_1, or LongDesc_2 in the database. Columns should have one purpose and one meaning that is clear to a new developer working in the code and conceptually plausible to anyone else trying to understand how the system works. One compromise, I'm not a huge fan of, would be to allow for arbitrary structured data via a column of type jsonb instead of text. That's not a great answer from a usability or db theory perspective, but it's slightly better than regex parsing.
Jonathan G
On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
+1, I am fine with it. That table already has a lot of columns, what's one
more!?
On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <mi...@gmail.com>
wrote:
> Sounds like server "notes" or a server "description". Seems like a fair
> ask. I don't see the harm in adding an optional column to the server table
> with type=text for this data.
>
> On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <ty...@husky.neu.edu> wrote:
>
> > Hello Traffic Controllers,
> >
> > I discussed this with a couple of ATC users. We have multiple
> > Description/text fields in Delivery Service configuration of TP.
> Similarly,
> > We should also have one text field in servers configuration. My use case
> is
> > to keep the service/serial id of the servers and any specific info for a
> > server for which no field is available.
> >
> > I have created an issue on GitHub for it earlier:
> > https://github.com/apache/trafficcontrol/issues/2764
> > <https://github.com/apache/trafficcontrol/issues/2764>
> >
> > It's not a major change so shouldn't be a problem. If everyone agrees,
> I'd
> > be interested to add that.
> >
> > Thank you
> > Anuj Tyagi
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
So the compromise of a jsonb column only partially addresses the data modeling issue. In my brain it should only be used for non-universal concepts like my Kabletown CDN identifies server assets using FINBINID string that would make no sense to anyone but me in my Kabletown CDN. Even if you use it for this purpose, you're still left with a tough issue when it comes to UI design to facilitate building out these fields that doesn't involve actually hand writing json.
Something else I just noticed is that we have a hwInfo table that's not the same thing as the server table; yet is linked to it. I'm seen other solutions which involved using views to represent forms which then use custom insert functions or strict adherence to certain database rules so that updatable views work.
Jonathan G
On 11/20/18, 10:52 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
column. Then it is up to whatever downstream components might be using
the column to make sure it works for their purposes. In the API we
could just validate that it's actually valid JSON, and I don't think
we'd need any other validation on it. Then it would be up to the
operator to come up with their own versioned JSON schema for the jsonb
column for their downstream components to use. Editing JSON directly
via TP is not ideal, but I'm sure there could be a way to generate a
usable input form given a JSON schema. The schema would be
version-controlled and placed down with TP via whatever configuration
management tool you're using. Then you can easily add whatever
arbitrary data you want to add to a server without polluting the DB
with multiple columns that aren't actually consumed by TC components.
Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
fit for that I think. Those columns aren't consumed by other TC
components directly and are mostly just informational or consumed by
things outside of Traffic Control, so maybe they'd be better off as
second-class citizens.
So basically this:
If the new column is consumed by a TC component for the purpose of
control flow, it deserves a first-class column in the DB. If the new
field is not consumed by TC components, meant to be consumed by non-TC
components (e.g. configuration management), or is purely informational
for human consumption, it can just be relegated to the jsonb column.
What do you all think?
- Rawlin
On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> DS lifecycle tracking was supposed to modeled via DSR Comments. Also, we do have 3 existing comment fields on a DS. They've all been co-opted for other various purposes already. That's why they use different names in TP than in the database.
>
> Jonathan G
>
>
> On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
>
> Right, we do... but try reading or writing paragraphs of info in them. The
> DB fields may support it, but the UI not so much.
>
> __Jason
>
> On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <Br...@comcast.com>
> wrote:
>
> > We have three such fields for Delivery Services, afaik :P
> > ________________________________________
> > From: Jason Tucker <ja...@gmail.com>
> > Sent: Tuesday, November 20, 2018 7:25 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
> >
> > I'm actually a fan of arbitrary text boxes for more than just server
> > objects. I've been hoping for something like this in delivery service
> > objects as well, as this sort of field can be used to help document
> > unusual/custom/snowflake behavior which may not necessarily be obvious to
> > those who come later with the intention of troubleshooting. Should be used
> > for communicating with humans, rather than systems. In lieu of versioning
> > of configs, it could be used to keep a change log for the object as well.
> > But, again, that's more applicable to DS objects rather than Server
> > objects, I think.
> >
> > __Jason
> >
> > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <Jo...@comcast.com>
> > wrote:
> >
> > > I'm -1 depending on what the intended use case is.
> > >
> > > Generic text fields should only be useful to human operators. In the
> > case
> > > where you intend anything to programmatically access that information and
> > > it's generally useful, you're better off with specific columns per point
> > of
> > > data. This is how we ended up with unparsable, yet critical, data in the
> > > comment fields of physical location table when we should have added real
> > > columns. The example in the issue is delivery services. The description
> > > field that I think is being referenced is one of LongDesc, LongDesc_1, or
> > > LongDesc_2 in the database. Columns should have one purpose and one
> > > meaning that is clear to a new developer working in the code and
> > > conceptually plausible to anyone else trying to understand how the system
> > > works. One compromise, I'm not a huge fan of, would be to allow for
> > > arbitrary structured data via a column of type jsonb instead of text.
> > > That's not a great answer from a usability or db theory perspective, but
> > > it's slightly better than regex parsing.
> > >
> > > Jonathan G
> > >
> > >
> > >
> > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > >
> > > +1, I am fine with it. That table already has a lot of columns,
> > > what's one
> > > more!?
> > >
> > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> > mitchell852@gmail.com
> > > >
> > > wrote:
> > >
> > > > Sounds like server "notes" or a server "description". Seems like a
> > > fair
> > > > ask. I don't see the harm in adding an optional column to the
> > server
> > > table
> > > > with type=text for this data.
> > > >
> > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <tyagi.an@husky.neu.edu
> > >
> > > wrote:
> > > >
> > > > > Hello Traffic Controllers,
> > > > >
> > > > > I discussed this with a couple of ATC users. We have multiple
> > > > > Description/text fields in Delivery Service configuration of TP.
> > > > Similarly,
> > > > > We should also have one text field in servers configuration. My
> > > use case
> > > > is
> > > > > to keep the service/serial id of the servers and any specific
> > info
> > > for a
> > > > > server for which no field is available.
> > > > >
> > > > > I have created an issue on GitHub for it earlier:
> > > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > > >
> > > > > It's not a major change so shouldn't be a problem. If everyone
> > > agrees,
> > > > I'd
> > > > > be interested to add that.
> > > > >
> > > > > Thank you
> > > > > Anuj Tyagi
> > > > >
> > > >
> > >
> > >
> > >
> >
>
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Jeff Elsloo <el...@apache.org>.
I agree with this last email from Rawlin.
I don't think that imposing structure here will gain us very much, and
one can still save JSON/whatever in the proposed field without the
additional complexity as Rawlin pointed out. In the discussion thus
far, we used a lot of prior examples as reasons to not do what Anuj
proposed, yet none of the prior examples actually do what he needs.
Instead, they showed how we have been undisciplined in the past with
our schema, and were used as reasons to not move forward. I don't
agree with that sentiment.
Specifically, Anuj would like a text field to store "service/serial id
of the servers and any specific info for a server for which no field
is available." The key part of this is the second half, not the first
half, which we could clearly accomplish with existing fields. I view
this specifically as a place to store data "for which no field is
available." I think that's a reasonable request. How this field is
used, or not, is solely up to the operator of the CDN.
If we have problems with the schema generally, instead of being -1 on
any new changes, we should evaluate the two issues separately. If we
don't like how the various fields or other aspects of the schema work,
then let's work toward resolving that instead of being -1 on new ideas
that may provide tangible benefit for our users. Perhaps this field
would need to move elsewhere if we refactored the server table in the
future, but let's not let that prevent us from moving forward today.
In general I'm fine with this feature request. I'm going to refrain
from using a +1 since this isn't a real vote. :)
--
Thanks,
Jeff
On Tue, Nov 20, 2018 at 3:13 PM Rawlin Peters <ra...@gmail.com> wrote:
>
> I think there's naturally going to be a lot of data overlap (IP
> addresses, hostnames, etc) between TO and whatever asset management
> system you're using for all your CDN servers, so I can see the benefit
> of trying to just stick it all in one place (TO) rather than having to
> deal with importing/exporting data to/from multiple asset management
> systems and keeping the data synced across all the systems.
>
> However, maybe we shouldn't require the field to be a json blob. If
> it's just an arbitrary text "description" field, you can really put
> whatever your heart desires into it (json, key/value pairs,
> base64-encoded data, etc). Then as a project we're not on the hook for
> making it easy to edit arbitrary json via the Traffic Portal UI, but
> end users can still build whatever they want into those fields for
> their own purposes.
>
> - Rawlin
>
> On Tue, Nov 20, 2018 at 2:36 PM Robert Butts <ro...@apache.org> wrote:
> >
> > This sounds like an Asset Management thing, not a CDN thing. We currently
> > have a number of "Asset Management" fields (like ASNs) which aren't
> > necessary for the CDN itself, and just asset tracking. Personally, I'd
> > rather see us move toward the CDN being a CDN, and operators using an Asset
> > Management system if they need one. Separation of Concerns and all that.
> >
> > The first hit searching for "open source asset management system" is
> > https://snipeitapp.com - but there are many. Shouldn't be too hard for an
> > operator to put TC names and IDs in one for cross-reference.
> >
> > Likewise for things like Ansible variables. Seems like the TO database
> > should be the TO/TC database, and data for other systems should be
> > maintained by them. Rather than shoving everything from every system into
> > the CDN database.
> >
> > But that's my opinion. If the quorum disagrees, it's not a hill I'm going
> > to die on.
> >
> > On Tue, Nov 20, 2018 at 2:30 PM Steve Malenfant <sm...@gmail.com>
> > wrote:
> >
> > > +1 json would help us with Ansible as well to store various variables.
> > > would be nice to have by cache group, servers, type... maybe not possible.
> > >
> > > On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
> > > wrote:
> > >
> > > > I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
> > > > column. Then it is up to whatever downstream components might be using
> > > > the column to make sure it works for their purposes. In the API we
> > > > could just validate that it's actually valid JSON, and I don't think
> > > > we'd need any other validation on it. Then it would be up to the
> > > > operator to come up with their own versioned JSON schema for the jsonb
> > > > column for their downstream components to use. Editing JSON directly
> > > > via TP is not ideal, but I'm sure there could be a way to generate a
> > > > usable input form given a JSON schema. The schema would be
> > > > version-controlled and placed down with TP via whatever configuration
> > > > management tool you're using. Then you can easily add whatever
> > > > arbitrary data you want to add to a server without polluting the DB
> > > > with multiple columns that aren't actually consumed by TC components.
> > > > Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
> > > > fit for that I think. Those columns aren't consumed by other TC
> > > > components directly and are mostly just informational or consumed by
> > > > things outside of Traffic Control, so maybe they'd be better off as
> > > > second-class citizens.
> > > >
> > > > So basically this:
> > > > If the new column is consumed by a TC component for the purpose of
> > > > control flow, it deserves a first-class column in the DB. If the new
> > > > field is not consumed by TC components, meant to be consumed by non-TC
> > > > components (e.g. configuration management), or is purely informational
> > > > for human consumption, it can just be relegated to the jsonb column.
> > > >
> > > > What do you all think?
> > > >
> > > > - Rawlin
> > > >
> > > > On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
> > > > <Jo...@comcast.com> wrote:
> > > > >
> > > > > DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
> > > > we do have 3 existing comment fields on a DS. They've all been co-opted
> > > > for other various purposes already. That's why they use different names
> > > in
> > > > TP than in the database.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > >
> > > > > On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
> > > > >
> > > > > Right, we do... but try reading or writing paragraphs of info in
> > > > them. The
> > > > > DB fields may support it, but the UI not so much.
> > > > >
> > > > > __Jason
> > > > >
> > > > > On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
> > > > Brennan_Fieck@comcast.com>
> > > > > wrote:
> > > > >
> > > > > > We have three such fields for Delivery Services, afaik :P
> > > > > > ________________________________________
> > > > > > From: Jason Tucker <ja...@gmail.com>
> > > > > > Sent: Tuesday, November 20, 2018 7:25 AM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config
> > > > of TP
> > > > > >
> > > > > > I'm actually a fan of arbitrary text boxes for more than just
> > > > server
> > > > > > objects. I've been hoping for something like this in delivery
> > > > service
> > > > > > objects as well, as this sort of field can be used to help
> > > document
> > > > > > unusual/custom/snowflake behavior which may not necessarily be
> > > > obvious to
> > > > > > those who come later with the intention of troubleshooting.
> > > Should
> > > > be used
> > > > > > for communicating with humans, rather than systems. In lieu of
> > > > versioning
> > > > > > of configs, it could be used to keep a change log for the object
> > > > as well.
> > > > > > But, again, that's more applicable to DS objects rather than
> > > Server
> > > > > > objects, I think.
> > > > > >
> > > > > > __Jason
> > > > > >
> > > > > > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
> > > > Jonathan_Gray@comcast.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I'm -1 depending on what the intended use case is.
> > > > > > >
> > > > > > > Generic text fields should only be useful to human operators.
> > > > In the
> > > > > > case
> > > > > > > where you intend anything to programmatically access that
> > > > information and
> > > > > > > it's generally useful, you're better off with specific columns
> > > > per point
> > > > > > of
> > > > > > > data. This is how we ended up with unparsable, yet critical,
> > > > data in the
> > > > > > > comment fields of physical location table when we should have
> > > > added real
> > > > > > > columns. The example in the issue is delivery services. The
> > > > description
> > > > > > > field that I think is being referenced is one of LongDesc,
> > > > LongDesc_1, or
> > > > > > > LongDesc_2 in the database. Columns should have one purpose
> > > and
> > > > one
> > > > > > > meaning that is clear to a new developer working in the code
> > > and
> > > > > > > conceptually plausible to anyone else trying to understand how
> > > > the system
> > > > > > > works. One compromise, I'm not a huge fan of, would be to
> > > allow
> > > > for
> > > > > > > arbitrary structured data via a column of type jsonb instead of
> > > > text.
> > > > > > > That's not a great answer from a usability or db theory
> > > > perspective, but
> > > > > > > it's slightly better than regex parsing.
> > > > > > >
> > > > > > > Jonathan G
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > > > > > >
> > > > > > > +1, I am fine with it. That table already has a lot of
> > > > columns,
> > > > > > > what's one
> > > > > > > more!?
> > > > > > >
> > > > > > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> > > > > > mitchell852@gmail.com
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Sounds like server "notes" or a server "description".
> > > > Seems like a
> > > > > > > fair
> > > > > > > > ask. I don't see the harm in adding an optional column to
> > > > the
> > > > > > server
> > > > > > > table
> > > > > > > > with type=text for this data.
> > > > > > > >
> > > > > > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
> > > > tyagi.an@husky.neu.edu
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello Traffic Controllers,
> > > > > > > > >
> > > > > > > > > I discussed this with a couple of ATC users. We have
> > > > multiple
> > > > > > > > > Description/text fields in Delivery Service
> > > > configuration of TP.
> > > > > > > > Similarly,
> > > > > > > > > We should also have one text field in servers
> > > > configuration. My
> > > > > > > use case
> > > > > > > > is
> > > > > > > > > to keep the service/serial id of the servers and any
> > > > specific
> > > > > > info
> > > > > > > for a
> > > > > > > > > server for which no field is available.
> > > > > > > > >
> > > > > > > > > I have created an issue on GitHub for it earlier:
> > > > > > > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > > > > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > > > > > > >
> > > > > > > > > It's not a major change so shouldn't be a problem. If
> > > > everyone
> > > > > > > agrees,
> > > > > > > > I'd
> > > > > > > > > be interested to add that.
> > > > > > > > >
> > > > > > > > > Thank you
> > > > > > > > > Anuj Tyagi
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Rawlin Peters <ra...@gmail.com>.
I think there's naturally going to be a lot of data overlap (IP
addresses, hostnames, etc) between TO and whatever asset management
system you're using for all your CDN servers, so I can see the benefit
of trying to just stick it all in one place (TO) rather than having to
deal with importing/exporting data to/from multiple asset management
systems and keeping the data synced across all the systems.
However, maybe we shouldn't require the field to be a json blob. If
it's just an arbitrary text "description" field, you can really put
whatever your heart desires into it (json, key/value pairs,
base64-encoded data, etc). Then as a project we're not on the hook for
making it easy to edit arbitrary json via the Traffic Portal UI, but
end users can still build whatever they want into those fields for
their own purposes.
- Rawlin
On Tue, Nov 20, 2018 at 2:36 PM Robert Butts <ro...@apache.org> wrote:
>
> This sounds like an Asset Management thing, not a CDN thing. We currently
> have a number of "Asset Management" fields (like ASNs) which aren't
> necessary for the CDN itself, and just asset tracking. Personally, I'd
> rather see us move toward the CDN being a CDN, and operators using an Asset
> Management system if they need one. Separation of Concerns and all that.
>
> The first hit searching for "open source asset management system" is
> https://snipeitapp.com - but there are many. Shouldn't be too hard for an
> operator to put TC names and IDs in one for cross-reference.
>
> Likewise for things like Ansible variables. Seems like the TO database
> should be the TO/TC database, and data for other systems should be
> maintained by them. Rather than shoving everything from every system into
> the CDN database.
>
> But that's my opinion. If the quorum disagrees, it's not a hill I'm going
> to die on.
>
> On Tue, Nov 20, 2018 at 2:30 PM Steve Malenfant <sm...@gmail.com>
> wrote:
>
> > +1 json would help us with Ansible as well to store various variables.
> > would be nice to have by cache group, servers, type... maybe not possible.
> >
> > On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
> > wrote:
> >
> > > I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
> > > column. Then it is up to whatever downstream components might be using
> > > the column to make sure it works for their purposes. In the API we
> > > could just validate that it's actually valid JSON, and I don't think
> > > we'd need any other validation on it. Then it would be up to the
> > > operator to come up with their own versioned JSON schema for the jsonb
> > > column for their downstream components to use. Editing JSON directly
> > > via TP is not ideal, but I'm sure there could be a way to generate a
> > > usable input form given a JSON schema. The schema would be
> > > version-controlled and placed down with TP via whatever configuration
> > > management tool you're using. Then you can easily add whatever
> > > arbitrary data you want to add to a server without polluting the DB
> > > with multiple columns that aren't actually consumed by TC components.
> > > Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
> > > fit for that I think. Those columns aren't consumed by other TC
> > > components directly and are mostly just informational or consumed by
> > > things outside of Traffic Control, so maybe they'd be better off as
> > > second-class citizens.
> > >
> > > So basically this:
> > > If the new column is consumed by a TC component for the purpose of
> > > control flow, it deserves a first-class column in the DB. If the new
> > > field is not consumed by TC components, meant to be consumed by non-TC
> > > components (e.g. configuration management), or is purely informational
> > > for human consumption, it can just be relegated to the jsonb column.
> > >
> > > What do you all think?
> > >
> > > - Rawlin
> > >
> > > On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
> > > <Jo...@comcast.com> wrote:
> > > >
> > > > DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
> > > we do have 3 existing comment fields on a DS. They've all been co-opted
> > > for other various purposes already. That's why they use different names
> > in
> > > TP than in the database.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > > On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
> > > >
> > > > Right, we do... but try reading or writing paragraphs of info in
> > > them. The
> > > > DB fields may support it, but the UI not so much.
> > > >
> > > > __Jason
> > > >
> > > > On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
> > > Brennan_Fieck@comcast.com>
> > > > wrote:
> > > >
> > > > > We have three such fields for Delivery Services, afaik :P
> > > > > ________________________________________
> > > > > From: Jason Tucker <ja...@gmail.com>
> > > > > Sent: Tuesday, November 20, 2018 7:25 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config
> > > of TP
> > > > >
> > > > > I'm actually a fan of arbitrary text boxes for more than just
> > > server
> > > > > objects. I've been hoping for something like this in delivery
> > > service
> > > > > objects as well, as this sort of field can be used to help
> > document
> > > > > unusual/custom/snowflake behavior which may not necessarily be
> > > obvious to
> > > > > those who come later with the intention of troubleshooting.
> > Should
> > > be used
> > > > > for communicating with humans, rather than systems. In lieu of
> > > versioning
> > > > > of configs, it could be used to keep a change log for the object
> > > as well.
> > > > > But, again, that's more applicable to DS objects rather than
> > Server
> > > > > objects, I think.
> > > > >
> > > > > __Jason
> > > > >
> > > > > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
> > > Jonathan_Gray@comcast.com>
> > > > > wrote:
> > > > >
> > > > > > I'm -1 depending on what the intended use case is.
> > > > > >
> > > > > > Generic text fields should only be useful to human operators.
> > > In the
> > > > > case
> > > > > > where you intend anything to programmatically access that
> > > information and
> > > > > > it's generally useful, you're better off with specific columns
> > > per point
> > > > > of
> > > > > > data. This is how we ended up with unparsable, yet critical,
> > > data in the
> > > > > > comment fields of physical location table when we should have
> > > added real
> > > > > > columns. The example in the issue is delivery services. The
> > > description
> > > > > > field that I think is being referenced is one of LongDesc,
> > > LongDesc_1, or
> > > > > > LongDesc_2 in the database. Columns should have one purpose
> > and
> > > one
> > > > > > meaning that is clear to a new developer working in the code
> > and
> > > > > > conceptually plausible to anyone else trying to understand how
> > > the system
> > > > > > works. One compromise, I'm not a huge fan of, would be to
> > allow
> > > for
> > > > > > arbitrary structured data via a column of type jsonb instead of
> > > text.
> > > > > > That's not a great answer from a usability or db theory
> > > perspective, but
> > > > > > it's slightly better than regex parsing.
> > > > > >
> > > > > > Jonathan G
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > > > > >
> > > > > > +1, I am fine with it. That table already has a lot of
> > > columns,
> > > > > > what's one
> > > > > > more!?
> > > > > >
> > > > > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> > > > > mitchell852@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Sounds like server "notes" or a server "description".
> > > Seems like a
> > > > > > fair
> > > > > > > ask. I don't see the harm in adding an optional column to
> > > the
> > > > > server
> > > > > > table
> > > > > > > with type=text for this data.
> > > > > > >
> > > > > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
> > > tyagi.an@husky.neu.edu
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Traffic Controllers,
> > > > > > > >
> > > > > > > > I discussed this with a couple of ATC users. We have
> > > multiple
> > > > > > > > Description/text fields in Delivery Service
> > > configuration of TP.
> > > > > > > Similarly,
> > > > > > > > We should also have one text field in servers
> > > configuration. My
> > > > > > use case
> > > > > > > is
> > > > > > > > to keep the service/serial id of the servers and any
> > > specific
> > > > > info
> > > > > > for a
> > > > > > > > server for which no field is available.
> > > > > > > >
> > > > > > > > I have created an issue on GitHub for it earlier:
> > > > > > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > > > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > > > > > >
> > > > > > > > It's not a major change so shouldn't be a problem. If
> > > everyone
> > > > > > agrees,
> > > > > > > I'd
> > > > > > > > be interested to add that.
> > > > > > > >
> > > > > > > > Thank you
> > > > > > > > Anuj Tyagi
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Robert Butts <ro...@apache.org>.
This sounds like an Asset Management thing, not a CDN thing. We currently
have a number of "Asset Management" fields (like ASNs) which aren't
necessary for the CDN itself, and just asset tracking. Personally, I'd
rather see us move toward the CDN being a CDN, and operators using an Asset
Management system if they need one. Separation of Concerns and all that.
The first hit searching for "open source asset management system" is
https://snipeitapp.com - but there are many. Shouldn't be too hard for an
operator to put TC names and IDs in one for cross-reference.
Likewise for things like Ansible variables. Seems like the TO database
should be the TO/TC database, and data for other systems should be
maintained by them. Rather than shoving everything from every system into
the CDN database.
But that's my opinion. If the quorum disagrees, it's not a hill I'm going
to die on.
On Tue, Nov 20, 2018 at 2:30 PM Steve Malenfant <sm...@gmail.com>
wrote:
> +1 json would help us with Ansible as well to store various variables.
> would be nice to have by cache group, servers, type... maybe not possible.
>
> On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
> wrote:
>
> > I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
> > column. Then it is up to whatever downstream components might be using
> > the column to make sure it works for their purposes. In the API we
> > could just validate that it's actually valid JSON, and I don't think
> > we'd need any other validation on it. Then it would be up to the
> > operator to come up with their own versioned JSON schema for the jsonb
> > column for their downstream components to use. Editing JSON directly
> > via TP is not ideal, but I'm sure there could be a way to generate a
> > usable input form given a JSON schema. The schema would be
> > version-controlled and placed down with TP via whatever configuration
> > management tool you're using. Then you can easily add whatever
> > arbitrary data you want to add to a server without polluting the DB
> > with multiple columns that aren't actually consumed by TC components.
> > Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
> > fit for that I think. Those columns aren't consumed by other TC
> > components directly and are mostly just informational or consumed by
> > things outside of Traffic Control, so maybe they'd be better off as
> > second-class citizens.
> >
> > So basically this:
> > If the new column is consumed by a TC component for the purpose of
> > control flow, it deserves a first-class column in the DB. If the new
> > field is not consumed by TC components, meant to be consumed by non-TC
> > components (e.g. configuration management), or is purely informational
> > for human consumption, it can just be relegated to the jsonb column.
> >
> > What do you all think?
> >
> > - Rawlin
> >
> > On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
> > <Jo...@comcast.com> wrote:
> > >
> > > DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
> > we do have 3 existing comment fields on a DS. They've all been co-opted
> > for other various purposes already. That's why they use different names
> in
> > TP than in the database.
> > >
> > > Jonathan G
> > >
> > >
> > > On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
> > >
> > > Right, we do... but try reading or writing paragraphs of info in
> > them. The
> > > DB fields may support it, but the UI not so much.
> > >
> > > __Jason
> > >
> > > On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com>
> > > wrote:
> > >
> > > > We have three such fields for Delivery Services, afaik :P
> > > > ________________________________________
> > > > From: Jason Tucker <ja...@gmail.com>
> > > > Sent: Tuesday, November 20, 2018 7:25 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config
> > of TP
> > > >
> > > > I'm actually a fan of arbitrary text boxes for more than just
> > server
> > > > objects. I've been hoping for something like this in delivery
> > service
> > > > objects as well, as this sort of field can be used to help
> document
> > > > unusual/custom/snowflake behavior which may not necessarily be
> > obvious to
> > > > those who come later with the intention of troubleshooting.
> Should
> > be used
> > > > for communicating with humans, rather than systems. In lieu of
> > versioning
> > > > of configs, it could be used to keep a change log for the object
> > as well.
> > > > But, again, that's more applicable to DS objects rather than
> Server
> > > > objects, I think.
> > > >
> > > > __Jason
> > > >
> > > > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
> > Jonathan_Gray@comcast.com>
> > > > wrote:
> > > >
> > > > > I'm -1 depending on what the intended use case is.
> > > > >
> > > > > Generic text fields should only be useful to human operators.
> > In the
> > > > case
> > > > > where you intend anything to programmatically access that
> > information and
> > > > > it's generally useful, you're better off with specific columns
> > per point
> > > > of
> > > > > data. This is how we ended up with unparsable, yet critical,
> > data in the
> > > > > comment fields of physical location table when we should have
> > added real
> > > > > columns. The example in the issue is delivery services. The
> > description
> > > > > field that I think is being referenced is one of LongDesc,
> > LongDesc_1, or
> > > > > LongDesc_2 in the database. Columns should have one purpose
> and
> > one
> > > > > meaning that is clear to a new developer working in the code
> and
> > > > > conceptually plausible to anyone else trying to understand how
> > the system
> > > > > works. One compromise, I'm not a huge fan of, would be to
> allow
> > for
> > > > > arbitrary structured data via a column of type jsonb instead of
> > text.
> > > > > That's not a great answer from a usability or db theory
> > perspective, but
> > > > > it's slightly better than regex parsing.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > >
> > > > >
> > > > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > > > >
> > > > > +1, I am fine with it. That table already has a lot of
> > columns,
> > > > > what's one
> > > > > more!?
> > > > >
> > > > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> > > > mitchell852@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Sounds like server "notes" or a server "description".
> > Seems like a
> > > > > fair
> > > > > > ask. I don't see the harm in adding an optional column to
> > the
> > > > server
> > > > > table
> > > > > > with type=text for this data.
> > > > > >
> > > > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
> > tyagi.an@husky.neu.edu
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > > Hello Traffic Controllers,
> > > > > > >
> > > > > > > I discussed this with a couple of ATC users. We have
> > multiple
> > > > > > > Description/text fields in Delivery Service
> > configuration of TP.
> > > > > > Similarly,
> > > > > > > We should also have one text field in servers
> > configuration. My
> > > > > use case
> > > > > > is
> > > > > > > to keep the service/serial id of the servers and any
> > specific
> > > > info
> > > > > for a
> > > > > > > server for which no field is available.
> > > > > > >
> > > > > > > I have created an issue on GitHub for it earlier:
> > > > > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > > > > >
> > > > > > > It's not a major change so shouldn't be a problem. If
> > everyone
> > > > > agrees,
> > > > > > I'd
> > > > > > > be interested to add that.
> > > > > > >
> > > > > > > Thank you
> > > > > > > Anuj Tyagi
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
Given only two options I'd go for the second one. We shouldn't be afraid to model our data in a database, especially since we're the driving force between every supported application that touches it. Adding more logic and safeties inside the TO API is just reimplementing the job that a database already does exceptionally well.
Jonathan G
On 11/26/18, 8:32 AM, "Dave Neuman" <ne...@apache.org> wrote:
This thread has died down a bit, but I don’t think we came to any real
consensus, so I going to attempt to get some consensus.
I think we have a need for two things: the ability to store serial numbers
for a server and the ability to store “notes” for a server.
The way I see it there are two ways (maybe more?) we can solve this without
getting too crazy:
The first way to solve this would be to use the hwinfo table. The hwinfo
table gives users the ability to store key/value pairs (in columns named
description and val) and has a foreign key to a server object. We can add
some fields to Traffic Portal and the API that store\get serial number and
notes keys in the hwinfo table and assign the foreign key to the server
object. Obviously using strings and key/value pairs is not the most ideal
way to do things, but if we implement the API correctly, we could be
reasonably safe in relying on the strings. The real benefit to this
solution is that it would not require and changes to the database schema,
just a few API and UI changes. For what it’s worth we have created a
precedent for doing this with profiles/parameters.
The other way we can accomplish this is to add two new columns to the
server table and make them available in the UI and API. The first column
would be serial_number and would have to be a varchar(n) (varchar(30)?),
but we could use a regex constraint to limit it to alphanumeric characters.
The second would be “note” and that would probably have to be a text field.
I personally prefer the first option myself, but we need to get some
consensus. Any thoughts? Other reasonable suggestions?
Unfortunately we can’t re-factor the whole database schema just because we
want to add a few new fields.
Thanks,
Dave
On Wed, Nov 21, 2018 at 5:28 PM Hank Beatty <hb...@apache.org> wrote:
> I'm +1 on adding a jsonb as well. As far as editing... What about using
> the text field that Jan presented at the TC summit? It could be written
> in YAML and easily converted to JSON.
>
> -Hank
>
> On 11/20/2018 04:29 PM, Steve Malenfant wrote:
> > +1 json would help us with Ansible as well to store various variables.
> > would be nice to have by cache group, servers, type... maybe not
> possible.
> >
> > On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
> > wrote:
> >
> >> I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
> >> column. Then it is up to whatever downstream components might be using
> >> the column to make sure it works for their purposes. In the API we
> >> could just validate that it's actually valid JSON, and I don't think
> >> we'd need any other validation on it. Then it would be up to the
> >> operator to come up with their own versioned JSON schema for the jsonb
> >> column for their downstream components to use. Editing JSON directly
> >> via TP is not ideal, but I'm sure there could be a way to generate a
> >> usable input form given a JSON schema. The schema would be
> >> version-controlled and placed down with TP via whatever configuration
> >> management tool you're using. Then you can easily add whatever
> >> arbitrary data you want to add to a server without polluting the DB
> >> with multiple columns that aren't actually consumed by TC components.
> >> Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
> >> fit for that I think. Those columns aren't consumed by other TC
> >> components directly and are mostly just informational or consumed by
> >> things outside of Traffic Control, so maybe they'd be better off as
> >> second-class citizens.
> >>
> >> So basically this:
> >> If the new column is consumed by a TC component for the purpose of
> >> control flow, it deserves a first-class column in the DB. If the new
> >> field is not consumed by TC components, meant to be consumed by non-TC
> >> components (e.g. configuration management), or is purely informational
> >> for human consumption, it can just be relegated to the jsonb column.
> >>
> >> What do you all think?
> >>
> >> - Rawlin
> >>
> >> On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
> >> <Jo...@comcast.com> wrote:
> >>>
> >>> DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
> >> we do have 3 existing comment fields on a DS. They've all been co-opted
> >> for other various purposes already. That's why they use different
> names in
> >> TP than in the database.
> >>>
> >>> Jonathan G
> >>>
> >>>
> >>> On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
> >>>
> >>> Right, we do... but try reading or writing paragraphs of info in
> >> them. The
> >>> DB fields may support it, but the UI not so much.
> >>>
> >>> __Jason
> >>>
> >>> On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
> >> Brennan_Fieck@comcast.com>
> >>> wrote:
> >>>
> >>> > We have three such fields for Delivery Services, afaik :P
> >>> > ________________________________________
> >>> > From: Jason Tucker <ja...@gmail.com>
> >>> > Sent: Tuesday, November 20, 2018 7:25 AM
> >>> > To: dev@trafficcontrol.apache.org
> >>> > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers
> config
> >> of TP
> >>> >
> >>> > I'm actually a fan of arbitrary text boxes for more than just
> >> server
> >>> > objects. I've been hoping for something like this in delivery
> >> service
> >>> > objects as well, as this sort of field can be used to help
> document
> >>> > unusual/custom/snowflake behavior which may not necessarily be
> >> obvious to
> >>> > those who come later with the intention of troubleshooting.
> Should
> >> be used
> >>> > for communicating with humans, rather than systems. In lieu of
> >> versioning
> >>> > of configs, it could be used to keep a change log for the object
> >> as well.
> >>> > But, again, that's more applicable to DS objects rather than
> Server
> >>> > objects, I think.
> >>> >
> >>> > __Jason
> >>> >
> >>> > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
> >> Jonathan_Gray@comcast.com>
> >>> > wrote:
> >>> >
> >>> > > I'm -1 depending on what the intended use case is.
> >>> > >
> >>> > > Generic text fields should only be useful to human operators.
> >> In the
> >>> > case
> >>> > > where you intend anything to programmatically access that
> >> information and
> >>> > > it's generally useful, you're better off with specific columns
> >> per point
> >>> > of
> >>> > > data. This is how we ended up with unparsable, yet critical,
> >> data in the
> >>> > > comment fields of physical location table when we should have
> >> added real
> >>> > > columns. The example in the issue is delivery services. The
> >> description
> >>> > > field that I think is being referenced is one of LongDesc,
> >> LongDesc_1, or
> >>> > > LongDesc_2 in the database. Columns should have one purpose
> and
> >> one
> >>> > > meaning that is clear to a new developer working in the code
> and
> >>> > > conceptually plausible to anyone else trying to understand how
> >> the system
> >>> > > works. One compromise, I'm not a huge fan of, would be to
> allow
> >> for
> >>> > > arbitrary structured data via a column of type jsonb instead
> of
> >> text.
> >>> > > That's not a great answer from a usability or db theory
> >> perspective, but
> >>> > > it's slightly better than regex parsing.
> >>> > >
> >>> > > Jonathan G
> >>> > >
> >>> > >
> >>> > >
> >>> > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org>
> wrote:
> >>> > >
> >>> > > +1, I am fine with it. That table already has a lot of
> >> columns,
> >>> > > what's one
> >>> > > more!?
> >>> > >
> >>> > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> >>> > mitchell852@gmail.com
> >>> > > >
> >>> > > wrote:
> >>> > >
> >>> > > > Sounds like server "notes" or a server "description".
> >> Seems like a
> >>> > > fair
> >>> > > > ask. I don't see the harm in adding an optional column
> to
> >> the
> >>> > server
> >>> > > table
> >>> > > > with type=text for this data.
> >>> > > >
> >>> > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
> >> tyagi.an@husky.neu.edu
> >>> > >
> >>> > > wrote:
> >>> > > >
> >>> > > > > Hello Traffic Controllers,
> >>> > > > >
> >>> > > > > I discussed this with a couple of ATC users. We have
> >> multiple
> >>> > > > > Description/text fields in Delivery Service
> >> configuration of TP.
> >>> > > > Similarly,
> >>> > > > > We should also have one text field in servers
> >> configuration. My
> >>> > > use case
> >>> > > > is
> >>> > > > > to keep the service/serial id of the servers and any
> >> specific
> >>> > info
> >>> > > for a
> >>> > > > > server for which no field is available.
> >>> > > > >
> >>> > > > > I have created an issue on GitHub for it earlier:
> >>> > > > > https://github.com/apache/trafficcontrol/issues/2764
> >>> > > > > <https://github.com/apache/trafficcontrol/issues/2764
> >
> >>> > > > >
> >>> > > > > It's not a major change so shouldn't be a problem. If
> >> everyone
> >>> > > agrees,
> >>> > > > I'd
> >>> > > > > be interested to add that.
> >>> > > > >
> >>> > > > > Thank you
> >>> > > > > Anuj Tyagi
> >>> > > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> >
> >>>
> >>>
> >>
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Dave Neuman <ne...@apache.org>.
This thread has died down a bit, but I don’t think we came to any real
consensus, so I going to attempt to get some consensus.
I think we have a need for two things: the ability to store serial numbers
for a server and the ability to store “notes” for a server.
The way I see it there are two ways (maybe more?) we can solve this without
getting too crazy:
The first way to solve this would be to use the hwinfo table. The hwinfo
table gives users the ability to store key/value pairs (in columns named
description and val) and has a foreign key to a server object. We can add
some fields to Traffic Portal and the API that store\get serial number and
notes keys in the hwinfo table and assign the foreign key to the server
object. Obviously using strings and key/value pairs is not the most ideal
way to do things, but if we implement the API correctly, we could be
reasonably safe in relying on the strings. The real benefit to this
solution is that it would not require and changes to the database schema,
just a few API and UI changes. For what it’s worth we have created a
precedent for doing this with profiles/parameters.
The other way we can accomplish this is to add two new columns to the
server table and make them available in the UI and API. The first column
would be serial_number and would have to be a varchar(n) (varchar(30)?),
but we could use a regex constraint to limit it to alphanumeric characters.
The second would be “note” and that would probably have to be a text field.
I personally prefer the first option myself, but we need to get some
consensus. Any thoughts? Other reasonable suggestions?
Unfortunately we can’t re-factor the whole database schema just because we
want to add a few new fields.
Thanks,
Dave
On Wed, Nov 21, 2018 at 5:28 PM Hank Beatty <hb...@apache.org> wrote:
> I'm +1 on adding a jsonb as well. As far as editing... What about using
> the text field that Jan presented at the TC summit? It could be written
> in YAML and easily converted to JSON.
>
> -Hank
>
> On 11/20/2018 04:29 PM, Steve Malenfant wrote:
> > +1 json would help us with Ansible as well to store various variables.
> > would be nice to have by cache group, servers, type... maybe not
> possible.
> >
> > On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
> > wrote:
> >
> >> I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
> >> column. Then it is up to whatever downstream components might be using
> >> the column to make sure it works for their purposes. In the API we
> >> could just validate that it's actually valid JSON, and I don't think
> >> we'd need any other validation on it. Then it would be up to the
> >> operator to come up with their own versioned JSON schema for the jsonb
> >> column for their downstream components to use. Editing JSON directly
> >> via TP is not ideal, but I'm sure there could be a way to generate a
> >> usable input form given a JSON schema. The schema would be
> >> version-controlled and placed down with TP via whatever configuration
> >> management tool you're using. Then you can easily add whatever
> >> arbitrary data you want to add to a server without polluting the DB
> >> with multiple columns that aren't actually consumed by TC components.
> >> Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
> >> fit for that I think. Those columns aren't consumed by other TC
> >> components directly and are mostly just informational or consumed by
> >> things outside of Traffic Control, so maybe they'd be better off as
> >> second-class citizens.
> >>
> >> So basically this:
> >> If the new column is consumed by a TC component for the purpose of
> >> control flow, it deserves a first-class column in the DB. If the new
> >> field is not consumed by TC components, meant to be consumed by non-TC
> >> components (e.g. configuration management), or is purely informational
> >> for human consumption, it can just be relegated to the jsonb column.
> >>
> >> What do you all think?
> >>
> >> - Rawlin
> >>
> >> On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
> >> <Jo...@comcast.com> wrote:
> >>>
> >>> DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
> >> we do have 3 existing comment fields on a DS. They've all been co-opted
> >> for other various purposes already. That's why they use different
> names in
> >> TP than in the database.
> >>>
> >>> Jonathan G
> >>>
> >>>
> >>> On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
> >>>
> >>> Right, we do... but try reading or writing paragraphs of info in
> >> them. The
> >>> DB fields may support it, but the UI not so much.
> >>>
> >>> __Jason
> >>>
> >>> On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
> >> Brennan_Fieck@comcast.com>
> >>> wrote:
> >>>
> >>> > We have three such fields for Delivery Services, afaik :P
> >>> > ________________________________________
> >>> > From: Jason Tucker <ja...@gmail.com>
> >>> > Sent: Tuesday, November 20, 2018 7:25 AM
> >>> > To: dev@trafficcontrol.apache.org
> >>> > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers
> config
> >> of TP
> >>> >
> >>> > I'm actually a fan of arbitrary text boxes for more than just
> >> server
> >>> > objects. I've been hoping for something like this in delivery
> >> service
> >>> > objects as well, as this sort of field can be used to help
> document
> >>> > unusual/custom/snowflake behavior which may not necessarily be
> >> obvious to
> >>> > those who come later with the intention of troubleshooting.
> Should
> >> be used
> >>> > for communicating with humans, rather than systems. In lieu of
> >> versioning
> >>> > of configs, it could be used to keep a change log for the object
> >> as well.
> >>> > But, again, that's more applicable to DS objects rather than
> Server
> >>> > objects, I think.
> >>> >
> >>> > __Jason
> >>> >
> >>> > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
> >> Jonathan_Gray@comcast.com>
> >>> > wrote:
> >>> >
> >>> > > I'm -1 depending on what the intended use case is.
> >>> > >
> >>> > > Generic text fields should only be useful to human operators.
> >> In the
> >>> > case
> >>> > > where you intend anything to programmatically access that
> >> information and
> >>> > > it's generally useful, you're better off with specific columns
> >> per point
> >>> > of
> >>> > > data. This is how we ended up with unparsable, yet critical,
> >> data in the
> >>> > > comment fields of physical location table when we should have
> >> added real
> >>> > > columns. The example in the issue is delivery services. The
> >> description
> >>> > > field that I think is being referenced is one of LongDesc,
> >> LongDesc_1, or
> >>> > > LongDesc_2 in the database. Columns should have one purpose
> and
> >> one
> >>> > > meaning that is clear to a new developer working in the code
> and
> >>> > > conceptually plausible to anyone else trying to understand how
> >> the system
> >>> > > works. One compromise, I'm not a huge fan of, would be to
> allow
> >> for
> >>> > > arbitrary structured data via a column of type jsonb instead
> of
> >> text.
> >>> > > That's not a great answer from a usability or db theory
> >> perspective, but
> >>> > > it's slightly better than regex parsing.
> >>> > >
> >>> > > Jonathan G
> >>> > >
> >>> > >
> >>> > >
> >>> > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org>
> wrote:
> >>> > >
> >>> > > +1, I am fine with it. That table already has a lot of
> >> columns,
> >>> > > what's one
> >>> > > more!?
> >>> > >
> >>> > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> >>> > mitchell852@gmail.com
> >>> > > >
> >>> > > wrote:
> >>> > >
> >>> > > > Sounds like server "notes" or a server "description".
> >> Seems like a
> >>> > > fair
> >>> > > > ask. I don't see the harm in adding an optional column
> to
> >> the
> >>> > server
> >>> > > table
> >>> > > > with type=text for this data.
> >>> > > >
> >>> > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
> >> tyagi.an@husky.neu.edu
> >>> > >
> >>> > > wrote:
> >>> > > >
> >>> > > > > Hello Traffic Controllers,
> >>> > > > >
> >>> > > > > I discussed this with a couple of ATC users. We have
> >> multiple
> >>> > > > > Description/text fields in Delivery Service
> >> configuration of TP.
> >>> > > > Similarly,
> >>> > > > > We should also have one text field in servers
> >> configuration. My
> >>> > > use case
> >>> > > > is
> >>> > > > > to keep the service/serial id of the servers and any
> >> specific
> >>> > info
> >>> > > for a
> >>> > > > > server for which no field is available.
> >>> > > > >
> >>> > > > > I have created an issue on GitHub for it earlier:
> >>> > > > > https://github.com/apache/trafficcontrol/issues/2764
> >>> > > > > <https://github.com/apache/trafficcontrol/issues/2764
> >
> >>> > > > >
> >>> > > > > It's not a major change so shouldn't be a problem. If
> >> everyone
> >>> > > agrees,
> >>> > > > I'd
> >>> > > > > be interested to add that.
> >>> > > > >
> >>> > > > > Thank you
> >>> > > > > Anuj Tyagi
> >>> > > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> >
> >>>
> >>>
> >>
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Hank Beatty <hb...@apache.org>.
I'm +1 on adding a jsonb as well. As far as editing... What about using
the text field that Jan presented at the TC summit? It could be written
in YAML and easily converted to JSON.
-Hank
On 11/20/2018 04:29 PM, Steve Malenfant wrote:
> +1 json would help us with Ansible as well to store various variables.
> would be nice to have by cache group, servers, type... maybe not possible.
>
> On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
> wrote:
>
>> I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
>> column. Then it is up to whatever downstream components might be using
>> the column to make sure it works for their purposes. In the API we
>> could just validate that it's actually valid JSON, and I don't think
>> we'd need any other validation on it. Then it would be up to the
>> operator to come up with their own versioned JSON schema for the jsonb
>> column for their downstream components to use. Editing JSON directly
>> via TP is not ideal, but I'm sure there could be a way to generate a
>> usable input form given a JSON schema. The schema would be
>> version-controlled and placed down with TP via whatever configuration
>> management tool you're using. Then you can easily add whatever
>> arbitrary data you want to add to a server without polluting the DB
>> with multiple columns that aren't actually consumed by TC components.
>> Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
>> fit for that I think. Those columns aren't consumed by other TC
>> components directly and are mostly just informational or consumed by
>> things outside of Traffic Control, so maybe they'd be better off as
>> second-class citizens.
>>
>> So basically this:
>> If the new column is consumed by a TC component for the purpose of
>> control flow, it deserves a first-class column in the DB. If the new
>> field is not consumed by TC components, meant to be consumed by non-TC
>> components (e.g. configuration management), or is purely informational
>> for human consumption, it can just be relegated to the jsonb column.
>>
>> What do you all think?
>>
>> - Rawlin
>>
>> On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
>> <Jo...@comcast.com> wrote:
>>>
>>> DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
>> we do have 3 existing comment fields on a DS. They've all been co-opted
>> for other various purposes already. That's why they use different names in
>> TP than in the database.
>>>
>>> Jonathan G
>>>
>>>
>>> On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
>>>
>>> Right, we do... but try reading or writing paragraphs of info in
>> them. The
>>> DB fields may support it, but the UI not so much.
>>>
>>> __Jason
>>>
>>> On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
>> Brennan_Fieck@comcast.com>
>>> wrote:
>>>
>>> > We have three such fields for Delivery Services, afaik :P
>>> > ________________________________________
>>> > From: Jason Tucker <ja...@gmail.com>
>>> > Sent: Tuesday, November 20, 2018 7:25 AM
>>> > To: dev@trafficcontrol.apache.org
>>> > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config
>> of TP
>>> >
>>> > I'm actually a fan of arbitrary text boxes for more than just
>> server
>>> > objects. I've been hoping for something like this in delivery
>> service
>>> > objects as well, as this sort of field can be used to help document
>>> > unusual/custom/snowflake behavior which may not necessarily be
>> obvious to
>>> > those who come later with the intention of troubleshooting. Should
>> be used
>>> > for communicating with humans, rather than systems. In lieu of
>> versioning
>>> > of configs, it could be used to keep a change log for the object
>> as well.
>>> > But, again, that's more applicable to DS objects rather than Server
>>> > objects, I think.
>>> >
>>> > __Jason
>>> >
>>> > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
>> Jonathan_Gray@comcast.com>
>>> > wrote:
>>> >
>>> > > I'm -1 depending on what the intended use case is.
>>> > >
>>> > > Generic text fields should only be useful to human operators.
>> In the
>>> > case
>>> > > where you intend anything to programmatically access that
>> information and
>>> > > it's generally useful, you're better off with specific columns
>> per point
>>> > of
>>> > > data. This is how we ended up with unparsable, yet critical,
>> data in the
>>> > > comment fields of physical location table when we should have
>> added real
>>> > > columns. The example in the issue is delivery services. The
>> description
>>> > > field that I think is being referenced is one of LongDesc,
>> LongDesc_1, or
>>> > > LongDesc_2 in the database. Columns should have one purpose and
>> one
>>> > > meaning that is clear to a new developer working in the code and
>>> > > conceptually plausible to anyone else trying to understand how
>> the system
>>> > > works. One compromise, I'm not a huge fan of, would be to allow
>> for
>>> > > arbitrary structured data via a column of type jsonb instead of
>> text.
>>> > > That's not a great answer from a usability or db theory
>> perspective, but
>>> > > it's slightly better than regex parsing.
>>> > >
>>> > > Jonathan G
>>> > >
>>> > >
>>> > >
>>> > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
>>> > >
>>> > > +1, I am fine with it. That table already has a lot of
>> columns,
>>> > > what's one
>>> > > more!?
>>> > >
>>> > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
>>> > mitchell852@gmail.com
>>> > > >
>>> > > wrote:
>>> > >
>>> > > > Sounds like server "notes" or a server "description".
>> Seems like a
>>> > > fair
>>> > > > ask. I don't see the harm in adding an optional column to
>> the
>>> > server
>>> > > table
>>> > > > with type=text for this data.
>>> > > >
>>> > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
>> tyagi.an@husky.neu.edu
>>> > >
>>> > > wrote:
>>> > > >
>>> > > > > Hello Traffic Controllers,
>>> > > > >
>>> > > > > I discussed this with a couple of ATC users. We have
>> multiple
>>> > > > > Description/text fields in Delivery Service
>> configuration of TP.
>>> > > > Similarly,
>>> > > > > We should also have one text field in servers
>> configuration. My
>>> > > use case
>>> > > > is
>>> > > > > to keep the service/serial id of the servers and any
>> specific
>>> > info
>>> > > for a
>>> > > > > server for which no field is available.
>>> > > > >
>>> > > > > I have created an issue on GitHub for it earlier:
>>> > > > > https://github.com/apache/trafficcontrol/issues/2764
>>> > > > > <https://github.com/apache/trafficcontrol/issues/2764>
>>> > > > >
>>> > > > > It's not a major change so shouldn't be a problem. If
>> everyone
>>> > > agrees,
>>> > > > I'd
>>> > > > > be interested to add that.
>>> > > > >
>>> > > > > Thank you
>>> > > > > Anuj Tyagi
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>>
>>
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Steve Malenfant <sm...@gmail.com>.
+1 json would help us with Ansible as well to store various variables.
would be nice to have by cache group, servers, type... maybe not possible.
On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <ra...@gmail.com>
wrote:
> I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
> column. Then it is up to whatever downstream components might be using
> the column to make sure it works for their purposes. In the API we
> could just validate that it's actually valid JSON, and I don't think
> we'd need any other validation on it. Then it would be up to the
> operator to come up with their own versioned JSON schema for the jsonb
> column for their downstream components to use. Editing JSON directly
> via TP is not ideal, but I'm sure there could be a way to generate a
> usable input form given a JSON schema. The schema would be
> version-controlled and placed down with TP via whatever configuration
> management tool you're using. Then you can easily add whatever
> arbitrary data you want to add to a server without polluting the DB
> with multiple columns that aren't actually consumed by TC components.
> Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
> fit for that I think. Those columns aren't consumed by other TC
> components directly and are mostly just informational or consumed by
> things outside of Traffic Control, so maybe they'd be better off as
> second-class citizens.
>
> So basically this:
> If the new column is consumed by a TC component for the purpose of
> control flow, it deserves a first-class column in the DB. If the new
> field is not consumed by TC components, meant to be consumed by non-TC
> components (e.g. configuration management), or is purely informational
> for human consumption, it can just be relegated to the jsonb column.
>
> What do you all think?
>
> - Rawlin
>
> On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
> <Jo...@comcast.com> wrote:
> >
> > DS lifecycle tracking was supposed to modeled via DSR Comments. Also,
> we do have 3 existing comment fields on a DS. They've all been co-opted
> for other various purposes already. That's why they use different names in
> TP than in the database.
> >
> > Jonathan G
> >
> >
> > On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
> >
> > Right, we do... but try reading or writing paragraphs of info in
> them. The
> > DB fields may support it, but the UI not so much.
> >
> > __Jason
> >
> > On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com>
> > wrote:
> >
> > > We have three such fields for Delivery Services, afaik :P
> > > ________________________________________
> > > From: Jason Tucker <ja...@gmail.com>
> > > Sent: Tuesday, November 20, 2018 7:25 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config
> of TP
> > >
> > > I'm actually a fan of arbitrary text boxes for more than just
> server
> > > objects. I've been hoping for something like this in delivery
> service
> > > objects as well, as this sort of field can be used to help document
> > > unusual/custom/snowflake behavior which may not necessarily be
> obvious to
> > > those who come later with the intention of troubleshooting. Should
> be used
> > > for communicating with humans, rather than systems. In lieu of
> versioning
> > > of configs, it could be used to keep a change log for the object
> as well.
> > > But, again, that's more applicable to DS objects rather than Server
> > > objects, I think.
> > >
> > > __Jason
> > >
> > > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <
> Jonathan_Gray@comcast.com>
> > > wrote:
> > >
> > > > I'm -1 depending on what the intended use case is.
> > > >
> > > > Generic text fields should only be useful to human operators.
> In the
> > > case
> > > > where you intend anything to programmatically access that
> information and
> > > > it's generally useful, you're better off with specific columns
> per point
> > > of
> > > > data. This is how we ended up with unparsable, yet critical,
> data in the
> > > > comment fields of physical location table when we should have
> added real
> > > > columns. The example in the issue is delivery services. The
> description
> > > > field that I think is being referenced is one of LongDesc,
> LongDesc_1, or
> > > > LongDesc_2 in the database. Columns should have one purpose and
> one
> > > > meaning that is clear to a new developer working in the code and
> > > > conceptually plausible to anyone else trying to understand how
> the system
> > > > works. One compromise, I'm not a huge fan of, would be to allow
> for
> > > > arbitrary structured data via a column of type jsonb instead of
> text.
> > > > That's not a great answer from a usability or db theory
> perspective, but
> > > > it's slightly better than regex parsing.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > >
> > > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > > >
> > > > +1, I am fine with it. That table already has a lot of
> columns,
> > > > what's one
> > > > more!?
> > > >
> > > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> > > mitchell852@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Sounds like server "notes" or a server "description".
> Seems like a
> > > > fair
> > > > > ask. I don't see the harm in adding an optional column to
> the
> > > server
> > > > table
> > > > > with type=text for this data.
> > > > >
> > > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <
> tyagi.an@husky.neu.edu
> > > >
> > > > wrote:
> > > > >
> > > > > > Hello Traffic Controllers,
> > > > > >
> > > > > > I discussed this with a couple of ATC users. We have
> multiple
> > > > > > Description/text fields in Delivery Service
> configuration of TP.
> > > > > Similarly,
> > > > > > We should also have one text field in servers
> configuration. My
> > > > use case
> > > > > is
> > > > > > to keep the service/serial id of the servers and any
> specific
> > > info
> > > > for a
> > > > > > server for which no field is available.
> > > > > >
> > > > > > I have created an issue on GitHub for it earlier:
> > > > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > > > >
> > > > > > It's not a major change so shouldn't be a problem. If
> everyone
> > > > agrees,
> > > > > I'd
> > > > > > be interested to add that.
> > > > > >
> > > > > > Thank you
> > > > > > Anuj Tyagi
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> >
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Rawlin Peters <ra...@gmail.com>.
I think I'm mostly +1 on the compromise of adding an arbitrary jsonb
column. Then it is up to whatever downstream components might be using
the column to make sure it works for their purposes. In the API we
could just validate that it's actually valid JSON, and I don't think
we'd need any other validation on it. Then it would be up to the
operator to come up with their own versioned JSON schema for the jsonb
column for their downstream components to use. Editing JSON directly
via TP is not ideal, but I'm sure there could be a way to generate a
usable input form given a JSON schema. The schema would be
version-controlled and placed down with TP via whatever configuration
management tool you're using. Then you can easily add whatever
arbitrary data you want to add to a server without polluting the DB
with multiple columns that aren't actually consumed by TC components.
Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good
fit for that I think. Those columns aren't consumed by other TC
components directly and are mostly just informational or consumed by
things outside of Traffic Control, so maybe they'd be better off as
second-class citizens.
So basically this:
If the new column is consumed by a TC component for the purpose of
control flow, it deserves a first-class column in the DB. If the new
field is not consumed by TC components, meant to be consumed by non-TC
components (e.g. configuration management), or is purely informational
for human consumption, it can just be relegated to the jsonb column.
What do you all think?
- Rawlin
On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> DS lifecycle tracking was supposed to modeled via DSR Comments. Also, we do have 3 existing comment fields on a DS. They've all been co-opted for other various purposes already. That's why they use different names in TP than in the database.
>
> Jonathan G
>
>
> On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
>
> Right, we do... but try reading or writing paragraphs of info in them. The
> DB fields may support it, but the UI not so much.
>
> __Jason
>
> On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <Br...@comcast.com>
> wrote:
>
> > We have three such fields for Delivery Services, afaik :P
> > ________________________________________
> > From: Jason Tucker <ja...@gmail.com>
> > Sent: Tuesday, November 20, 2018 7:25 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
> >
> > I'm actually a fan of arbitrary text boxes for more than just server
> > objects. I've been hoping for something like this in delivery service
> > objects as well, as this sort of field can be used to help document
> > unusual/custom/snowflake behavior which may not necessarily be obvious to
> > those who come later with the intention of troubleshooting. Should be used
> > for communicating with humans, rather than systems. In lieu of versioning
> > of configs, it could be used to keep a change log for the object as well.
> > But, again, that's more applicable to DS objects rather than Server
> > objects, I think.
> >
> > __Jason
> >
> > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <Jo...@comcast.com>
> > wrote:
> >
> > > I'm -1 depending on what the intended use case is.
> > >
> > > Generic text fields should only be useful to human operators. In the
> > case
> > > where you intend anything to programmatically access that information and
> > > it's generally useful, you're better off with specific columns per point
> > of
> > > data. This is how we ended up with unparsable, yet critical, data in the
> > > comment fields of physical location table when we should have added real
> > > columns. The example in the issue is delivery services. The description
> > > field that I think is being referenced is one of LongDesc, LongDesc_1, or
> > > LongDesc_2 in the database. Columns should have one purpose and one
> > > meaning that is clear to a new developer working in the code and
> > > conceptually plausible to anyone else trying to understand how the system
> > > works. One compromise, I'm not a huge fan of, would be to allow for
> > > arbitrary structured data via a column of type jsonb instead of text.
> > > That's not a great answer from a usability or db theory perspective, but
> > > it's slightly better than regex parsing.
> > >
> > > Jonathan G
> > >
> > >
> > >
> > > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > >
> > > +1, I am fine with it. That table already has a lot of columns,
> > > what's one
> > > more!?
> > >
> > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> > mitchell852@gmail.com
> > > >
> > > wrote:
> > >
> > > > Sounds like server "notes" or a server "description". Seems like a
> > > fair
> > > > ask. I don't see the harm in adding an optional column to the
> > server
> > > table
> > > > with type=text for this data.
> > > >
> > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <tyagi.an@husky.neu.edu
> > >
> > > wrote:
> > > >
> > > > > Hello Traffic Controllers,
> > > > >
> > > > > I discussed this with a couple of ATC users. We have multiple
> > > > > Description/text fields in Delivery Service configuration of TP.
> > > > Similarly,
> > > > > We should also have one text field in servers configuration. My
> > > use case
> > > > is
> > > > > to keep the service/serial id of the servers and any specific
> > info
> > > for a
> > > > > server for which no field is available.
> > > > >
> > > > > I have created an issue on GitHub for it earlier:
> > > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > > >
> > > > > It's not a major change so shouldn't be a problem. If everyone
> > > agrees,
> > > > I'd
> > > > > be interested to add that.
> > > > >
> > > > > Thank you
> > > > > Anuj Tyagi
> > > > >
> > > >
> > >
> > >
> > >
> >
>
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
DS lifecycle tracking was supposed to modeled via DSR Comments. Also, we do have 3 existing comment fields on a DS. They've all been co-opted for other various purposes already. That's why they use different names in TP than in the database.
Jonathan G
On 11/20/18, 8:54 AM, "Jason Tucker" <ja...@gmail.com> wrote:
Right, we do... but try reading or writing paragraphs of info in them. The
DB fields may support it, but the UI not so much.
__Jason
On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> We have three such fields for Delivery Services, afaik :P
> ________________________________________
> From: Jason Tucker <ja...@gmail.com>
> Sent: Tuesday, November 20, 2018 7:25 AM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
>
> I'm actually a fan of arbitrary text boxes for more than just server
> objects. I've been hoping for something like this in delivery service
> objects as well, as this sort of field can be used to help document
> unusual/custom/snowflake behavior which may not necessarily be obvious to
> those who come later with the intention of troubleshooting. Should be used
> for communicating with humans, rather than systems. In lieu of versioning
> of configs, it could be used to keep a change log for the object as well.
> But, again, that's more applicable to DS objects rather than Server
> objects, I think.
>
> __Jason
>
> On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <Jo...@comcast.com>
> wrote:
>
> > I'm -1 depending on what the intended use case is.
> >
> > Generic text fields should only be useful to human operators. In the
> case
> > where you intend anything to programmatically access that information and
> > it's generally useful, you're better off with specific columns per point
> of
> > data. This is how we ended up with unparsable, yet critical, data in the
> > comment fields of physical location table when we should have added real
> > columns. The example in the issue is delivery services. The description
> > field that I think is being referenced is one of LongDesc, LongDesc_1, or
> > LongDesc_2 in the database. Columns should have one purpose and one
> > meaning that is clear to a new developer working in the code and
> > conceptually plausible to anyone else trying to understand how the system
> > works. One compromise, I'm not a huge fan of, would be to allow for
> > arbitrary structured data via a column of type jsonb instead of text.
> > That's not a great answer from a usability or db theory perspective, but
> > it's slightly better than regex parsing.
> >
> > Jonathan G
> >
> >
> >
> > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> >
> > +1, I am fine with it. That table already has a lot of columns,
> > what's one
> > more!?
> >
> > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> mitchell852@gmail.com
> > >
> > wrote:
> >
> > > Sounds like server "notes" or a server "description". Seems like a
> > fair
> > > ask. I don't see the harm in adding an optional column to the
> server
> > table
> > > with type=text for this data.
> > >
> > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <tyagi.an@husky.neu.edu
> >
> > wrote:
> > >
> > > > Hello Traffic Controllers,
> > > >
> > > > I discussed this with a couple of ATC users. We have multiple
> > > > Description/text fields in Delivery Service configuration of TP.
> > > Similarly,
> > > > We should also have one text field in servers configuration. My
> > use case
> > > is
> > > > to keep the service/serial id of the servers and any specific
> info
> > for a
> > > > server for which no field is available.
> > > >
> > > > I have created an issue on GitHub for it earlier:
> > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > >
> > > > It's not a major change so shouldn't be a problem. If everyone
> > agrees,
> > > I'd
> > > > be interested to add that.
> > > >
> > > > Thank you
> > > > Anuj Tyagi
> > > >
> > >
> >
> >
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Jason Tucker <ja...@gmail.com>.
Right, we do... but try reading or writing paragraphs of info in them. The
DB fields may support it, but the UI not so much.
__Jason
On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> We have three such fields for Delivery Services, afaik :P
> ________________________________________
> From: Jason Tucker <ja...@gmail.com>
> Sent: Tuesday, November 20, 2018 7:25 AM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
>
> I'm actually a fan of arbitrary text boxes for more than just server
> objects. I've been hoping for something like this in delivery service
> objects as well, as this sort of field can be used to help document
> unusual/custom/snowflake behavior which may not necessarily be obvious to
> those who come later with the intention of troubleshooting. Should be used
> for communicating with humans, rather than systems. In lieu of versioning
> of configs, it could be used to keep a change log for the object as well.
> But, again, that's more applicable to DS objects rather than Server
> objects, I think.
>
> __Jason
>
> On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <Jo...@comcast.com>
> wrote:
>
> > I'm -1 depending on what the intended use case is.
> >
> > Generic text fields should only be useful to human operators. In the
> case
> > where you intend anything to programmatically access that information and
> > it's generally useful, you're better off with specific columns per point
> of
> > data. This is how we ended up with unparsable, yet critical, data in the
> > comment fields of physical location table when we should have added real
> > columns. The example in the issue is delivery services. The description
> > field that I think is being referenced is one of LongDesc, LongDesc_1, or
> > LongDesc_2 in the database. Columns should have one purpose and one
> > meaning that is clear to a new developer working in the code and
> > conceptually plausible to anyone else trying to understand how the system
> > works. One compromise, I'm not a huge fan of, would be to allow for
> > arbitrary structured data via a column of type jsonb instead of text.
> > That's not a great answer from a usability or db theory perspective, but
> > it's slightly better than regex parsing.
> >
> > Jonathan G
> >
> >
> >
> > On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
> >
> > +1, I am fine with it. That table already has a lot of columns,
> > what's one
> > more!?
> >
> > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <
> mitchell852@gmail.com
> > >
> > wrote:
> >
> > > Sounds like server "notes" or a server "description". Seems like a
> > fair
> > > ask. I don't see the harm in adding an optional column to the
> server
> > table
> > > with type=text for this data.
> > >
> > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <tyagi.an@husky.neu.edu
> >
> > wrote:
> > >
> > > > Hello Traffic Controllers,
> > > >
> > > > I discussed this with a couple of ATC users. We have multiple
> > > > Description/text fields in Delivery Service configuration of TP.
> > > Similarly,
> > > > We should also have one text field in servers configuration. My
> > use case
> > > is
> > > > to keep the service/serial id of the servers and any specific
> info
> > for a
> > > > server for which no field is available.
> > > >
> > > > I have created an issue on GitHub for it earlier:
> > > > https://github.com/apache/trafficcontrol/issues/2764
> > > > <https://github.com/apache/trafficcontrol/issues/2764>
> > > >
> > > > It's not a major change so shouldn't be a problem. If everyone
> > agrees,
> > > I'd
> > > > be interested to add that.
> > > >
> > > > Thank you
> > > > Anuj Tyagi
> > > >
> > >
> >
> >
> >
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by "Fieck, Brennan" <Br...@comcast.com>.
We have three such fields for Delivery Services, afaik :P
________________________________________
From: Jason Tucker <ja...@gmail.com>
Sent: Tuesday, November 20, 2018 7:25 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
I'm actually a fan of arbitrary text boxes for more than just server
objects. I've been hoping for something like this in delivery service
objects as well, as this sort of field can be used to help document
unusual/custom/snowflake behavior which may not necessarily be obvious to
those who come later with the intention of troubleshooting. Should be used
for communicating with humans, rather than systems. In lieu of versioning
of configs, it could be used to keep a change log for the object as well.
But, again, that's more applicable to DS objects rather than Server
objects, I think.
__Jason
On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <Jo...@comcast.com>
wrote:
> I'm -1 depending on what the intended use case is.
>
> Generic text fields should only be useful to human operators. In the case
> where you intend anything to programmatically access that information and
> it's generally useful, you're better off with specific columns per point of
> data. This is how we ended up with unparsable, yet critical, data in the
> comment fields of physical location table when we should have added real
> columns. The example in the issue is delivery services. The description
> field that I think is being referenced is one of LongDesc, LongDesc_1, or
> LongDesc_2 in the database. Columns should have one purpose and one
> meaning that is clear to a new developer working in the code and
> conceptually plausible to anyone else trying to understand how the system
> works. One compromise, I'm not a huge fan of, would be to allow for
> arbitrary structured data via a column of type jsonb instead of text.
> That's not a great answer from a usability or db theory perspective, but
> it's slightly better than regex parsing.
>
> Jonathan G
>
>
>
> On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
>
> +1, I am fine with it. That table already has a lot of columns,
> what's one
> more!?
>
> On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <mitchell852@gmail.com
> >
> wrote:
>
> > Sounds like server "notes" or a server "description". Seems like a
> fair
> > ask. I don't see the harm in adding an optional column to the server
> table
> > with type=text for this data.
> >
> > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <ty...@husky.neu.edu>
> wrote:
> >
> > > Hello Traffic Controllers,
> > >
> > > I discussed this with a couple of ATC users. We have multiple
> > > Description/text fields in Delivery Service configuration of TP.
> > Similarly,
> > > We should also have one text field in servers configuration. My
> use case
> > is
> > > to keep the service/serial id of the servers and any specific info
> for a
> > > server for which no field is available.
> > >
> > > I have created an issue on GitHub for it earlier:
> > > https://github.com/apache/trafficcontrol/issues/2764
> > > <https://github.com/apache/trafficcontrol/issues/2764>
> > >
> > > It's not a major change so shouldn't be a problem. If everyone
> agrees,
> > I'd
> > > be interested to add that.
> > >
> > > Thank you
> > > Anuj Tyagi
> > >
> >
>
>
>
Re: [EXTERNAL] Re: Adding a text field in Servers config of TP
Posted by Jason Tucker <ja...@gmail.com>.
I'm actually a fan of arbitrary text boxes for more than just server
objects. I've been hoping for something like this in delivery service
objects as well, as this sort of field can be used to help document
unusual/custom/snowflake behavior which may not necessarily be obvious to
those who come later with the intention of troubleshooting. Should be used
for communicating with humans, rather than systems. In lieu of versioning
of configs, it could be used to keep a change log for the object as well.
But, again, that's more applicable to DS objects rather than Server
objects, I think.
__Jason
On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan <Jo...@comcast.com>
wrote:
> I'm -1 depending on what the intended use case is.
>
> Generic text fields should only be useful to human operators. In the case
> where you intend anything to programmatically access that information and
> it's generally useful, you're better off with specific columns per point of
> data. This is how we ended up with unparsable, yet critical, data in the
> comment fields of physical location table when we should have added real
> columns. The example in the issue is delivery services. The description
> field that I think is being referenced is one of LongDesc, LongDesc_1, or
> LongDesc_2 in the database. Columns should have one purpose and one
> meaning that is clear to a new developer working in the code and
> conceptually plausible to anyone else trying to understand how the system
> works. One compromise, I'm not a huge fan of, would be to allow for
> arbitrary structured data via a column of type jsonb instead of text.
> That's not a great answer from a usability or db theory perspective, but
> it's slightly better than regex parsing.
>
> Jonathan G
>
>
>
> On 11/19/18, 3:09 PM, "Dave Neuman" <ne...@apache.org> wrote:
>
> +1, I am fine with it. That table already has a lot of columns,
> what's one
> more!?
>
> On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell <mitchell852@gmail.com
> >
> wrote:
>
> > Sounds like server "notes" or a server "description". Seems like a
> fair
> > ask. I don't see the harm in adding an optional column to the server
> table
> > with type=text for this data.
> >
> > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi <ty...@husky.neu.edu>
> wrote:
> >
> > > Hello Traffic Controllers,
> > >
> > > I discussed this with a couple of ATC users. We have multiple
> > > Description/text fields in Delivery Service configuration of TP.
> > Similarly,
> > > We should also have one text field in servers configuration. My
> use case
> > is
> > > to keep the service/serial id of the servers and any specific info
> for a
> > > server for which no field is available.
> > >
> > > I have created an issue on GitHub for it earlier:
> > > https://github.com/apache/trafficcontrol/issues/2764
> > > <https://github.com/apache/trafficcontrol/issues/2764>
> > >
> > > It's not a major change so shouldn't be a problem. If everyone
> agrees,
> > I'd
> > > be interested to add that.
> > >
> > > Thank you
> > > Anuj Tyagi
> > >
> >
>
>
>