Re: Adding some guard rails to Kudu

[Brock Noland <br...@phdata.io>]

+1

Very reasonable approach and like that there is a semi-hard to use safety
valve.

On Wednesday, November 30, 2016, Todd Lipcon <to...@cloudera.com> wrote:

> BTW I filed a JIRA here and started linking related issues to it:
> https://issues.apache.org/jira/browse/KUDU-1775
>
>
> On Wed, Nov 30, 2016 at 3:25 PM, Todd Lipcon <todd@cloudera.com
> <javascript:;>> wrote:
>
> > Hey folks,
> >
> > I've started working on a few patches to add "guard rails" to various
> > user-specified dimensions in Kudu. In particular, I'm planning to add
> > limits to the following:
> >
> > - max number of columns in a table (proposal: 300)
> > - max replication factor (proposal: 7)
> > - max table name or column name length (proposal: 256)
> > - max size of a binary/string column cell value (proposal: 64kb)
> >
> > The reasoning is that, even though in some cases we don't know a specific
> > issue that will happen outside these limits, we've done very little
> testing
> > (and have no automated testing) outside of these ranges. In some cases,
> we
> > do know that there is a certain threshold that will cause a big problem
> (eg
> > large cell sizes can cause tablet servers to crash). In other cases, it's
> > just "unknown territory".
> >
> > In all cases, I'm planning on making the limits overridable via an
> > "unsafe" configuration flag. That means that a user can run with
> > "--unlock_unsafe_flags --max_identifier_length=1000" if they want to, but
> > they're explicitly accepting some risk that they're entering untested
> > territory.
> >
> > Of course, in all cases, if we hear that there are people who are bumping
> > the maxes higher than the defaults and having good results, we can
> consider
> > raising the maximum, but I think it's smarter to start conservatively low
> > and raise later as we increase test coverage. Also, I'm sure down the
> road
> > we'll add features such as BLOB support or sparse column support, and at
> > that time we can remove the corresponding guard rails.
> >
> > I'm sending this note to both user@ and dev@ to solicit feedback. Are
> > there any other dimensions people can think of where we should probably
> add
> > guard-rails? Is anyone out there already outside of the above ranges and
> > can make a case that we're being too conservative?
> >
> > Thanks
> > -Todd
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>