[GitHub] [incubator-pinot] mcvsubbu commented on pull request #6737: Block backward incompatible schema change

[GitBox <gi...@apache.org>]

mcvsubbu commented on pull request #6737:
URL: https://github.com/apache/incubator-pinot/pull/6737#issuecomment-812701105


   > @siddharthteotia @mcvsubbu I can remove the force update flag. Then in order to make schema incompatible change, user has to drop and recreate the table. wdyt?
   
   That is totally fine I think. The way I look at this is as follows:
   
    If we have a production deployment, it is good  in multiple respects to NOT have backward incompatible changes. New columns can be added, and old columns just set to default value in future segments. In a columnar database, the only extra resource consumed is metadata for the column.
   
   If this is a non-production env (experimental) it should be ok to drop the data and recreate the table. Maybe we can provide some recipes for saving off existing segments and recreating the segments for the new table. The main problem that users may have is if they drop the table and recreate it the same instant (before the old table goes away). We could (should) add a check in table creation logic that there is no EXTERNALVIEW for a new table being created. If there is, fail the table with a good error message to ask the user to try later.
   
   Bottom line: Good to drop the table in either case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org