Need feedback on CASSANDRA-13066

[benjamin roth <br...@gmail.com>]

Hi guys,

I started working on 13066. My intention is to offer a table-setting that
allows a operator to optimize MV streaming in some cases or simply "on
purpose - i know what i do".

MV write path streaming can be ommitted e.g. if:
- data is append only
- no PK is added to MV so no stale data can be created on race conditions

This is a first patch:
https://github.com/Jaumo/cassandra/commit/0d4ce966f129e1b29098f194b5951a86dc8c585a

Please don't consider it as final. Some tests are missing and some logic is
still missing.

When introducing a table option what would be to prefer:
- mv_fast_stream: Does what it says, maybe even a more verbose name?
- append_only: To tell how data is filled. This could also be a hint for
future optimizations like CASSANDRA-9779
<https://issues.apache.org/jira/browse/CASSANDRA-9779> but would not allow
me just to tell CS to do that kind of streaming no matter how I treat my
data

Also still to be considered in this ticket:
- With "fast streaming" MVs MUST be repaired separately and explicitly
- With "write path repairs" MVs MUST NOT be included in KS repairs. Not
only that this is unnecessary repair-work - it could (or probably will)
break the local consistency between base table and MV.
- Manual of views that are normally repaired through the write path of the
base table should at least log a warning like "Manually repairing a
material view may lead to inconsistencies"

I'd really love to get some feedback before putting more effort in.
Thanks!