2.1 Release TODO

[Mike Miller <mm...@apache.org>]

Starting an email chain of things that folks want to finish for 2.1. Here
is what we currently have in the works that are most likely going into 2.1:
https://github.com/apache/accumulo/pull/2569
https://github.com/apache/accumulo/pull/2600
https://github.com/apache/accumulo/pull/2293

Some things that may go into 2.1:
https://github.com/apache/accumulo/pull/2422
https://github.com/apache/accumulo/pull/2475
https://github.com/apache/accumulo/pull/2197

I created a Project for follow on work to the ZK property change. I was
planning on putting tasks in there that we want to complete for 2.1. But we
could also use it for post 2.1 work.
https://github.com/apache/accumulo/projects/24
https://github.com/apache/accumulo/issues/2469

FYI a draft copy of the release notes has already been on the website:
https://accumulo.apache.org/release/accumulo-2.1.0/

This may be a good thread to discuss whether or not a task needs to go into
2.1 or should wait for the next version. We currently have 32 open pull
requests so please email me if there is one that you would like prioritized
for 2.1.

Re: Scan Server discussion [WAS: Re: 2.1 Release TODO]

[Christopher <ct...@apache.org>]

On Mon, Apr 4, 2022 at 1:11 PM Dave Marion <dm...@gmail.com> wrote:
>
> I understand the desire to see less coupling for the optional features, but
> getting to that point for ScanServers (and less so for ExternalCompactions)
> would be a ton of work I think.

The likelihood of it being a lot of work doesn't mean it shouldn't be
done. It's much more likely you could get help doing the work, if
needed, after the 2.1 release, when some of the time that people are
putting in 2.1 quality controls / testing, is freed up to work on
other tasks.

> The concern that I brought up in the "2.1
> Release TODOs" thread regarding planning has not been addressed. If there
> was a defined path forward, then that might make it easier to see how this
> feature gets added in the near-future in whatever form it takes.

LOL, I forked the mailing list conversation because Keith started
discussing the feature merits in the release planning thread, and it
seemed like a separate topic. Now, you're talking about release
planning in this thread. I'm not sure where to reply! 🤦 I guess the
lines are blurred. I'll reply here.

If we decide not to merge it into 2.1, what would be your preferred
path forward? What's your contingency plan for this feature if the
community doesn't include it in 2.1? Here's some possibilities:

* Could release in non-LTM 2.2 (release right away)
* Could release in non-LTM 3.0 (to include with other 3.0 changes)
* Could do some work to decouple, and create a separate optional
add-on jar for 2.1

By default, I would expect it to go into the next release (probably
named 3.0), unless there's another path proposed that we can get
consensus around. The more decoupled it is from Accumulo's internals,
though, makes it more likely to be usable with more versions, even
2.1. If completely decoupled, it could be released at any time on its
own schedule and be made to work with any version you like. It
wouldn't need to be released coupled to an Accumulo release.

>
> Regarding the concern about the readiness of the feature branch, Keith is
> doing a last pass review on the draft and then I believe we are ready to
> take it out of draft state. I think it will be before the end of this week.
> We have added six new integration tests and we have done some local and
> cluster testing.

Taking it out of draft state means that some people will begin to look
at it. But, others, whose focus is on polishing existing features and
testing for 2.1 won't have time, unless they prioritize your PR over
the existing prep for 2.1. I'm not sure how long that would take.

> Regarding the concern mentioned above, "availability of time to review/test
> such a big feature without delaying 2.1," I didn't realize that we had a
> schedule.

We don't have a strict schedule. We never have. It's informal. It's
just that 2.1 has been under development for so long already. I've
been feeling the pressure build to get it released. I think a lot of
our community has already started wanting a 2.x LTM release once we
started discussing the LTM concept, and they've been stuck with 2.0 as
the only 2.x version to work with. We only reluctantly did a CVE patch
for 2.0, when it was non-LTM, only because we were not able to get 2.1
out at that time. Mike also suggested in Slack months ago (January
27th), that we should consider branching a 2.2 for new features, so
2.1 doesn't get slowed down. So, I know I'm not the only one who has
felt 2.1 is getting a little fat with feature changes and overdue for
a release. You commented on that thread, so I know you had at least a
peripheral awareness at some point, that there was interest in
wrapping up 2.1 before you even started the ScanServer work, way back
when we were still cleaning up the tests from the breakages caused by
the big external compactions and thread pool changes.

>  Does it matter if it takes 2/4/6/8 weeks to test the 1000+
> completed issues in this release?

Yes. I think so.

First, most of those changes are trivial, and have already been
thoroughly tested and proven during development. While we do overall
testing near the end of development near a release, most of the
testing is done continuously throughout development, even for the
non-trivial changes. We often test features thoroughly as they are
added before we move on to the next feature to work on. So, it's
misleading to suggest that there are 1000+ untested features queued up
to be tested and that this is merely one more.

Second, while the specific duration 2 weeks vs. 8 weeks doesn't
necessarily matter, it does matter whether we're testing a moving
target or not. At this point, our testing target is still moving, but
it should be moving much more slowly than it was at the beginning of
the development cycle, and it should be slowing down, not speeding up
with new features. Many of us have already been thinking about a 2.1
release for several months and have been working on fixing tests,
making test improvements, wrapping up existing features, and doing
code quality checks, instead of doing new features, checking the open
tickets, and closing open PRs, specifically because we've moved
informally into a "wrapping up" phase. I myself only looked into
cleaning up ZooKeeper recently, because we had several tickets open
marked for 2.1 for singleton manager cleanup, and I wanted to see if
we could get them in or if they needed to be bumped... because I'm
focused on wrapping up 2.1. I suspect others are in a similar mindset
for 2.1.

> I know that we want to finish up the
> 2.1.0 release, but is there a target date?

No, there isn't a specific target date. We've never had a specific
target date. What we've had is a general need to have a reasonable cut
off point once momentum has started building, and consensus has grown
towards the conclusion that we're feature-sufficient for a release. I
think that momentum has already been building for a while now. We
don't need a specific target date, but we do need development to slow
down as we polish up, so we don't have a rapidly moving target to
chase.



Even if I understood the rush to get it released in 2.1, which I
don't, I still think it would likely delay 2.1 a lot, because we don't
have unlimited developer time and expertise. So we'd have to have
consensus on accepting that delay as a trade-off for including it.
Speaking only for myself, I would prefer we release without it, rather
than delay, and then focus on more big features for the next release,
and release more frequently... something this community has discussed
on the mailing list many times, but never managed to accomplish. Part
of the goal of LTM releases was so we could have more frequent non-LTM
releases. If we put so much importance into getting all our features
into LTM releases, regardless of how long it delays, or where we're at
in the development cycle, we're never going to do non-LTM releases,
because we just won't have the energy or motivation to do it.

Re: Scan Server discussion [WAS: Re: 2.1 Release TODO]

[Dave Marion <dm...@gmail.com>]

I understand the desire to see less coupling for the optional features, but
getting to that point for ScanServers (and less so for ExternalCompactions)
would be a ton of work I think. The concern that I brought up in the "2.1
Release TODOs" thread regarding planning has not been addressed. If there
was a defined path forward, then that might make it easier to see how this
feature gets added in the near-future in whatever form it takes.

Regarding the concern about the readiness of the feature branch, Keith is
doing a last pass review on the draft and then I believe we are ready to
take it out of draft state. I think it will be before the end of this week.
We have added six new integration tests and we have done some local and
cluster testing.

Regarding the concern mentioned above, "availability of time to review/test
such a big feature without delaying 2.1," I didn't realize that we had a
schedule.  Does it matter if it takes 2/4/6/8 weeks to test the 1000+
completed issues in this release? I know that we want to finish up the
2.1.0 release, but is there a target date?

On Mon, Apr 4, 2022 at 12:32 PM Christopher <ct...@apache.org> wrote:

> On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:
> >
> > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org> wrote:
> > >
> > > However, I'm reluctant to include #2422, because I don't think it's
> near
> > > ready enough, and by the time it is, it will be very last minute, and I
> > > don't want to delay 2.1 further for it. Even if it's included as an
> > > experimental feature, I think it has huge potential to be disruptive,
> or to
> > > have a lot of churn by the time people actually have a chance to
> review it
> > > thoroughly. Furthermore, I think there are possible alternatives (like
> a
> > > fully client-side implementation, based on offline scanners) that would
> > > avoid the tight coupling of a new service to Accumulo's core code. This
> >
> > There are some advantages to scan servers over direct file access to
> > consider.  One is scalability of computation, if a web server is
> > serving N client queries with scan servers those can potentially go to
> > different scan servers.  With direct file access, all N queries and
> > their iterator stacks would have to run in the web server.  Another is
> > scalability of caching/memory.  When web servers send queries to scan
> > servers using a sticky algorithm for assigning tablets to groups of
> > scan servers, it could lead to good cache utilization and sharing that
> > may not be possible when running scans directly in the web server. So
> > scan servers allow scaling cache and computations for queries
> > independently of web servers in way that may not be possible with
> > direct file access.
> >
> > Another advantage to consider is isolation.  With direct file access
> > and queries running directly in a web server, a bad query could bring
> > down a web server and lots of unrelated queries.  Having a bad query
> > bring down a scan server may be less disruptive.
> >
>
> I've forked this thread into its own discussion with a new subject
> line, because, as I suggested in my original reply, my intent was not
> to hijack the 2.1 planning thread with a discussion of the ScanServer
> implementation details.
>
> I'm fine with all those benefits (even if all the "could" and "may"
> were turned into concrete "will"). My objection is not an objection to
> the feature. It's an objection to including the feature in 2.1, based
> on:
>
> * readiness of the feature branch,
> * availability of time to review/test such a big feature without delaying
> 2.1,
> * its tight coupling to the core code in the implementation, and
> * the possibility that solutions may exist with the above benefits
> that are less tightly coupled has not yet been explored.
>
> I would be more okay with including it if:
>
> * it is ready,
> * it has been tested and reviewed by the wider community,
> * its coupling to the core Accumulo code is loosened, ideally if it's
> designed to use only API/SPI, and could be released as a separate,
> optional add-on. This might require improvements to API/SPI to expose
> the features needed to help it function. This could also be done by
> sub-classing the AccumuloClient. My concern here is the risk of
> technical debt and the extra maintenance costs of increased complexity
> for optional features that go unmaintained.
>
> We've been hurt by premature inclusion of optional/experimental
> features before that were rushed to release. No matter how awesome the
> feature is... if it's niche and optional, we should consider these
> risks and work to mitigate them. Otherwise, we'll be stuck with the
> technical debt for years to come. With a little bit of caution, we can
> make the feature available, without rushing, to satisfy the use case
> while reducing the risks.
>
> Also, one point of clarification: when I say "fully client side", I
> only mean relative to Accumulo, not necessarily in the client process.
> I'm lacking vocabulary to describe what I mean. As I understand it,
> the current client code has been modified to connect to ScanServers
> sitting off to the side of TabletServers, and the ScanServers are
> basically modified TabletServers with less functionality. What I mean
> is that instead of coupling the ScanServer to the TabletServer
> implementation, and coupling the ScanServer client to the
> AccumuloClient, there could be less coupling. The ScanServer itself
> could behave like a client to Accumulo and/or HDFS (and maybe even
> share some library code that we make public API, like RFile readers)
> and it could have its own client (this is just one very rough outline
> of an idea that could be explored). That way, the entire thing could
> be removed without any change in Accumulo's code, to make it truly
> optional (as in, optional to even have on the class path).
>

Scan Server discussion [WAS: Re: 2.1 Release TODO]

[Christopher <ct...@apache.org>]

On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:
>
> On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org> wrote:
> >
> > However, I'm reluctant to include #2422, because I don't think it's near
> > ready enough, and by the time it is, it will be very last minute, and I
> > don't want to delay 2.1 further for it. Even if it's included as an
> > experimental feature, I think it has huge potential to be disruptive, or to
> > have a lot of churn by the time people actually have a chance to review it
> > thoroughly. Furthermore, I think there are possible alternatives (like a
> > fully client-side implementation, based on offline scanners) that would
> > avoid the tight coupling of a new service to Accumulo's core code. This
>
> There are some advantages to scan servers over direct file access to
> consider.  One is scalability of computation, if a web server is
> serving N client queries with scan servers those can potentially go to
> different scan servers.  With direct file access, all N queries and
> their iterator stacks would have to run in the web server.  Another is
> scalability of caching/memory.  When web servers send queries to scan
> servers using a sticky algorithm for assigning tablets to groups of
> scan servers, it could lead to good cache utilization and sharing that
> may not be possible when running scans directly in the web server. So
> scan servers allow scaling cache and computations for queries
> independently of web servers in way that may not be possible with
> direct file access.
>
> Another advantage to consider is isolation.  With direct file access
> and queries running directly in a web server, a bad query could bring
> down a web server and lots of unrelated queries.  Having a bad query
> bring down a scan server may be less disruptive.
>

I've forked this thread into its own discussion with a new subject
line, because, as I suggested in my original reply, my intent was not
to hijack the 2.1 planning thread with a discussion of the ScanServer
implementation details.

I'm fine with all those benefits (even if all the "could" and "may"
were turned into concrete "will"). My objection is not an objection to
the feature. It's an objection to including the feature in 2.1, based
on:

* readiness of the feature branch,
* availability of time to review/test such a big feature without delaying 2.1,
* its tight coupling to the core code in the implementation, and
* the possibility that solutions may exist with the above benefits
that are less tightly coupled has not yet been explored.

I would be more okay with including it if:

* it is ready,
* it has been tested and reviewed by the wider community,
* its coupling to the core Accumulo code is loosened, ideally if it's
designed to use only API/SPI, and could be released as a separate,
optional add-on. This might require improvements to API/SPI to expose
the features needed to help it function. This could also be done by
sub-classing the AccumuloClient. My concern here is the risk of
technical debt and the extra maintenance costs of increased complexity
for optional features that go unmaintained.

We've been hurt by premature inclusion of optional/experimental
features before that were rushed to release. No matter how awesome the
feature is... if it's niche and optional, we should consider these
risks and work to mitigate them. Otherwise, we'll be stuck with the
technical debt for years to come. With a little bit of caution, we can
make the feature available, without rushing, to satisfy the use case
while reducing the risks.

Also, one point of clarification: when I say "fully client side", I
only mean relative to Accumulo, not necessarily in the client process.
I'm lacking vocabulary to describe what I mean. As I understand it,
the current client code has been modified to connect to ScanServers
sitting off to the side of TabletServers, and the ScanServers are
basically modified TabletServers with less functionality. What I mean
is that instead of coupling the ScanServer to the TabletServer
implementation, and coupling the ScanServer client to the
AccumuloClient, there could be less coupling. The ScanServer itself
could behave like a client to Accumulo and/or HDFS (and maybe even
share some library code that we make public API, like RFile readers)
and it could have its own client (this is just one very rough outline
of an idea that could be explored). That way, the entire thing could
be removed without any change in Accumulo's code, to make it truly
optional (as in, optional to even have on the class path).

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

2.1 update for July and August.
7 PRs in progress.
1,253 marked as DONE. (+87)
6 left marked as a Blocker for 2.1. [1]
2 left as TODO for Compaction follow on work. [2]
1 left as TODO for ZK follow on work. [3]
Once #2197 is completed and merged, there will be some follow on work.

[1]:
https://github.com/apache/accumulo/issues?q=is%3Aissue%20is%3Aopen%20label%3Ablocker%20project%3Aapache%2Faccumulo%2F3
[2]: https://github.com/apache/accumulo/projects/13
[3]: https://github.com/apache/accumulo/projects/24

On Wed, Jun 29, 2022 at 9:11 AM Mike Miller <mm...@apache.org> wrote:

> It has been a month so I am sending a 2.1 update.
> 10 PRs in progress.
> 1,166 marked as DONE. (+36)
> 33 left in the TODO state. I went through these and dropped some from 2.1.
> https://github.com/apache/accumulo/projects/3
>
> 3 marked as TODO as Compaction follow on work:
> https://github.com/apache/accumulo/projects/13
>
> 2 in progress & 2 TODO for the Single Node ZK follow on work.
> https://github.com/apache/accumulo/projects/24
>
>
> On Fri, May 27, 2022 at 11:30 AM Mike Miller <mm...@apache.org> wrote:
>
>> Update on 2.1 progress. Come on Folks, let's Hold The Line. [1]
>> 14 Pull requests in progress. (-7 digression from last week)
>> 1,130 marked as DONE. (+19)
>> https://github.com/apache/accumulo/projects/3
>>
>> 3 marked as TODO as Compaction follow on work:
>> https://github.com/apache/accumulo/projects/13
>>
>> 2 in progress & 2 TODO for the Single Node ZK follow on work.
>> https://github.com/apache/accumulo/projects/24
>>
>> [1]: https://www.youtube.com/watch?v=htgr3pvBr-I
>>
>> On Fri, May 20, 2022 at 11:25 AM Mike Miller <mm...@apache.org> wrote:
>>
>>> Update on 2.1 progress.
>>> 7 Pull requests in progress.
>>> 1111 marked as DONE. ( I just missed sending this at 11:11 EST)
>>> https://github.com/apache/accumulo/projects/3
>>>
>>> 3 marked as TODO as Compaction follow on work:
>>> https://github.com/apache/accumulo/projects/13
>>>
>>> 2 marked as TODO for the Single Node ZK follow on work.
>>> https://github.com/apache/accumulo/projects/24
>>>
>>>
>>>
>>> On Fri, May 13, 2022 at 9:38 AM Mike Miller <mm...@apache.org> wrote:
>>>
>>>> Update on 2.1 progress.
>>>> 10 Pull Requests in progress.
>>>> 1,097 marked as DONE.
>>>> https://github.com/apache/accumulo/projects/3
>>>>
>>>> 3 Tickets marked as TODO as Compaction follow on work:
>>>> https://github.com/apache/accumulo/projects/13
>>>>
>>>> Only 1 Ticket marked as TODO for the ZK follow on work. I thought there
>>>> would be more here:
>>>> https://github.com/apache/accumulo/projects/24
>>>>
>>>> On Fri, Apr 8, 2022 at 7:00 AM Mike Miller <mm...@apache.org> wrote:
>>>>
>>>>> Update on 2.1 progress. For pull requests:
>>>>> 15 currently in progress.
>>>>> 32 are open as TODO. But a lot of these will get bumped to the next
>>>>> version.
>>>>> 1,025 DONE. Wow! Good work everyone.
>>>>>
>>>>> https://github.com/apache/accumulo/projects/3
>>>>>
>>>>> On Wed, Apr 6, 2022 at 4:55 PM Christopher <ct...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> After some additional consideration, and getting a better
>>>>>> understanding of
>>>>>> how the code is expected to work from discussing it with Dave... I'm a
>>>>>> little more inclined to support #2422 in 2.1, provided:
>>>>>>
>>>>>> 1. There's time for me to review it,
>>>>>> 2. It is sufficiently decoupled from the existing code and marked
>>>>>> experimental, so that we have the flexibility to alter its design, if
>>>>>> it
>>>>>> seems appropriate after it gets some exposure after the release,
>>>>>> 3. Unit tests and integration tests are reliably passing (as stable
>>>>>> as, or
>>>>>> more stable than, they are currently),
>>>>>> 4. No serious issues are discovered during review, and
>>>>>> 5. It doesn't delay a release past early June, as I think this is a
>>>>>> reasonable target date.
>>>>>>
>>>>>> This my wishlist before I can get behind it with a +1 for 2.1. If
>>>>>> these
>>>>>> aren't met, I do not intend to veto, but I'd be a -0 on its inclusion
>>>>>> to
>>>>>> 2.1. Of course, once I review it, my thoughts may change a bit.
>>>>>>
>>>>>> On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> > I think I can finish the FATE refactor PR [1] for 2.1. I had been
>>>>>> keeping
>>>>>> > it up to date with the latest in main but stopped because it was
>>>>>> too much
>>>>>> > work. I was waiting until the ZK property changes are completed
>>>>>> before
>>>>>> > resolving the latest conflicts. I don't think it is much of a risk.
>>>>>> It is
>>>>>> > mostly cleanup and refactoring to remove generics from the
>>>>>> serialization
>>>>>> > code. It will be some work to revisit but I think the risk is
>>>>>> pretty low.
>>>>>> > It would allow changing the serialization, which we may be able to
>>>>>> get into
>>>>>> > 2.1 as well.
>>>>>> >
>>>>>> > [1] https://github.com/apache/accumulo/pull/2475
>>>>>> >
>>>>>> > On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org>
>>>>>> wrote:
>>>>>> > > >
>>>>>> > > > I haven't seen the metrics test fail very often lately. If it's
>>>>>> > stable, I
>>>>>> > > > don't mind removing the blocker on that issue, but I'd be
>>>>>> reluctant to
>>>>>> > > > close it entirely just yet, until we can verify it doesn't
>>>>>> happen
>>>>>> > > anymore.
>>>>>> > > >
>>>>>> > > > As for the original list of potential issues to include, I'm in
>>>>>> favor
>>>>>> > of
>>>>>> > > > trying to get #2197 in. It was started awhile ago, is
>>>>>> relatively simple
>>>>>> > > and
>>>>>> > > > well understood by several of us already... it just needs a bit
>>>>>> of
>>>>>> > > > attention to finalize reviews so it can be merged.
>>>>>> > > >
>>>>>> > > > However, I'm reluctant to include #2422, because I don't think
>>>>>> it's
>>>>>> > near
>>>>>> > > > ready enough, and by the time it is, it will be very last
>>>>>> minute, and I
>>>>>> > > > don't want to delay 2.1 further for it. Even if it's included
>>>>>> as an
>>>>>> > > > experimental feature, I think it has huge potential to be
>>>>>> disruptive,
>>>>>> > or
>>>>>> > > to
>>>>>> > > > have a lot of churn by the time people actually have a chance
>>>>>> to review
>>>>>> > > it
>>>>>> > > > thoroughly. Furthermore, I think there are possible
>>>>>> alternatives (like
>>>>>> > a
>>>>>> > > > fully client-side implementation, based on offline scanners)
>>>>>> that would
>>>>>> > > > avoid the tight coupling of a new service to Accumulo's core
>>>>>> code. This
>>>>>> > >
>>>>>> > > There are some advantages to scan servers over direct file access
>>>>>> to
>>>>>> > > consider.  One is scalability of computation, if a web server is
>>>>>> > > serving N client queries with scan servers those can potentially
>>>>>> go to
>>>>>> > > different scan servers.  With direct file access, all N queries
>>>>>> and
>>>>>> > > their iterator stacks would have to run in the web server.
>>>>>> Another is
>>>>>> > > scalability of caching/memory.  When web servers send queries to
>>>>>> scan
>>>>>> > > servers using a sticky algorithm for assigning tablets to groups
>>>>>> of
>>>>>> > > scan servers, it could lead to good cache utilization and sharing
>>>>>> that
>>>>>> > > may not be possible when running scans directly in the web
>>>>>> server. So
>>>>>> > > scan servers allow scaling cache and computations for queries
>>>>>> > > independently of web servers in way that may not be possible with
>>>>>> > > direct file access.
>>>>>> > >
>>>>>> > > Another advantage to consider is isolation.  With direct file
>>>>>> access
>>>>>> > > and queries running directly in a web server, a bad query could
>>>>>> bring
>>>>>> > > down a web server and lots of unrelated queries.  Having a bad
>>>>>> query
>>>>>> > > bring down a scan server may be less disruptive.
>>>>>> > >
>>>>>> > > > thread isn't for discussing this in depth, so we can have that
>>>>>> > discussion
>>>>>> > > > in a separate thread, but I'm generally opposed to including it
>>>>>> this
>>>>>> > late
>>>>>> > > > in 2.1's development, given the timing, size and scope, tight
>>>>>> coupling,
>>>>>> > > and
>>>>>> > > > current state.
>>>>>> > > >
>>>>>> > > > I don't know enough about #2475 to have a strong opinion, but
>>>>>> it looks
>>>>>> > > big,
>>>>>> > > > and possibly high-risk, given the critical code it touches. It
>>>>>> > currently
>>>>>> > > > has a substantial number of conflicts with the main branch.
>>>>>> However, I
>>>>>> > > was
>>>>>> > > > thinking that *some* minimal refactoring (like low-risk
>>>>>> automatic
>>>>>> > > > refactoring, like moving packages) could be done. So, if that's
>>>>>> all
>>>>>> > this
>>>>>> > > > does, it might be okay. Otherwise, maybe it can be simplified?
>>>>>> At the
>>>>>> > > very
>>>>>> > > > least, I was thinking it would be a good opportunity to move the
>>>>>> > > > `org.apache.accumulo.fate` packages into an appropriate
>>>>>> > > > `org.apache.accumulo.core` parent package (some would go to
>>>>>> > > o.a.a.core.fate
>>>>>> > > > and others might go to o.a.a.core.util or similar) to keep the
>>>>>> package
>>>>>> > > > namespaces standardized, which is helpful to avoid naming
>>>>>> collisions
>>>>>> > and
>>>>>> > > > jar sealing issues, as well as for less complicated jigsaw
>>>>>> module
>>>>>> > > > definitions in future. Since 2.1 FaTE is already incompatible
>>>>>> with
>>>>>> > prior
>>>>>> > > > versions, a rename at this time would be less disruptive.
>>>>>> > > >
>>>>>> > > > Another task I had wanted to be done for 2.1, before I got
>>>>>> distracted
>>>>>> > > > fixing test failures during and after Christmas and trying to
>>>>>> work
>>>>>> > > through
>>>>>> > > > the singleton manager zookeeper stuff to see what we could
>>>>>> simplify.
>>>>>> > > What I
>>>>>> > > > had wanted done was to standardize the way we pass table
>>>>>> identifiers
>>>>>> > > (name,
>>>>>> > > > IDs) across the RPC layer, since we currently do that
>>>>>> inconsistently. I
>>>>>> > > > don't remember if there's an existing ticket open for it, but I
>>>>>> have a
>>>>>> > > > working branch I had started working out of for it before
>>>>>> Christmas.
>>>>>> > It's
>>>>>> > > > relatively simple work, and would set us up for some much
>>>>>> better APIs
>>>>>> > > going
>>>>>> > > > forward, as well as help with logging information about table
>>>>>> actions.
>>>>>> > If
>>>>>> > > > necessary, it could be bumped to a future version, but then
>>>>>> we'd have
>>>>>> > > more
>>>>>> > > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to
>>>>>> avoid
>>>>>> > > that.
>>>>>> > > >
>>>>>> > > > As for planning, I was thinking early May for a code freeze
>>>>>> (except bug
>>>>>> > > > fixes and small improvements found during testing), so we can
>>>>>> try to
>>>>>> > > > release towards the end of May/early June. If we go with that
>>>>>> timeline,
>>>>>> > > > that's not a lot of time to wrap up features and have time for
>>>>>> > > > review/testing, so we may need to be selective about what we
>>>>>> hold off
>>>>>> > > until
>>>>>> > > > the next version, unless we want to further delay 2.1.
>>>>>> > > >
>>>>>> > > >
>>>>>> > > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dmarion18@gmail.com
>>>>>> >
>>>>>> > wrote:
>>>>>> > > >
>>>>>> > > > > I think [3] is OBE and can be closed.
>>>>>> > > > >
>>>>>> > > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <
>>>>>> mmiller@apache.org>
>>>>>> > wrote:
>>>>>> > > > >
>>>>>> > > > > > Yes I agree, that was the goal of this email thread. I
>>>>>> found a few
>>>>>> > > more
>>>>>> > > > > > tickets that should be addressed for the next release.
>>>>>> > > > > >
>>>>>> > > > > > Ivan - There was some work done on this PR but it has been
>>>>>> some
>>>>>> > > time. Do
>>>>>> > > > > > you want to take a look at it? Implement a Thread limit. [1]
>>>>>> > > > > > Keith T - I think we should get this one merged to fix that
>>>>>> > > consistency
>>>>>> > > > > > check bug I found. It looks like it is finished. [2]
>>>>>> > > > > > Dave & Dom - Were you guys able to figure out a fix for the
>>>>>> new
>>>>>> > > external
>>>>>> > > > > > compaction metrics test? [3]
>>>>>> > > > > >
>>>>>> > > > > > FYI we have 6 blockers for 2.1:
>>>>>> > > > > > https://github.com/apache/accumulo/labels/blocker
>>>>>> > > > > >
>>>>>> > > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
>>>>>> > > > > >
>>>>>> > > > > > [1] https://github.com/apache/accumulo/pull/1487
>>>>>> > > > > > [2] https://github.com/apache/accumulo/pull/2574
>>>>>> > > > > > [3] https://github.com/apache/accumulo/issues/2406
>>>>>> > > > > > [4] https://github.com/apache/accumulo/pull/2215
>>>>>> > > > > >
>>>>>> > > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <
>>>>>> dmarion18@gmail.com>
>>>>>> > > wrote:
>>>>>> > > > > >
>>>>>> > > > > > > I think it would be useful to do some release planning so
>>>>>> that we
>>>>>> > > know
>>>>>> > > > > > what
>>>>>> > > > > > > features we are working towards and in which release they
>>>>>> will be
>>>>>> > > in.
>>>>>> > > > > > This
>>>>>> > > > > > > would be helpful for determining what existing PRs need
>>>>>> to make
>>>>>> > it
>>>>>> > > into
>>>>>> > > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing
>>>>>> features
>>>>>> > > will
>>>>>> > > > > be
>>>>>> > > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as
>>>>>> defined in
>>>>>> > [1],
>>>>>> > > > > > features
>>>>>> > > > > > > that don't make it into 2.1.0 will go into the next
>>>>>> non-LTM
>>>>>> > release
>>>>>> > > > > > (2.2.0)
>>>>>> > > > > > > and any patches to bugs in those features will go into
>>>>>> the next
>>>>>> > > non-LTM
>>>>>> > > > > > > release after that (2.3.0).
>>>>>> > > > > > >
>>>>>> > > > > > > I'm not trying to hold up the 2.1.0 release by suggesting
>>>>>> that we
>>>>>> > > > > perform
>>>>>> > > > > > > this activity. I'm just asking what the future holds,
>>>>>> even if
>>>>>> > it's
>>>>>> > > just
>>>>>> > > > > > one
>>>>>> > > > > > > feature in the next non-LTM release. My concern is that
>>>>>> the next
>>>>>> > > > > release
>>>>>> > > > > > > will be open-ended and anything not included in 2.1.0
>>>>>> might not
>>>>>> > > get put
>>>>>> > > > > > > into a release for a very long time.
>>>>>> > > > > > >
>>>>>> > > > > > > [1]
>>>>>> https://accumulo.apache.org/contributor/versioning.html#LTM
>>>>>> > > > > > >
>>>>>> > > > > > >
>>>>>> > > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <
>>>>>> mmiller@apache.org
>>>>>> > >
>>>>>> > > > > wrote:
>>>>>> > > > > > >
>>>>>> > > > > > > > Starting an email chain of things that folks want to
>>>>>> finish for
>>>>>> > > 2.1.
>>>>>> > > > > > Here
>>>>>> > > > > > > > is what we currently have in the works that are most
>>>>>> likely
>>>>>> > going
>>>>>> > > > > into
>>>>>> > > > > > > 2.1:
>>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2569
>>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2600
>>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2293
>>>>>> > > > > > > >
>>>>>> > > > > > > > Some things that may go into 2.1:
>>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2422
>>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2475
>>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2197
>>>>>> > > > > > > >
>>>>>> > > > > > > > I created a Project for follow on work to the ZK
>>>>>> property
>>>>>> > > change. I
>>>>>> > > > > was
>>>>>> > > > > > > > planning on putting tasks in there that we want to
>>>>>> complete for
>>>>>> > > 2.1.
>>>>>> > > > > > But
>>>>>> > > > > > > we
>>>>>> > > > > > > > could also use it for post 2.1 work.
>>>>>> > > > > > > > https://github.com/apache/accumulo/projects/24
>>>>>> > > > > > > > https://github.com/apache/accumulo/issues/2469
>>>>>> > > > > > > >
>>>>>> > > > > > > > FYI a draft copy of the release notes has already been
>>>>>> on the
>>>>>> > > > > website:
>>>>>> > > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
>>>>>> > > > > > > >
>>>>>> > > > > > > > This may be a good thread to discuss whether or not a
>>>>>> task
>>>>>> > needs
>>>>>> > > to
>>>>>> > > > > go
>>>>>> > > > > > > into
>>>>>> > > > > > > > 2.1 or should wait for the next version. We currently
>>>>>> have 32
>>>>>> > > open
>>>>>> > > > > pull
>>>>>> > > > > > > > requests so please email me if there is one that you
>>>>>> would like
>>>>>> > > > > > > prioritized
>>>>>> > > > > > > > for 2.1.
>>>>>> > > > > > > >
>>>>>> > > > > > >
>>>>>> > > > > >
>>>>>> > > > >
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

It has been a month so I am sending a 2.1 update.
10 PRs in progress.
1,166 marked as DONE. (+36)
33 left in the TODO state. I went through these and dropped some from 2.1.
https://github.com/apache/accumulo/projects/3

3 marked as TODO as Compaction follow on work:
https://github.com/apache/accumulo/projects/13

2 in progress & 2 TODO for the Single Node ZK follow on work.
https://github.com/apache/accumulo/projects/24


On Fri, May 27, 2022 at 11:30 AM Mike Miller <mm...@apache.org> wrote:

> Update on 2.1 progress. Come on Folks, let's Hold The Line. [1]
> 14 Pull requests in progress. (-7 digression from last week)
> 1,130 marked as DONE. (+19)
> https://github.com/apache/accumulo/projects/3
>
> 3 marked as TODO as Compaction follow on work:
> https://github.com/apache/accumulo/projects/13
>
> 2 in progress & 2 TODO for the Single Node ZK follow on work.
> https://github.com/apache/accumulo/projects/24
>
> [1]: https://www.youtube.com/watch?v=htgr3pvBr-I
>
> On Fri, May 20, 2022 at 11:25 AM Mike Miller <mm...@apache.org> wrote:
>
>> Update on 2.1 progress.
>> 7 Pull requests in progress.
>> 1111 marked as DONE. ( I just missed sending this at 11:11 EST)
>> https://github.com/apache/accumulo/projects/3
>>
>> 3 marked as TODO as Compaction follow on work:
>> https://github.com/apache/accumulo/projects/13
>>
>> 2 marked as TODO for the Single Node ZK follow on work.
>> https://github.com/apache/accumulo/projects/24
>>
>>
>>
>> On Fri, May 13, 2022 at 9:38 AM Mike Miller <mm...@apache.org> wrote:
>>
>>> Update on 2.1 progress.
>>> 10 Pull Requests in progress.
>>> 1,097 marked as DONE.
>>> https://github.com/apache/accumulo/projects/3
>>>
>>> 3 Tickets marked as TODO as Compaction follow on work:
>>> https://github.com/apache/accumulo/projects/13
>>>
>>> Only 1 Ticket marked as TODO for the ZK follow on work. I thought there
>>> would be more here:
>>> https://github.com/apache/accumulo/projects/24
>>>
>>> On Fri, Apr 8, 2022 at 7:00 AM Mike Miller <mm...@apache.org> wrote:
>>>
>>>> Update on 2.1 progress. For pull requests:
>>>> 15 currently in progress.
>>>> 32 are open as TODO. But a lot of these will get bumped to the next
>>>> version.
>>>> 1,025 DONE. Wow! Good work everyone.
>>>>
>>>> https://github.com/apache/accumulo/projects/3
>>>>
>>>> On Wed, Apr 6, 2022 at 4:55 PM Christopher <ct...@apache.org> wrote:
>>>>
>>>>> After some additional consideration, and getting a better
>>>>> understanding of
>>>>> how the code is expected to work from discussing it with Dave... I'm a
>>>>> little more inclined to support #2422 in 2.1, provided:
>>>>>
>>>>> 1. There's time for me to review it,
>>>>> 2. It is sufficiently decoupled from the existing code and marked
>>>>> experimental, so that we have the flexibility to alter its design, if
>>>>> it
>>>>> seems appropriate after it gets some exposure after the release,
>>>>> 3. Unit tests and integration tests are reliably passing (as stable
>>>>> as, or
>>>>> more stable than, they are currently),
>>>>> 4. No serious issues are discovered during review, and
>>>>> 5. It doesn't delay a release past early June, as I think this is a
>>>>> reasonable target date.
>>>>>
>>>>> This my wishlist before I can get behind it with a +1 for 2.1. If these
>>>>> aren't met, I do not intend to veto, but I'd be a -0 on its inclusion
>>>>> to
>>>>> 2.1. Of course, once I review it, my thoughts may change a bit.
>>>>>
>>>>> On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org> wrote:
>>>>>
>>>>> > I think I can finish the FATE refactor PR [1] for 2.1. I had been
>>>>> keeping
>>>>> > it up to date with the latest in main but stopped because it was too
>>>>> much
>>>>> > work. I was waiting until the ZK property changes are completed
>>>>> before
>>>>> > resolving the latest conflicts. I don't think it is much of a risk.
>>>>> It is
>>>>> > mostly cleanup and refactoring to remove generics from the
>>>>> serialization
>>>>> > code. It will be some work to revisit but I think the risk is pretty
>>>>> low.
>>>>> > It would allow changing the serialization, which we may be able to
>>>>> get into
>>>>> > 2.1 as well.
>>>>> >
>>>>> > [1] https://github.com/apache/accumulo/pull/2475
>>>>> >
>>>>> > On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com>
>>>>> wrote:
>>>>> >
>>>>> > > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org>
>>>>> wrote:
>>>>> > > >
>>>>> > > > I haven't seen the metrics test fail very often lately. If it's
>>>>> > stable, I
>>>>> > > > don't mind removing the blocker on that issue, but I'd be
>>>>> reluctant to
>>>>> > > > close it entirely just yet, until we can verify it doesn't happen
>>>>> > > anymore.
>>>>> > > >
>>>>> > > > As for the original list of potential issues to include, I'm in
>>>>> favor
>>>>> > of
>>>>> > > > trying to get #2197 in. It was started awhile ago, is relatively
>>>>> simple
>>>>> > > and
>>>>> > > > well understood by several of us already... it just needs a bit
>>>>> of
>>>>> > > > attention to finalize reviews so it can be merged.
>>>>> > > >
>>>>> > > > However, I'm reluctant to include #2422, because I don't think
>>>>> it's
>>>>> > near
>>>>> > > > ready enough, and by the time it is, it will be very last
>>>>> minute, and I
>>>>> > > > don't want to delay 2.1 further for it. Even if it's included as
>>>>> an
>>>>> > > > experimental feature, I think it has huge potential to be
>>>>> disruptive,
>>>>> > or
>>>>> > > to
>>>>> > > > have a lot of churn by the time people actually have a chance to
>>>>> review
>>>>> > > it
>>>>> > > > thoroughly. Furthermore, I think there are possible alternatives
>>>>> (like
>>>>> > a
>>>>> > > > fully client-side implementation, based on offline scanners)
>>>>> that would
>>>>> > > > avoid the tight coupling of a new service to Accumulo's core
>>>>> code. This
>>>>> > >
>>>>> > > There are some advantages to scan servers over direct file access
>>>>> to
>>>>> > > consider.  One is scalability of computation, if a web server is
>>>>> > > serving N client queries with scan servers those can potentially
>>>>> go to
>>>>> > > different scan servers.  With direct file access, all N queries and
>>>>> > > their iterator stacks would have to run in the web server.
>>>>> Another is
>>>>> > > scalability of caching/memory.  When web servers send queries to
>>>>> scan
>>>>> > > servers using a sticky algorithm for assigning tablets to groups of
>>>>> > > scan servers, it could lead to good cache utilization and sharing
>>>>> that
>>>>> > > may not be possible when running scans directly in the web server.
>>>>> So
>>>>> > > scan servers allow scaling cache and computations for queries
>>>>> > > independently of web servers in way that may not be possible with
>>>>> > > direct file access.
>>>>> > >
>>>>> > > Another advantage to consider is isolation.  With direct file
>>>>> access
>>>>> > > and queries running directly in a web server, a bad query could
>>>>> bring
>>>>> > > down a web server and lots of unrelated queries.  Having a bad
>>>>> query
>>>>> > > bring down a scan server may be less disruptive.
>>>>> > >
>>>>> > > > thread isn't for discussing this in depth, so we can have that
>>>>> > discussion
>>>>> > > > in a separate thread, but I'm generally opposed to including it
>>>>> this
>>>>> > late
>>>>> > > > in 2.1's development, given the timing, size and scope, tight
>>>>> coupling,
>>>>> > > and
>>>>> > > > current state.
>>>>> > > >
>>>>> > > > I don't know enough about #2475 to have a strong opinion, but it
>>>>> looks
>>>>> > > big,
>>>>> > > > and possibly high-risk, given the critical code it touches. It
>>>>> > currently
>>>>> > > > has a substantial number of conflicts with the main branch.
>>>>> However, I
>>>>> > > was
>>>>> > > > thinking that *some* minimal refactoring (like low-risk automatic
>>>>> > > > refactoring, like moving packages) could be done. So, if that's
>>>>> all
>>>>> > this
>>>>> > > > does, it might be okay. Otherwise, maybe it can be simplified?
>>>>> At the
>>>>> > > very
>>>>> > > > least, I was thinking it would be a good opportunity to move the
>>>>> > > > `org.apache.accumulo.fate` packages into an appropriate
>>>>> > > > `org.apache.accumulo.core` parent package (some would go to
>>>>> > > o.a.a.core.fate
>>>>> > > > and others might go to o.a.a.core.util or similar) to keep the
>>>>> package
>>>>> > > > namespaces standardized, which is helpful to avoid naming
>>>>> collisions
>>>>> > and
>>>>> > > > jar sealing issues, as well as for less complicated jigsaw module
>>>>> > > > definitions in future. Since 2.1 FaTE is already incompatible
>>>>> with
>>>>> > prior
>>>>> > > > versions, a rename at this time would be less disruptive.
>>>>> > > >
>>>>> > > > Another task I had wanted to be done for 2.1, before I got
>>>>> distracted
>>>>> > > > fixing test failures during and after Christmas and trying to
>>>>> work
>>>>> > > through
>>>>> > > > the singleton manager zookeeper stuff to see what we could
>>>>> simplify.
>>>>> > > What I
>>>>> > > > had wanted done was to standardize the way we pass table
>>>>> identifiers
>>>>> > > (name,
>>>>> > > > IDs) across the RPC layer, since we currently do that
>>>>> inconsistently. I
>>>>> > > > don't remember if there's an existing ticket open for it, but I
>>>>> have a
>>>>> > > > working branch I had started working out of for it before
>>>>> Christmas.
>>>>> > It's
>>>>> > > > relatively simple work, and would set us up for some much better
>>>>> APIs
>>>>> > > going
>>>>> > > > forward, as well as help with logging information about table
>>>>> actions.
>>>>> > If
>>>>> > > > necessary, it could be bumped to a future version, but then we'd
>>>>> have
>>>>> > > more
>>>>> > > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to
>>>>> avoid
>>>>> > > that.
>>>>> > > >
>>>>> > > > As for planning, I was thinking early May for a code freeze
>>>>> (except bug
>>>>> > > > fixes and small improvements found during testing), so we can
>>>>> try to
>>>>> > > > release towards the end of May/early June. If we go with that
>>>>> timeline,
>>>>> > > > that's not a lot of time to wrap up features and have time for
>>>>> > > > review/testing, so we may need to be selective about what we
>>>>> hold off
>>>>> > > until
>>>>> > > > the next version, unless we want to further delay 2.1.
>>>>> > > >
>>>>> > > >
>>>>> > > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com>
>>>>> > wrote:
>>>>> > > >
>>>>> > > > > I think [3] is OBE and can be closed.
>>>>> > > > >
>>>>> > > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mmiller@apache.org
>>>>> >
>>>>> > wrote:
>>>>> > > > >
>>>>> > > > > > Yes I agree, that was the goal of this email thread. I found
>>>>> a few
>>>>> > > more
>>>>> > > > > > tickets that should be addressed for the next release.
>>>>> > > > > >
>>>>> > > > > > Ivan - There was some work done on this PR but it has been
>>>>> some
>>>>> > > time. Do
>>>>> > > > > > you want to take a look at it? Implement a Thread limit. [1]
>>>>> > > > > > Keith T - I think we should get this one merged to fix that
>>>>> > > consistency
>>>>> > > > > > check bug I found. It looks like it is finished. [2]
>>>>> > > > > > Dave & Dom - Were you guys able to figure out a fix for the
>>>>> new
>>>>> > > external
>>>>> > > > > > compaction metrics test? [3]
>>>>> > > > > >
>>>>> > > > > > FYI we have 6 blockers for 2.1:
>>>>> > > > > > https://github.com/apache/accumulo/labels/blocker
>>>>> > > > > >
>>>>> > > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
>>>>> > > > > >
>>>>> > > > > > [1] https://github.com/apache/accumulo/pull/1487
>>>>> > > > > > [2] https://github.com/apache/accumulo/pull/2574
>>>>> > > > > > [3] https://github.com/apache/accumulo/issues/2406
>>>>> > > > > > [4] https://github.com/apache/accumulo/pull/2215
>>>>> > > > > >
>>>>> > > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <
>>>>> dmarion18@gmail.com>
>>>>> > > wrote:
>>>>> > > > > >
>>>>> > > > > > > I think it would be useful to do some release planning so
>>>>> that we
>>>>> > > know
>>>>> > > > > > what
>>>>> > > > > > > features we are working towards and in which release they
>>>>> will be
>>>>> > > in.
>>>>> > > > > > This
>>>>> > > > > > > would be helpful for determining what existing PRs need to
>>>>> make
>>>>> > it
>>>>> > > into
>>>>> > > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing
>>>>> features
>>>>> > > will
>>>>> > > > > be
>>>>> > > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined
>>>>> in
>>>>> > [1],
>>>>> > > > > > features
>>>>> > > > > > > that don't make it into 2.1.0 will go into the next non-LTM
>>>>> > release
>>>>> > > > > > (2.2.0)
>>>>> > > > > > > and any patches to bugs in those features will go into the
>>>>> next
>>>>> > > non-LTM
>>>>> > > > > > > release after that (2.3.0).
>>>>> > > > > > >
>>>>> > > > > > > I'm not trying to hold up the 2.1.0 release by suggesting
>>>>> that we
>>>>> > > > > perform
>>>>> > > > > > > this activity. I'm just asking what the future holds, even
>>>>> if
>>>>> > it's
>>>>> > > just
>>>>> > > > > > one
>>>>> > > > > > > feature in the next non-LTM release. My concern is that
>>>>> the next
>>>>> > > > > release
>>>>> > > > > > > will be open-ended and anything not included in 2.1.0
>>>>> might not
>>>>> > > get put
>>>>> > > > > > > into a release for a very long time.
>>>>> > > > > > >
>>>>> > > > > > > [1]
>>>>> https://accumulo.apache.org/contributor/versioning.html#LTM
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <
>>>>> mmiller@apache.org
>>>>> > >
>>>>> > > > > wrote:
>>>>> > > > > > >
>>>>> > > > > > > > Starting an email chain of things that folks want to
>>>>> finish for
>>>>> > > 2.1.
>>>>> > > > > > Here
>>>>> > > > > > > > is what we currently have in the works that are most
>>>>> likely
>>>>> > going
>>>>> > > > > into
>>>>> > > > > > > 2.1:
>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2569
>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2600
>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2293
>>>>> > > > > > > >
>>>>> > > > > > > > Some things that may go into 2.1:
>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2422
>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2475
>>>>> > > > > > > > https://github.com/apache/accumulo/pull/2197
>>>>> > > > > > > >
>>>>> > > > > > > > I created a Project for follow on work to the ZK property
>>>>> > > change. I
>>>>> > > > > was
>>>>> > > > > > > > planning on putting tasks in there that we want to
>>>>> complete for
>>>>> > > 2.1.
>>>>> > > > > > But
>>>>> > > > > > > we
>>>>> > > > > > > > could also use it for post 2.1 work.
>>>>> > > > > > > > https://github.com/apache/accumulo/projects/24
>>>>> > > > > > > > https://github.com/apache/accumulo/issues/2469
>>>>> > > > > > > >
>>>>> > > > > > > > FYI a draft copy of the release notes has already been
>>>>> on the
>>>>> > > > > website:
>>>>> > > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
>>>>> > > > > > > >
>>>>> > > > > > > > This may be a good thread to discuss whether or not a
>>>>> task
>>>>> > needs
>>>>> > > to
>>>>> > > > > go
>>>>> > > > > > > into
>>>>> > > > > > > > 2.1 or should wait for the next version. We currently
>>>>> have 32
>>>>> > > open
>>>>> > > > > pull
>>>>> > > > > > > > requests so please email me if there is one that you
>>>>> would like
>>>>> > > > > > > prioritized
>>>>> > > > > > > > for 2.1.
>>>>> > > > > > > >
>>>>> > > > > > >
>>>>> > > > > >
>>>>> > > > >
>>>>> > >
>>>>> >
>>>>>
>>>>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

Update on 2.1 progress. Come on Folks, let's Hold The Line. [1]
14 Pull requests in progress. (-7 digression from last week)
1,130 marked as DONE. (+19)
https://github.com/apache/accumulo/projects/3

3 marked as TODO as Compaction follow on work:
https://github.com/apache/accumulo/projects/13

2 in progress & 2 TODO for the Single Node ZK follow on work.
https://github.com/apache/accumulo/projects/24

[1]: https://www.youtube.com/watch?v=htgr3pvBr-I

On Fri, May 20, 2022 at 11:25 AM Mike Miller <mm...@apache.org> wrote:

> Update on 2.1 progress.
> 7 Pull requests in progress.
> 1111 marked as DONE. ( I just missed sending this at 11:11 EST)
> https://github.com/apache/accumulo/projects/3
>
> 3 marked as TODO as Compaction follow on work:
> https://github.com/apache/accumulo/projects/13
>
> 2 marked as TODO for the Single Node ZK follow on work.
> https://github.com/apache/accumulo/projects/24
>
>
>
> On Fri, May 13, 2022 at 9:38 AM Mike Miller <mm...@apache.org> wrote:
>
>> Update on 2.1 progress.
>> 10 Pull Requests in progress.
>> 1,097 marked as DONE.
>> https://github.com/apache/accumulo/projects/3
>>
>> 3 Tickets marked as TODO as Compaction follow on work:
>> https://github.com/apache/accumulo/projects/13
>>
>> Only 1 Ticket marked as TODO for the ZK follow on work. I thought there
>> would be more here:
>> https://github.com/apache/accumulo/projects/24
>>
>> On Fri, Apr 8, 2022 at 7:00 AM Mike Miller <mm...@apache.org> wrote:
>>
>>> Update on 2.1 progress. For pull requests:
>>> 15 currently in progress.
>>> 32 are open as TODO. But a lot of these will get bumped to the next
>>> version.
>>> 1,025 DONE. Wow! Good work everyone.
>>>
>>> https://github.com/apache/accumulo/projects/3
>>>
>>> On Wed, Apr 6, 2022 at 4:55 PM Christopher <ct...@apache.org> wrote:
>>>
>>>> After some additional consideration, and getting a better understanding
>>>> of
>>>> how the code is expected to work from discussing it with Dave... I'm a
>>>> little more inclined to support #2422 in 2.1, provided:
>>>>
>>>> 1. There's time for me to review it,
>>>> 2. It is sufficiently decoupled from the existing code and marked
>>>> experimental, so that we have the flexibility to alter its design, if it
>>>> seems appropriate after it gets some exposure after the release,
>>>> 3. Unit tests and integration tests are reliably passing (as stable as,
>>>> or
>>>> more stable than, they are currently),
>>>> 4. No serious issues are discovered during review, and
>>>> 5. It doesn't delay a release past early June, as I think this is a
>>>> reasonable target date.
>>>>
>>>> This my wishlist before I can get behind it with a +1 for 2.1. If these
>>>> aren't met, I do not intend to veto, but I'd be a -0 on its inclusion to
>>>> 2.1. Of course, once I review it, my thoughts may change a bit.
>>>>
>>>> On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org> wrote:
>>>>
>>>> > I think I can finish the FATE refactor PR [1] for 2.1. I had been
>>>> keeping
>>>> > it up to date with the latest in main but stopped because it was too
>>>> much
>>>> > work. I was waiting until the ZK property changes are completed before
>>>> > resolving the latest conflicts. I don't think it is much of a risk.
>>>> It is
>>>> > mostly cleanup and refactoring to remove generics from the
>>>> serialization
>>>> > code. It will be some work to revisit but I think the risk is pretty
>>>> low.
>>>> > It would allow changing the serialization, which we may be able to
>>>> get into
>>>> > 2.1 as well.
>>>> >
>>>> > [1] https://github.com/apache/accumulo/pull/2475
>>>> >
>>>> > On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com>
>>>> wrote:
>>>> >
>>>> > > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org>
>>>> wrote:
>>>> > > >
>>>> > > > I haven't seen the metrics test fail very often lately. If it's
>>>> > stable, I
>>>> > > > don't mind removing the blocker on that issue, but I'd be
>>>> reluctant to
>>>> > > > close it entirely just yet, until we can verify it doesn't happen
>>>> > > anymore.
>>>> > > >
>>>> > > > As for the original list of potential issues to include, I'm in
>>>> favor
>>>> > of
>>>> > > > trying to get #2197 in. It was started awhile ago, is relatively
>>>> simple
>>>> > > and
>>>> > > > well understood by several of us already... it just needs a bit of
>>>> > > > attention to finalize reviews so it can be merged.
>>>> > > >
>>>> > > > However, I'm reluctant to include #2422, because I don't think
>>>> it's
>>>> > near
>>>> > > > ready enough, and by the time it is, it will be very last minute,
>>>> and I
>>>> > > > don't want to delay 2.1 further for it. Even if it's included as
>>>> an
>>>> > > > experimental feature, I think it has huge potential to be
>>>> disruptive,
>>>> > or
>>>> > > to
>>>> > > > have a lot of churn by the time people actually have a chance to
>>>> review
>>>> > > it
>>>> > > > thoroughly. Furthermore, I think there are possible alternatives
>>>> (like
>>>> > a
>>>> > > > fully client-side implementation, based on offline scanners) that
>>>> would
>>>> > > > avoid the tight coupling of a new service to Accumulo's core
>>>> code. This
>>>> > >
>>>> > > There are some advantages to scan servers over direct file access to
>>>> > > consider.  One is scalability of computation, if a web server is
>>>> > > serving N client queries with scan servers those can potentially go
>>>> to
>>>> > > different scan servers.  With direct file access, all N queries and
>>>> > > their iterator stacks would have to run in the web server.  Another
>>>> is
>>>> > > scalability of caching/memory.  When web servers send queries to
>>>> scan
>>>> > > servers using a sticky algorithm for assigning tablets to groups of
>>>> > > scan servers, it could lead to good cache utilization and sharing
>>>> that
>>>> > > may not be possible when running scans directly in the web server.
>>>> So
>>>> > > scan servers allow scaling cache and computations for queries
>>>> > > independently of web servers in way that may not be possible with
>>>> > > direct file access.
>>>> > >
>>>> > > Another advantage to consider is isolation.  With direct file access
>>>> > > and queries running directly in a web server, a bad query could
>>>> bring
>>>> > > down a web server and lots of unrelated queries.  Having a bad query
>>>> > > bring down a scan server may be less disruptive.
>>>> > >
>>>> > > > thread isn't for discussing this in depth, so we can have that
>>>> > discussion
>>>> > > > in a separate thread, but I'm generally opposed to including it
>>>> this
>>>> > late
>>>> > > > in 2.1's development, given the timing, size and scope, tight
>>>> coupling,
>>>> > > and
>>>> > > > current state.
>>>> > > >
>>>> > > > I don't know enough about #2475 to have a strong opinion, but it
>>>> looks
>>>> > > big,
>>>> > > > and possibly high-risk, given the critical code it touches. It
>>>> > currently
>>>> > > > has a substantial number of conflicts with the main branch.
>>>> However, I
>>>> > > was
>>>> > > > thinking that *some* minimal refactoring (like low-risk automatic
>>>> > > > refactoring, like moving packages) could be done. So, if that's
>>>> all
>>>> > this
>>>> > > > does, it might be okay. Otherwise, maybe it can be simplified? At
>>>> the
>>>> > > very
>>>> > > > least, I was thinking it would be a good opportunity to move the
>>>> > > > `org.apache.accumulo.fate` packages into an appropriate
>>>> > > > `org.apache.accumulo.core` parent package (some would go to
>>>> > > o.a.a.core.fate
>>>> > > > and others might go to o.a.a.core.util or similar) to keep the
>>>> package
>>>> > > > namespaces standardized, which is helpful to avoid naming
>>>> collisions
>>>> > and
>>>> > > > jar sealing issues, as well as for less complicated jigsaw module
>>>> > > > definitions in future. Since 2.1 FaTE is already incompatible with
>>>> > prior
>>>> > > > versions, a rename at this time would be less disruptive.
>>>> > > >
>>>> > > > Another task I had wanted to be done for 2.1, before I got
>>>> distracted
>>>> > > > fixing test failures during and after Christmas and trying to work
>>>> > > through
>>>> > > > the singleton manager zookeeper stuff to see what we could
>>>> simplify.
>>>> > > What I
>>>> > > > had wanted done was to standardize the way we pass table
>>>> identifiers
>>>> > > (name,
>>>> > > > IDs) across the RPC layer, since we currently do that
>>>> inconsistently. I
>>>> > > > don't remember if there's an existing ticket open for it, but I
>>>> have a
>>>> > > > working branch I had started working out of for it before
>>>> Christmas.
>>>> > It's
>>>> > > > relatively simple work, and would set us up for some much better
>>>> APIs
>>>> > > going
>>>> > > > forward, as well as help with logging information about table
>>>> actions.
>>>> > If
>>>> > > > necessary, it could be bumped to a future version, but then we'd
>>>> have
>>>> > > more
>>>> > > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to
>>>> avoid
>>>> > > that.
>>>> > > >
>>>> > > > As for planning, I was thinking early May for a code freeze
>>>> (except bug
>>>> > > > fixes and small improvements found during testing), so we can try
>>>> to
>>>> > > > release towards the end of May/early June. If we go with that
>>>> timeline,
>>>> > > > that's not a lot of time to wrap up features and have time for
>>>> > > > review/testing, so we may need to be selective about what we hold
>>>> off
>>>> > > until
>>>> > > > the next version, unless we want to further delay 2.1.
>>>> > > >
>>>> > > >
>>>> > > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com>
>>>> > wrote:
>>>> > > >
>>>> > > > > I think [3] is OBE and can be closed.
>>>> > > > >
>>>> > > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org>
>>>> > wrote:
>>>> > > > >
>>>> > > > > > Yes I agree, that was the goal of this email thread. I found
>>>> a few
>>>> > > more
>>>> > > > > > tickets that should be addressed for the next release.
>>>> > > > > >
>>>> > > > > > Ivan - There was some work done on this PR but it has been
>>>> some
>>>> > > time. Do
>>>> > > > > > you want to take a look at it? Implement a Thread limit. [1]
>>>> > > > > > Keith T - I think we should get this one merged to fix that
>>>> > > consistency
>>>> > > > > > check bug I found. It looks like it is finished. [2]
>>>> > > > > > Dave & Dom - Were you guys able to figure out a fix for the
>>>> new
>>>> > > external
>>>> > > > > > compaction metrics test? [3]
>>>> > > > > >
>>>> > > > > > FYI we have 6 blockers for 2.1:
>>>> > > > > > https://github.com/apache/accumulo/labels/blocker
>>>> > > > > >
>>>> > > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
>>>> > > > > >
>>>> > > > > > [1] https://github.com/apache/accumulo/pull/1487
>>>> > > > > > [2] https://github.com/apache/accumulo/pull/2574
>>>> > > > > > [3] https://github.com/apache/accumulo/issues/2406
>>>> > > > > > [4] https://github.com/apache/accumulo/pull/2215
>>>> > > > > >
>>>> > > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <
>>>> dmarion18@gmail.com>
>>>> > > wrote:
>>>> > > > > >
>>>> > > > > > > I think it would be useful to do some release planning so
>>>> that we
>>>> > > know
>>>> > > > > > what
>>>> > > > > > > features we are working towards and in which release they
>>>> will be
>>>> > > in.
>>>> > > > > > This
>>>> > > > > > > would be helpful for determining what existing PRs need to
>>>> make
>>>> > it
>>>> > > into
>>>> > > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing
>>>> features
>>>> > > will
>>>> > > > > be
>>>> > > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined
>>>> in
>>>> > [1],
>>>> > > > > > features
>>>> > > > > > > that don't make it into 2.1.0 will go into the next non-LTM
>>>> > release
>>>> > > > > > (2.2.0)
>>>> > > > > > > and any patches to bugs in those features will go into the
>>>> next
>>>> > > non-LTM
>>>> > > > > > > release after that (2.3.0).
>>>> > > > > > >
>>>> > > > > > > I'm not trying to hold up the 2.1.0 release by suggesting
>>>> that we
>>>> > > > > perform
>>>> > > > > > > this activity. I'm just asking what the future holds, even
>>>> if
>>>> > it's
>>>> > > just
>>>> > > > > > one
>>>> > > > > > > feature in the next non-LTM release. My concern is that the
>>>> next
>>>> > > > > release
>>>> > > > > > > will be open-ended and anything not included in 2.1.0 might
>>>> not
>>>> > > get put
>>>> > > > > > > into a release for a very long time.
>>>> > > > > > >
>>>> > > > > > > [1]
>>>> https://accumulo.apache.org/contributor/versioning.html#LTM
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <
>>>> mmiller@apache.org
>>>> > >
>>>> > > > > wrote:
>>>> > > > > > >
>>>> > > > > > > > Starting an email chain of things that folks want to
>>>> finish for
>>>> > > 2.1.
>>>> > > > > > Here
>>>> > > > > > > > is what we currently have in the works that are most
>>>> likely
>>>> > going
>>>> > > > > into
>>>> > > > > > > 2.1:
>>>> > > > > > > > https://github.com/apache/accumulo/pull/2569
>>>> > > > > > > > https://github.com/apache/accumulo/pull/2600
>>>> > > > > > > > https://github.com/apache/accumulo/pull/2293
>>>> > > > > > > >
>>>> > > > > > > > Some things that may go into 2.1:
>>>> > > > > > > > https://github.com/apache/accumulo/pull/2422
>>>> > > > > > > > https://github.com/apache/accumulo/pull/2475
>>>> > > > > > > > https://github.com/apache/accumulo/pull/2197
>>>> > > > > > > >
>>>> > > > > > > > I created a Project for follow on work to the ZK property
>>>> > > change. I
>>>> > > > > was
>>>> > > > > > > > planning on putting tasks in there that we want to
>>>> complete for
>>>> > > 2.1.
>>>> > > > > > But
>>>> > > > > > > we
>>>> > > > > > > > could also use it for post 2.1 work.
>>>> > > > > > > > https://github.com/apache/accumulo/projects/24
>>>> > > > > > > > https://github.com/apache/accumulo/issues/2469
>>>> > > > > > > >
>>>> > > > > > > > FYI a draft copy of the release notes has already been on
>>>> the
>>>> > > > > website:
>>>> > > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
>>>> > > > > > > >
>>>> > > > > > > > This may be a good thread to discuss whether or not a task
>>>> > needs
>>>> > > to
>>>> > > > > go
>>>> > > > > > > into
>>>> > > > > > > > 2.1 or should wait for the next version. We currently
>>>> have 32
>>>> > > open
>>>> > > > > pull
>>>> > > > > > > > requests so please email me if there is one that you
>>>> would like
>>>> > > > > > > prioritized
>>>> > > > > > > > for 2.1.
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > >
>>>> >
>>>>
>>>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

Update on 2.1 progress.
7 Pull requests in progress.
1111 marked as DONE. ( I just missed sending this at 11:11 EST)
https://github.com/apache/accumulo/projects/3

3 marked as TODO as Compaction follow on work:
https://github.com/apache/accumulo/projects/13

2 marked as TODO for the Single Node ZK follow on work.
https://github.com/apache/accumulo/projects/24



On Fri, May 13, 2022 at 9:38 AM Mike Miller <mm...@apache.org> wrote:

> Update on 2.1 progress.
> 10 Pull Requests in progress.
> 1,097 marked as DONE.
> https://github.com/apache/accumulo/projects/3
>
> 3 Tickets marked as TODO as Compaction follow on work:
> https://github.com/apache/accumulo/projects/13
>
> Only 1 Ticket marked as TODO for the ZK follow on work. I thought there
> would be more here:
> https://github.com/apache/accumulo/projects/24
>
> On Fri, Apr 8, 2022 at 7:00 AM Mike Miller <mm...@apache.org> wrote:
>
>> Update on 2.1 progress. For pull requests:
>> 15 currently in progress.
>> 32 are open as TODO. But a lot of these will get bumped to the next
>> version.
>> 1,025 DONE. Wow! Good work everyone.
>>
>> https://github.com/apache/accumulo/projects/3
>>
>> On Wed, Apr 6, 2022 at 4:55 PM Christopher <ct...@apache.org> wrote:
>>
>>> After some additional consideration, and getting a better understanding
>>> of
>>> how the code is expected to work from discussing it with Dave... I'm a
>>> little more inclined to support #2422 in 2.1, provided:
>>>
>>> 1. There's time for me to review it,
>>> 2. It is sufficiently decoupled from the existing code and marked
>>> experimental, so that we have the flexibility to alter its design, if it
>>> seems appropriate after it gets some exposure after the release,
>>> 3. Unit tests and integration tests are reliably passing (as stable as,
>>> or
>>> more stable than, they are currently),
>>> 4. No serious issues are discovered during review, and
>>> 5. It doesn't delay a release past early June, as I think this is a
>>> reasonable target date.
>>>
>>> This my wishlist before I can get behind it with a +1 for 2.1. If these
>>> aren't met, I do not intend to veto, but I'd be a -0 on its inclusion to
>>> 2.1. Of course, once I review it, my thoughts may change a bit.
>>>
>>> On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org> wrote:
>>>
>>> > I think I can finish the FATE refactor PR [1] for 2.1. I had been
>>> keeping
>>> > it up to date with the latest in main but stopped because it was too
>>> much
>>> > work. I was waiting until the ZK property changes are completed before
>>> > resolving the latest conflicts. I don't think it is much of a risk. It
>>> is
>>> > mostly cleanup and refactoring to remove generics from the
>>> serialization
>>> > code. It will be some work to revisit but I think the risk is pretty
>>> low.
>>> > It would allow changing the serialization, which we may be able to get
>>> into
>>> > 2.1 as well.
>>> >
>>> > [1] https://github.com/apache/accumulo/pull/2475
>>> >
>>> > On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:
>>> >
>>> > > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org>
>>> wrote:
>>> > > >
>>> > > > I haven't seen the metrics test fail very often lately. If it's
>>> > stable, I
>>> > > > don't mind removing the blocker on that issue, but I'd be
>>> reluctant to
>>> > > > close it entirely just yet, until we can verify it doesn't happen
>>> > > anymore.
>>> > > >
>>> > > > As for the original list of potential issues to include, I'm in
>>> favor
>>> > of
>>> > > > trying to get #2197 in. It was started awhile ago, is relatively
>>> simple
>>> > > and
>>> > > > well understood by several of us already... it just needs a bit of
>>> > > > attention to finalize reviews so it can be merged.
>>> > > >
>>> > > > However, I'm reluctant to include #2422, because I don't think it's
>>> > near
>>> > > > ready enough, and by the time it is, it will be very last minute,
>>> and I
>>> > > > don't want to delay 2.1 further for it. Even if it's included as an
>>> > > > experimental feature, I think it has huge potential to be
>>> disruptive,
>>> > or
>>> > > to
>>> > > > have a lot of churn by the time people actually have a chance to
>>> review
>>> > > it
>>> > > > thoroughly. Furthermore, I think there are possible alternatives
>>> (like
>>> > a
>>> > > > fully client-side implementation, based on offline scanners) that
>>> would
>>> > > > avoid the tight coupling of a new service to Accumulo's core code.
>>> This
>>> > >
>>> > > There are some advantages to scan servers over direct file access to
>>> > > consider.  One is scalability of computation, if a web server is
>>> > > serving N client queries with scan servers those can potentially go
>>> to
>>> > > different scan servers.  With direct file access, all N queries and
>>> > > their iterator stacks would have to run in the web server.  Another
>>> is
>>> > > scalability of caching/memory.  When web servers send queries to scan
>>> > > servers using a sticky algorithm for assigning tablets to groups of
>>> > > scan servers, it could lead to good cache utilization and sharing
>>> that
>>> > > may not be possible when running scans directly in the web server. So
>>> > > scan servers allow scaling cache and computations for queries
>>> > > independently of web servers in way that may not be possible with
>>> > > direct file access.
>>> > >
>>> > > Another advantage to consider is isolation.  With direct file access
>>> > > and queries running directly in a web server, a bad query could bring
>>> > > down a web server and lots of unrelated queries.  Having a bad query
>>> > > bring down a scan server may be less disruptive.
>>> > >
>>> > > > thread isn't for discussing this in depth, so we can have that
>>> > discussion
>>> > > > in a separate thread, but I'm generally opposed to including it
>>> this
>>> > late
>>> > > > in 2.1's development, given the timing, size and scope, tight
>>> coupling,
>>> > > and
>>> > > > current state.
>>> > > >
>>> > > > I don't know enough about #2475 to have a strong opinion, but it
>>> looks
>>> > > big,
>>> > > > and possibly high-risk, given the critical code it touches. It
>>> > currently
>>> > > > has a substantial number of conflicts with the main branch.
>>> However, I
>>> > > was
>>> > > > thinking that *some* minimal refactoring (like low-risk automatic
>>> > > > refactoring, like moving packages) could be done. So, if that's all
>>> > this
>>> > > > does, it might be okay. Otherwise, maybe it can be simplified? At
>>> the
>>> > > very
>>> > > > least, I was thinking it would be a good opportunity to move the
>>> > > > `org.apache.accumulo.fate` packages into an appropriate
>>> > > > `org.apache.accumulo.core` parent package (some would go to
>>> > > o.a.a.core.fate
>>> > > > and others might go to o.a.a.core.util or similar) to keep the
>>> package
>>> > > > namespaces standardized, which is helpful to avoid naming
>>> collisions
>>> > and
>>> > > > jar sealing issues, as well as for less complicated jigsaw module
>>> > > > definitions in future. Since 2.1 FaTE is already incompatible with
>>> > prior
>>> > > > versions, a rename at this time would be less disruptive.
>>> > > >
>>> > > > Another task I had wanted to be done for 2.1, before I got
>>> distracted
>>> > > > fixing test failures during and after Christmas and trying to work
>>> > > through
>>> > > > the singleton manager zookeeper stuff to see what we could
>>> simplify.
>>> > > What I
>>> > > > had wanted done was to standardize the way we pass table
>>> identifiers
>>> > > (name,
>>> > > > IDs) across the RPC layer, since we currently do that
>>> inconsistently. I
>>> > > > don't remember if there's an existing ticket open for it, but I
>>> have a
>>> > > > working branch I had started working out of for it before
>>> Christmas.
>>> > It's
>>> > > > relatively simple work, and would set us up for some much better
>>> APIs
>>> > > going
>>> > > > forward, as well as help with logging information about table
>>> actions.
>>> > If
>>> > > > necessary, it could be bumped to a future version, but then we'd
>>> have
>>> > > more
>>> > > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to
>>> avoid
>>> > > that.
>>> > > >
>>> > > > As for planning, I was thinking early May for a code freeze
>>> (except bug
>>> > > > fixes and small improvements found during testing), so we can try
>>> to
>>> > > > release towards the end of May/early June. If we go with that
>>> timeline,
>>> > > > that's not a lot of time to wrap up features and have time for
>>> > > > review/testing, so we may need to be selective about what we hold
>>> off
>>> > > until
>>> > > > the next version, unless we want to further delay 2.1.
>>> > > >
>>> > > >
>>> > > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com>
>>> > wrote:
>>> > > >
>>> > > > > I think [3] is OBE and can be closed.
>>> > > > >
>>> > > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org>
>>> > wrote:
>>> > > > >
>>> > > > > > Yes I agree, that was the goal of this email thread. I found a
>>> few
>>> > > more
>>> > > > > > tickets that should be addressed for the next release.
>>> > > > > >
>>> > > > > > Ivan - There was some work done on this PR but it has been some
>>> > > time. Do
>>> > > > > > you want to take a look at it? Implement a Thread limit. [1]
>>> > > > > > Keith T - I think we should get this one merged to fix that
>>> > > consistency
>>> > > > > > check bug I found. It looks like it is finished. [2]
>>> > > > > > Dave & Dom - Were you guys able to figure out a fix for the new
>>> > > external
>>> > > > > > compaction metrics test? [3]
>>> > > > > >
>>> > > > > > FYI we have 6 blockers for 2.1:
>>> > > > > > https://github.com/apache/accumulo/labels/blocker
>>> > > > > >
>>> > > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
>>> > > > > >
>>> > > > > > [1] https://github.com/apache/accumulo/pull/1487
>>> > > > > > [2] https://github.com/apache/accumulo/pull/2574
>>> > > > > > [3] https://github.com/apache/accumulo/issues/2406
>>> > > > > > [4] https://github.com/apache/accumulo/pull/2215
>>> > > > > >
>>> > > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <
>>> dmarion18@gmail.com>
>>> > > wrote:
>>> > > > > >
>>> > > > > > > I think it would be useful to do some release planning so
>>> that we
>>> > > know
>>> > > > > > what
>>> > > > > > > features we are working towards and in which release they
>>> will be
>>> > > in.
>>> > > > > > This
>>> > > > > > > would be helpful for determining what existing PRs need to
>>> make
>>> > it
>>> > > into
>>> > > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing
>>> features
>>> > > will
>>> > > > > be
>>> > > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in
>>> > [1],
>>> > > > > > features
>>> > > > > > > that don't make it into 2.1.0 will go into the next non-LTM
>>> > release
>>> > > > > > (2.2.0)
>>> > > > > > > and any patches to bugs in those features will go into the
>>> next
>>> > > non-LTM
>>> > > > > > > release after that (2.3.0).
>>> > > > > > >
>>> > > > > > > I'm not trying to hold up the 2.1.0 release by suggesting
>>> that we
>>> > > > > perform
>>> > > > > > > this activity. I'm just asking what the future holds, even if
>>> > it's
>>> > > just
>>> > > > > > one
>>> > > > > > > feature in the next non-LTM release. My concern is that the
>>> next
>>> > > > > release
>>> > > > > > > will be open-ended and anything not included in 2.1.0 might
>>> not
>>> > > get put
>>> > > > > > > into a release for a very long time.
>>> > > > > > >
>>> > > > > > > [1]
>>> https://accumulo.apache.org/contributor/versioning.html#LTM
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <
>>> mmiller@apache.org
>>> > >
>>> > > > > wrote:
>>> > > > > > >
>>> > > > > > > > Starting an email chain of things that folks want to
>>> finish for
>>> > > 2.1.
>>> > > > > > Here
>>> > > > > > > > is what we currently have in the works that are most likely
>>> > going
>>> > > > > into
>>> > > > > > > 2.1:
>>> > > > > > > > https://github.com/apache/accumulo/pull/2569
>>> > > > > > > > https://github.com/apache/accumulo/pull/2600
>>> > > > > > > > https://github.com/apache/accumulo/pull/2293
>>> > > > > > > >
>>> > > > > > > > Some things that may go into 2.1:
>>> > > > > > > > https://github.com/apache/accumulo/pull/2422
>>> > > > > > > > https://github.com/apache/accumulo/pull/2475
>>> > > > > > > > https://github.com/apache/accumulo/pull/2197
>>> > > > > > > >
>>> > > > > > > > I created a Project for follow on work to the ZK property
>>> > > change. I
>>> > > > > was
>>> > > > > > > > planning on putting tasks in there that we want to
>>> complete for
>>> > > 2.1.
>>> > > > > > But
>>> > > > > > > we
>>> > > > > > > > could also use it for post 2.1 work.
>>> > > > > > > > https://github.com/apache/accumulo/projects/24
>>> > > > > > > > https://github.com/apache/accumulo/issues/2469
>>> > > > > > > >
>>> > > > > > > > FYI a draft copy of the release notes has already been on
>>> the
>>> > > > > website:
>>> > > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
>>> > > > > > > >
>>> > > > > > > > This may be a good thread to discuss whether or not a task
>>> > needs
>>> > > to
>>> > > > > go
>>> > > > > > > into
>>> > > > > > > > 2.1 or should wait for the next version. We currently have
>>> 32
>>> > > open
>>> > > > > pull
>>> > > > > > > > requests so please email me if there is one that you would
>>> like
>>> > > > > > > prioritized
>>> > > > > > > > for 2.1.
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > >
>>> >
>>>
>>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

Update on 2.1 progress.
10 Pull Requests in progress.
1,097 marked as DONE.
https://github.com/apache/accumulo/projects/3

3 Tickets marked as TODO as Compaction follow on work:
https://github.com/apache/accumulo/projects/13

Only 1 Ticket marked as TODO for the ZK follow on work. I thought there
would be more here:
https://github.com/apache/accumulo/projects/24

On Fri, Apr 8, 2022 at 7:00 AM Mike Miller <mm...@apache.org> wrote:

> Update on 2.1 progress. For pull requests:
> 15 currently in progress.
> 32 are open as TODO. But a lot of these will get bumped to the next
> version.
> 1,025 DONE. Wow! Good work everyone.
>
> https://github.com/apache/accumulo/projects/3
>
> On Wed, Apr 6, 2022 at 4:55 PM Christopher <ct...@apache.org> wrote:
>
>> After some additional consideration, and getting a better understanding of
>> how the code is expected to work from discussing it with Dave... I'm a
>> little more inclined to support #2422 in 2.1, provided:
>>
>> 1. There's time for me to review it,
>> 2. It is sufficiently decoupled from the existing code and marked
>> experimental, so that we have the flexibility to alter its design, if it
>> seems appropriate after it gets some exposure after the release,
>> 3. Unit tests and integration tests are reliably passing (as stable as, or
>> more stable than, they are currently),
>> 4. No serious issues are discovered during review, and
>> 5. It doesn't delay a release past early June, as I think this is a
>> reasonable target date.
>>
>> This my wishlist before I can get behind it with a +1 for 2.1. If these
>> aren't met, I do not intend to veto, but I'd be a -0 on its inclusion to
>> 2.1. Of course, once I review it, my thoughts may change a bit.
>>
>> On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org> wrote:
>>
>> > I think I can finish the FATE refactor PR [1] for 2.1. I had been
>> keeping
>> > it up to date with the latest in main but stopped because it was too
>> much
>> > work. I was waiting until the ZK property changes are completed before
>> > resolving the latest conflicts. I don't think it is much of a risk. It
>> is
>> > mostly cleanup and refactoring to remove generics from the serialization
>> > code. It will be some work to revisit but I think the risk is pretty
>> low.
>> > It would allow changing the serialization, which we may be able to get
>> into
>> > 2.1 as well.
>> >
>> > [1] https://github.com/apache/accumulo/pull/2475
>> >
>> > On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:
>> >
>> > > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org>
>> wrote:
>> > > >
>> > > > I haven't seen the metrics test fail very often lately. If it's
>> > stable, I
>> > > > don't mind removing the blocker on that issue, but I'd be reluctant
>> to
>> > > > close it entirely just yet, until we can verify it doesn't happen
>> > > anymore.
>> > > >
>> > > > As for the original list of potential issues to include, I'm in
>> favor
>> > of
>> > > > trying to get #2197 in. It was started awhile ago, is relatively
>> simple
>> > > and
>> > > > well understood by several of us already... it just needs a bit of
>> > > > attention to finalize reviews so it can be merged.
>> > > >
>> > > > However, I'm reluctant to include #2422, because I don't think it's
>> > near
>> > > > ready enough, and by the time it is, it will be very last minute,
>> and I
>> > > > don't want to delay 2.1 further for it. Even if it's included as an
>> > > > experimental feature, I think it has huge potential to be
>> disruptive,
>> > or
>> > > to
>> > > > have a lot of churn by the time people actually have a chance to
>> review
>> > > it
>> > > > thoroughly. Furthermore, I think there are possible alternatives
>> (like
>> > a
>> > > > fully client-side implementation, based on offline scanners) that
>> would
>> > > > avoid the tight coupling of a new service to Accumulo's core code.
>> This
>> > >
>> > > There are some advantages to scan servers over direct file access to
>> > > consider.  One is scalability of computation, if a web server is
>> > > serving N client queries with scan servers those can potentially go to
>> > > different scan servers.  With direct file access, all N queries and
>> > > their iterator stacks would have to run in the web server.  Another is
>> > > scalability of caching/memory.  When web servers send queries to scan
>> > > servers using a sticky algorithm for assigning tablets to groups of
>> > > scan servers, it could lead to good cache utilization and sharing that
>> > > may not be possible when running scans directly in the web server. So
>> > > scan servers allow scaling cache and computations for queries
>> > > independently of web servers in way that may not be possible with
>> > > direct file access.
>> > >
>> > > Another advantage to consider is isolation.  With direct file access
>> > > and queries running directly in a web server, a bad query could bring
>> > > down a web server and lots of unrelated queries.  Having a bad query
>> > > bring down a scan server may be less disruptive.
>> > >
>> > > > thread isn't for discussing this in depth, so we can have that
>> > discussion
>> > > > in a separate thread, but I'm generally opposed to including it this
>> > late
>> > > > in 2.1's development, given the timing, size and scope, tight
>> coupling,
>> > > and
>> > > > current state.
>> > > >
>> > > > I don't know enough about #2475 to have a strong opinion, but it
>> looks
>> > > big,
>> > > > and possibly high-risk, given the critical code it touches. It
>> > currently
>> > > > has a substantial number of conflicts with the main branch.
>> However, I
>> > > was
>> > > > thinking that *some* minimal refactoring (like low-risk automatic
>> > > > refactoring, like moving packages) could be done. So, if that's all
>> > this
>> > > > does, it might be okay. Otherwise, maybe it can be simplified? At
>> the
>> > > very
>> > > > least, I was thinking it would be a good opportunity to move the
>> > > > `org.apache.accumulo.fate` packages into an appropriate
>> > > > `org.apache.accumulo.core` parent package (some would go to
>> > > o.a.a.core.fate
>> > > > and others might go to o.a.a.core.util or similar) to keep the
>> package
>> > > > namespaces standardized, which is helpful to avoid naming collisions
>> > and
>> > > > jar sealing issues, as well as for less complicated jigsaw module
>> > > > definitions in future. Since 2.1 FaTE is already incompatible with
>> > prior
>> > > > versions, a rename at this time would be less disruptive.
>> > > >
>> > > > Another task I had wanted to be done for 2.1, before I got
>> distracted
>> > > > fixing test failures during and after Christmas and trying to work
>> > > through
>> > > > the singleton manager zookeeper stuff to see what we could simplify.
>> > > What I
>> > > > had wanted done was to standardize the way we pass table identifiers
>> > > (name,
>> > > > IDs) across the RPC layer, since we currently do that
>> inconsistently. I
>> > > > don't remember if there's an existing ticket open for it, but I
>> have a
>> > > > working branch I had started working out of for it before Christmas.
>> > It's
>> > > > relatively simple work, and would set us up for some much better
>> APIs
>> > > going
>> > > > forward, as well as help with logging information about table
>> actions.
>> > If
>> > > > necessary, it could be bumped to a future version, but then we'd
>> have
>> > > more
>> > > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to avoid
>> > > that.
>> > > >
>> > > > As for planning, I was thinking early May for a code freeze (except
>> bug
>> > > > fixes and small improvements found during testing), so we can try to
>> > > > release towards the end of May/early June. If we go with that
>> timeline,
>> > > > that's not a lot of time to wrap up features and have time for
>> > > > review/testing, so we may need to be selective about what we hold
>> off
>> > > until
>> > > > the next version, unless we want to further delay 2.1.
>> > > >
>> > > >
>> > > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com>
>> > wrote:
>> > > >
>> > > > > I think [3] is OBE and can be closed.
>> > > > >
>> > > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org>
>> > wrote:
>> > > > >
>> > > > > > Yes I agree, that was the goal of this email thread. I found a
>> few
>> > > more
>> > > > > > tickets that should be addressed for the next release.
>> > > > > >
>> > > > > > Ivan - There was some work done on this PR but it has been some
>> > > time. Do
>> > > > > > you want to take a look at it? Implement a Thread limit. [1]
>> > > > > > Keith T - I think we should get this one merged to fix that
>> > > consistency
>> > > > > > check bug I found. It looks like it is finished. [2]
>> > > > > > Dave & Dom - Were you guys able to figure out a fix for the new
>> > > external
>> > > > > > compaction metrics test? [3]
>> > > > > >
>> > > > > > FYI we have 6 blockers for 2.1:
>> > > > > > https://github.com/apache/accumulo/labels/blocker
>> > > > > >
>> > > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
>> > > > > >
>> > > > > > [1] https://github.com/apache/accumulo/pull/1487
>> > > > > > [2] https://github.com/apache/accumulo/pull/2574
>> > > > > > [3] https://github.com/apache/accumulo/issues/2406
>> > > > > > [4] https://github.com/apache/accumulo/pull/2215
>> > > > > >
>> > > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dmarion18@gmail.com
>> >
>> > > wrote:
>> > > > > >
>> > > > > > > I think it would be useful to do some release planning so
>> that we
>> > > know
>> > > > > > what
>> > > > > > > features we are working towards and in which release they
>> will be
>> > > in.
>> > > > > > This
>> > > > > > > would be helpful for determining what existing PRs need to
>> make
>> > it
>> > > into
>> > > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing
>> features
>> > > will
>> > > > > be
>> > > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in
>> > [1],
>> > > > > > features
>> > > > > > > that don't make it into 2.1.0 will go into the next non-LTM
>> > release
>> > > > > > (2.2.0)
>> > > > > > > and any patches to bugs in those features will go into the
>> next
>> > > non-LTM
>> > > > > > > release after that (2.3.0).
>> > > > > > >
>> > > > > > > I'm not trying to hold up the 2.1.0 release by suggesting
>> that we
>> > > > > perform
>> > > > > > > this activity. I'm just asking what the future holds, even if
>> > it's
>> > > just
>> > > > > > one
>> > > > > > > feature in the next non-LTM release. My concern is that the
>> next
>> > > > > release
>> > > > > > > will be open-ended and anything not included in 2.1.0 might
>> not
>> > > get put
>> > > > > > > into a release for a very long time.
>> > > > > > >
>> > > > > > > [1]
>> https://accumulo.apache.org/contributor/versioning.html#LTM
>> > > > > > >
>> > > > > > >
>> > > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <
>> mmiller@apache.org
>> > >
>> > > > > wrote:
>> > > > > > >
>> > > > > > > > Starting an email chain of things that folks want to finish
>> for
>> > > 2.1.
>> > > > > > Here
>> > > > > > > > is what we currently have in the works that are most likely
>> > going
>> > > > > into
>> > > > > > > 2.1:
>> > > > > > > > https://github.com/apache/accumulo/pull/2569
>> > > > > > > > https://github.com/apache/accumulo/pull/2600
>> > > > > > > > https://github.com/apache/accumulo/pull/2293
>> > > > > > > >
>> > > > > > > > Some things that may go into 2.1:
>> > > > > > > > https://github.com/apache/accumulo/pull/2422
>> > > > > > > > https://github.com/apache/accumulo/pull/2475
>> > > > > > > > https://github.com/apache/accumulo/pull/2197
>> > > > > > > >
>> > > > > > > > I created a Project for follow on work to the ZK property
>> > > change. I
>> > > > > was
>> > > > > > > > planning on putting tasks in there that we want to complete
>> for
>> > > 2.1.
>> > > > > > But
>> > > > > > > we
>> > > > > > > > could also use it for post 2.1 work.
>> > > > > > > > https://github.com/apache/accumulo/projects/24
>> > > > > > > > https://github.com/apache/accumulo/issues/2469
>> > > > > > > >
>> > > > > > > > FYI a draft copy of the release notes has already been on
>> the
>> > > > > website:
>> > > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
>> > > > > > > >
>> > > > > > > > This may be a good thread to discuss whether or not a task
>> > needs
>> > > to
>> > > > > go
>> > > > > > > into
>> > > > > > > > 2.1 or should wait for the next version. We currently have
>> 32
>> > > open
>> > > > > pull
>> > > > > > > > requests so please email me if there is one that you would
>> like
>> > > > > > > prioritized
>> > > > > > > > for 2.1.
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > >
>> >
>>
>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

Update on 2.1 progress. For pull requests:
15 currently in progress.
32 are open as TODO. But a lot of these will get bumped to the next version.
1,025 DONE. Wow! Good work everyone.

https://github.com/apache/accumulo/projects/3

On Wed, Apr 6, 2022 at 4:55 PM Christopher <ct...@apache.org> wrote:

> After some additional consideration, and getting a better understanding of
> how the code is expected to work from discussing it with Dave... I'm a
> little more inclined to support #2422 in 2.1, provided:
>
> 1. There's time for me to review it,
> 2. It is sufficiently decoupled from the existing code and marked
> experimental, so that we have the flexibility to alter its design, if it
> seems appropriate after it gets some exposure after the release,
> 3. Unit tests and integration tests are reliably passing (as stable as, or
> more stable than, they are currently),
> 4. No serious issues are discovered during review, and
> 5. It doesn't delay a release past early June, as I think this is a
> reasonable target date.
>
> This my wishlist before I can get behind it with a +1 for 2.1. If these
> aren't met, I do not intend to veto, but I'd be a -0 on its inclusion to
> 2.1. Of course, once I review it, my thoughts may change a bit.
>
> On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org> wrote:
>
> > I think I can finish the FATE refactor PR [1] for 2.1. I had been keeping
> > it up to date with the latest in main but stopped because it was too much
> > work. I was waiting until the ZK property changes are completed before
> > resolving the latest conflicts. I don't think it is much of a risk. It is
> > mostly cleanup and refactoring to remove generics from the serialization
> > code. It will be some work to revisit but I think the risk is pretty low.
> > It would allow changing the serialization, which we may be able to get
> into
> > 2.1 as well.
> >
> > [1] https://github.com/apache/accumulo/pull/2475
> >
> > On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:
> >
> > > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org>
> wrote:
> > > >
> > > > I haven't seen the metrics test fail very often lately. If it's
> > stable, I
> > > > don't mind removing the blocker on that issue, but I'd be reluctant
> to
> > > > close it entirely just yet, until we can verify it doesn't happen
> > > anymore.
> > > >
> > > > As for the original list of potential issues to include, I'm in favor
> > of
> > > > trying to get #2197 in. It was started awhile ago, is relatively
> simple
> > > and
> > > > well understood by several of us already... it just needs a bit of
> > > > attention to finalize reviews so it can be merged.
> > > >
> > > > However, I'm reluctant to include #2422, because I don't think it's
> > near
> > > > ready enough, and by the time it is, it will be very last minute,
> and I
> > > > don't want to delay 2.1 further for it. Even if it's included as an
> > > > experimental feature, I think it has huge potential to be disruptive,
> > or
> > > to
> > > > have a lot of churn by the time people actually have a chance to
> review
> > > it
> > > > thoroughly. Furthermore, I think there are possible alternatives
> (like
> > a
> > > > fully client-side implementation, based on offline scanners) that
> would
> > > > avoid the tight coupling of a new service to Accumulo's core code.
> This
> > >
> > > There are some advantages to scan servers over direct file access to
> > > consider.  One is scalability of computation, if a web server is
> > > serving N client queries with scan servers those can potentially go to
> > > different scan servers.  With direct file access, all N queries and
> > > their iterator stacks would have to run in the web server.  Another is
> > > scalability of caching/memory.  When web servers send queries to scan
> > > servers using a sticky algorithm for assigning tablets to groups of
> > > scan servers, it could lead to good cache utilization and sharing that
> > > may not be possible when running scans directly in the web server. So
> > > scan servers allow scaling cache and computations for queries
> > > independently of web servers in way that may not be possible with
> > > direct file access.
> > >
> > > Another advantage to consider is isolation.  With direct file access
> > > and queries running directly in a web server, a bad query could bring
> > > down a web server and lots of unrelated queries.  Having a bad query
> > > bring down a scan server may be less disruptive.
> > >
> > > > thread isn't for discussing this in depth, so we can have that
> > discussion
> > > > in a separate thread, but I'm generally opposed to including it this
> > late
> > > > in 2.1's development, given the timing, size and scope, tight
> coupling,
> > > and
> > > > current state.
> > > >
> > > > I don't know enough about #2475 to have a strong opinion, but it
> looks
> > > big,
> > > > and possibly high-risk, given the critical code it touches. It
> > currently
> > > > has a substantial number of conflicts with the main branch. However,
> I
> > > was
> > > > thinking that *some* minimal refactoring (like low-risk automatic
> > > > refactoring, like moving packages) could be done. So, if that's all
> > this
> > > > does, it might be okay. Otherwise, maybe it can be simplified? At the
> > > very
> > > > least, I was thinking it would be a good opportunity to move the
> > > > `org.apache.accumulo.fate` packages into an appropriate
> > > > `org.apache.accumulo.core` parent package (some would go to
> > > o.a.a.core.fate
> > > > and others might go to o.a.a.core.util or similar) to keep the
> package
> > > > namespaces standardized, which is helpful to avoid naming collisions
> > and
> > > > jar sealing issues, as well as for less complicated jigsaw module
> > > > definitions in future. Since 2.1 FaTE is already incompatible with
> > prior
> > > > versions, a rename at this time would be less disruptive.
> > > >
> > > > Another task I had wanted to be done for 2.1, before I got distracted
> > > > fixing test failures during and after Christmas and trying to work
> > > through
> > > > the singleton manager zookeeper stuff to see what we could simplify.
> > > What I
> > > > had wanted done was to standardize the way we pass table identifiers
> > > (name,
> > > > IDs) across the RPC layer, since we currently do that
> inconsistently. I
> > > > don't remember if there's an existing ticket open for it, but I have
> a
> > > > working branch I had started working out of for it before Christmas.
> > It's
> > > > relatively simple work, and would set us up for some much better APIs
> > > going
> > > > forward, as well as help with logging information about table
> actions.
> > If
> > > > necessary, it could be bumped to a future version, but then we'd have
> > > more
> > > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to avoid
> > > that.
> > > >
> > > > As for planning, I was thinking early May for a code freeze (except
> bug
> > > > fixes and small improvements found during testing), so we can try to
> > > > release towards the end of May/early June. If we go with that
> timeline,
> > > > that's not a lot of time to wrap up features and have time for
> > > > review/testing, so we may need to be selective about what we hold off
> > > until
> > > > the next version, unless we want to further delay 2.1.
> > > >
> > > >
> > > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com>
> > wrote:
> > > >
> > > > > I think [3] is OBE and can be closed.
> > > > >
> > > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org>
> > wrote:
> > > > >
> > > > > > Yes I agree, that was the goal of this email thread. I found a
> few
> > > more
> > > > > > tickets that should be addressed for the next release.
> > > > > >
> > > > > > Ivan - There was some work done on this PR but it has been some
> > > time. Do
> > > > > > you want to take a look at it? Implement a Thread limit. [1]
> > > > > > Keith T - I think we should get this one merged to fix that
> > > consistency
> > > > > > check bug I found. It looks like it is finished. [2]
> > > > > > Dave & Dom - Were you guys able to figure out a fix for the new
> > > external
> > > > > > compaction metrics test? [3]
> > > > > >
> > > > > > FYI we have 6 blockers for 2.1:
> > > > > > https://github.com/apache/accumulo/labels/blocker
> > > > > >
> > > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
> > > > > >
> > > > > > [1] https://github.com/apache/accumulo/pull/1487
> > > > > > [2] https://github.com/apache/accumulo/pull/2574
> > > > > > [3] https://github.com/apache/accumulo/issues/2406
> > > > > > [4] https://github.com/apache/accumulo/pull/2215
> > > > > >
> > > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > I think it would be useful to do some release planning so that
> we
> > > know
> > > > > > what
> > > > > > > features we are working towards and in which release they will
> be
> > > in.
> > > > > > This
> > > > > > > would be helpful for determining what existing PRs need to make
> > it
> > > into
> > > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing
> features
> > > will
> > > > > be
> > > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in
> > [1],
> > > > > > features
> > > > > > > that don't make it into 2.1.0 will go into the next non-LTM
> > release
> > > > > > (2.2.0)
> > > > > > > and any patches to bugs in those features will go into the next
> > > non-LTM
> > > > > > > release after that (2.3.0).
> > > > > > >
> > > > > > > I'm not trying to hold up the 2.1.0 release by suggesting that
> we
> > > > > perform
> > > > > > > this activity. I'm just asking what the future holds, even if
> > it's
> > > just
> > > > > > one
> > > > > > > feature in the next non-LTM release. My concern is that the
> next
> > > > > release
> > > > > > > will be open-ended and anything not included in 2.1.0 might not
> > > get put
> > > > > > > into a release for a very long time.
> > > > > > >
> > > > > > > [1]
> https://accumulo.apache.org/contributor/versioning.html#LTM
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <
> mmiller@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Starting an email chain of things that folks want to finish
> for
> > > 2.1.
> > > > > > Here
> > > > > > > > is what we currently have in the works that are most likely
> > going
> > > > > into
> > > > > > > 2.1:
> > > > > > > > https://github.com/apache/accumulo/pull/2569
> > > > > > > > https://github.com/apache/accumulo/pull/2600
> > > > > > > > https://github.com/apache/accumulo/pull/2293
> > > > > > > >
> > > > > > > > Some things that may go into 2.1:
> > > > > > > > https://github.com/apache/accumulo/pull/2422
> > > > > > > > https://github.com/apache/accumulo/pull/2475
> > > > > > > > https://github.com/apache/accumulo/pull/2197
> > > > > > > >
> > > > > > > > I created a Project for follow on work to the ZK property
> > > change. I
> > > > > was
> > > > > > > > planning on putting tasks in there that we want to complete
> for
> > > 2.1.
> > > > > > But
> > > > > > > we
> > > > > > > > could also use it for post 2.1 work.
> > > > > > > > https://github.com/apache/accumulo/projects/24
> > > > > > > > https://github.com/apache/accumulo/issues/2469
> > > > > > > >
> > > > > > > > FYI a draft copy of the release notes has already been on the
> > > > > website:
> > > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
> > > > > > > >
> > > > > > > > This may be a good thread to discuss whether or not a task
> > needs
> > > to
> > > > > go
> > > > > > > into
> > > > > > > > 2.1 or should wait for the next version. We currently have 32
> > > open
> > > > > pull
> > > > > > > > requests so please email me if there is one that you would
> like
> > > > > > > prioritized
> > > > > > > > for 2.1.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>

Re: 2.1 Release TODO

[Christopher <ct...@apache.org>]

After some additional consideration, and getting a better understanding of
how the code is expected to work from discussing it with Dave... I'm a
little more inclined to support #2422 in 2.1, provided:

1. There's time for me to review it,
2. It is sufficiently decoupled from the existing code and marked
experimental, so that we have the flexibility to alter its design, if it
seems appropriate after it gets some exposure after the release,
3. Unit tests and integration tests are reliably passing (as stable as, or
more stable than, they are currently),
4. No serious issues are discovered during review, and
5. It doesn't delay a release past early June, as I think this is a
reasonable target date.

This my wishlist before I can get behind it with a +1 for 2.1. If these
aren't met, I do not intend to veto, but I'd be a -0 on its inclusion to
2.1. Of course, once I review it, my thoughts may change a bit.

On Mon, Apr 4, 2022 at 7:07 PM Mike Miller <mm...@apache.org> wrote:

> I think I can finish the FATE refactor PR [1] for 2.1. I had been keeping
> it up to date with the latest in main but stopped because it was too much
> work. I was waiting until the ZK property changes are completed before
> resolving the latest conflicts. I don't think it is much of a risk. It is
> mostly cleanup and refactoring to remove generics from the serialization
> code. It will be some work to revisit but I think the risk is pretty low.
> It would allow changing the serialization, which we may be able to get into
> 2.1 as well.
>
> [1] https://github.com/apache/accumulo/pull/2475
>
> On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:
>
> > On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org> wrote:
> > >
> > > I haven't seen the metrics test fail very often lately. If it's
> stable, I
> > > don't mind removing the blocker on that issue, but I'd be reluctant to
> > > close it entirely just yet, until we can verify it doesn't happen
> > anymore.
> > >
> > > As for the original list of potential issues to include, I'm in favor
> of
> > > trying to get #2197 in. It was started awhile ago, is relatively simple
> > and
> > > well understood by several of us already... it just needs a bit of
> > > attention to finalize reviews so it can be merged.
> > >
> > > However, I'm reluctant to include #2422, because I don't think it's
> near
> > > ready enough, and by the time it is, it will be very last minute, and I
> > > don't want to delay 2.1 further for it. Even if it's included as an
> > > experimental feature, I think it has huge potential to be disruptive,
> or
> > to
> > > have a lot of churn by the time people actually have a chance to review
> > it
> > > thoroughly. Furthermore, I think there are possible alternatives (like
> a
> > > fully client-side implementation, based on offline scanners) that would
> > > avoid the tight coupling of a new service to Accumulo's core code. This
> >
> > There are some advantages to scan servers over direct file access to
> > consider.  One is scalability of computation, if a web server is
> > serving N client queries with scan servers those can potentially go to
> > different scan servers.  With direct file access, all N queries and
> > their iterator stacks would have to run in the web server.  Another is
> > scalability of caching/memory.  When web servers send queries to scan
> > servers using a sticky algorithm for assigning tablets to groups of
> > scan servers, it could lead to good cache utilization and sharing that
> > may not be possible when running scans directly in the web server. So
> > scan servers allow scaling cache and computations for queries
> > independently of web servers in way that may not be possible with
> > direct file access.
> >
> > Another advantage to consider is isolation.  With direct file access
> > and queries running directly in a web server, a bad query could bring
> > down a web server and lots of unrelated queries.  Having a bad query
> > bring down a scan server may be less disruptive.
> >
> > > thread isn't for discussing this in depth, so we can have that
> discussion
> > > in a separate thread, but I'm generally opposed to including it this
> late
> > > in 2.1's development, given the timing, size and scope, tight coupling,
> > and
> > > current state.
> > >
> > > I don't know enough about #2475 to have a strong opinion, but it looks
> > big,
> > > and possibly high-risk, given the critical code it touches. It
> currently
> > > has a substantial number of conflicts with the main branch. However, I
> > was
> > > thinking that *some* minimal refactoring (like low-risk automatic
> > > refactoring, like moving packages) could be done. So, if that's all
> this
> > > does, it might be okay. Otherwise, maybe it can be simplified? At the
> > very
> > > least, I was thinking it would be a good opportunity to move the
> > > `org.apache.accumulo.fate` packages into an appropriate
> > > `org.apache.accumulo.core` parent package (some would go to
> > o.a.a.core.fate
> > > and others might go to o.a.a.core.util or similar) to keep the package
> > > namespaces standardized, which is helpful to avoid naming collisions
> and
> > > jar sealing issues, as well as for less complicated jigsaw module
> > > definitions in future. Since 2.1 FaTE is already incompatible with
> prior
> > > versions, a rename at this time would be less disruptive.
> > >
> > > Another task I had wanted to be done for 2.1, before I got distracted
> > > fixing test failures during and after Christmas and trying to work
> > through
> > > the singleton manager zookeeper stuff to see what we could simplify.
> > What I
> > > had wanted done was to standardize the way we pass table identifiers
> > (name,
> > > IDs) across the RPC layer, since we currently do that inconsistently. I
> > > don't remember if there's an existing ticket open for it, but I have a
> > > working branch I had started working out of for it before Christmas.
> It's
> > > relatively simple work, and would set us up for some much better APIs
> > going
> > > forward, as well as help with logging information about table actions.
> If
> > > necessary, it could be bumped to a future version, but then we'd have
> > more
> > > churn in the thrift layer. So, I'd prefer to get it for 2.1 to avoid
> > that.
> > >
> > > As for planning, I was thinking early May for a code freeze (except bug
> > > fixes and small improvements found during testing), so we can try to
> > > release towards the end of May/early June. If we go with that timeline,
> > > that's not a lot of time to wrap up features and have time for
> > > review/testing, so we may need to be selective about what we hold off
> > until
> > > the next version, unless we want to further delay 2.1.
> > >
> > >
> > > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com>
> wrote:
> > >
> > > > I think [3] is OBE and can be closed.
> > > >
> > > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org>
> wrote:
> > > >
> > > > > Yes I agree, that was the goal of this email thread. I found a few
> > more
> > > > > tickets that should be addressed for the next release.
> > > > >
> > > > > Ivan - There was some work done on this PR but it has been some
> > time. Do
> > > > > you want to take a look at it? Implement a Thread limit. [1]
> > > > > Keith T - I think we should get this one merged to fix that
> > consistency
> > > > > check bug I found. It looks like it is finished. [2]
> > > > > Dave & Dom - Were you guys able to figure out a fix for the new
> > external
> > > > > compaction metrics test? [3]
> > > > >
> > > > > FYI we have 6 blockers for 2.1:
> > > > > https://github.com/apache/accumulo/labels/blocker
> > > > >
> > > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
> > > > >
> > > > > [1] https://github.com/apache/accumulo/pull/1487
> > > > > [2] https://github.com/apache/accumulo/pull/2574
> > > > > [3] https://github.com/apache/accumulo/issues/2406
> > > > > [4] https://github.com/apache/accumulo/pull/2215
> > > > >
> > > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com>
> > wrote:
> > > > >
> > > > > > I think it would be useful to do some release planning so that we
> > know
> > > > > what
> > > > > > features we are working towards and in which release they will be
> > in.
> > > > > This
> > > > > > would be helpful for determining what existing PRs need to make
> it
> > into
> > > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing features
> > will
> > > > be
> > > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in
> [1],
> > > > > features
> > > > > > that don't make it into 2.1.0 will go into the next non-LTM
> release
> > > > > (2.2.0)
> > > > > > and any patches to bugs in those features will go into the next
> > non-LTM
> > > > > > release after that (2.3.0).
> > > > > >
> > > > > > I'm not trying to hold up the 2.1.0 release by suggesting that we
> > > > perform
> > > > > > this activity. I'm just asking what the future holds, even if
> it's
> > just
> > > > > one
> > > > > > feature in the next non-LTM release. My concern is that the next
> > > > release
> > > > > > will be open-ended and anything not included in 2.1.0 might not
> > get put
> > > > > > into a release for a very long time.
> > > > > >
> > > > > > [1] https://accumulo.apache.org/contributor/versioning.html#LTM
> > > > > >
> > > > > >
> > > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mmiller@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > > > Starting an email chain of things that folks want to finish for
> > 2.1.
> > > > > Here
> > > > > > > is what we currently have in the works that are most likely
> going
> > > > into
> > > > > > 2.1:
> > > > > > > https://github.com/apache/accumulo/pull/2569
> > > > > > > https://github.com/apache/accumulo/pull/2600
> > > > > > > https://github.com/apache/accumulo/pull/2293
> > > > > > >
> > > > > > > Some things that may go into 2.1:
> > > > > > > https://github.com/apache/accumulo/pull/2422
> > > > > > > https://github.com/apache/accumulo/pull/2475
> > > > > > > https://github.com/apache/accumulo/pull/2197
> > > > > > >
> > > > > > > I created a Project for follow on work to the ZK property
> > change. I
> > > > was
> > > > > > > planning on putting tasks in there that we want to complete for
> > 2.1.
> > > > > But
> > > > > > we
> > > > > > > could also use it for post 2.1 work.
> > > > > > > https://github.com/apache/accumulo/projects/24
> > > > > > > https://github.com/apache/accumulo/issues/2469
> > > > > > >
> > > > > > > FYI a draft copy of the release notes has already been on the
> > > > website:
> > > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
> > > > > > >
> > > > > > > This may be a good thread to discuss whether or not a task
> needs
> > to
> > > > go
> > > > > > into
> > > > > > > 2.1 or should wait for the next version. We currently have 32
> > open
> > > > pull
> > > > > > > requests so please email me if there is one that you would like
> > > > > > prioritized
> > > > > > > for 2.1.
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

I think I can finish the FATE refactor PR [1] for 2.1. I had been keeping
it up to date with the latest in main but stopped because it was too much
work. I was waiting until the ZK property changes are completed before
resolving the latest conflicts. I don't think it is much of a risk. It is
mostly cleanup and refactoring to remove generics from the serialization
code. It will be some work to revisit but I think the risk is pretty low.
It would allow changing the serialization, which we may be able to get into
2.1 as well.

[1] https://github.com/apache/accumulo/pull/2475

On Mon, Apr 4, 2022 at 11:50 AM Keith Turner <ke...@deenlo.com> wrote:

> On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org> wrote:
> >
> > I haven't seen the metrics test fail very often lately. If it's stable, I
> > don't mind removing the blocker on that issue, but I'd be reluctant to
> > close it entirely just yet, until we can verify it doesn't happen
> anymore.
> >
> > As for the original list of potential issues to include, I'm in favor of
> > trying to get #2197 in. It was started awhile ago, is relatively simple
> and
> > well understood by several of us already... it just needs a bit of
> > attention to finalize reviews so it can be merged.
> >
> > However, I'm reluctant to include #2422, because I don't think it's near
> > ready enough, and by the time it is, it will be very last minute, and I
> > don't want to delay 2.1 further for it. Even if it's included as an
> > experimental feature, I think it has huge potential to be disruptive, or
> to
> > have a lot of churn by the time people actually have a chance to review
> it
> > thoroughly. Furthermore, I think there are possible alternatives (like a
> > fully client-side implementation, based on offline scanners) that would
> > avoid the tight coupling of a new service to Accumulo's core code. This
>
> There are some advantages to scan servers over direct file access to
> consider.  One is scalability of computation, if a web server is
> serving N client queries with scan servers those can potentially go to
> different scan servers.  With direct file access, all N queries and
> their iterator stacks would have to run in the web server.  Another is
> scalability of caching/memory.  When web servers send queries to scan
> servers using a sticky algorithm for assigning tablets to groups of
> scan servers, it could lead to good cache utilization and sharing that
> may not be possible when running scans directly in the web server. So
> scan servers allow scaling cache and computations for queries
> independently of web servers in way that may not be possible with
> direct file access.
>
> Another advantage to consider is isolation.  With direct file access
> and queries running directly in a web server, a bad query could bring
> down a web server and lots of unrelated queries.  Having a bad query
> bring down a scan server may be less disruptive.
>
> > thread isn't for discussing this in depth, so we can have that discussion
> > in a separate thread, but I'm generally opposed to including it this late
> > in 2.1's development, given the timing, size and scope, tight coupling,
> and
> > current state.
> >
> > I don't know enough about #2475 to have a strong opinion, but it looks
> big,
> > and possibly high-risk, given the critical code it touches. It currently
> > has a substantial number of conflicts with the main branch. However, I
> was
> > thinking that *some* minimal refactoring (like low-risk automatic
> > refactoring, like moving packages) could be done. So, if that's all this
> > does, it might be okay. Otherwise, maybe it can be simplified? At the
> very
> > least, I was thinking it would be a good opportunity to move the
> > `org.apache.accumulo.fate` packages into an appropriate
> > `org.apache.accumulo.core` parent package (some would go to
> o.a.a.core.fate
> > and others might go to o.a.a.core.util or similar) to keep the package
> > namespaces standardized, which is helpful to avoid naming collisions and
> > jar sealing issues, as well as for less complicated jigsaw module
> > definitions in future. Since 2.1 FaTE is already incompatible with prior
> > versions, a rename at this time would be less disruptive.
> >
> > Another task I had wanted to be done for 2.1, before I got distracted
> > fixing test failures during and after Christmas and trying to work
> through
> > the singleton manager zookeeper stuff to see what we could simplify.
> What I
> > had wanted done was to standardize the way we pass table identifiers
> (name,
> > IDs) across the RPC layer, since we currently do that inconsistently. I
> > don't remember if there's an existing ticket open for it, but I have a
> > working branch I had started working out of for it before Christmas. It's
> > relatively simple work, and would set us up for some much better APIs
> going
> > forward, as well as help with logging information about table actions. If
> > necessary, it could be bumped to a future version, but then we'd have
> more
> > churn in the thrift layer. So, I'd prefer to get it for 2.1 to avoid
> that.
> >
> > As for planning, I was thinking early May for a code freeze (except bug
> > fixes and small improvements found during testing), so we can try to
> > release towards the end of May/early June. If we go with that timeline,
> > that's not a lot of time to wrap up features and have time for
> > review/testing, so we may need to be selective about what we hold off
> until
> > the next version, unless we want to further delay 2.1.
> >
> >
> > On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com> wrote:
> >
> > > I think [3] is OBE and can be closed.
> > >
> > > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org> wrote:
> > >
> > > > Yes I agree, that was the goal of this email thread. I found a few
> more
> > > > tickets that should be addressed for the next release.
> > > >
> > > > Ivan - There was some work done on this PR but it has been some
> time. Do
> > > > you want to take a look at it? Implement a Thread limit. [1]
> > > > Keith T - I think we should get this one merged to fix that
> consistency
> > > > check bug I found. It looks like it is finished. [2]
> > > > Dave & Dom - Were you guys able to figure out a fix for the new
> external
> > > > compaction metrics test? [3]
> > > >
> > > > FYI we have 6 blockers for 2.1:
> > > > https://github.com/apache/accumulo/labels/blocker
> > > >
> > > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
> > > >
> > > > [1] https://github.com/apache/accumulo/pull/1487
> > > > [2] https://github.com/apache/accumulo/pull/2574
> > > > [3] https://github.com/apache/accumulo/issues/2406
> > > > [4] https://github.com/apache/accumulo/pull/2215
> > > >
> > > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com>
> wrote:
> > > >
> > > > > I think it would be useful to do some release planning so that we
> know
> > > > what
> > > > > features we are working towards and in which release they will be
> in.
> > > > This
> > > > > would be helpful for determining what existing PRs need to make it
> into
> > > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing features
> will
> > > be
> > > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in [1],
> > > > features
> > > > > that don't make it into 2.1.0 will go into the next non-LTM release
> > > > (2.2.0)
> > > > > and any patches to bugs in those features will go into the next
> non-LTM
> > > > > release after that (2.3.0).
> > > > >
> > > > > I'm not trying to hold up the 2.1.0 release by suggesting that we
> > > perform
> > > > > this activity. I'm just asking what the future holds, even if it's
> just
> > > > one
> > > > > feature in the next non-LTM release. My concern is that the next
> > > release
> > > > > will be open-ended and anything not included in 2.1.0 might not
> get put
> > > > > into a release for a very long time.
> > > > >
> > > > > [1] https://accumulo.apache.org/contributor/versioning.html#LTM
> > > > >
> > > > >
> > > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mm...@apache.org>
> > > wrote:
> > > > >
> > > > > > Starting an email chain of things that folks want to finish for
> 2.1.
> > > > Here
> > > > > > is what we currently have in the works that are most likely going
> > > into
> > > > > 2.1:
> > > > > > https://github.com/apache/accumulo/pull/2569
> > > > > > https://github.com/apache/accumulo/pull/2600
> > > > > > https://github.com/apache/accumulo/pull/2293
> > > > > >
> > > > > > Some things that may go into 2.1:
> > > > > > https://github.com/apache/accumulo/pull/2422
> > > > > > https://github.com/apache/accumulo/pull/2475
> > > > > > https://github.com/apache/accumulo/pull/2197
> > > > > >
> > > > > > I created a Project for follow on work to the ZK property
> change. I
> > > was
> > > > > > planning on putting tasks in there that we want to complete for
> 2.1.
> > > > But
> > > > > we
> > > > > > could also use it for post 2.1 work.
> > > > > > https://github.com/apache/accumulo/projects/24
> > > > > > https://github.com/apache/accumulo/issues/2469
> > > > > >
> > > > > > FYI a draft copy of the release notes has already been on the
> > > website:
> > > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
> > > > > >
> > > > > > This may be a good thread to discuss whether or not a task needs
> to
> > > go
> > > > > into
> > > > > > 2.1 or should wait for the next version. We currently have 32
> open
> > > pull
> > > > > > requests so please email me if there is one that you would like
> > > > > prioritized
> > > > > > for 2.1.
> > > > > >
> > > > >
> > > >
> > >
>

Re: 2.1 Release TODO

[Keith Turner <ke...@deenlo.com>]

On Mon, Apr 4, 2022 at 11:17 AM Christopher <ct...@apache.org> wrote:
>
> I haven't seen the metrics test fail very often lately. If it's stable, I
> don't mind removing the blocker on that issue, but I'd be reluctant to
> close it entirely just yet, until we can verify it doesn't happen anymore.
>
> As for the original list of potential issues to include, I'm in favor of
> trying to get #2197 in. It was started awhile ago, is relatively simple and
> well understood by several of us already... it just needs a bit of
> attention to finalize reviews so it can be merged.
>
> However, I'm reluctant to include #2422, because I don't think it's near
> ready enough, and by the time it is, it will be very last minute, and I
> don't want to delay 2.1 further for it. Even if it's included as an
> experimental feature, I think it has huge potential to be disruptive, or to
> have a lot of churn by the time people actually have a chance to review it
> thoroughly. Furthermore, I think there are possible alternatives (like a
> fully client-side implementation, based on offline scanners) that would
> avoid the tight coupling of a new service to Accumulo's core code. This

There are some advantages to scan servers over direct file access to
consider.  One is scalability of computation, if a web server is
serving N client queries with scan servers those can potentially go to
different scan servers.  With direct file access, all N queries and
their iterator stacks would have to run in the web server.  Another is
scalability of caching/memory.  When web servers send queries to scan
servers using a sticky algorithm for assigning tablets to groups of
scan servers, it could lead to good cache utilization and sharing that
may not be possible when running scans directly in the web server. So
scan servers allow scaling cache and computations for queries
independently of web servers in way that may not be possible with
direct file access.

Another advantage to consider is isolation.  With direct file access
and queries running directly in a web server, a bad query could bring
down a web server and lots of unrelated queries.  Having a bad query
bring down a scan server may be less disruptive.

> thread isn't for discussing this in depth, so we can have that discussion
> in a separate thread, but I'm generally opposed to including it this late
> in 2.1's development, given the timing, size and scope, tight coupling, and
> current state.
>
> I don't know enough about #2475 to have a strong opinion, but it looks big,
> and possibly high-risk, given the critical code it touches. It currently
> has a substantial number of conflicts with the main branch. However, I was
> thinking that *some* minimal refactoring (like low-risk automatic
> refactoring, like moving packages) could be done. So, if that's all this
> does, it might be okay. Otherwise, maybe it can be simplified? At the very
> least, I was thinking it would be a good opportunity to move the
> `org.apache.accumulo.fate` packages into an appropriate
> `org.apache.accumulo.core` parent package (some would go to o.a.a.core.fate
> and others might go to o.a.a.core.util or similar) to keep the package
> namespaces standardized, which is helpful to avoid naming collisions and
> jar sealing issues, as well as for less complicated jigsaw module
> definitions in future. Since 2.1 FaTE is already incompatible with prior
> versions, a rename at this time would be less disruptive.
>
> Another task I had wanted to be done for 2.1, before I got distracted
> fixing test failures during and after Christmas and trying to work through
> the singleton manager zookeeper stuff to see what we could simplify. What I
> had wanted done was to standardize the way we pass table identifiers (name,
> IDs) across the RPC layer, since we currently do that inconsistently. I
> don't remember if there's an existing ticket open for it, but I have a
> working branch I had started working out of for it before Christmas. It's
> relatively simple work, and would set us up for some much better APIs going
> forward, as well as help with logging information about table actions. If
> necessary, it could be bumped to a future version, but then we'd have more
> churn in the thrift layer. So, I'd prefer to get it for 2.1 to avoid that.
>
> As for planning, I was thinking early May for a code freeze (except bug
> fixes and small improvements found during testing), so we can try to
> release towards the end of May/early June. If we go with that timeline,
> that's not a lot of time to wrap up features and have time for
> review/testing, so we may need to be selective about what we hold off until
> the next version, unless we want to further delay 2.1.
>
>
> On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com> wrote:
>
> > I think [3] is OBE and can be closed.
> >
> > On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org> wrote:
> >
> > > Yes I agree, that was the goal of this email thread. I found a few more
> > > tickets that should be addressed for the next release.
> > >
> > > Ivan - There was some work done on this PR but it has been some time. Do
> > > you want to take a look at it? Implement a Thread limit. [1]
> > > Keith T - I think we should get this one merged to fix that consistency
> > > check bug I found. It looks like it is finished. [2]
> > > Dave & Dom - Were you guys able to figure out a fix for the new external
> > > compaction metrics test? [3]
> > >
> > > FYI we have 6 blockers for 2.1:
> > > https://github.com/apache/accumulo/labels/blocker
> > >
> > > This is almost definitely going into 2.1 [4]. Thanks Jeff!
> > >
> > > [1] https://github.com/apache/accumulo/pull/1487
> > > [2] https://github.com/apache/accumulo/pull/2574
> > > [3] https://github.com/apache/accumulo/issues/2406
> > > [4] https://github.com/apache/accumulo/pull/2215
> > >
> > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com> wrote:
> > >
> > > > I think it would be useful to do some release planning so that we know
> > > what
> > > > features we are working towards and in which release they will be in.
> > > This
> > > > would be helpful for determining what existing PRs need to make it into
> > > > 2.1.0. 2.1.0 is the LTM release, so patches for existing features will
> > be
> > > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in [1],
> > > features
> > > > that don't make it into 2.1.0 will go into the next non-LTM release
> > > (2.2.0)
> > > > and any patches to bugs in those features will go into the next non-LTM
> > > > release after that (2.3.0).
> > > >
> > > > I'm not trying to hold up the 2.1.0 release by suggesting that we
> > perform
> > > > this activity. I'm just asking what the future holds, even if it's just
> > > one
> > > > feature in the next non-LTM release. My concern is that the next
> > release
> > > > will be open-ended and anything not included in 2.1.0 might not get put
> > > > into a release for a very long time.
> > > >
> > > > [1] https://accumulo.apache.org/contributor/versioning.html#LTM
> > > >
> > > >
> > > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mm...@apache.org>
> > wrote:
> > > >
> > > > > Starting an email chain of things that folks want to finish for 2.1.
> > > Here
> > > > > is what we currently have in the works that are most likely going
> > into
> > > > 2.1:
> > > > > https://github.com/apache/accumulo/pull/2569
> > > > > https://github.com/apache/accumulo/pull/2600
> > > > > https://github.com/apache/accumulo/pull/2293
> > > > >
> > > > > Some things that may go into 2.1:
> > > > > https://github.com/apache/accumulo/pull/2422
> > > > > https://github.com/apache/accumulo/pull/2475
> > > > > https://github.com/apache/accumulo/pull/2197
> > > > >
> > > > > I created a Project for follow on work to the ZK property change. I
> > was
> > > > > planning on putting tasks in there that we want to complete for 2.1.
> > > But
> > > > we
> > > > > could also use it for post 2.1 work.
> > > > > https://github.com/apache/accumulo/projects/24
> > > > > https://github.com/apache/accumulo/issues/2469
> > > > >
> > > > > FYI a draft copy of the release notes has already been on the
> > website:
> > > > > https://accumulo.apache.org/release/accumulo-2.1.0/
> > > > >
> > > > > This may be a good thread to discuss whether or not a task needs to
> > go
> > > > into
> > > > > 2.1 or should wait for the next version. We currently have 32 open
> > pull
> > > > > requests so please email me if there is one that you would like
> > > > prioritized
> > > > > for 2.1.
> > > > >
> > > >
> > >
> >

Re: 2.1 Release TODO

[Christopher <ct...@apache.org>]

I haven't seen the metrics test fail very often lately. If it's stable, I
don't mind removing the blocker on that issue, but I'd be reluctant to
close it entirely just yet, until we can verify it doesn't happen anymore.

As for the original list of potential issues to include, I'm in favor of
trying to get #2197 in. It was started awhile ago, is relatively simple and
well understood by several of us already... it just needs a bit of
attention to finalize reviews so it can be merged.

However, I'm reluctant to include #2422, because I don't think it's near
ready enough, and by the time it is, it will be very last minute, and I
don't want to delay 2.1 further for it. Even if it's included as an
experimental feature, I think it has huge potential to be disruptive, or to
have a lot of churn by the time people actually have a chance to review it
thoroughly. Furthermore, I think there are possible alternatives (like a
fully client-side implementation, based on offline scanners) that would
avoid the tight coupling of a new service to Accumulo's core code. This
thread isn't for discussing this in depth, so we can have that discussion
in a separate thread, but I'm generally opposed to including it this late
in 2.1's development, given the timing, size and scope, tight coupling, and
current state.

I don't know enough about #2475 to have a strong opinion, but it looks big,
and possibly high-risk, given the critical code it touches. It currently
has a substantial number of conflicts with the main branch. However, I was
thinking that *some* minimal refactoring (like low-risk automatic
refactoring, like moving packages) could be done. So, if that's all this
does, it might be okay. Otherwise, maybe it can be simplified? At the very
least, I was thinking it would be a good opportunity to move the
`org.apache.accumulo.fate` packages into an appropriate
`org.apache.accumulo.core` parent package (some would go to o.a.a.core.fate
and others might go to o.a.a.core.util or similar) to keep the package
namespaces standardized, which is helpful to avoid naming collisions and
jar sealing issues, as well as for less complicated jigsaw module
definitions in future. Since 2.1 FaTE is already incompatible with prior
versions, a rename at this time would be less disruptive.

Another task I had wanted to be done for 2.1, before I got distracted
fixing test failures during and after Christmas and trying to work through
the singleton manager zookeeper stuff to see what we could simplify. What I
had wanted done was to standardize the way we pass table identifiers (name,
IDs) across the RPC layer, since we currently do that inconsistently. I
don't remember if there's an existing ticket open for it, but I have a
working branch I had started working out of for it before Christmas. It's
relatively simple work, and would set us up for some much better APIs going
forward, as well as help with logging information about table actions. If
necessary, it could be bumped to a future version, but then we'd have more
churn in the thrift layer. So, I'd prefer to get it for 2.1 to avoid that.

As for planning, I was thinking early May for a code freeze (except bug
fixes and small improvements found during testing), so we can try to
release towards the end of May/early June. If we go with that timeline,
that's not a lot of time to wrap up features and have time for
review/testing, so we may need to be selective about what we hold off until
the next version, unless we want to further delay 2.1.


On Mon, Apr 4, 2022 at 9:13 AM Dave Marion <dm...@gmail.com> wrote:

> I think [3] is OBE and can be closed.
>
> On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org> wrote:
>
> > Yes I agree, that was the goal of this email thread. I found a few more
> > tickets that should be addressed for the next release.
> >
> > Ivan - There was some work done on this PR but it has been some time. Do
> > you want to take a look at it? Implement a Thread limit. [1]
> > Keith T - I think we should get this one merged to fix that consistency
> > check bug I found. It looks like it is finished. [2]
> > Dave & Dom - Were you guys able to figure out a fix for the new external
> > compaction metrics test? [3]
> >
> > FYI we have 6 blockers for 2.1:
> > https://github.com/apache/accumulo/labels/blocker
> >
> > This is almost definitely going into 2.1 [4]. Thanks Jeff!
> >
> > [1] https://github.com/apache/accumulo/pull/1487
> > [2] https://github.com/apache/accumulo/pull/2574
> > [3] https://github.com/apache/accumulo/issues/2406
> > [4] https://github.com/apache/accumulo/pull/2215
> >
> > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com> wrote:
> >
> > > I think it would be useful to do some release planning so that we know
> > what
> > > features we are working towards and in which release they will be in.
> > This
> > > would be helpful for determining what existing PRs need to make it into
> > > 2.1.0. 2.1.0 is the LTM release, so patches for existing features will
> be
> > > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in [1],
> > features
> > > that don't make it into 2.1.0 will go into the next non-LTM release
> > (2.2.0)
> > > and any patches to bugs in those features will go into the next non-LTM
> > > release after that (2.3.0).
> > >
> > > I'm not trying to hold up the 2.1.0 release by suggesting that we
> perform
> > > this activity. I'm just asking what the future holds, even if it's just
> > one
> > > feature in the next non-LTM release. My concern is that the next
> release
> > > will be open-ended and anything not included in 2.1.0 might not get put
> > > into a release for a very long time.
> > >
> > > [1] https://accumulo.apache.org/contributor/versioning.html#LTM
> > >
> > >
> > > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mm...@apache.org>
> wrote:
> > >
> > > > Starting an email chain of things that folks want to finish for 2.1.
> > Here
> > > > is what we currently have in the works that are most likely going
> into
> > > 2.1:
> > > > https://github.com/apache/accumulo/pull/2569
> > > > https://github.com/apache/accumulo/pull/2600
> > > > https://github.com/apache/accumulo/pull/2293
> > > >
> > > > Some things that may go into 2.1:
> > > > https://github.com/apache/accumulo/pull/2422
> > > > https://github.com/apache/accumulo/pull/2475
> > > > https://github.com/apache/accumulo/pull/2197
> > > >
> > > > I created a Project for follow on work to the ZK property change. I
> was
> > > > planning on putting tasks in there that we want to complete for 2.1.
> > But
> > > we
> > > > could also use it for post 2.1 work.
> > > > https://github.com/apache/accumulo/projects/24
> > > > https://github.com/apache/accumulo/issues/2469
> > > >
> > > > FYI a draft copy of the release notes has already been on the
> website:
> > > > https://accumulo.apache.org/release/accumulo-2.1.0/
> > > >
> > > > This may be a good thread to discuss whether or not a task needs to
> go
> > > into
> > > > 2.1 or should wait for the next version. We currently have 32 open
> pull
> > > > requests so please email me if there is one that you would like
> > > prioritized
> > > > for 2.1.
> > > >
> > >
> >
>

Re: 2.1 Release TODO

[Dave Marion <dm...@gmail.com>]

I think [3] is OBE and can be closed.

On Mon, Apr 4, 2022 at 9:11 AM Mike Miller <mm...@apache.org> wrote:

> Yes I agree, that was the goal of this email thread. I found a few more
> tickets that should be addressed for the next release.
>
> Ivan - There was some work done on this PR but it has been some time. Do
> you want to take a look at it? Implement a Thread limit. [1]
> Keith T - I think we should get this one merged to fix that consistency
> check bug I found. It looks like it is finished. [2]
> Dave & Dom - Were you guys able to figure out a fix for the new external
> compaction metrics test? [3]
>
> FYI we have 6 blockers for 2.1:
> https://github.com/apache/accumulo/labels/blocker
>
> This is almost definitely going into 2.1 [4]. Thanks Jeff!
>
> [1] https://github.com/apache/accumulo/pull/1487
> [2] https://github.com/apache/accumulo/pull/2574
> [3] https://github.com/apache/accumulo/issues/2406
> [4] https://github.com/apache/accumulo/pull/2215
>
> On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com> wrote:
>
> > I think it would be useful to do some release planning so that we know
> what
> > features we are working towards and in which release they will be in.
> This
> > would be helpful for determining what existing PRs need to make it into
> > 2.1.0. 2.1.0 is the LTM release, so patches for existing features will be
> > backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in [1],
> features
> > that don't make it into 2.1.0 will go into the next non-LTM release
> (2.2.0)
> > and any patches to bugs in those features will go into the next non-LTM
> > release after that (2.3.0).
> >
> > I'm not trying to hold up the 2.1.0 release by suggesting that we perform
> > this activity. I'm just asking what the future holds, even if it's just
> one
> > feature in the next non-LTM release. My concern is that the next release
> > will be open-ended and anything not included in 2.1.0 might not get put
> > into a release for a very long time.
> >
> > [1] https://accumulo.apache.org/contributor/versioning.html#LTM
> >
> >
> > On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mm...@apache.org> wrote:
> >
> > > Starting an email chain of things that folks want to finish for 2.1.
> Here
> > > is what we currently have in the works that are most likely going into
> > 2.1:
> > > https://github.com/apache/accumulo/pull/2569
> > > https://github.com/apache/accumulo/pull/2600
> > > https://github.com/apache/accumulo/pull/2293
> > >
> > > Some things that may go into 2.1:
> > > https://github.com/apache/accumulo/pull/2422
> > > https://github.com/apache/accumulo/pull/2475
> > > https://github.com/apache/accumulo/pull/2197
> > >
> > > I created a Project for follow on work to the ZK property change. I was
> > > planning on putting tasks in there that we want to complete for 2.1.
> But
> > we
> > > could also use it for post 2.1 work.
> > > https://github.com/apache/accumulo/projects/24
> > > https://github.com/apache/accumulo/issues/2469
> > >
> > > FYI a draft copy of the release notes has already been on the website:
> > > https://accumulo.apache.org/release/accumulo-2.1.0/
> > >
> > > This may be a good thread to discuss whether or not a task needs to go
> > into
> > > 2.1 or should wait for the next version. We currently have 32 open pull
> > > requests so please email me if there is one that you would like
> > prioritized
> > > for 2.1.
> > >
> >
>

Re: 2.1 Release TODO

[Mike Miller <mm...@apache.org>]

Yes I agree, that was the goal of this email thread. I found a few more
tickets that should be addressed for the next release.

Ivan - There was some work done on this PR but it has been some time. Do
you want to take a look at it? Implement a Thread limit. [1]
Keith T - I think we should get this one merged to fix that consistency
check bug I found. It looks like it is finished. [2]
Dave & Dom - Were you guys able to figure out a fix for the new external
compaction metrics test? [3]

FYI we have 6 blockers for 2.1:
https://github.com/apache/accumulo/labels/blocker

This is almost definitely going into 2.1 [4]. Thanks Jeff!

[1] https://github.com/apache/accumulo/pull/1487
[2] https://github.com/apache/accumulo/pull/2574
[3] https://github.com/apache/accumulo/issues/2406
[4] https://github.com/apache/accumulo/pull/2215

On Fri, Apr 1, 2022 at 2:21 PM Dave Marion <dm...@gmail.com> wrote:

> I think it would be useful to do some release planning so that we know what
> features we are working towards and in which release they will be in. This
> would be helpful for determining what existing PRs need to make it into
> 2.1.0. 2.1.0 is the LTM release, so patches for existing features will be
> backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in [1], features
> that don't make it into 2.1.0 will go into the next non-LTM release (2.2.0)
> and any patches to bugs in those features will go into the next non-LTM
> release after that (2.3.0).
>
> I'm not trying to hold up the 2.1.0 release by suggesting that we perform
> this activity. I'm just asking what the future holds, even if it's just one
> feature in the next non-LTM release. My concern is that the next release
> will be open-ended and anything not included in 2.1.0 might not get put
> into a release for a very long time.
>
> [1] https://accumulo.apache.org/contributor/versioning.html#LTM
>
>
> On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mm...@apache.org> wrote:
>
> > Starting an email chain of things that folks want to finish for 2.1. Here
> > is what we currently have in the works that are most likely going into
> 2.1:
> > https://github.com/apache/accumulo/pull/2569
> > https://github.com/apache/accumulo/pull/2600
> > https://github.com/apache/accumulo/pull/2293
> >
> > Some things that may go into 2.1:
> > https://github.com/apache/accumulo/pull/2422
> > https://github.com/apache/accumulo/pull/2475
> > https://github.com/apache/accumulo/pull/2197
> >
> > I created a Project for follow on work to the ZK property change. I was
> > planning on putting tasks in there that we want to complete for 2.1. But
> we
> > could also use it for post 2.1 work.
> > https://github.com/apache/accumulo/projects/24
> > https://github.com/apache/accumulo/issues/2469
> >
> > FYI a draft copy of the release notes has already been on the website:
> > https://accumulo.apache.org/release/accumulo-2.1.0/
> >
> > This may be a good thread to discuss whether or not a task needs to go
> into
> > 2.1 or should wait for the next version. We currently have 32 open pull
> > requests so please email me if there is one that you would like
> prioritized
> > for 2.1.
> >
>

Re: 2.1 Release TODO

[Dave Marion <dm...@gmail.com>]

I think it would be useful to do some release planning so that we know what
features we are working towards and in which release they will be in. This
would be helpful for determining what existing PRs need to make it into
2.1.0. 2.1.0 is the LTM release, so patches for existing features will be
backported (2.1.1, 2.1.2, 2.1.3, etc.) However, as defined in [1], features
that don't make it into 2.1.0 will go into the next non-LTM release (2.2.0)
and any patches to bugs in those features will go into the next non-LTM
release after that (2.3.0).

I'm not trying to hold up the 2.1.0 release by suggesting that we perform
this activity. I'm just asking what the future holds, even if it's just one
feature in the next non-LTM release. My concern is that the next release
will be open-ended and anything not included in 2.1.0 might not get put
into a release for a very long time.

[1] https://accumulo.apache.org/contributor/versioning.html#LTM


On Thu, Mar 31, 2022 at 11:43 AM Mike Miller <mm...@apache.org> wrote:

> Starting an email chain of things that folks want to finish for 2.1. Here
> is what we currently have in the works that are most likely going into 2.1:
> https://github.com/apache/accumulo/pull/2569
> https://github.com/apache/accumulo/pull/2600
> https://github.com/apache/accumulo/pull/2293
>
> Some things that may go into 2.1:
> https://github.com/apache/accumulo/pull/2422
> https://github.com/apache/accumulo/pull/2475
> https://github.com/apache/accumulo/pull/2197
>
> I created a Project for follow on work to the ZK property change. I was
> planning on putting tasks in there that we want to complete for 2.1. But we
> could also use it for post 2.1 work.
> https://github.com/apache/accumulo/projects/24
> https://github.com/apache/accumulo/issues/2469
>
> FYI a draft copy of the release notes has already been on the website:
> https://accumulo.apache.org/release/accumulo-2.1.0/
>
> This may be a good thread to discuss whether or not a task needs to go into
> 2.1 or should wait for the next version. We currently have 32 open pull
> requests so please email me if there is one that you would like prioritized
> for 2.1.
>