You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by David Smiley <da...@gmail.com> on 2020/03/02 18:07:16 UTC

CHANGES.txt and issue categorization

I'd like us to reflect on how we categorize issues in CHANGES.txt.  We have
these categories:
(Lucene) 'API Changes', 'New Features', 'Improvements', 'Optimizations',
'Bug Fixes', 'Other'
(Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes', 'Other
Changes'
(I lifted these from dev-tools/scripts/addVersion.py line 215)

In particular, I'm often surprised at how some of us categorize New
Features or Improvements that should better be categorized as something
else.  I think the root cause of these problems may be that we don't have
JIRA categories that directly align.  Furthermore, our dev practices will
typically result in a CHANGES.txt being added out of band from the
code-review process, and thus no peer-review on ideal placement.
Furthermore the message itself is often not code reviewed but should be.
Perhaps we can simply get in the habit of adding a JIRA comment (or GH code
review) what we propose the category & issue summary should be.

Here is my attempt at a definition for _some_ of these categories.  I don't
pretend to think we all agree 100% but it's up for discussion:
========
* New Features:  A user-visible new capability.  Usually opt-in.

* Improvements:  A user-visible improvement to an existing capability that
somehow expands its ability or that which improves the behavior.  Not a
refactoring, not an optimization.

* Optimizations: Something is now more efficient.  Usually automatic (not
opt-in).

* Other:  Anything else: Refactorings, tests, build, docs, etc.  And adding
log statements.
========

I recommend the following changes to Lucene 8.5:

These are "Improvements" that I think are better categorized as
"Optimizations"
* LUCENE-9211: Add compression for Binary doc value fields. (Mark Harwood)
* LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
* LUCENE-9228: Sort dvUpdates in the term order before applying if they all
update a
  single field to the same value. This optimization can reduce the flush
time by around
  20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand,
Simon Willnauer)
* LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant,
Robert Muir)
* LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)

These "Improvements" I think are better categorized as "Other":
* LUCENE-9109: Backport some changes from master (except StackWalker) to
improve
  TestSecurityManager (Uwe Schindler)
* LUCENE-9110: Backport refactored stack analysis in tests to use
generalized
  LuceneTestCase methods (Uwe Schindler)
* LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract
class called LatLonGeometry. Queries are
  executed with input objects that extend such interface. (Ignacio Vera)
* LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract class
called XYGeometry. Queries are
  executed with input objects that extend such interface. (Ignacio Vera)

Maybe this "Other" item should be  "Optimization"? (not sure):
* LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward,
Mike Drob)

Solr:

"New Features" that maybe should be "Improvements":
 * SOLR-13892: New "top-level" docValues join implementation (Jason
Gerlowski, Joel Bernstein)
 * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or
shapes. (Adrien Grand)

"Improvements" that maybe should be "Optimizations":
* SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query DSL
are cached by default (Mikhail Khludnev)

"Improvements" that maybe should be "Other":
* SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in
production (janhoy)

Thoughts?

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

Re: CHANGES.txt and issue categorization

Posted by Jason Gerlowski <ge...@gmail.com>.
> Furthermore the message itself is often not code reviewed but should be.

+1 to that point especially.  I know on a few things I've worked on
that once you get down in the weeds of the implementation, tests, etc.
... coming up with a good high-level "why might a user care" sentence
can be tough.  We should push each other more on getting that
peer-reviewed for the sake of users who have to dig through
CHANGES.txt.

On Thu, Mar 5, 2020 at 5:17 PM Houston Putman <ho...@gmail.com> wrote:
>
> +1 to move the entries.
>
> I would suggest that we document this organization somewhere though, so that future developers can adhere to the guidelines without finding this thread. I don't have a strong opinion on where this would go, maybe a legend at the top of CHANGES.txt or in the developer docs.
>
> I do agree that the JIRA categories should be aligned at some point, as that would likely help a lot.
>
> - Houston
>
> On Thu, Mar 5, 2020 at 5:00 PM Bruno Roustant <br...@gmail.com> wrote:
>>
>> +1 to move these entries. And I agree with the categories definitions.
>>
>> Le mer. 4 mars 2020 à 10:24, Adrien Grand <jp...@gmail.com> a écrit :
>>>
>>> +1 to move these entries.
>>>
>>> On Wed, Mar 4, 2020 at 4:27 AM David Smiley <da...@gmail.com> wrote:
>>>>
>>>> I'll simply move these items around tomorrow this time, unless I hear feedback to the contrary.
>>>>
>>>> ~ David Smiley
>>>> Apache Lucene/Solr Search Developer
>>>> http://www.linkedin.com/in/davidwsmiley
>>>>
>>>>
>>>> On Mon, Mar 2, 2020 at 1:07 PM David Smiley <da...@gmail.com> wrote:
>>>>>
>>>>> I'd like us to reflect on how we categorize issues in CHANGES.txt.  We have these categories:
>>>>> (Lucene) 'API Changes', 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes', 'Other'
>>>>> (Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes', 'Other Changes'
>>>>> (I lifted these from dev-tools/scripts/addVersion.py line 215)
>>>>>
>>>>> In particular, I'm often surprised at how some of us categorize New Features or Improvements that should better be categorized as something else.  I think the root cause of these problems may be that we don't have JIRA categories that directly align.  Furthermore, our dev practices will typically result in a CHANGES.txt being added out of band from the code-review process, and thus no peer-review on ideal placement.  Furthermore the message itself is often not code reviewed but should be.  Perhaps we can simply get in the habit of adding a JIRA comment (or GH code review) what we propose the category & issue summary should be.
>>>>>
>>>>> Here is my attempt at a definition for _some_ of these categories.  I don't pretend to think we all agree 100% but it's up for discussion:
>>>>> ========
>>>>> * New Features:  A user-visible new capability.  Usually opt-in.
>>>>>
>>>>> * Improvements:  A user-visible improvement to an existing capability that somehow expands its ability or that which improves the behavior.  Not a refactoring, not an optimization.
>>>>>
>>>>> * Optimizations: Something is now more efficient.  Usually automatic (not opt-in).
>>>>>
>>>>> * Other:  Anything else: Refactorings, tests, build, docs, etc.  And adding log statements.
>>>>> ========
>>>>>
>>>>> I recommend the following changes to Lucene 8.5:
>>>>>
>>>>> These are "Improvements" that I think are better categorized as "Optimizations"
>>>>> * LUCENE-9211: Add compression for Binary doc value fields. (Mark Harwood)
>>>>> * LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
>>>>> * LUCENE-9228: Sort dvUpdates in the term order before applying if they all update a
>>>>>   single field to the same value. This optimization can reduce the flush time by around
>>>>>   20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand, Simon Willnauer)
>>>>> * LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant, Robert Muir)
>>>>> * LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)
>>>>>
>>>>> These "Improvements" I think are better categorized as "Other":
>>>>> * LUCENE-9109: Backport some changes from master (except StackWalker) to improve
>>>>>   TestSecurityManager (Uwe Schindler)
>>>>> * LUCENE-9110: Backport refactored stack analysis in tests to use generalized
>>>>>   LuceneTestCase methods (Uwe Schindler)
>>>>> * LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract class called LatLonGeometry. Queries are
>>>>>   executed with input objects that extend such interface. (Ignacio Vera)
>>>>> * LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract class called XYGeometry. Queries are
>>>>>   executed with input objects that extend such interface. (Ignacio Vera)
>>>>>
>>>>> Maybe this "Other" item should be  "Optimization"? (not sure):
>>>>> * LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward, Mike Drob)
>>>>>
>>>>> Solr:
>>>>>
>>>>> "New Features" that maybe should be "Improvements":
>>>>>  * SOLR-13892: New "top-level" docValues join implementation (Jason Gerlowski, Joel Bernstein)
>>>>>  * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or shapes. (Adrien Grand)
>>>>>
>>>>> "Improvements" that maybe should be "Optimizations":
>>>>> * SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query DSL are cached by default (Mikhail Khludnev)
>>>>>
>>>>> "Improvements" that maybe should be "Other":
>>>>> * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in production (janhoy)
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> ~ David Smiley
>>>>> Apache Lucene/Solr Search Developer
>>>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>>
>>> --
>>> Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: CHANGES.txt and issue categorization

Posted by Houston Putman <ho...@gmail.com>.
+1 to move the entries.

I would suggest that we document this organization somewhere though, so
that future developers can adhere to the guidelines without finding this
thread. I don't have a strong opinion on where this would go, maybe a
legend at the top of CHANGES.txt or in the developer docs.

I do agree that the JIRA categories should be aligned at some point, as
that would likely help a lot.

- Houston

On Thu, Mar 5, 2020 at 5:00 PM Bruno Roustant <br...@gmail.com>
wrote:

> +1 to move these entries. And I agree with the categories definitions.
>
> Le mer. 4 mars 2020 à 10:24, Adrien Grand <jp...@gmail.com> a écrit :
>
>> +1 to move these entries.
>>
>> On Wed, Mar 4, 2020 at 4:27 AM David Smiley <da...@gmail.com>
>> wrote:
>>
>>> I'll simply move these items around tomorrow this time, unless I hear
>>> feedback to the contrary.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Mon, Mar 2, 2020 at 1:07 PM David Smiley <da...@gmail.com>
>>> wrote:
>>>
>>>> I'd like us to reflect on how we categorize issues in CHANGES.txt.  We
>>>> have these categories:
>>>> (Lucene) 'API Changes', 'New Features', 'Improvements',
>>>> 'Optimizations', 'Bug Fixes', 'Other'
>>>> (Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes',
>>>> 'Other Changes'
>>>> (I lifted these from dev-tools/scripts/addVersion.py line 215)
>>>>
>>>> In particular, I'm often surprised at how some of us categorize New
>>>> Features or Improvements that should better be categorized as something
>>>> else.  I think the root cause of these problems may be that we don't have
>>>> JIRA categories that directly align.  Furthermore, our dev practices will
>>>> typically result in a CHANGES.txt being added out of band from the
>>>> code-review process, and thus no peer-review on ideal placement.
>>>> Furthermore the message itself is often not code reviewed but should be.
>>>> Perhaps we can simply get in the habit of adding a JIRA comment (or GH code
>>>> review) what we propose the category & issue summary should be.
>>>>
>>>> Here is my attempt at a definition for _some_ of these categories.  I
>>>> don't pretend to think we all agree 100% but it's up for discussion:
>>>> ========
>>>> * New Features:  A user-visible new capability.  Usually opt-in.
>>>>
>>>> * Improvements:  A user-visible improvement to an existing capability
>>>> that somehow expands its ability or that which improves the behavior.  Not
>>>> a refactoring, not an optimization.
>>>>
>>>> * Optimizations: Something is now more efficient.  Usually automatic
>>>> (not opt-in).
>>>>
>>>> * Other:  Anything else: Refactorings, tests, build, docs, etc.  And
>>>> adding log statements.
>>>> ========
>>>>
>>>> I recommend the following changes to Lucene 8.5:
>>>>
>>>> These are "Improvements" that I think are better categorized as
>>>> "Optimizations"
>>>> * LUCENE-9211: Add compression for Binary doc value fields. (Mark
>>>> Harwood)
>>>> * LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
>>>> * LUCENE-9228: Sort dvUpdates in the term order before applying if they
>>>> all update a
>>>>   single field to the same value. This optimization can reduce the
>>>> flush time by around
>>>>   20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand,
>>>> Simon Willnauer)
>>>> * LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant,
>>>> Robert Muir)
>>>> * LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)
>>>>
>>>> These "Improvements" I think are better categorized as "Other":
>>>> * LUCENE-9109: Backport some changes from master (except StackWalker)
>>>> to improve
>>>>   TestSecurityManager (Uwe Schindler)
>>>> * LUCENE-9110: Backport refactored stack analysis in tests to use
>>>> generalized
>>>>   LuceneTestCase methods (Uwe Schindler)
>>>> * LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract
>>>> class called LatLonGeometry. Queries are
>>>>   executed with input objects that extend such interface. (Ignacio Vera)
>>>> * LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract
>>>> class called XYGeometry. Queries are
>>>>   executed with input objects that extend such interface. (Ignacio Vera)
>>>>
>>>> Maybe this "Other" item should be  "Optimization"? (not sure):
>>>> * LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward,
>>>> Mike Drob)
>>>>
>>>> Solr:
>>>>
>>>> "New Features" that maybe should be "Improvements":
>>>>  * SOLR-13892: New "top-level" docValues join implementation (Jason
>>>> Gerlowski, Joel Bernstein)
>>>>  * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges
>>>> or shapes. (Adrien Grand)
>>>>
>>>> "Improvements" that maybe should be "Optimizations":
>>>> * SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query
>>>> DSL are cached by default (Mikhail Khludnev)
>>>>
>>>> "Improvements" that maybe should be "Other":
>>>> * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in
>>>> production (janhoy)
>>>>
>>>> Thoughts?
>>>>
>>>> ~ David Smiley
>>>> Apache Lucene/Solr Search Developer
>>>> http://www.linkedin.com/in/davidwsmiley
>>>>
>>>
>>
>> --
>> Adrien
>>
>

Re: CHANGES.txt and issue categorization

Posted by Bruno Roustant <br...@gmail.com>.
+1 to move these entries. And I agree with the categories definitions.

Le mer. 4 mars 2020 à 10:24, Adrien Grand <jp...@gmail.com> a écrit :

> +1 to move these entries.
>
> On Wed, Mar 4, 2020 at 4:27 AM David Smiley <da...@gmail.com>
> wrote:
>
>> I'll simply move these items around tomorrow this time, unless I hear
>> feedback to the contrary.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Mon, Mar 2, 2020 at 1:07 PM David Smiley <da...@gmail.com>
>> wrote:
>>
>>> I'd like us to reflect on how we categorize issues in CHANGES.txt.  We
>>> have these categories:
>>> (Lucene) 'API Changes', 'New Features', 'Improvements', 'Optimizations',
>>> 'Bug Fixes', 'Other'
>>> (Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes',
>>> 'Other Changes'
>>> (I lifted these from dev-tools/scripts/addVersion.py line 215)
>>>
>>> In particular, I'm often surprised at how some of us categorize New
>>> Features or Improvements that should better be categorized as something
>>> else.  I think the root cause of these problems may be that we don't have
>>> JIRA categories that directly align.  Furthermore, our dev practices will
>>> typically result in a CHANGES.txt being added out of band from the
>>> code-review process, and thus no peer-review on ideal placement.
>>> Furthermore the message itself is often not code reviewed but should be.
>>> Perhaps we can simply get in the habit of adding a JIRA comment (or GH code
>>> review) what we propose the category & issue summary should be.
>>>
>>> Here is my attempt at a definition for _some_ of these categories.  I
>>> don't pretend to think we all agree 100% but it's up for discussion:
>>> ========
>>> * New Features:  A user-visible new capability.  Usually opt-in.
>>>
>>> * Improvements:  A user-visible improvement to an existing capability
>>> that somehow expands its ability or that which improves the behavior.  Not
>>> a refactoring, not an optimization.
>>>
>>> * Optimizations: Something is now more efficient.  Usually automatic
>>> (not opt-in).
>>>
>>> * Other:  Anything else: Refactorings, tests, build, docs, etc.  And
>>> adding log statements.
>>> ========
>>>
>>> I recommend the following changes to Lucene 8.5:
>>>
>>> These are "Improvements" that I think are better categorized as
>>> "Optimizations"
>>> * LUCENE-9211: Add compression for Binary doc value fields. (Mark
>>> Harwood)
>>> * LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
>>> * LUCENE-9228: Sort dvUpdates in the term order before applying if they
>>> all update a
>>>   single field to the same value. This optimization can reduce the flush
>>> time by around
>>>   20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand,
>>> Simon Willnauer)
>>> * LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant,
>>> Robert Muir)
>>> * LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)
>>>
>>> These "Improvements" I think are better categorized as "Other":
>>> * LUCENE-9109: Backport some changes from master (except StackWalker) to
>>> improve
>>>   TestSecurityManager (Uwe Schindler)
>>> * LUCENE-9110: Backport refactored stack analysis in tests to use
>>> generalized
>>>   LuceneTestCase methods (Uwe Schindler)
>>> * LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract
>>> class called LatLonGeometry. Queries are
>>>   executed with input objects that extend such interface. (Ignacio Vera)
>>> * LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract class
>>> called XYGeometry. Queries are
>>>   executed with input objects that extend such interface. (Ignacio Vera)
>>>
>>> Maybe this "Other" item should be  "Optimization"? (not sure):
>>> * LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward,
>>> Mike Drob)
>>>
>>> Solr:
>>>
>>> "New Features" that maybe should be "Improvements":
>>>  * SOLR-13892: New "top-level" docValues join implementation (Jason
>>> Gerlowski, Joel Bernstein)
>>>  * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or
>>> shapes. (Adrien Grand)
>>>
>>> "Improvements" that maybe should be "Optimizations":
>>> * SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query
>>> DSL are cached by default (Mikhail Khludnev)
>>>
>>> "Improvements" that maybe should be "Other":
>>> * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in
>>> production (janhoy)
>>>
>>> Thoughts?
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>
>
> --
> Adrien
>

Re: CHANGES.txt and issue categorization

Posted by Adrien Grand <jp...@gmail.com>.
+1 to move these entries.

On Wed, Mar 4, 2020 at 4:27 AM David Smiley <da...@gmail.com>
wrote:

> I'll simply move these items around tomorrow this time, unless I hear
> feedback to the contrary.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, Mar 2, 2020 at 1:07 PM David Smiley <da...@gmail.com>
> wrote:
>
>> I'd like us to reflect on how we categorize issues in CHANGES.txt.  We
>> have these categories:
>> (Lucene) 'API Changes', 'New Features', 'Improvements', 'Optimizations',
>> 'Bug Fixes', 'Other'
>> (Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes',
>> 'Other Changes'
>> (I lifted these from dev-tools/scripts/addVersion.py line 215)
>>
>> In particular, I'm often surprised at how some of us categorize New
>> Features or Improvements that should better be categorized as something
>> else.  I think the root cause of these problems may be that we don't have
>> JIRA categories that directly align.  Furthermore, our dev practices will
>> typically result in a CHANGES.txt being added out of band from the
>> code-review process, and thus no peer-review on ideal placement.
>> Furthermore the message itself is often not code reviewed but should be.
>> Perhaps we can simply get in the habit of adding a JIRA comment (or GH code
>> review) what we propose the category & issue summary should be.
>>
>> Here is my attempt at a definition for _some_ of these categories.  I
>> don't pretend to think we all agree 100% but it's up for discussion:
>> ========
>> * New Features:  A user-visible new capability.  Usually opt-in.
>>
>> * Improvements:  A user-visible improvement to an existing capability
>> that somehow expands its ability or that which improves the behavior.  Not
>> a refactoring, not an optimization.
>>
>> * Optimizations: Something is now more efficient.  Usually automatic (not
>> opt-in).
>>
>> * Other:  Anything else: Refactorings, tests, build, docs, etc.  And
>> adding log statements.
>> ========
>>
>> I recommend the following changes to Lucene 8.5:
>>
>> These are "Improvements" that I think are better categorized as
>> "Optimizations"
>> * LUCENE-9211: Add compression for Binary doc value fields. (Mark Harwood)
>> * LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
>> * LUCENE-9228: Sort dvUpdates in the term order before applying if they
>> all update a
>>   single field to the same value. This optimization can reduce the flush
>> time by around
>>   20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand,
>> Simon Willnauer)
>> * LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant,
>> Robert Muir)
>> * LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)
>>
>> These "Improvements" I think are better categorized as "Other":
>> * LUCENE-9109: Backport some changes from master (except StackWalker) to
>> improve
>>   TestSecurityManager (Uwe Schindler)
>> * LUCENE-9110: Backport refactored stack analysis in tests to use
>> generalized
>>   LuceneTestCase methods (Uwe Schindler)
>> * LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract
>> class called LatLonGeometry. Queries are
>>   executed with input objects that extend such interface. (Ignacio Vera)
>> * LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract class
>> called XYGeometry. Queries are
>>   executed with input objects that extend such interface. (Ignacio Vera)
>>
>> Maybe this "Other" item should be  "Optimization"? (not sure):
>> * LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward,
>> Mike Drob)
>>
>> Solr:
>>
>> "New Features" that maybe should be "Improvements":
>>  * SOLR-13892: New "top-level" docValues join implementation (Jason
>> Gerlowski, Joel Bernstein)
>>  * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or
>> shapes. (Adrien Grand)
>>
>> "Improvements" that maybe should be "Optimizations":
>> * SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query
>> DSL are cached by default (Mikhail Khludnev)
>>
>> "Improvements" that maybe should be "Other":
>> * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in
>> production (janhoy)
>>
>> Thoughts?
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>

-- 
Adrien

Re: CHANGES.txt and issue categorization

Posted by David Smiley <da...@gmail.com>.
I'll simply move these items around tomorrow this time, unless I hear
feedback to the contrary.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Mar 2, 2020 at 1:07 PM David Smiley <da...@gmail.com>
wrote:

> I'd like us to reflect on how we categorize issues in CHANGES.txt.  We
> have these categories:
> (Lucene) 'API Changes', 'New Features', 'Improvements', 'Optimizations',
> 'Bug Fixes', 'Other'
> (Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes',
> 'Other Changes'
> (I lifted these from dev-tools/scripts/addVersion.py line 215)
>
> In particular, I'm often surprised at how some of us categorize New
> Features or Improvements that should better be categorized as something
> else.  I think the root cause of these problems may be that we don't have
> JIRA categories that directly align.  Furthermore, our dev practices will
> typically result in a CHANGES.txt being added out of band from the
> code-review process, and thus no peer-review on ideal placement.
> Furthermore the message itself is often not code reviewed but should be.
> Perhaps we can simply get in the habit of adding a JIRA comment (or GH code
> review) what we propose the category & issue summary should be.
>
> Here is my attempt at a definition for _some_ of these categories.  I
> don't pretend to think we all agree 100% but it's up for discussion:
> ========
> * New Features:  A user-visible new capability.  Usually opt-in.
>
> * Improvements:  A user-visible improvement to an existing capability that
> somehow expands its ability or that which improves the behavior.  Not a
> refactoring, not an optimization.
>
> * Optimizations: Something is now more efficient.  Usually automatic (not
> opt-in).
>
> * Other:  Anything else: Refactorings, tests, build, docs, etc.  And
> adding log statements.
> ========
>
> I recommend the following changes to Lucene 8.5:
>
> These are "Improvements" that I think are better categorized as
> "Optimizations"
> * LUCENE-9211: Add compression for Binary doc value fields. (Mark Harwood)
> * LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
> * LUCENE-9228: Sort dvUpdates in the term order before applying if they
> all update a
>   single field to the same value. This optimization can reduce the flush
> time by around
>   20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand,
> Simon Willnauer)
> * LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant,
> Robert Muir)
> * LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)
>
> These "Improvements" I think are better categorized as "Other":
> * LUCENE-9109: Backport some changes from master (except StackWalker) to
> improve
>   TestSecurityManager (Uwe Schindler)
> * LUCENE-9110: Backport refactored stack analysis in tests to use
> generalized
>   LuceneTestCase methods (Uwe Schindler)
> * LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract
> class called LatLonGeometry. Queries are
>   executed with input objects that extend such interface. (Ignacio Vera)
> * LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract class
> called XYGeometry. Queries are
>   executed with input objects that extend such interface. (Ignacio Vera)
>
> Maybe this "Other" item should be  "Optimization"? (not sure):
> * LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward,
> Mike Drob)
>
> Solr:
>
> "New Features" that maybe should be "Improvements":
>  * SOLR-13892: New "top-level" docValues join implementation (Jason
> Gerlowski, Joel Bernstein)
>  * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or
> shapes. (Adrien Grand)
>
> "Improvements" that maybe should be "Optimizations":
> * SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query
> DSL are cached by default (Mikhail Khludnev)
>
> "Improvements" that maybe should be "Other":
> * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in
> production (janhoy)
>
> Thoughts?
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>

Re: CHANGES.txt and issue categorization

Posted by Mikhail Khludnev <mk...@apache.org>.
I'm ok with it. Thank you, David. Will you put it somewhere on wiki?

On Mon, Mar 2, 2020 at 10:07 AM David Smiley <da...@gmail.com>
wrote:

> I'd like us to reflect on how we categorize issues in CHANGES.txt.  We
> have these categories:
> (Lucene) 'API Changes', 'New Features', 'Improvements', 'Optimizations',
> 'Bug Fixes', 'Other'
> (Solr) 'New Features', 'Improvements', 'Optimizations', 'Bug Fixes',
> 'Other Changes'
> (I lifted these from dev-tools/scripts/addVersion.py line 215)
>
> In particular, I'm often surprised at how some of us categorize New
> Features or Improvements that should better be categorized as something
> else.  I think the root cause of these problems may be that we don't have
> JIRA categories that directly align.  Furthermore, our dev practices will
> typically result in a CHANGES.txt being added out of band from the
> code-review process, and thus no peer-review on ideal placement.
> Furthermore the message itself is often not code reviewed but should be.
> Perhaps we can simply get in the habit of adding a JIRA comment (or GH code
> review) what we propose the category & issue summary should be.
>
> Here is my attempt at a definition for _some_ of these categories.  I
> don't pretend to think we all agree 100% but it's up for discussion:
> ========
> * New Features:  A user-visible new capability.  Usually opt-in.
>
> * Improvements:  A user-visible improvement to an existing capability that
> somehow expands its ability or that which improves the behavior.  Not a
> refactoring, not an optimization.
>
> * Optimizations: Something is now more efficient.  Usually automatic (not
> opt-in).
>
> * Other:  Anything else: Refactorings, tests, build, docs, etc.  And
> adding log statements.
> ========
>
> I recommend the following changes to Lucene 8.5:
>
> These are "Improvements" that I think are better categorized as
> "Optimizations"
> * LUCENE-9211: Add compression for Binary doc value fields. (Mark Harwood)
> * LUCENE-4702: Better compression of terms dictionaries. (Adrien Grand)
> * LUCENE-9228: Sort dvUpdates in the term order before applying if they
> all update a
>   single field to the same value. This optimization can reduce the flush
> time by around
>   20% for the docValues update user cases. (Nhat Nguyen, Adrien Grand,
> Simon Willnauer)
> * LUCENE-9245: Reduce AutomatonTermsEnum memory usage. (Bruno Roustant,
> Robert Muir)
> * LUCENE-9237: Faster UniformSplit intersect TermsEnum. (Bruno Roustant)
>
> These "Improvements" I think are better categorized as "Other":
> * LUCENE-9109: Backport some changes from master (except StackWalker) to
> improve
>   TestSecurityManager (Uwe Schindler)
> * LUCENE-9110: Backport refactored stack analysis in tests to use
> generalized
>   LuceneTestCase methods (Uwe Schindler)
> * LUCENE-9141: Simplify LatLonShapeXQuery API by adding a new abstract
> class called LatLonGeometry. Queries are
>   executed with input objects that extend such interface. (Ignacio Vera)
> * LUCENE-9194: Simplify XYShapeXQuery API by adding a new abstract class
> called XYGeometry. Queries are
>   executed with input objects that extend such interface. (Ignacio Vera)
>
> Maybe this "Other" item should be  "Optimization"? (not sure):
> * LUCENE-9068: FuzzyQuery builds its Automaton up-front (Alan Woodward,
> Mike Drob)
>
> Solr:
>
> "New Features" that maybe should be "Improvements":
>  * SOLR-13892: New "top-level" docValues join implementation (Jason
> Gerlowski, Joel Bernstein)
>  * SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or
> shapes. (Adrien Grand)
>
> "Improvements" that maybe should be "Optimizations":
> * SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query
> DSL are cached by default (Mikhail Khludnev)
>
> "Improvements" that maybe should be "Other":
> * SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in
> production (janhoy)
>
> Thoughts?
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>


-- 
Sincerely yours
Mikhail Khludnev