You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by Jan Høydahl <ja...@cominvent.com> on 2022/03/01 09:46:27 UTC

Re: New branch and feature freeze for Solr 9.0.0

Hi, and welcome to March!

Our initial goal of a RC1 within February slipped, but we are still in a good position.
I'll try to summarize the current code blockers:


SOLR-16061  Decouple CloudSolrClient from ZkStateReader

This is new, a spin-off from SOLR-15342 to prepare for solrj modularization. There is already a draft PR. Hope there will be progress on this so we don't have to delay solrj modularization until 10.0

SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x

This has not seen any movement despite repeated reminders, so unless there is progress within a few days I'll remove it as blocker and add a note to the release notes that users relying on running test framework locally should wait for a later release.

SOLR-14401  "distrib" request handler metrics should only be tracked on...

There is a PR, not sure how close to merge it is though.

Also, David found a new blocker bug yesterday - the "jetty" metrics group is missing in 9.x. There will likely be another blocker due to this.


Appreciate an update in this thread on the ETA for each of these.

Jan

> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
> 
> I created a new blocker
> 
> SOLR-16040  Fix split packages in hdfs module
> 
> Not sure if it needs to be a blocker though, but we should try to avoid split packages as far as we can, and this cannot be done in 9.x.
> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
> Jenkins is now mostly green after some turmoil!
> 
> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can lend a hand there?
> 
> We also discussed in SOLR-15342 <https://issues.apache.org/jira/browse/SOLR-15342> whether refactoring CloudSolrClient to untangle ZkStateProvider should be done now, and also rename solr-solrj as solr-solrj-all so that we can continue with the solrj modularization in 9.x without back-compat breaks. 
> It seems worthy of a blocker to me, but we need someone willing to do the work in the next few days. Anyone?
> 
> I also created SOLR-16041 <https://issues.apache.org/jira/browse/SOLR-16041> (not blocker) to try to setup nightly smoketestRelease Jenkins jobs, I may try to give it a go.
> 
> Assuming progress on the above, I'm still hopeful for an RC1 in the timeframe of next week.
> 
> Jan
> 
>> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>> 
>> These are the three main code-blockers for doing 9.0.0 RC1:
>> 
>> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>> 
>> The other blockers are mostly about the release process itself, including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>> When these are resolved, I'll prepare RC1. That means we're really close now!!
>> 
>> Anshum is preparing a release notes draft, and we also need to complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing the guide.
>> 
>> Jan
>> 
>>> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>>> 
>>> Congrats on HDFS as a package! Huge win! Also some other blockers have been closed recently.
>>> 
>>> Status on the 9.0 release, one week into February.
>>> 
>>> - I have done a dry-run of an RC and the smoketester. Think the release scripts are ready!
>>> - 11 open blockers:
>>> 
>>> (P) SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude     
>>> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett       
>>> (P) SOLR-15557  Figure out how to handle ref guide page renames/redirects          janhoy      ctargett       
>>> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>>> (A) SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley        
>>> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus            
>>> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley        
>>> (S) SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman        
>>> (U) SOLR-15941  Wire Antora refGuide into the release scripts                                  janhoy
>>> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy         
>>> (D) SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>>> 
>>> Based on a quick look, I assigned a readiness-code for each:
>>> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed? (S): Stalled. (U): Unassigned
>>> 
>>> Would be great to have a status and ETA for the (S)talled issues. Perhaps we can push some to 9.0.1 or 9.1?
>>> 
>>> Wrt publishing the ref-guide and docker image, those could be done semi-manual for 9.0.0 if need be.
>>> 
>>> Jan
>>> 
>>>> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <compuwizard123@gmail.com <ma...@gmail.com>>:
>>>> 
>>>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>>>> 
>>>> This was just merged. Big shout out to Istvan Farkas for doing 99% of the heavy lifting here!
>>>>  
>>>> Kevin Risden
>>>> 
>>>> 
>>>> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
>>>> Hi,
>>>> 
>>>> Nine days ago we had 24 blockers. Several blockers have since been added and several resolved, and today we are at 19.
>>>> 
>>>> I notified in the previous email that I'd remove inactive blockers. Here is the list of JIRAs for which I intend to remove the blocker flag on February 1st:
>>>> 
>>>> SOLR-14097  coreRootDirectory should be solr_home/cores
>>>> SOLR-15064  Atomic/partial updates to nested docs should not assume _route_ param is the root ID
>>>> SOLR-15242  Consolidate README.md with solr/README.md
>>>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly degraded in Java 11 v 8
>>>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as "optional" in SolrJ
>>>> SOLR-15835  Collection creation failing with https
>>>> 
>>>> Shout out if you are working on one of these and expect it to be finished soon-ish.
>>>> 
>>>> That leaves 13 blockers:
>>>> 
>>>> Issue key   Summary                                                            Assignee    Reporter       
>>>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>>>> SOLR-15956  Add documentation for creating a docker image from the binary dis  houston     houston        
>>>> SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek     
>>>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett       
>>>> SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley        
>>>> SOLR-15926  Fix version specification in the Solr Ref Guide                                houston        
>>>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus            
>>>> SOLR-12901  Make UnifiedHighlighter the default                                dsmiley     dsmiley        
>>>> SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley        
>>>> SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude     
>>>> SOLR-15557  Figure out how to handle ref guide page renames/redirects                      ctargett       
>>>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy         
>>>> SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman        
>>>> 
>>>> It seems like most of these have had some recent activity.
>>>> Please everyone, have a look if you can lend a hand with any of these, so we can get the list to zero early in February and do the first RC.
>>>> I'd appreciate some help on SOLR-15898, consolidating and structuring the "Major changes" chapter of the reference guide.
>>>> 
>>>> Jan
>>>> 
>>>> 
>>>>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> The list of release blockers can be seen with this JIRA filter: https://issues.apache.org/jira/issues/?filter=12351219 <https://issues.apache.org/jira/issues/?filter=12351219>
>>>>> 
>>>>> After the committer's meeting today we decided to add these to the blockers list:
>>>>> - SOLR-15556	Ref Guide Redesign Phase 3: Replace Jekyll
>>>>> - SOLR-15917	Rename 'contrib' as 'module'
>>>>> - SOLR-15880	Introduce Support to K Nearest Neighbors Search
>>>>> - SOLR-14660	Migrating HDFS into a package
>>>>> - SOLR-12901	Make UnifiedHighlighter the default
>>>>> - SOLR-15914 Make it super simple to add a contrib module to shared classpath
>>>>> - And probably some minor ones too
>>>>> 
>>>>> Each of these are in-flight and are expected to be ready really soon™.
>>>>> 
>>>>> There are currently 24 blockers, but some of those are Unsassigned and/or have not been given any attention for some time.
>>>>> On Feb 1st I'll take the freedom to remove blocker flag for those that have not moved anywhere since. 
>>>>> If you want to own one of them, please assign yourself and communicate progress and an ETA.
>>>>> 
>>>>> Jan
>>>>> 
>>>> 
>>> 
>> 
>

Re: New branch and feature freeze for Solr 9.0.0

Posted by Jan Høydahl <ja...@cominvent.com>.
The blockers are once again resolved.

Building RC1 has failed 4 times in a row due to flaky tests.
Also, the last 4 runs of "Solr-Check-9.0" in Jenkins have failed: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.0/


These are the tests failing in recent Jenkins runs:
- org.apache.solr.search.TestCancellableCollector.classMethod

- org.apache.solr.cloud.OverseerTest.testShardLeaderChange
- org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod
- org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots

- org.apache.solr.client.solrj.impl.CloudSolrClientTest.testPerReplicaStateCollection

- org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod
- org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnLeaderRestart
- org.apache.solr.client.solrj.impl.CloudSolrClientCacheTest.classMethod


I BadApple'd TestLeaderElectionZkExpiry (SOLR-16122), and will try build the RC again.
Appreciate help with fixing some of these other tests.

Jan

> 26. mar. 2022 kl. 13:41 skrev Jan Høydahl <ja...@cominvent.com>:
> 
> Hi,
> 
> I filed these new blockers for build bugs, with attached PRs. Please do a quick review:
> 
> https://issues.apache.org/jira/browse/SOLR-16117 <https://issues.apache.org/jira/browse/SOLR-16117>  Do not publish an empty solr-server.jar to maven
> https://issues.apache.org/jira/browse/SOLR-16118 <https://issues.apache.org/jira/browse/SOLR-16118>  Solr source jars should contain NOTICE, LICENSE files
> 
> Also some more smokeTester changes were necessary:
> 
> https://github.com/apache/solr/pull/763 <https://github.com/apache/solr/pull/763> (SOLR-15852: Smoketester fixes for maven artifact comparison)
> 
> 
> 
> I just built a release with these three PRs applied, and then the smoke tester passes for me. So once I get at least one thumbs up for each of them, I'll merge and build RC1.
> 
> Jan
> 
> 
>> 26. mar. 2022 kl. 04:25 skrev Houston Putman <houston@apache.org <ma...@apache.org>>:
>> 
>> Seriously monumental effort everyone!
>> 
>> Thanks for keeping this on track Jan!
>> 
>> - Houston
>> 
>> On Fri, Mar 25, 2022 at 10:36 PM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>> Woohoo!
>> 
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>> 
>> On Fri, Mar 25, 2022 at 4:16 PM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
>> Hi,
>> 
>> All code blockers are now cleared: https://issues.apache.org/jira/issues/?filter=12351219 <https://issues.apache.org/jira/issues/?filter=12351219>
>> Some work remains for RefGuide and docker release procedures, we can continue on those in parallell with the RCs.
>> 
>> I'll proceed with RC1.
>> 
>> Jan
>> 
>>> 4. mar. 2022 kl. 21:52 skrev Houston Putman <houston@apache.org <ma...@apache.org>>:
>>> 
>>> I think we have another blocker for 9.0. Basically there is a bug in the updated version of commons-io that causes index files to be set to read-only in the filesystem occasionally. The solution is to upgrade commons-io, or find a workaround in Solr, but we can have that discussion on the JIRA.
>>> 
>>> More info here: https://issues.apache.org/jira/browse/SOLR-16074 <https://issues.apache.org/jira/browse/SOLR-16074>
>>> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>>> I suppose the biggest spots for peer review are:
>>> * use of brackets [ ] in the metric name where the request handler is.  Thus "/select[shard]"
>>> * There is a fundamental difference in how the metrics are tracked on a handler.  Previously, there were metrics for all of /select (no matter how it was invoked), and a few for .distrib. & .shard. depending on how it was invoked.  Now, the request is classified to be a shard request, or not a shard request, after which separate metrics (same type/semantics) are manipulated based on that classification, kind of as if there are two distinct request handlers even though just one is registered.  I think the PR makes this clear.  While I like it, the main trade-off is that a user would be forced to aggregate metrics if they wanted a single metric for the handler.  I think the isShard=true request changes the personality/mode of the handler so much that I prefer to present it as its own identity from a metrics standpoint.
>>> 
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>>> 
>>> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <thelabdude@gmail.com <ma...@gmail.com>> wrote:
>>> Hi David,
>>> 
>>> I read your note about SOLR-14401 but not clear what you need from me?
>>> Seems like you're renaming existing metrics and removing "distrib"
>>> from handlers that don't support a distrib mode, seems right to me.
>>> 
>>> I actually haven't done much work on the metrics backend. For Grafana,
>>> it's a JSON file so search / replace the metrics you're changing. The
>>> Solr operator makes it really easy to set up Solr + ZK + Grafana +
>>> Prometheus + Exporter to test out your changes. It'll be pretty
>>> obvious if the dashboard is broken.
>>> 
>>> Tim
>>> 
>>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>>> >
>>> >
>>> >
>>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
>>> >>
>>> >> Hi, and welcome to March!
>>> >>
>>> >> Our initial goal of a RC1 within February slipped, but we are still in a good position.
>>> >> I'll try to summarize the current code blockers:
>>> >>
>>> >>
>>> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
>>> >>
>>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj modularization. There is already a draft PR. Hope there will be progress on this so we don't have to delay solrj modularization until 10.0
>>> >
>>> >
>>> > I'm working with Haythem on this (a colleague).  I think it's close; it's "just" a refactoring.  The main constraint on this is Haythem's time.
>>> >
>>> >>
>>> >>
>>> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x
>>> >>
>>> >> This has not seen any movement despite repeated reminders, so unless there is progress within a few days I'll remove it as blocker and add a note to the release notes that users relying on running test framework locally should wait for a later release.
>>> >
>>> >
>>> > I'm interested in looking but not until I get through the other two.
>>> >
>>> >>
>>> >> SOLR-14401  "distrib" request handler metrics should only be tracked on...
>>> >>
>>> >> There is a PR, not sure how close to merge it is though.
>>> >
>>> >
>>> > I think the core of the change is fine but there are downstream changes needed.  First level is the prometheus exporter configuration to not look for ".distrib." vs ".local."; it's different now.  Next level is the Grafana dashboard.  I don't normally play with JQ, Prometheus or Grafana so it's taking me some time this week.  I'd appreciate any feedback on the choices here; so far only Houston has weigh'ed in.  I think Tim Potter, if you're reading this, would be useful given you did major work here.
>>> >
>>> >>
>>> >>
>>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics group is missing in 9.x. There will likely be another blocker due to this.
>>> >>
>>> >>
>>> >> Appreciate an update in this thread on the ETA for each of these.
>>> >>
>>> >> Jan
>>> >>
>>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>>> >>
>>> >> I created a new blocker
>>> >>
>>> >> SOLR-16040  Fix split packages in hdfs module
>>> >>
>>> >> Not sure if it needs to be a blocker though, but we should try to avoid split packages as far as we can, and this cannot be done in 9.x.
>>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
>>> >> Jenkins is now mostly green after some turmoil!
>>> >>
>>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can lend a hand there?
>>> >>
>>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to untangle ZkStateProvider should be done now, and also rename solr-solrj as solr-solrj-all so that we can continue with the solrj modularization in 9.x without back-compat breaks.
>>> >> It seems worthy of a blocker to me, but we need someone willing to do the work in the next few days. Anyone?
>>> >>
>>> >> I also created SOLR-16041 (not blocker) to try to setup nightly smoketestRelease Jenkins jobs, I may try to give it a go.
>>> >>
>>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the timeframe of next week.
>>> >>
>>> >> Jan
>>> >>
>>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>>> >>
>>> >> These are the three main code-blockers for doing 9.0.0 RC1:
>>> >>
>>> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>>> >>
>>> >> The other blockers are mostly about the release process itself, including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>>> >> When these are resolved, I'll prepare RC1. That means we're really close now!!
>>> >>
>>> >> Anshum is preparing a release notes draft, and we also need to complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing the guide.
>>> >>
>>> >> Jan
>>> >>
>>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>>> >>
>>> >> Congrats on HDFS as a package! Huge win! Also some other blockers have been closed recently.
>>> >>
>>> >> Status on the 9.0 release, one week into February.
>>> >>
>>> >> - I have done a dry-run of an RC and the smoketester. Think the release scripts are ready!
>>> >> - 11 open blockers:
>>> >>
>>> >> (P) SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
>>> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
>>> >> (P) SOLR-15557  Figure out how to handle ref guide page renames/redirects          janhoy      ctargett
>>> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>>> >> (A) SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
>>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>>> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
>>> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts                                  janhoy
>>> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
>>> >> (D) SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>>> >>
>>> >> Based on a quick look, I assigned a readiness-code for each:
>>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed? (S): Stalled. (U): Unassigned
>>> >>
>>> >> Would be great to have a status and ETA for the (S)talled issues. Perhaps we can push some to 9.0.1 or 9.1?
>>> >>
>>> >> Wrt publishing the ref-guide and docker image, those could be done semi-manual for 9.0.0 if need be.
>>> >>
>>> >> Jan
>>> >>
>>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <compuwizard123@gmail.com <ma...@gmail.com>>:
>>> >>
>>> >>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>>> >>
>>> >>
>>> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of the heavy lifting here!
>>> >>
>>> >> Kevin Risden
>>> >>
>>> >>
>>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> Nine days ago we had 24 blockers. Several blockers have since been added and several resolved, and today we are at 19.
>>> >>>
>>> >>> I notified in the previous email that I'd remove inactive blockers. Here is the list of JIRAs for which I intend to remove the blocker flag on February 1st:
>>> >>>
>>> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
>>> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume _route_ param is the root ID
>>> >>> SOLR-15242  Consolidate README.md with solr/README.md
>>> >>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly degraded in Java 11 v 8
>>> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as "optional" in SolrJ
>>> >>> SOLR-15835  Collection creation failing with https
>>> >>>
>>> >>> Shout out if you are working on one of these and expect it to be finished soon-ish.
>>> >>>
>>> >>> That leaves 13 blockers:
>>> >>>
>>> >>> Issue key   Summary                                                            Assignee    Reporter
>>> >>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>>> >>> SOLR-15956  Add documentation for creating a docker image from the binary dis  houston     houston
>>> >>> SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>>> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
>>> >>> SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
>>> >>> SOLR-15926  Fix version specification in the Solr Ref Guide                                houston
>>> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>>> >>> SOLR-12901  Make UnifiedHighlighter the default                                dsmiley     dsmiley
>>> >>> SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>>> >>> SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
>>> >>> SOLR-15557  Figure out how to handle ref guide page renames/redirects                      ctargett
>>> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
>>> >>> SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
>>> >>>
>>> >>> It seems like most of these have had some recent activity.
>>> >>> Please everyone, have a look if you can lend a hand with any of these, so we can get the list to zero early in February and do the first RC.
>>> >>> I'd appreciate some help on SOLR-15898, consolidating and structuring the "Major changes" chapter of the reference guide.
>>> >>>
>>> >>> Jan
>>> >>>
>>> >>>
>>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> The list of release blockers can be seen with this JIRA filter: https://issues.apache.org/jira/issues/?filter=12351219 <https://issues.apache.org/jira/issues/?filter=12351219>
>>> >>>
>>> >>> After the committer's meeting today we decided to add these to the blockers list:
>>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>>> >>> - SOLR-15917 Rename 'contrib' as 'module'
>>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>>> >>> - SOLR-14660 Migrating HDFS into a package
>>> >>> - SOLR-12901 Make UnifiedHighlighter the default
>>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared classpath
>>> >>> - And probably some minor ones too
>>> >>>
>>> >>> Each of these are in-flight and are expected to be ready really soon™.
>>> >>>
>>> >>> There are currently 24 blockers, but some of those are Unsassigned and/or have not been given any attention for some time.
>>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those that have not moved anywhere since.
>>> >>> If you want to own one of them, please assign yourself and communicate progress and an ETA.
>>> >>>
>>> >>> Jan
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >>
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <ma...@solr.apache.org>
>>> For additional commands, e-mail: dev-help@solr.apache.org <ma...@solr.apache.org>
>>> 
>> 
>

Re: New branch and feature freeze for Solr 9.0.0

Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,

I filed these new blockers for build bugs, with attached PRs. Please do a quick review:

https://issues.apache.org/jira/browse/SOLR-16117  Do not publish an empty solr-server.jar to maven
https://issues.apache.org/jira/browse/SOLR-16118  Solr source jars should contain NOTICE, LICENSE files

Also some more smokeTester changes were necessary:

https://github.com/apache/solr/pull/763 (SOLR-15852: Smoketester fixes for maven artifact comparison)



I just built a release with these three PRs applied, and then the smoke tester passes for me. So once I get at least one thumbs up for each of them, I'll merge and build RC1.

Jan


> 26. mar. 2022 kl. 04:25 skrev Houston Putman <ho...@apache.org>:
> 
> Seriously monumental effort everyone!
> 
> Thanks for keeping this on track Jan!
> 
> - Houston
> 
> On Fri, Mar 25, 2022 at 10:36 PM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
> Woohoo!
> 
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
> 
> On Fri, Mar 25, 2022 at 4:16 PM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
> Hi,
> 
> All code blockers are now cleared: https://issues.apache.org/jira/issues/?filter=12351219 <https://issues.apache.org/jira/issues/?filter=12351219>
> Some work remains for RefGuide and docker release procedures, we can continue on those in parallell with the RCs.
> 
> I'll proceed with RC1.
> 
> Jan
> 
>> 4. mar. 2022 kl. 21:52 skrev Houston Putman <houston@apache.org <ma...@apache.org>>:
>> 
>> I think we have another blocker for 9.0. Basically there is a bug in the updated version of commons-io that causes index files to be set to read-only in the filesystem occasionally. The solution is to upgrade commons-io, or find a workaround in Solr, but we can have that discussion on the JIRA.
>> 
>> More info here: https://issues.apache.org/jira/browse/SOLR-16074 <https://issues.apache.org/jira/browse/SOLR-16074>
>> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>> I suppose the biggest spots for peer review are:
>> * use of brackets [ ] in the metric name where the request handler is.  Thus "/select[shard]"
>> * There is a fundamental difference in how the metrics are tracked on a handler.  Previously, there were metrics for all of /select (no matter how it was invoked), and a few for .distrib. & .shard. depending on how it was invoked.  Now, the request is classified to be a shard request, or not a shard request, after which separate metrics (same type/semantics) are manipulated based on that classification, kind of as if there are two distinct request handlers even though just one is registered.  I think the PR makes this clear.  While I like it, the main trade-off is that a user would be forced to aggregate metrics if they wanted a single metric for the handler.  I think the isShard=true request changes the personality/mode of the handler so much that I prefer to present it as its own identity from a metrics standpoint.
>> 
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>> 
>> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <thelabdude@gmail.com <ma...@gmail.com>> wrote:
>> Hi David,
>> 
>> I read your note about SOLR-14401 but not clear what you need from me?
>> Seems like you're renaming existing metrics and removing "distrib"
>> from handlers that don't support a distrib mode, seems right to me.
>> 
>> I actually haven't done much work on the metrics backend. For Grafana,
>> it's a JSON file so search / replace the metrics you're changing. The
>> Solr operator makes it really easy to set up Solr + ZK + Grafana +
>> Prometheus + Exporter to test out your changes. It'll be pretty
>> obvious if the dashboard is broken.
>> 
>> Tim
>> 
>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>> >
>> >
>> >
>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
>> >>
>> >> Hi, and welcome to March!
>> >>
>> >> Our initial goal of a RC1 within February slipped, but we are still in a good position.
>> >> I'll try to summarize the current code blockers:
>> >>
>> >>
>> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
>> >>
>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj modularization. There is already a draft PR. Hope there will be progress on this so we don't have to delay solrj modularization until 10.0
>> >
>> >
>> > I'm working with Haythem on this (a colleague).  I think it's close; it's "just" a refactoring.  The main constraint on this is Haythem's time.
>> >
>> >>
>> >>
>> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x
>> >>
>> >> This has not seen any movement despite repeated reminders, so unless there is progress within a few days I'll remove it as blocker and add a note to the release notes that users relying on running test framework locally should wait for a later release.
>> >
>> >
>> > I'm interested in looking but not until I get through the other two.
>> >
>> >>
>> >> SOLR-14401  "distrib" request handler metrics should only be tracked on...
>> >>
>> >> There is a PR, not sure how close to merge it is though.
>> >
>> >
>> > I think the core of the change is fine but there are downstream changes needed.  First level is the prometheus exporter configuration to not look for ".distrib." vs ".local."; it's different now.  Next level is the Grafana dashboard.  I don't normally play with JQ, Prometheus or Grafana so it's taking me some time this week.  I'd appreciate any feedback on the choices here; so far only Houston has weigh'ed in.  I think Tim Potter, if you're reading this, would be useful given you did major work here.
>> >
>> >>
>> >>
>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics group is missing in 9.x. There will likely be another blocker due to this.
>> >>
>> >>
>> >> Appreciate an update in this thread on the ETA for each of these.
>> >>
>> >> Jan
>> >>
>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>> >>
>> >> I created a new blocker
>> >>
>> >> SOLR-16040  Fix split packages in hdfs module
>> >>
>> >> Not sure if it needs to be a blocker though, but we should try to avoid split packages as far as we can, and this cannot be done in 9.x.
>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
>> >> Jenkins is now mostly green after some turmoil!
>> >>
>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can lend a hand there?
>> >>
>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to untangle ZkStateProvider should be done now, and also rename solr-solrj as solr-solrj-all so that we can continue with the solrj modularization in 9.x without back-compat breaks.
>> >> It seems worthy of a blocker to me, but we need someone willing to do the work in the next few days. Anyone?
>> >>
>> >> I also created SOLR-16041 (not blocker) to try to setup nightly smoketestRelease Jenkins jobs, I may try to give it a go.
>> >>
>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the timeframe of next week.
>> >>
>> >> Jan
>> >>
>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>> >>
>> >> These are the three main code-blockers for doing 9.0.0 RC1:
>> >>
>> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>> >>
>> >> The other blockers are mostly about the release process itself, including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>> >> When these are resolved, I'll prepare RC1. That means we're really close now!!
>> >>
>> >> Anshum is preparing a release notes draft, and we also need to complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing the guide.
>> >>
>> >> Jan
>> >>
>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>> >>
>> >> Congrats on HDFS as a package! Huge win! Also some other blockers have been closed recently.
>> >>
>> >> Status on the 9.0 release, one week into February.
>> >>
>> >> - I have done a dry-run of an RC and the smoketester. Think the release scripts are ready!
>> >> - 11 open blockers:
>> >>
>> >> (P) SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
>> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
>> >> (P) SOLR-15557  Figure out how to handle ref guide page renames/redirects          janhoy      ctargett
>> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>> >> (A) SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
>> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts                                  janhoy
>> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
>> >> (D) SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>> >>
>> >> Based on a quick look, I assigned a readiness-code for each:
>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed? (S): Stalled. (U): Unassigned
>> >>
>> >> Would be great to have a status and ETA for the (S)talled issues. Perhaps we can push some to 9.0.1 or 9.1?
>> >>
>> >> Wrt publishing the ref-guide and docker image, those could be done semi-manual for 9.0.0 if need be.
>> >>
>> >> Jan
>> >>
>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <compuwizard123@gmail.com <ma...@gmail.com>>:
>> >>
>> >>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>> >>
>> >>
>> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of the heavy lifting here!
>> >>
>> >> Kevin Risden
>> >>
>> >>
>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> Nine days ago we had 24 blockers. Several blockers have since been added and several resolved, and today we are at 19.
>> >>>
>> >>> I notified in the previous email that I'd remove inactive blockers. Here is the list of JIRAs for which I intend to remove the blocker flag on February 1st:
>> >>>
>> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
>> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume _route_ param is the root ID
>> >>> SOLR-15242  Consolidate README.md with solr/README.md
>> >>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly degraded in Java 11 v 8
>> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as "optional" in SolrJ
>> >>> SOLR-15835  Collection creation failing with https
>> >>>
>> >>> Shout out if you are working on one of these and expect it to be finished soon-ish.
>> >>>
>> >>> That leaves 13 blockers:
>> >>>
>> >>> Issue key   Summary                                                            Assignee    Reporter
>> >>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>> >>> SOLR-15956  Add documentation for creating a docker image from the binary dis  houston     houston
>> >>> SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
>> >>> SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
>> >>> SOLR-15926  Fix version specification in the Solr Ref Guide                                houston
>> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>> >>> SOLR-12901  Make UnifiedHighlighter the default                                dsmiley     dsmiley
>> >>> SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>> >>> SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
>> >>> SOLR-15557  Figure out how to handle ref guide page renames/redirects                      ctargett
>> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
>> >>> SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
>> >>>
>> >>> It seems like most of these have had some recent activity.
>> >>> Please everyone, have a look if you can lend a hand with any of these, so we can get the list to zero early in February and do the first RC.
>> >>> I'd appreciate some help on SOLR-15898, consolidating and structuring the "Major changes" chapter of the reference guide.
>> >>>
>> >>> Jan
>> >>>
>> >>>
>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
>> >>>
>> >>> Hi,
>> >>>
>> >>> The list of release blockers can be seen with this JIRA filter: https://issues.apache.org/jira/issues/?filter=12351219 <https://issues.apache.org/jira/issues/?filter=12351219>
>> >>>
>> >>> After the committer's meeting today we decided to add these to the blockers list:
>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>> >>> - SOLR-15917 Rename 'contrib' as 'module'
>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>> >>> - SOLR-14660 Migrating HDFS into a package
>> >>> - SOLR-12901 Make UnifiedHighlighter the default
>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared classpath
>> >>> - And probably some minor ones too
>> >>>
>> >>> Each of these are in-flight and are expected to be ready really soon™.
>> >>>
>> >>> There are currently 24 blockers, but some of those are Unsassigned and/or have not been given any attention for some time.
>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those that have not moved anywhere since.
>> >>> If you want to own one of them, please assign yourself and communicate progress and an ETA.
>> >>>
>> >>> Jan
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <ma...@solr.apache.org>
>> For additional commands, e-mail: dev-help@solr.apache.org <ma...@solr.apache.org>
>> 
>

Re: New branch and feature freeze for Solr 9.0.0

Posted by Houston Putman <ho...@apache.org>.
Seriously monumental effort everyone!

Thanks for keeping this on track Jan!

- Houston

On Fri, Mar 25, 2022 at 10:36 PM David Smiley <ds...@apache.org> wrote:

> Woohoo!
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Mar 25, 2022 at 4:16 PM Jan Høydahl <ja...@cominvent.com> wrote:
>
>> Hi,
>>
>> All code blockers are now cleared:
>> https://issues.apache.org/jira/issues/?filter=12351219
>> Some work remains for RefGuide and docker release procedures, we can
>> continue on those in parallell with the RCs.
>>
>> I'll proceed with RC1.
>>
>> Jan
>>
>> 4. mar. 2022 kl. 21:52 skrev Houston Putman <ho...@apache.org>:
>>
>> I think we have another blocker for 9.0. Basically there is a bug in the
>> updated version of commons-io that causes index files to be set to
>> read-only in the filesystem occasionally. The solution is to upgrade
>> commons-io, or find a workaround in Solr, but we can have that discussion
>> on the JIRA.
>>
>> More info here: https://issues.apache.org/jira/browse/SOLR-16074
>>
>> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <ds...@apache.org> wrote:
>>
>>> I suppose the biggest spots for peer review are:
>>> * use of brackets [ ] in the metric name where the request handler is.
>>> Thus "/select[shard]"
>>> * There is a fundamental difference in how the metrics are tracked on a
>>> handler.  Previously, there were metrics for all of /select (no matter how
>>> it was invoked), and a few for .distrib. & .shard. depending on how it was
>>> invoked.  Now, the request is classified to be a shard request, or not a
>>> shard request, after which separate metrics (same type/semantics) are
>>> manipulated based on that classification, kind of as if there are two
>>> distinct request handlers even though just one is registered.  I think
>>> the PR makes this clear.  While I like it, the main trade-off is that a
>>> user would be forced to aggregate metrics if they wanted a single metric
>>> for the handler.  I think the isShard=true request changes the
>>> personality/mode of the handler so much that I prefer to present it as its
>>> own identity from a metrics standpoint.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <th...@gmail.com>
>>> wrote:
>>>
>>>> Hi David,
>>>>
>>>> I read your note about SOLR-14401 but not clear what you need from me?
>>>> Seems like you're renaming existing metrics and removing "distrib"
>>>> from handlers that don't support a distrib mode, seems right to me.
>>>>
>>>> I actually haven't done much work on the metrics backend. For Grafana,
>>>> it's a JSON file so search / replace the metrics you're changing. The
>>>> Solr operator makes it really easy to set up Solr + ZK + Grafana +
>>>> Prometheus + Exporter to test out your changes. It'll be pretty
>>>> obvious if the dashboard is broken.
>>>>
>>>> Tim
>>>>
>>>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <ds...@apache.org> wrote:
>>>> >
>>>> >
>>>> >
>>>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <ja...@cominvent.com>
>>>> wrote:
>>>> >>
>>>> >> Hi, and welcome to March!
>>>> >>
>>>> >> Our initial goal of a RC1 within February slipped, but we are still
>>>> in a good position.
>>>> >> I'll try to summarize the current code blockers:
>>>> >>
>>>> >>
>>>> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
>>>> >>
>>>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj
>>>> modularization. There is already a draft PR. Hope there will be progress on
>>>> this so we don't have to delay solrj modularization until 10.0
>>>> >
>>>> >
>>>> > I'm working with Haythem on this (a colleague).  I think it's close;
>>>> it's "just" a refactoring.  The main constraint on this is Haythem's time.
>>>> >
>>>> >>
>>>> >>
>>>> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>>>> master/9.x
>>>> >>
>>>> >> This has not seen any movement despite repeated reminders, so unless
>>>> there is progress within a few days I'll remove it as blocker and add a
>>>> note to the release notes that users relying on running test framework
>>>> locally should wait for a later release.
>>>> >
>>>> >
>>>> > I'm interested in looking but not until I get through the other two.
>>>> >
>>>> >>
>>>> >> SOLR-14401  "distrib" request handler metrics should only be tracked
>>>> on...
>>>> >>
>>>> >> There is a PR, not sure how close to merge it is though.
>>>> >
>>>> >
>>>> > I think the core of the change is fine but there are downstream
>>>> changes needed.  First level is the prometheus exporter configuration to
>>>> not look for ".distrib." vs ".local."; it's different now.  Next level is
>>>> the Grafana dashboard.  I don't normally play with JQ, Prometheus or
>>>> Grafana so it's taking me some time this week.  I'd appreciate any feedback
>>>> on the choices here; so far only Houston has weigh'ed in.  I think Tim
>>>> Potter, if you're reading this, would be useful given you did major work
>>>> here.
>>>> >
>>>> >>
>>>> >>
>>>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics
>>>> group is missing in 9.x. There will likely be another blocker due to this.
>>>> >>
>>>> >>
>>>> >> Appreciate an update in this thread on the ETA for each of these.
>>>> >>
>>>> >> Jan
>>>> >>
>>>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
>>>> >>
>>>> >> I created a new blocker
>>>> >>
>>>> >> SOLR-16040  Fix split packages in hdfs module
>>>> >>
>>>> >> Not sure if it needs to be a blocker though, but we should try to
>>>> avoid split packages as far as we can, and this cannot be done in 9.x.
>>>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review
>>>> phase.
>>>> >> Jenkins is now mostly green after some turmoil!
>>>> >>
>>>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can
>>>> lend a hand there?
>>>> >>
>>>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient
>>>> to untangle ZkStateProvider should be done now, and also rename solr-solrj
>>>> as solr-solrj-all so that we can continue with the solrj modularization in
>>>> 9.x without back-compat breaks.
>>>> >> It seems worthy of a blocker to me, but we need someone willing to
>>>> do the work in the next few days. Anyone?
>>>> >>
>>>> >> I also created SOLR-16041 (not blocker) to try to setup nightly
>>>> smoketestRelease Jenkins jobs, I may try to give it a go.
>>>> >>
>>>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the
>>>> timeframe of next week.
>>>> >>
>>>> >> Jan
>>>> >>
>>>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <ja...@cominvent.com>:
>>>> >>
>>>> >> These are the three main code-blockers for doing 9.0.0 RC1:
>>>> >>
>>>> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not
>>>> assume _route_    dsmiley     dsmiley
>>>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage
>>>> for master/9.x   gus         gus
>>>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
>>>> tracked on pe  dsmiley     dsmiley
>>>> >>
>>>> >> The other blockers are mostly about the release process itself,
>>>> including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>>>> >> When these are resolved, I'll prepare RC1. That means we're really
>>>> close now!!
>>>> >>
>>>> >> Anshum is preparing a release notes draft, and we also need to
>>>> complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before
>>>> publishing the guide.
>>>> >>
>>>> >> Jan
>>>> >>
>>>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <ja...@cominvent.com>:
>>>> >>
>>>> >> Congrats on HDFS as a package! Huge win! Also some other blockers
>>>> have been closed recently.
>>>> >>
>>>> >> Status on the 9.0 release, one week into February.
>>>> >>
>>>> >> - I have done a dry-run of an RC and the smoketester. Think the
>>>> release scripts are ready!
>>>> >> - 11 open blockers:
>>>> >>
>>>> >> (P) SOLR-15587  Replicas end up with base_url as http on client side
>>>> even if clus  thelabdude  thelabdude
>>>> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>>>                ctargett    ctargett
>>>> >> (P) SOLR-15557  Figure out how to handle ref guide page
>>>> renames/redirects          janhoy      ctargett
>>>> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not
>>>> assume _route_    dsmiley     dsmiley
>>>> >> (A) SOLR-15949  Use Java 17 in docker
>>>>               dsmiley     dsmiley
>>>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage
>>>> for master/9.x   gus         gus
>>>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
>>>> tracked on pe  dsmiley     dsmiley
>>>> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official""
>>>> Dockerfiles   houston     hossman
>>>> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts
>>>>                           janhoy
>>>> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie
>>>> for 9.0.0                 janhoy
>>>> >> (D) SOLR-13138  Remove deprecated code prior to 9.0
>>>>                           romseygeek
>>>> >>
>>>> >> Based on a quick look, I assigned a readiness-code for each:
>>>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be
>>>> closed? (S): Stalled. (U): Unassigned
>>>> >>
>>>> >> Would be great to have a status and ETA for the (S)talled issues.
>>>> Perhaps we can push some to 9.0.1 or 9.1?
>>>> >>
>>>> >> Wrt publishing the ref-guide and docker image, those could be done
>>>> semi-manual for 9.0.0 if need be.
>>>> >>
>>>> >> Jan
>>>> >>
>>>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <compuwizard123@gmail.com
>>>> >:
>>>> >>
>>>> >>> SOLR-14660  Migrating HDFS into a module
>>>>            krisden     ichattopadhyaya
>>>> >>
>>>> >>
>>>> >> This was just merged. Big shout out to Istvan Farkas for doing 99%
>>>> of the heavy lifting here!
>>>> >>
>>>> >> Kevin Risden
>>>> >>
>>>> >>
>>>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <ja...@cominvent.com>
>>>> wrote:
>>>> >>>
>>>> >>> Hi,
>>>> >>>
>>>> >>> Nine days ago we had 24 blockers. Several blockers have since been
>>>> added and several resolved, and today we are at 19.
>>>> >>>
>>>> >>> I notified in the previous email that I'd remove inactive blockers.
>>>> Here is the list of JIRAs for which I intend to remove the blocker flag on
>>>> February 1st:
>>>> >>>
>>>> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
>>>> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume
>>>> _route_ param is the root ID
>>>> >>> SOLR-15242  Consolidate README.md with solr/README.md
>>>> >>> SOLR-15096  [REGRESSION] Collection Delete Performance
>>>> significantly degraded in Java 11 v 8
>>>> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as
>>>> "optional" in SolrJ
>>>> >>> SOLR-15835  Collection creation failing with https
>>>> >>>
>>>> >>> Shout out if you are working on one of these and expect it to be
>>>> finished soon-ish.
>>>> >>>
>>>> >>> That leaves 13 blockers:
>>>> >>>
>>>> >>> Issue key   Summary
>>>>             Assignee    Reporter
>>>> >>> SOLR-14660  Migrating HDFS into a module
>>>>            krisden     ichattopadhyaya
>>>> >>> SOLR-15956  Add documentation for creating a docker image from the
>>>> binary dis  houston     houston
>>>> >>> SOLR-13138  Remove deprecated code prior to 9.0
>>>>                         romseygeek
>>>> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>>>            ctargett    ctargett
>>>> >>> SOLR-15949  Use Java 17 in docker
>>>>             dsmiley     dsmiley
>>>> >>> SOLR-15926  Fix version specification in the Solr Ref Guide
>>>>                         houston
>>>> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>>>> master/9.x   gus         gus
>>>> >>> SOLR-12901  Make UnifiedHighlighter the default
>>>>             dsmiley     dsmiley
>>>> >>> SOLR-14401  """distrib"" request handler metrics should only be
>>>> tracked on pe  dsmiley     dsmiley
>>>> >>> SOLR-15587  Replicas end up with base_url as http on client side
>>>> even if clus  thelabdude  thelabdude
>>>> >>> SOLR-15557  Figure out how to handle ref guide page
>>>> renames/redirects                      ctargett
>>>> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie
>>>> for 9.0.0                 janhoy
>>>> >>> SOLR-15321  "Flesh out process for managing/storing ""official""
>>>> Dockerfiles   houston     hossman
>>>> >>>
>>>> >>> It seems like most of these have had some recent activity.
>>>> >>> Please everyone, have a look if you can lend a hand with any of
>>>> these, so we can get the list to zero early in February and do the first RC.
>>>> >>> I'd appreciate some help on SOLR-15898, consolidating and
>>>> structuring the "Major changes" chapter of the reference guide.
>>>> >>>
>>>> >>> Jan
>>>> >>>
>>>> >>>
>>>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <ja...@cominvent.com>:
>>>> >>>
>>>> >>> Hi,
>>>> >>>
>>>> >>> The list of release blockers can be seen with this JIRA filter:
>>>> https://issues.apache.org/jira/issues/?filter=12351219
>>>> >>>
>>>> >>> After the committer's meeting today we decided to add these to the
>>>> blockers list:
>>>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>>>> >>> - SOLR-15917 Rename 'contrib' as 'module'
>>>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>>>> >>> - SOLR-14660 Migrating HDFS into a package
>>>> >>> - SOLR-12901 Make UnifiedHighlighter the default
>>>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared
>>>> classpath
>>>> >>> - And probably some minor ones too
>>>> >>>
>>>> >>> Each of these are in-flight and are expected to be ready really
>>>> soon™.
>>>> >>>
>>>> >>> There are currently 24 blockers, but some of those are Unsassigned
>>>> and/or have not been given any attention for some time.
>>>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those
>>>> that have not moved anywhere since.
>>>> >>> If you want to own one of them, please assign yourself and
>>>> communicate progress and an ETA.
>>>> >>>
>>>> >>> Jan
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>>
>>>>
>>

Re: New branch and feature freeze for Solr 9.0.0

Posted by David Smiley <ds...@apache.org>.
Woohoo!

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Mar 25, 2022 at 4:16 PM Jan Høydahl <ja...@cominvent.com> wrote:

> Hi,
>
> All code blockers are now cleared:
> https://issues.apache.org/jira/issues/?filter=12351219
> Some work remains for RefGuide and docker release procedures, we can
> continue on those in parallell with the RCs.
>
> I'll proceed with RC1.
>
> Jan
>
> 4. mar. 2022 kl. 21:52 skrev Houston Putman <ho...@apache.org>:
>
> I think we have another blocker for 9.0. Basically there is a bug in the
> updated version of commons-io that causes index files to be set to
> read-only in the filesystem occasionally. The solution is to upgrade
> commons-io, or find a workaround in Solr, but we can have that discussion
> on the JIRA.
>
> More info here: https://issues.apache.org/jira/browse/SOLR-16074
>
> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <ds...@apache.org> wrote:
>
>> I suppose the biggest spots for peer review are:
>> * use of brackets [ ] in the metric name where the request handler is.
>> Thus "/select[shard]"
>> * There is a fundamental difference in how the metrics are tracked on a
>> handler.  Previously, there were metrics for all of /select (no matter how
>> it was invoked), and a few for .distrib. & .shard. depending on how it was
>> invoked.  Now, the request is classified to be a shard request, or not a
>> shard request, after which separate metrics (same type/semantics) are
>> manipulated based on that classification, kind of as if there are two
>> distinct request handlers even though just one is registered.  I think
>> the PR makes this clear.  While I like it, the main trade-off is that a
>> user would be forced to aggregate metrics if they wanted a single metric
>> for the handler.  I think the isShard=true request changes the
>> personality/mode of the handler so much that I prefer to present it as its
>> own identity from a metrics standpoint.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <th...@gmail.com>
>> wrote:
>>
>>> Hi David,
>>>
>>> I read your note about SOLR-14401 but not clear what you need from me?
>>> Seems like you're renaming existing metrics and removing "distrib"
>>> from handlers that don't support a distrib mode, seems right to me.
>>>
>>> I actually haven't done much work on the metrics backend. For Grafana,
>>> it's a JSON file so search / replace the metrics you're changing. The
>>> Solr operator makes it really easy to set up Solr + ZK + Grafana +
>>> Prometheus + Exporter to test out your changes. It'll be pretty
>>> obvious if the dashboard is broken.
>>>
>>> Tim
>>>
>>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <ds...@apache.org> wrote:
>>> >
>>> >
>>> >
>>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <ja...@cominvent.com>
>>> wrote:
>>> >>
>>> >> Hi, and welcome to March!
>>> >>
>>> >> Our initial goal of a RC1 within February slipped, but we are still
>>> in a good position.
>>> >> I'll try to summarize the current code blockers:
>>> >>
>>> >>
>>> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
>>> >>
>>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj
>>> modularization. There is already a draft PR. Hope there will be progress on
>>> this so we don't have to delay solrj modularization until 10.0
>>> >
>>> >
>>> > I'm working with Haythem on this (a colleague).  I think it's close;
>>> it's "just" a refactoring.  The main constraint on this is Haythem's time.
>>> >
>>> >>
>>> >>
>>> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>>> master/9.x
>>> >>
>>> >> This has not seen any movement despite repeated reminders, so unless
>>> there is progress within a few days I'll remove it as blocker and add a
>>> note to the release notes that users relying on running test framework
>>> locally should wait for a later release.
>>> >
>>> >
>>> > I'm interested in looking but not until I get through the other two.
>>> >
>>> >>
>>> >> SOLR-14401  "distrib" request handler metrics should only be tracked
>>> on...
>>> >>
>>> >> There is a PR, not sure how close to merge it is though.
>>> >
>>> >
>>> > I think the core of the change is fine but there are downstream
>>> changes needed.  First level is the prometheus exporter configuration to
>>> not look for ".distrib." vs ".local."; it's different now.  Next level is
>>> the Grafana dashboard.  I don't normally play with JQ, Prometheus or
>>> Grafana so it's taking me some time this week.  I'd appreciate any feedback
>>> on the choices here; so far only Houston has weigh'ed in.  I think Tim
>>> Potter, if you're reading this, would be useful given you did major work
>>> here.
>>> >
>>> >>
>>> >>
>>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics
>>> group is missing in 9.x. There will likely be another blocker due to this.
>>> >>
>>> >>
>>> >> Appreciate an update in this thread on the ETA for each of these.
>>> >>
>>> >> Jan
>>> >>
>>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
>>> >>
>>> >> I created a new blocker
>>> >>
>>> >> SOLR-16040  Fix split packages in hdfs module
>>> >>
>>> >> Not sure if it needs to be a blocker though, but we should try to
>>> avoid split packages as far as we can, and this cannot be done in 9.x.
>>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review
>>> phase.
>>> >> Jenkins is now mostly green after some turmoil!
>>> >>
>>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can
>>> lend a hand there?
>>> >>
>>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient
>>> to untangle ZkStateProvider should be done now, and also rename solr-solrj
>>> as solr-solrj-all so that we can continue with the solrj modularization in
>>> 9.x without back-compat breaks.
>>> >> It seems worthy of a blocker to me, but we need someone willing to do
>>> the work in the next few days. Anyone?
>>> >>
>>> >> I also created SOLR-16041 (not blocker) to try to setup nightly
>>> smoketestRelease Jenkins jobs, I may try to give it a go.
>>> >>
>>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the
>>> timeframe of next week.
>>> >>
>>> >> Jan
>>> >>
>>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <ja...@cominvent.com>:
>>> >>
>>> >> These are the three main code-blockers for doing 9.0.0 RC1:
>>> >>
>>> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not
>>> assume _route_    dsmiley     dsmiley
>>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>>> master/9.x   gus         gus
>>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
>>> tracked on pe  dsmiley     dsmiley
>>> >>
>>> >> The other blockers are mostly about the release process itself,
>>> including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>>> >> When these are resolved, I'll prepare RC1. That means we're really
>>> close now!!
>>> >>
>>> >> Anshum is preparing a release notes draft, and we also need to
>>> complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before
>>> publishing the guide.
>>> >>
>>> >> Jan
>>> >>
>>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <ja...@cominvent.com>:
>>> >>
>>> >> Congrats on HDFS as a package! Huge win! Also some other blockers
>>> have been closed recently.
>>> >>
>>> >> Status on the 9.0 release, one week into February.
>>> >>
>>> >> - I have done a dry-run of an RC and the smoketester. Think the
>>> release scripts are ready!
>>> >> - 11 open blockers:
>>> >>
>>> >> (P) SOLR-15587  Replicas end up with base_url as http on client side
>>> even if clus  thelabdude  thelabdude
>>> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>>              ctargett    ctargett
>>> >> (P) SOLR-15557  Figure out how to handle ref guide page
>>> renames/redirects          janhoy      ctargett
>>> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not
>>> assume _route_    dsmiley     dsmiley
>>> >> (A) SOLR-15949  Use Java 17 in docker
>>>               dsmiley     dsmiley
>>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>>> master/9.x   gus         gus
>>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
>>> tracked on pe  dsmiley     dsmiley
>>> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official""
>>> Dockerfiles   houston     hossman
>>> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts
>>>                           janhoy
>>> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie
>>> for 9.0.0                 janhoy
>>> >> (D) SOLR-13138  Remove deprecated code prior to 9.0
>>>                           romseygeek
>>> >>
>>> >> Based on a quick look, I assigned a readiness-code for each:
>>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be
>>> closed? (S): Stalled. (U): Unassigned
>>> >>
>>> >> Would be great to have a status and ETA for the (S)talled issues.
>>> Perhaps we can push some to 9.0.1 or 9.1?
>>> >>
>>> >> Wrt publishing the ref-guide and docker image, those could be done
>>> semi-manual for 9.0.0 if need be.
>>> >>
>>> >> Jan
>>> >>
>>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <compuwizard123@gmail.com
>>> >:
>>> >>
>>> >>> SOLR-14660  Migrating HDFS into a module
>>>            krisden     ichattopadhyaya
>>> >>
>>> >>
>>> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of
>>> the heavy lifting here!
>>> >>
>>> >> Kevin Risden
>>> >>
>>> >>
>>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <ja...@cominvent.com>
>>> wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> Nine days ago we had 24 blockers. Several blockers have since been
>>> added and several resolved, and today we are at 19.
>>> >>>
>>> >>> I notified in the previous email that I'd remove inactive blockers.
>>> Here is the list of JIRAs for which I intend to remove the blocker flag on
>>> February 1st:
>>> >>>
>>> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
>>> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume
>>> _route_ param is the root ID
>>> >>> SOLR-15242  Consolidate README.md with solr/README.md
>>> >>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly
>>> degraded in Java 11 v 8
>>> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as
>>> "optional" in SolrJ
>>> >>> SOLR-15835  Collection creation failing with https
>>> >>>
>>> >>> Shout out if you are working on one of these and expect it to be
>>> finished soon-ish.
>>> >>>
>>> >>> That leaves 13 blockers:
>>> >>>
>>> >>> Issue key   Summary
>>>           Assignee    Reporter
>>> >>> SOLR-14660  Migrating HDFS into a module
>>>            krisden     ichattopadhyaya
>>> >>> SOLR-15956  Add documentation for creating a docker image from the
>>> binary dis  houston     houston
>>> >>> SOLR-13138  Remove deprecated code prior to 9.0
>>>                       romseygeek
>>> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>>            ctargett    ctargett
>>> >>> SOLR-15949  Use Java 17 in docker
>>>           dsmiley     dsmiley
>>> >>> SOLR-15926  Fix version specification in the Solr Ref Guide
>>>                       houston
>>> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>>> master/9.x   gus         gus
>>> >>> SOLR-12901  Make UnifiedHighlighter the default
>>>           dsmiley     dsmiley
>>> >>> SOLR-14401  """distrib"" request handler metrics should only be
>>> tracked on pe  dsmiley     dsmiley
>>> >>> SOLR-15587  Replicas end up with base_url as http on client side
>>> even if clus  thelabdude  thelabdude
>>> >>> SOLR-15557  Figure out how to handle ref guide page
>>> renames/redirects                      ctargett
>>> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for
>>> 9.0.0                 janhoy
>>> >>> SOLR-15321  "Flesh out process for managing/storing ""official""
>>> Dockerfiles   houston     hossman
>>> >>>
>>> >>> It seems like most of these have had some recent activity.
>>> >>> Please everyone, have a look if you can lend a hand with any of
>>> these, so we can get the list to zero early in February and do the first RC.
>>> >>> I'd appreciate some help on SOLR-15898, consolidating and
>>> structuring the "Major changes" chapter of the reference guide.
>>> >>>
>>> >>> Jan
>>> >>>
>>> >>>
>>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <ja...@cominvent.com>:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> The list of release blockers can be seen with this JIRA filter:
>>> https://issues.apache.org/jira/issues/?filter=12351219
>>> >>>
>>> >>> After the committer's meeting today we decided to add these to the
>>> blockers list:
>>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>>> >>> - SOLR-15917 Rename 'contrib' as 'module'
>>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>>> >>> - SOLR-14660 Migrating HDFS into a package
>>> >>> - SOLR-12901 Make UnifiedHighlighter the default
>>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared
>>> classpath
>>> >>> - And probably some minor ones too
>>> >>>
>>> >>> Each of these are in-flight and are expected to be ready really
>>> soon™.
>>> >>>
>>> >>> There are currently 24 blockers, but some of those are Unsassigned
>>> and/or have not been given any attention for some time.
>>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those
>>> that have not moved anywhere since.
>>> >>> If you want to own one of them, please assign yourself and
>>> communicate progress and an ETA.
>>> >>>
>>> >>> Jan
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>
>>>
>

Re: New branch and feature freeze for Solr 9.0.0

Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,

All code blockers are now cleared: https://issues.apache.org/jira/issues/?filter=12351219
Some work remains for RefGuide and docker release procedures, we can continue on those in parallell with the RCs.

I'll proceed with RC1.

Jan

> 4. mar. 2022 kl. 21:52 skrev Houston Putman <ho...@apache.org>:
> 
> I think we have another blocker for 9.0. Basically there is a bug in the updated version of commons-io that causes index files to be set to read-only in the filesystem occasionally. The solution is to upgrade commons-io, or find a workaround in Solr, but we can have that discussion on the JIRA.
> 
> More info here: https://issues.apache.org/jira/browse/SOLR-16074 <https://issues.apache.org/jira/browse/SOLR-16074>
> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
> I suppose the biggest spots for peer review are:
> * use of brackets [ ] in the metric name where the request handler is.  Thus "/select[shard]"
> * There is a fundamental difference in how the metrics are tracked on a handler.  Previously, there were metrics for all of /select (no matter how it was invoked), and a few for .distrib. & .shard. depending on how it was invoked.  Now, the request is classified to be a shard request, or not a shard request, after which separate metrics (same type/semantics) are manipulated based on that classification, kind of as if there are two distinct request handlers even though just one is registered.  I think the PR makes this clear.  While I like it, the main trade-off is that a user would be forced to aggregate metrics if they wanted a single metric for the handler.  I think the isShard=true request changes the personality/mode of the handler so much that I prefer to present it as its own identity from a metrics standpoint.
> 
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
> 
> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <thelabdude@gmail.com <ma...@gmail.com>> wrote:
> Hi David,
> 
> I read your note about SOLR-14401 but not clear what you need from me?
> Seems like you're renaming existing metrics and removing "distrib"
> from handlers that don't support a distrib mode, seems right to me.
> 
> I actually haven't done much work on the metrics backend. For Grafana,
> it's a JSON file so search / replace the metrics you're changing. The
> Solr operator makes it really easy to set up Solr + ZK + Grafana +
> Prometheus + Exporter to test out your changes. It'll be pretty
> obvious if the dashboard is broken.
> 
> Tim
> 
> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
> >
> >
> >
> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
> >>
> >> Hi, and welcome to March!
> >>
> >> Our initial goal of a RC1 within February slipped, but we are still in a good position.
> >> I'll try to summarize the current code blockers:
> >>
> >>
> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
> >>
> >> This is new, a spin-off from SOLR-15342 to prepare for solrj modularization. There is already a draft PR. Hope there will be progress on this so we don't have to delay solrj modularization until 10.0
> >
> >
> > I'm working with Haythem on this (a colleague).  I think it's close; it's "just" a refactoring.  The main constraint on this is Haythem's time.
> >
> >>
> >>
> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x
> >>
> >> This has not seen any movement despite repeated reminders, so unless there is progress within a few days I'll remove it as blocker and add a note to the release notes that users relying on running test framework locally should wait for a later release.
> >
> >
> > I'm interested in looking but not until I get through the other two.
> >
> >>
> >> SOLR-14401  "distrib" request handler metrics should only be tracked on...
> >>
> >> There is a PR, not sure how close to merge it is though.
> >
> >
> > I think the core of the change is fine but there are downstream changes needed.  First level is the prometheus exporter configuration to not look for ".distrib." vs ".local."; it's different now.  Next level is the Grafana dashboard.  I don't normally play with JQ, Prometheus or Grafana so it's taking me some time this week.  I'd appreciate any feedback on the choices here; so far only Houston has weigh'ed in.  I think Tim Potter, if you're reading this, would be useful given you did major work here.
> >
> >>
> >>
> >> Also, David found a new blocker bug yesterday - the "jetty" metrics group is missing in 9.x. There will likely be another blocker due to this.
> >>
> >>
> >> Appreciate an update in this thread on the ETA for each of these.
> >>
> >> Jan
> >>
> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
> >>
> >> I created a new blocker
> >>
> >> SOLR-16040  Fix split packages in hdfs module
> >>
> >> Not sure if it needs to be a blocker though, but we should try to avoid split packages as far as we can, and this cannot be done in 9.x.
> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
> >> Jenkins is now mostly green after some turmoil!
> >>
> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can lend a hand there?
> >>
> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to untangle ZkStateProvider should be done now, and also rename solr-solrj as solr-solrj-all so that we can continue with the solrj modularization in 9.x without back-compat breaks.
> >> It seems worthy of a blocker to me, but we need someone willing to do the work in the next few days. Anyone?
> >>
> >> I also created SOLR-16041 (not blocker) to try to setup nightly smoketestRelease Jenkins jobs, I may try to give it a go.
> >>
> >> Assuming progress on the above, I'm still hopeful for an RC1 in the timeframe of next week.
> >>
> >> Jan
> >>
> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
> >>
> >> These are the three main code-blockers for doing 9.0.0 RC1:
> >>
> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
> >> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
> >>
> >> The other blockers are mostly about the release process itself, including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
> >> When these are resolved, I'll prepare RC1. That means we're really close now!!
> >>
> >> Anshum is preparing a release notes draft, and we also need to complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing the guide.
> >>
> >> Jan
> >>
> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
> >>
> >> Congrats on HDFS as a package! Huge win! Also some other blockers have been closed recently.
> >>
> >> Status on the 9.0 release, one week into February.
> >>
> >> - I have done a dry-run of an RC and the smoketester. Think the release scripts are ready!
> >> - 11 open blockers:
> >>
> >> (P) SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
> >> (P) SOLR-15557  Figure out how to handle ref guide page renames/redirects          janhoy      ctargett
> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
> >> (A) SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
> >> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts                                  janhoy
> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
> >> (D) SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
> >>
> >> Based on a quick look, I assigned a readiness-code for each:
> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed? (S): Stalled. (U): Unassigned
> >>
> >> Would be great to have a status and ETA for the (S)talled issues. Perhaps we can push some to 9.0.1 or 9.1?
> >>
> >> Wrt publishing the ref-guide and docker image, those could be done semi-manual for 9.0.0 if need be.
> >>
> >> Jan
> >>
> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <compuwizard123@gmail.com <ma...@gmail.com>>:
> >>
> >>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
> >>
> >>
> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of the heavy lifting here!
> >>
> >> Kevin Risden
> >>
> >>
> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Nine days ago we had 24 blockers. Several blockers have since been added and several resolved, and today we are at 19.
> >>>
> >>> I notified in the previous email that I'd remove inactive blockers. Here is the list of JIRAs for which I intend to remove the blocker flag on February 1st:
> >>>
> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume _route_ param is the root ID
> >>> SOLR-15242  Consolidate README.md with solr/README.md
> >>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly degraded in Java 11 v 8
> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as "optional" in SolrJ
> >>> SOLR-15835  Collection creation failing with https
> >>>
> >>> Shout out if you are working on one of these and expect it to be finished soon-ish.
> >>>
> >>> That leaves 13 blockers:
> >>>
> >>> Issue key   Summary                                                            Assignee    Reporter
> >>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
> >>> SOLR-15956  Add documentation for creating a docker image from the binary dis  houston     houston
> >>> SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
> >>> SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
> >>> SOLR-15926  Fix version specification in the Solr Ref Guide                                houston
> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
> >>> SOLR-12901  Make UnifiedHighlighter the default                                dsmiley     dsmiley
> >>> SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
> >>> SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
> >>> SOLR-15557  Figure out how to handle ref guide page renames/redirects                      ctargett
> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
> >>> SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
> >>>
> >>> It seems like most of these have had some recent activity.
> >>> Please everyone, have a look if you can lend a hand with any of these, so we can get the list to zero early in February and do the first RC.
> >>> I'd appreciate some help on SOLR-15898, consolidating and structuring the "Major changes" chapter of the reference guide.
> >>>
> >>> Jan
> >>>
> >>>
> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <jan.asf@cominvent.com <ma...@cominvent.com>>:
> >>>
> >>> Hi,
> >>>
> >>> The list of release blockers can be seen with this JIRA filter: https://issues.apache.org/jira/issues/?filter=12351219 <https://issues.apache.org/jira/issues/?filter=12351219>
> >>>
> >>> After the committer's meeting today we decided to add these to the blockers list:
> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
> >>> - SOLR-15917 Rename 'contrib' as 'module'
> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
> >>> - SOLR-14660 Migrating HDFS into a package
> >>> - SOLR-12901 Make UnifiedHighlighter the default
> >>> - SOLR-15914 Make it super simple to add a contrib module to shared classpath
> >>> - And probably some minor ones too
> >>>
> >>> Each of these are in-flight and are expected to be ready really soon™.
> >>>
> >>> There are currently 24 blockers, but some of those are Unsassigned and/or have not been given any attention for some time.
> >>> On Feb 1st I'll take the freedom to remove blocker flag for those that have not moved anywhere since.
> >>> If you want to own one of them, please assign yourself and communicate progress and an ETA.
> >>>
> >>> Jan
> >>>
> >>>
> >>
> >>
> >>
> >>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <ma...@solr.apache.org>
> For additional commands, e-mail: dev-help@solr.apache.org <ma...@solr.apache.org>
>

Re: New branch and feature freeze for Solr 9.0.0

Posted by Houston Putman <ho...@apache.org>.
I think we have another blocker for 9.0. Basically there is a bug in the
updated version of commons-io that causes index files to be set to
read-only in the filesystem occasionally. The solution is to upgrade
commons-io, or find a workaround in Solr, but we can have that discussion
on the JIRA.

More info here: https://issues.apache.org/jira/browse/SOLR-16074

On Tue, Mar 1, 2022 at 5:02 PM David Smiley <ds...@apache.org> wrote:

> I suppose the biggest spots for peer review are:
> * use of brackets [ ] in the metric name where the request handler is.
> Thus "/select[shard]"
> * There is a fundamental difference in how the metrics are tracked on a
> handler.  Previously, there were metrics for all of /select (no matter how
> it was invoked), and a few for .distrib. & .shard. depending on how it was
> invoked.  Now, the request is classified to be a shard request, or not a
> shard request, after which separate metrics (same type/semantics) are
> manipulated based on that classification, kind of as if there are two
> distinct request handlers even though just one is registered.  I think
> the PR makes this clear.  While I like it, the main trade-off is that a
> user would be forced to aggregate metrics if they wanted a single metric
> for the handler.  I think the isShard=true request changes the
> personality/mode of the handler so much that I prefer to present it as its
> own identity from a metrics standpoint.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <th...@gmail.com>
> wrote:
>
>> Hi David,
>>
>> I read your note about SOLR-14401 but not clear what you need from me?
>> Seems like you're renaming existing metrics and removing "distrib"
>> from handlers that don't support a distrib mode, seems right to me.
>>
>> I actually haven't done much work on the metrics backend. For Grafana,
>> it's a JSON file so search / replace the metrics you're changing. The
>> Solr operator makes it really easy to set up Solr + ZK + Grafana +
>> Prometheus + Exporter to test out your changes. It'll be pretty
>> obvious if the dashboard is broken.
>>
>> Tim
>>
>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <ds...@apache.org> wrote:
>> >
>> >
>> >
>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>> >>
>> >> Hi, and welcome to March!
>> >>
>> >> Our initial goal of a RC1 within February slipped, but we are still in
>> a good position.
>> >> I'll try to summarize the current code blockers:
>> >>
>> >>
>> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
>> >>
>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj
>> modularization. There is already a draft PR. Hope there will be progress on
>> this so we don't have to delay solrj modularization until 10.0
>> >
>> >
>> > I'm working with Haythem on this (a colleague).  I think it's close;
>> it's "just" a refactoring.  The main constraint on this is Haythem's time.
>> >
>> >>
>> >>
>> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>> master/9.x
>> >>
>> >> This has not seen any movement despite repeated reminders, so unless
>> there is progress within a few days I'll remove it as blocker and add a
>> note to the release notes that users relying on running test framework
>> locally should wait for a later release.
>> >
>> >
>> > I'm interested in looking but not until I get through the other two.
>> >
>> >>
>> >> SOLR-14401  "distrib" request handler metrics should only be tracked
>> on...
>> >>
>> >> There is a PR, not sure how close to merge it is though.
>> >
>> >
>> > I think the core of the change is fine but there are downstream changes
>> needed.  First level is the prometheus exporter configuration to not look
>> for ".distrib." vs ".local."; it's different now.  Next level is the
>> Grafana dashboard.  I don't normally play with JQ, Prometheus or Grafana so
>> it's taking me some time this week.  I'd appreciate any feedback on the
>> choices here; so far only Houston has weigh'ed in.  I think Tim Potter, if
>> you're reading this, would be useful given you did major work here.
>> >
>> >>
>> >>
>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics
>> group is missing in 9.x. There will likely be another blocker due to this.
>> >>
>> >>
>> >> Appreciate an update in this thread on the ETA for each of these.
>> >>
>> >> Jan
>> >>
>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
>> >>
>> >> I created a new blocker
>> >>
>> >> SOLR-16040  Fix split packages in hdfs module
>> >>
>> >> Not sure if it needs to be a blocker though, but we should try to
>> avoid split packages as far as we can, and this cannot be done in 9.x.
>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
>> >> Jenkins is now mostly green after some turmoil!
>> >>
>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can
>> lend a hand there?
>> >>
>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to
>> untangle ZkStateProvider should be done now, and also rename solr-solrj as
>> solr-solrj-all so that we can continue with the solrj modularization in 9.x
>> without back-compat breaks.
>> >> It seems worthy of a blocker to me, but we need someone willing to do
>> the work in the next few days. Anyone?
>> >>
>> >> I also created SOLR-16041 (not blocker) to try to setup nightly
>> smoketestRelease Jenkins jobs, I may try to give it a go.
>> >>
>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the
>> timeframe of next week.
>> >>
>> >> Jan
>> >>
>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <ja...@cominvent.com>:
>> >>
>> >> These are the three main code-blockers for doing 9.0.0 RC1:
>> >>
>> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not
>> assume _route_    dsmiley     dsmiley
>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>> master/9.x   gus         gus
>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
>> tracked on pe  dsmiley     dsmiley
>> >>
>> >> The other blockers are mostly about the release process itself,
>> including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>> >> When these are resolved, I'll prepare RC1. That means we're really
>> close now!!
>> >>
>> >> Anshum is preparing a release notes draft, and we also need to
>> complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before
>> publishing the guide.
>> >>
>> >> Jan
>> >>
>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <ja...@cominvent.com>:
>> >>
>> >> Congrats on HDFS as a package! Huge win! Also some other blockers have
>> been closed recently.
>> >>
>> >> Status on the 9.0 release, one week into February.
>> >>
>> >> - I have done a dry-run of an RC and the smoketester. Think the
>> release scripts are ready!
>> >> - 11 open blockers:
>> >>
>> >> (P) SOLR-15587  Replicas end up with base_url as http on client side
>> even if clus  thelabdude  thelabdude
>> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>              ctargett    ctargett
>> >> (P) SOLR-15557  Figure out how to handle ref guide page
>> renames/redirects          janhoy      ctargett
>> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not
>> assume _route_    dsmiley     dsmiley
>> >> (A) SOLR-15949  Use Java 17 in docker
>>             dsmiley     dsmiley
>> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>> master/9.x   gus         gus
>> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
>> tracked on pe  dsmiley     dsmiley
>> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official""
>> Dockerfiles   houston     hossman
>> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts
>>                         janhoy
>> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie
>> for 9.0.0                 janhoy
>> >> (D) SOLR-13138  Remove deprecated code prior to 9.0
>>                         romseygeek
>> >>
>> >> Based on a quick look, I assigned a readiness-code for each:
>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed?
>> (S): Stalled. (U): Unassigned
>> >>
>> >> Would be great to have a status and ETA for the (S)talled issues.
>> Perhaps we can push some to 9.0.1 or 9.1?
>> >>
>> >> Wrt publishing the ref-guide and docker image, those could be done
>> semi-manual for 9.0.0 if need be.
>> >>
>> >> Jan
>> >>
>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <co...@gmail.com>:
>> >>
>> >>> SOLR-14660  Migrating HDFS into a module
>>          krisden     ichattopadhyaya
>> >>
>> >>
>> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of
>> the heavy lifting here!
>> >>
>> >> Kevin Risden
>> >>
>> >>
>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> Nine days ago we had 24 blockers. Several blockers have since been
>> added and several resolved, and today we are at 19.
>> >>>
>> >>> I notified in the previous email that I'd remove inactive blockers.
>> Here is the list of JIRAs for which I intend to remove the blocker flag on
>> February 1st:
>> >>>
>> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
>> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume
>> _route_ param is the root ID
>> >>> SOLR-15242  Consolidate README.md with solr/README.md
>> >>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly
>> degraded in Java 11 v 8
>> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as
>> "optional" in SolrJ
>> >>> SOLR-15835  Collection creation failing with https
>> >>>
>> >>> Shout out if you are working on one of these and expect it to be
>> finished soon-ish.
>> >>>
>> >>> That leaves 13 blockers:
>> >>>
>> >>> Issue key   Summary
>>           Assignee    Reporter
>> >>> SOLR-14660  Migrating HDFS into a module
>>          krisden     ichattopadhyaya
>> >>> SOLR-15956  Add documentation for creating a docker image from the
>> binary dis  houston     houston
>> >>> SOLR-13138  Remove deprecated code prior to 9.0
>>                       romseygeek
>> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>          ctargett    ctargett
>> >>> SOLR-15949  Use Java 17 in docker
>>           dsmiley     dsmiley
>> >>> SOLR-15926  Fix version specification in the Solr Ref Guide
>>                       houston
>> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>> master/9.x   gus         gus
>> >>> SOLR-12901  Make UnifiedHighlighter the default
>>           dsmiley     dsmiley
>> >>> SOLR-14401  """distrib"" request handler metrics should only be
>> tracked on pe  dsmiley     dsmiley
>> >>> SOLR-15587  Replicas end up with base_url as http on client side even
>> if clus  thelabdude  thelabdude
>> >>> SOLR-15557  Figure out how to handle ref guide page
>> renames/redirects                      ctargett
>> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for
>> 9.0.0                 janhoy
>> >>> SOLR-15321  "Flesh out process for managing/storing ""official""
>> Dockerfiles   houston     hossman
>> >>>
>> >>> It seems like most of these have had some recent activity.
>> >>> Please everyone, have a look if you can lend a hand with any of
>> these, so we can get the list to zero early in February and do the first RC.
>> >>> I'd appreciate some help on SOLR-15898, consolidating and structuring
>> the "Major changes" chapter of the reference guide.
>> >>>
>> >>> Jan
>> >>>
>> >>>
>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <ja...@cominvent.com>:
>> >>>
>> >>> Hi,
>> >>>
>> >>> The list of release blockers can be seen with this JIRA filter:
>> https://issues.apache.org/jira/issues/?filter=12351219
>> >>>
>> >>> After the committer's meeting today we decided to add these to the
>> blockers list:
>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>> >>> - SOLR-15917 Rename 'contrib' as 'module'
>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>> >>> - SOLR-14660 Migrating HDFS into a package
>> >>> - SOLR-12901 Make UnifiedHighlighter the default
>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared
>> classpath
>> >>> - And probably some minor ones too
>> >>>
>> >>> Each of these are in-flight and are expected to be ready really soon™.
>> >>>
>> >>> There are currently 24 blockers, but some of those are Unsassigned
>> and/or have not been given any attention for some time.
>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those
>> that have not moved anywhere since.
>> >>> If you want to own one of them, please assign yourself and
>> communicate progress and an ETA.
>> >>>
>> >>> Jan
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>>

Re: New branch and feature freeze for Solr 9.0.0

Posted by David Smiley <ds...@apache.org>.
I suppose the biggest spots for peer review are:
* use of brackets [ ] in the metric name where the request handler is.
Thus "/select[shard]"
* There is a fundamental difference in how the metrics are tracked on a
handler.  Previously, there were metrics for all of /select (no matter how
it was invoked), and a few for .distrib. & .shard. depending on how it was
invoked.  Now, the request is classified to be a shard request, or not a
shard request, after which separate metrics (same type/semantics) are
manipulated based on that classification, kind of as if there are two
distinct request handlers even though just one is registered.  I think
the PR makes this clear.  While I like it, the main trade-off is that a
user would be forced to aggregate metrics if they wanted a single metric
for the handler.  I think the isShard=true request changes the
personality/mode of the handler so much that I prefer to present it as its
own identity from a metrics standpoint.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <th...@gmail.com> wrote:

> Hi David,
>
> I read your note about SOLR-14401 but not clear what you need from me?
> Seems like you're renaming existing metrics and removing "distrib"
> from handlers that don't support a distrib mode, seems right to me.
>
> I actually haven't done much work on the metrics backend. For Grafana,
> it's a JSON file so search / replace the metrics you're changing. The
> Solr operator makes it really easy to set up Solr + ZK + Grafana +
> Prometheus + Exporter to test out your changes. It'll be pretty
> obvious if the dashboard is broken.
>
> Tim
>
> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <ds...@apache.org> wrote:
> >
> >
> >
> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> >>
> >> Hi, and welcome to March!
> >>
> >> Our initial goal of a RC1 within February slipped, but we are still in
> a good position.
> >> I'll try to summarize the current code blockers:
> >>
> >>
> >> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
> >>
> >> This is new, a spin-off from SOLR-15342 to prepare for solrj
> modularization. There is already a draft PR. Hope there will be progress on
> this so we don't have to delay solrj modularization until 10.0
> >
> >
> > I'm working with Haythem on this (a colleague).  I think it's close;
> it's "just" a refactoring.  The main constraint on this is Haythem's time.
> >
> >>
> >>
> >> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x
> >>
> >> This has not seen any movement despite repeated reminders, so unless
> there is progress within a few days I'll remove it as blocker and add a
> note to the release notes that users relying on running test framework
> locally should wait for a later release.
> >
> >
> > I'm interested in looking but not until I get through the other two.
> >
> >>
> >> SOLR-14401  "distrib" request handler metrics should only be tracked
> on...
> >>
> >> There is a PR, not sure how close to merge it is though.
> >
> >
> > I think the core of the change is fine but there are downstream changes
> needed.  First level is the prometheus exporter configuration to not look
> for ".distrib." vs ".local."; it's different now.  Next level is the
> Grafana dashboard.  I don't normally play with JQ, Prometheus or Grafana so
> it's taking me some time this week.  I'd appreciate any feedback on the
> choices here; so far only Houston has weigh'ed in.  I think Tim Potter, if
> you're reading this, would be useful given you did major work here.
> >
> >>
> >>
> >> Also, David found a new blocker bug yesterday - the "jetty" metrics
> group is missing in 9.x. There will likely be another blocker due to this.
> >>
> >>
> >> Appreciate an update in this thread on the ETA for each of these.
> >>
> >> Jan
> >>
> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
> >>
> >> I created a new blocker
> >>
> >> SOLR-16040  Fix split packages in hdfs module
> >>
> >> Not sure if it needs to be a blocker though, but we should try to avoid
> split packages as far as we can, and this cannot be done in 9.x.
> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
> >> Jenkins is now mostly green after some turmoil!
> >>
> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can
> lend a hand there?
> >>
> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to
> untangle ZkStateProvider should be done now, and also rename solr-solrj as
> solr-solrj-all so that we can continue with the solrj modularization in 9.x
> without back-compat breaks.
> >> It seems worthy of a blocker to me, but we need someone willing to do
> the work in the next few days. Anyone?
> >>
> >> I also created SOLR-16041 (not blocker) to try to setup nightly
> smoketestRelease Jenkins jobs, I may try to give it a go.
> >>
> >> Assuming progress on the above, I'm still hopeful for an RC1 in the
> timeframe of next week.
> >>
> >> Jan
> >>
> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <ja...@cominvent.com>:
> >>
> >> These are the three main code-blockers for doing 9.0.0 RC1:
> >>
> >> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume
> _route_    dsmiley     dsmiley
> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x   gus         gus
> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
> tracked on pe  dsmiley     dsmiley
> >>
> >> The other blockers are mostly about the release process itself,
> including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
> >> When these are resolved, I'll prepare RC1. That means we're really
> close now!!
> >>
> >> Anshum is preparing a release notes draft, and we also need to complete
> "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing
> the guide.
> >>
> >> Jan
> >>
> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <ja...@cominvent.com>:
> >>
> >> Congrats on HDFS as a package! Huge win! Also some other blockers have
> been closed recently.
> >>
> >> Status on the 9.0 release, one week into February.
> >>
> >> - I have done a dry-run of an RC and the smoketester. Think the release
> scripts are ready!
> >> - 11 open blockers:
> >>
> >> (P) SOLR-15587  Replicas end up with base_url as http on client side
> even if clus  thelabdude  thelabdude
> >> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>            ctargett    ctargett
> >> (P) SOLR-15557  Figure out how to handle ref guide page
> renames/redirects          janhoy      ctargett
> >> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume
> _route_    dsmiley     dsmiley
> >> (A) SOLR-15949  Use Java 17 in docker
>             dsmiley     dsmiley
> >> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x   gus         gus
> >> (S) SOLR-14401  """distrib"" request handler metrics should only be
> tracked on pe  dsmiley     dsmiley
> >> (S) SOLR-15321  "Flesh out process for managing/storing ""official""
> Dockerfiles   houston     hossman
> >> (U) SOLR-15941  Wire Antora refGuide into the release scripts
>                         janhoy
> >> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie
> for 9.0.0                 janhoy
> >> (D) SOLR-13138  Remove deprecated code prior to 9.0
>                         romseygeek
> >>
> >> Based on a quick look, I assigned a readiness-code for each:
> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed?
> (S): Stalled. (U): Unassigned
> >>
> >> Would be great to have a status and ETA for the (S)talled issues.
> Perhaps we can push some to 9.0.1 or 9.1?
> >>
> >> Wrt publishing the ref-guide and docker image, those could be done
> semi-manual for 9.0.0 if need be.
> >>
> >> Jan
> >>
> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <co...@gmail.com>:
> >>
> >>> SOLR-14660  Migrating HDFS into a module
>          krisden     ichattopadhyaya
> >>
> >>
> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of
> the heavy lifting here!
> >>
> >> Kevin Risden
> >>
> >>
> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <ja...@cominvent.com>
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Nine days ago we had 24 blockers. Several blockers have since been
> added and several resolved, and today we are at 19.
> >>>
> >>> I notified in the previous email that I'd remove inactive blockers.
> Here is the list of JIRAs for which I intend to remove the blocker flag on
> February 1st:
> >>>
> >>> SOLR-14097  coreRootDirectory should be solr_home/cores
> >>> SOLR-15064  Atomic/partial updates to nested docs should not assume
> _route_ param is the root ID
> >>> SOLR-15242  Consolidate README.md with solr/README.md
> >>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly
> degraded in Java 11 v 8
> >>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as
> "optional" in SolrJ
> >>> SOLR-15835  Collection creation failing with https
> >>>
> >>> Shout out if you are working on one of these and expect it to be
> finished soon-ish.
> >>>
> >>> That leaves 13 blockers:
> >>>
> >>> Issue key   Summary
>         Assignee    Reporter
> >>> SOLR-14660  Migrating HDFS into a module
>          krisden     ichattopadhyaya
> >>> SOLR-15956  Add documentation for creating a docker image from the
> binary dis  houston     houston
> >>> SOLR-13138  Remove deprecated code prior to 9.0
>                     romseygeek
> >>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>          ctargett    ctargett
> >>> SOLR-15949  Use Java 17 in docker
>         dsmiley     dsmiley
> >>> SOLR-15926  Fix version specification in the Solr Ref Guide
>                     houston
> >>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x   gus         gus
> >>> SOLR-12901  Make UnifiedHighlighter the default
>         dsmiley     dsmiley
> >>> SOLR-14401  """distrib"" request handler metrics should only be
> tracked on pe  dsmiley     dsmiley
> >>> SOLR-15587  Replicas end up with base_url as http on client side even
> if clus  thelabdude  thelabdude
> >>> SOLR-15557  Figure out how to handle ref guide page renames/redirects
>                     ctargett
> >>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for
> 9.0.0                 janhoy
> >>> SOLR-15321  "Flesh out process for managing/storing ""official""
> Dockerfiles   houston     hossman
> >>>
> >>> It seems like most of these have had some recent activity.
> >>> Please everyone, have a look if you can lend a hand with any of these,
> so we can get the list to zero early in February and do the first RC.
> >>> I'd appreciate some help on SOLR-15898, consolidating and structuring
> the "Major changes" chapter of the reference guide.
> >>>
> >>> Jan
> >>>
> >>>
> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <ja...@cominvent.com>:
> >>>
> >>> Hi,
> >>>
> >>> The list of release blockers can be seen with this JIRA filter:
> https://issues.apache.org/jira/issues/?filter=12351219
> >>>
> >>> After the committer's meeting today we decided to add these to the
> blockers list:
> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
> >>> - SOLR-15917 Rename 'contrib' as 'module'
> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
> >>> - SOLR-14660 Migrating HDFS into a package
> >>> - SOLR-12901 Make UnifiedHighlighter the default
> >>> - SOLR-15914 Make it super simple to add a contrib module to shared
> classpath
> >>> - And probably some minor ones too
> >>>
> >>> Each of these are in-flight and are expected to be ready really soon™.
> >>>
> >>> There are currently 24 blockers, but some of those are Unsassigned
> and/or have not been given any attention for some time.
> >>> On Feb 1st I'll take the freedom to remove blocker flag for those that
> have not moved anywhere since.
> >>> If you want to own one of them, please assign yourself and communicate
> progress and an ETA.
> >>>
> >>> Jan
> >>>
> >>>
> >>
> >>
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: New branch and feature freeze for Solr 9.0.0

Posted by Timothy Potter <th...@gmail.com>.
Hi David,

I read your note about SOLR-14401 but not clear what you need from me?
Seems like you're renaming existing metrics and removing "distrib"
from handlers that don't support a distrib mode, seems right to me.

I actually haven't done much work on the metrics backend. For Grafana,
it's a JSON file so search / replace the metrics you're changing. The
Solr operator makes it really easy to set up Solr + ZK + Grafana +
Prometheus + Exporter to test out your changes. It'll be pretty
obvious if the dashboard is broken.

Tim

On Tue, Mar 1, 2022 at 7:01 AM David Smiley <ds...@apache.org> wrote:
>
>
>
> On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <ja...@cominvent.com> wrote:
>>
>> Hi, and welcome to March!
>>
>> Our initial goal of a RC1 within February slipped, but we are still in a good position.
>> I'll try to summarize the current code blockers:
>>
>>
>> SOLR-16061  Decouple CloudSolrClient from ZkStateReader
>>
>> This is new, a spin-off from SOLR-15342 to prepare for solrj modularization. There is already a draft PR. Hope there will be progress on this so we don't have to delay solrj modularization until 10.0
>
>
> I'm working with Haythem on this (a colleague).  I think it's close; it's "just" a refactoring.  The main constraint on this is Haythem's time.
>
>>
>>
>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x
>>
>> This has not seen any movement despite repeated reminders, so unless there is progress within a few days I'll remove it as blocker and add a note to the release notes that users relying on running test framework locally should wait for a later release.
>
>
> I'm interested in looking but not until I get through the other two.
>
>>
>> SOLR-14401  "distrib" request handler metrics should only be tracked on...
>>
>> There is a PR, not sure how close to merge it is though.
>
>
> I think the core of the change is fine but there are downstream changes needed.  First level is the prometheus exporter configuration to not look for ".distrib." vs ".local."; it's different now.  Next level is the Grafana dashboard.  I don't normally play with JQ, Prometheus or Grafana so it's taking me some time this week.  I'd appreciate any feedback on the choices here; so far only Houston has weigh'ed in.  I think Tim Potter, if you're reading this, would be useful given you did major work here.
>
>>
>>
>> Also, David found a new blocker bug yesterday - the "jetty" metrics group is missing in 9.x. There will likely be another blocker due to this.
>>
>>
>> Appreciate an update in this thread on the ETA for each of these.
>>
>> Jan
>>
>> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
>>
>> I created a new blocker
>>
>> SOLR-16040  Fix split packages in hdfs module
>>
>> Not sure if it needs to be a blocker though, but we should try to avoid split packages as far as we can, and this cannot be done in 9.x.
>> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
>> Jenkins is now mostly green after some turmoil!
>>
>> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can lend a hand there?
>>
>> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to untangle ZkStateProvider should be done now, and also rename solr-solrj as solr-solrj-all so that we can continue with the solrj modularization in 9.x without back-compat breaks.
>> It seems worthy of a blocker to me, but we need someone willing to do the work in the next few days. Anyone?
>>
>> I also created SOLR-16041 (not blocker) to try to setup nightly smoketestRelease Jenkins jobs, I may try to give it a go.
>>
>> Assuming progress on the above, I'm still hopeful for an RC1 in the timeframe of next week.
>>
>> Jan
>>
>> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <ja...@cominvent.com>:
>>
>> These are the three main code-blockers for doing 9.0.0 RC1:
>>
>> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>>
>> The other blockers are mostly about the release process itself, including docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
>> When these are resolved, I'll prepare RC1. That means we're really close now!!
>>
>> Anshum is preparing a release notes draft, and we also need to complete "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing the guide.
>>
>> Jan
>>
>> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <ja...@cominvent.com>:
>>
>> Congrats on HDFS as a package! Huge win! Also some other blockers have been closed recently.
>>
>> Status on the 9.0 release, one week into February.
>>
>> - I have done a dry-run of an RC and the smoketester. Think the release scripts are ready!
>> - 11 open blockers:
>>
>> (P) SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
>> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
>> (P) SOLR-15557  Figure out how to handle ref guide page renames/redirects          janhoy      ctargett
>> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume _route_    dsmiley     dsmiley
>> (A) SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
>> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>> (S) SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>> (S) SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
>> (U) SOLR-15941  Wire Antora refGuide into the release scripts                                  janhoy
>> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
>> (D) SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>>
>> Based on a quick look, I assigned a readiness-code for each:
>> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed? (S): Stalled. (U): Unassigned
>>
>> Would be great to have a status and ETA for the (S)talled issues. Perhaps we can push some to 9.0.1 or 9.1?
>>
>> Wrt publishing the ref-guide and docker image, those could be done semi-manual for 9.0.0 if need be.
>>
>> Jan
>>
>> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <co...@gmail.com>:
>>
>>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>>
>>
>> This was just merged. Big shout out to Istvan Farkas for doing 99% of the heavy lifting here!
>>
>> Kevin Risden
>>
>>
>> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <ja...@cominvent.com> wrote:
>>>
>>> Hi,
>>>
>>> Nine days ago we had 24 blockers. Several blockers have since been added and several resolved, and today we are at 19.
>>>
>>> I notified in the previous email that I'd remove inactive blockers. Here is the list of JIRAs for which I intend to remove the blocker flag on February 1st:
>>>
>>> SOLR-14097  coreRootDirectory should be solr_home/cores
>>> SOLR-15064  Atomic/partial updates to nested docs should not assume _route_ param is the root ID
>>> SOLR-15242  Consolidate README.md with solr/README.md
>>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly degraded in Java 11 v 8
>>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as "optional" in SolrJ
>>> SOLR-15835  Collection creation failing with https
>>>
>>> Shout out if you are working on one of these and expect it to be finished soon-ish.
>>>
>>> That leaves 13 blockers:
>>>
>>> Issue key   Summary                                                            Assignee    Reporter
>>> SOLR-14660  Migrating HDFS into a module                                       krisden     ichattopadhyaya
>>> SOLR-15956  Add documentation for creating a docker image from the binary dis  houston     houston
>>> SOLR-13138  Remove deprecated code prior to 9.0                                            romseygeek
>>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll                         ctargett    ctargett
>>> SOLR-15949  Use Java 17 in docker                                              dsmiley     dsmiley
>>> SOLR-15926  Fix version specification in the Solr Ref Guide                                houston
>>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for master/9.x   gus         gus
>>> SOLR-12901  Make UnifiedHighlighter the default                                dsmiley     dsmiley
>>> SOLR-14401  """distrib"" request handler metrics should only be tracked on pe  dsmiley     dsmiley
>>> SOLR-15587  Replicas end up with base_url as http on client side even if clus  thelabdude  thelabdude
>>> SOLR-15557  Figure out how to handle ref guide page renames/redirects                      ctargett
>>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for 9.0.0                 janhoy
>>> SOLR-15321  "Flesh out process for managing/storing ""official"" Dockerfiles   houston     hossman
>>>
>>> It seems like most of these have had some recent activity.
>>> Please everyone, have a look if you can lend a hand with any of these, so we can get the list to zero early in February and do the first RC.
>>> I'd appreciate some help on SOLR-15898, consolidating and structuring the "Major changes" chapter of the reference guide.
>>>
>>> Jan
>>>
>>>
>>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <ja...@cominvent.com>:
>>>
>>> Hi,
>>>
>>> The list of release blockers can be seen with this JIRA filter: https://issues.apache.org/jira/issues/?filter=12351219
>>>
>>> After the committer's meeting today we decided to add these to the blockers list:
>>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>>> - SOLR-15917 Rename 'contrib' as 'module'
>>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>>> - SOLR-14660 Migrating HDFS into a package
>>> - SOLR-12901 Make UnifiedHighlighter the default
>>> - SOLR-15914 Make it super simple to add a contrib module to shared classpath
>>> - And probably some minor ones too
>>>
>>> Each of these are in-flight and are expected to be ready really soon™.
>>>
>>> There are currently 24 blockers, but some of those are Unsassigned and/or have not been given any attention for some time.
>>> On Feb 1st I'll take the freedom to remove blocker flag for those that have not moved anywhere since.
>>> If you want to own one of them, please assign yourself and communicate progress and an ETA.
>>>
>>> Jan
>>>
>>>
>>
>>
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: New branch and feature freeze for Solr 9.0.0

Posted by David Smiley <ds...@apache.org>.
On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <ja...@cominvent.com> wrote:

> Hi, and welcome to March!
>
> Our initial goal of a RC1 within February slipped, but we are still in a
> good position.
> I'll try to summarize the current code blockers:
>
>
> *SOLR-16061  Decouple CloudSolrClient from ZkStateReader*
>
> This is new, a spin-off from SOLR-15342 to prepare for solrj
> modularization. There is already a draft PR. Hope there will be progress on
> this so we don't have to delay solrj modularization until 10.0
>

I'm working with Haythem on this (a colleague).  I think it's close; it's
"just" a refactoring.  The main constraint on this is Haythem's time.


>
> *SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x*
>
> This has not seen any movement despite repeated reminders, so unless there
> is progress within a few days I'll remove it as blocker and add a note to
> the release notes that users relying on running test framework locally
> should wait for a later release.
>

I'm interested in looking but not until I get through the other two.


> *SOLR-14401  "distrib" request handler metrics should only be tracked
> on...*
>
> There is a PR, not sure how close to merge it is though.
>

I think the core of the change is fine but there are downstream changes
needed.  First level is the prometheus exporter configuration to not look
for ".distrib." vs ".local."; it's different now.  Next level is
the Grafana dashboard.  I don't normally play with JQ, Prometheus or
Grafana so it's taking me some time this week.  I'd appreciate any feedback
on the choices here; so far only Houston has weigh'ed in.  I think Tim
Potter, if you're reading this, would be useful given you did major work
here.


>
> Also, David found a new blocker bug yesterday - the "jetty" metrics group
> is missing in 9.x. There will likely be another blocker due to this.
>
>
> Appreciate an update in this thread on the ETA for each of these.
>
> Jan
>
> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <ja...@cominvent.com>:
>
> I created a new blocker
>
> SOLR-16040  Fix split packages in hdfs module
>
> Not sure if it needs to be a blocker though, but we should try to avoid
> split packages as far as we can, and this cannot be done in 9.x.
> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase.
> Jenkins is now mostly green after some turmoil!
>
> *SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can lend
> a hand there?*
>
> We also discussed in SOLR-15342
> <https://issues.apache.org/jira/browse/SOLR-15342> whether refactoring
> CloudSolrClient to untangle ZkStateProvider should be done now, and also
> rename solr-solrj as solr-solrj-all so that we can continue with the solrj
> modularization in 9.x without back-compat breaks.
> It seems worthy of a blocker to me, but we need someone willing to do the
> work in the next few days. Anyone?
>
> I also created SOLR-16041
> <https://issues.apache.org/jira/browse/SOLR-16041> (not blocker) to try
> to setup nightly smoketestRelease Jenkins jobs, I may try to give it a go.
>
> Assuming progress on the above, I'm still hopeful for an RC1 in the
> timeframe of next week.
>
> Jan
>
> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <ja...@cominvent.com>:
>
> These are the three main code-blockers for doing 9.0.0 RC1:
>
> (P) SOLR-15064  Atomic/partial updates to nested docs should not assume
> _route_    dsmiley     dsmiley
> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x   gus         gus
> (S) SOLR-14401  """distrib"" request handler metrics should only be
> tracked on pe  dsmiley     dsmiley
>
> The other blockers are mostly about the release process itself, including
> docker and refguide. I'm doing a clean-up of 9.0  CHANGES too.
> When these are resolved, I'll prepare RC1. That means *we're really close
> *now!!
>
> Anshum is preparing a release notes draft, and we also need to complete
> "Major Changes in 9.0"  and "Upgrade Notes" in ref-guide before publishing
> the guide.
>
> Jan
>
> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <ja...@cominvent.com>:
>
> Congrats on HDFS as a package! Huge win! Also some other blockers have
> been closed recently.
>
> Status on the 9.0 release, one week into February.
>
> - I have done a dry-run of an RC and the smoketester. Think the release
> scripts are ready!
> - 11 open blockers:
>
> (P) SOLR-15587  Replicas end up with base_url as http on client side even
> if clus  thelabdude  thelabdude
> (P) SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>         ctargett    ctargett
> (P) SOLR-15557  Figure out how to handle ref guide page renames/redirects
>          janhoy      ctargett
> (A) SOLR-15064  Atomic/partial updates to nested docs should not assume
> _route_    dsmiley     dsmiley
> (A) SOLR-15949  Use Java 17 in docker
>          dsmiley     dsmiley
> (S) SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
> master/9.x   gus         gus
> (S) SOLR-14401  """distrib"" request handler metrics should only be
> tracked on pe  dsmiley     dsmiley
> (S) SOLR-15321  "Flesh out process for managing/storing ""official""
> Dockerfiles   houston     hossman
> (U) SOLR-15941  Wire Antora refGuide into the release scripts
>                      janhoy
> (U) SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for
> 9.0.0                 janhoy
> (D) SOLR-13138  Remove deprecated code prior to 9.0
>                      romseygeek
>
> Based on a quick look, I assigned a readiness-code for each:
> *(A)*: Active, pending PR *(P)*: PR in progress, *(D)*: Done, can be
> closed? *(S)*: Stalled. *(U)*: Unassigned
>
> Would be great to have a status and ETA for the (S)talled issues. Perhaps
> we can push some to 9.0.1 or 9.1?
>
> Wrt publishing the ref-guide and docker image, those could be done
> semi-manual for 9.0.0 if need be.
>
> Jan
>
> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <co...@gmail.com>:
>
> SOLR-14660  Migrating HDFS into a module
>>     krisden     ichattopadhyaya
>
>
> This was just merged. Big shout out to Istvan Farkas for doing 99% of the
> heavy lifting here!
>
> Kevin Risden
>
>
> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <ja...@cominvent.com> wrote:
>
>> Hi,
>>
>> Nine days ago we had 24 blockers. Several blockers have since been added
>> and several resolved, and today we are at 19.
>>
>> I notified in the previous email that I'd remove inactive blockers. Here
>> is the list of JIRAs for which I intend to remove the blocker flag on
>> February 1st:
>>
>> SOLR-14097  coreRootDirectory should be solr_home/cores
>> SOLR-15064  Atomic/partial updates to nested docs should not assume
>> _route_ param is the root ID
>> SOLR-15242  Consolidate README.md with solr/README.md
>> SOLR-15096  [REGRESSION] Collection Delete Performance significantly
>> degraded in Java 11 v 8
>> SOLR-15223  Deprecate HttpSolrClient, mark httpcomponents dep as
>> "optional" in SolrJ
>> SOLR-15835  Collection creation failing with https
>>
>> Shout out if you are working on one of these and expect it to be finished
>> soon-ish.
>>
>> That leaves 13 blockers:
>>
>> Issue key   Summary
>>      Assignee    Reporter
>> SOLR-14660  Migrating HDFS into a module
>>       krisden     ichattopadhyaya
>> SOLR-15956  Add documentation for creating a docker image from the binary
>> dis  houston     houston
>> SOLR-13138  Remove deprecated code prior to 9.0
>>                  romseygeek
>> SOLR-15556  Ref Guide Redesign Phase 3: Replace Jekyll
>>       ctargett    ctargett
>> SOLR-15949  Use Java 17 in docker
>>      dsmiley     dsmiley
>> SOLR-15926  Fix version specification in the Solr Ref Guide
>>                  houston
>> SOLR-14290  Fix NPE in SolrTestCaseJ4 breaking external usage for
>> master/9.x   gus         gus
>> SOLR-12901  Make UnifiedHighlighter the default
>>      dsmiley     dsmiley
>> SOLR-14401  """distrib"" request handler metrics should only be tracked
>> on pe  dsmiley     dsmiley
>> SOLR-15587  Replicas end up with base_url as http on client side even if
>> clus  thelabdude  thelabdude
>> SOLR-15557  Figure out how to handle ref guide page renames/redirects
>>                  ctargett
>> SOLR-15898  Complete Major changes and Upgrade Notes in RefGudie for
>> 9.0.0                 janhoy
>> SOLR-15321  "Flesh out process for managing/storing ""official""
>> Dockerfiles   houston     hossman
>>
>> It seems like most of these have had some recent activity.
>> Please everyone, have a look if you can lend a hand with any of these, so
>> we can get the list to zero early in February and do the first RC.
>> I'd appreciate some help on SOLR-15898, consolidating and structuring the
>> "Major changes" chapter of the reference guide.
>>
>> Jan
>>
>>
>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <ja...@cominvent.com>:
>>
>> Hi,
>>
>> The list of release blockers can be seen with this JIRA filter:
>> https://issues.apache.org/jira/issues/?filter=12351219
>>
>> After the committer's meeting today we decided to add these to the
>> blockers list:
>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll
>> - SOLR-15917 Rename 'contrib' as 'module'
>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search
>> - SOLR-14660 Migrating HDFS into a package
>> - SOLR-12901 Make UnifiedHighlighter the default
>> - SOLR-15914 Make it super simple to add a contrib module to shared
>> classpath
>> - And probably some minor ones too
>>
>> Each of these are in-flight and are expected to be ready really soon™.
>>
>> There are currently 24 blockers, but some of those are Unsassigned and/or
>> have not been given any attention for some time.
>> On Feb 1st I'll take the freedom to remove blocker flag for those that
>> have not moved anywhere since.
>> If you want to own one of them, please assign yourself and communicate
>> progress and an ETA.
>>
>> Jan
>>
>>
>>
>
>
>
>