You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@netbeans.apache.org by "mbien (via GitHub)" <gi...@apache.org> on 2023/05/24 02:46:19 UTC
[GitHub] [netbeans] mbien opened a new pull request, #5984: use SMO service for class name and SHA1 queries.
mbien opened a new pull request, #5984:
URL: https://github.com/apache/netbeans/pull/5984
service https://github.com/apache/maven-indexer/tree/master/search-backend-smo (search maven org)
- the remote index does not contain class name information anymore
- sha1 queries are rarely used and contribute to index size, which makes them a good candidate for SMO too
how to test:
open/create maven project, type some class name, use the hint to do a maven search. Results should contain both central and local entries. Should work for simple class names and fully qualified class names.
I am not sure how useful the sha1 query actually is, since features like attach src/doc do seem to work even with indexing disabled since it uses GAV. The local repo index does also contain hash information which means that as soon the project is primed, the local index could be used for it.
https://github.com/search?q=repo%3Aapache%2Fnetbeans+findBySHA1&type=code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] matthiasblaesing commented on a diff in pull request #5984: use SMO service for class name and SHA1 queries.
Posted by "matthiasblaesing (via GitHub)" <gi...@apache.org>.
matthiasblaesing commented on code in PR #5984:
URL: https://github.com/apache/netbeans/pull/5984#discussion_r1209504971
##########
java/maven.indexer/src/org/netbeans/modules/maven/indexer/CompositeResult.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.netbeans.modules.maven.indexer;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import org.netbeans.modules.maven.indexer.spi.ResultImplementation;
+
+/**
+ * Merged view over multiple {@link ResultImplementation}s.
+ *
+ * @author mbien
+ */
+final class CompositeResult<T> implements ResultImplementation<T> {
+
+ private final List<ResultImplementation<T>> results;
+
+ public CompositeResult(List<ResultImplementation<T>> results) {
+ this.results = results;
+ }
+
+ public CompositeResult(ResultImplementation<T> first, ResultImplementation<T> second) {
+ this(List.of(first, second));
+ }
+
+ @Override
+ public boolean isPartial() {
+ for (ResultImplementation<T> result : results) {
+ if (result.isPartial()) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public void waitForSkipped() {
+ for (ResultImplementation<T> result : results) {
+ result.waitForSkipped();
+ }
+ }
+
+ @Override
+ public List<T> getResults() {
+ return results.stream()
+ .flatMap(r -> r.getResults().stream())
+ .distinct()
Review Comment:
The results should be sorted. Without that the user will first get the results of the first resultset and then of the second.
```suggestion
.distinct()
.sorted()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] mbien commented on a diff in pull request #5984: use SMO service for class name and SHA1 queries.
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien commented on code in PR #5984:
URL: https://github.com/apache/netbeans/pull/5984#discussion_r1209520835
##########
java/maven.indexer/src/org/netbeans/modules/maven/indexer/CompositeResult.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.netbeans.modules.maven.indexer;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import org.netbeans.modules.maven.indexer.spi.ResultImplementation;
+
+/**
+ * Merged view over multiple {@link ResultImplementation}s.
+ *
+ * @author mbien
+ */
+final class CompositeResult<T> implements ResultImplementation<T> {
+
+ private final List<ResultImplementation<T>> results;
+
+ public CompositeResult(List<ResultImplementation<T>> results) {
+ this.results = results;
+ }
+
+ public CompositeResult(ResultImplementation<T> first, ResultImplementation<T> second) {
+ this(List.of(first, second));
+ }
+
+ @Override
+ public boolean isPartial() {
+ for (ResultImplementation<T> result : results) {
+ if (result.isPartial()) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public void waitForSkipped() {
+ for (ResultImplementation<T> result : results) {
+ result.waitForSkipped();
+ }
+ }
+
+ @Override
+ public List<T> getResults() {
+ return results.stream()
+ .flatMap(r -> r.getResults().stream())
+ .distinct()
Review Comment:
good catch. Done. I moved distinct after sorted since deduplication should be faster on already sorted streams if I remember correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] matthiasblaesing commented on a diff in pull request #5984: use SMO service for class name and SHA1 queries.
Posted by "matthiasblaesing (via GitHub)" <gi...@apache.org>.
matthiasblaesing commented on code in PR #5984:
URL: https://github.com/apache/netbeans/pull/5984#discussion_r1209523811
##########
java/maven.indexer/src/org/netbeans/modules/maven/indexer/CompositeResult.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.netbeans.modules.maven.indexer;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import org.netbeans.modules.maven.indexer.spi.ResultImplementation;
+
+/**
+ * Merged view over multiple {@link ResultImplementation}s.
+ *
+ * @author mbien
+ */
+final class CompositeResult<T> implements ResultImplementation<T> {
+
+ private final List<ResultImplementation<T>> results;
+
+ public CompositeResult(List<ResultImplementation<T>> results) {
+ this.results = results;
+ }
+
+ public CompositeResult(ResultImplementation<T> first, ResultImplementation<T> second) {
+ this(List.of(first, second));
+ }
+
+ @Override
+ public boolean isPartial() {
+ for (ResultImplementation<T> result : results) {
+ if (result.isPartial()) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public void waitForSkipped() {
+ for (ResultImplementation<T> result : results) {
+ result.waitForSkipped();
+ }
+ }
+
+ @Override
+ public List<T> getResults() {
+ return results.stream()
+ .flatMap(r -> r.getResults().stream())
+ .distinct()
Review Comment:
Yeah, makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] mbien commented on pull request #5984: use SMO service for class name and SHA1 queries.
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien commented on PR #5984:
URL: https://github.com/apache/netbeans/pull/5984#issuecomment-1563681809
looks like the search service had a bad day yesterday, the latency is better today. This is the `StringUtils` search repeated from above:
```
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=0}, query=cn:StringUtils} finished in 1,735 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=1}, query=cn:StringUtils} finished in 1,044 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=2}, query=cn:StringUtils} finished in 909 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=3}, query=cn:StringUtils} finished in 567 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=4}, query=cn:StringUtils} finished in 608 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=5}, query=cn:StringUtils} finished in 710 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=6}, query=cn:StringUtils} finished in 935 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=7}, query=cn:StringUtils} finished in 847 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=8}, query=cn:StringUtils} finished in 798 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=9}, query=cn:StringUtils} finished in 498 ms
INFO [org.netbeans.modules.maven.indexer.SMORequestResult]: SMO SearchRequest{paging=Paging{pageSize=128, pageOffset=10}, query=cn:StringUtils} finished in 407 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lbownik commented on a diff in pull request #5984: use SMO service for class name and SHA1 queries.
Posted by "lbownik (via GitHub)" <gi...@apache.org>.
lbownik commented on code in PR #5984:
URL: https://github.com/apache/netbeans/pull/5984#discussion_r1203551404
##########
java/maven.indexer/src/org/netbeans/modules/maven/indexer/NexusRepositoryIndexerImpl.java:
##########
@@ -1196,12 +1206,38 @@ private ResultImplementation<NBVersionInfo> getVersions(final String groupId, fi
@Override
public ResultImplementation<NBVersionInfo> findVersionsByClass(final String className, List<RepositoryInfo> repos) {
- ResultImpl<NBVersionInfo> result = new ResultImpl<>((ResultImpl<NBVersionInfo> result1) -> {
- findVersionsByClass(className, result1, result1.getSkipped(), false);
- });
- return findVersionsByClass(className, result, repos, true);
+
+ Optional<RepositoryInfo> central = repos.stream()
+ .filter(repo -> repo.getId().equals(smo.getRepositoryId()))
+ .findFirst();
+
+ // remote index contains no class data -> use web service
+ if (central.isPresent()) {
+ List<RepositoryInfo> otherRepos = new ArrayList<>(repos);
+ otherRepos.remove(central.get());
+
+ SearchRequest request = new SearchRequest(new Paging(64),
+ FieldQuery.fieldQuery(className.contains(".") ?
+ org.apache.maven.search.MAVEN.FQ_CLASS_NAME
+ : org.apache.maven.search.MAVEN.CLASS_NAME, className));
+
+ return mergeResults(
+ () -> (ResultImpl) findVersionsByClass(className, otherRepos),
+ () -> {
Review Comment:
IMHO making this lambda a method would make this code more readable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] mbien merged pull request #5984: use SMO service for class name and SHA1 queries.
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien merged PR #5984:
URL: https://github.com/apache/netbeans/pull/5984
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists