You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by GitBox <gi...@apache.org> on 2022/02/01 20:22:11 UTC

[GitHub] [solr] markrmiller opened a new pull request #585: SOLR-15955 Update Jetty dependency to 10

markrmiller opened a new pull request #585:
URL: https://github.com/apache/solr/pull/585


   My local .git folder got mangled somehow so I'm transferring changes to a fresh checkout - still a few things that are off and I'm working on syncing up. Have a couple nits left to work out as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #585:
URL: https://github.com/apache/solr/pull/585#issuecomment-1045911960


   Thanks for taking a look. I have a n update I'll push I'll push tomorrow. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819239275



##########
File path: gradle/solr/force-versions.gradle
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+// Force slf4j version so that it does not get auto promoted from 1x to 2-alpha due to Jetty dependency.
+
+project(":solr").subprojects {
+  plugins.withType(JavaPlugin) {
+    dependencies {
+      implementation enforcedPlatform('org.slf4j:slf4j-api:1.7.35')

Review comment:
       I think this was done this way because you cannot easily do that here, or I assume Dawid would have, but I'll see what happens.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r810457300



##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -320,17 +340,113 @@ protected URI getBaseUri() throws URISyntaxException {
   }
 
   private void pushObject(String path, String content) {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       s3.putObject(b -> b.bucket(BUCKET_NAME).key(path), RequestBody.fromString(content));
     }
   }
 
   private File pullObject(String path) throws IOException {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       File file = temporaryFolder.newFile();
       InputStream input = s3.getObject(b -> b.bucket(BUCKET_NAME).key(path));
       FileUtils.copyInputStreamToFile(input, file);
       return file;
     }
   }
+
+  public S3Client createS3ClientV2() {

Review comment:
       This should not be a change in this issue. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r810558736



##########
File path: gradle/testing/randomization/policies/solr-tests.policy
##########
@@ -38,6 +38,13 @@ grant {
   // TestConfigSets messes with these (wtf?)
   permission java.io.FilePermission "/path/to/solr/home/lib", "read";
 
+  // hdfs name node uses jetty, ends up with a JarResource in AllowedResourceAliasChecker#doStart
+  // which ends up needing to be able to read '/', see HttpServer2 - there is a comment I saw that
+  // indicated there may be a way to supply a custom AliasChecker
+  permission java.io.FilePermission "/", "read";
+
+  permission java.io.FilePermission "/tmp/inst1/conf/solrcore.properties", "read";

Review comment:
       Isn't this the same as line 36?

##########
File path: solr/modules/hdfs/src/test/org/apache/solr/cloud/hdfs/HdfsTestUtil.java
##########
@@ -176,11 +176,19 @@ public static MiniDFSCluster setupClass(String dir, boolean safeModeTesting, boo
 
     Configuration conf = getBasicConfiguration(new Configuration());
     conf.set("hdfs.minidfs.basedir", dir + File.separator + "hdfsBaseDir");
+    String hdfsBaseDir = dir + File.separator + "hdfsBaseDir";
+    conf.set("hdfs.minidfs.basedir.provided", hdfsBaseDir);

Review comment:
       can we also `conf.set("dfs.webhdfs.enabled", "false");`
   

##########
File path: gradle/testing/randomization/policies/solr-tests.policy
##########
@@ -38,6 +38,13 @@ grant {
   // TestConfigSets messes with these (wtf?)
   permission java.io.FilePermission "/path/to/solr/home/lib", "read";
 
+  // hdfs name node uses jetty, ends up with a JarResource in AllowedResourceAliasChecker#doStart
+  // which ends up needing to be able to read '/', see HttpServer2 - there is a comment I saw that
+  // indicated there may be a way to supply a custom AliasChecker
+  permission java.io.FilePermission "/", "read";

Review comment:
       I think this is fixed in Jetty 10.0.8 via https://github.com/eclipse/jetty.project/pull/7076




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #585:
URL: https://github.com/apache/solr/pull/585#issuecomment-1068380050


   > I don't think we need this - tried running tests locally without it and using S3_MOCK_RULE to build the client directly and things seemed fine.
   
   I actually took things out one by one from this class on the last update, and I left this because the tests passed as I removed up to this point and then failed when I removed this. But I tried again, and it seems to have passed, so perhaps some stale result issue I saw. Looks like it can come out if we go by your result and my latest.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r821894872



##########
File path: solr/core/build.gradle
##########
@@ -90,6 +90,7 @@ dependencies {
   implementation 'org.apache.httpcomponents:httpcore'
 
   compileOnlyApi 'javax.servlet:javax.servlet-api'
+  compileOnlyApi 'org.eclipse.jetty.toolchain:jetty-servlet-api'

Review comment:
       I think having both of these is incorrect? From a visual inspection it looks like jetty-servlet-api has everything that java.servlet-api has, and more. They definitely both end up in `server/lib` not sure if that's from here or from somewhere else.

##########
File path: solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
##########
@@ -68,8 +69,12 @@ public static void processParams(SolrRequestHandler handler, SolrQueryRequest re
       String[] jsonFromParams = map.remove(JSON);  // params from the query string should come after (and hence override) JSON content streams
 
       for (ContentStream cs : req.getContentStreams()) {
+        // if BinaryResponseParser.BINARY_CONTENT_TYPE, let the following fail below - we may have adjusted the content without updating the content type
+        // problem in this case happens in a few tests, one seems to happen with kerberos and remote node query (HttpSolrCall's request proxy)

Review comment:
       It took a bit of digging, but I finally figured it out. When Jetty receives a request with no body content, it creates a placeholder zero-length application/octet-stream content object. Then when it gets a redirect (like an explicit redirect handler, or some authentication plugins cause), it copies that body to the new request. When the new request gets to our search handlers, we see the body and save it as a stream. So I think that we could check for that in `SolrRequestParsers.parseParamsAndFillStreams` - the combination of a GET request with a zero length octet-stream body seems like a safe instance to drop the stream. Maybe we could drop all the stream bodies for all get requests, I'm not sure. This could be a bug in Jetty, but I haven't figured out a simple repro case for them yet.

##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -320,17 +340,113 @@ protected URI getBaseUri() throws URISyntaxException {
   }
 
   private void pushObject(String path, String content) {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       s3.putObject(b -> b.bucket(BUCKET_NAME).key(path), RequestBody.fromString(content));
     }
   }
 
   private File pullObject(String path) throws IOException {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       File file = temporaryFolder.newFile();
       InputStream input = s3.getObject(b -> b.bucket(BUCKET_NAME).key(path));
       FileUtils.copyInputStreamToFile(input, file);
       return file;
     }
   }
+
+  public S3Client createS3ClientV2() {

Review comment:
       I don't think we need this - tried running tests locally without it and using S3_MOCK_RULE to build the client directly and things seemed fine.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r827341958



##########
File path: solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
##########
@@ -68,8 +69,12 @@ public static void processParams(SolrRequestHandler handler, SolrQueryRequest re
       String[] jsonFromParams = map.remove(JSON);  // params from the query string should come after (and hence override) JSON content streams
 
       for (ContentStream cs : req.getContentStreams()) {
+        // if BinaryResponseParser.BINARY_CONTENT_TYPE, let the following fail below - we may have adjusted the content without updating the content type
+        // problem in this case happens in a few tests, one seems to happen with kerberos and remote node query (HttpSolrCall's request proxy)

Review comment:
       Good find, and that does jive with what was happening when I found the fail I saw here pop up on the mailing list in the past.
   
   So I think the diagnosis makes sense, and the change sounds reasonable high level, but I am concerned that it requires that you can inspect the stream or count on the content-size header to properly describe the stream. You cannot generally do that though. The size is not required, usually not given in Solr, and you can't reasonable inspect a stream in a webapp/servlet unless you buffer it, which is also problematic. Given we are talking about a steram on a GET request, maybe somehow you can get away with this, but it seems pretty fragile?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819239772



##########
File path: solr/modules/hadoop-auth/src/test/org/apache/solr/client/solrj/impl/Krb5HttpClientUtils.java
##########
@@ -36,10 +36,11 @@
   public static void setup(Http2SolrClient http2Client, String principalName) {
     HttpAuthenticationStore authenticationStore = new HttpAuthenticationStore();
     authenticationStore.addAuthentication(createSPNEGOAuthentication(principalName));
-    http2Client.getHttpClient().setAuthenticationStore(authenticationStore);
+    http2Client.setAuthenticationStore(authenticationStore);
     http2Client
         .getProtocolHandlers()
         .put(new WWWAuthenticationProtocolHandler(http2Client.getHttpClient()));
+    http2Client.setAuthenticationStore(authenticationStore);

Review comment:
       came in during rebase reconciliation I think, back to one




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r807527032



##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -246,7 +266,7 @@ private void doRandomAccessTest(String content, int position) throws Exception {
 
     try (S3BackupRepository repo = getRepository()) {
       // Open an index input on a file
-      pushObject("/my-repo/content", content);
+      pushObject("my-repo/content", content);

Review comment:
       @HoustonPutman I recall you fighting with S3 and Mock-S3 and how it handled directories...

##########
File path: gradle/ant-compat/force-versions.gradle
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+// Force versions of certain components to align them with ant build.

Review comment:
       Can you add more comments explaining why we need this, since earlier we felt like it would have been safe to remove?

##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3ReadWriteTest.java
##########
@@ -91,7 +91,7 @@ public void testDirectoryLength() throws Exception {
             "Getting length on a dir should throw exception",
             S3Exception.class,
             () -> client.length("/directory"));
-    assertEquals("Path is Directory", exception.getMessage());
+    assertTrue(exception.getMessage(), exception.getMessage().contains("Path is Directory"));

Review comment:
       nit: `assertThat` would get us a nice error message for free

##########
File path: solr/solrj/src/java/org/apache/solr/client/solrj/util/SolrBasicAuthentication.java
##########
@@ -35,7 +35,7 @@
   private final String value;
 
   public SolrBasicAuthentication(String user, String password) {
-    this.value = "Basic " + B64Code.encode(user + ":" + password, StandardCharsets.ISO_8859_1);
+    this.value = "Basic " + Base64.getEncoder().encodeToString((user + ":" + password + "").getBytes(StandardCharsets.ISO_8859_1));

Review comment:
       nit: extra `+ ""` in here

##########
File path: solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
##########
@@ -68,8 +69,12 @@ public static void processParams(SolrRequestHandler handler, SolrQueryRequest re
       String[] jsonFromParams = map.remove(JSON);  // params from the query string should come after (and hence override) JSON content streams
 
       for (ContentStream cs : req.getContentStreams()) {
+        // if BinaryResponseParser.BINARY_CONTENT_TYPE, let the following fail below - we may have adjusted the content without updating the content type
+        // problem in this case happens in a few tests, one seems to happen with kerberos and remote node query (HttpSolrCall's request proxy)
+
         String contentType = cs.getContentType();
-        if (contentType==null || !contentType.contains("/json")) {
+        if (contentType==null || (!contentType.contains("/json") && !contentType.contains(
+            BinaryResponseParser.BINARY_CONTENT_TYPE))) {

Review comment:
       I saw that you switched a bunch of places away from BINARY_CONTENT_TYPE to multi-part -- does Jetty 10 not like octet-stream? What's the motivation?

##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -320,17 +340,113 @@ protected URI getBaseUri() throws URISyntaxException {
   }
 
   private void pushObject(String path, String content) {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       s3.putObject(b -> b.bucket(BUCKET_NAME).key(path), RequestBody.fromString(content));
     }
   }
 
   private File pullObject(String path) throws IOException {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       File file = temporaryFolder.newFile();
       InputStream input = s3.getObject(b -> b.bucket(BUCKET_NAME).key(path));
       FileUtils.copyInputStreamToFile(input, file);
       return file;
     }
   }
+
+  public S3Client createS3ClientV2() {
+    return S3Client.builder()
+        .region(Region.of("us-east-1"))
+        .credentialsProvider(
+            StaticCredentialsProvider.create(AwsBasicCredentials.create("foo", "bar")))
+        .endpointOverride(URI.create(S3_MOCK_RULE.getServiceEndpoint()))
+        .httpClient(
+            UrlConnectionHttpClient.builder()
+                .buildWithDefaults(
+                    AttributeMap.builder()
+                        .put(
+                            new HttpConfigurationOption("secureConnection", Boolean.class),
+                            Boolean.FALSE)
+                        .build()))
+        .build();
+  }
+
+  public ClientConfiguration configureClientToIgnoreInvalidSslCertificates(
+      final ClientConfiguration clientConfiguration) {
+
+    clientConfiguration
+        .getApacheHttpClientConfig()
+        .withSslSocketFactory(
+            new SSLConnectionSocketFactory(
+                createBlindlyTrustingSslContext(), NoopHostnameVerifier.INSTANCE));
+
+    return clientConfiguration;
+  }
+
+  private SSLContext createBlindlyTrustingSslContext() {
+    try {
+      final SSLContext sc = SSLContext.getInstance("TLS");
+
+      sc.init(
+          null,
+          new TrustManager[] {
+            new X509ExtendedTrustManager() {
+              @Override
+              public java.security.cert.X509Certificate[] getAcceptedIssuers() {
+                return null;
+              }
+
+              @Override
+              public void checkClientTrusted(
+                  final X509Certificate[] arg0, final String arg1, final Socket arg2) {
+                // no-op
+              }
+
+              @Override
+              public void checkClientTrusted(
+                  final X509Certificate[] arg0, final String arg1, final SSLEngine arg2) {
+                // no-op
+              }
+
+              @Override
+              public void checkClientTrusted(final X509Certificate[] certs, final String authType) {
+                // no-op
+              }
+
+              @Override
+              public void checkServerTrusted(final X509Certificate[] certs, final String authType) {
+                // no-op
+              }
+
+              @Override
+              public void checkServerTrusted(
+                  final X509Certificate[] arg0, final String arg1, final Socket arg2) {
+                // no-op
+              }
+
+              @Override
+              public void checkServerTrusted(
+                  final X509Certificate[] arg0, final String arg1, final SSLEngine arg2) {
+                // no-op
+              }
+            }
+          },
+          new java.security.SecureRandom());

Review comment:
       Should we use NotSecurePseudoRandom here?

##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -320,17 +340,113 @@ protected URI getBaseUri() throws URISyntaxException {
   }
 
   private void pushObject(String path, String content) {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       s3.putObject(b -> b.bucket(BUCKET_NAME).key(path), RequestBody.fromString(content));
     }
   }
 
   private File pullObject(String path) throws IOException {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       File file = temporaryFolder.newFile();
       InputStream input = s3.getObject(b -> b.bucket(BUCKET_NAME).key(path));
       FileUtils.copyInputStreamToFile(input, file);
       return file;
     }
   }
+
+  public S3Client createS3ClientV2() {

Review comment:
       Is this copied directly from adobe's S3MockStarter? If so, I'd like to see it in a separate file that we can properly attribute (and keep up to date periodically as needed)

##########
File path: solr/test-framework/src/java/org/apache/solr/cloud/api/collections/AbstractBackupRepositoryTest.java
##########
@@ -103,6 +104,11 @@ public void testCanDistinguishBetweenFilesAndDirectories() throws Exception {
             repo.createDirectory(nonEmptyDirUri);
             addFile(repo, nestedFileUri);
 
+            String[] list = repo.listAll(repo.resolve(nonEmptyDirUri));

Review comment:
       left over from debugging?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r811482790



##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -246,7 +266,7 @@ private void doRandomAccessTest(String content, int position) throws Exception {
 
     try (S3BackupRepository repo = getRepository()) {
       // Open an index input on a file
-      pushObject("/my-repo/content", content);
+      pushObject("my-repo/content", content);

Review comment:
       something changed with the version change of the mock lib




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r811479886



##########
File path: solr/test-framework/src/java/org/apache/solr/cloud/api/collections/AbstractBackupRepositoryTest.java
##########
@@ -103,6 +104,11 @@ public void testCanDistinguishBetweenFilesAndDirectories() throws Exception {
             repo.createDirectory(nonEmptyDirUri);
             addFile(repo, nestedFileUri);
 
+            String[] list = repo.listAll(repo.resolve(nonEmptyDirUri));

Review comment:
       yup




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #585:
URL: https://github.com/apache/solr/pull/585#issuecomment-1046519892


   A couple of those are extra lines that came in when I merged to a clean repo (test policy and s3 build.gradle I believe).
   
   I'll get the follow up pushed tomorrow - I rebased yesterday and after resolving new conflicts, tests were flakier (a handful failed first run) and one test failed consistently (managed storage test I believe). So I held back to look into that a bit, but in the in end, it seems I see the same behavior on a clean checkout, so should not be related this work.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r811482515



##########
File path: .github/workflows/manual.yml
##########
@@ -0,0 +1,21 @@
+name: Scheduled Merge Remote Action

Review comment:
       strange, don't even see it locally




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819243244



##########
File path: gradle/solr/force-versions.gradle
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+// Force slf4j version so that it does not get auto promoted from 1x to 2-alpha due to Jetty dependency.
+
+project(":solr").subprojects {
+  plugins.withType(JavaPlugin) {
+    dependencies {
+      implementation enforcedPlatform('org.slf4j:slf4j-api:1.7.35')

Review comment:
       Think it actually works.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r827346145



##########
File path: solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
##########
@@ -68,8 +69,12 @@ public static void processParams(SolrRequestHandler handler, SolrQueryRequest re
       String[] jsonFromParams = map.remove(JSON);  // params from the query string should come after (and hence override) JSON content streams
 
       for (ContentStream cs : req.getContentStreams()) {
+        // if BinaryResponseParser.BINARY_CONTENT_TYPE, let the following fail below - we may have adjusted the content without updating the content type
+        // problem in this case happens in a few tests, one seems to happen with kerberos and remote node query (HttpSolrCall's request proxy)

Review comment:
       RawRequestParser.parseParamsAndFillStreams will proactively save the stream if there is a content length `>= 0` - that should probably be strictly `> 0`. Because then in the else block, if it's not a GET we try to buffer the first byte - if it exists, save the stream again and if not then we don't need it. I think we need to fix RawRequestParser with this PR to accommodate the change that Jetty made - it's likely a regression but I couldn't find an existing bug report or an easy reproducer for them.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r827511207



##########
File path: solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
##########
@@ -68,8 +69,12 @@ public static void processParams(SolrRequestHandler handler, SolrQueryRequest re
       String[] jsonFromParams = map.remove(JSON);  // params from the query string should come after (and hence override) JSON content streams
 
       for (ContentStream cs : req.getContentStreams()) {
+        // if BinaryResponseParser.BINARY_CONTENT_TYPE, let the following fail below - we may have adjusted the content without updating the content type
+        // problem in this case happens in a few tests, one seems to happen with kerberos and remote node query (HttpSolrCall's request proxy)

Review comment:
       Hmmm, it shouldn't save the steam content I hope, that would prevent actual streaming. But in either case, that does remind me: I think there is a tiny little buffer that is used for content type auto detect in RequestParsers...we would know if a stream is empty or not there. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819198076



##########
File path: gradle/solr/force-versions.gradle
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+// Force slf4j version so that it does not get auto promoted from 1x to 2-alpha due to Jetty dependency.
+
+project(":solr").subprojects {
+  plugins.withType(JavaPlugin) {
+    dependencies {
+      implementation enforcedPlatform('org.slf4j:slf4j-api:1.7.35')

Review comment:
       Should we set this version in version.props like the rest of our libraries?

##########
File path: versions.lock
##########
@@ -259,9 +258,9 @@ org.ow2.asm:asm:9.2 (4 constraints: 4931393a)
 org.ow2.asm:asm-commons:7.2 (1 constraints: 6b0f7267)
 org.quicktheories:quicktheories:0.26 (1 constraints: dc04f530)
 org.reactivestreams:reactive-streams:1.0.3 (3 constraints: 3c2b02fd)
-org.slf4j:jcl-over-slf4j:1.7.36 (3 constraints: 05188eb8)
-org.slf4j:jul-to-slf4j:1.7.36 (2 constraints: 1723911a)
-org.slf4j:slf4j-api:1.7.36 (38 constraints: 9dd389ac)
+org.slf4j:jcl-over-slf4j:1.7.35 (3 constraints: 05188eb8)
+org.slf4j:jul-to-slf4j:1.7.35 (2 constraints: 1723911a)
+org.slf4j:slf4j-api:1.7.35 (57 constraints: a71af872)

Review comment:
       Downgrade?

##########
File path: solr/modules/hadoop-auth/src/test/org/apache/solr/client/solrj/impl/Krb5HttpClientUtils.java
##########
@@ -36,10 +36,11 @@
   public static void setup(Http2SolrClient http2Client, String principalName) {
     HttpAuthenticationStore authenticationStore = new HttpAuthenticationStore();
     authenticationStore.addAuthentication(createSPNEGOAuthentication(principalName));
-    http2Client.getHttpClient().setAuthenticationStore(authenticationStore);
+    http2Client.setAuthenticationStore(authenticationStore);
     http2Client
         .getProtocolHandlers()
         .put(new WWWAuthenticationProtocolHandler(http2Client.getHttpClient()));
+    http2Client.setAuthenticationStore(authenticationStore);

Review comment:
       We call this twice?

##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -41,21 +41,25 @@ dependencies {
   runtimeOnly(group: 'com.fasterxml.woodstox', name: 'woodstox-core')
   runtimeOnly(group: 'org.codehaus.woodstox', name: 'stax2-api')
 
-  testImplementation project(':solr:test-framework')
-  testImplementation 'org.apache.lucene:lucene-test-framework'
-  testImplementation 'com.carrotsearch.randomizedtesting:randomizedtesting-runner'
   testImplementation 'junit:junit'
+  testImplementation 'com.carrotsearch.randomizedtesting:randomizedtesting-runner'
+  testImplementation 'org.apache.lucene:lucene-test-framework'
+  testImplementation project(':solr:test-framework')
+
+  // overrides spring-boot pulled by com.adobe.testing:s3mock-junit4 to allow using a version
+  // compatible with recent Jetty
+  testImplementation 'org.springframework.boot:spring-boot'
 
   testImplementation('com.adobe.testing:s3mock-junit4') {
     // Don't pull in separate versions of these libs, just use what Solr already has
     exclude group: 'org.apache.logging.log4j', module: 'log4j-to-slf4j'
     exclude group: 'ch.qos.logback', module: 'logback-classic'
     exclude group: 'org.eclipse.jetty', module: '*'
     exclude group: 'io.netty', module: '*'
-    exclude group: 'commons-io', module: 'commons-io'
+    exclude group: 'software.amazon.awssdk', module: 'netty-nio-client'
     exclude group: 'commons-codec', module: 'commons-codec'
+    exclude group: 'commons-io', module: 'commons-io'
     exclude group: 'org.apache.commons', module: 'commons-lang3'
-    exclude group: 'software.amazon.awssdk', module: 'netty-nio-client'

Review comment:
       The rearranging here looks weird




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r811513061



##########
File path: gradle/testing/randomization/policies/solr-tests.policy
##########
@@ -38,6 +38,13 @@ grant {
   // TestConfigSets messes with these (wtf?)
   permission java.io.FilePermission "/path/to/solr/home/lib", "read";
 
+  // hdfs name node uses jetty, ends up with a JarResource in AllowedResourceAliasChecker#doStart
+  // which ends up needing to be able to read '/', see HttpServer2 - there is a comment I saw that
+  // indicated there may be a way to supply a custom AliasChecker
+  permission java.io.FilePermission "/", "read";

Review comment:
       10.0.8 breaks the S3 Spring dependency again. I'll see if a different version works out.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r811482134



##########
File path: solr/modules/hdfs/src/test/org/apache/solr/cloud/hdfs/HdfsTestUtil.java
##########
@@ -208,6 +216,32 @@ public static MiniDFSCluster setupClass(String dir, boolean safeModeTesting, boo
 
     MiniDFSCluster dfsCluster = dfsClusterBuilder.build();
     HdfsUtil.TEST_CONF = getClientConfiguration(dfsCluster);
+    
+    // nocommit TODO remove

Review comment:
       unrelated tweaks for hdfs tests since I was running them a fair bit when trying for jetty 11




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r810457269



##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -320,17 +340,113 @@ protected URI getBaseUri() throws URISyntaxException {
   }
 
   private void pushObject(String path, String content) {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       s3.putObject(b -> b.bucket(BUCKET_NAME).key(path), RequestBody.fromString(content));
     }
   }
 
   private File pullObject(String path) throws IOException {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       File file = temporaryFolder.newFile();
       InputStream input = s3.getObject(b -> b.bucket(BUCKET_NAME).key(path));
       FileUtils.copyInputStreamToFile(input, file);
       return file;
     }
   }
+
+  public S3Client createS3ClientV2() {
+    return S3Client.builder()
+        .region(Region.of("us-east-1"))
+        .credentialsProvider(
+            StaticCredentialsProvider.create(AwsBasicCredentials.create("foo", "bar")))
+        .endpointOverride(URI.create(S3_MOCK_RULE.getServiceEndpoint()))
+        .httpClient(
+            UrlConnectionHttpClient.builder()
+                .buildWithDefaults(
+                    AttributeMap.builder()
+                        .put(
+                            new HttpConfigurationOption("secureConnection", Boolean.class),
+                            Boolean.FALSE)
+                        .build()))
+        .build();
+  }
+
+  public ClientConfiguration configureClientToIgnoreInvalidSslCertificates(
+      final ClientConfiguration clientConfiguration) {
+
+    clientConfiguration
+        .getApacheHttpClientConfig()
+        .withSslSocketFactory(
+            new SSLConnectionSocketFactory(
+                createBlindlyTrustingSslContext(), NoopHostnameVerifier.INSTANCE));
+
+    return clientConfiguration;
+  }
+
+  private SSLContext createBlindlyTrustingSslContext() {
+    try {
+      final SSLContext sc = SSLContext.getInstance("TLS");
+
+      sc.init(
+          null,
+          new TrustManager[] {
+            new X509ExtendedTrustManager() {
+              @Override
+              public java.security.cert.X509Certificate[] getAcceptedIssuers() {
+                return null;
+              }
+
+              @Override
+              public void checkClientTrusted(
+                  final X509Certificate[] arg0, final String arg1, final Socket arg2) {
+                // no-op
+              }
+
+              @Override
+              public void checkClientTrusted(
+                  final X509Certificate[] arg0, final String arg1, final SSLEngine arg2) {
+                // no-op
+              }
+
+              @Override
+              public void checkClientTrusted(final X509Certificate[] certs, final String authType) {
+                // no-op
+              }
+
+              @Override
+              public void checkServerTrusted(final X509Certificate[] certs, final String authType) {
+                // no-op
+              }
+
+              @Override
+              public void checkServerTrusted(
+                  final X509Certificate[] arg0, final String arg1, final Socket arg2) {
+                // no-op
+              }
+
+              @Override
+              public void checkServerTrusted(
+                  final X509Certificate[] arg0, final String arg1, final SSLEngine arg2) {
+                // no-op
+              }
+            }
+          },
+          new java.security.SecureRandom());

Review comment:
       Maybe, but unrelated to the jetty changes. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819240165



##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -41,21 +41,25 @@ dependencies {
   runtimeOnly(group: 'com.fasterxml.woodstox', name: 'woodstox-core')
   runtimeOnly(group: 'org.codehaus.woodstox', name: 'stax2-api')
 
-  testImplementation project(':solr:test-framework')
-  testImplementation 'org.apache.lucene:lucene-test-framework'
-  testImplementation 'com.carrotsearch.randomizedtesting:randomizedtesting-runner'
   testImplementation 'junit:junit'
+  testImplementation 'com.carrotsearch.randomizedtesting:randomizedtesting-runner'
+  testImplementation 'org.apache.lucene:lucene-test-framework'
+  testImplementation project(':solr:test-framework')
+
+  // overrides spring-boot pulled by com.adobe.testing:s3mock-junit4 to allow using a version
+  // compatible with recent Jetty
+  testImplementation 'org.springframework.boot:spring-boot'
 
   testImplementation('com.adobe.testing:s3mock-junit4') {
     // Don't pull in separate versions of these libs, just use what Solr already has
     exclude group: 'org.apache.logging.log4j', module: 'log4j-to-slf4j'
     exclude group: 'ch.qos.logback', module: 'logback-classic'
     exclude group: 'org.eclipse.jetty', module: '*'
     exclude group: 'io.netty', module: '*'
-    exclude group: 'commons-io', module: 'commons-io'
+    exclude group: 'software.amazon.awssdk', module: 'netty-nio-client'
     exclude group: 'commons-codec', module: 'commons-codec'
+    exclude group: 'commons-io', module: 'commons-io'
     exclude group: 'org.apache.commons', module: 'commons-lang3'
-    exclude group: 'software.amazon.awssdk', module: 'netty-nio-client'

Review comment:
       Not really sure why that is there, probably conflict merging over time and changes.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #585:
URL: https://github.com/apache/solr/pull/585#issuecomment-1058725898


   That was a bit more effort than expected, but updated jetty from 10.0.7 to 10.0.8. That removes the need for the security policy read permission for "/".
   
   Rebased to latest main, let's see if these build checks pass, did a couple test runs without issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819245105



##########
File path: solr/modules/s3-repository/src/test/org/apache/solr/s3/S3BackupRepositoryTest.java
##########
@@ -320,17 +340,113 @@ protected URI getBaseUri() throws URISyntaxException {
   }
 
   private void pushObject(String path, String content) {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       s3.putObject(b -> b.bucket(BUCKET_NAME).key(path), RequestBody.fromString(content));
     }
   }
 
   private File pullObject(String path) throws IOException {
-    try (S3Client s3 = S3_MOCK_RULE.createS3ClientV2()) {
+    try (S3Client s3 = createS3ClientV2()) {
       File file = temporaryFolder.newFile();
       InputStream input = s3.getObject(b -> b.bucket(BUCKET_NAME).key(path));
       FileUtils.copyInputStreamToFile(input, file);
       return file;
     }
   }
+
+  public S3Client createS3ClientV2() {

Review comment:
       Actually, I'm going to have check if that is still necessary or not. Was working on making some different version combo work properly.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r826207829



##########
File path: solr/core/build.gradle
##########
@@ -90,6 +90,7 @@ dependencies {
   implementation 'org.apache.httpcomponents:httpcore'
 
   compileOnlyApi 'javax.servlet:javax.servlet-api'
+  compileOnlyApi 'org.eclipse.jetty.toolchain:jetty-servlet-api'

Review comment:
       > I don't think we need this 
   
   I tried without it and it failed with a client/server using SSL mismatch before putting this iteration up. I can take one more look at it.
   
   > I think having both of these is incorrect? 
   
   I never actually added that myself, it did exist when I started, it was removed in a rebase I did, and then, and if I remove it now, analyzeClassesDependencies fails for me with (I'm digging around a bit, not sure why it existed, or why it could be removed but then not in this issue):
   
   
   FAILURE: Build failed with an exception.
   
   * What went wrong:
   Execution failed for task ':solr:modules:jwt-auth:analyzeClassesDependencies'.
   > Dependency analysis found issues.
     usedUndeclaredArtifacts
      - org.eclipse.jetty.toolchain:jetty-servlet-api:4.0.6@jar
   
   
   Execution failed for task ':solr:modules:jwt-auth:analyzeClassesDependencies'.
   > Dependency analysis found issues.
     usedUndeclaredArtifacts
      - org.eclipse.jetty.toolchain:jetty-servlet-api:4.0.6@jar
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #585:
URL: https://github.com/apache/solr/pull/585#issuecomment-1074584006


   Okay, I had missed in the patch you passed me how you proposed dealing with the toolchain dep issue - just using that instead of servlet-api - that seems fine to me.
   
   I spent a bit of time on the 0 byte inputstream issue, but in the end, I think your approach is likely fine to start with - given that it works in the case that we currently know causes an issue, and if it stopped working for that case, tests would fail, I've come to think this solution is fine until we find it not to be.
   
   This cleanup push that just went up includes a couple required changes to jetty config that I had forgotten are required and introduce the biggest upgrade issues a user would face - the existing config will not work. There is a fix required in the main jetty xml config where the rewrite handler is referenced by id instead of refid - that fails on jetty 10. And there is gzip config that references methods that are now removed and for the most part without replacement. They have to be changed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r810457205



##########
File path: solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
##########
@@ -68,8 +69,12 @@ public static void processParams(SolrRequestHandler handler, SolrQueryRequest re
       String[] jsonFromParams = map.remove(JSON);  // params from the query string should come after (and hence override) JSON content streams
 
       for (ContentStream cs : req.getContentStreams()) {
+        // if BinaryResponseParser.BINARY_CONTENT_TYPE, let the following fail below - we may have adjusted the content without updating the content type
+        // problem in this case happens in a few tests, one seems to happen with kerberos and remote node query (HttpSolrCall's request proxy)
+
         String contentType = cs.getContentType();
-        if (contentType==null || !contentType.contains("/json")) {
+        if (contentType==null || (!contentType.contains("/json") && !contentType.contains(
+            BinaryResponseParser.BINARY_CONTENT_TYPE))) {

Review comment:
       3 or 4 tests fail due to the search handler and it's content header check. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r810457581



##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -39,6 +39,10 @@ dependencies {
     runtimeOnly (group: 'com.fasterxml.woodstox', name: 'woodstox-core')
     runtimeOnly (group: 'org.codehaus.woodstox', name: 'stax2-api')
 
+    implementation 'org.springframework.boot:spring-boot'

Review comment:
       This was already pulled in, I had to control certain versions. A few changes since, I'll see what's still required. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r803126032



##########
File path: solr/core/src/java/org/apache/solr/servlet/SolrRequestParsers.java
##########
@@ -631,6 +630,11 @@ public InputStream getStream() throws IOException {
     }
   }
 
+  static boolean isMultipart(HttpServletRequest req) {

Review comment:
       This is repeated?

##########
File path: solr/modules/hdfs/src/test/org/apache/solr/cloud/hdfs/HdfsTestUtil.java
##########
@@ -208,6 +216,32 @@ public static MiniDFSCluster setupClass(String dir, boolean safeModeTesting, boo
 
     MiniDFSCluster dfsCluster = dfsClusterBuilder.build();
     HdfsUtil.TEST_CONF = getClientConfiguration(dfsCluster);
+    
+    // nocommit TODO remove

Review comment:
       ?

##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -39,6 +39,10 @@ dependencies {
     runtimeOnly (group: 'com.fasterxml.woodstox', name: 'woodstox-core')
     runtimeOnly (group: 'org.codehaus.woodstox', name: 'stax2-api')
 
+    implementation 'org.springframework.boot:spring-boot'

Review comment:
       This seems likely excessive?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r810560965



##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -39,6 +39,10 @@ dependencies {
     runtimeOnly (group: 'com.fasterxml.woodstox', name: 'woodstox-core')
     runtimeOnly (group: 'org.codehaus.woodstox', name: 'stax2-api')
 
+    implementation 'org.springframework.boot:spring-boot'

Review comment:
       That is to say, what's wrong with using the existing s3mock dependency resolution




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r811482212



##########
File path: .github/workflows/manual.yml
##########
@@ -0,0 +1,21 @@
+name: Scheduled Merge Remote Action

Review comment:
       unintended




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] sonatype-lift[bot] commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r803155178



##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -39,6 +39,10 @@ dependencies {
     runtimeOnly (group: 'com.fasterxml.woodstox', name: 'woodstox-core')
     runtimeOnly (group: 'org.codehaus.woodstox', name: 'stax2-api')
 
+    implementation 'org.springframework.boot:spring-boot'
+    implementation 'org.springframework:spring-core'

Review comment:
       *Severe OSS Vulnerability:*
   ### pkg:maven/org.springframework/spring-core@5.3.9
   0 Critical, 1 Severe, 0 Moderate, 0 Unknown vulnerabilities have been found across 1 dependencies
   
   <details>
     <summary><b>Components</b></summary><br/>
     <ul>
         <details>
           <summary><b>pkg:maven/org.springframework/spring-core@5.3.9</b></summary>
           <ul>
     <details>
       <summary><b>SEVERE Vulnerabilities (1)</b></summary><br/>
   <ul>
   
   > #### [CVE-2021-22096] In Spring Framework versions 5.3.0 - 5.3.10, 5.2.0 - 5.2.17, and older unsupport...
   > In Spring Framework versions 5.3.0 - 5.3.10, 5.2.0 - 5.2.17, and older unsupported versions, it is possible for a user to provide malicious input to cause the insertion of additional log entries.
   >
   > **CVSS Score:** 4.3
   >
   > **CVSS Vector:** CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:L/A:N
   
   </ul>
       </details>
           </ul>
         </details>
     </ul>
   </details>
   (at-me [in a reply](https://help.sonatype.com/lift/talking-to-lift) with `help` or `ignore`)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] madrob commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r804920634



##########
File path: .github/workflows/manual.yml
##########
@@ -0,0 +1,21 @@
+name: Scheduled Merge Remote Action

Review comment:
       I don't think we want this? Might be surprising for some folks...




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r827337036



##########
File path: solr/core/build.gradle
##########
@@ -90,6 +90,7 @@ dependencies {
   implementation 'org.apache.httpcomponents:httpcore'
 
   compileOnlyApi 'javax.servlet:javax.servlet-api'
+  compileOnlyApi 'org.eclipse.jetty.toolchain:jetty-servlet-api'

Review comment:
       Not sure what you want to do here. If I remove that, analyzeClassDependencies will fail.
   
   Here is the details on that toolchain module, appears to be part of what is needed to build Jetty: https://github.com/eclipse/jetty.toolchain
   
   Here is a dependency insight call on that lib:
   
   org.eclipse.jetty.toolchain:jetty-servlet-api:4.0.6
   +--- project :
   |    \--- runtimeClasspath
   +--- org.eclipse.jetty:jetty-rewrite:10.0.8
   |    +--- runtimeClasspath
   |    \--- project : (requested org.eclipse.jetty:jetty-rewrite:{strictly 10.0.8}) (*)
   \--- org.eclipse.jetty:jetty-server:10.0.8
        +--- runtimeClasspath
        +--- project : (requested org.eclipse.jetty:jetty-server:{strictly 10.0.8}) (*)
        +--- io.dropwizard.metrics:metrics-jetty9:4.1.5 (requested org.eclipse.jetty:jetty-server:9.4.27.v20200227)
        |    +--- runtimeClasspath
        |    \--- project : (requested io.dropwizard.metrics:metrics-jetty9:{strictly 4.1.5}) (*)
        +--- org.eclipse.jetty:jetty-rewrite:10.0.8 (*)
        +--- org.eclipse.jetty:jetty-security:10.0.8
        |    +--- runtimeClasspath
        |    +--- project : (requested org.eclipse.jetty:jetty-security:{strictly 10.0.8}) (*)
        |    \--- org.eclipse.jetty:jetty-servlet:10.0.8
        |         +--- runtimeClasspath
        |         +--- project : (requested org.eclipse.jetty:jetty-servlet:{strictly 10.0.8}) (*)
        |         \--- org.eclipse.jetty:jetty-webapp:10.0.8
        |              +--- runtimeClasspath
        |              +--- project : (requested org.eclipse.jetty:jetty-webapp:{strictly 10.0.8}) (*)
        |              \--- org.eclipse.jetty:jetty-deploy:10.0.8
        |                   +--- runtimeClasspath
        |                   \--- project : (requested org.eclipse.jetty:jetty-deploy:{strictly 10.0.8}) (*)
        +--- org.eclipse.jetty:jetty-alpn-server:10.0.8
        |    +--- runtimeClasspath
        |    +--- project : (requested org.eclipse.jetty:jetty-alpn-server:{strictly 10.0.8}) (*)
        |    \--- org.eclipse.jetty:jetty-alpn-java-server:10.0.8
        |         +--- runtimeClasspath
        |         \--- project : (requested org.eclipse.jetty:jetty-alpn-java-server:{strictly 10.0.8}) (*)
        \--- org.eclipse.jetty.http2:http2-server:10.0.8
             +--- runtimeClasspath
             \--- project : (requested org.eclipse.j




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] markrmiller commented on a change in pull request #585: SOLR-15955 Update Jetty dependency to 10

Posted by GitBox <gi...@apache.org>.
markrmiller commented on a change in pull request #585:
URL: https://github.com/apache/solr/pull/585#discussion_r819244284



##########
File path: solr/modules/s3-repository/build.gradle
##########
@@ -39,6 +39,10 @@ dependencies {
     runtimeOnly (group: 'com.fasterxml.woodstox', name: 'woodstox-core')
     runtimeOnly (group: 'org.codehaus.woodstox', name: 'stax2-api')
 
+    implementation 'org.springframework.boot:spring-boot'

Review comment:
       That creates the Jetty 10 issues.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org