You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by GitBox <gi...@apache.org> on 2022/09/21 10:04:45 UTC
[GitHub] [maven-resolver] cstamas opened a new pull request, #199: [MRESOLVER-269] Compact format for provided checksums
cstamas opened a new pull request, #199:
URL: https://github.com/apache/maven-resolver/pull/199
New provided checksums implementation that uses more compact format: a "summary" file that contains list of Artifact IDs and provided checksum per one line.
This format is more VCS friendly, and also easier to handle then sparse directories.
Perfomed cleanup around provided checksum sources as well, old one was in wrong place, dropped it (as it was final class), now we have two sources:
* `file-sparse` -- is exactly same as dropped one, expects provided checksums in "local repo"-like layout
* `file-compact` -- is the new format, where one file `checksums.${checksumExt}` is expected to contain Artifact ID and checksum per line (separated by space)
---
https://issues.apache.org/jira/browse/MRESOLVER-269
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r985443290
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
LocalPathPrefixComposer is a component, and in that case we could have it behave same as LRM. In other words, if you "split" LRM you'd need to split this one as well, and other way around. I don't see the two "same".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r986066117
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
OK, now I understand it makes sense. Would it at least, for consistency reasons, make sense to rename `.originAware` to `.splitRepository`? This term is already established with its semantics?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r984935204
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ }
+
+ @SuppressWarnings( "unchecked" )
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
Review Comment:
I mixed up things, cannot be null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1264295715
Going through again...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r983824905
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,149 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like bqsedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean) must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string, path) directory from where implementation
+ * can use files. If unset, default value is ".checksums" and is resolved from local repository basedir.</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.originAware</pre> (boolean) whether to make implementation
+ * "originAware", to factor in origin repository ID as well or not.</li>
+ * </ul>
+ * <p>
+ * This implementation ensures that implementations have "name" property, used in configuration properties above.
+ *
+ * @since TBD
+ */
+abstract class FileTrustedChecksumsSourceSupport
+ implements TrustedChecksumsSource
+{
+ private static final String CONFIG_PROP_PREFIX = "aether.trustedChecksumsSource.";
+
+ private static final String CONF_NAME_ENABLED = "enabled";
+
+ private static final String CONF_NAME_BASEDIR = "basedir";
+
+ private static final String CONF_NAME_ORIGIN_AWARE = "originAware";
+
+ /**
+ * Visible for testing.
+ */
+ static final String LOCAL_REPO_PREFIX = ".checksums";
+
+ private final String name;
+
+ FileTrustedChecksumsSourceSupport( String name )
+ {
+ this.name = requireNonNull( name );
+ }
+
+ /**
+ * The implementation will call into underlying code only if enabled, chosen basedir exists, and requested
+ * checksum algorithms are not empty.
+ */
+ @Override
+ public Map<String, String> getTrustedArtifactChecksums( RepositorySystemSession session,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ boolean enabled = ConfigUtils.getBoolean( session, false, configPropKey( CONF_NAME_ENABLED ) );
+ if ( enabled )
+ {
+ Path baseDir = getBasedir( session );
+ if ( baseDir != null && !checksumAlgorithmFactories.isEmpty() )
+ {
+ Map<String, String> result = performLookup(
+ session, baseDir, artifact, artifactRepository, checksumAlgorithmFactories );
+
+ return result == null || result.isEmpty() ? null : result;
+ }
+ }
+ return null;
+ }
+
+ protected abstract Map<String, String> performLookup( RepositorySystemSession session,
+ Path baseDir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories );
+
+ /**
+ * To be used by underlying implementations to form configuration property keys properly scoped.
+ */
+ protected String configPropKey( String name )
+ {
+ requireNonNull( name );
+ return CONFIG_PROP_PREFIX + this.name + "." + name;
+ }
+
+ /**
+ * Returns {@code true} if session configuration contains "originAware" property set to {@code true}.
+ */
+ protected boolean isOriginAware( RepositorySystemSession session )
+ {
+ return ConfigUtils.getBoolean( session, false, configPropKey( CONF_NAME_ORIGIN_AWARE ) );
+ }
+
+ /**
+ * Uses common {@link DirectoryUtils} to calculate (but not) create basedir for this implementation. Returns
+ * {@code null} if the calculated basedir does not exist.
+ */
+ private Path getBasedir( RepositorySystemSession session )
+ {
+ try
+ {
+ Path basedir = DirectoryUtils.resolveDirectory(
+ session, LOCAL_REPO_PREFIX, configPropKey( CONF_NAME_BASEDIR ), false );
+ if ( !Files.isDirectory( basedir ) )
Review Comment:
Alright, makes sense. The developer has to take care of that, not us.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r985076402
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,154 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like basedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean) must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string, path) directory from where implementation
+ * can use files. May be relative path (the is resolved from local repository basedir) or absolute. If unset,
+ * default value is ".checksums" and is resolved from local repository basedir.</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.originAware</pre> (boolean) whether to make implementation
+ * "originAware", to factor in origin repository ID as well or not.</li>
+ * </ul>
+ * <p>
+ * This implementation ensures that implementations have "name" property, used in configuration properties above.
+ *
+ * @since TBD
+ */
+abstract class FileTrustedChecksumsSourceSupport
+ implements TrustedChecksumsSource
+{
+ private static final String CONFIG_PROP_PREFIX = "aether.trustedChecksumsSource.";
+
+ private static final String CONF_NAME_ENABLED = "enabled";
+
+ private static final String CONF_NAME_BASEDIR = "basedir";
+
+ private static final String CONF_NAME_ORIGIN_AWARE = "originAware";
+
+ /**
+ * Visible for testing.
+ */
+ static final String LOCAL_REPO_PREFIX = ".checksums";
Review Comment:
This should rather be `LOCAL_REPO_PREFIX_DIR` since it stands as a dir name .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r985076081
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
Call me stupid, but why don't we use the `LocalPathPrefixComposer` for this. It will behave identically to `EnhancedLocalRepositoryManager`?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,154 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like basedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean) must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string, path) directory from where implementation
+ * can use files. May be relative path (the is resolved from local repository basedir) or absolute. If unset,
+ * default value is ".checksums" and is resolved from local repository basedir.</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.originAware</pre> (boolean) whether to make implementation
+ * "originAware", to factor in origin repository ID as well or not.</li>
+ * </ul>
+ * <p>
+ * This implementation ensures that implementations have "name" property, used in configuration properties above.
+ *
+ * @since TBD
+ */
+abstract class FileTrustedChecksumsSourceSupport
+ implements TrustedChecksumsSource
+{
+ private static final String CONFIG_PROP_PREFIX = "aether.trustedChecksumsSource.";
+
+ private static final String CONF_NAME_ENABLED = "enabled";
+
+ private static final String CONF_NAME_BASEDIR = "basedir";
+
+ private static final String CONF_NAME_ORIGIN_AWARE = "originAware";
+
+ /**
+ * Visible for testing.
+ */
+ static final String LOCAL_REPO_PREFIX = ".checksums";
Review Comment:
This should be rather `LOCAL_REPO_PREFIX_DIR` since it stands as a dir name .
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,162 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "checksums-central.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUMS_FILE_PREFIX = "checksums";
+
+ private static final String CHECKSUMS_CACHE_KEY = CompactFileTrustedChecksumsSource.class.getName() + ".checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ }
+
+ @SuppressWarnings( "unchecked" )
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ prefix = CHECKSUMS_FILE_PREFIX + "-" + artifactRepository.getId() + ".";
+ }
+ else
+ {
+ prefix = CHECKSUMS_FILE_PREFIX + ".";
+ }
+
+ final ConcurrentHashMap<String, ConcurrentHashMap<String, String>> basedirProvidedChecksums =
+ (ConcurrentHashMap<String, ConcurrentHashMap<String, String>>) session.getData()
+ .computeIfAbsent( CHECKSUMS_CACHE_KEY, ConcurrentHashMap::new );
+
+ final HashMap<String, String> checksums = new HashMap<>();
+ for ( ChecksumAlgorithmFactory checksumAlgorithmFactory : checksumAlgorithmFactories )
+ {
+ ConcurrentHashMap<String, String> algorithmChecksums = basedirProvidedChecksums.computeIfAbsent(
+ checksumAlgorithmFactory.getName(),
+ algName -> loadProvidedChecksums(
+ basedir.resolve( prefix + checksumAlgorithmFactory.getFileExtension() )
+ )
+ );
+ String checksum = algorithmChecksums.get( ArtifactIdUtils.toId( artifact ) );
+ if ( checksum != null )
+ {
+ checksums.put( checksumAlgorithmFactory.getName(), checksum );
+ }
+ }
+ return checksums;
+ }
+
+ private ConcurrentHashMap<String, String> loadProvidedChecksums( Path checksumFile )
+ {
+ ConcurrentHashMap<String, String> result = new ConcurrentHashMap<>();
+ try
+ {
+ try ( BufferedReader reader = Files.newBufferedReader( checksumFile, StandardCharsets.UTF_8 ) )
+ {
+ LOGGER.debug( "Loading provided checksums file '{}'", checksumFile );
+ String line;
+ while ( ( line = reader.readLine() ) != null )
+ {
+ if ( !line.startsWith( "#" ) && !line.isEmpty() )
+ {
+ String[] parts = line.split( " ", 2 );
+ if ( parts.length == 2 )
+ {
+ String old = result.put( parts[0], parts[1] );
+ if ( old != null )
+ {
+ LOGGER.warn( "Checksums file '{}' contains duplicate checksums for artifact {}: "
+ + "old '{}' replaced by new '{}'", checksumFile, parts[0], old, parts[1] );
+ }
+ }
+ else
+ {
+ LOGGER.warn( "Checksums file '{}' ignored malformed line '{}'", checksumFile, line );
+ }
+ }
+ }
+ }
+ }
+ catch ( NoSuchFileException e )
+ {
+ // ignore, will return empty result
+ LOGGER.debug( "Checksums file '{}' not found", checksumFile );
Review Comment:
This is an antipattern according to Joshua Bloch. Test for existance instead of reyling on such exceptions.
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
+ * <p>
+ * The name of this implementation is "file-sparse".
+ *
+ * @see LocalPathComposer
+ * @since TBD
+ */
+@Singleton
+@Named( SparseFileTrustedChecksumsSource.NAME )
+public final class SparseFileTrustedChecksumsSource
Review Comment:
I know what sparse means here and I think it is used incorrectly. Your understanding: A spare directory tree where only a limited amount of files exist for artifacts required. Meaning of sparse file: https://en.wikipedia.org/wiki/Sparse_file
I think the term is abused.
It should be `SparseDirectoryFileTrustedChecksumsSource` or `SparseDirectoryTrustedChecksumsSource`.
See impl of a sparse dir tree: https://svnbook.red-bean.com/en/1.7/svn.advanced.sparsedirs.html
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,162 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
Review Comment:
Stupid question: If you say yourself that it uses a summary file, why just not call it so? `SummaryFileTrustedChecksumsSource`, DI name `summary-file`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r986063529
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
It can, but I tend to NOT make it 100% same for several reasons:
* if we'd use LocalPathPrefixComposer, then if you use split LRM you MUST use split (originAware) SparseDirectory as well. So it's both or nothing.
* given we want to distinguish here the ORIGIN of checksums, I would even lessen the "resemblency" to local repo (to not have anyone "tempted" to just copy the hashes from his local repo for example)
* both (non origin aware and origin aware) has it's "pros": one can ship "curated" checksums per project (non-origin aware), or one can ship "curated" checksum per repo
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
It can, but I tend to NOT make it 100% same for several reasons:
* ~~if we'd use LocalPathPrefixComposer, then if you use split LRM you MUST use split (originAware) SparseDirectory as well. So it's both or nothing~~ I feel I don't quite understand your proposal
* given we want to distinguish here the ORIGIN of checksums, I would even lessen the "resemblency" to local repo (to not have anyone "tempted" to just copy the hashes from his local repo for example)
* both (non origin aware and origin aware) has it's "pros": one can ship "curated" checksums per project (non-origin aware), or one can ship "curated" checksum per repo
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
IMHO nope, partially due that above ("lessen the resemblency to LRM"), and partially as these are "trusted checksums" ONLY (that may or may not have origin noted), while "split repository" is, well, as it's name say a "repository", that again implies layout and contents (artifacts, not checksums). Plus, then we'd have a "repository embedded within repository", as `$LOCAL_REPO/.splitRepository` would imply that, no?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
IMHO nope, partially due that above ("lessen the resemblency to LRM"), and partially as these are "trusted checksums" ONLY (that may or may not have origin noted), while "split repository" is, well, as it's name say a "repository", that again implies layout and contents (artifacts, not checksums).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r984929491
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ @Inject
Review Comment:
isn;t, is explicit but removed it (as guice will figure)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #199: [MRESOLVER-269] Compact format for provided checksums
Posted by GitBox <gi...@apache.org>.
cstamas commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1253488044
This PR obsoletes https://github.com/apache/maven-resolver/pull/192 as it provides 100% same change as in that PR but this PR also cleans up in some areas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r986017262
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
Can you rephrase your answer?! I frankly do not understand whether everything under `.checksums/` can or cannot behave like with LRM treewise. This should also make `.originAware` obsolete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas merged pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas merged PR #199:
URL: https://github.com/apache/maven-resolver/pull/199
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1265103866
> Yet another issue with the `.enabled`: I think this is again inconsistent with our existing boolean properties: https://maven.apache.org/resolver/configuration.html They use the name as boolean property with a default value. So to apply this appoach `.enabled` just can be dropped and its parent used.
This is fixed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1265735368
> > Yet another question: It is worthwhile to enable it by default and add checksums on the fly when they are present?
>
> This could be similar as "record" functionality is for remote repository filter feature. Yes, it could be possible, but unsure is it worth it: the whole idea of this is to get checksums for _other trusted source_ than remote...
ОК, let's put this aside.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r982431840
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,149 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like bqsedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean) must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string, path) directory from where implementation
+ * can use files. If unset, default value is ".checksums" and is resolved from local repository basedir.</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.originAware</pre> (boolean) whether to make implementation
+ * "originAware", to factor in origin repository ID as well or not.</li>
+ * </ul>
+ * <p>
+ * This implementation ensures that implementations have "name" property, used in configuration properties above.
+ *
+ * @since TBD
+ */
+abstract class FileTrustedChecksumsSourceSupport
+ implements TrustedChecksumsSource
+{
+ private static final String CONFIG_PROP_PREFIX = "aether.trustedChecksumsSource.";
+
+ private static final String CONF_NAME_ENABLED = "enabled";
+
+ private static final String CONF_NAME_BASEDIR = "basedir";
+
+ private static final String CONF_NAME_ORIGIN_AWARE = "originAware";
+
+ /**
+ * Visible for testing.
+ */
+ static final String LOCAL_REPO_PREFIX = ".checksums";
+
+ private final String name;
+
+ FileTrustedChecksumsSourceSupport( String name )
+ {
+ this.name = requireNonNull( name );
+ }
+
+ /**
+ * The implementation will call into underlying code only if enabled, chosen basedir exists, and requested
+ * checksum algorithms are not empty.
+ */
+ @Override
+ public Map<String, String> getTrustedArtifactChecksums( RepositorySystemSession session,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ boolean enabled = ConfigUtils.getBoolean( session, false, configPropKey( CONF_NAME_ENABLED ) );
+ if ( enabled )
+ {
+ Path baseDir = getBasedir( session );
+ if ( baseDir != null && !checksumAlgorithmFactories.isEmpty() )
+ {
+ Map<String, String> result = performLookup(
+ session, baseDir, artifact, artifactRepository, checksumAlgorithmFactories );
+
+ return result == null || result.isEmpty() ? null : result;
+ }
+ }
+ return null;
+ }
+
+ protected abstract Map<String, String> performLookup( RepositorySystemSession session,
+ Path baseDir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories );
+
+ /**
+ * To be used by underlying implementations to form configuration property keys properly scoped.
+ */
+ protected String configPropKey( String name )
+ {
+ requireNonNull( name );
+ return CONFIG_PROP_PREFIX + this.name + "." + name;
+ }
+
+ /**
+ * Returns {@code true} if session configuration contains "originAware" property set to {@code true}.
+ */
+ protected boolean isOriginAware( RepositorySystemSession session )
+ {
+ return ConfigUtils.getBoolean( session, false, configPropKey( CONF_NAME_ORIGIN_AWARE ) );
+ }
+
+ /**
+ * Uses common {@link DirectoryUtils} to calculate (but not) create basedir for this implementation. Returns
+ * {@code null} if the calculated basedir does not exist.
+ */
+ private Path getBasedir( RepositorySystemSession session )
+ {
+ try
+ {
+ Path basedir = DirectoryUtils.resolveDirectory(
+ session, LOCAL_REPO_PREFIX, configPropKey( CONF_NAME_BASEDIR ), false );
+ if ( !Files.isDirectory( basedir ) )
Review Comment:
It is for "not exists" case (and create=false), so yes, if exists and is NOT a directlry, DirectoryUtils will throw already. So when here, it either exists (and is directory) or not exists (and isDirectory returns false). IMHO the `Files.exists` is not appropriate here, as would not reflect the real intent.
TL;DR: if exists and is not directory, code does not even arrive here, but I want to reflect the fact basedir is a directory (if exists) or does not exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r983853854
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ }
+
+ @SuppressWarnings( "unchecked" )
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
+ {
+ prefix = artifactRepository.getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ else
+ {
+ prefix = session.getLocalRepository().getId() + "-" + CHECKSUM_FILE_PREFIX;
Review Comment:
Do we compare checksums for local files?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ @Inject
Review Comment:
Isn't that noop?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ }
+
+ @SuppressWarnings( "unchecked" )
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
Review Comment:
How can this be `null`?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,130 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
+ * <p>
+ * The name of this implementation is "file-sparse".
+ *
+ * @see LocalPathComposer
+ * @since TBD
+ */
+@Singleton
+@Named( SparseFileTrustedChecksumsSource.NAME )
+public final class SparseFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-sparse";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( SparseFileTrustedChecksumsSource.class );
+
+ private final FileProcessor fileProcessor;
+
+ private final LocalPathComposer localPathComposer;
+
+ @Inject
+ public SparseFileTrustedChecksumsSource( FileProcessor fileProcessor, LocalPathComposer localPathComposer )
+ {
+ super( NAME );
+ this.fileProcessor = requireNonNull( fileProcessor );
+ this.localPathComposer = requireNonNull( localPathComposer );
+ }
+
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
+ {
+ prefix = artifactRepository.getId() + "/";
+ }
+ else
+ {
+ prefix = session.getLocalRepository().getId() + "/";
+ }
+ }
+ else
+ {
+ prefix = "";
+ }
+
+ final HashMap<String, String> checksums = new HashMap<>();
+ final String artifactPath = localPathComposer.getPathForArtifact( artifact, false );
+ for ( ChecksumAlgorithmFactory checksumAlgorithmFactory : checksumAlgorithmFactories )
+ {
+ Path checksumPath = basedir.resolve(
+ prefix + artifactPath + "." + checksumAlgorithmFactory.getFileExtension() );
+
+ if ( !Files.isRegularFile( checksumPath ) )
+ {
+ continue;
+ }
Review Comment:
I think this deserves a log message. I guess developer provided us nonsense.
##########
maven-resolver-impl/src/test/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceTestSupport.java:
##########
@@ -0,0 +1,90 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.Collections;
+import java.util.Map;
+
+import org.eclipse.aether.DefaultRepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.artifact.DefaultArtifact;
+import org.eclipse.aether.internal.test.util.TestUtils;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.junit.Before;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+
+public abstract class FileTrustedChecksumsSourceTestSupport
+{
+ protected static final Artifact ARTIFACT_WITHOUT_CHECKSUM = new DefaultArtifact( "test:test:1.0" );
+
+ protected static final Artifact ARTIFACT_WITH_CHECKSUM = new DefaultArtifact( "test:test:2.0" );
+
+ protected static final String ARTIFACT_TRUSTED_CHECKSUM = "trustedChecksum";
+
+ protected DefaultRepositorySystemSession session;
+
+ protected ChecksumAlgorithmFactory checksumAlgorithmFactory;
+
+ private FileTrustedChecksumsSourceSupport subject;
+
+ @Before
+ public void before() throws Exception
+ {
+ session = TestUtils.newSession();
+ // populate local repository
+ Path baseDir = session.getLocalRepository().getBasedir().toPath()
Review Comment:
basedir
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ }
+
+ @SuppressWarnings( "unchecked" )
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
+ {
+ prefix = artifactRepository.getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ else
+ {
+ prefix = session.getLocalRepository().getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ }
+ else
+ {
+ prefix = CHECKSUM_FILE_PREFIX;
+ }
+
+ final ConcurrentHashMap<String, ConcurrentHashMap<String, String>> basedirProvidedChecksums =
+ (ConcurrentHashMap<String, ConcurrentHashMap<String, String>>) session.getData()
+ .computeIfAbsent( CHECKSUMS_CACHE_KEY, ConcurrentHashMap::new );
+
+ final HashMap<String, String> checksums = new HashMap<>();
+ for ( ChecksumAlgorithmFactory checksumAlgorithmFactory : checksumAlgorithmFactories )
+ {
+ ConcurrentHashMap<String, String> algorithmChecksums = basedirProvidedChecksums.computeIfAbsent(
+ checksumAlgorithmFactory.getName(),
+ algName -> loadProvidedChecksums(
+ basedir.resolve( prefix + checksumAlgorithmFactory.getFileExtension() )
+ )
+ );
+ String checksum = algorithmChecksums.get( ArtifactIdUtils.toId( artifact ) );
+ if ( checksum != null )
+ {
+ checksums.put( checksumAlgorithmFactory.getName(), checksum );
+ }
+ }
+ return checksums;
+ }
+
+ private ConcurrentHashMap<String, String> loadProvidedChecksums( Path checksumFile )
+ {
+ ConcurrentHashMap<String, String> result = new ConcurrentHashMap<>();
+ try
+ {
+ try ( BufferedReader reader = Files.newBufferedReader( checksumFile, StandardCharsets.UTF_8 ) )
+ {
+ LOGGER.debug( "Loading provided checksums file '{}'", checksumFile );
+ String line = reader.readLine();
+ while ( line != null )
+ {
+ if ( !line.startsWith( "#" ) && !line.isEmpty() )
+ {
+ String[] parts = line.split( " ", 2 );
+ if ( parts.length == 2 )
+ {
+ String old = result.put( parts[0], parts[1] );
+ if ( old != null )
+ {
+ LOGGER.warn( "Checksums file '{}' contains duplicate checksums for artifact {}: "
+ + "old '{}' replaced by new '{}'", checksumFile, parts[0], old, parts[1] );
+ }
+ }
+ else
+ {
+ LOGGER.warn( "Checksums file '{}' ignored malformed line '{}'", checksumFile, line );
+ }
+ }
+ line = reader.readLine();
Review Comment:
Why not put `reader.readLine()` in the while condition?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final String CHECKSUMS_CACHE_KEY = NAME + "-checksums";
Review Comment:
This might be nitpicking, but of this cache is an impl detail I think that the key should be prefixed with the class name.
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
Review Comment:
This class has one inconsistency: All our meta files have the form: `{metafile}.{ext}` or `{metafile}-{repoId}.{ext}`. This should apply here as well. E.g., `./org/apache/maven/resolver/maven-metadata-local.xml`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1260956232
> * Since the `LocalPathComposer` is used throughout the docs of the sources should mention that this substructure is also reflected in `.checksums/`
The javadoc of sparse source says:
```
* Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
* directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
* coordinates solely to form path from basedir, pretty much as Maven local repository does.
```
> * As for the file-compact. I like the idea. Does it make sense to reuse rather a well-known format like BSD checksum file? See also [nicoulaj/checksum-maven-plugin#127](https://github.com/nicoulaj/checksum-maven-plugin/issues/127)
IMHO no, as BSD checksum file contains filename, here, we operate with ArtifactId ("file format is artifact ID and checksum separated by space per line"), see https://github.com/apache/maven-resolver/blob/master/maven-resolver-util/src/main/java/org/eclipse/aether/util/artifact/ArtifactIdUtils.java#L45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1262576217
> > * Since the `LocalPathComposer` is used throughout the docs of the sources should mention that this substructure is also reflected in `.checksums/`
>
> The javadoc of sparse source says:
>
> ```
> * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
> * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
> * coordinates solely to form path from basedir, pretty much as Maven local repository does.
> ```
I don't see how this answers my question... ?!!
> > * As for the file-compact. I like the idea. Does it make sense to reuse rather a well-known format like BSD checksum file? See also [nicoulaj/checksum-maven-plugin#127](https://github.com/nicoulaj/checksum-maven-plugin/issues/127)
>
> IMHO no, as BSD checksum file contains filename, here, we operate with ArtifactId ("file format is artifact ID and checksum separated by space per line"), see https://github.com/apache/maven-resolver/blob/master/maven-resolver-util/src/main/java/org/eclipse/aether/util/artifact/ArtifactIdUtils.java#L45
hm...we basically use our current format, but compile multiple files and key them by artifact id. Agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1264310466
Yet another question. It is worthwhile to enable it by default and add checksums on the fly when they are present?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] raphw commented on pull request #199: [MRESOLVER-269] Compact format for provided checksums
Posted by GitBox <gi...@apache.org>.
raphw commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1253552663
This looks really good, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r982378937
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,149 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like bqsedir calculation, "enabled" flag and "originAware" flag.
Review Comment:
typo
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,166 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ private final ConcurrentHashMap<Path, ConcurrentHashMap<String, ConcurrentHashMap<String, String>>> checksumCache;
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ this.checksumCache = new ConcurrentHashMap<>();
+ }
+
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final HashMap<String, String> checksums = new HashMap<>();
+ final ConcurrentHashMap<String, ConcurrentHashMap<String, String>> baseDirProvidedHashes = checksumCache
Review Comment:
basedirProvidedChecksums
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,166 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ private final ConcurrentHashMap<Path, ConcurrentHashMap<String, ConcurrentHashMap<String, String>>> checksumCache;
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ this.checksumCache = new ConcurrentHashMap<>();
+ }
+
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final HashMap<String, String> checksums = new HashMap<>();
+ final ConcurrentHashMap<String, ConcurrentHashMap<String, String>> baseDirProvidedHashes = checksumCache
+ .computeIfAbsent( basedir, b -> new ConcurrentHashMap<>() );
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
+ {
+ prefix = artifactRepository.getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ else
+ {
+ prefix = session.getLocalRepository().getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ }
+ else
+ {
+ prefix = CHECKSUM_FILE_PREFIX;
+ }
+
+ for ( ChecksumAlgorithmFactory checksumAlgorithmFactory : checksumAlgorithmFactories )
+ {
+ ConcurrentHashMap<String, String> algorithmHashes = baseDirProvidedHashes.computeIfAbsent(
+ checksumAlgorithmFactory.getName(),
+ algName -> loadProvidedHashes(
+ basedir.resolve( prefix + checksumAlgorithmFactory.getFileExtension() )
+ )
+ );
+ String checksum = algorithmHashes.get( ArtifactIdUtils.toId( artifact ) );
+ if ( checksum != null )
+ {
+ checksums.put( checksumAlgorithmFactory.getName(), checksum );
+ }
+ }
+ return checksums;
+ }
+
+ private ConcurrentHashMap<String, String> loadProvidedHashes( Path checksumFile )
Review Comment:
Same here
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,149 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like bqsedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean) must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string, path) directory from where implementation
+ * can use files. If unset, default value is ".checksums" and is resolved from local repository basedir.</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.originAware</pre> (boolean) whether to make implementation
+ * "originAware", to factor in origin repository ID as well or not.</li>
+ * </ul>
+ * <p>
+ * This implementation ensures that implementations have "name" property, used in configuration properties above.
+ *
+ * @since TBD
+ */
+abstract class FileTrustedChecksumsSourceSupport
+ implements TrustedChecksumsSource
+{
+ private static final String CONFIG_PROP_PREFIX = "aether.trustedChecksumsSource.";
+
+ private static final String CONF_NAME_ENABLED = "enabled";
+
+ private static final String CONF_NAME_BASEDIR = "basedir";
+
+ private static final String CONF_NAME_ORIGIN_AWARE = "originAware";
+
+ /**
+ * Visible for testing.
+ */
+ static final String LOCAL_REPO_PREFIX = ".checksums";
+
+ private final String name;
+
+ FileTrustedChecksumsSourceSupport( String name )
+ {
+ this.name = requireNonNull( name );
+ }
+
+ /**
+ * The implementation will call into underlying code only if enabled, chosen basedir exists, and requested
+ * checksum algorithms are not empty.
+ */
+ @Override
+ public Map<String, String> getTrustedArtifactChecksums( RepositorySystemSession session,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ boolean enabled = ConfigUtils.getBoolean( session, false, configPropKey( CONF_NAME_ENABLED ) );
+ if ( enabled )
+ {
+ Path baseDir = getBasedir( session );
+ if ( baseDir != null && !checksumAlgorithmFactories.isEmpty() )
+ {
+ Map<String, String> result = performLookup(
+ session, baseDir, artifact, artifactRepository, checksumAlgorithmFactories );
+
+ return result == null || result.isEmpty() ? null : result;
+ }
+ }
+ return null;
+ }
+
+ protected abstract Map<String, String> performLookup( RepositorySystemSession session,
+ Path baseDir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories );
+
+ /**
+ * To be used by underlying implementations to form configuration property keys properly scoped.
+ */
+ protected String configPropKey( String name )
+ {
+ requireNonNull( name );
+ return CONFIG_PROP_PREFIX + this.name + "." + name;
+ }
+
+ /**
+ * Returns {@code true} if session configuration contains "originAware" property set to {@code true}.
+ */
+ protected boolean isOriginAware( RepositorySystemSession session )
+ {
+ return ConfigUtils.getBoolean( session, false, configPropKey( CONF_NAME_ORIGIN_AWARE ) );
+ }
+
+ /**
+ * Uses common {@link DirectoryUtils} to calculate (but not) create basedir for this implementation. Returns
+ * {@code null} if the calculated basedir does not exist.
+ */
+ private Path getBasedir( RepositorySystemSession session )
+ {
+ try
+ {
+ Path basedir = DirectoryUtils.resolveDirectory(
+ session, LOCAL_REPO_PREFIX, configPropKey( CONF_NAME_BASEDIR ), false );
+ if ( !Files.isDirectory( basedir ) )
Review Comment:
Won't this throw an exception if it isn't a dir already?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,166 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUM_FILE_PREFIX = "checksums.";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger( CompactFileTrustedChecksumsSource.class );
+
+ private final ConcurrentHashMap<Path, ConcurrentHashMap<String, ConcurrentHashMap<String, String>>> checksumCache;
+
+ @Inject
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ this.checksumCache = new ConcurrentHashMap<>();
+ }
+
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository artifactRepository,
+ List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final HashMap<String, String> checksums = new HashMap<>();
+ final ConcurrentHashMap<String, ConcurrentHashMap<String, String>> baseDirProvidedHashes = checksumCache
+ .computeIfAbsent( basedir, b -> new ConcurrentHashMap<>() );
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ if ( artifactRepository != null )
+ {
+ prefix = artifactRepository.getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ else
+ {
+ prefix = session.getLocalRepository().getId() + "-" + CHECKSUM_FILE_PREFIX;
+ }
+ }
+ else
+ {
+ prefix = CHECKSUM_FILE_PREFIX;
+ }
+
+ for ( ChecksumAlgorithmFactory checksumAlgorithmFactory : checksumAlgorithmFactories )
+ {
+ ConcurrentHashMap<String, String> algorithmHashes = baseDirProvidedHashes.computeIfAbsent(
Review Comment:
algorithmChecksums
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,149 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by local filesystem. It implements basic support
+ * like bqsedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean) must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string, path) directory from where implementation
+ * can use files. If unset, default value is ".checksums" and is resolved from local repository basedir.</li>
Review Comment:
Docs should state, relative or absolute or both.
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,128 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
+ * coordinates solely to form path from baseDir, pretty much as Maven local repository does.
Review Comment:
basedir
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r984942284
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,172 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
+ * directory, where it expects a "summary" file named as "checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will factor in origin repository ID as well into
+ * file name (for example "central-checksums.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
Review Comment:
made it `checksums-central.sha1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1264016126
> I don't see how this answers my question... ?!!
I might missed the real intent of your question then...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
michael-o commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1264310181
> > I don't see how this answers my question... ?!!
>
> I might missed the real intent of your question then...
Forget it, I understand now. No further explanation necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #199: [MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source
Posted by GitBox <gi...@apache.org>.
cstamas commented on PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#issuecomment-1265038787
> Yet another question: It is worthwhile to enable it by default and add checksums on the fly when they are present?
This could be similar as "record" functionality is for remote repository filter feature. Yes, it could be possible, but unsure is it worths it: the whole idea of this is to get checksums for _other trusted source_ than remote...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org