You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Eric Chang (JIRA)" <ji...@apache.org> on 2018/07/23 23:40:00 UTC

[jira] [Created] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

Eric Chang created SPARK-24895:
----------------------------------

             Summary: Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames
                 Key: SPARK-24895
                 URL: https://issues.apache.org/jira/browse/SPARK-24895
             Project: Spark
          Issue Type: Bug
          Components: Build
    Affects Versions: 2.4.0
            Reporter: Eric Chang


Spark 2.4.0 has maven build errors because artifacts uploaded to apache maven repo has mismatched filenames:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-banned-dependencies) on project spark_2.4: Execution enforce-banned-dependencies of goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce failed: org.apache.maven.shared.dependency.graph.DependencyGraphBuilderException: Could not resolve following dependencies: [org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT (compile), org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT (compile), org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT (compile)]: Could not resolve dependencies for project com.databricks:spark_2.4:pom:1: The following artifacts could not be resolved: org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT, org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT, org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT: Could not find artifact org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-20180723.232411-177 in apache-snapshots ([https://repository.apache.org/snapshots/]) -> [Help 1]

 

If you check the artifact metadata you will see the pom and jar files are 2.4.0-20180723.232411-177 instead of 2.4.0-20180723.232410-177:
{code:xml}
<metadata modelVersion="1.1.0">
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib-local_2.11</artifactId>
<version>2.4.0-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20180723.232411</timestamp>
<buildNumber>177</buildNumber>
</snapshot>
<lastUpdated>20180723232411</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>2.4.0-20180723.232411-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>2.4.0-20180723.232411-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>tests</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>sources</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>test-sources</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
</metadata>
{code}
 
 This behavior is very similar to this issue: https://issues.apache.org/jira/browse/MDEPLOY-221

Since 2.3.0 snapshots work with the same maven 3.3.9 version and maven deploy 2.8.2 plugin, it is highly possible that we introduced a new plugin that causes this. 

The most recent addition is the spot-bugs plugin, which is known to have incompatibilities with other plugins: [https://github.com/spotbugs/spotbugs-maven-plugin/issues/21]

We may want to try building without it to sanity check.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org