You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/11/12 10:47:59 UTC
[GitHub] [incubator-druid] fstolba opened a new issue #8858: export-metadata
throws java.lang.NullPointerException
fstolba opened a new issue #8858: export-metadata throws java.lang.NullPointerException
URL: https://github.com/apache/incubator-druid/issues/8858
I'm trying to migrate my segments from local storage to S3. When running `export-metadata` with parameters `-b`/`-k` (rewriting deep storage location to S3), it crashes while exporting the segments table. Exporting without rewriting works fine.
I suspect it might be due to the fact that not all segments in the table refer to the same storage location. Recently created and new segments are actually stored in the target bucket already (storage location in `common/runtime.conf` was changed to reflect the S3 configuration).
### Affected Version
0.16
### Description
```
bash-4.2# java -classpath "lib/*" -Dlog4j.configurationFile=conf/druid/cluster/_common/log4j2.xml -Ddruid.extensions.directory="extensions" -Ddruid.extensions.loadList=[] org.apache.druid.cli.Main tools export-metadata --connectURI "jdbc:derby://localhost:1527/var/druid/metadata.db;" -o /tmp -b druid-bucket -k druid/segments
Nov 12, 2019 10:11:27 AM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 5.1.3.Final
2019-11-12T10:11:27,504 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.common.config.NullValueHandlingConfig] from props[druid.generic.] as [org.apache.druid.common.config.NullValueHandlingConfig@3da05287]
2019-11-12T10:11:27,552 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='extensions', useExtensionClassloaderFirst=false, hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', addExtensionsToHadoopContainer=false, loadList=[]}]
2019-11-12T10:11:27,640 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.ModulesConfig] from props[druid.modules.] as [ModulesConfig{excludeList=[]}]
2019-11-12T10:11:27,795 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.common.aws.AWSModule] for class [interface org.apache.druid.initialization.DruidModule] from classpath extension
2019-11-12T10:11:27,797 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.common.gcp.GcpModule] for class [interface org.apache.druid.initialization.DruidModule] from classpath extension
2019-11-12T10:11:28,042 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.server.StatusResource
2019-11-12T10:11:28,476 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.server.security.Escalator] from props[druid.escalator.] as [NoopEscalator{}]
2019-11-12T10:11:28,486 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.curator.CuratorConfig] from props[druid.zk.service.] as [org.apache.druid.curator.CuratorConfig@3015db78]
2019-11-12T10:11:28,490 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.curator.ExhibitorConfig] from props[druid.exhibitor.service.] as [org.apache.druid.curator.ExhibitorConfig@6b760460]
2019-11-12T10:11:28,543 INFO [main] org.apache.curator.utils.Compatibility - Running in ZooKeeper 3.4.x compatibility mode
2019-11-12T10:11:28,545 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29
2019-11-12T10:11:28,583 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [org.apache.druid.server.initialization.ZkPathsConfig@58d3f4be]
2019-11-12T10:11:28,591 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.security.AuthConfig] from props[druid.auth.] as [AuthConfig{authenticatorChain=null, authorizers=null, unsecuredPaths=[], allowUnauthenticatedHttpOptions=false}]
2019-11-12T10:11:28,642 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [org.apache.druid.server.metrics.DruidMonitorSchedulerConfig@7bc6d27a]
2019-11-12T10:11:28,646 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[]}]
2019-11-12T10:11:28,647 INFO [main] org.apache.druid.server.emitter.EmitterModule - Underlying emitter for ServiceEmitter: org.apache.druid.java.util.emitter.core.NoopEmitter@1ac45389
2019-11-12T10:11:28,647 INFO [main] org.apache.druid.server.emitter.EmitterModule - Extra service dimensions: {version=0.16.0-incubating}
2019-11-12T10:11:28,648 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.query.ExecutorServiceMonitor@10bd9df0]
2019-11-12T10:11:28,648 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.server.initialization.jetty.JettyServerModule$JettyMonitor@732c9b5c]
2019-11-12T10:11:28,720 INFO [main] org.apache.druid.metadata.storage.derby.DerbyConnector - Derby connector instantiated with metadata storage [org.apache.druid.metadata.NoopMetadataStorageProvider$1].
2019-11-12T10:11:28,720 INFO [main] org.apache.druid.cli.ExportMetadata - Exporting datasource table: druid_dataSource
2019-11-12T10:11:28,862 INFO [main] org.apache.druid.cli.ExportMetadata - Exporting segments table: druid_segments
Exception in thread "main" java.lang.NullPointerException
at org.apache.druid.cli.ExportMetadata.getUniqueIDFromLocalLoadSpec(ExportMetadata.java:537)
at org.apache.druid.cli.ExportMetadata.makePayloadWithConvertedLoadSpec(ExportMetadata.java:429)
at org.apache.druid.cli.ExportMetadata.rewriteSegmentsExport(ExportMetadata.java:407)
at org.apache.druid.cli.ExportMetadata.run(ExportMetadata.java:225)
at org.apache.druid.cli.Main.main(Main.java:113)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org