You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Dinesh Chitlangia (Jira)" <ji...@apache.org> on 2020/03/25 04:35:00 UTC
[jira] [Created] (HDDS-3272) Smoke Test: hdfs commands failing on
hadoop 27 docker-compose
Dinesh Chitlangia created HDDS-3272:
---------------------------------------
Summary: Smoke Test: hdfs commands failing on hadoop 27 docker-compose
Key: HDDS-3272
URL: https://issues.apache.org/jira/browse/HDDS-3272
Project: Hadoop Distributed Data Store
Issue Type: Bug
Affects Versions: 0.5.0
Reporter: Dinesh Chitlangia
Discovered by [~bharat] when testing 0.5.0-beta RC2.
issue when running hdfs commands on hadoop 27
docker-compose. I see the same test failing when running the smoke test.
$ docker exec -it c7fe17804044 bash
bash-4.4$ hdfs dfs -put /opt/hadoop/NOTICE.txt o3fs://bucket1.vol1/kk
2020-03-22 04:40:14 WARN NativeCodeLoader:60 - Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2020-03-22 04:40:15 INFO MetricsConfig:118 - Loaded properties from
hadoop-metrics2.properties
2020-03-22 04:40:16 INFO MetricsSystemImpl:374 - Scheduled Metric snapshot
period at 10 second(s).
2020-03-22 04:40:16 INFO MetricsSystemImpl:191 - XceiverClientMetrics
metrics system started
-put: Fatal internal error
java.lang.NullPointerException: client is null
at java.util.Objects.requireNonNull(Objects.java:228)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.getClient(XceiverClientRatis.java:201)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.sendRequestAsync(XceiverClientRatis.java:227)
at
org.apache.hadoop.hdds.scm.XceiverClientRatis.sendCommandAsync(XceiverClientRatis.java:305)
at
org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.writeChunkAsync(ContainerProtocolCalls.java:315)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.writeChunkToContainer(BlockOutputStream.java:599)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.writeChunk(BlockOutputStream.java:452)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.handleFlush(BlockOutputStream.java:463)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.close(BlockOutputStream.java:486)
at
org.apache.hadoop.ozone.[client.io|http://client.io/].BlockOutputStreamEntry.close(BlockOutputStreamEntry.java:144)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.handleStreamAction(KeyOutputStream.java:481)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.handleFlushOrClose(KeyOutputStream.java:455)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.close(KeyOutputStream.java:508)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.close(OzoneFSOutputStream.java:56)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:62)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:120)
at
org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.writeStreamToFile(CommandWithDestination.java:466)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:391)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:328)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:263)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:248)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:317)
at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:289)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:243)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:271)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:255)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:220)
at
org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:267)
at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201)
at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
The same command when using ozone fs is working fine.
docker exec -it fe5d39cf6eed bash
bash-4.2$ ozone fs -put /opt/hadoop/NOTICE.txt o3fs://bucket1.vol1/kk
2020-03-22 04:41:10,999 [main] INFO impl.MetricsConfig: Loaded properties
from hadoop-metrics2.properties
2020-03-22 04:41:11,123 [main] INFO impl.MetricsSystemImpl: Scheduled
Metric snapshot period at 10 second(s).
2020-03-22 04:41:11,127 [main] INFO impl.MetricsSystemImpl:
XceiverClientMetrics metrics system started
bash-4.2$ ozone fs -ls o3fs://bucket1.vol1/
Found 1 items
-rw-rw-rw- 3 hadoop hadoop 17540 2020-03-22 04:41
o3fs://bucket1.vol1/kk
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org