You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crail.apache.org by Ana Klimovic <an...@gmail.com> on 2018/04/05 08:18:26 UTC
crail-S3 tier
Hello,
I have a question regarding the Crail S3 storage tier:
https://github.com/zrlio/crail-s3
I am running Crail on Amazon EC2 and getting a connection refused error to
localhost on the client. I have the namenode and s3 datanode running on the
same EC2 instance and the client is on a separate EC2 instance, but all
instances are in the same VPC and subnet. Below is the full output on the
client. Please let me know if you have ideas what I may be missing.
$CRAIL_HOME/bin/crail iobench -t write -f /test.data -s 4096 -k 1 -w 0
18/04/05 07:24:01 INFO crail: creating singleton crail file system
18/04/05 07:24:01 INFO crail: crail.version 3004
18/04/05 07:24:01 INFO crail: crail.directorydepth 16
18/04/05 07:24:01 INFO crail: crail.tokenexpiration 10
18/04/05 07:24:01 INFO crail: crail.blocksize 65536
18/04/05 07:24:01 INFO crail: crail.cachelimit 53687091200
18/04/05 07:24:01 INFO crail: crail.cachepath /dev/hugepages/cache
18/04/05 07:24:01 INFO crail: crail.user stu
18/04/05 07:24:01 INFO crail: crail.shadowreplication 1
18/04/05 07:24:01 INFO crail: crail.debug false
18/04/05 07:24:01 INFO crail: crail.statistics true
18/04/05 07:24:01 INFO crail: crail.rpctimeout 4000
18/04/05 07:24:01 INFO crail: crail.datatimeout 1000
18/04/05 07:24:01 INFO crail: crail.buffersize 524288
18/04/05 07:24:01 INFO crail: crail.slicesize 262144
18/04/05 07:24:01 INFO crail: crail.singleton true
18/04/05 07:24:01 INFO crail: crail.regionsize 1073741824
18/04/05 07:24:01 INFO crail: crail.directoryrecord 512
18/04/05 07:24:01 INFO crail: crail.directoryrandomize true
18/04/05 07:24:01 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
18/04/05 07:24:01 INFO crail: crail.locationmap
18/04/05 07:24:01 INFO crail: crail.namenode.address crail://
10.1.217.199:9070
18/04/05 07:24:01 INFO crail: crail.namenode.blockselection roundrobin
18/04/05 07:24:01 INFO crail: crail.namenode.fileblocks 16
18/04/05 07:24:01 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
18/04/05 07:24:01 INFO crail: crail.namenode.log
18/04/05 07:24:01 INFO crail: crail.storage.types
com.ibm.crail.storage.object.ObjectStorageTier
18/04/05 07:24:01 INFO crail: crail.storage.classes 1
18/04/05 07:24:01 INFO crail: crail.storage.rootclass 0
18/04/05 07:24:01 INFO crail: crail.storage.keepalive 2
18/04/05 07:24:01 INFO crail: crail.storage.object.storagelimit = 1073741824
18/04/05 07:24:01 INFO crail: crail.storage.object.allocationsize = 1048576
18/04/05 07:24:01 INFO crail: crail.storage.object.datanode = localhost
18/04/05 07:24:01 INFO crail: crail.storage.object.datanodeport = 50080
18/04/05 07:24:01 INFO crail: crail.storage.object.objectprefix = crail
18/04/05 07:24:01 INFO crail: crail.storage.object.cleanuponexit = false
18/04/05 07:24:01 INFO crail: crail.storage.object.profile = false
18/04/05 07:24:01 INFO crail: crail.storage.object.s3accesskey = XXX
18/04/05 07:24:01 INFO crail: crail.storage.object.s3secretkey = XXXXX
18/04/05 07:24:01 INFO crail: crail.storage.object.s3region = us-west-2
18/04/05 07:24:01 INFO crail: crail.storage.object.s3bucket = anakli-pocket
18/04/05 07:24:01 INFO crail: crail.storage.object.s3endpoint =
s3.us-west-2.amazonaws.com
18/04/05 07:24:01 INFO crail: crail.storage.object.s3protocol = HTTP
18/04/05 07:24:01 INFO crail: crail.storage.object.s3signer = null
18/04/05 07:24:01 INFO narpc: new NaRPC server group v1.0, queueDepth 1024,
messageSize 512, nodealy true
18/04/05 07:24:01 INFO crail: crail.namenode.tcp.queueDepth 1024
18/04/05 07:24:01 INFO crail: crail.namenode.tcp.messageSize 512
18/04/05 07:24:01 INFO crail: crail.namenode.tcp.cores 1
18/04/05 07:24:01 INFO crail: connected to namenode(s) /10.1.217.199:9070
18/04/05 07:24:01 INFO crail: buffer cache, allocationCount 50, bufferCount
2048
write, filename /test.data, size 4096, loop 1, storageClass 0,
locationClass 0, buffered true
warmUp, warmupFile /test.data-972347181, operations 0
starting benchmark...
18/04/05 07:24:01 INFO crail: ERROR: failed data operation
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
refused: localhost/127.0.0.1:50080
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:640)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" java.util.concurrent.ExecutionException:
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
refused: localhost/127.0.0.1:50080
at
org.apache.crail.core.CoreMetaDataOperation.get(CoreMetaDataOperation.java:98)
at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:108)
at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1071)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException:
Connection refused: localhost/127.0.0.1:50080
Re: crail-S3 tier
Posted by ra...@gmail.com,
ra...@gmail.com.
Looks like the S3 datanode is either not running or you used "localhost" instead of the public IP of the machine running the datanode. The S3 datanode handles metadata requests rather than user data (it maps Crail blocks to S3 objects) and has to be accessible from all Crail clients.
I suggest running one instance of this S3 datanode type on the same machine as the namenode and setting the appropriate IP address. For testing purposes, after starting the Crail namenode, you can run the S3 datanode directly in a terminal:
./bin/crail datanode -t com.ibm.crail.storage.object.ObjectStoreTier
On 2018/04/05 08:18:26, Ana Klimovic <an...@gmail.com> wrote:
> Hello,
>
> I have a question regarding the Crail S3 storage tier:
> https://github.com/zrlio/crail-s3
>
> I am running Crail on Amazon EC2 and getting a connection refused error to
> localhost on the client. I have the namenode and s3 datanode running on the
> same EC2 instance and the client is on a separate EC2 instance, but all
> instances are in the same VPC and subnet. Below is the full output on the
> client. Please let me know if you have ideas what I may be missing.
>
>
> $CRAIL_HOME/bin/crail iobench -t write -f /test.data -s 4096 -k 1 -w 0
> 18/04/05 07:24:01 INFO crail: creating singleton crail file system
> 18/04/05 07:24:01 INFO crail: crail.version 3004
> 18/04/05 07:24:01 INFO crail: crail.directorydepth 16
> 18/04/05 07:24:01 INFO crail: crail.tokenexpiration 10
> 18/04/05 07:24:01 INFO crail: crail.blocksize 65536
> 18/04/05 07:24:01 INFO crail: crail.cachelimit 53687091200
> 18/04/05 07:24:01 INFO crail: crail.cachepath /dev/hugepages/cache
> 18/04/05 07:24:01 INFO crail: crail.user stu
> 18/04/05 07:24:01 INFO crail: crail.shadowreplication 1
> 18/04/05 07:24:01 INFO crail: crail.debug false
> 18/04/05 07:24:01 INFO crail: crail.statistics true
> 18/04/05 07:24:01 INFO crail: crail.rpctimeout 4000
> 18/04/05 07:24:01 INFO crail: crail.datatimeout 1000
> 18/04/05 07:24:01 INFO crail: crail.buffersize 524288
> 18/04/05 07:24:01 INFO crail: crail.slicesize 262144
> 18/04/05 07:24:01 INFO crail: crail.singleton true
> 18/04/05 07:24:01 INFO crail: crail.regionsize 1073741824
> 18/04/05 07:24:01 INFO crail: crail.directoryrecord 512
> 18/04/05 07:24:01 INFO crail: crail.directoryrandomize true
> 18/04/05 07:24:01 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 18/04/05 07:24:01 INFO crail: crail.locationmap
> 18/04/05 07:24:01 INFO crail: crail.namenode.address crail://
> 10.1.217.199:9070
> 18/04/05 07:24:01 INFO crail: crail.namenode.blockselection roundrobin
> 18/04/05 07:24:01 INFO crail: crail.namenode.fileblocks 16
> 18/04/05 07:24:01 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 18/04/05 07:24:01 INFO crail: crail.namenode.log
> 18/04/05 07:24:01 INFO crail: crail.storage.types
> com.ibm.crail.storage.object.ObjectStorageTier
> 18/04/05 07:24:01 INFO crail: crail.storage.classes 1
> 18/04/05 07:24:01 INFO crail: crail.storage.rootclass 0
> 18/04/05 07:24:01 INFO crail: crail.storage.keepalive 2
> 18/04/05 07:24:01 INFO crail: crail.storage.object.storagelimit = 1073741824
> 18/04/05 07:24:01 INFO crail: crail.storage.object.allocationsize = 1048576
> 18/04/05 07:24:01 INFO crail: crail.storage.object.datanode = localhost
> 18/04/05 07:24:01 INFO crail: crail.storage.object.datanodeport = 50080
> 18/04/05 07:24:01 INFO crail: crail.storage.object.objectprefix = crail
> 18/04/05 07:24:01 INFO crail: crail.storage.object.cleanuponexit = false
> 18/04/05 07:24:01 INFO crail: crail.storage.object.profile = false
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3accesskey = XXX
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3secretkey = XXXXX
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3region = us-west-2
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3bucket = anakli-pocket
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3endpoint =
> s3.us-west-2.amazonaws.com
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3protocol = HTTP
> 18/04/05 07:24:01 INFO crail: crail.storage.object.s3signer = null
> 18/04/05 07:24:01 INFO narpc: new NaRPC server group v1.0, queueDepth 1024,
> messageSize 512, nodealy true
> 18/04/05 07:24:01 INFO crail: crail.namenode.tcp.queueDepth 1024
> 18/04/05 07:24:01 INFO crail: crail.namenode.tcp.messageSize 512
> 18/04/05 07:24:01 INFO crail: crail.namenode.tcp.cores 1
> 18/04/05 07:24:01 INFO crail: connected to namenode(s) /10.1.217.199:9070
> 18/04/05 07:24:01 INFO crail: buffer cache, allocationCount 50, bufferCount
> 2048
> write, filename /test.data, size 4096, loop 1, storageClass 0,
> locationClass 0, buffered true
> warmUp, warmupFile /test.data-972347181, operations 0
> starting benchmark...
> 18/04/05 07:24:01 INFO crail: ERROR: failed data operation
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: localhost/127.0.0.1:50080
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257)
> at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:640)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> at java.lang.Thread.run(Thread.java:748)
> Exception in thread "main" java.util.concurrent.ExecutionException:
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: localhost/127.0.0.1:50080
> at
> org.apache.crail.core.CoreMetaDataOperation.get(CoreMetaDataOperation.java:98)
> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:108)
> at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1071)
> Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException:
> Connection refused: localhost/127.0.0.1:50080
>