You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crail.apache.org by Lou DeGenaro <lo...@gmail.com> on 2020/02/07 20:15:22 UTC
iobench
Still a noob. Got namenode and datanode running as docker images on my
VM. I'm looking for the simplest example of writing something then reading
something.
[root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f /filename
-s 1024 -k 1
20/02/07 14:10:22 INFO crail: creating singleton crail file system
20/02/07 14:10:23 INFO crail: crail.version 3101
20/02/07 14:10:23 INFO crail: crail.directorydepth 16
20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/07 14:10:23 INFO crail: crail.user crail
20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
20/02/07 14:10:23 INFO crail: crail.debug false
20/02/07 14:10:23 INFO crail: crail.statistics true
20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
20/02/07 14:10:23 INFO crail: crail.slicesize 524288
20/02/07 14:10:23 INFO crail: crail.singleton true
20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
20/02/07 14:10:23 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/07 14:10:23 INFO crail: crail.locationmap
20/02/07 14:10:23 INFO crail: crail.namenode.address crail://abbey-road:9060
20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/07 14:10:23 INFO crail: crail.namenode.log
20/02/07 14:10:23 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/07 14:10:23 INFO crail: crail.storage.classes 1
20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
10.114.222.23:9060
write, filename /filename, size 1024, loop 1, storageClass 0, locationClass
0, buffered true
Exception in thread "main" java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
at
org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
at org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
at
org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
... 5 more
RE: iobench
Posted by David Crespi <da...@storedgesystems.com>.
Lou,
Did you try the commands I gave you? That’s the simplest way of determining
what’s going on. You can also move files into the datastore using the fs command
set. That way you can see better what’s going on. It appears from the error that
you’ve exceeded the size of the memory (i.e. storage). I told you it was small 😊
You could try something smaller if you’d rather start with iobench perhaps.
Regards,
David
From: Lou DeGenaro<ma...@gmail.com>
Sent: Friday, February 7, 2020 12:15 PM
To: dev@crail.apache.org<ma...@crail.apache.org>
Subject: iobench
Still a noob. Got namenode and datanode running as docker images on my
VM. I'm looking for the simplest example of writing something then reading
something.
[root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f /filename
-s 1024 -k 1
20/02/07 14:10:22 INFO crail: creating singleton crail file system
20/02/07 14:10:23 INFO crail: crail.version 3101
20/02/07 14:10:23 INFO crail: crail.directorydepth 16
20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/07 14:10:23 INFO crail: crail.user crail
20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
20/02/07 14:10:23 INFO crail: crail.debug false
20/02/07 14:10:23 INFO crail: crail.statistics true
20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
20/02/07 14:10:23 INFO crail: crail.slicesize 524288
20/02/07 14:10:23 INFO crail: crail.singleton true
20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
20/02/07 14:10:23 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/07 14:10:23 INFO crail: crail.locationmap
20/02/07 14:10:23 INFO crail: crail.namenode.address crail://abbey-road:9060
20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/07 14:10:23 INFO crail: crail.namenode.log
20/02/07 14:10:23 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/07 14:10:23 INFO crail: crail.storage.classes 1
20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
10.114.222.23:9060
write, filename /filename, size 1024, loop 1, storageClass 0, locationClass
0, buffered true
Exception in thread "main" java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
at
org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
at org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
at
org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
... 5 more
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
For the record, docker containers now work by specifying DATAPATH and
CACHEPATH:
docker run -it --network host -e NAMENODE_HOST=sgt-pepper -e
DATAPATH=/tmp/hugepages/data -e CACHEPATH=/tmp/hugepages/cache
apache/incubator-crail:1.2 namenode
docker run -it --network host -e NAMENODE_HOST=sgt-pepper -e
DATAPATH=/tmp/hugepages/data -e CACHEPATH=/tmp/hugepages/cache
apache/incubator-crail:1.2 dataenode
And CLI downloaded, installed and configured from bin tgz worked just the
same as reported above.
I guess /dev was the blocker. But I'm mystified how I got crail to work
the first time when using /dev. Oh well, /tmp works and I'm going with
that for now...
Lou.
On Tue, Feb 11, 2020 at 10:36 AM David Crespi <
david.crespi@storedgesystems.com> wrote:
> Congrats and no worries… I doubt you’ll have a problem with docker.
> Happy Crailing 😉
>
> Regards,
> David
>
> From: Lou DeGenaro<ma...@gmail.com>
> Sent: Tuesday, February 11, 2020 7:03 AM
> To: dev@crail.apache.org<ma...@crail.apache.org>
> Subject: Re: iobench
>
> Success!
>
> [root@sgt-pepper ~]# $CRAIL_HOME/bin/crail fs -ls /foo
> ...
> Found 1 items
> -rw-rw-rw- 1 crail crail 1079 2020-02-11 08:58 /foo/favicon.png
>
> Thanks for the help and suggestions.
>
> Now to try all over again with docker...
>
> Lou.
>
>
> On Tue, Feb 11, 2020 at 9:47 AM Lou DeGenaro <lo...@gmail.com>
> wrote:
>
> > Changed the default in conf from /dev to /tmp as you suggest and that
> > helped!
> >
> > [root@sgt-pepper conf]# $CRAIL_HOME/bin/crail datanode
> > 20/02/11 08:43:36 INFO crail: crail.version 3101
> > 20/02/11 08:43:36 INFO crail: crail.directorydepth 16
> > 20/02/11 08:43:36 INFO crail: crail.tokenexpiration 10
> > 20/02/11 08:43:36 INFO crail: crail.blocksize 1048576
> > 20/02/11 08:43:36 INFO crail: crail.cachelimit 1073741824
> > 20/02/11 08:43:36 INFO crail: crail.cachepath /tmp/hugepages/cache
> > 20/02/11 08:43:36 INFO crail: crail.user crail
> > 20/02/11 08:43:36 INFO crail: crail.shadowreplication 1
> > 20/02/11 08:43:36 INFO crail: crail.debug false
> > 20/02/11 08:43:36 INFO crail: crail.statistics true
> > 20/02/11 08:43:36 INFO crail: crail.rpctimeout 1000
> > 20/02/11 08:43:36 INFO crail: crail.datatimeout 1000
> > 20/02/11 08:43:36 INFO crail: crail.buffersize 1048576
> > 20/02/11 08:43:36 INFO crail: crail.slicesize 524288
> > 20/02/11 08:43:36 INFO crail: crail.singleton true
> > 20/02/11 08:43:36 INFO crail: crail.regionsize 1073741824
> > 20/02/11 08:43:36 INFO crail: crail.directoryrecord 512
> > 20/02/11 08:43:36 INFO crail: crail.directoryrandomize true
> > 20/02/11 08:43:36 INFO crail: crail.cacheimpl
> > org.apache.crail.memory.MappedBufferCache
> > 20/02/11 08:43:36 INFO crail: crail.locationmap
> > 20/02/11 08:43:36 INFO crail: crail.namenode.address
> > crail://sgt-pepper:9060
> > 20/02/11 08:43:36 INFO crail: crail.namenode.blockselection roundrobin
> > 20/02/11 08:43:36 INFO crail: crail.namenode.fileblocks 16
> > 20/02/11 08:43:36 INFO crail: crail.namenode.rpctype
> > org.apache.crail.namenode.rpc.tcp.TcpNameNode
> > 20/02/11 08:43:36 INFO crail: crail.namenode.log
> > 20/02/11 08:43:36 INFO crail: crail.storage.types
> > org.apache.crail.storage.tcp.TcpStorageTier
> > 20/02/11 08:43:36 INFO crail: crail.storage.classes 1
> > 20/02/11 08:43:36 INFO crail: crail.storage.rootclass 0
> > 20/02/11 08:43:36 INFO crail: crail.storage.keepalive 2
> > 20/02/11 08:43:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> > 16, messageSize 2097152, nodealy false, cores 1
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.interface eth0
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.port 50020
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.datapath
> > /tmp/hugepages/data
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.queuedepth 16
> > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.cores 1
> > 20/02/11 08:43:36 INFO crail: running TCP storage server, address /
> > 10.114.222.23:50020
> > 20/02/11 08:43:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> > 32, messageSize 512, nodealy true
> > 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.queueDepth 32
> > 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.messageSize 512
> > 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.cores 1
> > 20/02/11 08:43:36 INFO crail: connected to namenode(s) sgt-pepper/
> > 10.114.222.23:9060
> > 20/02/11 08:43:36 INFO crail: datanode statistics, freeBlocks 1024
> > 20/02/11 08:43:37 INFO crail: datanode statistics, freeBlocks 1024
> > 20/02/11 08:43:39 INFO crail: datanode statistics, freeBlocks 1024
> > 20/02/11 08:43:41 INFO crail: datanode statistics, freeBlocks 1024
> > 20/02/11 08:43:43 INFO crail: datanode statistics, freeBlocks 1024
> > ...
> >
> > Now to see if I can use the CLI...
> >
> > Lou.
> >
> >
> >
> > On Tue, Feb 11, 2020 at 9:40 AM Lou DeGenaro <lo...@gmail.com>
> > wrote:
> >
> >> ...and if it makes any difference:
> >>
> >> [root@sgt-pepper conf]# which java
> >> /usr/bin/java
> >> [root@sgt-pepper conf]# java -version
> >> openjdk version "1.8.0_242"
> >> OpenJDK Runtime Environment (build 1.8.0_242-b08)
> >> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
> >>
> >> [root@sgt-pepper ~]# vmstat -s
> >> 8003404 K total memory
> >> 906212 K used memory
> >> 1658432 K active memory
> >> 2281696 K inactive memory
> >> 3692808 K free memory
> >> 123776 K buffer memory
> >> 3280608 K swap cache
> >> 0 K total swap
> >> 0 K used swap
> >> 0 K free swap
> >> 59564 non-nice user cpu ticks
> >> 359 nice user cpu ticks
> >> 22392 system cpu ticks
> >> 1165850 idle cpu ticks
> >> 9712 IO-wait cpu ticks
> >> 0 IRQ cpu ticks
> >> 1288 softirq cpu ticks
> >> 7522 stolen cpu ticks
> >> 313554 pages paged in
> >> 4620824 pages paged out
> >> 0 pages swapped in
> >> 0 pages swapped out
> >> 12475579 interrupts
> >> 20632577 CPU context switches
> >> 1581425491 boot time
> >> 86760 forks
> >>
> >> Lou.
> >>
> >> On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lo...@gmail.com>
> >> wrote:
> >>
> >>> I've now removed docker from the picture. I simply download the bin
> >>> distribution onto my 8GB VM and copy the templates in conf w/o changes.
> >>> ======
> >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> >>> namenode
> >>> 20/02/11 08:09:08 INFO crail: initalizing namenode
> >>> 20/02/11 08:09:08 INFO crail: crail.version 3101
> >>> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16
> >>> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
> >>> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
> >>> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
> >>> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
> >>> 20/02/11 08:09:08 INFO crail: crail.user crail
> >>> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
> >>> 20/02/11 08:09:08 INFO crail: crail.debug false
> >>> 20/02/11 08:09:08 INFO crail: crail.statistics true
> >>> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
> >>> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
> >>> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
> >>> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288
> >>> 20/02/11 08:09:08 INFO crail: crail.singleton true
> >>> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
> >>> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
> >>> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
> >>> 20/02/11 08:09:08 INFO crail: crail.cacheimpl
> >>> org.apache.crail.memory.MappedBufferCache
> >>> 20/02/11 08:09:08 INFO crail: crail.locationmap
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.address
> >>> crail://localhost:9060?id=0&size=1
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
> >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.log
> >>> 20/02/11 08:09:08 INFO crail: crail.storage.types
> >>> org.apache.crail.storage.tcp.TcpStorageTier
> >>> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1
> >>> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
> >>> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
> >>> 20/02/11 08:09:08 INFO crail: round robin block selection
> >>> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> >>> 32, messageSize 512, nodealy true, cores 1
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
> >>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
> >>> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
> >>> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
> >>> 127.0.0.1:40578
> >>> =====
> >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> >>> datanode
> >>> 20/02/11 08:11:28 INFO crail: crail.version 3101
> >>> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16
> >>> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
> >>> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
> >>> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
> >>> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
> >>> 20/02/11 08:11:28 INFO crail: crail.user crail
> >>> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
> >>> 20/02/11 08:11:28 INFO crail: crail.debug false
> >>> 20/02/11 08:11:28 INFO crail: crail.statistics true
> >>> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
> >>> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
> >>> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
> >>> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288
> >>> 20/02/11 08:11:28 INFO crail: crail.singleton true
> >>> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
> >>> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
> >>> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
> >>> 20/02/11 08:11:28 INFO crail: crail.cacheimpl
> >>> org.apache.crail.memory.MappedBufferCache
> >>> 20/02/11 08:11:28 INFO crail: crail.locationmap
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.address
> >>> crail://localhost:9060
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
> >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.log
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.types
> >>> org.apache.crail.storage.tcp.TcpStorageTier
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
> >>> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> >>> 16, messageSize 2097152, nodealy false, cores 1
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize
> 1073741824
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath
> >>> /dev/hugepages/data
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
> >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
> >>> 20/02/11 08:11:28 INFO crail: running TCP storage server, address /
> >>> 10.114.222.23:50020
> >>> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> >>> 32, messageSize 512, nodealy true
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
> >>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
> >>> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
> >>> 127.0.0.1:9060
> >>> Exception in thread "main" java.io.IOException: Map failed
> >>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> >>> at
> >>>
> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
> >>> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
> >>> Caused by: java.lang.OutOfMemoryError: Map failed
> >>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> >>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> >>> ... 2 more
> >>> =====
> >>> Sorry for being a noob. Thx for the help.
> >>>
> >>> Lou.
> >>>
> >>> On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
> >>> david.crespi@storedgesystems.com> wrote:
> >>>
> >>>> Was does your core-site and crail-site look like.
> >>>> At first thought, it may not be resolving “sgt-pepper”.
> >>>> Why not try the actual IP address to eliminate.
> >>>>
> >>>> Regards,
> >>>> David
> >>>>
> >>>> (C) 714-476-2692
> >>>>
> >>>> From: Lou DeGenaro<ma...@gmail.com>
> >>>> Sent: Monday, February 10, 2020 1:12 PM
> >>>> To: dev@crail.apache.org<ma...@crail.apache.org>
> >>>> Subject: Re: iobench
> >>>>
> >>>> I seem to have gone backwards? I started over with new VM. I
> started 2
> >>>> docker containers, one each for name and data:
> >>>>
> >>>> [root@sgt-pepper ~]# docker run -it --network host -e
> >>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
> >>>> 20/02/10 20:32:44 INFO crail: initalizing namenode
> >>>> 20/02/10 20:32:44 INFO crail: crail.version 3101
> >>>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
> >>>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
> >>>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
> >>>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
> >>>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
> >>>> 20/02/10 20:32:44 INFO crail: crail.user crail
> >>>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
> >>>> 20/02/10 20:32:44 INFO crail: crail.debug false
> >>>> 20/02/10 20:32:44 INFO crail: crail.statistics true
> >>>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
> >>>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
> >>>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
> >>>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
> >>>> 20/02/10 20:32:44 INFO crail: crail.singleton true
> >>>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
> >>>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
> >>>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
> >>>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
> >>>> org.apache.crail.memory.MappedBufferCache
> >>>> 20/02/10 20:32:44 INFO crail: crail.locationmap
> >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.address
> >>>> crail://sgt-pepper:9060?id=0&size=1
> >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
> >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
> >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
> >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.log
> >>>> 20/02/10 20:32:44 INFO crail: crail.storage.types
> >>>> org.apache.crail.storage.tcp.TcpStorageTier
> >>>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
> >>>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
> >>>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
> >>>> 20/02/10 20:32:44 INFO crail: round robin block selection
> >>>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0,
> queueDepth
> >>>> 32,
> >>>> messageSize 512, nodealy true, cores 1
> >>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
> >>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
> >>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
> >>>> 20/02/10 20:35:36 INFO crail: new connection from /
> 10.114.222.82:37328
> >>>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
> >>>> 10.114.222.82:37328
> >>>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of
> >>>> type
> >>>> 0 to storage class 0
> >>>>
> >>>> [root@sgt-pepper ~]# docker run -it --network host -e
> >>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
> >>>> 20/02/10 20:35:36 INFO crail: crail.version 3101
> >>>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
> >>>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
> >>>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
> >>>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
> >>>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
> >>>> 20/02/10 20:35:36 INFO crail: crail.user crail
> >>>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
> >>>> 20/02/10 20:35:36 INFO crail: crail.debug false
> >>>> 20/02/10 20:35:36 INFO crail: crail.statistics true
> >>>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
> >>>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
> >>>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
> >>>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
> >>>> 20/02/10 20:35:36 INFO crail: crail.singleton true
> >>>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
> >>>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
> >>>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
> >>>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
> >>>> org.apache.crail.memory.MappedBufferCache
> >>>> 20/02/10 20:35:36 INFO crail: crail.locationmap
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.address
> >>>> crail://sgt-pepper:9060
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
> >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.log
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.types
> >>>> org.apache.crail.storage.tcp.TcpStorageTier
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
> >>>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0,
> queueDepth
> >>>> 16,
> >>>> messageSize 2097152, nodealy false, cores 1
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit
> 1073741824
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize
> >>>> 1073741824
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
> >>>> /dev/hugepages/data
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
> >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
> >>>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
> >>>> 10.114.222.82:50020
> >>>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 32,
> >>>> messageSize 512, nodealy true
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
> >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
> >>>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
> >>>> 10.114.222.82:9060
> >>>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
> >>>> ...
> >>>>
> >>>> This seems to be OK so far, since we see *adding datanode
> >>>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to
> storage
> >>>> class 0* in the namenode log.
> >>>>
> >>>> Then I download the Non-Official Binary Release from here:
> >>>> https://crail.apache.org/download/ and I extract and create the 4
> conf
> >>>> files and the revised log4j properties file:
> >>>> [root@sgt-pepper conf]# ls -atl
> >>>> total 40
> >>>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
> >>>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
> >>>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
> >>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
> >>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
> >>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
> >>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
> >>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
> >>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
> >>>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
> >>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
> >>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
> >>>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
> >>>> 2c2
> >>>> < log4j.rootCategory=DEBUG, console
> >>>> ---
> >>>> > log4j.rootCategory=INFO, console
> >>>>
> >>>> I setup CRAIL_HOME:
> >>>>
> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
> >>>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
> >>>>
> >>>> =====
> >>>>
> >>>> Now I try using the CLI:
> >>>>
> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> >>>> fs -ls
> >>>> /
> >>>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
> >>>> directory
> >>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> >>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> >>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> >>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> >>>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
> >>>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
> >>>>
> >>>>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> >>>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
> >>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> >>>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
> >>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> >>>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
> >>>> java.io.BufferedInputStream@71c7db30
> >>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>>
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[Rate
> >>>> of
> >>>> successful kerberos logins and latency (milliseconds)])
> >>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>>
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[Rate
> >>>> of
> >>>> failed kerberos logins and latency (milliseconds)])
> >>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
> >>>> with
> >>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> >>>> value=[GetGroups])
> >>>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> >>>> related metrics
> >>>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
> >>>> found, setting default realm to empty
> >>>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
> >>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the
> >>>> custom-built
> >>>> native-hadoop library...
> >>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop
> >>>> with
> >>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> >>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
> >>>>
> >>>>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >>>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
> >>>> library for your platform... using builtin-java classes where
> applicable
> >>>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell
> based
> >>>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> >>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> >>>> 20/02/10 14:54:52 DEBUG Groups: Group mapping
> >>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> >>>> cacheTimeout=300000; warningDeltaMs=5000
> >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
> >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
> >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
> >>>> user:UnixPrincipal: root
> >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user:
> >>>> "UnixPrincipal:
> >>>> root" with name root
> >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
> >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
> >>>> (auth:SIMPLE)
> >>>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
> >>>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
> >>>> 20/02/10 14:54:52 INFO crail: crail.version 3101
> >>>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
> >>>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
> >>>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
> >>>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
> >>>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
> >>>> 20/02/10 14:54:52 INFO crail: crail.user crail
> >>>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
> >>>> 20/02/10 14:54:52 INFO crail: crail.debug false
> >>>> 20/02/10 14:54:52 INFO crail: crail.statistics true
> >>>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
> >>>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
> >>>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
> >>>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
> >>>> 20/02/10 14:54:52 INFO crail: crail.singleton true
> >>>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
> >>>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
> >>>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
> >>>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
> >>>> org.apache.crail.memory.MappedBufferCache
> >>>> 20/02/10 14:54:52 INFO crail: crail.locationmap
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.address
> >>>> crail://localhost:9060
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
> >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.log
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.types
> >>>> org.apache.crail.storage.tcp.TcpStorageTier
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
> >>>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1,
> >>>> bufferCount
> >>>> 1024
> >>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 16,
> >>>> messageSize 2097152, nodealy false
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit
> 1073741824
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize
> >>>> 1073741824
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
> >>>> /dev/hugepages/data
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
> >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
> >>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 32,
> >>>> messageSize 512, nodealy true
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
> >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
> >>>> ls: java.io.IOException: java.net.ConnectException: Connection refused
> >>>>
> >>>> =====
> >>>>
> >>>> Since I get connection refused, I try changing "localhost" to
> >>>> "sgt-pepper"
> >>>> in the conf files:
> >>>>
> >>>> [root@sgt-pepper conf]# grep sgt-pepper *
> >>>> core-site.xml: <value>crail://sgt-pepper:9060</value>
> >>>> crail-site.conf:crail.namenode.address
> >>>> crail://sgt-pepper:9060
> >>>>
> >>>> Then I try using the CLI again:
> >>>>
> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> >>>> fs -ls
> >>>> /
> >>>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
> >>>> directory
> >>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> >>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> >>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> >>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> >>>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
> >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
> >>>>
> >>>>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
> >>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
> >>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
> >>>> java.io.BufferedInputStream@71c7db30
> >>>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>>
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[Rate
> >>>> of
> >>>> successful kerberos logins and latency (milliseconds)])
> >>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>>
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[Rate
> >>>> of
> >>>> failed kerberos logins and latency (milliseconds)])
> >>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
> >>>> with
> >>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> >>>> value=[GetGroups])
> >>>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> >>>> related metrics
> >>>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
> >>>> found, setting default realm to empty
> >>>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
> >>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the
> >>>> custom-built
> >>>> native-hadoop library...
> >>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop
> >>>> with
> >>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> >>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
> >>>>
> >>>>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >>>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
> >>>> library for your platform... using builtin-java classes where
> applicable
> >>>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell
> based
> >>>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> >>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> >>>> 20/02/10 15:00:28 DEBUG Groups: Group mapping
> >>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> >>>> cacheTimeout=300000; warningDeltaMs=5000
> >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
> >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
> >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
> >>>> user:UnixPrincipal: root
> >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user:
> >>>> "UnixPrincipal:
> >>>> root" with name root
> >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
> >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
> >>>> (auth:SIMPLE)
> >>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
> >>>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
> >>>> 20/02/10 15:00:28 INFO crail: crail.version 3101
> >>>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
> >>>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
> >>>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
> >>>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
> >>>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
> >>>> 20/02/10 15:00:28 INFO crail: crail.user crail
> >>>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
> >>>> 20/02/10 15:00:28 INFO crail: crail.debug false
> >>>> 20/02/10 15:00:28 INFO crail: crail.statistics true
> >>>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
> >>>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
> >>>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
> >>>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
> >>>> 20/02/10 15:00:28 INFO crail: crail.singleton true
> >>>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
> >>>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
> >>>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
> >>>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
> >>>> org.apache.crail.memory.MappedBufferCache
> >>>> 20/02/10 15:00:28 INFO crail: crail.locationmap
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.address
> >>>> crail://sgt-pepper:9060
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
> >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.log
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.types
> >>>> org.apache.crail.storage.tcp.TcpStorageTier
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
> >>>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1,
> >>>> bufferCount
> >>>> 1024
> >>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 16,
> >>>> messageSize 2097152, nodealy false
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit
> 1073741824
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize
> >>>> 1073741824
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
> >>>> /dev/hugepages/data
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
> >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
> >>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 32,
> >>>> messageSize 512, nodealy true
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
> >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
> >>>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
> >>>> 10.114.222.82:9060
> >>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
> >>>> done..
> >>>> ls: /
> >>>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
> >>>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
> >>>> 20/02/10 15:00:28 INFO crail: mapped client cache closed
> >>>>
> >>>> Seem better, but not really.
> >>>>
> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> fs
> >>>> -mkdir /foobar
> >>>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
> >>>> directory
> >>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> >>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> >>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> >>>> at
> >>>>
> >>>>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> >>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> >>>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
> >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
> >>>>
> >>>>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
> >>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
> >>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
> >>>> java.io.BufferedInputStream@71c7db30
> >>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>>
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[Rate
> >>>> of
> >>>> successful kerberos logins and latency (milliseconds)])
> >>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>>
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[Rate
> >>>> of
> >>>> failed kerberos logins and latency (milliseconds)])
> >>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> >>>> org.apache.hadoop.metrics2.lib.MutableRate
> >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
> >>>> with
> >>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> >>>> value=[GetGroups])
> >>>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> >>>> related metrics
> >>>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
> >>>> found, setting default realm to empty
> >>>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
> >>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the
> >>>> custom-built
> >>>> native-hadoop library...
> >>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop
> >>>> with
> >>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> >>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
> >>>>
> >>>>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >>>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
> >>>> library for your platform... using builtin-java classes where
> applicable
> >>>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell
> based
> >>>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> >>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> >>>> 20/02/10 15:06:14 DEBUG Groups: Group mapping
> >>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> >>>> cacheTimeout=300000; warningDeltaMs=5000
> >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
> >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
> >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
> >>>> user:UnixPrincipal: root
> >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user:
> >>>> "UnixPrincipal:
> >>>> root" with name root
> >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
> >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
> >>>> (auth:SIMPLE)
> >>>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
> >>>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
> >>>> 20/02/10 15:06:14 INFO crail: crail.version 3101
> >>>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
> >>>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
> >>>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
> >>>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
> >>>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
> >>>> 20/02/10 15:06:14 INFO crail: crail.user crail
> >>>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
> >>>> 20/02/10 15:06:14 INFO crail: crail.debug false
> >>>> 20/02/10 15:06:14 INFO crail: crail.statistics true
> >>>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
> >>>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
> >>>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
> >>>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
> >>>> 20/02/10 15:06:14 INFO crail: crail.singleton true
> >>>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
> >>>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
> >>>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
> >>>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
> >>>> org.apache.crail.memory.MappedBufferCache
> >>>> 20/02/10 15:06:14 INFO crail: crail.locationmap
> >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.address
> >>>> crail://sgt-pepper:9060
> >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
> >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
> >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
> >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.log
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.types
> >>>> org.apache.crail.storage.tcp.TcpStorageTier
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
> >>>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1,
> >>>> bufferCount
> >>>> 1024
> >>>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 16,
> >>>> messageSize 2097152, nodealy false
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit
> 1073741824
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize
> >>>> 1073741824
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
> >>>> /dev/hugepages/data
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
> >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
> >>>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> >>>> 32,
> >>>> messageSize 512, nodealy true
> >>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
> >>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
> >>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
> >>>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
> >>>> 10.114.222.82:9060
> >>>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
> >>>> done..
> >>>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException:
> Map
> >>>> failed
> >>>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
> >>>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
> >>>> 20/02/10 15:06:15 INFO crail: mapped client cache closed
> >>>>
> >>>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
> >>>> java.io.IOException: Map failed *is not good.
> >>>>
> >>>> Sorry for the long append. Probably something really dumb, but I'm
> >>>> sratching my head...
> >>>>
> >>>> Lou.
> >>>>
> >>>>
> >>>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
> >>>> david.crespi@storedgesystems.com> wrote:
> >>>>
> >>>> > Looks like both cmds are working, but it’s not really finding the
> >>>> > datastore.
> >>>> > You’re getting info into the namenode, but not the datanode (which
> >>>> means
> >>>> > you’re really not writing to the datanode). The
> >>>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
> >>>> > native-hadoop library for your platform)
> >>>> >
> >>>> > you may want to turn on debug to see more of what’s going on.
> >>>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
> >>>> >
> >>>> > You should probably also look at the individual docker logs.
> >>>> > docker logs YourNamenodeName & docker logs YourDatanodeName
> >>>> >
> >>>> > may give you some more hints of what’s happening.
> >>>> >
> >>>> >
> >>>> > Regards,
> >>>> > David
> >>>> >
> >>>> >
> >>>> > From: Lou DeGenaro<ma...@gmail.com>
> >>>> > Sent: Monday, February 10, 2020 8:12 AM
> >>>> > To: dev@crail.apache.org<ma...@crail.apache.org>
> >>>> > Subject: Re: iobench
> >>>> >
> >>>> > OK, I take it back. Having trouble with crail fs.
> >>>> >
> >>>> > =====
> >>>> >
> >>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> >>>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load
> native-hadoop
> >>>> > library for your platform... using builtin-java classes where
> >>>> applicable
> >>>> > ...
> >>>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
> >>>> > 10.114.222.23:9060
> >>>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs
> initialization
> >>>> > done..
> >>>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException:
> >>>> Map
> >>>> > failed
> >>>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
> >>>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
> >>>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
> >>>> >
> >>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
> >>>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load
> native-hadoop
> >>>> > library for your platform... using builtin-java classes where
> >>>> applicable
> >>>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs
> >>>> initialization
> >>>> > done..
> >>>> > rmdir: /foobar
> >>>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
> >>>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
> >>>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
> >>>> >
> >>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> >>>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load
> native-hadoop
> >>>> > library for your platform... using builtin-java classes where
> >>>> applicable
> >>>> > ...
> >>>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs
> initialization
> >>>> > done..
> >>>> > mkdir: `/foobar': File exists
> >>>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
> >>>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
> >>>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
> >>>> >
> >>>> > =====
> >>>> >
> >>>> > Seems that mkdir works, but rmdir does not? Also, IOException was
> >>>> reported
> >>>> > on mkdir, which seems worrisome.
> >>>> >
> >>>> > Lou.
> >>>> >
> >>>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <
> lou.degenaro@gmail.com>
> >>>> > wrote:
> >>>> >
> >>>> > > David,
> >>>> > >
> >>>> > > Thanks. I'm able to use *crail fs *successfully to
> >>>> create/list/delete.
> >>>> > > Will try to increase available storage space next...
> >>>> > >
> >>>> > > Lou.
> >>>> > >
> >>>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> >>>> > > david.crespi@storedgesystems.com> wrote:
> >>>> > >
> >>>> > >> Oh, and it may have already written something on the datastore,
> so
> >>>> you’d
> >>>> > >> need
> >>>> > >> to check it and remove it with the fs command anyway.
> >>>> > >>
> >>>> > >> crail fs -ls -R /
> >>>> > >>
> >>>> > >> would show you what you have there.
> >>>> > >>
> >>>> > >> Regards,
> >>>> > >> David
> >>>> > >>
> >>>> > >> (C) 714-476-2692
> >>>> > >>
> >>>> > >> From: Lou DeGenaro<ma...@gmail.com>
> >>>> > >> Sent: Friday, February 7, 2020 12:15 PM
> >>>> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
> >>>> > >> Subject: iobench
> >>>> > >>
> >>>> > >> Still a noob. Got namenode and datanode running as docker images
> >>>> on my
> >>>> > >> VM. I'm looking for the simplest example of writing something
> then
> >>>> > >> reading
> >>>> > >> something.
> >>>> > >>
> >>>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write
> -f
> >>>> > >> /filename
> >>>> > >> -s 1024 -k 1
> >>>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file
> system
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath
> /dev/hugepages/cache
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
> >>>> > >> org.apache.crail.memory.MappedBufferCache
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
> >>>> > >> crail://abbey-road:9060
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection
> >>>> roundrobin
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
> >>>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
> >>>> > >> org.apache.crail.storage.tcp.TcpStorageTier
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
> >>>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
> >>>> > bufferCount
> >>>> > >> 1024
> >>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
> >>>> queueDepth
> >>>> > >> 16,
> >>>> > >> messageSize 2097152, nodealy false
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
> >>>> 1073741824
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
> >>>> > 1073741824
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
> >>>> > >> /dev/hugepages/data
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
> >>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
> >>>> queueDepth
> >>>> > >> 32,
> >>>> > >> messageSize 512, nodealy true
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
> >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
> >>>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s)
> abbey-road/
> >>>> > >> 10.114.222.23:9060
> >>>> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
> >>>> > >> locationClass
> >>>> > >> 0, buffered true
> >>>> > >> Exception in thread "main" java.io.IOException: Map failed
> >>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> >>>> > >> at
> >>>> > >>
> >>>> > >>
> >>>> >
> >>>>
> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
> >>>> > >> at
> >>>> >
> >>>>
> org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
> >>>> > >> at
> >>>> > >>
> >>>> >
> >>>>
> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
> >>>> > >> at
> >>>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
> >>>> > >> at
> >>>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
> >>>> > >> Caused by: java.lang.OutOfMemoryError: Map failed
> >>>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> >>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> >>>> > >> ... 5 more
> >>>> > >>
> >>>> > >>
> >>>> >
> >>>> >
> >>>>
> >>>>
>
>
RE: iobench
Posted by David Crespi <da...@storedgesystems.com>.
Congrats and no worries… I doubt you’ll have a problem with docker.
Happy Crailing 😉
Regards,
David
From: Lou DeGenaro<ma...@gmail.com>
Sent: Tuesday, February 11, 2020 7:03 AM
To: dev@crail.apache.org<ma...@crail.apache.org>
Subject: Re: iobench
Success!
[root@sgt-pepper ~]# $CRAIL_HOME/bin/crail fs -ls /foo
...
Found 1 items
-rw-rw-rw- 1 crail crail 1079 2020-02-11 08:58 /foo/favicon.png
Thanks for the help and suggestions.
Now to try all over again with docker...
Lou.
On Tue, Feb 11, 2020 at 9:47 AM Lou DeGenaro <lo...@gmail.com> wrote:
> Changed the default in conf from /dev to /tmp as you suggest and that
> helped!
>
> [root@sgt-pepper conf]# $CRAIL_HOME/bin/crail datanode
> 20/02/11 08:43:36 INFO crail: crail.version 3101
> 20/02/11 08:43:36 INFO crail: crail.directorydepth 16
> 20/02/11 08:43:36 INFO crail: crail.tokenexpiration 10
> 20/02/11 08:43:36 INFO crail: crail.blocksize 1048576
> 20/02/11 08:43:36 INFO crail: crail.cachelimit 1073741824
> 20/02/11 08:43:36 INFO crail: crail.cachepath /tmp/hugepages/cache
> 20/02/11 08:43:36 INFO crail: crail.user crail
> 20/02/11 08:43:36 INFO crail: crail.shadowreplication 1
> 20/02/11 08:43:36 INFO crail: crail.debug false
> 20/02/11 08:43:36 INFO crail: crail.statistics true
> 20/02/11 08:43:36 INFO crail: crail.rpctimeout 1000
> 20/02/11 08:43:36 INFO crail: crail.datatimeout 1000
> 20/02/11 08:43:36 INFO crail: crail.buffersize 1048576
> 20/02/11 08:43:36 INFO crail: crail.slicesize 524288
> 20/02/11 08:43:36 INFO crail: crail.singleton true
> 20/02/11 08:43:36 INFO crail: crail.regionsize 1073741824
> 20/02/11 08:43:36 INFO crail: crail.directoryrecord 512
> 20/02/11 08:43:36 INFO crail: crail.directoryrandomize true
> 20/02/11 08:43:36 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/11 08:43:36 INFO crail: crail.locationmap
> 20/02/11 08:43:36 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/11 08:43:36 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/11 08:43:36 INFO crail: crail.namenode.fileblocks 16
> 20/02/11 08:43:36 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/11 08:43:36 INFO crail: crail.namenode.log
> 20/02/11 08:43:36 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/11 08:43:36 INFO crail: crail.storage.classes 1
> 20/02/11 08:43:36 INFO crail: crail.storage.rootclass 0
> 20/02/11 08:43:36 INFO crail: crail.storage.keepalive 2
> 20/02/11 08:43:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> 16, messageSize 2097152, nodealy false, cores 1
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.interface eth0
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.port 50020
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.datapath
> /tmp/hugepages/data
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.cores 1
> 20/02/11 08:43:36 INFO crail: running TCP storage server, address /
> 10.114.222.23:50020
> 20/02/11 08:43:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> 32, messageSize 512, nodealy true
> 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.cores 1
> 20/02/11 08:43:36 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.23:9060
> 20/02/11 08:43:36 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:37 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:39 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:41 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:43 INFO crail: datanode statistics, freeBlocks 1024
> ...
>
> Now to see if I can use the CLI...
>
> Lou.
>
>
>
> On Tue, Feb 11, 2020 at 9:40 AM Lou DeGenaro <lo...@gmail.com>
> wrote:
>
>> ...and if it makes any difference:
>>
>> [root@sgt-pepper conf]# which java
>> /usr/bin/java
>> [root@sgt-pepper conf]# java -version
>> openjdk version "1.8.0_242"
>> OpenJDK Runtime Environment (build 1.8.0_242-b08)
>> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
>>
>> [root@sgt-pepper ~]# vmstat -s
>> 8003404 K total memory
>> 906212 K used memory
>> 1658432 K active memory
>> 2281696 K inactive memory
>> 3692808 K free memory
>> 123776 K buffer memory
>> 3280608 K swap cache
>> 0 K total swap
>> 0 K used swap
>> 0 K free swap
>> 59564 non-nice user cpu ticks
>> 359 nice user cpu ticks
>> 22392 system cpu ticks
>> 1165850 idle cpu ticks
>> 9712 IO-wait cpu ticks
>> 0 IRQ cpu ticks
>> 1288 softirq cpu ticks
>> 7522 stolen cpu ticks
>> 313554 pages paged in
>> 4620824 pages paged out
>> 0 pages swapped in
>> 0 pages swapped out
>> 12475579 interrupts
>> 20632577 CPU context switches
>> 1581425491 boot time
>> 86760 forks
>>
>> Lou.
>>
>> On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lo...@gmail.com>
>> wrote:
>>
>>> I've now removed docker from the picture. I simply download the bin
>>> distribution onto my 8GB VM and copy the templates in conf w/o changes.
>>> ======
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>> namenode
>>> 20/02/11 08:09:08 INFO crail: initalizing namenode
>>> 20/02/11 08:09:08 INFO crail: crail.version 3101
>>> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16
>>> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
>>> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
>>> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
>>> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/11 08:09:08 INFO crail: crail.user crail
>>> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
>>> 20/02/11 08:09:08 INFO crail: crail.debug false
>>> 20/02/11 08:09:08 INFO crail: crail.statistics true
>>> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
>>> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
>>> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
>>> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288
>>> 20/02/11 08:09:08 INFO crail: crail.singleton true
>>> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
>>> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
>>> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
>>> 20/02/11 08:09:08 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/11 08:09:08 INFO crail: crail.locationmap
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.address
>>> crail://localhost:9060?id=0&size=1
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.log
>>> 20/02/11 08:09:08 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1
>>> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
>>> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
>>> 20/02/11 08:09:08 INFO crail: round robin block selection
>>> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>> 32, messageSize 512, nodealy true, cores 1
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
>>> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
>>> 127.0.0.1:40578
>>> =====
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>> datanode
>>> 20/02/11 08:11:28 INFO crail: crail.version 3101
>>> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16
>>> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
>>> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
>>> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/11 08:11:28 INFO crail: crail.user crail
>>> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
>>> 20/02/11 08:11:28 INFO crail: crail.debug false
>>> 20/02/11 08:11:28 INFO crail: crail.statistics true
>>> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
>>> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
>>> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
>>> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288
>>> 20/02/11 08:11:28 INFO crail: crail.singleton true
>>> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
>>> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
>>> 20/02/11 08:11:28 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/11 08:11:28 INFO crail: crail.locationmap
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.address
>>> crail://localhost:9060
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.log
>>> 20/02/11 08:11:28 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1
>>> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
>>> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
>>> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>> 16, messageSize 2097152, nodealy false, cores 1
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath
>>> /dev/hugepages/data
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
>>> 20/02/11 08:11:28 INFO crail: running TCP storage server, address /
>>> 10.114.222.23:50020
>>> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 32, messageSize 512, nodealy true
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
>>> 127.0.0.1:9060
>>> Exception in thread "main" java.io.IOException: Map failed
>>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>>> at
>>> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
>>> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
>>> Caused by: java.lang.OutOfMemoryError: Map failed
>>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>>> ... 2 more
>>> =====
>>> Sorry for being a noob. Thx for the help.
>>>
>>> Lou.
>>>
>>> On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
>>> david.crespi@storedgesystems.com> wrote:
>>>
>>>> Was does your core-site and crail-site look like.
>>>> At first thought, it may not be resolving “sgt-pepper”.
>>>> Why not try the actual IP address to eliminate.
>>>>
>>>> Regards,
>>>> David
>>>>
>>>> (C) 714-476-2692
>>>>
>>>> From: Lou DeGenaro<ma...@gmail.com>
>>>> Sent: Monday, February 10, 2020 1:12 PM
>>>> To: dev@crail.apache.org<ma...@crail.apache.org>
>>>> Subject: Re: iobench
>>>>
>>>> I seem to have gone backwards? I started over with new VM. I started 2
>>>> docker containers, one each for name and data:
>>>>
>>>> [root@sgt-pepper ~]# docker run -it --network host -e
>>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
>>>> 20/02/10 20:32:44 INFO crail: initalizing namenode
>>>> 20/02/10 20:32:44 INFO crail: crail.version 3101
>>>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
>>>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
>>>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 20:32:44 INFO crail: crail.user crail
>>>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 20:32:44 INFO crail: crail.debug false
>>>> 20/02/10 20:32:44 INFO crail: crail.statistics true
>>>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
>>>> 20/02/10 20:32:44 INFO crail: crail.singleton true
>>>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 20:32:44 INFO crail: crail.locationmap
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060?id=0&size=1
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.log
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 20:32:44 INFO crail: round robin block selection
>>>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true, cores 1
>>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
>>>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
>>>> 10.114.222.82:37328
>>>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of
>>>> type
>>>> 0 to storage class 0
>>>>
>>>> [root@sgt-pepper ~]# docker run -it --network host -e
>>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
>>>> 20/02/10 20:35:36 INFO crail: crail.version 3101
>>>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
>>>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
>>>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 20:35:36 INFO crail: crail.user crail
>>>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 20:35:36 INFO crail: crail.debug false
>>>> 20/02/10 20:35:36 INFO crail: crail.statistics true
>>>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
>>>> 20/02/10 20:35:36 INFO crail: crail.singleton true
>>>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 20:35:36 INFO crail: crail.locationmap
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.log
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false, cores 1
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
>>>> 10.114.222.82:50020
>>>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
>>>> 10.114.222.82:9060
>>>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
>>>> ...
>>>>
>>>> This seems to be OK so far, since we see *adding datanode
>>>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
>>>> class 0* in the namenode log.
>>>>
>>>> Then I download the Non-Official Binary Release from here:
>>>> https://crail.apache.org/download/ and I extract and create the 4 conf
>>>> files and the revised log4j properties file:
>>>> [root@sgt-pepper conf]# ls -atl
>>>> total 40
>>>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
>>>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
>>>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
>>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
>>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
>>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
>>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
>>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
>>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
>>>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
>>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
>>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
>>>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
>>>> 2c2
>>>> < log4j.rootCategory=DEBUG, console
>>>> ---
>>>> > log4j.rootCategory=INFO, console
>>>>
>>>> I setup CRAIL_HOME:
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
>>>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>>>>
>>>> =====
>>>>
>>>> Now I try using the CLI:
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>>> fs -ls
>>>> /
>>>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
>>>> directory
>>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
>>>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
>>>>
>>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
>>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
>>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
>>>> java.io.BufferedInputStream@71c7db30
>>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> successful kerberos logins and latency (milliseconds)])
>>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> failed kerberos logins and latency (milliseconds)])
>>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
>>>> with
>>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>>> value=[GetGroups])
>>>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>>> related metrics
>>>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
>>>> found, setting default realm to empty
>>>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
>>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the
>>>> custom-built
>>>> native-hadoop library...
>>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>>> with
>>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
>>>>
>>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> library for your platform... using builtin-java classes where applicable
>>>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
>>>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>>> 20/02/10 14:54:52 DEBUG Groups: Group mapping
>>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>>> cacheTimeout=300000; warningDeltaMs=5000
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
>>>> user:UnixPrincipal: root
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user:
>>>> "UnixPrincipal:
>>>> root" with name root
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
>>>> (auth:SIMPLE)
>>>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
>>>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
>>>> 20/02/10 14:54:52 INFO crail: crail.version 3101
>>>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
>>>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 14:54:52 INFO crail: crail.user crail
>>>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 14:54:52 INFO crail: crail.debug false
>>>> 20/02/10 14:54:52 INFO crail: crail.statistics true
>>>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
>>>> 20/02/10 14:54:52 INFO crail: crail.singleton true
>>>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 14:54:52 INFO crail: crail.locationmap
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.address
>>>> crail://localhost:9060
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.log
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1,
>>>> bufferCount
>>>> 1024
>>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
>>>> ls: java.io.IOException: java.net.ConnectException: Connection refused
>>>>
>>>> =====
>>>>
>>>> Since I get connection refused, I try changing "localhost" to
>>>> "sgt-pepper"
>>>> in the conf files:
>>>>
>>>> [root@sgt-pepper conf]# grep sgt-pepper *
>>>> core-site.xml: <value>crail://sgt-pepper:9060</value>
>>>> crail-site.conf:crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>>
>>>> Then I try using the CLI again:
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>>> fs -ls
>>>> /
>>>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
>>>> directory
>>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>>>
>>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>>>> java.io.BufferedInputStream@71c7db30
>>>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> successful kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> failed kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
>>>> with
>>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>>> value=[GetGroups])
>>>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>>> related metrics
>>>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
>>>> found, setting default realm to empty
>>>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
>>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the
>>>> custom-built
>>>> native-hadoop library...
>>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>>> with
>>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
>>>>
>>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> library for your platform... using builtin-java classes where applicable
>>>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
>>>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>>> 20/02/10 15:00:28 DEBUG Groups: Group mapping
>>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>>> cacheTimeout=300000; warningDeltaMs=5000
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
>>>> user:UnixPrincipal: root
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user:
>>>> "UnixPrincipal:
>>>> root" with name root
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
>>>> (auth:SIMPLE)
>>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
>>>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
>>>> 20/02/10 15:00:28 INFO crail: crail.version 3101
>>>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
>>>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 15:00:28 INFO crail: crail.user crail
>>>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 15:00:28 INFO crail: crail.debug false
>>>> 20/02/10 15:00:28 INFO crail: crail.statistics true
>>>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
>>>> 20/02/10 15:00:28 INFO crail: crail.singleton true
>>>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 15:00:28 INFO crail: crail.locationmap
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.log
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1,
>>>> bufferCount
>>>> 1024
>>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
>>>> 10.114.222.82:9060
>>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
>>>> done..
>>>> ls: /
>>>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
>>>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
>>>> 20/02/10 15:00:28 INFO crail: mapped client cache closed
>>>>
>>>> Seem better, but not really.
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>>>> -mkdir /foobar
>>>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
>>>> directory
>>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>>>
>>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>>>> java.io.BufferedInputStream@71c7db30
>>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> successful kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> failed kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
>>>> with
>>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>>> value=[GetGroups])
>>>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>>> related metrics
>>>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
>>>> found, setting default realm to empty
>>>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
>>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the
>>>> custom-built
>>>> native-hadoop library...
>>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>>> with
>>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
>>>>
>>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> library for your platform... using builtin-java classes where applicable
>>>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
>>>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>>> 20/02/10 15:06:14 DEBUG Groups: Group mapping
>>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>>> cacheTimeout=300000; warningDeltaMs=5000
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
>>>> user:UnixPrincipal: root
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user:
>>>> "UnixPrincipal:
>>>> root" with name root
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
>>>> (auth:SIMPLE)
>>>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
>>>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
>>>> 20/02/10 15:06:14 INFO crail: crail.version 3101
>>>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
>>>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 15:06:14 INFO crail: crail.user crail
>>>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 15:06:14 INFO crail: crail.debug false
>>>> 20/02/10 15:06:14 INFO crail: crail.statistics true
>>>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
>>>> 20/02/10 15:06:14 INFO crail: crail.singleton true
>>>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 15:06:14 INFO crail: crail.locationmap
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.log
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1,
>>>> bufferCount
>>>> 1024
>>>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
>>>> 10.114.222.82:9060
>>>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
>>>> done..
>>>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
>>>> failed
>>>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
>>>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
>>>> 20/02/10 15:06:15 INFO crail: mapped client cache closed
>>>>
>>>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
>>>> java.io.IOException: Map failed *is not good.
>>>>
>>>> Sorry for the long append. Probably something really dumb, but I'm
>>>> sratching my head...
>>>>
>>>> Lou.
>>>>
>>>>
>>>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
>>>> david.crespi@storedgesystems.com> wrote:
>>>>
>>>> > Looks like both cmds are working, but it’s not really finding the
>>>> > datastore.
>>>> > You’re getting info into the namenode, but not the datanode (which
>>>> means
>>>> > you’re really not writing to the datanode). The
>>>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
>>>> > native-hadoop library for your platform)
>>>> >
>>>> > you may want to turn on debug to see more of what’s going on.
>>>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
>>>> >
>>>> > You should probably also look at the individual docker logs.
>>>> > docker logs YourNamenodeName & docker logs YourDatanodeName
>>>> >
>>>> > may give you some more hints of what’s happening.
>>>> >
>>>> >
>>>> > Regards,
>>>> > David
>>>> >
>>>> >
>>>> > From: Lou DeGenaro<ma...@gmail.com>
>>>> > Sent: Monday, February 10, 2020 8:12 AM
>>>> > To: dev@crail.apache.org<ma...@crail.apache.org>
>>>> > Subject: Re: iobench
>>>> >
>>>> > OK, I take it back. Having trouble with crail fs.
>>>> >
>>>> > =====
>>>> >
>>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>>>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> > library for your platform... using builtin-java classes where
>>>> applicable
>>>> > ...
>>>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
>>>> > 10.114.222.23:9060
>>>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
>>>> > done..
>>>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException:
>>>> Map
>>>> > failed
>>>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
>>>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
>>>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
>>>> >
>>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
>>>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> > library for your platform... using builtin-java classes where
>>>> applicable
>>>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs
>>>> initialization
>>>> > done..
>>>> > rmdir: /foobar
>>>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
>>>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
>>>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
>>>> >
>>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>>>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> > library for your platform... using builtin-java classes where
>>>> applicable
>>>> > ...
>>>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
>>>> > done..
>>>> > mkdir: `/foobar': File exists
>>>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
>>>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
>>>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
>>>> >
>>>> > =====
>>>> >
>>>> > Seems that mkdir works, but rmdir does not? Also, IOException was
>>>> reported
>>>> > on mkdir, which seems worrisome.
>>>> >
>>>> > Lou.
>>>> >
>>>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > David,
>>>> > >
>>>> > > Thanks. I'm able to use *crail fs *successfully to
>>>> create/list/delete.
>>>> > > Will try to increase available storage space next...
>>>> > >
>>>> > > Lou.
>>>> > >
>>>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
>>>> > > david.crespi@storedgesystems.com> wrote:
>>>> > >
>>>> > >> Oh, and it may have already written something on the datastore, so
>>>> you’d
>>>> > >> need
>>>> > >> to check it and remove it with the fs command anyway.
>>>> > >>
>>>> > >> crail fs -ls -R /
>>>> > >>
>>>> > >> would show you what you have there.
>>>> > >>
>>>> > >> Regards,
>>>> > >> David
>>>> > >>
>>>> > >> (C) 714-476-2692
>>>> > >>
>>>> > >> From: Lou DeGenaro<ma...@gmail.com>
>>>> > >> Sent: Friday, February 7, 2020 12:15 PM
>>>> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
>>>> > >> Subject: iobench
>>>> > >>
>>>> > >> Still a noob. Got namenode and datanode running as docker images
>>>> on my
>>>> > >> VM. I'm looking for the simplest example of writing something then
>>>> > >> reading
>>>> > >> something.
>>>> > >>
>>>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
>>>> > >> /filename
>>>> > >> -s 1024 -k 1
>>>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
>>>> > >> org.apache.crail.memory.MappedBufferCache
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
>>>> > >> crail://abbey-road:9060
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection
>>>> roundrobin
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
>>>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
>>>> > >> org.apache.crail.storage.tcp.TcpStorageTier
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
>>>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
>>>> > bufferCount
>>>> > >> 1024
>>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>>>> queueDepth
>>>> > >> 16,
>>>> > >> messageSize 2097152, nodealy false
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
>>>> 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
>>>> > 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
>>>> > >> /dev/hugepages/data
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
>>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>>>> queueDepth
>>>> > >> 32,
>>>> > >> messageSize 512, nodealy true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
>>>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
>>>> > >> 10.114.222.23:9060
>>>> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
>>>> > >> locationClass
>>>> > >> 0, buffered true
>>>> > >> Exception in thread "main" java.io.IOException: Map failed
>>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>>>> > >> at
>>>> > >>
>>>> > >>
>>>> >
>>>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
>>>> > >> at
>>>> >
>>>> org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
>>>> > >> at
>>>> > >>
>>>> >
>>>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
>>>> > >> at
>>>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
>>>> > >> at
>>>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
>>>> > >> Caused by: java.lang.OutOfMemoryError: Map failed
>>>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>>>> > >> ... 5 more
>>>> > >>
>>>> > >>
>>>> >
>>>> >
>>>>
>>>>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
Success!
[root@sgt-pepper ~]# $CRAIL_HOME/bin/crail fs -ls /foo
...
Found 1 items
-rw-rw-rw- 1 crail crail 1079 2020-02-11 08:58 /foo/favicon.png
Thanks for the help and suggestions.
Now to try all over again with docker...
Lou.
On Tue, Feb 11, 2020 at 9:47 AM Lou DeGenaro <lo...@gmail.com> wrote:
> Changed the default in conf from /dev to /tmp as you suggest and that
> helped!
>
> [root@sgt-pepper conf]# $CRAIL_HOME/bin/crail datanode
> 20/02/11 08:43:36 INFO crail: crail.version 3101
> 20/02/11 08:43:36 INFO crail: crail.directorydepth 16
> 20/02/11 08:43:36 INFO crail: crail.tokenexpiration 10
> 20/02/11 08:43:36 INFO crail: crail.blocksize 1048576
> 20/02/11 08:43:36 INFO crail: crail.cachelimit 1073741824
> 20/02/11 08:43:36 INFO crail: crail.cachepath /tmp/hugepages/cache
> 20/02/11 08:43:36 INFO crail: crail.user crail
> 20/02/11 08:43:36 INFO crail: crail.shadowreplication 1
> 20/02/11 08:43:36 INFO crail: crail.debug false
> 20/02/11 08:43:36 INFO crail: crail.statistics true
> 20/02/11 08:43:36 INFO crail: crail.rpctimeout 1000
> 20/02/11 08:43:36 INFO crail: crail.datatimeout 1000
> 20/02/11 08:43:36 INFO crail: crail.buffersize 1048576
> 20/02/11 08:43:36 INFO crail: crail.slicesize 524288
> 20/02/11 08:43:36 INFO crail: crail.singleton true
> 20/02/11 08:43:36 INFO crail: crail.regionsize 1073741824
> 20/02/11 08:43:36 INFO crail: crail.directoryrecord 512
> 20/02/11 08:43:36 INFO crail: crail.directoryrandomize true
> 20/02/11 08:43:36 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/11 08:43:36 INFO crail: crail.locationmap
> 20/02/11 08:43:36 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/11 08:43:36 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/11 08:43:36 INFO crail: crail.namenode.fileblocks 16
> 20/02/11 08:43:36 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/11 08:43:36 INFO crail: crail.namenode.log
> 20/02/11 08:43:36 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/11 08:43:36 INFO crail: crail.storage.classes 1
> 20/02/11 08:43:36 INFO crail: crail.storage.rootclass 0
> 20/02/11 08:43:36 INFO crail: crail.storage.keepalive 2
> 20/02/11 08:43:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> 16, messageSize 2097152, nodealy false, cores 1
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.interface eth0
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.port 50020
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.datapath
> /tmp/hugepages/data
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/11 08:43:36 INFO crail: crail.storage.tcp.cores 1
> 20/02/11 08:43:36 INFO crail: running TCP storage server, address /
> 10.114.222.23:50020
> 20/02/11 08:43:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> 32, messageSize 512, nodealy true
> 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.cores 1
> 20/02/11 08:43:36 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.23:9060
> 20/02/11 08:43:36 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:37 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:39 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:41 INFO crail: datanode statistics, freeBlocks 1024
> 20/02/11 08:43:43 INFO crail: datanode statistics, freeBlocks 1024
> ...
>
> Now to see if I can use the CLI...
>
> Lou.
>
>
>
> On Tue, Feb 11, 2020 at 9:40 AM Lou DeGenaro <lo...@gmail.com>
> wrote:
>
>> ...and if it makes any difference:
>>
>> [root@sgt-pepper conf]# which java
>> /usr/bin/java
>> [root@sgt-pepper conf]# java -version
>> openjdk version "1.8.0_242"
>> OpenJDK Runtime Environment (build 1.8.0_242-b08)
>> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
>>
>> [root@sgt-pepper ~]# vmstat -s
>> 8003404 K total memory
>> 906212 K used memory
>> 1658432 K active memory
>> 2281696 K inactive memory
>> 3692808 K free memory
>> 123776 K buffer memory
>> 3280608 K swap cache
>> 0 K total swap
>> 0 K used swap
>> 0 K free swap
>> 59564 non-nice user cpu ticks
>> 359 nice user cpu ticks
>> 22392 system cpu ticks
>> 1165850 idle cpu ticks
>> 9712 IO-wait cpu ticks
>> 0 IRQ cpu ticks
>> 1288 softirq cpu ticks
>> 7522 stolen cpu ticks
>> 313554 pages paged in
>> 4620824 pages paged out
>> 0 pages swapped in
>> 0 pages swapped out
>> 12475579 interrupts
>> 20632577 CPU context switches
>> 1581425491 boot time
>> 86760 forks
>>
>> Lou.
>>
>> On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lo...@gmail.com>
>> wrote:
>>
>>> I've now removed docker from the picture. I simply download the bin
>>> distribution onto my 8GB VM and copy the templates in conf w/o changes.
>>> ======
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>> namenode
>>> 20/02/11 08:09:08 INFO crail: initalizing namenode
>>> 20/02/11 08:09:08 INFO crail: crail.version 3101
>>> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16
>>> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
>>> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
>>> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
>>> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/11 08:09:08 INFO crail: crail.user crail
>>> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
>>> 20/02/11 08:09:08 INFO crail: crail.debug false
>>> 20/02/11 08:09:08 INFO crail: crail.statistics true
>>> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
>>> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
>>> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
>>> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288
>>> 20/02/11 08:09:08 INFO crail: crail.singleton true
>>> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
>>> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
>>> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
>>> 20/02/11 08:09:08 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/11 08:09:08 INFO crail: crail.locationmap
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.address
>>> crail://localhost:9060?id=0&size=1
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.log
>>> 20/02/11 08:09:08 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1
>>> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
>>> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
>>> 20/02/11 08:09:08 INFO crail: round robin block selection
>>> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>> 32, messageSize 512, nodealy true, cores 1
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
>>> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
>>> 127.0.0.1:40578
>>> =====
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>> datanode
>>> 20/02/11 08:11:28 INFO crail: crail.version 3101
>>> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16
>>> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
>>> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
>>> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/11 08:11:28 INFO crail: crail.user crail
>>> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
>>> 20/02/11 08:11:28 INFO crail: crail.debug false
>>> 20/02/11 08:11:28 INFO crail: crail.statistics true
>>> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
>>> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
>>> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
>>> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288
>>> 20/02/11 08:11:28 INFO crail: crail.singleton true
>>> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
>>> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
>>> 20/02/11 08:11:28 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/11 08:11:28 INFO crail: crail.locationmap
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.address
>>> crail://localhost:9060
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.log
>>> 20/02/11 08:11:28 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1
>>> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
>>> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
>>> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>> 16, messageSize 2097152, nodealy false, cores 1
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath
>>> /dev/hugepages/data
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
>>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
>>> 20/02/11 08:11:28 INFO crail: running TCP storage server, address /
>>> 10.114.222.23:50020
>>> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 32, messageSize 512, nodealy true
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
>>> 127.0.0.1:9060
>>> Exception in thread "main" java.io.IOException: Map failed
>>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>>> at
>>> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
>>> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
>>> Caused by: java.lang.OutOfMemoryError: Map failed
>>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>>> ... 2 more
>>> =====
>>> Sorry for being a noob. Thx for the help.
>>>
>>> Lou.
>>>
>>> On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
>>> david.crespi@storedgesystems.com> wrote:
>>>
>>>> Was does your core-site and crail-site look like.
>>>> At first thought, it may not be resolving “sgt-pepper”.
>>>> Why not try the actual IP address to eliminate.
>>>>
>>>> Regards,
>>>> David
>>>>
>>>> (C) 714-476-2692
>>>>
>>>> From: Lou DeGenaro<ma...@gmail.com>
>>>> Sent: Monday, February 10, 2020 1:12 PM
>>>> To: dev@crail.apache.org<ma...@crail.apache.org>
>>>> Subject: Re: iobench
>>>>
>>>> I seem to have gone backwards? I started over with new VM. I started 2
>>>> docker containers, one each for name and data:
>>>>
>>>> [root@sgt-pepper ~]# docker run -it --network host -e
>>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
>>>> 20/02/10 20:32:44 INFO crail: initalizing namenode
>>>> 20/02/10 20:32:44 INFO crail: crail.version 3101
>>>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
>>>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
>>>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 20:32:44 INFO crail: crail.user crail
>>>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 20:32:44 INFO crail: crail.debug false
>>>> 20/02/10 20:32:44 INFO crail: crail.statistics true
>>>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
>>>> 20/02/10 20:32:44 INFO crail: crail.singleton true
>>>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 20:32:44 INFO crail: crail.locationmap
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060?id=0&size=1
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 20:32:44 INFO crail: crail.namenode.log
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 20:32:44 INFO crail: round robin block selection
>>>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true, cores 1
>>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
>>>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
>>>> 10.114.222.82:37328
>>>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of
>>>> type
>>>> 0 to storage class 0
>>>>
>>>> [root@sgt-pepper ~]# docker run -it --network host -e
>>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
>>>> 20/02/10 20:35:36 INFO crail: crail.version 3101
>>>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
>>>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
>>>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 20:35:36 INFO crail: crail.user crail
>>>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 20:35:36 INFO crail: crail.debug false
>>>> 20/02/10 20:35:36 INFO crail: crail.statistics true
>>>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
>>>> 20/02/10 20:35:36 INFO crail: crail.singleton true
>>>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 20:35:36 INFO crail: crail.locationmap
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.log
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false, cores 1
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
>>>> 10.114.222.82:50020
>>>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
>>>> 10.114.222.82:9060
>>>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
>>>> ...
>>>>
>>>> This seems to be OK so far, since we see *adding datanode
>>>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
>>>> class 0* in the namenode log.
>>>>
>>>> Then I download the Non-Official Binary Release from here:
>>>> https://crail.apache.org/download/ and I extract and create the 4 conf
>>>> files and the revised log4j properties file:
>>>> [root@sgt-pepper conf]# ls -atl
>>>> total 40
>>>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
>>>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
>>>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
>>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
>>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
>>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
>>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
>>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
>>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
>>>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
>>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
>>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
>>>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
>>>> 2c2
>>>> < log4j.rootCategory=DEBUG, console
>>>> ---
>>>> > log4j.rootCategory=INFO, console
>>>>
>>>> I setup CRAIL_HOME:
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
>>>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>>>>
>>>> =====
>>>>
>>>> Now I try using the CLI:
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>>> fs -ls
>>>> /
>>>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
>>>> directory
>>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
>>>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
>>>>
>>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
>>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
>>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
>>>> java.io.BufferedInputStream@71c7db30
>>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> successful kerberos logins and latency (milliseconds)])
>>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> failed kerberos logins and latency (milliseconds)])
>>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
>>>> with
>>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>>> value=[GetGroups])
>>>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>>> related metrics
>>>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
>>>> found, setting default realm to empty
>>>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
>>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the
>>>> custom-built
>>>> native-hadoop library...
>>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>>> with
>>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
>>>>
>>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> library for your platform... using builtin-java classes where applicable
>>>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
>>>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>>> 20/02/10 14:54:52 DEBUG Groups: Group mapping
>>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>>> cacheTimeout=300000; warningDeltaMs=5000
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
>>>> user:UnixPrincipal: root
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user:
>>>> "UnixPrincipal:
>>>> root" with name root
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
>>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
>>>> (auth:SIMPLE)
>>>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
>>>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
>>>> 20/02/10 14:54:52 INFO crail: crail.version 3101
>>>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
>>>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 14:54:52 INFO crail: crail.user crail
>>>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 14:54:52 INFO crail: crail.debug false
>>>> 20/02/10 14:54:52 INFO crail: crail.statistics true
>>>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
>>>> 20/02/10 14:54:52 INFO crail: crail.singleton true
>>>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 14:54:52 INFO crail: crail.locationmap
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.address
>>>> crail://localhost:9060
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.log
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1,
>>>> bufferCount
>>>> 1024
>>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
>>>> ls: java.io.IOException: java.net.ConnectException: Connection refused
>>>>
>>>> =====
>>>>
>>>> Since I get connection refused, I try changing "localhost" to
>>>> "sgt-pepper"
>>>> in the conf files:
>>>>
>>>> [root@sgt-pepper conf]# grep sgt-pepper *
>>>> core-site.xml: <value>crail://sgt-pepper:9060</value>
>>>> crail-site.conf:crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>>
>>>> Then I try using the CLI again:
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>>>> fs -ls
>>>> /
>>>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
>>>> directory
>>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>>>
>>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>>>> java.io.BufferedInputStream@71c7db30
>>>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> successful kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> failed kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
>>>> with
>>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>>> value=[GetGroups])
>>>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>>> related metrics
>>>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
>>>> found, setting default realm to empty
>>>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
>>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the
>>>> custom-built
>>>> native-hadoop library...
>>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>>> with
>>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
>>>>
>>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> library for your platform... using builtin-java classes where applicable
>>>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
>>>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>>> 20/02/10 15:00:28 DEBUG Groups: Group mapping
>>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>>> cacheTimeout=300000; warningDeltaMs=5000
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
>>>> user:UnixPrincipal: root
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user:
>>>> "UnixPrincipal:
>>>> root" with name root
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
>>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
>>>> (auth:SIMPLE)
>>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
>>>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
>>>> 20/02/10 15:00:28 INFO crail: crail.version 3101
>>>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
>>>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 15:00:28 INFO crail: crail.user crail
>>>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 15:00:28 INFO crail: crail.debug false
>>>> 20/02/10 15:00:28 INFO crail: crail.statistics true
>>>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
>>>> 20/02/10 15:00:28 INFO crail: crail.singleton true
>>>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 15:00:28 INFO crail: crail.locationmap
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.log
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1,
>>>> bufferCount
>>>> 1024
>>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
>>>> 10.114.222.82:9060
>>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
>>>> done..
>>>> ls: /
>>>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
>>>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
>>>> 20/02/10 15:00:28 INFO crail: mapped client cache closed
>>>>
>>>> Seem better, but not really.
>>>>
>>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>>>> -mkdir /foobar
>>>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
>>>> directory
>>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>>> at
>>>>
>>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>>>
>>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>>>> java.io.BufferedInputStream@71c7db30
>>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> successful kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>>> of
>>>> failed kerberos logins and latency (milliseconds)])
>>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>>> org.apache.hadoop.metrics2.lib.MutableRate
>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups
>>>> with
>>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>>> value=[GetGroups])
>>>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>>> related metrics
>>>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
>>>> found, setting default realm to empty
>>>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
>>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the
>>>> custom-built
>>>> native-hadoop library...
>>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>>> with
>>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
>>>>
>>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> library for your platform... using builtin-java classes where applicable
>>>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
>>>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>>> 20/02/10 15:06:14 DEBUG Groups: Group mapping
>>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>>> cacheTimeout=300000; warningDeltaMs=5000
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
>>>> user:UnixPrincipal: root
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user:
>>>> "UnixPrincipal:
>>>> root" with name root
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
>>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
>>>> (auth:SIMPLE)
>>>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
>>>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
>>>> 20/02/10 15:06:14 INFO crail: crail.version 3101
>>>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
>>>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
>>>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
>>>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> 20/02/10 15:06:14 INFO crail: crail.user crail
>>>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
>>>> 20/02/10 15:06:14 INFO crail: crail.debug false
>>>> 20/02/10 15:06:14 INFO crail: crail.statistics true
>>>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
>>>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
>>>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
>>>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
>>>> 20/02/10 15:06:14 INFO crail: crail.singleton true
>>>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
>>>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
>>>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
>>>> org.apache.crail.memory.MappedBufferCache
>>>> 20/02/10 15:06:14 INFO crail: crail.locationmap
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.address
>>>> crail://sgt-pepper:9060
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
>>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> 20/02/10 15:06:14 INFO crail: crail.namenode.log
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.types
>>>> org.apache.crail.storage.tcp.TcpStorageTier
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
>>>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1,
>>>> bufferCount
>>>> 1024
>>>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 16,
>>>> messageSize 2097152, nodealy false
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize
>>>> 1073741824
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
>>>> /dev/hugepages/data
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
>>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
>>>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>>> 32,
>>>> messageSize 512, nodealy true
>>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
>>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
>>>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
>>>> 10.114.222.82:9060
>>>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
>>>> done..
>>>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
>>>> failed
>>>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
>>>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
>>>> 20/02/10 15:06:15 INFO crail: mapped client cache closed
>>>>
>>>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
>>>> java.io.IOException: Map failed *is not good.
>>>>
>>>> Sorry for the long append. Probably something really dumb, but I'm
>>>> sratching my head...
>>>>
>>>> Lou.
>>>>
>>>>
>>>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
>>>> david.crespi@storedgesystems.com> wrote:
>>>>
>>>> > Looks like both cmds are working, but it’s not really finding the
>>>> > datastore.
>>>> > You’re getting info into the namenode, but not the datanode (which
>>>> means
>>>> > you’re really not writing to the datanode). The
>>>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
>>>> > native-hadoop library for your platform)
>>>> >
>>>> > you may want to turn on debug to see more of what’s going on.
>>>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
>>>> >
>>>> > You should probably also look at the individual docker logs.
>>>> > docker logs YourNamenodeName & docker logs YourDatanodeName
>>>> >
>>>> > may give you some more hints of what’s happening.
>>>> >
>>>> >
>>>> > Regards,
>>>> > David
>>>> >
>>>> >
>>>> > From: Lou DeGenaro<ma...@gmail.com>
>>>> > Sent: Monday, February 10, 2020 8:12 AM
>>>> > To: dev@crail.apache.org<ma...@crail.apache.org>
>>>> > Subject: Re: iobench
>>>> >
>>>> > OK, I take it back. Having trouble with crail fs.
>>>> >
>>>> > =====
>>>> >
>>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>>>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> > library for your platform... using builtin-java classes where
>>>> applicable
>>>> > ...
>>>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
>>>> > 10.114.222.23:9060
>>>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
>>>> > done..
>>>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException:
>>>> Map
>>>> > failed
>>>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
>>>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
>>>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
>>>> >
>>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
>>>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> > library for your platform... using builtin-java classes where
>>>> applicable
>>>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs
>>>> initialization
>>>> > done..
>>>> > rmdir: /foobar
>>>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
>>>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
>>>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
>>>> >
>>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>>>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
>>>> > library for your platform... using builtin-java classes where
>>>> applicable
>>>> > ...
>>>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
>>>> > done..
>>>> > mkdir: `/foobar': File exists
>>>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
>>>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
>>>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
>>>> >
>>>> > =====
>>>> >
>>>> > Seems that mkdir works, but rmdir does not? Also, IOException was
>>>> reported
>>>> > on mkdir, which seems worrisome.
>>>> >
>>>> > Lou.
>>>> >
>>>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > David,
>>>> > >
>>>> > > Thanks. I'm able to use *crail fs *successfully to
>>>> create/list/delete.
>>>> > > Will try to increase available storage space next...
>>>> > >
>>>> > > Lou.
>>>> > >
>>>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
>>>> > > david.crespi@storedgesystems.com> wrote:
>>>> > >
>>>> > >> Oh, and it may have already written something on the datastore, so
>>>> you’d
>>>> > >> need
>>>> > >> to check it and remove it with the fs command anyway.
>>>> > >>
>>>> > >> crail fs -ls -R /
>>>> > >>
>>>> > >> would show you what you have there.
>>>> > >>
>>>> > >> Regards,
>>>> > >> David
>>>> > >>
>>>> > >> (C) 714-476-2692
>>>> > >>
>>>> > >> From: Lou DeGenaro<ma...@gmail.com>
>>>> > >> Sent: Friday, February 7, 2020 12:15 PM
>>>> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
>>>> > >> Subject: iobench
>>>> > >>
>>>> > >> Still a noob. Got namenode and datanode running as docker images
>>>> on my
>>>> > >> VM. I'm looking for the simplest example of writing something then
>>>> > >> reading
>>>> > >> something.
>>>> > >>
>>>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
>>>> > >> /filename
>>>> > >> -s 1024 -k 1
>>>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
>>>> > >> org.apache.crail.memory.MappedBufferCache
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
>>>> > >> crail://abbey-road:9060
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection
>>>> roundrobin
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
>>>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
>>>> > >> org.apache.crail.storage.tcp.TcpStorageTier
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
>>>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
>>>> > bufferCount
>>>> > >> 1024
>>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>>>> queueDepth
>>>> > >> 16,
>>>> > >> messageSize 2097152, nodealy false
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
>>>> 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
>>>> > 1073741824
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
>>>> > >> /dev/hugepages/data
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
>>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>>>> queueDepth
>>>> > >> 32,
>>>> > >> messageSize 512, nodealy true
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
>>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
>>>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
>>>> > >> 10.114.222.23:9060
>>>> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
>>>> > >> locationClass
>>>> > >> 0, buffered true
>>>> > >> Exception in thread "main" java.io.IOException: Map failed
>>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>>>> > >> at
>>>> > >>
>>>> > >>
>>>> >
>>>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
>>>> > >> at
>>>> >
>>>> org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
>>>> > >> at
>>>> > >>
>>>> >
>>>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
>>>> > >> at
>>>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
>>>> > >> at
>>>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
>>>> > >> Caused by: java.lang.OutOfMemoryError: Map failed
>>>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>>>> > >> ... 5 more
>>>> > >>
>>>> > >>
>>>> >
>>>> >
>>>>
>>>>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
Changed the default in conf from /dev to /tmp as you suggest and that
helped!
[root@sgt-pepper conf]# $CRAIL_HOME/bin/crail datanode
20/02/11 08:43:36 INFO crail: crail.version 3101
20/02/11 08:43:36 INFO crail: crail.directorydepth 16
20/02/11 08:43:36 INFO crail: crail.tokenexpiration 10
20/02/11 08:43:36 INFO crail: crail.blocksize 1048576
20/02/11 08:43:36 INFO crail: crail.cachelimit 1073741824
20/02/11 08:43:36 INFO crail: crail.cachepath /tmp/hugepages/cache
20/02/11 08:43:36 INFO crail: crail.user crail
20/02/11 08:43:36 INFO crail: crail.shadowreplication 1
20/02/11 08:43:36 INFO crail: crail.debug false
20/02/11 08:43:36 INFO crail: crail.statistics true
20/02/11 08:43:36 INFO crail: crail.rpctimeout 1000
20/02/11 08:43:36 INFO crail: crail.datatimeout 1000
20/02/11 08:43:36 INFO crail: crail.buffersize 1048576
20/02/11 08:43:36 INFO crail: crail.slicesize 524288
20/02/11 08:43:36 INFO crail: crail.singleton true
20/02/11 08:43:36 INFO crail: crail.regionsize 1073741824
20/02/11 08:43:36 INFO crail: crail.directoryrecord 512
20/02/11 08:43:36 INFO crail: crail.directoryrandomize true
20/02/11 08:43:36 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/11 08:43:36 INFO crail: crail.locationmap
20/02/11 08:43:36 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/11 08:43:36 INFO crail: crail.namenode.blockselection roundrobin
20/02/11 08:43:36 INFO crail: crail.namenode.fileblocks 16
20/02/11 08:43:36 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/11 08:43:36 INFO crail: crail.namenode.log
20/02/11 08:43:36 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/11 08:43:36 INFO crail: crail.storage.classes 1
20/02/11 08:43:36 INFO crail: crail.storage.rootclass 0
20/02/11 08:43:36 INFO crail: crail.storage.keepalive 2
20/02/11 08:43:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false, cores 1
20/02/11 08:43:36 INFO crail: crail.storage.tcp.interface eth0
20/02/11 08:43:36 INFO crail: crail.storage.tcp.port 50020
20/02/11 08:43:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/11 08:43:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/11 08:43:36 INFO crail: crail.storage.tcp.datapath /tmp/hugepages/data
20/02/11 08:43:36 INFO crail: crail.storage.tcp.queuedepth 16
20/02/11 08:43:36 INFO crail: crail.storage.tcp.cores 1
20/02/11 08:43:36 INFO crail: running TCP storage server, address /
10.114.222.23:50020
20/02/11 08:43:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/11 08:43:36 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/11 08:43:36 INFO crail: crail.namenode.tcp.messageSize 512
20/02/11 08:43:36 INFO crail: crail.namenode.tcp.cores 1
20/02/11 08:43:36 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.23:9060
20/02/11 08:43:36 INFO crail: datanode statistics, freeBlocks 1024
20/02/11 08:43:37 INFO crail: datanode statistics, freeBlocks 1024
20/02/11 08:43:39 INFO crail: datanode statistics, freeBlocks 1024
20/02/11 08:43:41 INFO crail: datanode statistics, freeBlocks 1024
20/02/11 08:43:43 INFO crail: datanode statistics, freeBlocks 1024
...
Now to see if I can use the CLI...
Lou.
On Tue, Feb 11, 2020 at 9:40 AM Lou DeGenaro <lo...@gmail.com> wrote:
> ...and if it makes any difference:
>
> [root@sgt-pepper conf]# which java
> /usr/bin/java
> [root@sgt-pepper conf]# java -version
> openjdk version "1.8.0_242"
> OpenJDK Runtime Environment (build 1.8.0_242-b08)
> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
>
> [root@sgt-pepper ~]# vmstat -s
> 8003404 K total memory
> 906212 K used memory
> 1658432 K active memory
> 2281696 K inactive memory
> 3692808 K free memory
> 123776 K buffer memory
> 3280608 K swap cache
> 0 K total swap
> 0 K used swap
> 0 K free swap
> 59564 non-nice user cpu ticks
> 359 nice user cpu ticks
> 22392 system cpu ticks
> 1165850 idle cpu ticks
> 9712 IO-wait cpu ticks
> 0 IRQ cpu ticks
> 1288 softirq cpu ticks
> 7522 stolen cpu ticks
> 313554 pages paged in
> 4620824 pages paged out
> 0 pages swapped in
> 0 pages swapped out
> 12475579 interrupts
> 20632577 CPU context switches
> 1581425491 boot time
> 86760 forks
>
> Lou.
>
> On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lo...@gmail.com>
> wrote:
>
>> I've now removed docker from the picture. I simply download the bin
>> distribution onto my 8GB VM and copy the templates in conf w/o changes.
>> ======
>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>> namenode
>> 20/02/11 08:09:08 INFO crail: initalizing namenode
>> 20/02/11 08:09:08 INFO crail: crail.version 3101
>> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16
>> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
>> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
>> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
>> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/11 08:09:08 INFO crail: crail.user crail
>> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
>> 20/02/11 08:09:08 INFO crail: crail.debug false
>> 20/02/11 08:09:08 INFO crail: crail.statistics true
>> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
>> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
>> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
>> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288
>> 20/02/11 08:09:08 INFO crail: crail.singleton true
>> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
>> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
>> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
>> 20/02/11 08:09:08 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/11 08:09:08 INFO crail: crail.locationmap
>> 20/02/11 08:09:08 INFO crail: crail.namenode.address
>> crail://localhost:9060?id=0&size=1
>> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
>> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/11 08:09:08 INFO crail: crail.namenode.log
>> 20/02/11 08:09:08 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1
>> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
>> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
>> 20/02/11 08:09:08 INFO crail: round robin block selection
>> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>> 32, messageSize 512, nodealy true, cores 1
>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
>> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
>> 127.0.0.1:40578
>> =====
>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
>> datanode
>> 20/02/11 08:11:28 INFO crail: crail.version 3101
>> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16
>> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
>> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
>> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
>> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/11 08:11:28 INFO crail: crail.user crail
>> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
>> 20/02/11 08:11:28 INFO crail: crail.debug false
>> 20/02/11 08:11:28 INFO crail: crail.statistics true
>> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
>> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
>> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
>> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288
>> 20/02/11 08:11:28 INFO crail: crail.singleton true
>> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
>> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
>> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
>> 20/02/11 08:11:28 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/11 08:11:28 INFO crail: crail.locationmap
>> 20/02/11 08:11:28 INFO crail: crail.namenode.address
>> crail://localhost:9060
>> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
>> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/11 08:11:28 INFO crail: crail.namenode.log
>> 20/02/11 08:11:28 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1
>> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
>> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
>> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>> 16, messageSize 2097152, nodealy false, cores 1
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
>> 20/02/11 08:11:28 INFO crail: running TCP storage server, address /
>> 10.114.222.23:50020
>> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32, messageSize 512, nodealy true
>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
>> 127.0.0.1:9060
>> Exception in thread "main" java.io.IOException: Map failed
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>> at
>> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
>> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
>> Caused by: java.lang.OutOfMemoryError: Map failed
>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>> ... 2 more
>> =====
>> Sorry for being a noob. Thx for the help.
>>
>> Lou.
>>
>> On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
>> david.crespi@storedgesystems.com> wrote:
>>
>>> Was does your core-site and crail-site look like.
>>> At first thought, it may not be resolving “sgt-pepper”.
>>> Why not try the actual IP address to eliminate.
>>>
>>> Regards,
>>> David
>>>
>>> (C) 714-476-2692
>>>
>>> From: Lou DeGenaro<ma...@gmail.com>
>>> Sent: Monday, February 10, 2020 1:12 PM
>>> To: dev@crail.apache.org<ma...@crail.apache.org>
>>> Subject: Re: iobench
>>>
>>> I seem to have gone backwards? I started over with new VM. I started 2
>>> docker containers, one each for name and data:
>>>
>>> [root@sgt-pepper ~]# docker run -it --network host -e
>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
>>> 20/02/10 20:32:44 INFO crail: initalizing namenode
>>> 20/02/10 20:32:44 INFO crail: crail.version 3101
>>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
>>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
>>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
>>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
>>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/10 20:32:44 INFO crail: crail.user crail
>>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
>>> 20/02/10 20:32:44 INFO crail: crail.debug false
>>> 20/02/10 20:32:44 INFO crail: crail.statistics true
>>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
>>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
>>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
>>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
>>> 20/02/10 20:32:44 INFO crail: crail.singleton true
>>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
>>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
>>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
>>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/10 20:32:44 INFO crail: crail.locationmap
>>> 20/02/10 20:32:44 INFO crail: crail.namenode.address
>>> crail://sgt-pepper:9060?id=0&size=1
>>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/10 20:32:44 INFO crail: crail.namenode.log
>>> 20/02/10 20:32:44 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
>>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
>>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
>>> 20/02/10 20:32:44 INFO crail: round robin block selection
>>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>> 32,
>>> messageSize 512, nodealy true, cores 1
>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
>>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
>>> 10.114.222.82:37328
>>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of
>>> type
>>> 0 to storage class 0
>>>
>>> [root@sgt-pepper ~]# docker run -it --network host -e
>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
>>> 20/02/10 20:35:36 INFO crail: crail.version 3101
>>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
>>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
>>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
>>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
>>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/10 20:35:36 INFO crail: crail.user crail
>>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
>>> 20/02/10 20:35:36 INFO crail: crail.debug false
>>> 20/02/10 20:35:36 INFO crail: crail.statistics true
>>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
>>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
>>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
>>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
>>> 20/02/10 20:35:36 INFO crail: crail.singleton true
>>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
>>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
>>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
>>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/10 20:35:36 INFO crail: crail.locationmap
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.address
>>> crail://sgt-pepper:9060
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.log
>>> 20/02/10 20:35:36 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
>>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
>>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
>>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>>> 16,
>>> messageSize 2097152, nodealy false, cores 1
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
>>> /dev/hugepages/data
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
>>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
>>> 10.114.222.82:50020
>>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 32,
>>> messageSize 512, nodealy true
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
>>> 10.114.222.82:9060
>>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
>>> ...
>>>
>>> This seems to be OK so far, since we see *adding datanode
>>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
>>> class 0* in the namenode log.
>>>
>>> Then I download the Non-Official Binary Release from here:
>>> https://crail.apache.org/download/ and I extract and create the 4 conf
>>> files and the revised log4j properties file:
>>> [root@sgt-pepper conf]# ls -atl
>>> total 40
>>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
>>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
>>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
>>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
>>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
>>> 2c2
>>> < log4j.rootCategory=DEBUG, console
>>> ---
>>> > log4j.rootCategory=INFO, console
>>>
>>> I setup CRAIL_HOME:
>>>
>>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
>>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
>>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>>>
>>> =====
>>>
>>> Now I try using the CLI:
>>>
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>>> -ls
>>> /
>>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
>>> directory
>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
>>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
>>>
>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
>>> java.io.BufferedInputStream@71c7db30
>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>> of
>>> successful kerberos logins and latency (milliseconds)])
>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>> of
>>> failed kerberos logins and latency (milliseconds)])
>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>> value=[GetGroups])
>>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>> related metrics
>>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
>>> found, setting default realm to empty
>>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built
>>> native-hadoop library...
>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>> with
>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
>>>
>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
>>> library for your platform... using builtin-java classes where applicable
>>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
>>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>> 20/02/10 14:54:52 DEBUG Groups: Group mapping
>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>> cacheTimeout=300000; warningDeltaMs=5000
>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
>>> user:UnixPrincipal: root
>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
>>> root" with name root
>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
>>> (auth:SIMPLE)
>>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
>>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
>>> 20/02/10 14:54:52 INFO crail: crail.version 3101
>>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
>>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
>>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
>>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
>>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/10 14:54:52 INFO crail: crail.user crail
>>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
>>> 20/02/10 14:54:52 INFO crail: crail.debug false
>>> 20/02/10 14:54:52 INFO crail: crail.statistics true
>>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
>>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
>>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
>>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
>>> 20/02/10 14:54:52 INFO crail: crail.singleton true
>>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
>>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
>>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
>>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/10 14:54:52 INFO crail: crail.locationmap
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.address
>>> crail://localhost:9060
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.log
>>> 20/02/10 14:54:52 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
>>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
>>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
>>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1,
>>> bufferCount
>>> 1024
>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 16,
>>> messageSize 2097152, nodealy false
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
>>> /dev/hugepages/data
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 32,
>>> messageSize 512, nodealy true
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
>>> ls: java.io.IOException: java.net.ConnectException: Connection refused
>>>
>>> =====
>>>
>>> Since I get connection refused, I try changing "localhost" to
>>> "sgt-pepper"
>>> in the conf files:
>>>
>>> [root@sgt-pepper conf]# grep sgt-pepper *
>>> core-site.xml: <value>crail://sgt-pepper:9060</value>
>>> crail-site.conf:crail.namenode.address crail://sgt-pepper:9060
>>>
>>> Then I try using the CLI again:
>>>
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>>> -ls
>>> /
>>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
>>> directory
>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>>
>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>>> java.io.BufferedInputStream@71c7db30
>>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>> of
>>> successful kerberos logins and latency (milliseconds)])
>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>> of
>>> failed kerberos logins and latency (milliseconds)])
>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>> value=[GetGroups])
>>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>> related metrics
>>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
>>> found, setting default realm to empty
>>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built
>>> native-hadoop library...
>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>> with
>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
>>>
>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
>>> library for your platform... using builtin-java classes where applicable
>>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
>>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>> 20/02/10 15:00:28 DEBUG Groups: Group mapping
>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>> cacheTimeout=300000; warningDeltaMs=5000
>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
>>> user:UnixPrincipal: root
>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
>>> root" with name root
>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
>>> (auth:SIMPLE)
>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
>>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
>>> 20/02/10 15:00:28 INFO crail: crail.version 3101
>>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
>>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
>>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
>>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
>>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/10 15:00:28 INFO crail: crail.user crail
>>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
>>> 20/02/10 15:00:28 INFO crail: crail.debug false
>>> 20/02/10 15:00:28 INFO crail: crail.statistics true
>>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
>>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
>>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
>>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
>>> 20/02/10 15:00:28 INFO crail: crail.singleton true
>>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
>>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
>>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
>>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/10 15:00:28 INFO crail: crail.locationmap
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.address
>>> crail://sgt-pepper:9060
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.log
>>> 20/02/10 15:00:28 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
>>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
>>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
>>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1,
>>> bufferCount
>>> 1024
>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 16,
>>> messageSize 2097152, nodealy false
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
>>> /dev/hugepages/data
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 32,
>>> messageSize 512, nodealy true
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
>>> 10.114.222.82:9060
>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
>>> done..
>>> ls: /
>>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
>>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
>>> 20/02/10 15:00:28 INFO crail: mapped client cache closed
>>>
>>> Seem better, but not really.
>>>
>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>>> -mkdir /foobar
>>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
>>> directory
>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>>> at
>>>
>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>>
>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>>> java.io.BufferedInputStream@71c7db30
>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>> of
>>> successful kerberos logins and latency (milliseconds)])
>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate
>>> of
>>> failed kerberos logins and latency (milliseconds)])
>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>>> org.apache.hadoop.metrics2.lib.MutableRate
>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>>> value=[GetGroups])
>>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>>> related metrics
>>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
>>> found, setting default realm to empty
>>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built
>>> native-hadoop library...
>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop
>>> with
>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
>>>
>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
>>> library for your platform... using builtin-java classes where applicable
>>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
>>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>>> 20/02/10 15:06:14 DEBUG Groups: Group mapping
>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>>> cacheTimeout=300000; warningDeltaMs=5000
>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
>>> user:UnixPrincipal: root
>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
>>> root" with name root
>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
>>> (auth:SIMPLE)
>>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
>>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
>>> 20/02/10 15:06:14 INFO crail: crail.version 3101
>>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
>>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
>>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
>>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
>>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
>>> 20/02/10 15:06:14 INFO crail: crail.user crail
>>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
>>> 20/02/10 15:06:14 INFO crail: crail.debug false
>>> 20/02/10 15:06:14 INFO crail: crail.statistics true
>>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
>>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
>>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
>>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
>>> 20/02/10 15:06:14 INFO crail: crail.singleton true
>>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
>>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
>>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
>>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
>>> org.apache.crail.memory.MappedBufferCache
>>> 20/02/10 15:06:14 INFO crail: crail.locationmap
>>> 20/02/10 15:06:14 INFO crail: crail.namenode.address
>>> crail://sgt-pepper:9060
>>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
>>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
>>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> 20/02/10 15:06:14 INFO crail: crail.namenode.log
>>> 20/02/10 15:06:14 INFO crail: crail.storage.types
>>> org.apache.crail.storage.tcp.TcpStorageTier
>>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
>>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
>>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
>>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1,
>>> bufferCount
>>> 1024
>>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 16,
>>> messageSize 2097152, nodealy false
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
>>> /dev/hugepages/data
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
>>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>>> 32,
>>> messageSize 512, nodealy true
>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
>>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
>>> 10.114.222.82:9060
>>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
>>> done..
>>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
>>> failed
>>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
>>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
>>> 20/02/10 15:06:15 INFO crail: mapped client cache closed
>>>
>>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
>>> java.io.IOException: Map failed *is not good.
>>>
>>> Sorry for the long append. Probably something really dumb, but I'm
>>> sratching my head...
>>>
>>> Lou.
>>>
>>>
>>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
>>> david.crespi@storedgesystems.com> wrote:
>>>
>>> > Looks like both cmds are working, but it’s not really finding the
>>> > datastore.
>>> > You’re getting info into the namenode, but not the datanode (which
>>> means
>>> > you’re really not writing to the datanode). The
>>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
>>> > native-hadoop library for your platform)
>>> >
>>> > you may want to turn on debug to see more of what’s going on.
>>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
>>> >
>>> > You should probably also look at the individual docker logs.
>>> > docker logs YourNamenodeName & docker logs YourDatanodeName
>>> >
>>> > may give you some more hints of what’s happening.
>>> >
>>> >
>>> > Regards,
>>> > David
>>> >
>>> >
>>> > From: Lou DeGenaro<ma...@gmail.com>
>>> > Sent: Monday, February 10, 2020 8:12 AM
>>> > To: dev@crail.apache.org<ma...@crail.apache.org>
>>> > Subject: Re: iobench
>>> >
>>> > OK, I take it back. Having trouble with crail fs.
>>> >
>>> > =====
>>> >
>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
>>> > library for your platform... using builtin-java classes where
>>> applicable
>>> > ...
>>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
>>> > 10.114.222.23:9060
>>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
>>> > done..
>>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException:
>>> Map
>>> > failed
>>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
>>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
>>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
>>> >
>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
>>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
>>> > library for your platform... using builtin-java classes where
>>> applicable
>>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs
>>> initialization
>>> > done..
>>> > rmdir: /foobar
>>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
>>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
>>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
>>> >
>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
>>> > library for your platform... using builtin-java classes where
>>> applicable
>>> > ...
>>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
>>> > done..
>>> > mkdir: `/foobar': File exists
>>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
>>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
>>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
>>> >
>>> > =====
>>> >
>>> > Seems that mkdir works, but rmdir does not? Also, IOException was
>>> reported
>>> > on mkdir, which seems worrisome.
>>> >
>>> > Lou.
>>> >
>>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
>>> > wrote:
>>> >
>>> > > David,
>>> > >
>>> > > Thanks. I'm able to use *crail fs *successfully to
>>> create/list/delete.
>>> > > Will try to increase available storage space next...
>>> > >
>>> > > Lou.
>>> > >
>>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
>>> > > david.crespi@storedgesystems.com> wrote:
>>> > >
>>> > >> Oh, and it may have already written something on the datastore, so
>>> you’d
>>> > >> need
>>> > >> to check it and remove it with the fs command anyway.
>>> > >>
>>> > >> crail fs -ls -R /
>>> > >>
>>> > >> would show you what you have there.
>>> > >>
>>> > >> Regards,
>>> > >> David
>>> > >>
>>> > >> (C) 714-476-2692
>>> > >>
>>> > >> From: Lou DeGenaro<ma...@gmail.com>
>>> > >> Sent: Friday, February 7, 2020 12:15 PM
>>> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
>>> > >> Subject: iobench
>>> > >>
>>> > >> Still a noob. Got namenode and datanode running as docker images
>>> on my
>>> > >> VM. I'm looking for the simplest example of writing something then
>>> > >> reading
>>> > >> something.
>>> > >>
>>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
>>> > >> /filename
>>> > >> -s 1024 -k 1
>>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
>>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
>>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
>>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
>>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
>>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
>>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
>>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
>>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
>>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
>>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
>>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
>>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
>>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
>>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
>>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
>>> > >> org.apache.crail.memory.MappedBufferCache
>>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
>>> > >> crail://abbey-road:9060
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection
>>> roundrobin
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
>>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
>>> > >> org.apache.crail.storage.tcp.TcpStorageTier
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
>>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
>>> > bufferCount
>>> > >> 1024
>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>>> queueDepth
>>> > >> 16,
>>> > >> messageSize 2097152, nodealy false
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
>>> 1073741824
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
>>> > 1073741824
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
>>> > >> /dev/hugepages/data
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>>> queueDepth
>>> > >> 32,
>>> > >> messageSize 512, nodealy true
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
>>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
>>> > >> 10.114.222.23:9060
>>> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
>>> > >> locationClass
>>> > >> 0, buffered true
>>> > >> Exception in thread "main" java.io.IOException: Map failed
>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>>> > >> at
>>> > >>
>>> > >>
>>> >
>>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
>>> > >> at
>>> > org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
>>> > >> at
>>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
>>> > >> at
>>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
>>> > >> Caused by: java.lang.OutOfMemoryError: Map failed
>>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>>> > >> ... 5 more
>>> > >>
>>> > >>
>>> >
>>> >
>>>
>>>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
...and if it makes any difference:
[root@sgt-pepper conf]# which java
/usr/bin/java
[root@sgt-pepper conf]# java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
[root@sgt-pepper ~]# vmstat -s
8003404 K total memory
906212 K used memory
1658432 K active memory
2281696 K inactive memory
3692808 K free memory
123776 K buffer memory
3280608 K swap cache
0 K total swap
0 K used swap
0 K free swap
59564 non-nice user cpu ticks
359 nice user cpu ticks
22392 system cpu ticks
1165850 idle cpu ticks
9712 IO-wait cpu ticks
0 IRQ cpu ticks
1288 softirq cpu ticks
7522 stolen cpu ticks
313554 pages paged in
4620824 pages paged out
0 pages swapped in
0 pages swapped out
12475579 interrupts
20632577 CPU context switches
1581425491 boot time
86760 forks
Lou.
On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lo...@gmail.com> wrote:
> I've now removed docker from the picture. I simply download the bin
> distribution onto my 8GB VM and copy the templates in conf w/o changes.
> ======
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> namenode
> 20/02/11 08:09:08 INFO crail: initalizing namenode
> 20/02/11 08:09:08 INFO crail: crail.version 3101
> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16
> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/11 08:09:08 INFO crail: crail.user crail
> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
> 20/02/11 08:09:08 INFO crail: crail.debug false
> 20/02/11 08:09:08 INFO crail: crail.statistics true
> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288
> 20/02/11 08:09:08 INFO crail: crail.singleton true
> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
> 20/02/11 08:09:08 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/11 08:09:08 INFO crail: crail.locationmap
> 20/02/11 08:09:08 INFO crail: crail.namenode.address
> crail://localhost:9060?id=0&size=1
> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/11 08:09:08 INFO crail: crail.namenode.log
> 20/02/11 08:09:08 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1
> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
> 20/02/11 08:09:08 INFO crail: round robin block selection
> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> 32, messageSize 512, nodealy true, cores 1
> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
> 127.0.0.1:40578
> =====
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
> datanode
> 20/02/11 08:11:28 INFO crail: crail.version 3101
> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16
> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/11 08:11:28 INFO crail: crail.user crail
> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
> 20/02/11 08:11:28 INFO crail: crail.debug false
> 20/02/11 08:11:28 INFO crail: crail.statistics true
> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288
> 20/02/11 08:11:28 INFO crail: crail.singleton true
> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
> 20/02/11 08:11:28 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/11 08:11:28 INFO crail: crail.locationmap
> 20/02/11 08:11:28 INFO crail: crail.namenode.address crail://localhost:9060
> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/11 08:11:28 INFO crail: crail.namenode.log
> 20/02/11 08:11:28 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1
> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth
> 16, messageSize 2097152, nodealy false, cores 1
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
> 20/02/11 08:11:28 INFO crail: running TCP storage server, address /
> 10.114.222.23:50020
> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> 32, messageSize 512, nodealy true
> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
> 127.0.0.1:9060
> Exception in thread "main" java.io.IOException: Map failed
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> at
> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
> Caused by: java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> ... 2 more
> =====
> Sorry for being a noob. Thx for the help.
>
> Lou.
>
> On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
> david.crespi@storedgesystems.com> wrote:
>
>> Was does your core-site and crail-site look like.
>> At first thought, it may not be resolving “sgt-pepper”.
>> Why not try the actual IP address to eliminate.
>>
>> Regards,
>> David
>>
>> (C) 714-476-2692
>>
>> From: Lou DeGenaro<ma...@gmail.com>
>> Sent: Monday, February 10, 2020 1:12 PM
>> To: dev@crail.apache.org<ma...@crail.apache.org>
>> Subject: Re: iobench
>>
>> I seem to have gone backwards? I started over with new VM. I started 2
>> docker containers, one each for name and data:
>>
>> [root@sgt-pepper ~]# docker run -it --network host -e
>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
>> 20/02/10 20:32:44 INFO crail: initalizing namenode
>> 20/02/10 20:32:44 INFO crail: crail.version 3101
>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/10 20:32:44 INFO crail: crail.user crail
>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
>> 20/02/10 20:32:44 INFO crail: crail.debug false
>> 20/02/10 20:32:44 INFO crail: crail.statistics true
>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
>> 20/02/10 20:32:44 INFO crail: crail.singleton true
>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/10 20:32:44 INFO crail: crail.locationmap
>> 20/02/10 20:32:44 INFO crail: crail.namenode.address
>> crail://sgt-pepper:9060?id=0&size=1
>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/10 20:32:44 INFO crail: crail.namenode.log
>> 20/02/10 20:32:44 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
>> 20/02/10 20:32:44 INFO crail: round robin block selection
>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true, cores 1
>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
>> 10.114.222.82:37328
>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of
>> type
>> 0 to storage class 0
>>
>> [root@sgt-pepper ~]# docker run -it --network host -e
>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
>> 20/02/10 20:35:36 INFO crail: crail.version 3101
>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/10 20:35:36 INFO crail: crail.user crail
>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
>> 20/02/10 20:35:36 INFO crail: crail.debug false
>> 20/02/10 20:35:36 INFO crail: crail.statistics true
>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
>> 20/02/10 20:35:36 INFO crail: crail.singleton true
>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/10 20:35:36 INFO crail: crail.locationmap
>> 20/02/10 20:35:36 INFO crail: crail.namenode.address
>> crail://sgt-pepper:9060
>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/10 20:35:36 INFO crail: crail.namenode.log
>> 20/02/10 20:35:36 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth
>> 16,
>> messageSize 2097152, nodealy false, cores 1
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
>> 10.114.222.82:50020
>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true
>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
>> 10.114.222.82:9060
>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
>> ...
>>
>> This seems to be OK so far, since we see *adding datanode
>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
>> class 0* in the namenode log.
>>
>> Then I download the Non-Official Binary Release from here:
>> https://crail.apache.org/download/ and I extract and create the 4 conf
>> files and the revised log4j properties file:
>> [root@sgt-pepper conf]# ls -atl
>> total 40
>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
>> 2c2
>> < log4j.rootCategory=DEBUG, console
>> ---
>> > log4j.rootCategory=INFO, console
>>
>> I setup CRAIL_HOME:
>>
>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>>
>> =====
>>
>> Now I try using the CLI:
>>
>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>> -ls
>> /
>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
>> directory
>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
>>
>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
>> java.io.BufferedInputStream@71c7db30
>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
>> successful kerberos logins and latency (milliseconds)])
>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
>> failed kerberos logins and latency (milliseconds)])
>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>> value=[GetGroups])
>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>> related metrics
>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
>> found, setting default realm to empty
>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built
>> native-hadoop library...
>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop
>> with
>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
>>
>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>> 20/02/10 14:54:52 DEBUG Groups: Group mapping
>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>> cacheTimeout=300000; warningDeltaMs=5000
>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
>> user:UnixPrincipal: root
>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
>> root" with name root
>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
>> (auth:SIMPLE)
>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
>> 20/02/10 14:54:52 INFO crail: crail.version 3101
>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/10 14:54:52 INFO crail: crail.user crail
>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
>> 20/02/10 14:54:52 INFO crail: crail.debug false
>> 20/02/10 14:54:52 INFO crail: crail.statistics true
>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
>> 20/02/10 14:54:52 INFO crail: crail.singleton true
>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/10 14:54:52 INFO crail: crail.locationmap
>> 20/02/10 14:54:52 INFO crail: crail.namenode.address
>> crail://localhost:9060
>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/10 14:54:52 INFO crail: crail.namenode.log
>> 20/02/10 14:54:52 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, bufferCount
>> 1024
>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 16,
>> messageSize 2097152, nodealy false
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true
>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
>> ls: java.io.IOException: java.net.ConnectException: Connection refused
>>
>> =====
>>
>> Since I get connection refused, I try changing "localhost" to "sgt-pepper"
>> in the conf files:
>>
>> [root@sgt-pepper conf]# grep sgt-pepper *
>> core-site.xml: <value>crail://sgt-pepper:9060</value>
>> crail-site.conf:crail.namenode.address crail://sgt-pepper:9060
>>
>> Then I try using the CLI again:
>>
>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>> -ls
>> /
>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
>> directory
>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>>
>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
>> java.io.BufferedInputStream@71c7db30
>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
>> successful kerberos logins and latency (milliseconds)])
>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
>> failed kerberos logins and latency (milliseconds)])
>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>> value=[GetGroups])
>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>> related metrics
>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
>> found, setting default realm to empty
>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built
>> native-hadoop library...
>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop
>> with
>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
>>
>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>> 20/02/10 15:00:28 DEBUG Groups: Group mapping
>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>> cacheTimeout=300000; warningDeltaMs=5000
>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
>> user:UnixPrincipal: root
>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
>> root" with name root
>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
>> (auth:SIMPLE)
>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
>> 20/02/10 15:00:28 INFO crail: crail.version 3101
>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/10 15:00:28 INFO crail: crail.user crail
>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
>> 20/02/10 15:00:28 INFO crail: crail.debug false
>> 20/02/10 15:00:28 INFO crail: crail.statistics true
>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
>> 20/02/10 15:00:28 INFO crail: crail.singleton true
>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/10 15:00:28 INFO crail: crail.locationmap
>> 20/02/10 15:00:28 INFO crail: crail.namenode.address
>> crail://sgt-pepper:9060
>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/10 15:00:28 INFO crail: crail.namenode.log
>> 20/02/10 15:00:28 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, bufferCount
>> 1024
>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 16,
>> messageSize 2097152, nodealy false
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true
>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
>> 10.114.222.82:9060
>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
>> done..
>> ls: /
>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
>> 20/02/10 15:00:28 INFO crail: mapped client cache closed
>>
>> Seem better, but not really.
>>
>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
>> -mkdir /foobar
>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
>> directory
>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
>> at
>>
>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>>
>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
>> java.io.BufferedInputStream@71c7db30
>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
>> successful kerberos logins and latency (milliseconds)])
>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
>> failed kerberos logins and latency (milliseconds)])
>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
>> org.apache.hadoop.metrics2.lib.MutableRate
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
>> value=[GetGroups])
>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
>> related metrics
>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
>> found, setting default realm to empty
>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built
>> native-hadoop library...
>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop
>> with
>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
>> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
>>
>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
>> 20/02/10 15:06:14 DEBUG Groups: Group mapping
>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
>> cacheTimeout=300000; warningDeltaMs=5000
>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
>> user:UnixPrincipal: root
>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
>> root" with name root
>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
>> (auth:SIMPLE)
>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
>> 20/02/10 15:06:14 INFO crail: crail.version 3101
>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/10 15:06:14 INFO crail: crail.user crail
>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
>> 20/02/10 15:06:14 INFO crail: crail.debug false
>> 20/02/10 15:06:14 INFO crail: crail.statistics true
>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
>> 20/02/10 15:06:14 INFO crail: crail.singleton true
>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/10 15:06:14 INFO crail: crail.locationmap
>> 20/02/10 15:06:14 INFO crail: crail.namenode.address
>> crail://sgt-pepper:9060
>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/10 15:06:14 INFO crail: crail.namenode.log
>> 20/02/10 15:06:14 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, bufferCount
>> 1024
>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 16,
>> messageSize 2097152, nodealy false
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true
>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
>> 10.114.222.82:9060
>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
>> done..
>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
>> failed
>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
>> 20/02/10 15:06:15 INFO crail: mapped client cache closed
>>
>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
>> java.io.IOException: Map failed *is not good.
>>
>> Sorry for the long append. Probably something really dumb, but I'm
>> sratching my head...
>>
>> Lou.
>>
>>
>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
>> david.crespi@storedgesystems.com> wrote:
>>
>> > Looks like both cmds are working, but it’s not really finding the
>> > datastore.
>> > You’re getting info into the namenode, but not the datanode (which means
>> > you’re really not writing to the datanode). The
>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
>> > native-hadoop library for your platform)
>> >
>> > you may want to turn on debug to see more of what’s going on.
>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
>> >
>> > You should probably also look at the individual docker logs.
>> > docker logs YourNamenodeName & docker logs YourDatanodeName
>> >
>> > may give you some more hints of what’s happening.
>> >
>> >
>> > Regards,
>> > David
>> >
>> >
>> > From: Lou DeGenaro<ma...@gmail.com>
>> > Sent: Monday, February 10, 2020 8:12 AM
>> > To: dev@crail.apache.org<ma...@crail.apache.org>
>> > Subject: Re: iobench
>> >
>> > OK, I take it back. Having trouble with crail fs.
>> >
>> > =====
>> >
>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
>> > library for your platform... using builtin-java classes where applicable
>> > ...
>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
>> > 10.114.222.23:9060
>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
>> > done..
>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
>> > failed
>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
>> >
>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
>> > library for your platform... using builtin-java classes where applicable
>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
>> > done..
>> > rmdir: /foobar
>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
>> >
>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
>> > library for your platform... using builtin-java classes where applicable
>> > ...
>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
>> > done..
>> > mkdir: `/foobar': File exists
>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
>> >
>> > =====
>> >
>> > Seems that mkdir works, but rmdir does not? Also, IOException was
>> reported
>> > on mkdir, which seems worrisome.
>> >
>> > Lou.
>> >
>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
>> > wrote:
>> >
>> > > David,
>> > >
>> > > Thanks. I'm able to use *crail fs *successfully to
>> create/list/delete.
>> > > Will try to increase available storage space next...
>> > >
>> > > Lou.
>> > >
>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
>> > > david.crespi@storedgesystems.com> wrote:
>> > >
>> > >> Oh, and it may have already written something on the datastore, so
>> you’d
>> > >> need
>> > >> to check it and remove it with the fs command anyway.
>> > >>
>> > >> crail fs -ls -R /
>> > >>
>> > >> would show you what you have there.
>> > >>
>> > >> Regards,
>> > >> David
>> > >>
>> > >> (C) 714-476-2692
>> > >>
>> > >> From: Lou DeGenaro<ma...@gmail.com>
>> > >> Sent: Friday, February 7, 2020 12:15 PM
>> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
>> > >> Subject: iobench
>> > >>
>> > >> Still a noob. Got namenode and datanode running as docker images on
>> my
>> > >> VM. I'm looking for the simplest example of writing something then
>> > >> reading
>> > >> something.
>> > >>
>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
>> > >> /filename
>> > >> -s 1024 -k 1
>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
>> > >> org.apache.crail.memory.MappedBufferCache
>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
>> > >> crail://abbey-road:9060
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection
>> roundrobin
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
>> > >> org.apache.crail.storage.tcp.TcpStorageTier
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
>> > bufferCount
>> > >> 1024
>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>> queueDepth
>> > >> 16,
>> > >> messageSize 2097152, nodealy false
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
>> 1073741824
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
>> > 1073741824
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
>> > >> /dev/hugepages/data
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
>> queueDepth
>> > >> 32,
>> > >> messageSize 512, nodealy true
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
>> > >> 10.114.222.23:9060
>> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
>> > >> locationClass
>> > >> 0, buffered true
>> > >> Exception in thread "main" java.io.IOException: Map failed
>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>> > >> at
>> > >>
>> > >>
>> >
>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
>> > >> at
>> > org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
>> > >> at
>> > >>
>> >
>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
>> > >> at
>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
>> > >> at
>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
>> > >> Caused by: java.lang.OutOfMemoryError: Map failed
>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>> > >> ... 5 more
>> > >>
>> > >>
>> >
>> >
>>
>>
RE: iobench
Posted by David Crespi <da...@storedgesystems.com>.
If I remember correctly, a “map error” is telling you that you’ve run out of memory.
You’re asking for 1GB for each, I assume you have that free? I also see you’re not
using hugepages, so there “shouldn’t” be anything at /dev/hugepages/cache or dev/hugepages/data,
but you have those as the path… /dev is protected, maybe you should try starting as sudo or
move the path to somewhere open (like /tmp).
I think I saw that your cache size was set to 0, so that one won’t matter. I run my containers as --privledged also.
Regards,
David
From: Lou DeGenaro<ma...@gmail.com>
Sent: Tuesday, February 11, 2020 6:19 AM
To: dev@crail.apache.org<ma...@crail.apache.org>
Subject: Re: iobench
I've now removed docker from the picture. I simply download the bin
distribution onto my 8GB VM and copy the templates in conf w/o changes.
======
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
namenode
20/02/11 08:09:08 INFO crail: initalizing namenode
20/02/11 08:09:08 INFO crail: crail.version 3101
20/02/11 08:09:08 INFO crail: crail.directorydepth 16
20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/11 08:09:08 INFO crail: crail.user crail
20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
20/02/11 08:09:08 INFO crail: crail.debug false
20/02/11 08:09:08 INFO crail: crail.statistics true
20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
20/02/11 08:09:08 INFO crail: crail.slicesize 524288
20/02/11 08:09:08 INFO crail: crail.singleton true
20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
20/02/11 08:09:08 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/11 08:09:08 INFO crail: crail.locationmap
20/02/11 08:09:08 INFO crail: crail.namenode.address
crail://localhost:9060?id=0&size=1
20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/11 08:09:08 INFO crail: crail.namenode.log
20/02/11 08:09:08 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/11 08:09:08 INFO crail: crail.storage.classes 1
20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
20/02/11 08:09:08 INFO crail: round robin block selection
20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth 32,
messageSize 512, nodealy true, cores 1
20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
127.0.0.1:40578
=====
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
datanode
20/02/11 08:11:28 INFO crail: crail.version 3101
20/02/11 08:11:28 INFO crail: crail.directorydepth 16
20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/11 08:11:28 INFO crail: crail.user crail
20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
20/02/11 08:11:28 INFO crail: crail.debug false
20/02/11 08:11:28 INFO crail: crail.statistics true
20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
20/02/11 08:11:28 INFO crail: crail.slicesize 524288
20/02/11 08:11:28 INFO crail: crail.singleton true
20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
20/02/11 08:11:28 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/11 08:11:28 INFO crail: crail.locationmap
20/02/11 08:11:28 INFO crail: crail.namenode.address crail://localhost:9060
20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/11 08:11:28 INFO crail: crail.namenode.log
20/02/11 08:11:28 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/11 08:11:28 INFO crail: crail.storage.classes 1
20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false, cores 1
20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
20/02/11 08:11:28 INFO crail: running TCP storage server, address /
10.114.222.23:50020
20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
127.0.0.1:9060
Exception in thread "main" java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
at
org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
... 2 more
=====
Sorry for being a noob. Thx for the help.
Lou.
On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
david.crespi@storedgesystems.com> wrote:
> Was does your core-site and crail-site look like.
> At first thought, it may not be resolving “sgt-pepper”.
> Why not try the actual IP address to eliminate.
>
> Regards,
> David
>
> (C) 714-476-2692
>
> From: Lou DeGenaro<ma...@gmail.com>
> Sent: Monday, February 10, 2020 1:12 PM
> To: dev@crail.apache.org<ma...@crail.apache.org>
> Subject: Re: iobench
>
> I seem to have gone backwards? I started over with new VM. I started 2
> docker containers, one each for name and data:
>
> [root@sgt-pepper ~]# docker run -it --network host -e
> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
> 20/02/10 20:32:44 INFO crail: initalizing namenode
> 20/02/10 20:32:44 INFO crail: crail.version 3101
> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 20:32:44 INFO crail: crail.user crail
> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
> 20/02/10 20:32:44 INFO crail: crail.debug false
> 20/02/10 20:32:44 INFO crail: crail.statistics true
> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
> 20/02/10 20:32:44 INFO crail: crail.singleton true
> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 20:32:44 INFO crail: crail.locationmap
> 20/02/10 20:32:44 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060?id=0&size=1
> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 20:32:44 INFO crail: crail.namenode.log
> 20/02/10 20:32:44 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
> 20/02/10 20:32:44 INFO crail: round robin block selection
> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true, cores 1
> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
> 10.114.222.82:37328
> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of type
> 0 to storage class 0
>
> [root@sgt-pepper ~]# docker run -it --network host -e
> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
> 20/02/10 20:35:36 INFO crail: crail.version 3101
> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 20:35:36 INFO crail: crail.user crail
> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
> 20/02/10 20:35:36 INFO crail: crail.debug false
> 20/02/10 20:35:36 INFO crail: crail.statistics true
> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
> 20/02/10 20:35:36 INFO crail: crail.singleton true
> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 20:35:36 INFO crail: crail.locationmap
> 20/02/10 20:35:36 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 20:35:36 INFO crail: crail.namenode.log
> 20/02/10 20:35:36 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false, cores 1
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
> 10.114.222.82:50020
> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.82:9060
> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
> ...
>
> This seems to be OK so far, since we see *adding datanode
> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
> class 0* in the namenode log.
>
> Then I download the Non-Official Binary Release from here:
> https://crail.apache.org/download/ and I extract and create the 4 conf
> files and the revised log4j properties file:
> [root@sgt-pepper conf]# ls -atl
> total 40
> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
> 2c2
> < log4j.rootCategory=DEBUG, console
> ---
> > log4j.rootCategory=INFO, console
>
> I setup CRAIL_HOME:
>
> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>
> =====
>
> Now I try using the CLI:
>
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
> -ls
> /
> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
> directory
> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
> java.io.BufferedInputStream@71c7db30
> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> successful kerberos logins and latency (milliseconds)])
> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> failed kerberos logins and latency (milliseconds)])
> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[GetGroups])
> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built
> native-hadoop library...
> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 20/02/10 14:54:52 DEBUG Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
> user:UnixPrincipal: root
> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
> root" with name root
> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
> (auth:SIMPLE)
> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
> 20/02/10 14:54:52 INFO crail: crail.version 3101
> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 14:54:52 INFO crail: crail.user crail
> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
> 20/02/10 14:54:52 INFO crail: crail.debug false
> 20/02/10 14:54:52 INFO crail: crail.statistics true
> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
> 20/02/10 14:54:52 INFO crail: crail.singleton true
> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 14:54:52 INFO crail: crail.locationmap
> 20/02/10 14:54:52 INFO crail: crail.namenode.address crail://localhost:9060
> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 14:54:52 INFO crail: crail.namenode.log
> 20/02/10 14:54:52 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
> ls: java.io.IOException: java.net.ConnectException: Connection refused
>
> =====
>
> Since I get connection refused, I try changing "localhost" to "sgt-pepper"
> in the conf files:
>
> [root@sgt-pepper conf]# grep sgt-pepper *
> core-site.xml: <value>crail://sgt-pepper:9060</value>
> crail-site.conf:crail.namenode.address crail://sgt-pepper:9060
>
> Then I try using the CLI again:
>
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
> -ls
> /
> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
> directory
> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
> java.io.BufferedInputStream@71c7db30
> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> successful kerberos logins and latency (milliseconds)])
> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> failed kerberos logins and latency (milliseconds)])
> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[GetGroups])
> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built
> native-hadoop library...
> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 20/02/10 15:00:28 DEBUG Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
> user:UnixPrincipal: root
> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
> root" with name root
> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
> (auth:SIMPLE)
> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
> 20/02/10 15:00:28 INFO crail: crail.version 3101
> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 15:00:28 INFO crail: crail.user crail
> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
> 20/02/10 15:00:28 INFO crail: crail.debug false
> 20/02/10 15:00:28 INFO crail: crail.statistics true
> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
> 20/02/10 15:00:28 INFO crail: crail.singleton true
> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 15:00:28 INFO crail: crail.locationmap
> 20/02/10 15:00:28 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 15:00:28 INFO crail: crail.namenode.log
> 20/02/10 15:00:28 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.82:9060
> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> ls: /
> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
> 20/02/10 15:00:28 INFO crail: mapped client cache closed
>
> Seem better, but not really.
>
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
> -mkdir /foobar
> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
> directory
> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
> java.io.BufferedInputStream@71c7db30
> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> successful kerberos logins and latency (milliseconds)])
> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> failed kerberos logins and latency (milliseconds)])
> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[GetGroups])
> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built
> native-hadoop library...
> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 20/02/10 15:06:14 DEBUG Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
> user:UnixPrincipal: root
> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
> root" with name root
> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
> (auth:SIMPLE)
> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
> 20/02/10 15:06:14 INFO crail: crail.version 3101
> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 15:06:14 INFO crail: crail.user crail
> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
> 20/02/10 15:06:14 INFO crail: crail.debug false
> 20/02/10 15:06:14 INFO crail: crail.statistics true
> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
> 20/02/10 15:06:14 INFO crail: crail.singleton true
> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 15:06:14 INFO crail: crail.locationmap
> 20/02/10 15:06:14 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 15:06:14 INFO crail: crail.namenode.log
> 20/02/10 15:06:14 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.82:9060
> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
> failed
> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
> 20/02/10 15:06:15 INFO crail: mapped client cache closed
>
> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
> java.io.IOException: Map failed *is not good.
>
> Sorry for the long append. Probably something really dumb, but I'm
> sratching my head...
>
> Lou.
>
>
> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
> david.crespi@storedgesystems.com> wrote:
>
> > Looks like both cmds are working, but it’s not really finding the
> > datastore.
> > You’re getting info into the namenode, but not the datanode (which means
> > you’re really not writing to the datanode). The
> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
> > native-hadoop library for your platform)
> >
> > you may want to turn on debug to see more of what’s going on.
> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
> >
> > You should probably also look at the individual docker logs.
> > docker logs YourNamenodeName & docker logs YourDatanodeName
> >
> > may give you some more hints of what’s happening.
> >
> >
> > Regards,
> > David
> >
> >
> > From: Lou DeGenaro<ma...@gmail.com>
> > Sent: Monday, February 10, 2020 8:12 AM
> > To: dev@crail.apache.org<ma...@crail.apache.org>
> > Subject: Re: iobench
> >
> > OK, I take it back. Having trouble with crail fs.
> >
> > =====
> >
> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > ...
> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
> > 10.114.222.23:9060
> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
> > done..
> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
> > failed
> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
> >
> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
> > done..
> > rmdir: /foobar
> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
> >
> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > ...
> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
> > done..
> > mkdir: `/foobar': File exists
> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
> >
> > =====
> >
> > Seems that mkdir works, but rmdir does not? Also, IOException was
> reported
> > on mkdir, which seems worrisome.
> >
> > Lou.
> >
> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
> > wrote:
> >
> > > David,
> > >
> > > Thanks. I'm able to use *crail fs *successfully to create/list/delete.
> > > Will try to increase available storage space next...
> > >
> > > Lou.
> > >
> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> > > david.crespi@storedgesystems.com> wrote:
> > >
> > >> Oh, and it may have already written something on the datastore, so
> you’d
> > >> need
> > >> to check it and remove it with the fs command anyway.
> > >>
> > >> crail fs -ls -R /
> > >>
> > >> would show you what you have there.
> > >>
> > >> Regards,
> > >> David
> > >>
> > >> (C) 714-476-2692
> > >>
> > >> From: Lou DeGenaro<ma...@gmail.com>
> > >> Sent: Friday, February 7, 2020 12:15 PM
> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
> > >> Subject: iobench
> > >>
> > >> Still a noob. Got namenode and datanode running as docker images on
> my
> > >> VM. I'm looking for the simplest example of writing something then
> > >> reading
> > >> something.
> > >>
> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
> > >> /filename
> > >> -s 1024 -k 1
> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
> > >> org.apache.crail.memory.MappedBufferCache
> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
> > >> crail://abbey-road:9060
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
> > >> org.apache.crail.storage.tcp.TcpStorageTier
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
> > bufferCount
> > >> 1024
> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> > >> 16,
> > >> messageSize 2097152, nodealy false
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
> 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
> > 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
> > >> /dev/hugepages/data
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> > >> 32,
> > >> messageSize 512, nodealy true
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
> > >> 10.114.222.23:9060
> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
> > >> locationClass
> > >> 0, buffered true
> > >> Exception in thread "main" java.io.IOException: Map failed
> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> > >> at
> > >>
> > >>
> >
> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
> > >> at
> > org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
> > >> at
> > >>
> >
> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
> > >> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
> > >> at
> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
> > >> Caused by: java.lang.OutOfMemoryError: Map failed
> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> > >> ... 5 more
> > >>
> > >>
> >
> >
>
>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
I've now removed docker from the picture. I simply download the bin
distribution onto my 8GB VM and copy the templates in conf w/o changes.
======
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
namenode
20/02/11 08:09:08 INFO crail: initalizing namenode
20/02/11 08:09:08 INFO crail: crail.version 3101
20/02/11 08:09:08 INFO crail: crail.directorydepth 16
20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10
20/02/11 08:09:08 INFO crail: crail.blocksize 1048576
20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824
20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/11 08:09:08 INFO crail: crail.user crail
20/02/11 08:09:08 INFO crail: crail.shadowreplication 1
20/02/11 08:09:08 INFO crail: crail.debug false
20/02/11 08:09:08 INFO crail: crail.statistics true
20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000
20/02/11 08:09:08 INFO crail: crail.datatimeout 1000
20/02/11 08:09:08 INFO crail: crail.buffersize 1048576
20/02/11 08:09:08 INFO crail: crail.slicesize 524288
20/02/11 08:09:08 INFO crail: crail.singleton true
20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824
20/02/11 08:09:08 INFO crail: crail.directoryrecord 512
20/02/11 08:09:08 INFO crail: crail.directoryrandomize true
20/02/11 08:09:08 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/11 08:09:08 INFO crail: crail.locationmap
20/02/11 08:09:08 INFO crail: crail.namenode.address
crail://localhost:9060?id=0&size=1
20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin
20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16
20/02/11 08:09:08 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/11 08:09:08 INFO crail: crail.namenode.log
20/02/11 08:09:08 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/11 08:09:08 INFO crail: crail.storage.classes 1
20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0
20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2
20/02/11 08:09:08 INFO crail: round robin block selection
20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth 32,
messageSize 512, nodealy true, cores 1
20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512
20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1
20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578
20/02/11 08:11:28 INFO narpc: adding new channel to selector, from /
127.0.0.1:40578
=====
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail
datanode
20/02/11 08:11:28 INFO crail: crail.version 3101
20/02/11 08:11:28 INFO crail: crail.directorydepth 16
20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10
20/02/11 08:11:28 INFO crail: crail.blocksize 1048576
20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824
20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/11 08:11:28 INFO crail: crail.user crail
20/02/11 08:11:28 INFO crail: crail.shadowreplication 1
20/02/11 08:11:28 INFO crail: crail.debug false
20/02/11 08:11:28 INFO crail: crail.statistics true
20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000
20/02/11 08:11:28 INFO crail: crail.datatimeout 1000
20/02/11 08:11:28 INFO crail: crail.buffersize 1048576
20/02/11 08:11:28 INFO crail: crail.slicesize 524288
20/02/11 08:11:28 INFO crail: crail.singleton true
20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824
20/02/11 08:11:28 INFO crail: crail.directoryrecord 512
20/02/11 08:11:28 INFO crail: crail.directoryrandomize true
20/02/11 08:11:28 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/11 08:11:28 INFO crail: crail.locationmap
20/02/11 08:11:28 INFO crail: crail.namenode.address crail://localhost:9060
20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin
20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16
20/02/11 08:11:28 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/11 08:11:28 INFO crail: crail.namenode.log
20/02/11 08:11:28 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/11 08:11:28 INFO crail: crail.storage.classes 1
20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0
20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2
20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false, cores 1
20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0
20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020
20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16
20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1
20/02/11 08:11:28 INFO crail: running TCP storage server, address /
10.114.222.23:50020
20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512
20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1
20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/
127.0.0.1:9060
Exception in thread "main" java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
at
org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83)
at org.apache.crail.storage.StorageServer.main(StorageServer.java:152)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
... 2 more
=====
Sorry for being a noob. Thx for the help.
Lou.
On Mon, Feb 10, 2020 at 6:31 PM David Crespi <
david.crespi@storedgesystems.com> wrote:
> Was does your core-site and crail-site look like.
> At first thought, it may not be resolving “sgt-pepper”.
> Why not try the actual IP address to eliminate.
>
> Regards,
> David
>
> (C) 714-476-2692
>
> From: Lou DeGenaro<ma...@gmail.com>
> Sent: Monday, February 10, 2020 1:12 PM
> To: dev@crail.apache.org<ma...@crail.apache.org>
> Subject: Re: iobench
>
> I seem to have gone backwards? I started over with new VM. I started 2
> docker containers, one each for name and data:
>
> [root@sgt-pepper ~]# docker run -it --network host -e
> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
> 20/02/10 20:32:44 INFO crail: initalizing namenode
> 20/02/10 20:32:44 INFO crail: crail.version 3101
> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16
> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0
> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 20:32:44 INFO crail: crail.user crail
> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
> 20/02/10 20:32:44 INFO crail: crail.debug false
> 20/02/10 20:32:44 INFO crail: crail.statistics true
> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288
> 20/02/10 20:32:44 INFO crail: crail.singleton true
> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
> 20/02/10 20:32:44 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 20:32:44 INFO crail: crail.locationmap
> 20/02/10 20:32:44 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060?id=0&size=1
> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 20:32:44 INFO crail: crail.namenode.log
> 20/02/10 20:32:44 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1
> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
> 20/02/10 20:32:44 INFO crail: round robin block selection
> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true, cores 1
> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
> 10.114.222.82:37328
> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of type
> 0 to storage class 0
>
> [root@sgt-pepper ~]# docker run -it --network host -e
> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
> 20/02/10 20:35:36 INFO crail: crail.version 3101
> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16
> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0
> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 20:35:36 INFO crail: crail.user crail
> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
> 20/02/10 20:35:36 INFO crail: crail.debug false
> 20/02/10 20:35:36 INFO crail: crail.statistics true
> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288
> 20/02/10 20:35:36 INFO crail: crail.singleton true
> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
> 20/02/10 20:35:36 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 20:35:36 INFO crail: crail.locationmap
> 20/02/10 20:35:36 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 20:35:36 INFO crail: crail.namenode.log
> 20/02/10 20:35:36 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1
> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false, cores 1
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 20:35:36 INFO crail: running TCP storage server, address /
> 10.114.222.82:50020
> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.82:9060
> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
> ...
>
> This seems to be OK so far, since we see *adding datanode
> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
> class 0* in the namenode log.
>
> Then I download the Non-Official Binary Release from here:
> https://crail.apache.org/download/ and I extract and create the 4 conf
> files and the revised log4j properties file:
> [root@sgt-pepper conf]# ls -atl
> total 40
> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
> 2c2
> < log4j.rootCategory=DEBUG, console
> ---
> > log4j.rootCategory=INFO, console
>
> I setup CRAIL_HOME:
>
> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>
> =====
>
> Now I try using the CLI:
>
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
> -ls
> /
> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
> directory
> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
> 20/02/10 14:54:51 DEBUG Configuration: parsing URL
>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> 20/02/10 14:54:52 DEBUG Configuration: parsing URL
> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream
> java.io.BufferedInputStream@71c7db30
> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> successful kerberos logins and latency (milliseconds)])
> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> failed kerberos logins and latency (milliseconds)])
> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[GetGroups])
> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built
> native-hadoop library...
> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> 20/02/10 14:54:52 DEBUG NativeCodeLoader:
>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 20/02/10 14:54:52 DEBUG Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local
> user:UnixPrincipal: root
> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
> root" with name root
> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
> (auth:SIMPLE)
> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
> 20/02/10 14:54:52 INFO crail: creating singleton crail file system
> 20/02/10 14:54:52 INFO crail: crail.version 3101
> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16
> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 14:54:52 INFO crail: crail.user crail
> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
> 20/02/10 14:54:52 INFO crail: crail.debug false
> 20/02/10 14:54:52 INFO crail: crail.statistics true
> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288
> 20/02/10 14:54:52 INFO crail: crail.singleton true
> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
> 20/02/10 14:54:52 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 14:54:52 INFO crail: crail.locationmap
> 20/02/10 14:54:52 INFO crail: crail.namenode.address crail://localhost:9060
> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 14:54:52 INFO crail: crail.namenode.log
> 20/02/10 14:54:52 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1
> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
> ls: java.io.IOException: java.net.ConnectException: Connection refused
>
> =====
>
> Since I get connection refused, I try changing "localhost" to "sgt-pepper"
> in the conf files:
>
> [root@sgt-pepper conf]# grep sgt-pepper *
> core-site.xml: <value>crail://sgt-pepper:9060</value>
> crail-site.conf:crail.namenode.address crail://sgt-pepper:9060
>
> Then I try using the CLI again:
>
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
> -ls
> /
> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
> directory
> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> 20/02/10 15:00:27 DEBUG Configuration: parsing URL
> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream
> java.io.BufferedInputStream@71c7db30
> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> successful kerberos logins and latency (milliseconds)])
> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> failed kerberos logins and latency (milliseconds)])
> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[GetGroups])
> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built
> native-hadoop library...
> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> 20/02/10 15:00:28 DEBUG NativeCodeLoader:
>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 20/02/10 15:00:28 DEBUG Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local
> user:UnixPrincipal: root
> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
> root" with name root
> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
> (auth:SIMPLE)
> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
> 20/02/10 15:00:28 INFO crail: creating singleton crail file system
> 20/02/10 15:00:28 INFO crail: crail.version 3101
> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16
> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 15:00:28 INFO crail: crail.user crail
> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
> 20/02/10 15:00:28 INFO crail: crail.debug false
> 20/02/10 15:00:28 INFO crail: crail.statistics true
> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288
> 20/02/10 15:00:28 INFO crail: crail.singleton true
> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
> 20/02/10 15:00:28 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 15:00:28 INFO crail: crail.locationmap
> 20/02/10 15:00:28 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 15:00:28 INFO crail: crail.namenode.log
> 20/02/10 15:00:28 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1
> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.82:9060
> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> ls: /
> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
> 20/02/10 15:00:28 INFO crail: mapped client cache closed
>
> Seem better, but not really.
>
> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
> -mkdir /foobar
> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
> directory
> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
>
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
>
> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
> 20/02/10 15:06:14 DEBUG Configuration: parsing URL
> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream
> java.io.BufferedInputStream@71c7db30
> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> successful kerberos logins and latency (milliseconds)])
> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
> failed kerberos logins and latency (milliseconds)])
> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
> value=[GetGroups])
> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built
> native-hadoop library...
> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> 20/02/10 15:06:14 DEBUG NativeCodeLoader:
>
> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 20/02/10 15:06:14 DEBUG Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local
> user:UnixPrincipal: root
> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
> root" with name root
> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
> (auth:SIMPLE)
> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
> 20/02/10 15:06:14 INFO crail: creating singleton crail file system
> 20/02/10 15:06:14 INFO crail: crail.version 3101
> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16
> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/10 15:06:14 INFO crail: crail.user crail
> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
> 20/02/10 15:06:14 INFO crail: crail.debug false
> 20/02/10 15:06:14 INFO crail: crail.statistics true
> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288
> 20/02/10 15:06:14 INFO crail: crail.singleton true
> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
> 20/02/10 15:06:14 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/10 15:06:14 INFO crail: crail.locationmap
> 20/02/10 15:06:14 INFO crail: crail.namenode.address
> crail://sgt-pepper:9060
> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/10 15:06:14 INFO crail: crail.namenode.log
> 20/02/10 15:06:14 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1
> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
> 10.114.222.82:9060
> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
> failed
> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
> 20/02/10 15:06:15 INFO crail: mapped client cache closed
>
> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
> java.io.IOException: Map failed *is not good.
>
> Sorry for the long append. Probably something really dumb, but I'm
> sratching my head...
>
> Lou.
>
>
> On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
> david.crespi@storedgesystems.com> wrote:
>
> > Looks like both cmds are working, but it’s not really finding the
> > datastore.
> > You’re getting info into the namenode, but not the datanode (which means
> > you’re really not writing to the datanode). The
> > First warning you can ignore (WARN NativeCodeLoader: Unable to load
> > native-hadoop library for your platform)
> >
> > you may want to turn on debug to see more of what’s going on.
> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
> >
> > You should probably also look at the individual docker logs.
> > docker logs YourNamenodeName & docker logs YourDatanodeName
> >
> > may give you some more hints of what’s happening.
> >
> >
> > Regards,
> > David
> >
> >
> > From: Lou DeGenaro<ma...@gmail.com>
> > Sent: Monday, February 10, 2020 8:12 AM
> > To: dev@crail.apache.org<ma...@crail.apache.org>
> > Subject: Re: iobench
> >
> > OK, I take it back. Having trouble with crail fs.
> >
> > =====
> >
> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > ...
> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
> > 10.114.222.23:9060
> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
> > done..
> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
> > failed
> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
> > 20/02/10 10:07:16 INFO crail: mapped client cache closed
> >
> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
> > done..
> > rmdir: /foobar
> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
> > 20/02/10 10:08:55 INFO crail: mapped client cache closed
> >
> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > ...
> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
> > done..
> > mkdir: `/foobar': File exists
> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
> > 20/02/10 10:09:43 INFO crail: mapped client cache closed
> >
> > =====
> >
> > Seems that mkdir works, but rmdir does not? Also, IOException was
> reported
> > on mkdir, which seems worrisome.
> >
> > Lou.
> >
> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
> > wrote:
> >
> > > David,
> > >
> > > Thanks. I'm able to use *crail fs *successfully to create/list/delete.
> > > Will try to increase available storage space next...
> > >
> > > Lou.
> > >
> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> > > david.crespi@storedgesystems.com> wrote:
> > >
> > >> Oh, and it may have already written something on the datastore, so
> you’d
> > >> need
> > >> to check it and remove it with the fs command anyway.
> > >>
> > >> crail fs -ls -R /
> > >>
> > >> would show you what you have there.
> > >>
> > >> Regards,
> > >> David
> > >>
> > >> (C) 714-476-2692
> > >>
> > >> From: Lou DeGenaro<ma...@gmail.com>
> > >> Sent: Friday, February 7, 2020 12:15 PM
> > >> To: dev@crail.apache.org<ma...@crail.apache.org>
> > >> Subject: iobench
> > >>
> > >> Still a noob. Got namenode and datanode running as docker images on
> my
> > >> VM. I'm looking for the simplest example of writing something then
> > >> reading
> > >> something.
> > >>
> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
> > >> /filename
> > >> -s 1024 -k 1
> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101
> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
> > >> 20/02/07 14:10:23 INFO crail: crail.user crail
> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
> > >> 20/02/07 14:10:23 INFO crail: crail.debug false
> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true
> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true
> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
> > >> org.apache.crail.memory.MappedBufferCache
> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
> > >> crail://abbey-road:9060
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types
> > >> org.apache.crail.storage.tcp.TcpStorageTier
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
> > bufferCount
> > >> 1024
> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> > >> 16,
> > >> messageSize 2097152, nodealy false
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit
> 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
> > 1073741824
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
> > >> /dev/hugepages/data
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0,
> queueDepth
> > >> 32,
> > >> messageSize 512, nodealy true
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
> > >> 10.114.222.23:9060
> > >> write, filename /filename, size 1024, loop 1, storageClass 0,
> > >> locationClass
> > >> 0, buffered true
> > >> Exception in thread "main" java.io.IOException: Map failed
> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> > >> at
> > >>
> > >>
> >
> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
> > >> at
> > org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
> > >> at
> > >>
> >
> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
> > >> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
> > >> at
> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
> > >> Caused by: java.lang.OutOfMemoryError: Map failed
> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> > >> ... 5 more
> > >>
> > >>
> >
> >
>
>
RE: iobench
Posted by David Crespi <da...@storedgesystems.com>.
Was does your core-site and crail-site look like.
At first thought, it may not be resolving “sgt-pepper”.
Why not try the actual IP address to eliminate.
Regards,
David
(C) 714-476-2692
From: Lou DeGenaro<ma...@gmail.com>
Sent: Monday, February 10, 2020 1:12 PM
To: dev@crail.apache.org<ma...@crail.apache.org>
Subject: Re: iobench
I seem to have gone backwards? I started over with new VM. I started 2
docker containers, one each for name and data:
[root@sgt-pepper ~]# docker run -it --network host -e
NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
20/02/10 20:32:44 INFO crail: initalizing namenode
20/02/10 20:32:44 INFO crail: crail.version 3101
20/02/10 20:32:44 INFO crail: crail.directorydepth 16
20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
20/02/10 20:32:44 INFO crail: crail.cachelimit 0
20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 20:32:44 INFO crail: crail.user crail
20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
20/02/10 20:32:44 INFO crail: crail.debug false
20/02/10 20:32:44 INFO crail: crail.statistics true
20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
20/02/10 20:32:44 INFO crail: crail.slicesize 524288
20/02/10 20:32:44 INFO crail: crail.singleton true
20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
20/02/10 20:32:44 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 20:32:44 INFO crail: crail.locationmap
20/02/10 20:32:44 INFO crail: crail.namenode.address
crail://sgt-pepper:9060?id=0&size=1
20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 20:32:44 INFO crail: crail.namenode.log
20/02/10 20:32:44 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 20:32:44 INFO crail: crail.storage.classes 1
20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
20/02/10 20:32:44 INFO crail: round robin block selection
20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth 32,
messageSize 512, nodealy true, cores 1
20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
10.114.222.82:37328
20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of type
0 to storage class 0
[root@sgt-pepper ~]# docker run -it --network host -e
NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
20/02/10 20:35:36 INFO crail: crail.version 3101
20/02/10 20:35:36 INFO crail: crail.directorydepth 16
20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
20/02/10 20:35:36 INFO crail: crail.cachelimit 0
20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 20:35:36 INFO crail: crail.user crail
20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
20/02/10 20:35:36 INFO crail: crail.debug false
20/02/10 20:35:36 INFO crail: crail.statistics true
20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
20/02/10 20:35:36 INFO crail: crail.slicesize 524288
20/02/10 20:35:36 INFO crail: crail.singleton true
20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
20/02/10 20:35:36 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 20:35:36 INFO crail: crail.locationmap
20/02/10 20:35:36 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 20:35:36 INFO crail: crail.namenode.log
20/02/10 20:35:36 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 20:35:36 INFO crail: crail.storage.classes 1
20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false, cores 1
20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
20/02/10 20:35:36 INFO crail: running TCP storage server, address /
10.114.222.82:50020
20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.82:9060
20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
...
This seems to be OK so far, since we see *adding datanode
/10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
class 0* in the namenode log.
Then I download the Non-Official Binary Release from here:
https://crail.apache.org/download/ and I extract and create the 4 conf
files and the revised log4j properties file:
[root@sgt-pepper conf]# ls -atl
total 40
drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
-rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
-rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
-rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
-rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
-rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
-rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
-rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
-rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
-rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
-rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
[root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
2c2
< log4j.rootCategory=DEBUG, console
---
> log4j.rootCategory=INFO, console
I setup CRAIL_HOME:
[root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
[root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
=====
Now I try using the CLI:
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs -ls
/
20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
20/02/10 14:54:51 DEBUG Configuration: parsing URL
jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
20/02/10 14:54:51 DEBUG Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
20/02/10 14:54:52 DEBUG Configuration: parsing URL
file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
20/02/10 14:54:52 DEBUG Configuration: parsing input stream
java.io.BufferedInputStream@71c7db30
20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
successful kerberos logins and latency (milliseconds)])
20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
failed kerberos logins and latency (milliseconds)])
20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
value=[GetGroups])
20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
related metrics
20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
20/02/10 14:54:52 DEBUG NativeCodeLoader:
java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
20/02/10 14:54:52 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
20/02/10 14:54:52 DEBUG UserGroupInformation: using local
user:UnixPrincipal: root
20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
root" with name root
20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
(auth:SIMPLE)
20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
20/02/10 14:54:52 INFO crail: creating singleton crail file system
20/02/10 14:54:52 INFO crail: crail.version 3101
20/02/10 14:54:52 INFO crail: crail.directorydepth 16
20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 14:54:52 INFO crail: crail.user crail
20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
20/02/10 14:54:52 INFO crail: crail.debug false
20/02/10 14:54:52 INFO crail: crail.statistics true
20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
20/02/10 14:54:52 INFO crail: crail.slicesize 524288
20/02/10 14:54:52 INFO crail: crail.singleton true
20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
20/02/10 14:54:52 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 14:54:52 INFO crail: crail.locationmap
20/02/10 14:54:52 INFO crail: crail.namenode.address crail://localhost:9060
20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 14:54:52 INFO crail: crail.namenode.log
20/02/10 14:54:52 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 14:54:52 INFO crail: crail.storage.classes 1
20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
ls: java.io.IOException: java.net.ConnectException: Connection refused
=====
Since I get connection refused, I try changing "localhost" to "sgt-pepper"
in the conf files:
[root@sgt-pepper conf]# grep sgt-pepper *
core-site.xml: <value>crail://sgt-pepper:9060</value>
crail-site.conf:crail.namenode.address crail://sgt-pepper:9060
Then I try using the CLI again:
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs -ls
/
20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
20/02/10 15:00:27 DEBUG Configuration: parsing URL
jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
20/02/10 15:00:27 DEBUG Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
20/02/10 15:00:27 DEBUG Configuration: parsing URL
file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
20/02/10 15:00:27 DEBUG Configuration: parsing input stream
java.io.BufferedInputStream@71c7db30
20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
successful kerberos logins and latency (milliseconds)])
20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
failed kerberos logins and latency (milliseconds)])
20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
value=[GetGroups])
20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
related metrics
20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
20/02/10 15:00:28 DEBUG NativeCodeLoader:
java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
20/02/10 15:00:28 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
20/02/10 15:00:28 DEBUG UserGroupInformation: using local
user:UnixPrincipal: root
20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
root" with name root
20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
(auth:SIMPLE)
20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
20/02/10 15:00:28 INFO crail: creating singleton crail file system
20/02/10 15:00:28 INFO crail: crail.version 3101
20/02/10 15:00:28 INFO crail: crail.directorydepth 16
20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 15:00:28 INFO crail: crail.user crail
20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
20/02/10 15:00:28 INFO crail: crail.debug false
20/02/10 15:00:28 INFO crail: crail.statistics true
20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
20/02/10 15:00:28 INFO crail: crail.slicesize 524288
20/02/10 15:00:28 INFO crail: crail.singleton true
20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
20/02/10 15:00:28 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 15:00:28 INFO crail: crail.locationmap
20/02/10 15:00:28 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 15:00:28 INFO crail: crail.namenode.log
20/02/10 15:00:28 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 15:00:28 INFO crail: crail.storage.classes 1
20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.82:9060
20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization done..
ls: /
20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
20/02/10 15:00:28 INFO crail: mapped client cache closed
Seem better, but not really.
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
-mkdir /foobar
20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
20/02/10 15:06:14 DEBUG Configuration: parsing URL
jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
20/02/10 15:06:14 DEBUG Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
20/02/10 15:06:14 DEBUG Configuration: parsing URL
file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
20/02/10 15:06:14 DEBUG Configuration: parsing input stream
java.io.BufferedInputStream@71c7db30
20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
successful kerberos logins and latency (milliseconds)])
20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
failed kerberos logins and latency (milliseconds)])
20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
value=[GetGroups])
20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
related metrics
20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
20/02/10 15:06:14 DEBUG NativeCodeLoader:
java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
20/02/10 15:06:14 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
20/02/10 15:06:14 DEBUG UserGroupInformation: using local
user:UnixPrincipal: root
20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
root" with name root
20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
(auth:SIMPLE)
20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
20/02/10 15:06:14 INFO crail: creating singleton crail file system
20/02/10 15:06:14 INFO crail: crail.version 3101
20/02/10 15:06:14 INFO crail: crail.directorydepth 16
20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 15:06:14 INFO crail: crail.user crail
20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
20/02/10 15:06:14 INFO crail: crail.debug false
20/02/10 15:06:14 INFO crail: crail.statistics true
20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
20/02/10 15:06:14 INFO crail: crail.slicesize 524288
20/02/10 15:06:14 INFO crail: crail.singleton true
20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
20/02/10 15:06:14 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 15:06:14 INFO crail: crail.locationmap
20/02/10 15:06:14 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 15:06:14 INFO crail: crail.namenode.log
20/02/10 15:06:14 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 15:06:14 INFO crail: crail.storage.classes 1
20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.82:9060
20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization done..
mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
failed
20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
20/02/10 15:06:15 INFO crail: mapped client cache closed
I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
java.io.IOException: Map failed *is not good.
Sorry for the long append. Probably something really dumb, but I'm
sratching my head...
Lou.
On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
david.crespi@storedgesystems.com> wrote:
> Looks like both cmds are working, but it’s not really finding the
> datastore.
> You’re getting info into the namenode, but not the datanode (which means
> you’re really not writing to the datanode). The
> First warning you can ignore (WARN NativeCodeLoader: Unable to load
> native-hadoop library for your platform)
>
> you may want to turn on debug to see more of what’s going on.
> Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
>
> You should probably also look at the individual docker logs.
> docker logs YourNamenodeName & docker logs YourDatanodeName
>
> may give you some more hints of what’s happening.
>
>
> Regards,
> David
>
>
> From: Lou DeGenaro<ma...@gmail.com>
> Sent: Monday, February 10, 2020 8:12 AM
> To: dev@crail.apache.org<ma...@crail.apache.org>
> Subject: Re: iobench
>
> OK, I take it back. Having trouble with crail fs.
>
> =====
>
> [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ...
> 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
> 10.114.222.23:9060
> 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
> failed
> 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
> 20/02/10 10:07:16 INFO crail: mapped client cache closed
>
> [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
> 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> rmdir: /foobar
> 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
> 20/02/10 10:08:55 INFO crail: mapped client cache closed
>
> [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ...
> 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> mkdir: `/foobar': File exists
> 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
> 20/02/10 10:09:43 INFO crail: mapped client cache closed
>
> =====
>
> Seems that mkdir works, but rmdir does not? Also, IOException was reported
> on mkdir, which seems worrisome.
>
> Lou.
>
> On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
> wrote:
>
> > David,
> >
> > Thanks. I'm able to use *crail fs *successfully to create/list/delete.
> > Will try to increase available storage space next...
> >
> > Lou.
> >
> > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> > david.crespi@storedgesystems.com> wrote:
> >
> >> Oh, and it may have already written something on the datastore, so you’d
> >> need
> >> to check it and remove it with the fs command anyway.
> >>
> >> crail fs -ls -R /
> >>
> >> would show you what you have there.
> >>
> >> Regards,
> >> David
> >>
> >> (C) 714-476-2692
> >>
> >> From: Lou DeGenaro<ma...@gmail.com>
> >> Sent: Friday, February 7, 2020 12:15 PM
> >> To: dev@crail.apache.org<ma...@crail.apache.org>
> >> Subject: iobench
> >>
> >> Still a noob. Got namenode and datanode running as docker images on my
> >> VM. I'm looking for the simplest example of writing something then
> >> reading
> >> something.
> >>
> >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
> >> /filename
> >> -s 1024 -k 1
> >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
> >> 20/02/07 14:10:23 INFO crail: crail.version 3101
> >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
> >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
> >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
> >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
> >> 20/02/07 14:10:23 INFO crail: crail.user crail
> >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
> >> 20/02/07 14:10:23 INFO crail: crail.debug false
> >> 20/02/07 14:10:23 INFO crail: crail.statistics true
> >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
> >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
> >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
> >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
> >> 20/02/07 14:10:23 INFO crail: crail.singleton true
> >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
> >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
> >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
> >> org.apache.crail.memory.MappedBufferCache
> >> 20/02/07 14:10:23 INFO crail: crail.locationmap
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
> >> crail://abbey-road:9060
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
> >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
> >> 20/02/07 14:10:23 INFO crail: crail.storage.types
> >> org.apache.crail.storage.tcp.TcpStorageTier
> >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
> >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
> >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
> >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
> bufferCount
> >> 1024
> >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> >> 16,
> >> messageSize 2097152, nodealy false
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
> 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
> >> /dev/hugepages/data
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
> >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> >> 32,
> >> messageSize 512, nodealy true
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
> >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
> >> 10.114.222.23:9060
> >> write, filename /filename, size 1024, loop 1, storageClass 0,
> >> locationClass
> >> 0, buffered true
> >> Exception in thread "main" java.io.IOException: Map failed
> >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> >> at
> >>
> >>
> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
> >> at
> org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
> >> at
> >>
> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
> >> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
> >> at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
> >> Caused by: java.lang.OutOfMemoryError: Map failed
> >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> >> ... 5 more
> >>
> >>
>
>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
I seem to have gone backwards? I started over with new VM. I started 2
docker containers, one each for name and data:
[root@sgt-pepper ~]# docker run -it --network host -e
NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode
20/02/10 20:32:44 INFO crail: initalizing namenode
20/02/10 20:32:44 INFO crail: crail.version 3101
20/02/10 20:32:44 INFO crail: crail.directorydepth 16
20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10
20/02/10 20:32:44 INFO crail: crail.blocksize 1048576
20/02/10 20:32:44 INFO crail: crail.cachelimit 0
20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 20:32:44 INFO crail: crail.user crail
20/02/10 20:32:44 INFO crail: crail.shadowreplication 1
20/02/10 20:32:44 INFO crail: crail.debug false
20/02/10 20:32:44 INFO crail: crail.statistics true
20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000
20/02/10 20:32:44 INFO crail: crail.datatimeout 1000
20/02/10 20:32:44 INFO crail: crail.buffersize 1048576
20/02/10 20:32:44 INFO crail: crail.slicesize 524288
20/02/10 20:32:44 INFO crail: crail.singleton true
20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824
20/02/10 20:32:44 INFO crail: crail.directoryrecord 512
20/02/10 20:32:44 INFO crail: crail.directoryrandomize true
20/02/10 20:32:44 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 20:32:44 INFO crail: crail.locationmap
20/02/10 20:32:44 INFO crail: crail.namenode.address
crail://sgt-pepper:9060?id=0&size=1
20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16
20/02/10 20:32:44 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 20:32:44 INFO crail: crail.namenode.log
20/02/10 20:32:44 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 20:32:44 INFO crail: crail.storage.classes 1
20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0
20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2
20/02/10 20:32:44 INFO crail: round robin block selection
20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth 32,
messageSize 512, nodealy true, cores 1
20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1
20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328
20/02/10 20:35:36 INFO narpc: adding new channel to selector, from /
10.114.222.82:37328
20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of type
0 to storage class 0
[root@sgt-pepper ~]# docker run -it --network host -e
NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode
20/02/10 20:35:36 INFO crail: crail.version 3101
20/02/10 20:35:36 INFO crail: crail.directorydepth 16
20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10
20/02/10 20:35:36 INFO crail: crail.blocksize 1048576
20/02/10 20:35:36 INFO crail: crail.cachelimit 0
20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 20:35:36 INFO crail: crail.user crail
20/02/10 20:35:36 INFO crail: crail.shadowreplication 1
20/02/10 20:35:36 INFO crail: crail.debug false
20/02/10 20:35:36 INFO crail: crail.statistics true
20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000
20/02/10 20:35:36 INFO crail: crail.datatimeout 1000
20/02/10 20:35:36 INFO crail: crail.buffersize 1048576
20/02/10 20:35:36 INFO crail: crail.slicesize 524288
20/02/10 20:35:36 INFO crail: crail.singleton true
20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824
20/02/10 20:35:36 INFO crail: crail.directoryrecord 512
20/02/10 20:35:36 INFO crail: crail.directoryrandomize true
20/02/10 20:35:36 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 20:35:36 INFO crail: crail.locationmap
20/02/10 20:35:36 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16
20/02/10 20:35:36 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 20:35:36 INFO crail: crail.namenode.log
20/02/10 20:35:36 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 20:35:36 INFO crail: crail.storage.classes 1
20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0
20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2
20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false, cores 1
20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0
20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020
20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1
20/02/10 20:35:36 INFO crail: running TCP storage server, address /
10.114.222.82:50020
20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1
20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.82:9060
20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024
...
This seems to be OK so far, since we see *adding datanode
/10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage
class 0* in the namenode log.
Then I download the Non-Official Binary Release from here:
https://crail.apache.org/download/ and I extract and create the 4 conf
files and the revised log4j properties file:
[root@sgt-pepper conf]# ls -atl
total 40
drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 .
-rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties
drwxr-xr-x. 8 root root 4096 Feb 10 13:52 ..
-rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml
-rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template
-rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh
-rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template
-rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf
-rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template
-rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig
-rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves
-rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template
[root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig
2c2
< log4j.rootCategory=DEBUG, console
---
> log4j.rootCategory=INFO, console
I setup CRAIL_HOME:
[root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD
[root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL
CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
=====
Now I try using the CLI:
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs -ls
/
20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0
20/02/10 14:54:51 DEBUG Configuration: parsing URL
jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
20/02/10 14:54:51 DEBUG Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
20/02/10 14:54:52 DEBUG Configuration: parsing URL
file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
20/02/10 14:54:52 DEBUG Configuration: parsing input stream
java.io.BufferedInputStream@71c7db30
20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
successful kerberos logins and latency (milliseconds)])
20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
failed kerberos logins and latency (milliseconds)])
20/02/10 14:54:52 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
value=[GetGroups])
20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group
related metrics
20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
20/02/10 14:54:52 DEBUG Groups: Creating new Groups object
20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
20/02/10 14:54:52 DEBUG NativeCodeLoader:
java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based
20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
20/02/10 14:54:52 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login
20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit
20/02/10 14:54:52 DEBUG UserGroupInformation: using local
user:UnixPrincipal: root
20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
root" with name root
20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root"
20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root
(auth:SIMPLE)
20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction
20/02/10 14:54:52 INFO crail: creating singleton crail file system
20/02/10 14:54:52 INFO crail: crail.version 3101
20/02/10 14:54:52 INFO crail: crail.directorydepth 16
20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10
20/02/10 14:54:52 INFO crail: crail.blocksize 1048576
20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824
20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 14:54:52 INFO crail: crail.user crail
20/02/10 14:54:52 INFO crail: crail.shadowreplication 1
20/02/10 14:54:52 INFO crail: crail.debug false
20/02/10 14:54:52 INFO crail: crail.statistics true
20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000
20/02/10 14:54:52 INFO crail: crail.datatimeout 1000
20/02/10 14:54:52 INFO crail: crail.buffersize 1048576
20/02/10 14:54:52 INFO crail: crail.slicesize 524288
20/02/10 14:54:52 INFO crail: crail.singleton true
20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824
20/02/10 14:54:52 INFO crail: crail.directoryrecord 512
20/02/10 14:54:52 INFO crail: crail.directoryrandomize true
20/02/10 14:54:52 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 14:54:52 INFO crail: crail.locationmap
20/02/10 14:54:52 INFO crail: crail.namenode.address crail://localhost:9060
20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16
20/02/10 14:54:52 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 14:54:52 INFO crail: crail.namenode.log
20/02/10 14:54:52 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 14:54:52 INFO crail: crail.storage.classes 1
20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0
20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2
20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0
20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020
20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1
20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1
ls: java.io.IOException: java.net.ConnectException: Connection refused
=====
Since I get connection refused, I try changing "localhost" to "sgt-pepper"
in the conf files:
[root@sgt-pepper conf]# grep sgt-pepper *
core-site.xml: <value>crail://sgt-pepper:9060</value>
crail-site.conf:crail.namenode.address crail://sgt-pepper:9060
Then I try using the CLI again:
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs -ls
/
20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0
20/02/10 15:00:27 DEBUG Configuration: parsing URL
jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
20/02/10 15:00:27 DEBUG Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
20/02/10 15:00:27 DEBUG Configuration: parsing URL
file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
20/02/10 15:00:27 DEBUG Configuration: parsing input stream
java.io.BufferedInputStream@71c7db30
20/02/10 15:00:27 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
successful kerberos logins and latency (milliseconds)])
20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
failed kerberos logins and latency (milliseconds)])
20/02/10 15:00:28 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
value=[GetGroups])
20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group
related metrics
20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
20/02/10 15:00:28 DEBUG Groups: Creating new Groups object
20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
20/02/10 15:00:28 DEBUG NativeCodeLoader:
java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based
20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
20/02/10 15:00:28 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login
20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit
20/02/10 15:00:28 DEBUG UserGroupInformation: using local
user:UnixPrincipal: root
20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
root" with name root
20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root"
20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root
(auth:SIMPLE)
20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction
20/02/10 15:00:28 INFO crail: creating singleton crail file system
20/02/10 15:00:28 INFO crail: crail.version 3101
20/02/10 15:00:28 INFO crail: crail.directorydepth 16
20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10
20/02/10 15:00:28 INFO crail: crail.blocksize 1048576
20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824
20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 15:00:28 INFO crail: crail.user crail
20/02/10 15:00:28 INFO crail: crail.shadowreplication 1
20/02/10 15:00:28 INFO crail: crail.debug false
20/02/10 15:00:28 INFO crail: crail.statistics true
20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000
20/02/10 15:00:28 INFO crail: crail.datatimeout 1000
20/02/10 15:00:28 INFO crail: crail.buffersize 1048576
20/02/10 15:00:28 INFO crail: crail.slicesize 524288
20/02/10 15:00:28 INFO crail: crail.singleton true
20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824
20/02/10 15:00:28 INFO crail: crail.directoryrecord 512
20/02/10 15:00:28 INFO crail: crail.directoryrandomize true
20/02/10 15:00:28 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 15:00:28 INFO crail: crail.locationmap
20/02/10 15:00:28 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16
20/02/10 15:00:28 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 15:00:28 INFO crail: crail.namenode.log
20/02/10 15:00:28 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 15:00:28 INFO crail: crail.storage.classes 1
20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0
20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2
20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0
20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020
20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1
20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1
20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.82:9060
20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization done..
ls: /
20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem
20/02/10 15:00:28 INFO crail: Closing CrailFS singleton
20/02/10 15:00:28 INFO crail: mapped client cache closed
Seem better, but not really.
[root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs
-mkdir /foobar
20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0
20/02/10 15:06:14 DEBUG Configuration: parsing URL
jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml
20/02/10 15:06:14 DEBUG Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8
20/02/10 15:06:14 DEBUG Configuration: parsing URL
file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml
20/02/10 15:06:14 DEBUG Configuration: parsing input stream
java.io.BufferedInputStream@71c7db30
20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
successful kerberos logins and latency (milliseconds)])
20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of
failed kerberos logins and latency (milliseconds)])
20/02/10 15:06:14 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
sampleName=Ops, always=false, type=DEFAULT, valueName=Time,
value=[GetGroups])
20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group
related metrics
20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
20/02/10 15:06:14 DEBUG Groups: Creating new Groups object
20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
20/02/10 15:06:14 DEBUG NativeCodeLoader:
java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based
20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
20/02/10 15:06:14 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login
20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit
20/02/10 15:06:14 DEBUG UserGroupInformation: using local
user:UnixPrincipal: root
20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
root" with name root
20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root"
20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root
(auth:SIMPLE)
20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction
20/02/10 15:06:14 INFO crail: creating singleton crail file system
20/02/10 15:06:14 INFO crail: crail.version 3101
20/02/10 15:06:14 INFO crail: crail.directorydepth 16
20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10
20/02/10 15:06:14 INFO crail: crail.blocksize 1048576
20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824
20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/10 15:06:14 INFO crail: crail.user crail
20/02/10 15:06:14 INFO crail: crail.shadowreplication 1
20/02/10 15:06:14 INFO crail: crail.debug false
20/02/10 15:06:14 INFO crail: crail.statistics true
20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000
20/02/10 15:06:14 INFO crail: crail.datatimeout 1000
20/02/10 15:06:14 INFO crail: crail.buffersize 1048576
20/02/10 15:06:14 INFO crail: crail.slicesize 524288
20/02/10 15:06:14 INFO crail: crail.singleton true
20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824
20/02/10 15:06:14 INFO crail: crail.directoryrecord 512
20/02/10 15:06:14 INFO crail: crail.directoryrandomize true
20/02/10 15:06:14 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/10 15:06:14 INFO crail: crail.locationmap
20/02/10 15:06:14 INFO crail: crail.namenode.address crail://sgt-pepper:9060
20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin
20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16
20/02/10 15:06:14 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/10 15:06:14 INFO crail: crail.namenode.log
20/02/10 15:06:14 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/10 15:06:14 INFO crail: crail.storage.classes 1
20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0
20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2
20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0
20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020
20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16
20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1
20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512
20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1
20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/
10.114.222.82:9060
20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization done..
mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
failed
20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem
20/02/10 15:06:15 INFO crail: Closing CrailFS singleton
20/02/10 15:06:15 INFO crail: mapped client cache closed
I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException:
java.io.IOException: Map failed *is not good.
Sorry for the long append. Probably something really dumb, but I'm
sratching my head...
Lou.
On Mon, Feb 10, 2020 at 12:13 PM David Crespi <
david.crespi@storedgesystems.com> wrote:
> Looks like both cmds are working, but it’s not really finding the
> datastore.
> You’re getting info into the namenode, but not the datanode (which means
> you’re really not writing to the datanode). The
> First warning you can ignore (WARN NativeCodeLoader: Unable to load
> native-hadoop library for your platform)
>
> you may want to turn on debug to see more of what’s going on.
> Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
>
> You should probably also look at the individual docker logs.
> docker logs YourNamenodeName & docker logs YourDatanodeName
>
> may give you some more hints of what’s happening.
>
>
> Regards,
> David
>
>
> From: Lou DeGenaro<ma...@gmail.com>
> Sent: Monday, February 10, 2020 8:12 AM
> To: dev@crail.apache.org<ma...@crail.apache.org>
> Subject: Re: iobench
>
> OK, I take it back. Having trouble with crail fs.
>
> =====
>
> [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ...
> 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
> 10.114.222.23:9060
> 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
> failed
> 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
> 20/02/10 10:07:16 INFO crail: mapped client cache closed
>
> [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
> 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> rmdir: /foobar
> 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
> 20/02/10 10:08:55 INFO crail: mapped client cache closed
>
> [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
> 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> ...
> 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization
> done..
> mkdir: `/foobar': File exists
> 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
> 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
> 20/02/10 10:09:43 INFO crail: mapped client cache closed
>
> =====
>
> Seems that mkdir works, but rmdir does not? Also, IOException was reported
> on mkdir, which seems worrisome.
>
> Lou.
>
> On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com>
> wrote:
>
> > David,
> >
> > Thanks. I'm able to use *crail fs *successfully to create/list/delete.
> > Will try to increase available storage space next...
> >
> > Lou.
> >
> > On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> > david.crespi@storedgesystems.com> wrote:
> >
> >> Oh, and it may have already written something on the datastore, so you’d
> >> need
> >> to check it and remove it with the fs command anyway.
> >>
> >> crail fs -ls -R /
> >>
> >> would show you what you have there.
> >>
> >> Regards,
> >> David
> >>
> >> (C) 714-476-2692
> >>
> >> From: Lou DeGenaro<ma...@gmail.com>
> >> Sent: Friday, February 7, 2020 12:15 PM
> >> To: dev@crail.apache.org<ma...@crail.apache.org>
> >> Subject: iobench
> >>
> >> Still a noob. Got namenode and datanode running as docker images on my
> >> VM. I'm looking for the simplest example of writing something then
> >> reading
> >> something.
> >>
> >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
> >> /filename
> >> -s 1024 -k 1
> >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
> >> 20/02/07 14:10:23 INFO crail: crail.version 3101
> >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
> >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
> >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
> >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
> >> 20/02/07 14:10:23 INFO crail: crail.user crail
> >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
> >> 20/02/07 14:10:23 INFO crail: crail.debug false
> >> 20/02/07 14:10:23 INFO crail: crail.statistics true
> >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
> >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
> >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
> >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
> >> 20/02/07 14:10:23 INFO crail: crail.singleton true
> >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
> >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
> >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
> >> org.apache.crail.memory.MappedBufferCache
> >> 20/02/07 14:10:23 INFO crail: crail.locationmap
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.address
> >> crail://abbey-road:9060
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
> >> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.log
> >> 20/02/07 14:10:23 INFO crail: crail.storage.types
> >> org.apache.crail.storage.tcp.TcpStorageTier
> >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
> >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
> >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
> >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1,
> bufferCount
> >> 1024
> >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> >> 16,
> >> messageSize 2097152, nodealy false
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize
> 1073741824
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
> >> /dev/hugepages/data
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
> >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
> >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
> >> 32,
> >> messageSize 512, nodealy true
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
> >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
> >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
> >> 10.114.222.23:9060
> >> write, filename /filename, size 1024, loop 1, storageClass 0,
> >> locationClass
> >> 0, buffered true
> >> Exception in thread "main" java.io.IOException: Map failed
> >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> >> at
> >>
> >>
> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
> >> at
> org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
> >> at
> >>
> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
> >> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
> >> at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
> >> Caused by: java.lang.OutOfMemoryError: Map failed
> >> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> >> ... 5 more
> >>
> >>
>
>
RE: iobench
Posted by David Crespi <da...@storedgesystems.com>.
Looks like both cmds are working, but it’s not really finding the datastore.
You’re getting info into the namenode, but not the datanode (which means
you’re really not writing to the datanode). The
First warning you can ignore (WARN NativeCodeLoader: Unable to load
native-hadoop library for your platform)
you may want to turn on debug to see more of what’s going on.
Edit /crail/conf/log4j.properties and change from INFO to DEBUG.
You should probably also look at the individual docker logs.
docker logs YourNamenodeName & docker logs YourDatanodeName
may give you some more hints of what’s happening.
Regards,
David
From: Lou DeGenaro<ma...@gmail.com>
Sent: Monday, February 10, 2020 8:12 AM
To: dev@crail.apache.org<ma...@crail.apache.org>
Subject: Re: iobench
OK, I take it back. Having trouble with crail fs.
=====
[root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
...
20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
10.114.222.23:9060
20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization done..
mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
failed
20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
20/02/10 10:07:16 INFO crail: mapped client cache closed
[root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
done..
rmdir: /foobar
20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
20/02/10 10:08:55 INFO crail: mapped client cache closed
[root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
...
20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization done..
mkdir: `/foobar': File exists
20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
20/02/10 10:09:43 INFO crail: mapped client cache closed
=====
Seems that mkdir works, but rmdir does not? Also, IOException was reported
on mkdir, which seems worrisome.
Lou.
On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com> wrote:
> David,
>
> Thanks. I'm able to use *crail fs *successfully to create/list/delete.
> Will try to increase available storage space next...
>
> Lou.
>
> On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> david.crespi@storedgesystems.com> wrote:
>
>> Oh, and it may have already written something on the datastore, so you’d
>> need
>> to check it and remove it with the fs command anyway.
>>
>> crail fs -ls -R /
>>
>> would show you what you have there.
>>
>> Regards,
>> David
>>
>> (C) 714-476-2692
>>
>> From: Lou DeGenaro<ma...@gmail.com>
>> Sent: Friday, February 7, 2020 12:15 PM
>> To: dev@crail.apache.org<ma...@crail.apache.org>
>> Subject: iobench
>>
>> Still a noob. Got namenode and datanode running as docker images on my
>> VM. I'm looking for the simplest example of writing something then
>> reading
>> something.
>>
>> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
>> /filename
>> -s 1024 -k 1
>> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
>> 20/02/07 14:10:23 INFO crail: crail.version 3101
>> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
>> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
>> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
>> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/07 14:10:23 INFO crail: crail.user crail
>> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
>> 20/02/07 14:10:23 INFO crail: crail.debug false
>> 20/02/07 14:10:23 INFO crail: crail.statistics true
>> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
>> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
>> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
>> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
>> 20/02/07 14:10:23 INFO crail: crail.singleton true
>> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
>> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
>> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/07 14:10:23 INFO crail: crail.locationmap
>> 20/02/07 14:10:23 INFO crail: crail.namenode.address
>> crail://abbey-road:9060
>> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
>> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/07 14:10:23 INFO crail: crail.namenode.log
>> 20/02/07 14:10:23 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
>> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
>> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
>> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, bufferCount
>> 1024
>> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 16,
>> messageSize 2097152, nodealy false
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
>> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true
>> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
>> 10.114.222.23:9060
>> write, filename /filename, size 1024, loop 1, storageClass 0,
>> locationClass
>> 0, buffered true
>> Exception in thread "main" java.io.IOException: Map failed
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>> at
>>
>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
>> at org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
>> at
>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
>> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
>> at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
>> Caused by: java.lang.OutOfMemoryError: Map failed
>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>> ... 5 more
>>
>>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
OK, I take it back. Having trouble with crail fs.
=====
[root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
...
20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/
10.114.222.23:9060
20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization done..
mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map
failed
20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem
20/02/10 10:07:16 INFO crail: Closing CrailFS singleton
20/02/10 10:07:16 INFO crail: mapped client cache closed
[root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar
20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs initialization
done..
rmdir: /foobar
20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem
20/02/10 10:08:55 INFO crail: Closing CrailFS singleton
20/02/10 10:08:55 INFO crail: mapped client cache closed
[root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar
20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
...
20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization done..
mkdir: `/foobar': File exists
20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem
20/02/10 10:09:43 INFO crail: Closing CrailFS singleton
20/02/10 10:09:43 INFO crail: mapped client cache closed
=====
Seems that mkdir works, but rmdir does not? Also, IOException was reported
on mkdir, which seems worrisome.
Lou.
On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lo...@gmail.com> wrote:
> David,
>
> Thanks. I'm able to use *crail fs *successfully to create/list/delete.
> Will try to increase available storage space next...
>
> Lou.
>
> On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
> david.crespi@storedgesystems.com> wrote:
>
>> Oh, and it may have already written something on the datastore, so you’d
>> need
>> to check it and remove it with the fs command anyway.
>>
>> crail fs -ls -R /
>>
>> would show you what you have there.
>>
>> Regards,
>> David
>>
>> (C) 714-476-2692
>>
>> From: Lou DeGenaro<ma...@gmail.com>
>> Sent: Friday, February 7, 2020 12:15 PM
>> To: dev@crail.apache.org<ma...@crail.apache.org>
>> Subject: iobench
>>
>> Still a noob. Got namenode and datanode running as docker images on my
>> VM. I'm looking for the simplest example of writing something then
>> reading
>> something.
>>
>> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
>> /filename
>> -s 1024 -k 1
>> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
>> 20/02/07 14:10:23 INFO crail: crail.version 3101
>> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
>> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
>> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
>> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
>> 20/02/07 14:10:23 INFO crail: crail.user crail
>> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
>> 20/02/07 14:10:23 INFO crail: crail.debug false
>> 20/02/07 14:10:23 INFO crail: crail.statistics true
>> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
>> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
>> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
>> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
>> 20/02/07 14:10:23 INFO crail: crail.singleton true
>> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
>> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
>> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
>> org.apache.crail.memory.MappedBufferCache
>> 20/02/07 14:10:23 INFO crail: crail.locationmap
>> 20/02/07 14:10:23 INFO crail: crail.namenode.address
>> crail://abbey-road:9060
>> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
>> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
>> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
>> org.apache.crail.namenode.rpc.tcp.TcpNameNode
>> 20/02/07 14:10:23 INFO crail: crail.namenode.log
>> 20/02/07 14:10:23 INFO crail: crail.storage.types
>> org.apache.crail.storage.tcp.TcpStorageTier
>> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
>> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
>> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
>> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, bufferCount
>> 1024
>> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 16,
>> messageSize 2097152, nodealy false
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize 1073741824
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
>> /dev/hugepages/data
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
>> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
>> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth
>> 32,
>> messageSize 512, nodealy true
>> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
>> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
>> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
>> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
>> 10.114.222.23:9060
>> write, filename /filename, size 1024, loop 1, storageClass 0,
>> locationClass
>> 0, buffered true
>> Exception in thread "main" java.io.IOException: Map failed
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
>> at
>>
>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
>> at org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
>> at
>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
>> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
>> at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
>> Caused by: java.lang.OutOfMemoryError: Map failed
>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
>> ... 5 more
>>
>>
Re: iobench
Posted by Lou DeGenaro <lo...@gmail.com>.
David,
Thanks. I'm able to use *crail fs *successfully to create/list/delete.
Will try to increase available storage space next...
Lou.
On Fri, Feb 7, 2020 at 6:37 PM David Crespi <
david.crespi@storedgesystems.com> wrote:
> Oh, and it may have already written something on the datastore, so you’d
> need
> to check it and remove it with the fs command anyway.
>
> crail fs -ls -R /
>
> would show you what you have there.
>
> Regards,
> David
>
> (C) 714-476-2692
>
> From: Lou DeGenaro<ma...@gmail.com>
> Sent: Friday, February 7, 2020 12:15 PM
> To: dev@crail.apache.org<ma...@crail.apache.org>
> Subject: iobench
>
> Still a noob. Got namenode and datanode running as docker images on my
> VM. I'm looking for the simplest example of writing something then reading
> something.
>
> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f
> /filename
> -s 1024 -k 1
> 20/02/07 14:10:22 INFO crail: creating singleton crail file system
> 20/02/07 14:10:23 INFO crail: crail.version 3101
> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16
> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
> 20/02/07 14:10:23 INFO crail: crail.user crail
> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
> 20/02/07 14:10:23 INFO crail: crail.debug false
> 20/02/07 14:10:23 INFO crail: crail.statistics true
> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288
> 20/02/07 14:10:23 INFO crail: crail.singleton true
> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
> 20/02/07 14:10:23 INFO crail: crail.cacheimpl
> org.apache.crail.memory.MappedBufferCache
> 20/02/07 14:10:23 INFO crail: crail.locationmap
> 20/02/07 14:10:23 INFO crail: crail.namenode.address
> crail://abbey-road:9060
> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
> org.apache.crail.namenode.rpc.tcp.TcpNameNode
> 20/02/07 14:10:23 INFO crail: crail.namenode.log
> 20/02/07 14:10:23 INFO crail: crail.storage.types
> org.apache.crail.storage.tcp.TcpStorageTier
> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1
> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, bufferCount
> 1024
> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
> messageSize 2097152, nodealy false
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize 1073741824
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath
> /dev/hugepages/data
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
> messageSize 512, nodealy true
> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
> 10.114.222.23:9060
> write, filename /filename, size 1024, loop 1, storageClass 0, locationClass
> 0, buffered true
> Exception in thread "main" java.io.IOException: Map failed
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
> at
>
> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
> at org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
> at
> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
> at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
> at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
> Caused by: java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
> ... 5 more
>
>
RE: iobench
Posted by David Crespi <da...@storedgesystems.com>.
Oh, and it may have already written something on the datastore, so you’d need
to check it and remove it with the fs command anyway.
crail fs -ls -R /
would show you what you have there.
Regards,
David
(C) 714-476-2692
From: Lou DeGenaro<ma...@gmail.com>
Sent: Friday, February 7, 2020 12:15 PM
To: dev@crail.apache.org<ma...@crail.apache.org>
Subject: iobench
Still a noob. Got namenode and datanode running as docker images on my
VM. I'm looking for the simplest example of writing something then reading
something.
[root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f /filename
-s 1024 -k 1
20/02/07 14:10:22 INFO crail: creating singleton crail file system
20/02/07 14:10:23 INFO crail: crail.version 3101
20/02/07 14:10:23 INFO crail: crail.directorydepth 16
20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10
20/02/07 14:10:23 INFO crail: crail.blocksize 1048576
20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824
20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache
20/02/07 14:10:23 INFO crail: crail.user crail
20/02/07 14:10:23 INFO crail: crail.shadowreplication 1
20/02/07 14:10:23 INFO crail: crail.debug false
20/02/07 14:10:23 INFO crail: crail.statistics true
20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000
20/02/07 14:10:23 INFO crail: crail.datatimeout 1000
20/02/07 14:10:23 INFO crail: crail.buffersize 1048576
20/02/07 14:10:23 INFO crail: crail.slicesize 524288
20/02/07 14:10:23 INFO crail: crail.singleton true
20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824
20/02/07 14:10:23 INFO crail: crail.directoryrecord 512
20/02/07 14:10:23 INFO crail: crail.directoryrandomize true
20/02/07 14:10:23 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
20/02/07 14:10:23 INFO crail: crail.locationmap
20/02/07 14:10:23 INFO crail: crail.namenode.address crail://abbey-road:9060
20/02/07 14:10:23 INFO crail: crail.namenode.blockselection roundrobin
20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16
20/02/07 14:10:23 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
20/02/07 14:10:23 INFO crail: crail.namenode.log
20/02/07 14:10:23 INFO crail: crail.storage.types
org.apache.crail.storage.tcp.TcpStorageTier
20/02/07 14:10:23 INFO crail: crail.storage.classes 1
20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0
20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2
20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, bufferCount
1024
20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 16,
messageSize 2097152, nodealy false
20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0
20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020
20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit 1073741824
20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize 1073741824
20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath /dev/hugepages/data
20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16
20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1
20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32,
messageSize 512, nodealy true
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512
20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1
20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/
10.114.222.23:9060
write, filename /filename, size 1024, loop 1, storageClass 0, locationClass
0, buffered true
Exception in thread "main" java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938)
at
org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94)
at org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95)
at
org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482)
at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87)
at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935)
... 5 more