simple experiment yields namenode exception

[Lou DeGenaro <lo...@gmail.com>]

beware: I'm a noob.

The objective is to put a 1GB file into crail DRAM, then measure how fast
it can be retrieved into a client's memory.

I have an 8GB VM running CentOS.  I have 2 docker containers, one each for
namenode and datanode.  Configuration is for 4GB of storage for crail.

# docker run -it --network host -e NAMENODE_HOST=sgt-pepper -e
> DATAPATH=/tmp/hugepages/data -e CACHEPATH=/tmp/hugepages/cache -e
> STORAGELIMIT=4294967296 apache/incubator-crail:1.2 namenode
>

# docker run -it --network host -e NAMENODE_HOST=sgt-pepper -e
> DATAPATH=/tmp/hugepages/data -e CACHEPATH=/tmp/hugepages/cache -e
> STORAGELIMIT=4294967296 apache/incubator-cra:1.2 datanode
>

To put 1GB file into crail, I use:

#  /usr/local/apache-crail-1.2-incubating/bin/crail fs -copyFromLocal
> /root/G1.txt /
>

Seems fine:


> /usr/local/apache-crail-1.2-incubating/bin/crail fs -ls /
>

Found 1 items
> -rw-rw-rw-   1 crail crail 1073741824 2020-02-18 09:02 /G1.txt
>

# java -version
> openjdk version "1.8.0_242"
> OpenJDK Runtime Environment (build 1.8.0_242-b08)
> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)


# env | grep CLASSPATH
> CLASSPATH=/usr/local/apache-crail-1.2-incubating/jars/*
>

# env | grep CRAIL_HOME
> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating
>

To fetch from crail into memory, I use:

# java org.apache.crail.tools.CrailBenchmark -f G1.txt -t readSequential
> experiment 0
> log4j:WARN No appenders could be found for logger (org.apache.crail).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> readSequential, filename G1.txt, size 1048576, loop 1, buffered true
> warmUp, warmupFile G1.txt-709380522, operations 32
> starting benchmark...
> execution time 0.01
> ops 1.0
> sumbytes 1048576.0
> throughput 838.8608
> latency 10000.0
>

Not 1GB, but seems OK so far.  But on namenode console, I see:

20/02/18 15:06:14 INFO crail: new connection from /10.114.222.22:55364
> 20/02/18 15:06:14 INFO narpc: adding new channel to selector, from /
> 10.114.222.22:55364
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
> at com.ibm.narpc.NaRPCProtocol.fetchBuffer(NaRPCProtocol.java:29)
> at
> com.ibm.narpc.NaRPCServerChannel.receiveMessage(NaRPCServerChannel.java:38)
> at com.ibm.narpc.NaRPCDispatcher.run(NaRPCDispatcher.java:84)
> at java.lang.Thread.run(Thread.java:748)
> 20/02/18 15:06:15 INFO narpc: closing the select call
>

And more, if I try readSequentioal again, the requester is frozen and on
namenode console all I see is:

20/02/18 15:21:33 INFO crail: new connection from /10.114.222.22:55496
>

Thanks for any insights.

BTW, the CrailBenchmark code has sparse comments.  Would be nice if there
was more explanation.  Or is there some better place to look for code to
emulate?

 Lou.