[GitHub] milleruntime opened a new issue #497: LogSorter InputStream closes prematurely

[GitBox <gi...@apache.org>]

milleruntime opened a new issue #497: LogSorter InputStream closes prematurely
URL: https://github.com/apache/accumulo/issues/497
 
 
   Some time ago, LogSorter was modified to handle partial headers and the FSDataInputStream in sort() was moved to a try-with-resources.  It appears this change caused the FSDataInputStream to close prematurely, before the finally can call the close() method.  The close method gets the bytesCopied before closing the input so this probably results in bytesCopied in the Monitor to always be -1.  I think this happens every time a WAL is sorted on recovery, showing this error:
   <pre>
   2018-05-23 14:19:55,298 [log.LogSorter] ERROR: Error during cleanup sort/copy 2841a75d-8086-4fdb-a736-5f0bf60ff42e
   java.io.IOException: Stream is closed!
           at org.apache.hadoop.fs.BufferedFSInputStream.getPos(BufferedFSInputStream.java:56)
           at org.apache.hadoop.fs.FSDataInputStream.getPos(FSDataInputStream.java:72)
           at org.apache.accumulo.tserver.log.LogSorter$LogProcessor.close(LogSorter.java:198)
           at org.apache.accumulo.tserver.log.LogSorter$LogProcessor.sort(LogSorter.java:169)
           at org.apache.accumulo.tserver.log.LogSorter$LogProcessor.process(LogSorter.java:96)
           at org.apache.accumulo.server.zookeeper.DistributedWorkQueue$1.run(DistributedWorkQueue.java:109)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
           at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
           at java.lang.Thread.run(Thread.java:748)
   </pre>
   
   We could move the FSDataInputStream back to a regular try/catch or refactor the sort method to close things properly.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services