[jira] [Commented] (HADOOP-16921) NPE in s3a byte buffer block upload

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-16921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057746#comment-17057746 ] 

Steve Loughran commented on HADOOP-16921:
-----------------------------------------

{code}
 java.io.IOException: regular upload failed: java.lang.NullPointerException at org.apache.hadoop.fs.s3a.S3AUtils.extractException(S3AUtils.java:338) at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.putObject(S3ABlockOutputStream.java:454) at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.close(S3ABlockOutputStream.java:366) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:70) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:129) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:415) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:387) at org.apache.hadoop.hive.common.FileUtils.copy(FileUtils.java:666) at org.apache.hadoop.hive.common.FileUtils.copy(FileUtils.java:633) at org.apache.hadoop.hive.ql.metadata.Hive.mvFile(Hive.java:4436) at org.apache.hadoop.hive.ql.metadata.Hive.access$100(Hive.java:221) at org.apache.hadoop.hive.ql.metadata.Hive$5.call(Hive.java:4296) ... 5 moreCaused by: java.lang.NullPointerException at org.apache.hadoop.fs.s3a.S3ADataBlocks$ByteBufferBlockFactory$ByteBufferBlock$ByteBufferInputStream.position(S3ADataBlocks.java:708) at org.apache.hadoop.fs.s3a.S3ADataBlocks$ByteBufferBlockFactory$ByteBufferBlock$ByteBufferInputStream.mark(S3ADataBlocks.java:721) at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:114) at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:114) at com.amazonaws.util.LengthCheckInputStream.mark(LengthCheckInputStream.java:116) at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:114) at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:114) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1067) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4920) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4866) at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:389) at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5800) at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1789) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1749) at org.apache.hadoop.fs.s3a.S3AFileSystem.putObjectDirect(S3AFileSystem.java:2136) at org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$putObject$5(WriteOperationHelper.java:462) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:110) at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:315) at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:407) at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:311) at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:286) at org.apache.hadoop.fs.s3a.WriteOperationHelper.retry(WriteOperationHelper.java:150) at org.apache.hadoop.fs.s3a.WriteOperationHelper.putObject(WriteOperationHelper.java:460) at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.lambda$putObject$0(S3ABlockOutputStream.java:439) at org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219) at org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) ... 3 more
{code}

> NPE in s3a byte buffer block upload
> -----------------------------------
>
>                 Key: HADOOP-16921
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16921
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.3.0
>            Reporter: Steve Loughran
>            Priority: Minor
>
> NPE in s3a upload when fs.s3a.fast.upload.buffer = bytebuffer



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org