[jira] [Commented] (HADOOP-15796) ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs caching is disabled

["Gabor Bota (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634181#comment-16634181 ] 

Gabor Bota commented on HADOOP-15796:
-------------------------------------

Thanks for looking in this [~stevel@apache.org]!
I understand that those test should not run if fs caching is off, but throwing an exception like this is not the nicest nor the most informative way to handle this. 

You think that forcing fs caching on for those tests will not make the {{java.io.IOException: s3a://TEST-BUCKET: FileSystem is closed!}} failure happen for other tests?

> ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs caching is disabled
> -------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-15796
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15796
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.1.1
>            Reporter: Gabor Bota
>            Priority: Major
>
> Sometimes we get {{java.io.IOException: s3a://TEST-BUCKET: FileSystem is closed!}} when running tests, because of filesystem caching: an fs instance is closed from a test teardown while another test is still using it. (That should be fixed in another issue.)
> To avoid test errors introduced by closed fs I've disabled fs caching while testing. This caused the following issue in lots of test:
> {noformat}
> [ERROR] testPartitionsResolution(org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing)  Time elapsed: 1.994 s  <<< ERROR!
> java.lang.ClassCastException: org.apache.hadoop.fs.s3a.S3AFileSystem cannot be cast to org.apache.hadoop.fs.s3a.MockS3AFileSystem
> 	at org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase.lookupWrapperFS(StagingTestBase.java:145)
> 	at org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase$JobCommitterTest.setupJob(StagingTestBase.java:294)
> {noformat}
> We should fix the casting during these tests.
> To switch of fs caching during a test (mvn verify) run, add the following to the config:
> {code:java}
>   <property>
>     <name>fs.s3a.impl.disable.cache</name>
>     <value>true</value>
>   </property>
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org