You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/09/01 16:12:00 UTC
[jira] [Commented] (IMPALA-10087) IMPALA-6050 causes alluxio not
to be supported
[ https://issues.apache.org/jira/browse/IMPALA-10087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188603#comment-17188603 ]
ASF subversion and git services commented on IMPALA-10087:
----------------------------------------------------------
Commit 69d0d0af471f7013627ead3dff86a402ebc263a6 in impala's branch refs/heads/master from abeltian
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=69d0d0a ]
IMPALA-10087: IMPALA-6050 causes alluxio not to be supported
This change adds file type support for alluxio.
Alluxio URLs have a different prefix
such as：alluxio://zk@zk-1:2181,zk-2:2181,zk-3:2181/path/
Testing:
Add unit test for alluxio file system type checks.
Change-Id: Id92ec9cb0ee241a039fe4a96e1bc2ab3eaaf8f77
Reviewed-on: http://gerrit.cloudera.org:8080/16379
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> IMPALA-6050 causes alluxio not to be supported
> -----------------------------------------------
>
> Key: IMPALA-10087
> URL: https://issues.apache.org/jira/browse/IMPALA-10087
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.3.0, Impala 3.4.0
> Reporter: abeltian
> Assignee: abeltian
> Priority: Major
> Attachments: image-2020-08-18-20-09-31-714.png
>
>
> IMPALA-6050 causes alluxio not to be supported.
> {code:java}
> if (sampledFiles != null) {
> numPartitionsPerFs_ = sampledFiles.keySet().stream().collect(Collectors.groupingBy(
> SampledPartitionMetadata::getPartitionFsType, Collectors.counting()));
> } else {
> numPartitionsPerFs_.putAll(partitions_.stream().collect(
> Collectors.groupingBy(FeFsPartition::getFsType, Collectors.counting())));
> }{code}
> {code:java}
> public enum FsType {
> ADLS,
> HDFS,
> LOCAL,
> S3,
> OZONE;
> private static final Map<String, FsType> SCHEME_TO_FS_MAPPING =
> ImmutableMap.<String, FsType>builder()
> .put("abfs", ADLS)
> .put("abfss", ADLS)
> .put("adl", ADLS)
> .put("file", LOCAL)
> .put("hdfs", HDFS)
> .put("s3a", S3)
> .put("o3fs", OZONE)
> .build();{code}
> {code:java}
> 10:58:47.793 AM INFO cc:288 10:58:47.793 AM INFO cc:288 0543f5853d94c127:7a4e836900000000] java.lang.NullPointerException: element cannot be mapped to a null key
> at java.util.Objects.requireNonNull(Objects.java:228) at java.util.stream.Collectors.lambda$groupingBy$45(Collectors.java:907)
> at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at org.apache.impala.planner.HdfsScanNode.computeScanRangeLocations(HdfsScanNode.java:822)
> at org.apache.impala.planner.HdfsScanNode.init(HdfsScanNode.java:381)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org