You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ku...@apache.org on 2020/06/01 08:52:06 UTC
[carbondata] branch branch-2.0 created (now 8947655)
This is an automated email from the ASF dual-hosted git repository.
kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.
at 8947655 [CARBONDATA-3840] Mark features as experimental
This branch includes the following new commits:
new 316ea18 [CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object
new 8947655 [CARBONDATA-3840] Mark features as experimental
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[carbondata] 01/02: [CARBONDATA-3839]Fix rename file failed for
FilterFileSystem DFS object
Posted by ku...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git
commit 316ea18a8e3825090f1e9083b67dee278bb9a98c
Author: akashrn5 <ak...@gmail.com>
AuthorDate: Mon Jun 1 09:58:31 2020 +0530
[CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object
Why is this PR needed?
Rename file fails in HDFS when the FS object is of FilterFileSystem,
(which basically can contain any filesystem to use as basic filesystem)
What changes were proposed in this PR?
While rename force, have a check for this FS object
This closes #3781
---
.../apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
index ac508c3..7d36f0e 100644
--- a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
+++ b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
@@ -23,6 +23,7 @@ import org.apache.carbondata.common.logging.LogServiceFactory;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FilterFileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hdfs.DistributedFileSystem;
import org.apache.log4j.Logger;
@@ -77,6 +78,11 @@ public class HDFSCarbonFile extends AbstractDFSCarbonFile {
((DistributedFileSystem) fileSystem).rename(path, new Path(changetoName),
org.apache.hadoop.fs.Options.Rename.OVERWRITE);
return true;
+ } else if ((fileSystem instanceof FilterFileSystem) && (((FilterFileSystem) fileSystem)
+ .getRawFileSystem() instanceof DistributedFileSystem)) {
+ ((DistributedFileSystem) ((FilterFileSystem) fileSystem).getRawFileSystem())
+ .rename(path, new Path(changetoName), org.apache.hadoop.fs.Options.Rename.OVERWRITE);
+ return true;
} else {
return fileSystem.rename(path, new Path(changetoName));
}
[carbondata] 02/02: [CARBONDATA-3840] Mark features as experimental
Posted by ku...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git
commit 8947655194db7c76987e97ebb5d01bbadb587b8a
Author: kunal642 <ku...@gmail.com>
AuthorDate: Mon Jun 1 13:07:24 2020 +0530
[CARBONDATA-3840] Mark features as experimental
Why is this PR needed?
Mark features as experimental because they are subject to change in future.
What changes were proposed in this PR?
Mark features as experimental because they are subject to change in future.
Does this PR introduce any user interface change?
No
Is any new testcase added?
No
This closes #3783
---
README.md | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/README.md b/README.md
index 2d9fcc7..0293c4c 100644
--- a/README.md
+++ b/README.md
@@ -69,6 +69,14 @@ CarbonData is built using Apache Maven, to [build CarbonData](https://github.com
* [Carbon as Spark's Datasource](https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md)
* [FAQs](https://github.com/apache/carbondata/blob/master/docs/faq.md)
+## Experimental Features
+
+Some features are marked as experimental because the syntax/implementation might change in the future.
+1. Hybrid format table using Add Segment.
+2. Accelerating performance using MV on parquet/orc.
+3. Merge API for Spark DataFrame.
+4. Hive write for non-transactional table.
+
## Integration
* [Hive](https://github.com/apache/carbondata/blob/master/docs/hive-guide.md)
* [Presto](https://github.com/apache/carbondata/blob/master/docs/prestodb-guide.md)