You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ra...@apache.org on 2019/04/08 07:47:17 UTC
[carbondata] branch online_segment updated (53dbb45 -> 7f05e69)
This is an automated email from the ASF dual-hosted git repository.
ravipesala pushed a change to branch online_segment
in repository https://gitbox.apache.org/repos/asf/carbondata.git.
from 53dbb45 [CARBONDATA-3161]Pipe dilimiter is not working for streaming table
add 34923db [CARBONDATA-3149]Support alter table column rename
add 96b2ea3 [CARBONDATA-3179] Map Data Load Failure and Struct Projection Pushdown Issue
add bd752e9 [CARBONDATA-3127]Fix the HiveExample & TestCarbonSerde exception
add 10bc5c2 [CARBONDATA-3186]Avoid creating empty carbondata file when all the records are bad record with action redirect.
add f4c1c67 [CARBONDATA-3192] Fix for compaction compatibilty issue
add 0855361 [CARBONDATA-3184]Fix DataLoad Failure with 'using carbondata'
add 128a6c8 [CARBONDATA-3080] Supporting local dictionary enable by default for SDK
add 04b5256 [CARBONDATA-3142]Add timestamp with thread name which created by CarbonThreadFactory
add ca32374 [CARBONDATA-3176] Optimize quick-start-guide documentation
add 7fec2c5 [CARBONDATA-3188] Create carbon table as hive understandable metastore table needed by Presto and Hive
add 56aecb6 [CARBONDATA-3165]Protection of Bloom Null Exception
add d5a2c69 [HOTFIX] rename field thread_pool_size to match camel case
add e8cf14a [CARBONDATA-2218] AlluxioCarbonFile while trying to force rename causes a FileSytem error and is not a DistributionFileSystem.(Adding Alluxio Support)
add d85d543 [Carbondata-3173] Add the hive/presto documents index to the root of file ReadMe.md
add f5c1b7b [CARBONDATA-3195]Added validation for Inverted Index columns and added a test case in case of varchar
add 7c4e79f [CARBONDATA-3196] [CARBONDATA-3203]Fixed Compaction for Complex types with Dictionary Include and also supported Compaction for restructured table
add e193df0 [CARBONDATA-3194] Integrating Carbon with Presto using hive connector
add 3e4638b [CARBONDATA-3126] Correct some spell errors in CarbonData
add b0733ec [CARBONDATA-3202]update the schema to session catalog after add column, drop column and column rename
add 21330b8 [CARBONDATA-3208] Remove unused parameters, imports and optimize the spell errors
add 9c9cd7f [CARBONDATA-3209] Remove unused import
add ed9f445 [CARBONDATA-3213] Add license for java and md files
add 28432e3 [CARBONDATA-3206] Fix some spell errors in CarbonData
add 7477527 [CARBONDATA-3205]Fix Get Local Dictionary for empty Array of String
add f8697b1 [CARBONDATA-3218] Fix schema refresh and wrong query result issues in presto.
add deb08c3 [CARBONDATA-3212] Fixed NegativeArraySizeException while querying in specific scenario
add 81bf02f [HOTFIX] Optimize the code style in csdk/sdk markdown doc
add 9fa045d [CARBONDATA-3216] Fix enableLocalDictionary with false issue in CSDK
add bc1e944 [CARBONDATA-3217] Optimize implicit filter expression performance by removing extra serialization
add fc4d511 [CARBONDATA-3149] Documentation for alter table column rename
add 923dab1 [CARBONDATA-3226] Remove duplicated and useless files
add 72da334 [CARBONDATA-3223] Fixed Wrong Datasize and Indexsize calculation for old store using Show Segments
add aad9aab [CARBONDATA-3189] Fix PreAggregate Datamap Issue
add 5a03626 [CARBONDATA-3222]Fix dataload failure after creation of preaggregate datamap on main table with long_string_columns
add 03ec2e0 [CARBONDATA-3224] Support SDK validate the improper value when using withLoadOptions
add 0276250 [CARBONDATA-3227] Fix some spell errors in the project
add 8002c59 [CARBONDATA-3211] Optimize the documentation
add 45951c7 [CARBONDATA-3219] Support range partition the input data for local_sort/global sort data loading
add c0ba982 [CARBONDATA-3221] Fix the error of SDK don't support read multiple file from S3
add 4e27b86 [CARBONDATA-3230] Add alter test case for datasource
add 77d2b4e [CARBONDATA-3201] Added load level SORT_SCOPE Added SORT_SCOPE in Load Options & in SET Command
add 3a41ee5 [CARBONDATA-3210] Merge common method into CarbonSparkUtil and fix example error
add dd2fff2 [CARBONDATA-3235] Fix Rename-Fail & Datamap-creation-Fail
add 3a5572e [CARBONDATA-3236] Fix for JVM Crash for insert into new table from old table
add 1b45c41 [CARBONDATA-3200] No-Sort compaction
add 8e6def9 [CARBONDATA-3237] Fix presto carbon issues in dictionary include scenario
add d78db8f [CARBONDATA-3220] Support presto to read stream segment data
add 0221526 [CARBONDATA-3226] Remove duplicated and useless files
add 86713f5 [CARBONDATA-3241] Refactor the requested scan columns and the projection columns
add c2003b0 [HOTFIX] Fix compile error after merging PR#3001
add 9608413 [HOTFIX] Exclude filter doesn't work in presto carbon in cluster
add 1a2c051 [CARBONDATA-3239] Fix ArrayIndexOutOfBoundsException in DataSkewRangePartitioner
add 92c9ce3 [CARBONDATA-3233]Fix JVM crash issue in snappy compressor and update the pagesize correctly
add 39cd2f4 [HOTFIX][DataLoad]fix task assignment issue using NODE_MIN_SIZE_FIRST block assignment strategy
add b828d0d [CARBONDATA-3215] Optimize the documentation
add 5a7dc79 [CARBONDATA-3251] Fix spark-2.1 UT errors
add 9804873 [CARBONDATA-3238] Solve StackOverflowError using MV datamap
add d51cbc5 [HOTFIX] Fixed NPE during query with Local Dictionary
add af297a9 [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False
add 733ce21 [CARBONDATA-3253] Remove test case of bloom datamap using search mode
add 6605b54 Optimize presto-guide
add 9f13d2b [CARBONDATA-3242] Move Range_Column into the table level properties
add e485e14 [SDV] Fix SDV testcase failures
add 556e220 [CARBONDATA-3261] support float and byte data type reading from presto carbon
add cd51220 [CARBONDATA-3260] Fix the Hive stats issue in carbon catalog table
add a001d9d [CARBONDATA-3259] Added documentation for new complex delimiters
add aadfbeb [CARBONDATA-3243] Updated DOC for No-Sort Compaction and a few Fixes
add df3d4c8 [DOC] Update range_info partition example in ddl description
add 0f1d98f [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
add b8def42 [DOC] Updated Presto guide
add 134df2b [CARBONDATA-3263] Update doc for RANGE_COLUMN
add 7916aa6 [CARBONDATA-3257] Fix for NO_SORT load and describe formatted being in NO_SORT flow even with Sort Columns given
add 64e8df3 [CARBONDATA-3262] Fix merge index failure handling for compacted segment
add f4e4629 [CARBONDATA-3267]Fixed Range Sort OOM Issue
add 2728332 [CARBONDATA-3265] Fixed memory leak in Range Sort
add d63b2c0 [HOTFIX] presto carbon doesn't work with Hadoop conf in cluster.
add 06977de [CARBONDATA-3252] Remove unused import
add 028eb25 [CARBONDATA-3232] Add example and doc for alluxio integration
add 8e39ee1 [CARBONDATA-3264] Added SORT_SCOPE in ALTER TABLE SET
add aa07020 [CARBONDATA-3275][TEST] Fix errors in tests
add 75d9eae [CARBONDATA-3235] Fixed Alter Table Rename
add e34ad2f [CARBONDATA-3268] Fix for query on Varchar Columns showing Null in Presto
add d149732 [CARBONDATA-3269] Fix ArrayIndexOutOfBoundsException of Range_Column when using KryoSerializer
add c55f1d7 [HOTFIX] Fix select query on varchar column with large data fails with jvm crash
add faa85fb [CARBONDATA-3272]fix ArrayIndexOutOfBoundsException of horizontal compaction during update, when cardinality changes within a segment
add b82cd1b [CARBONDATA-3273] [CARBONDATA-3274] Fix for SORT_SCOPE in CarbonLoadDataCommand
add 3f63f91 [HOTFIX] Upgraded jars to work S3 with presto
add 74ce5fa [CARBONDATA-3282] set hadoop conf to thread local for file factory usage in presto carbon
add 8ff04e6 [HOTFIX] SDV framework for presto cluster test suite
add ba139b6 [CARBONDATA-3280] Fix the issue of SDK assert can't work
add 2ecf30c [DOC] Document Update for default sort scope
add 6091eb7 [CARBONDATA-3287]Remove the validation for same schema in a location and fix drop datamap issue
add e7b5745 [CARBONDATA-3284] [CARBONDATA-3285] Workaround for Create-PreAgg Datamap Fail & Sort-Columns Fix
add d84f721 [CARBONDATA-3278] Remove duplicate code to get filter string of date/timestamp
add 9672a10 [CARBONDATA-3107] Optimize error/exception coding for better debugging
add 1470f78 [CARBONDATA-2447] Block update operation on range/list/hash partition table
add c56f810 [CARBONDATA-3299] Desc Formatted Issue Fixed
add 99dfcbe [CARBONDATA-3276] Compacting table that do not exist should modify the message of MalformedCarbonCommandException
add 9051ab7 [CARBONDATA-3298]Removed Log Message for Already Deleted Segments
add 718be37 [CARBONDATA-3305] Support show metacache command to list the cache sizes for all tables
add 3f98c51 [CARBONDATA-3305] Added DDL to drop cache for a table
add e443a94 [CARBONDATA-3281] Add validation for the size of the LRU cache
add a5fc19d [CARBONDATA-3301]Fix inserting null values to Array<date> columns in carbon file format data load
add 182eeb0 [LOG] Optimize the logs of CarbonProperties
add f5e4793 [CARBONDATA-3307] Fix Performance Issue in No Sort
add 6840a18 [CARBONDATA-3297] Fix that the IndexoutOfBoundsException when creating table and dropping table are at the same time
add dda9c4d [CARBONDATA-3300] Fixed ClassNotFoundException when using UDF in spark-shell
add 1825861 [DOC] Update the doc of "Show DataMap"
add c55a5d0 [CARBONDATA-3304] Distinguish the thread names created by thread pool of CarbonThreadFactory
add 89c3873 [CARBONDATA-3313] count(*) is not invalidating the invalid segments cache
add 8328cc7 [CARBONDATA-3315] Fix for Range Filter failing with two between clauses as children of OR expression
add 5fbc644 [CARBONDATA-3314] Fix for Index Cache Size in SHOW METACACHE DDL
add 1c8e9b6 [CARBONDATA-3317] Fix NPE when execute 'show segments' command for stream table
add cdcea33 [CARBONDATA-3311] support presto 0.217 #3142
add dda3e7d [TestCase][HOTFIX] Added drop database in beforeEach to avoid exception
add 66982f3 [CARBONDATA-3302] [Spark-Integration] code cleaning related to CarbonCreateTable command
add 2980803 [CARBONDATA-3318] Added PreAgg & Bloom Event-Listener for ShowCacheCommmand
add 6fd81af [CARBONDATA-3293] Prune datamaps improvement for count(*)
add e61b97e [CARBONDATA-3321] Improved Single/Concurrent query Performance
add 9854f20 [DOC] Fix the spell mistake of enable.unsafe.in.query.processing
add 5c52876 [CARBONDATA-3322] [CARBONDATA-3323] Added check for invalid tables in ShowCacheCommand & Standard output on ShowCacheCommand on table
add e5a97d0 [CARBONDATA-3320]fix number of partitions issue in describe formatted and drop partition issue
add dc3d5ab [CARBONDATA-3329] Fixed deadlock issue during failed query
add 91c6758 [CARBONDATA-3328]Fixed performance issue with merge small files distribution
add 726bb7e [HOTFIX][DOC] Optimize quick-start-guide.md and dml-of-carbondata.md
add 176d6a7 [CARBONDATA-3319][TestCase]Added condition to check if datamap exist or not before caching
add 35e63b1 [CARBONDATA-3330] Fix Invalid Exception while clearing datamap from SDK carbon reader
add 4a7e3bd [CARBONDATA-3333]Fixed No Sort Store Size issue and Compatibility issue after alter added column done in 1.1 and load in 1.5
add 46006ab [Document] update doc about presto version support details #3163
add 36e6840 [CARBONDATA-3332] Blocked concurrent compaction and update/delete
add 38c81e2 [CARBONDATA-3335]Fixed load and compaction failure after alter done in older version
add 0b93ba3 [HOTFIX]Fixed data map loading issue when number of segments are high
add 7f05e69 [HOTFIX]fixed loading issue for legacy store
No new revisions were added by this update.
Summary of changes:
README.md | 5 +
common/pom.xml | 8 +
.../java/org/apache/carbondata/common/Strings.java | 6 +-
.../org/apache/carbondata/common/StringsSuite.java | 2 +-
.../carbondata/core/cache/CacheProvider.java | 4 +
.../carbondata/core/cache/CarbonLRUCache.java | 59 +-
.../cache/dictionary/AbstractDictionaryCache.java | 4 +-
.../cache/dictionary/ForwardDictionaryCache.java | 4 +-
.../cache/dictionary/ReverseDictionaryCache.java | 4 +-
.../core/constants/CarbonCommonConstants.java | 47 +-
.../constants/CarbonCommonConstantsInternal.java | 2 +
.../core/constants/CarbonLoadOptionConstants.java | 7 +
.../core/constants/SortScopeOptions.java | 4 +-
.../core/datamap/DataMapStoreManager.java | 39 +-
.../carbondata/core/datamap/DataMapUtil.java | 2 +-
.../core/datamap/DistributableDataMapFormat.java | 1 +
.../apache/carbondata/core/datamap/Segment.java | 24 +-
.../carbondata/core/datamap/TableDataMap.java | 118 ++-
.../carbondata/core/datamap/dev/DataMap.java | 19 +-
.../datamap/dev/cgdatamap/CoarseGrainDataMap.java | 18 +-
.../datamap/dev/fgdatamap/FineGrainDataMap.java | 17 +-
.../core/datastore/ReusableDataBuffer.java | 9 +-
.../carbondata/core/datastore/TableSpec.java | 86 +-
.../core/datastore/block/SegmentProperties.java | 20 +-
.../block/SegmentPropertiesAndSchemaHolder.java | 62 +-
.../core/datastore/block/TableBlockInfo.java | 11 +
.../chunk/impl/DimensionRawColumnChunk.java | 6 +-
.../UnsafeAbstractDimensionDataChunkStore.java | 10 +-
.../filesystem/AbstractDFSCarbonFile.java | 14 +-
.../datastore/filesystem/AlluxioCarbonFile.java | 89 +-
.../core/datastore/filesystem/CarbonFile.java | 2 +-
.../core/datastore/filesystem/HDFSCarbonFile.java | 2 +-
.../core/datastore/filesystem/LocalCarbonFile.java | 4 +-
.../core/datastore/filesystem/S3CarbonFile.java | 2 +-
.../datastore/filesystem/ViewFSCarbonFile.java | 2 +-
.../core/datastore/impl/FileFactory.java | 81 +-
.../carbondata/core/datastore/page/ColumnPage.java | 6 +-
.../core/datastore/page/LocalDictColumnPage.java | 3 +
.../datastore/page/UnsafeFixLengthColumnPage.java | 29 +-
.../client/NonSecureDictionaryClient.java | 2 +-
.../client/NonSecureDictionaryClientHandler.java | 4 +-
.../generator/TableDictionaryGenerator.java | 2 +-
.../server/NonSecureDictionaryServerHandler.java | 2 +-
.../service/AbstractDictionaryServer.java | 8 +-
.../carbondata/core/features/TableOperation.java | 1 +
.../fileoperations/AtomicFileOperationFactory.java | 2 +-
.../core/indexstore/BlockletDataMapIndexStore.java | 4 +-
.../core/indexstore/BlockletDetailInfo.java | 2 +-
.../core/indexstore/ExtendedBlocklet.java | 97 +-
.../core/indexstore/SegmentPropertiesFetcher.java | 3 +
.../TableBlockIndexUniqueIdentifier.java | 5 +-
.../core/indexstore/UnsafeMemoryDMStore.java | 161 +++-
.../indexstore/blockletindex/BlockDataMap.java | 160 ++--
.../indexstore/blockletindex/BlockletDataMap.java | 29 +-
.../blockletindex/BlockletDataMapFactory.java | 9 +-
.../blockletindex/BlockletDataMapRowIndexes.java | 14 +-
.../carbondata/core/indexstore/row/DataMapRow.java | 12 +-
.../core/indexstore/row/UnsafeDataMapRow.java | 217 +----
.../core/indexstore/schema/CarbonRowSchema.java | 8 +
.../core/indexstore/schema/SchemaGenerator.java | 72 ++
.../timestamp/DateDirectDictionaryGenerator.java | 4 +-
.../{S3FileLock.java => AlluxioFileLock.java} | 53 +-
.../carbondata/core/locks/CarbonLockFactory.java | 21 +-
.../apache/carbondata/core/locks/LockUsage.java | 1 +
.../carbondata/core/locks/ZookeeperInit.java | 2 +-
.../carbondata/core/memory/IntPointerBuffer.java | 10 +-
.../core/memory/UnsafeMemoryManager.java | 3 +-
.../core/memory/UnsafeSortMemoryManager.java | 77 +-
.../carbondata/core/metadata/SegmentFileStore.java | 21 +-
.../core/metadata/blocklet/BlockletInfo.java | 23 +-
.../core/metadata/blocklet/DataFileFooter.java | 13 +
.../ThriftWrapperSchemaConverterImpl.java | 1 +
.../core/metadata/schema/table/CarbonTable.java | 61 +-
.../carbondata/core/mutate/CarbonUpdateUtil.java | 14 +-
.../core/reader/CarbonDeleteFilesDataReader.java | 14 +-
.../impl/DictionaryBasedResultCollector.java | 8 +-
.../impl/DictionaryBasedVectorResultCollector.java | 11 +-
.../core/scan/complextypes/ArrayQueryType.java | 14 +-
.../core/scan/complextypes/ComplexQueryType.java | 6 +-
.../core/scan/complextypes/MapQueryType.java | 4 +-
.../core/scan/complextypes/PrimitiveQueryType.java | 16 +-
.../core/scan/complextypes/StructQueryType.java | 18 +-
.../scan/executor/impl/AbstractQueryExecutor.java | 26 +-
.../scan/expression/RangeExpressionEvaluator.java | 30 +-
.../expression/conditional/ImplicitExpression.java | 109 +++
.../core/scan/filter/ColumnFilterInfo.java | 43 +-
.../carbondata/core/scan/filter/FilterUtil.java | 89 +-
.../core/scan/filter/GenericQueryType.java | 4 +-
.../ImplicitIncludeFilterExecutorImpl.java | 23 +-
.../core/scan/filter/intf/ExpressionType.java | 3 +-
.../visitor/ImplicitColumnVisitor.java | 24 +-
.../carbondata/core/scan/model/QueryModel.java | 64 +-
.../core/scan/partition/PartitionUtil.java | 2 +-
.../core/scan/result/BlockletScannedResult.java | 6 +-
.../AbstractDetailQueryResultIterator.java | 2 +-
.../core/scan/wrappers/ByteArrayWrapper.java | 4 +
.../core/scan/wrappers/IntArrayWrapper.java | 31 +-
.../core/statusmanager/LoadMetadataDetails.java | 8 +-
.../core/statusmanager/SegmentStatusManager.java | 8 +-
.../statusmanager/SegmentUpdateStatusManager.java | 3 +-
.../core/util/AbstractDataFileFooterConverter.java | 7 +
.../carbondata/core/util/BlockletDataMapUtil.java | 33 +-
.../carbondata/core/util/CarbonProperties.java | 205 ++--
.../carbondata/core/util/CarbonThreadFactory.java | 13 +-
.../apache/carbondata/core/util/CarbonUtil.java | 48 +-
.../core/util/DataFileFooterConverterV3.java | 5 +
.../apache/carbondata/core/util/DataTypeUtil.java | 8 +-
.../carbondata/core/util/DeleteLoadFolders.java | 4 -
.../core/util/ObjectSerializationUtil.java | 7 +-
.../apache/carbondata/core/util/SessionParams.java | 8 +-
.../carbondata/core/util/ThreadLocalTaskInfo.java | 7 +
.../carbondata/core/util/path/CarbonTablePath.java | 27 +-
.../carbondata/core/util/path/HDFSLeaseUtils.java | 2 +-
.../core/writer/CarbonIndexFileMergeWriter.java | 47 +-
.../apache/carbondata/hadoop/CarbonInputSplit.java | 387 ++++++--
.../hadoop/internal/ObjectArrayWritable.java | 0
.../carbondata/hadoop/internal/index/Block.java | 0
.../core/CarbonPropertiesValidationTest.java | 2 +-
.../carbondata/core/cache/CarbonLRUCacheTest.java | 7 +
.../filesystem/AlluxioCarbonFileTest.java | 214 ++++-
.../datastore/filesystem/HDFSCarbonFileTest.java | 2 +-
.../blockletindex/TestBlockletDataMap.java | 17 +
.../core/load/LoadMetadataDetailsUnitTest.java | 2 +-
.../core/scan/complextypes/ArrayQueryTypeTest.java | 2 +-
.../scan/complextypes/StructQueryTypeTest.java | 2 +-
.../scan/filter/FilterExpressionProcessorTest.java | 17 +
.../carbondata/core/util/DataTypeUtilTest.java | 1 -
.../datamap/bloom/BloomCacheKeyValue.java | 2 +-
.../datamap/bloom/BloomCoarseGrainDataMap.java | 51 +-
.../bloom/BloomCoarseGrainDataMapFactory.java | 18 +-
.../datamap/lucene/LuceneDataMapWriter.java | 2 +-
.../datamap/lucene/LuceneFineGrainDataMap.java | 6 +-
.../lucene/LuceneFineGrainDataMapFactory.java | 6 +-
.../apache/carbondata/mv/datamap/MVHelper.scala | 12 +-
.../carbondata/mv/rewrite/DefaultMatchMaker.scala | 20 +-
.../carbondata/mv/rewrite/MVCreateTestCase.scala | 25 +
.../mv/plans/util/BirdcageOptimizer.scala | 4 +-
docs/alluxio-guide.md | 136 +++
docs/configuration-parameters.md | 12 +-
docs/csdk-guide.md | 713 +++++++-------
docs/datamap-developer-guide.md | 25 +-
docs/datamap/bloomfilter-datamap-guide.md | 40 +-
docs/datamap/datamap-management.md | 15 +-
docs/datamap/timeseries-datamap-guide.md | 43 +-
docs/ddl-of-carbondata.md | 443 +++++----
docs/dml-of-carbondata.md | 78 +-
docs/documentation.md | 6 +-
docs/faq.md | 51 +-
docs/hive-guide.md | 8 +-
docs/introduction.md | 21 +-
docs/presto-guide.md | 298 ++++++
docs/quick-start-guide.md | 243 ++---
docs/s3-guide.md | 9 +-
docs/sdk-guide.md | 1008 ++++++++++----------
docs/streaming-guide.md | 14 +-
docs/usecases.md | 21 +-
examples/flink/pom.xml | 10 +-
examples/spark2/pom.xml | 24 +-
.../examples/sdk/CarbonReaderExample.java | 237 ++---
.../carbondata/examples/sdk/SDKS3Example.java | 55 +-
.../spark2/src/main/resources/sample.csv | 0
.../carbondata/examples/AlluxioExample.scala | 115 ++-
.../apache/carbondata/examples}/HiveExample.scala | 100 +-
.../org/apache/carbondata/examples/S3Example.scala | 34 +-
.../carbondata/examples/S3UsingSDkExample.scala | 51 +-
.../carbondata/examples/SparkSessionExample.scala | 6 +-
.../carbondata/examples/util/ExampleUtils.scala | 22 +-
.../apache/carbondata/examplesCI/RunExamples.scala | 8 +-
format/pom.xml | 8 +
format/src/main/thrift/carbondata_index.thrift | 1 +
.../carbondata/hadoop/CarbonMultiBlockSplit.java | 23 +-
.../carbondata/hadoop/CarbonRecordReader.java | 4 +-
.../hadoop/api/CarbonFileInputFormat.java | 8 +-
.../carbondata/hadoop/api/CarbonInputFormat.java | 114 ++-
.../hadoop/api/CarbonTableInputFormat.java | 117 +--
.../hadoop/api/CarbonTableOutputFormat.java | 3 +-
.../hadoop/stream}/CarbonStreamInputFormat.java | 5 +-
.../hadoop/stream}/CarbonStreamUtils.java | 2 +-
.../hadoop/stream}/StreamBlockletReader.java | 2 +-
.../hadoop/stream/StreamRecordReader.java | 212 +---
.../hadoop/util/CarbonVectorizedRecordReader.java | 65 +-
.../hive/CarbonDictionaryDecodeReadSupport.java | 2 +-
.../carbondata/hive/CarbonHiveInputSplit.java | 35 +-
.../carbondata/hive/MapredCarbonInputFormat.java | 17 +-
.../apache/carbondata/hive/TestCarbonSerDe.java | 137 +++
.../apache/carbondata/hive/TestCarbonSerde.java | 133 ---
.../hiveexampleCI/RunHiveExampleTest.scala | 43 -
.../presto/Presto_Cluster_Setup_For_Carbondata.md | 135 ---
integration/presto/README.md | 148 ---
integration/presto/pom.xml | 57 +-
.../carbondata/presto/CarbonVectorBatch.java | 26 +-
.../carbondata/presto/CarbondataColumnHandle.java | 143 ---
.../carbondata/presto/CarbondataConnector.java | 78 --
.../presto/CarbondataConnectorFactory.java | 160 +++-
.../carbondata/presto/CarbondataConnectorId.java | 52 -
.../carbondata/presto/CarbondataErrorCode.java | 38 -
.../presto/CarbondataHandleResolver.java | 43 -
.../carbondata/presto/CarbondataMetadata.java | 272 ------
.../apache/carbondata/presto/CarbondataModule.java | 155 ++-
.../carbondata/presto/CarbondataPageSource.java | 318 +++++-
.../presto/CarbondataPageSourceProvider.java | 209 +---
.../apache/carbondata/presto/CarbondataSplit.java | 102 --
.../carbondata/presto/CarbondataSplitManager.java | 154 +--
.../carbondata/presto/CarbondataTableHandle.java | 71 --
.../presto/CarbondataTableLayoutHandle.java | 71 --
.../presto/CarbondataTransactionHandle.java | 65 --
.../presto/PrestoCarbonVectorizedRecordReader.java | 14 +-
.../apache/carbondata/presto/PrestoFilterUtil.java | 126 +--
.../presto/impl/CarbonLocalInputSplit.java | 48 +-
.../presto/impl/CarbonLocalMultiBlockSplit.java | 16 +-
.../presto/impl/CarbonTableCacheModel.java | 29 +-
.../carbondata/presto/impl/CarbonTableConfig.java | 35 -
.../carbondata/presto/impl/CarbonTableReader.java | 405 +++-----
.../presto/readers/BooleanStreamReader.java | 17 +
...leanStreamReader.java => ByteStreamReader.java} | 48 +-
.../presto/readers/DecimalSliceStreamReader.java | 33 +-
.../presto/readers/DoubleStreamReader.java | 24 +
...ortStreamReader.java => FloatStreamReader.java} | 46 +-
.../presto/readers/IntegerStreamReader.java | 19 +
.../presto/readers/LongStreamReader.java | 24 +
.../presto/readers/ObjectStreamReader.java | 6 +
.../presto/readers/ShortStreamReader.java | 24 +
.../presto/readers/SliceStreamReader.java | 80 +-
.../presto/readers/TimestampStreamReader.java | 24 +
.../presto/CarbonDictionaryDecodeReadSupport.scala | 22 +-
.../PrestoAllDataTypeLocalDictTest.scala | 16 +-
.../integrationtest/PrestoAllDataTypeTest.scala | 14 +-
.../PrestoTestNonTransactionalTableFiles.scala | 124 ++-
.../carbondata/presto/server/PrestoServer.scala | 62 +-
.../presto/util/CarbonDataStoreCreator.scala | 3 +-
integration/spark-common-cluster-test/pom.xml | 17 +
.../cluster/sdv/generated/AlterTableTestCase.scala | 10 +-
.../cluster/sdv/generated/ColumndictTestCase.scala | 4 +-
.../sdv/generated/DataLoadingTestCase.scala | 2 +-
.../sdv/generated/PrestoSampleTestCase.scala | 56 ++
.../cluster/sdv/generated/QueriesBVATestCase.scala | 4 +-
.../sdv/generated/SetParameterTestCase.scala | 8 +-
...teTableUsingSparkCarbonFileFormatTestCase.scala | 7 +-
.../datasource/SparkCarbonDataSourceTestCase.scala | 19 +-
.../carbondata/cluster/sdv/suite/SDVSuites.scala | 18 +-
.../sql/common/util/DataSourceTestUtil.scala} | 90 +-
.../apache/spark/sql/common/util/QueryTest.scala | 101 +-
.../src/test/resources/range_column/dataskew.csv | 20 +
.../src/test/resources/structofarray.csv | 10 +
.../lucene/LuceneFineGrainDataMapSuite.scala | 7 +-
...eEncodingSafeColumnPageForComplexDataType.scala | 6 -
...ncodingUnsafeColumnPageForComplexDataType.scala | 12 +-
.../complexType/TestCompactionComplexType.scala | 65 ++
.../complexType/TestComplexDataType.scala | 8 +-
.../complexType/TestComplexTypeQuery.scala | 20 +-
.../testsuite/dataload/TestLoadDataGeneral.scala | 9 +-
.../dataload/TestNoInvertedIndexLoadAndQuery.scala | 8 +-
.../preaggregate/TestPreAggCreateCommand.scala | 46 -
...ryWithColumnMetCacheAndCacheLevelProperty.scala | 5 +-
.../badrecordloger/BadRecordLoggerTest.scala | 61 +-
.../compaction/TestHybridCompaction.scala | 262 +++++
.../TestLoadTableConcurrentScenario.scala | 0
.../TestCreateDDLForComplexMapType.scala | 71 +-
.../TestCreateHiveTableWithCarbonDS.scala | 86 ++
.../createTable/TestCreateTableWithSortScope.scala | 19 -
.../TestNonTransactionalCarbonTable.scala | 49 +-
...adDataWithMalformedCarbonCommandException.scala | 8 +-
.../dataload/TestRangeColumnDataLoad.scala | 266 ++++++
.../testsuite/datamap/TestDataMapCommand.scala | 2 +-
.../dblocation/DBLocationCarbonTableTestCase.scala | 26 +-
.../detailquery/RangeFilterTestCase.scala | 38 +
.../ValueCompressionDataTypeTestCase.scala | 22 +-
.../TimestampNoDictionaryColumnTestCase.scala | 2 -
.../filterexpr/TestImplicitFilterExpression.scala | 117 +++
.../longstring/VarcharDataTypesBasicTestCase.scala | 46 +-
.../TestAllDataTypeForPartitionTable.scala | 4 +-
.../partition/TestDDLForPartitionTable.scala | 13 +-
.../partition/TestUpdateForPartitionTable.scala | 71 ++
.../StandardPartitionTableQueryTestCase.scala | 10 +-
...StandardPartitionWithPreaggregateTestCase.scala | 22 +
.../sql/commands/TestCarbonDropCacheCommand.scala | 200 ++++
.../sql/commands/TestCarbonShowCacheCommand.scala | 233 +++++
.../client/SecureDictionaryClientHandler.java | 4 +-
.../server/SecureDictionaryServerHandler.java | 2 +-
.../org/apache/carbondata/spark/util/Util.java | 3 +-
.../org/apache/carbondata/api/CarbonStore.scala | 22 +-
.../carbondata/events/AlterTableEvents.scala | 5 +-
.../{CleanFilesEvents.scala => CacheEvents.scala} | 25 +-
.../org/apache/carbondata/events/Events.scala | 14 +
.../carbondata/spark/CarbonSparkFactory.scala | 2 +-
.../apache/carbondata/spark/PartitionFactory.scala | 2 +-
.../carbondata/spark/load/CsvRDDHelper.scala | 3 +
.../spark/load/DataLoadProcessBuilderOnSpark.scala | 226 ++++-
.../spark/load/DataLoadProcessorStepOnSpark.scala | 183 +++-
.../carbondata/spark/load/GlobalSortHelper.scala | 2 -
.../spark/rdd/AlterTableLoadPartitionRDD.scala | 6 +-
.../spark/rdd/CarbonDropPartitionRDD.scala | 2 +-
.../spark/rdd/CarbonGlobalDictionaryRDD.scala | 18 +-
.../carbondata/spark/rdd/CarbonMergeFilesRDD.scala | 0
.../carbondata/spark/rdd/CarbonMergerRDD.scala | 57 +-
.../carbondata/spark/rdd/CarbonScanRDD.scala | 44 +-
.../spark/rdd/NewCarbonDataLoadRDD.scala | 26 +-
.../spark/rdd/QueryTaskCompletionListener.scala | 1 +
.../carbondata/spark/rdd/StreamHandoffRDD.scala | 6 +-
.../spark/tasks/SortIndexWriterTask.scala | 1 -
.../carbondata/spark/util/CarbonScalaUtil.scala | 55 +-
.../apache/carbondata/spark/util/CommonUtil.scala | 42 +-
.../spark/util/GlobalDictionaryUtil.scala | 11 +-
.../streaming/CarbonStreamingQueryListener.scala | 1 -
.../carbondata/streaming/StreamSinkFactory.scala | 2 +-
.../apache/spark/DataSkewRangePartitioner.scala | 370 +++++++
.../spark/sql/catalyst/CarbonDDLSqlParser.scala | 133 ++-
.../command/carbonTableSchemaCommon.scala | 20 +-
.../apache/spark/sql/test/TestQueryExecutor.scala | 2 +-
.../org/apache/spark/sql/test/util/QueryTest.scala | 4 +-
.../apache/spark/util/CarbonReflectionUtils.scala | 40 +
.../vectorreader/VectorizedCarbonRecordReader.java | 4 +-
.../datasources/SparkCarbonFileFormat.scala | 19 +-
.../CarbonTaskCompletionListener.scala | 4 +-
.../org/apache/spark/sql/CarbonVectorProxy.java | 1 +
.../src/test/resources/test_json.json | 1 +
.../datasource/SparkCarbonDataSourceTest.scala | 372 +++++++-
...TestCreateTableUsingSparkCarbonFileFormat.scala | 22 +
.../spark/sql/carbondata/datasource/TestUtil.scala | 2 +-
integration/spark2/pom.xml | 1 -
.../sql/hive/CarbonInMemorySessionState.scala | 15 +-
.../apache/spark/sql/hive/CarbonSessionState.scala | 55 +-
.../apache/spark/sql/hive/CarbonSessionUtil.scala | 72 +-
.../spark/sql/hive/SqlAstBuilderHelper.scala | 24 +-
.../spark/rdd/CarbonDataRDDFactory.scala | 70 +-
.../spark/thriftserver/CarbonThriftServer.scala | 37 +-
.../carbondata/spark/util/CarbonSparkUtil.scala | 40 +-
.../stream/CarbonStreamRecordReader.java | 684 ++-----------
.../org/apache/spark/sql/CarbonCountStar.scala | 4 +-
.../spark/sql/CarbonDatasourceHadoopRelation.scala | 43 +-
.../scala/org/apache/spark/sql/CarbonEnv.scala | 8 +-
.../scala/org/apache/spark/sql/CarbonSession.scala | 3 +-
.../scala/org/apache/spark/sql/CarbonSource.scala | 3 +-
.../sql/execution/command/cache/CacheUtil.scala | 114 +++
.../command/cache/CarbonDropCacheCommand.scala | 66 ++
.../command/cache/CarbonShowCacheCommand.scala | 225 +++++
.../command/cache/DropCacheEventListeners.scala | 121 +++
.../command/cache/ShowCacheEventListeners.scala | 126 +++
.../sql/execution/command/carbonTableSchema.scala | 0
.../command/datamap/CarbonDropDataMapCommand.scala | 2 +-
.../CarbonAlterTableCompactionCommand.scala | 59 +-
.../CarbonAlterTableFinishStreaming.scala | 2 +-
.../command/management/CarbonLoadDataCommand.scala | 98 +-
.../management/CarbonShowLoadsCommand.scala | 2 +-
.../mutation/CarbonProjectForDeleteCommand.scala | 18 +-
.../mutation/CarbonProjectForUpdateCommand.scala | 99 +-
.../command/mutation/DeleteExecution.scala | 4 +-
.../spark/sql/execution/command/package.scala | 7 +-
.../CarbonAlterTableDropHivePartitionCommand.scala | 3 +-
.../preaaggregate/PreAggregateListeners.scala | 20 +-
.../preaaggregate/PreAggregateTableHelper.scala | 27 +-
.../command/preaaggregate/PreAggregateUtil.scala | 10 +-
.../schema/CarbonAlterTableAddColumnCommand.scala | 12 +-
...nAlterTableColRenameDataTypeChangeCommand.scala | 336 +++++++
.../CarbonAlterTableDataTypeChangeCommand.scala | 180 ----
.../schema/CarbonAlterTableDropColumnCommand.scala | 9 +-
.../schema/CarbonAlterTableRenameCommand.scala | 44 +-
.../command/table/CarbonCreateTableCommand.scala | 12 +-
.../table/CarbonDescribeFormattedCommand.scala | 57 +-
.../command/table/CarbonDropTableCommand.scala | 32 +-
.../datasources/SparkCarbonTableFormat.scala | 4 -
.../strategy/CarbonLateDecodeStrategy.scala | 35 +-
.../spark/sql/execution/strategy/DDLStrategy.scala | 26 +-
.../strategy/StreamingTableStrategy.scala | 12 +-
.../spark/sql/hive/CarbonFileMetastore.scala | 37 +-
.../apache/spark/sql/hive/CarbonMetaStore.scala | 2 +-
.../spark/sql/hive/CarbonPreAggregateRules.scala | 38 +-
.../spark/sql/hive/CarbonSessionCatalog.scala | 26 +-
.../execution/command/CarbonHiveCommands.scala | 19 +-
.../apache/spark/sql/optimizer/CarbonFilters.scala | 15 +-
.../spark/sql/optimizer/CarbonLateDecodeRule.scala | 27 +-
.../spark/sql/parser/CarbonSpark2SqlParser.scala | 46 +-
.../spark/sql/parser/CarbonSparkSqlParser.scala | 1 -
.../spark/sql/test/Spark2TestQueryExecutor.scala | 9 +-
.../org/apache/spark/util/AlterTableUtil.scala | 123 ++-
.../apache/spark/sql/CarbonToSparkAdapater.scala | 11 +-
.../apache/spark/sql/hive/CarbonSessionState.scala | 31 +-
...rkAdapater.scala => CarbonToSparkAdapter.scala} | 12 +-
.../apache/spark/sql/CarbonToSparkAdapater.scala | 11 +-
.../stream/CarbonStreamRecordReaderTest.java | 2 +-
.../BloomCoarseGrainDataMapFunctionSuite.scala | 2 +-
.../bloom/BloomCoarseGrainDataMapSuite.scala | 53 +-
.../bloom/BloomCoarseGrainDataMapTestUtil.scala | 2 +-
.../InsertIntoCarbonTableSpark2TestCase.scala | 3 -
.../spark/util/AllDictionaryTestCase.scala | 20 +-
.../spark/util/DictionaryLRUCacheTestCase.scala | 2 -
.../util/ExternalColumnDictionaryTestCase.scala | 4 +-
.../carbondata/BadRecordPathLoadOptionTest.scala | 11 +-
.../carbondata/TestStreamingTableOpName.scala | 4 +
.../carbondata/commands/SetCommandTestCase.scala | 28 +
.../datatype/NumericDimensionBadRecordTest.scala | 2 -
.../iud/DeleteCarbonTableSubqueryTestCase.scala | 3 -
.../register/TestRegisterCarbonTable.scala | 24 +-
.../restructure/AlterTableRevertTestCase.scala | 1 -
.../restructure/AlterTableValidationTestCase.scala | 69 ++
.../AlterTableColumnRenameTestCase.scala | 356 +++++++
.../vectorreader/ChangeDataTypeTestCases.scala | 2 +-
.../sql/CarbonGetTableDetailComandTestCase.scala | 0
.../spark/sql/common/util/Spark2QueryTest.scala | 3 +-
.../org/apache/spark/util/CarbonCommandSuite.scala | 1 -
pom.xml | 6 +-
processing/pom.xml | 8 +
.../processing/datamap/DataMapWriterListener.java | 2 +-
.../processing/datatypes/ArrayDataType.java | 20 +-
.../processing/datatypes/GenericDataType.java | 2 +-
.../processing/datatypes/PrimitiveDataType.java | 29 +-
.../processing/datatypes/StructDataType.java | 22 +-
.../loading/AbstractDataLoadProcessorStep.java | 2 +-
.../loading/CarbonDataLoadConfiguration.java | 32 +-
.../loading/TableProcessingOperations.java | 3 +-
.../converter/impl/MeasureFieldConverterImpl.java | 8 +-
.../impl/NonDictionaryFieldConverterImpl.java | 8 +-
.../loading/converter/impl/RowConverterImpl.java | 4 +-
.../loading/csvinput/CSVInputFormat.java | 10 +
.../processing/loading/events/LoadEvents.java | 13 +-
.../processing/loading/model/CarbonLoadModel.java | 47 +
.../loading/model/CarbonLoadModelBuilder.java | 50 +-
.../processing/loading/model/LoadOption.java | 3 +
.../loading/parser/impl/JsonRowParser.java | 2 +-
.../loading/parser/impl/RangeColumnParserImpl.java | 56 ++
.../loading/row/IntermediateSortTempRow.java | 8 +
.../loading/sort/SortStepRowHandler.java | 7 +-
.../sort/impl/ParallelReadMergeSorterImpl.java | 5 +-
...ParallelReadMergeSorterWithColumnRangeImpl.java | 2 +-
.../UnsafeBatchParallelReadMergeSorterImpl.java | 4 +-
.../impl/UnsafeParallelReadMergeSorterImpl.java | 5 +-
...ParallelReadMergeSorterWithColumnRangeImpl.java | 2 +-
.../loading/sort/unsafe/UnsafeCarbonRowPage.java | 17 +-
.../loading/sort/unsafe/UnsafeSortDataRows.java | 51 +-
.../holder/UnsafeSortTempFileChunkHolder.java | 26 +-
.../merger/UnsafeIntermediateFileMerger.java | 2 +-
.../unsafe/merger/UnsafeIntermediateMerger.java | 11 +-
.../UnsafeSingleThreadFinalSortFilesMerger.java | 4 +-
.../CarbonRowDataWriterProcessorStepImpl.java | 16 +-
.../steps/DataWriterBatchProcessorStepImpl.java | 10 +-
.../loading/steps/DataWriterProcessorStepImpl.java | 8 +-
.../loading/steps/InputProcessorStepImpl.java | 2 +-
.../processing/merger/AbstractResultProcessor.java | 6 +-
.../merger/CarbonCompactionExecutor.java | 96 +-
.../processing/merger/CarbonCompactionUtil.java | 96 +-
.../processing/merger/CarbonDataMergerUtil.java | 6 +-
.../merger/CompactionResultSortProcessor.java | 61 +-
.../merger/RowResultMergerProcessor.java | 13 +-
.../partition/spliter/RowResultProcessor.java | 6 +-
.../DummyRowUpdater.java} | 25 +-
.../processing/sort/SchemaBasedRowUpdater.java | 91 ++
.../SortTempRowUpdater.java} | 27 +-
.../sort/sortdata/InMemorySortTempChunkHolder.java | 147 +++
.../sortdata/SingleThreadFinalSortFilesMerger.java | 56 +-
.../processing/sort/sortdata/SortDataRows.java | 5 +-
.../sort/sortdata/SortIntermediateFileMerger.java | 3 +-
.../processing/sort/sortdata/SortParameters.java | 59 +-
.../sort/sortdata/SortTempFileChunkHolder.java | 46 +-
.../processing/sort/sortdata/TableFieldStat.java | 50 +-
.../store/CarbonFactDataHandlerColumnar.java | 20 +-
.../store/CarbonFactDataHandlerModel.java | 41 +-
.../processing/store/CarbonFactHandlerFactory.java | 1 -
.../carbondata/processing/store/TablePage.java | 6 +-
.../store/writer/AbstractFactDataWriter.java | 54 +-
.../writer/v3/CarbonFactDataWriterImplV3.java | 5 +-
.../processing/util/CarbonDataProcessorUtil.java | 33 +
.../processing/util/CarbonLoaderUtil.java | 24 +-
store/CSDK/src/CarbonWriter.cpp | 3 -
store/CSDK/test/main.cpp | 75 +-
store/sdk/pom.xml | 8 +
.../carbondata/sdk/file/CarbonWriterBuilder.java | 36 +-
.../apache/carbondata/store/LocalCarbonStore.java | 4 +-
.../carbondata/sdk/file/CarbonReaderTest.java | 350 ++++++-
.../sdk/file/ConcurrentSdkReaderTest.java | 1 -
streaming/pom.xml | 35 +-
.../streaming/parser/FieldConverter.scala | 44 +-
.../streaming/parser/RowStreamParserImp.scala | 15 +-
tools/cli/pom.xml | 8 +
.../java/org/apache/carbondata/tool/CarbonCli.java | 4 +-
.../org/apache/carbondata/tool/CarbonCliTest.java | 69 +-
475 files changed, 14274 insertions(+), 7695 deletions(-)
copy core/src/main/java/org/apache/carbondata/core/locks/{S3FileLock.java => AlluxioFileLock.java} (54%)
create mode 100644 core/src/main/java/org/apache/carbondata/core/scan/expression/conditional/ImplicitExpression.java
copy processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtilResult.java => core/src/main/java/org/apache/carbondata/core/scan/wrappers/IntArrayWrapper.java (61%)
rename {hadoop => core}/src/main/java/org/apache/carbondata/hadoop/CarbonInputSplit.java (51%)
rename {hadoop => core}/src/main/java/org/apache/carbondata/hadoop/internal/ObjectArrayWritable.java (100%)
rename {hadoop => core}/src/main/java/org/apache/carbondata/hadoop/internal/index/Block.java (100%)
create mode 100644 docs/alluxio-guide.md
create mode 100644 docs/presto-guide.md
rename integration/hive/src/main/resources/data.csv => examples/spark2/src/main/resources/sample.csv (100%)
rename {integration/hive/src/main/scala/org/apache/carbondata/hiveexample => examples/spark2/src/main/scala/org/apache/carbondata/examples}/HiveExample.scala (71%)
rename {streaming/src/main/java/org/apache/carbondata/streaming => hadoop/src/main/java/org/apache/carbondata/hadoop/stream}/CarbonStreamInputFormat.java (97%)
rename {streaming/src/main/java/org/apache/carbondata/streaming => hadoop/src/main/java/org/apache/carbondata/hadoop/stream}/CarbonStreamUtils.java (97%)
rename {streaming/src/main/java/org/apache/carbondata/streaming => hadoop/src/main/java/org/apache/carbondata/hadoop/stream}/StreamBlockletReader.java (99%)
copy integration/spark2/src/main/scala/org/apache/carbondata/stream/CarbonStreamRecordReader.java => hadoop/src/main/java/org/apache/carbondata/hadoop/stream/StreamRecordReader.java (78%)
create mode 100644 integration/hive/src/test/java/org/apache/carbondata/hive/TestCarbonSerDe.java
delete mode 100644 integration/hive/src/test/java/org/apache/carbondata/hive/TestCarbonSerde.java
delete mode 100644 integration/hive/src/test/scala/org/apache/carbondata/hiveexampleCI/RunHiveExampleTest.scala
delete mode 100644 integration/presto/Presto_Cluster_Setup_For_Carbondata.md
delete mode 100644 integration/presto/README.md
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataColumnHandle.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataConnector.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataConnectorId.java
delete mode 100644 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataErrorCode.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataHandleResolver.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataMetadata.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataSplit.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataTableHandle.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataTableLayoutHandle.java
delete mode 100755 integration/presto/src/main/java/org/apache/carbondata/presto/CarbondataTransactionHandle.java
copy integration/presto/src/main/java/org/apache/carbondata/presto/readers/{BooleanStreamReader.java => ByteStreamReader.java} (70%)
copy integration/presto/src/main/java/org/apache/carbondata/presto/readers/{ShortStreamReader.java => FloatStreamReader.java} (64%)
create mode 100644 integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/PrestoSampleTestCase.scala
copy integration/{spark-datasource/src/test/scala/org/apache/spark/sql/carbondata/datasource/TestUtil.scala => spark-common-cluster-test/src/test/scala/org/apache/spark/sql/common/util/DataSourceTestUtil.scala} (66%)
create mode 100644 integration/spark-common-test/src/test/resources/range_column/dataskew.csv
create mode 100644 integration/spark-common-test/src/test/resources/structofarray.csv
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/compaction/TestHybridCompaction.scala
delete mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/concurrent/TestLoadTableConcurrentScenario.scala
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/createTable/TestCreateHiveTableWithCarbonDS.scala
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestRangeColumnDataLoad.scala
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/filterexpr/TestImplicitFilterExpression.scala
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/partition/TestUpdateForPartitionTable.scala
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/sql/commands/TestCarbonDropCacheCommand.scala
create mode 100644 integration/spark-common-test/src/test/scala/org/apache/carbondata/sql/commands/TestCarbonShowCacheCommand.scala
copy integration/spark-common/src/main/scala/org/apache/carbondata/events/{CleanFilesEvents.scala => CacheEvents.scala} (72%)
delete mode 100644 integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonMergeFilesRDD.scala
create mode 100644 integration/spark-common/src/main/scala/org/apache/spark/DataSkewRangePartitioner.scala
create mode 100644 integration/spark-datasource/src/test/resources/test_json.json
create mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CacheUtil.scala
create mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonDropCacheCommand.scala
create mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/CarbonShowCacheCommand.scala
create mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/DropCacheEventListeners.scala
create mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/cache/ShowCacheEventListeners.scala
delete mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchema.scala
create mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableColRenameDataTypeChangeCommand.scala
delete mode 100644 integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDataTypeChangeCommand.scala
rename integration/spark2/src/main/spark2.2/org/apache/spark/sql/{CarbonToSparkAdapater.scala => CarbonToSparkAdapter.scala} (89%)
create mode 100644 integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/vectorreader/AlterTableColumnRenameTestCase.scala
delete mode 100644 integration/spark2/src/test/scala/org/apache/spark/sql/CarbonGetTableDetailComandTestCase.scala
create mode 100644 processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/RangeColumnParserImpl.java
copy processing/src/main/java/org/apache/carbondata/processing/{loading/sort/unsafe/holder/SortTempChunkHolder.java => sort/DummyRowUpdater.java} (54%)
create mode 100644 processing/src/main/java/org/apache/carbondata/processing/sort/SchemaBasedRowUpdater.java
copy processing/src/main/java/org/apache/carbondata/processing/{loading/sort/unsafe/holder/SortTempChunkHolder.java => sort/SortTempRowUpdater.java} (53%)
create mode 100644 processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/InMemorySortTempChunkHolder.java