You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by da...@apache.org on 2018/10/23 23:49:27 UTC
[1/2] hive git commit: RELEASE_NOTES.txt for release 3.1.1
Repository: hive
Updated Branches:
refs/heads/branch-3.1 a01c3827b -> 436d01a36
http://git-wip-us.apache.org/repos/asf/hive/blob/436d01a3/RELEASE_NOTES.txt
----------------------------------------------------------------------
diff --git a/RELEASE_NOTES.txt b/RELEASE_NOTES.txt
index bde815e..100d35e 100644
--- a/RELEASE_NOTES.txt
+++ b/RELEASE_NOTES.txt
@@ -1,2156 +1,11 @@
-Release Notes - Hive - Version 3.1.0
+Release Notes - Hive - Version 3.1.1
-** Sub-task
- * [HIVE-12192] - Hive should carry out timestamp computations in UTC
- * [HIVE-17227] - Incremental replication load should create tasks in execution phase rather than semantic phase
- * [HIVE-17657] - export/import for MM tables is broken
- * [HIVE-18193] - Migrate existing ACID tables to use write id per table rather than global transaction id
- * [HIVE-18748] - Rename table impacts the ACID behavior as table names are not updated in meta-tables.
- * [HIVE-18840] - CachedStore: Prioritize loading of recently accessed tables during prewarm
- * [HIVE-18946] - Fix columnstats merge NPE
- * [HIVE-18988] - Support bootstrap replication of ACID tables
- * [HIVE-19009] - Retain and use runtime statistics during hs2 lifetime
- * [HIVE-19096] - query result cache interferes with explain analyze
- * [HIVE-19126] - CachedStore: Use memory estimation to limit cache size during prewarm
- * [HIVE-19128] - Update golden files for spark perf tests
- * [HIVE-19135] - Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
- * [HIVE-19141] - TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
- * [HIVE-19159] - TestMTQueries.testMTQueries1 failure
- * [HIVE-19164] - TestMetastoreVersion failures
- * [HIVE-19171] - Persist runtime statistics in metastore
- * [HIVE-19193] - TestActivePassiveHA fails
- * [HIVE-19194] - TestDruidStorageHandler fails
- * [HIVE-19195] - Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.
- * [HIVE-19196] - TestTriggersMoveWorkloadManager is flaky
- * [HIVE-19206] - Automatic memory management for open streaming writers
- * [HIVE-19209] - Streaming ingest record writers should accept input stream
- * [HIVE-19210] - Create separate module for streaming ingest
- * [HIVE-19211] - New streaming ingest API and support for dynamic partitioning
- * [HIVE-19214] - High throughput ingest ORC format
- * [HIVE-19222] - TestNegativeCliDriver tests are failing due to "java.lang.OutOfMemoryError: GC overhead limit exceeded"
- * [HIVE-19243] - Upgrade hadoop.version to 3.1.0
- * [HIVE-19252] - TestJdbcWithMiniKdcCookie.testCookieNegative is failing consistently
- * [HIVE-19274] - Add an OpTreeSignature persistence checker hook
- * [HIVE-19332] - Disable compute.query.using.stats for external table
- * [HIVE-19336] - Disable SMB/Bucketmap join for external tables
- * [HIVE-19340] - Disable timeout of transactions opened by replication task at target cluster
- * [HIVE-19347] - TestTriggersWorkloadManager tests are failing consistently
- * [HIVE-19348] - org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
- * [HIVE-19366] - Vectorization causing TestStreaming.testStreamBucketingMatchesRegularBucketing to fail
- * [HIVE-19374] - Parse and process ALTER TABLE SET OWNER command syntax
- * [HIVE-19409] - Disable incremental rewriting with outdated materialized views
- * [HIVE-19472] - HiveStreamingConnection swallows exception on partition creation
- * [HIVE-19494] - Accept shade prefix during reflective instantiation of output format
- * [HIVE-19495] - Arrow SerDe itest failure
- * [HIVE-19499] - Bootstrap REPL LOAD shall add tasks to create checkpoints for db/tables/partitions.
- * [HIVE-19500] - Prevent multiple selectivity estimations for the same variable in conjuctions
- * [HIVE-19562] - Flaky test: TestMiniSparkOnYarn FileNotFoundException in spark-submit
- * [HIVE-19598] - Add Acid V1 to V2 upgrade module
- * [HIVE-19637] - Add slow test report script to testutils
- * [HIVE-19688] - Make catalogs updatable
- * [HIVE-19727] - Fix Signature matching of table aliases
- * [HIVE-19739] - Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
- * [HIVE-19758] - Set hadoop.version=3.1.0 in standalone-metastore
- * [HIVE-19768] - Utility to convert tables to conform to Hive strict managed tables mode
- * [HIVE-19799] - remove jasper dependency
- * [HIVE-19815] - Repl dump should not propagate the checkpoint and repl source properties
- * [HIVE-19851] - upgrade jQuery version
- * [HIVE-19852] - update jackson to latest
- * [HIVE-19868] - Add support for float aggregator
- * [HIVE-19892] - Disable query results cache for for HiveServer2 doAs=true
- * [HIVE-19923] - Follow up of HIVE-19615, use UnaryFunction instead of prefix
** Bug
- * [HIVE-15190] - Field names are not preserved in ORC files written with ACID
- * [HIVE-18434] - Type is not determined correctly for comparison between decimal column and string constant
- * [HIVE-18816] - CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
- * [HIVE-19016] - Vectorization and Parquet: Disable vectorization for nested complex types
- * [HIVE-19054] - Function replication shall use "hive.repl.replica.functions.root.dir" as root
- * [HIVE-19108] - Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
- * [HIVE-19109] - Vectorization: Enabling vectorization causes TestCliDriver delete_orig_table.q to produce Wrong Results
- * [HIVE-19110] - Vectorization: Enabling vectorization causes TestContribCliDriver udf_example_arraymapstruct.q to produce Wrong Results
- * [HIVE-19118] - Vectorization: Turning on vectorization in escape_crlf produces wrong results
- * [HIVE-19120] - catalog not properly set for some tables in SQL upgrade scripts
- * [HIVE-19131] - DecimalColumnStatsMergerTest comparison review
- * [HIVE-19155] - Day time saving cause Druid inserts to fail with org.apache.hive.druid.io.druid.java.util.common.UOE: Cannot add overlapping segments
- * [HIVE-19157] - Assert that Insert into Druid Table fails if the publishing of metadata by HS2 fails
- * [HIVE-19167] - Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)
- * [HIVE-19168] - Ranger changes for llap commands
- * [HIVE-19186] - Multi Table INSERT statements query has a flaw for partitioned table when INSERT INTO and INSERT OVERWRITE are used
- * [HIVE-19200] - Vectorization: Disable vectorization for LLAP I/O when a non-VECTORIZED_INPUT_FILE_FORMAT mode is needed (i.e. rows) and data type conversion is needed
- * [HIVE-19219] - Incremental REPL DUMP should throw error if requested events are cleaned-up.
- * [HIVE-19230] - Schema column width inconsistency in Oracle
- * [HIVE-19231] - Beeline generates garbled output when using UnsupportedTerminal
- * [HIVE-19237] - Only use an operatorId once in a plan
- * [HIVE-19247] - StatsOptimizer: Missing stats fast-path for Date
- * [HIVE-19248] - REPL LOAD couldn't copy file from source CM path and also doesn't throw error if file copy fails.
- * [HIVE-19258] - add originals support to MM tables (and make the conversion a metadata only operation)
- * [HIVE-19264] - Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
- * [HIVE-19269] - Vectorization: Turn On by Default
- * [HIVE-19275] - Vectorization: Defer Wrong Results / Execution Failures when Vectorization turned on
- * [HIVE-19277] - Active/Passive HA web endpoints does not allow cross origin requests
- * [HIVE-19312] - MM tables don't work with BucketizedHIF
- * [HIVE-19317] - Handle schema evolution from int like types to decimal
- * [HIVE-19327] - qroupby_rollup_empty.q fails for insert-only transactional tables
- * [HIVE-19331] - Repl load config in "with" clause not pass to Context.getStagingDir
- * [HIVE-19350] - Vectorization: Turn off vectorization for explainuser_1.q / spark_explainuser_1
- * [HIVE-19352] - Vectorization: Disable vectorization for org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
- * [HIVE-19357] - Vectorization: assert_true HiveException erroneously gets suppressed to NULL
- * [HIVE-19358] - CBO decorrelation logic should generate Hive operators
- * [HIVE-19365] - Index on COMPLETED_TXN_COMPONENTS in Metastore RDBMS has different names in different scripts
- * [HIVE-19370] - Issue: ADD Months function on timestamp datatype fields in hive
- * [HIVE-19381] - Function replication in cloud fail when download resource from AWS
- * [HIVE-19382] - Acquire locks before generating valid transaction list for some operations
- * [HIVE-19384] - Vectorization: IfExprTimestamp* do not handle NULLs correctly
- * [HIVE-19385] - Optional hive env variable to redirect bin/hive to use Beeline
- * [HIVE-19389] - Schematool: For Hive's Information Schema, use embedded HS2 as default
- * [HIVE-19410] - don't create serde reader in LLAP if there's no cache
- * [HIVE-19418] - add background stats updater similar to compactor
- * [HIVE-19423] - REPL LOAD creates staging directory in source dump directory instead of table data location
- * [HIVE-19433] - HiveJoinPushTransitivePredicatesRule hangs
- * [HIVE-19435] - Incremental replication cause data loss if a table is dropped followed by create and insert-into with different partition type.
- * [HIVE-19454] - Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
- * [HIVE-19460] - Improve stats estimations for NOT IN operator
- * [HIVE-19463] - TezTask - getting groups may fail (PartialGroupNameException in some tests)
- * [HIVE-19467] - Make storage format configurable for temp tables created using LLAP external client
- * [HIVE-19474] - Decimal type should be casted as part of the CTAS or INSERT Clause.
- * [HIVE-19479] - encoded stream seek is incorrect for 0-length RGs in LLAP IO
- * [HIVE-19481] - Tablesample uses incorrect logic to pick files corresponding to buckets.
- * [HIVE-19485] - dump directory for non native tables should not be created
- * [HIVE-19493] - VectorUDFDateDiffColCol copySelected does not handle nulls correctly
- * [HIVE-19496] - Check untar folder
- * [HIVE-19498] - Vectorization: CAST expressions produce wrong results
- * [HIVE-19504] - Change default value for hive.auto.convert.join.shuffle.max.size property
- * [HIVE-19516] - TestNegative merge_negative_5 and mm_concatenate are causing timeouts
- * [HIVE-19529] - Vectorization: Date/Timestamp NULL issues
- * [HIVE-19557] - stats: filters for dates are not taking advantage of min/max values
- * [HIVE-19565] - Vectorization: Fix NULL / Wrong Results issues in STRING Functions
- * [HIVE-19567] - Fix flakiness in TestTriggers
- * [HIVE-19569] - alter table db1.t1 rename db2.t2 generates MetaStoreEventListener.onDropTable()
- * [HIVE-19575] - TestAutoPurgeTables seems flaky
- * [HIVE-19577] - CREATE TEMPORARY TABLE LIKE and INSERT generate output format mismatch errors
- * [HIVE-19578] - HLL merges tempList on every add
- * [HIVE-19588] - Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
- * [HIVE-19589] - Disable TestAutoPurge tests and annotate TestTriggersWorkloadManager with retry
- * [HIVE-19590] - mask stats in llap_smb
- * [HIVE-19592] - TestWorkloadManager - add retry for now
- * [HIVE-19594] - Add custom tmp folders to tests to avoid collisions
- * [HIVE-19595] - Regenerate webui port in MiniHS2
- * [HIVE-19604] - Incorrect Handling of Boolean in DruidSerde
- * [HIVE-19605] - TAB_COL_STATS table has no index on db/table name
- * [HIVE-19608] - disable flaky tests 2
- * [HIVE-19613] - GenericUDTFGetSplits should handle fetch task with temp table rewrite
- * [HIVE-19614] - GenericUDTFGetSplits does not honor ORDER BY
- * [HIVE-19615] - Proper handling of is null and not is null predicate when pushed to Druid
- * [HIVE-19619] - Allow comparisons between doubles and bigints
- * [HIVE-19629] - Enable Decimal64 reader after orc version upgrade
- * [HIVE-19631] - reduce epic locking in AbstractService
- * [HIVE-19632] - Remove webapps directory from standalone jar
- * [HIVE-19639] - a transactional Hive table cannot be imported as an external table
- * [HIVE-19643] - MM table conversion doesn't need full ACID structure checks
- * [HIVE-19644] - change WM syntax to avoid conflicts with identifiers starting with a number
- * [HIVE-19646] - Filesystem closed error in HiveProtoLoggingHook
- * [HIVE-19660] - update branch-3 to be version 3.1 and fix storage-api mismatch
- * [HIVE-19675] - Cast to timestamps on Druid time column leads to an exception
- * [HIVE-19677] - Disable sample6.q
- * [HIVE-19680] - Push down limit is not applied for Druid storage handler.
- * [HIVE-19684] - Hive stats optimizer wrongly uses stats against non native tables
- * [HIVE-19687] - Export table on acid partitioned table is failing
- * [HIVE-19690] - multi-insert query with multiple GBY, and distinct in only some branches can produce incorrect results
- * [HIVE-19691] - Start SessionState in materialized views registry
- * [HIVE-19695] - Year Month Day extraction functions need to add an implicit cast for column that are String types
- * [HIVE-19697] - TestReOptimization#testStatCachingMetaStore is flaky
- * [HIVE-19698] - TestAMReporter#testMultipleAM is flaky
- * [HIVE-19700] - Workaround for JLine issue with UnsupportedTerminal
- * [HIVE-19713] - itests/hive-jmh should not reference a concreate storage-api version
- * [HIVE-19723] - Arrow serde: "Unsupported data type: Timestamp(NANOSECOND, null)"
- * [HIVE-19726] - ORC date PPD is broken
- * [HIVE-19728] - beeline with USE_BEELINE_FOR_HIVE_CLI fails when trying to set hive.aux.jars.path
- * [HIVE-19734] - Beeline: When beeline-site.xml is present, beeline does not honor -n (username) and -p (password) arguments
- * [HIVE-19744] - In Beeline if -u is specified the default connection should not be tried at all
- * [HIVE-19750] - Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
- * [HIVE-19753] - Strict managed tables mode in Hive
- * [HIVE-19754] - vector_decimal_2 failing on branch-3
- * [HIVE-19755] - insertsel_fail.q.out needs to be updated on branch-3
- * [HIVE-19762] - Druid Queries containing Joins gives wrong results.
- * [HIVE-19771] - allowNullColumnForMissingStats should not be false when column stats are estimated
- * [HIVE-19772] - Streaming ingest V2 API can generate invalid orc file if interrupted
- * [HIVE-19773] - CBO exception while running queries with tables that are not present in materialized views
- * [HIVE-19777] - NPE in TezSessionState
- * [HIVE-19789] - reenable orc_llap test
- * [HIVE-19793] - disable LLAP IO batch-to-row wrapper for ACID deletes/updates
- * [HIVE-19794] - Disable removing order by from subquery in GenericUDTFGetSplits
- * [HIVE-19796] - Push Down TRUNC Fn to Druid Storage Handler
- * [HIVE-19801] - JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
- * [HIVE-19808] - GenericUDTFGetSplits should support ACID reads in the temp. table read path
- * [HIVE-19810] - StorageHandler fail to ship jars in Tez intermittently
- * [HIVE-19813] - SessionState.start don't have to be synchronized
- * [HIVE-19817] - Hive streaming API + dynamic partitioning + json/regex writer does not work
- * [HIVE-19826] - OrcRawRecordMerger doesn't work for more than one file in non vectorized case
- * [HIVE-19827] - hiveserver2 startup should provide a way to override TEZ_CONF_DIR
- * [HIVE-19833] - reduce LLAP IO min allocation to match ORC variable CB size
- * [HIVE-19837] - Setting to have different default location for external tables
- * [HIVE-19838] - simplify & fix ColumnizedDeleteEventRegistry load loop
- * [HIVE-19853] - Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
- * [HIVE-19857] - Set 3.1.0 for sys db version
- * [HIVE-19859] - Inspect lock components for DBHiveLock while verifying whether transaction list is valid
- * [HIVE-19861] - Fix temp table path generation for acid table export
- * [HIVE-19862] - Postgres init script has a glitch around UNIQUE_DATABASE
- * [HIVE-19864] - Address TestTriggersWorkloadManager flakiness
- * [HIVE-19866] - improve LLAP cache purge
- * [HIVE-19869] - Remove double formatting bug followup of HIVE-19382
- * [HIVE-19872] - hive-schema-3.1.0.hive.sql is missing on master and branch-3
- * [HIVE-19873] - Cleanup operation log on query cancellation after some delay
- * [HIVE-19875] - increase LLAP IO queue size for perf
- * [HIVE-19876] - Multiple fixes for Driver.isValidTxnListState
- * [HIVE-19877] - Remove setting hive.execution.engine as mr in HiveStreamingConnection
- * [HIVE-19879] - Remove unused calcite sql operator.
- * [HIVE-19884] - Invalidation cache may throw NPE when there is no data in table used by materialized view
- * [HIVE-19889] - Wrong results due to PPD of non deterministic functions with CBO
- * [HIVE-19890] - ACID: Inherit bucket-id from original ROW_ID for delete deltas
- * [HIVE-19898] - Disable TransactionalValidationListener when the table is not in the Hive catalog
- * [HIVE-19903] - Disable temporary insert-only transactional table
- * [HIVE-19904] - Load data rewrite into Tez job fails for ACID
- * [HIVE-19908] - Block Insert Overwrite with Union All on full CRUD ACID tables using HIVE_UNION_SUBDIR_
- * [HIVE-19912] - Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
- * [HIVE-19917] - Export of full CRUD transactional table fails if table is not in default database
- * [HIVE-19920] - Schematool fails in embedded mode when auth is on
- * [HIVE-19921] - Fix perf duration and queue name in HiveProtoLoggingHook
- * [HIVE-19938] - Upgrade scripts for information schema
- * [HIVE-19941] - Row based Filters added via Hive Ranger policies are not pushed to druid
- * [HIVE-19946] - VectorizedRowBatchCtx.recordIdColumnVector cannot be shared between different JVMs
- * [HIVE-19951] - Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)
- * [HIVE-19956] - Include yarn registry classes to jdbc standalone jar
- * [HIVE-19964] - Apply resource plan fails if trigger expression has quotes
- * [HIVE-19965] - Make HiveEndPoint use IMetaStoreClient.add_partition
- * [HIVE-19972] - Followup to HIVE-19928 : Fix the check for managed table
- * [HIVE-19973] - Enable materialized view rewriting by default
- * [HIVE-19980] - GenericUDTFGetSplits fails when order by query returns 0 rows
- * [HIVE-19997] - Batches for TestMiniDruidCliDriver
- * [HIVE-20002] - Shipping jdbd-storage-handler dependency jars in LLAP
- * [HIVE-20010] - Fix create view over literals
- * [HIVE-20076] - ACID: Fix Synthetic ROW__ID generation for vectorized orc readers
+ * [HIVE-18778] - Needs to capture input/output entities in explain
** New Feature
- * [HIVE-18739] - Add support for Import/Export from Acid table
- * [HIVE-19307] - Support ArrowOutputStream in LlapOutputFormatService
-
-
-** Improvement
- * [HIVE-17824] - msck repair table should drop the missing partitions from metastore
- * [HIVE-18079] - Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
- * [HIVE-18394] - Materialized view: "Create Materialized View" should default to rewritable ones
- * [HIVE-18410] - [Performance][Avro] Reading flat Avro tables is very expensive in Hive
- * [HIVE-18743] - CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
- * [HIVE-18792] - Allow standard compliant syntax for insert on partitioned tables
- * [HIVE-18866] - Semijoin and analyze: Implement a Long -> Hash64 vector fast-path
- * [HIVE-19027] - Make materializations invalidation cache work with multiple active remote metastores
- * [HIVE-19161] - Add authorizations to information schema
- * [HIVE-19228] - Remove commons-httpclient 3.x usage
- * [HIVE-19259] - Create view on tables having union all fail with "Table not found"
- * [HIVE-19344] - Change default value of msck.repair.batch.size
- * [HIVE-19390] - Useless error messages logged for dummy table stats
- * [HIVE-19415] - Support CORS for all HS2 web endpoints
- * [HIVE-19421] - Upgrade version of Jetty to 9.3.20.v20170531
- * [HIVE-19440] - Make StorageBasedAuthorizer work with information schema
- * [HIVE-19462] - Fix mapping for char_length function to enable pushdown to Druid.
- * [HIVE-19464] - Upgrade Parquet to 1.10.0
- * [HIVE-19465] - Upgrade ORC to 1.5.0
- * [HIVE-19466] - Update constraint violation error message
- * [HIVE-19490] - Locking on Insert into for non native and managed tables.
- * [HIVE-19534] - Allow implementations to access member variables of AbstractRecordWriter
- * [HIVE-19560] - Retry test runner and retry rule for flaky tests
- * [HIVE-19572] - Add option to mask stats and data size in q files
- * [HIVE-19586] - Optimize Count(distinct X) pushdown based on the storage capabilities
- * [HIVE-19669] - Upgrade ORC to 1.5.1
- * [HIVE-19682] - Provide option for GenericUDTFGetSplits to return only schema metadata
- * [HIVE-19775] - Schematool should use HS2 embedded mode in privileged auth mode
- * [HIVE-19776] - HiveServer2.startHiveServer2 retries of start has concurrency issues
- * [HIVE-19824] - Improve online datasize estimations for MapJoins
- * [HIVE-19885] - Druid Kafka Ingestion - Allow user to set kafka consumer properties via table properties
- * [HIVE-20102] - Add a couple of additional tests for query parsing
- * [HIVE-20135] - Fix incompatible change in TimestampColumnVector to default to UTC
-
-** Test
- * [HIVE-19271] - TestMiniLlapLocalCliDriver default_constraint and check_constraint failing
- * [HIVE-19515] - TestRpc.testServerPort is consistently failing
- * [HIVE-19555] - Enable TestMiniLlapLocalCliDriver#tez_dynpart_hashjoin_1.q and TestMiniLlapLocalCliDriver#tez_vector_dynpart_hashjoin_1.q
- * [HIVE-19573] - Fix flaky TestMiniLlapLocalCliDriver#explainuser_4.q
- * [HIVE-19612] - Add option to mask lineage in q files
- * [HIVE-19617] - Rename test tables to avoid collisions during execution in batches
- * [HIVE-19620] - Change tmp directory used by PigServer in HCat tests
- * [HIVE-19626] - Change tmp staging mapred directory for CliDriver
- * [HIVE-19654] - Change tmp staging mapred directory for TestBlobstoreCliDriver
- * [HIVE-19655] - Mask stats for TestMiniLlapLocalCliDriver#smb_mapjoin_15
- * [HIVE-19699] - Re-enable TestReOptimization
- * [HIVE-19706] - Disable TestJdbcWithMiniHS2#testHttpRetryOnServerIdleTimeout
- * [HIVE-19731] - Change staging tmp directory used by TestHCatLoaderComplexSchema
- * [HIVE-20123] - Fix masking tests after HIVE-19617
-
-
-** Task
- * [HIVE-18875] - Enable SMB Join by default in Tez
- * [HIVE-19134] - Update copyright NOTICE and fix rat check failures
- * [HIVE-19140] - Update metastore upgrade scripts to prepare for 3.1.0 development
- * [HIVE-19257] - HIVE-19157 commit references wrong jira
- * [HIVE-19306] - Arrow batch serializer
- * [HIVE-19308] - Provide an Arrow stream reader for external LLAP clients
- * [HIVE-19323] - Create metastore SQL install and upgrade scripts for 3.1
- * [HIVE-19488] - Enable CM root based on db parameter, identifying a db as source of replication.
- * [HIVE-19509] - Disable tests that are failing continuously
- * [HIVE-19512] - If parallel execution is enabled, metastore is throwing out of sequence error.
- * [HIVE-19708] - Repl copy retrying with cm path even if the failure is due to network issue
- * [HIVE-19725] - Add ability to dump non-native tables in replication metadata dump
- * [HIVE-19880] - Repl Load to return recoverable vs non-recoverable error codes
- * [HIVE-19881] - Allow metadata-only dump for database which are not source of replication
- * [HIVE-19928] - Load Data for managed tables should set the owner of loaded files to a configurable user
- * [HIVE-19978] - Backport HIVE-18037 to branch-3
-
-
-Release Notes - Hive - Version 3.0.0
-
-** Sub-task
- * [HIVE-11133] - Support hive.explain.user for Spark
- * [HIVE-11418] - Dropping a database in an encryption zone with CASCADE and trash enabled fails
- * [HIVE-13567] - Enable auto-gather column stats by default
- * [HIVE-13583] - E061-14: Search Conditions
- * [HIVE-13673] - LLAP: handle case where no service instance is found on the host specified in the input split
- * [HIVE-14412] - Add timestamp with time zone
- * [HIVE-14487] - Add REBUILD statement for materialized views
- * [HIVE-14495] - Add SHOW MATERIALIZED VIEWS statement
- * [HIVE-14498] - Freshness period for query rewriting using materialized views
- * [HIVE-14518] - Support 'having' translation for Druid GroupBy queries
- * [HIVE-14747] - Remove JAVA paths from profiles by sending them from ptest-client
- * [HIVE-14947] - Add support for Acid 2 in Merge
- * [HIVE-15016] - Run tests with Hadoop 3.0.0-beta1
- * [HIVE-15018] - ALTER rewriting flag in materialized view
- * [HIVE-15051] - Test framework integration with findbugs, rat checks etc.
- * [HIVE-15173] - Allow dec as an alias for decimal
- * [HIVE-15212] - merge branch into master
- * [HIVE-15326] - Hive shims report Unrecognized Hadoop major version number: 3.0.0-alpha2-SNAPSHOT
- * [HIVE-15436] - Enhancing metastore APIs to retrieve only materialized views
- * [HIVE-15490] - REPL LOAD & DUMP support for INSERT events with change management
- * [HIVE-15619] - Column pruner should handle DruidQuery
- * [HIVE-15642] - Replicate Insert Overwrites, Dynamic Partition Inserts and Loads
- * [HIVE-15673] - Allow multiple queries with disjunction
- * [HIVE-15705] - Event replication for constraints
- * [HIVE-15725] - Make it possible to run checkstyle for a specific module
- * [HIVE-15758] - Allow correlated scalar subqueries with aggregates which has non-equi join predicates
- * [HIVE-15834] - Add unit tests for org.json usage on master
- * [HIVE-15899] - Make CTAS with acid target table and insert into acid_tbl select ... union all ... work
- * [HIVE-15939] - Make cast expressions comply more to sql2011
- * [HIVE-15982] - Support the width_bucket function
- * [HIVE-15986] - Support "is [not] distinct from"
- * [HIVE-16171] - Support replication of truncate table
- * [HIVE-16186] - REPL DUMP shows last event ID of the database even if we use LIMIT option.
- * [HIVE-16197] - Incremental insert into a partitioned table doesn't get replicated.
- * [HIVE-16207] - Add support for Complex Types in Fast SerDe
- * [HIVE-16228] - Support subqueries in complex expression in SELECT clause
- * [HIVE-16256] - Flaky test: TestCliDriver.testCliDriver[comments]
- * [HIVE-16266] - Enable function metadata to be written during bootstrap
- * [HIVE-16267] - Enable bootstrap function metadata to be loaded in repl load
- * [HIVE-16268] - enable incremental repl dump to handle functions metadata
- * [HIVE-16269] - enable incremental function dump to be loaded via repl load
- * [HIVE-16272] - support for drop function in incremental replication
- * [HIVE-16276] - Fix NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
- * [HIVE-16294] - Support snapshot for truncate table
- * [HIVE-16312] - Flaky test: TestHCatClient.testTransportFailure
- * [HIVE-16313] - Flaky test: TestBeeLineDriver[drop_with_concurrency]
- * [HIVE-16320] - Flaky test: TestBeeLineDriver.testCliDriver[escape_comments]
- * [HIVE-16330] - Improve plans for scalar subquery with aggregates
- * [HIVE-16344] - Test and support replication of exchange partition
- * [HIVE-16372] - Enable DDL statement for non-native tables (add/remove table properties)
- * [HIVE-16400] - Fix the MDC reference to use slf4j rather than log4j
- * [HIVE-16416] - Service: move constants out from HiveAuthFactory
- * [HIVE-16467] - Flaky test: TestCliDriver.testCliDriver[vector_order_null]
- * [HIVE-16488] - Support replicating into existing db if the db is empty
- * [HIVE-16493] - Skip column stats when colStats is empty
- * [HIVE-16504] - Addition of binary licenses broke rat check
- * [HIVE-16530] - Add HS2 operation logs and improve logs for REPL commands
- * [HIVE-16532] - HIVE on hadoop 3 build failed due to hdfs client/server jar separation
- * [HIVE-16535] - Hive fails to build from source code tarball
- * [HIVE-16542] - make merge that targets acid 2.0 table fail-fast
- * [HIVE-16555] - Add a new thrift API call for get_metastore_uuid
- * [HIVE-16556] - Modify schematool scripts to initialize and create METASTORE_DB_PROPERTIES table
- * [HIVE-16566] - Set column stats default as true when creating new tables/partitions
- * [HIVE-16568] - Support complex types in external LLAP InputFormat
- * [HIVE-16579] - CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table
- * [HIVE-16586] - Fix Unit test failures when CachedStore is enabled
- * [HIVE-16591] - DR for function Binaries on HDFS
- * [HIVE-16600] - Refactor SetSparkReducerParallelism#needSetParallelism to enable parallel order by in multi_insert cases
- * [HIVE-16601] - Display Session Id and Query Name / Id in Spark UI
- * [HIVE-16617] - Clean up javadoc from errors in module hive-shims
- * [HIVE-16618] - Clean up javadoc from errors in module hive-common
- * [HIVE-16619] - Clean up javadoc from errors in module hive-serde
- * [HIVE-16628] - Fix query25 when it uses a mix of MergeJoin and MapJoin
- * [HIVE-16637] - Improve end-of-data checking for LLAP input format
- * [HIVE-16642] - New Events created as part of replv2 potentially break replv1
- * [HIVE-16644] - Hook Change Manager to Insert Overwrite
- * [HIVE-16647] - Improve the validation output to make the output to stderr and stdout more consistent
- * [HIVE-16651] - LlapProtocolClientProxy stack trace when using llap input format
- * [HIVE-16652] - LlapInputFormat: Seeing "output error" WARN message
- * [HIVE-16653] - Mergejoin should give itself a correct tag
- * [HIVE-16672] - Parquet vectorization doesn't work for tables with partition info
- * [HIVE-16684] - Bootstrap REPL DUMP shouldn't fail when table is dropped after fetching the table names.
- * [HIVE-16686] - repl invocations of distcp needs additional handling
- * [HIVE-16688] - Make sure Alter Table to set transaction=true acquires X lock
- * [HIVE-16691] - Add test for more datatypes for LlapInputFormat
- * [HIVE-16697] - Schema table validator should return a sorted list of missing tables
- * [HIVE-16702] - Use LazyBinarySerDe for LLAP InputFormat
- * [HIVE-16706] - Bootstrap REPL DUMP shouldn't fail when a partition is dropped/renamed when dump in progress.
- * [HIVE-16714] - make Task Dependency on Repl Load more intuitive
- * [HIVE-16715] - Clean up javadoc from errors in modules llap-client, metastore, spark-client
- * [HIVE-16722] - Converting bucketed non-acid table to acid should perform validation
- * [HIVE-16727] - REPL DUMP for insert event should't fail if the table is already dropped.
- * [HIVE-16729] - Improve location validator to check for blank paths.
- * [HIVE-16747] - Remove YETUS*.sh files after a YETUS release
- * [HIVE-16748] - Integreate YETUS to Pre-Commit
- * [HIVE-16750] - Support change management for rename table/partition.
- * [HIVE-16764] - Support numeric as same as decimal
- * [HIVE-16765] - ParquetFileReader should be closed to avoid resource leak
- * [HIVE-16774] - Support position in ORDER BY when using SELECT *
- * [HIVE-16775] - Fix HiveFilterAggregateTransposeRule when filter is always false
- * [HIVE-16779] - CachedStore leak PersistenceManager resources
- * [HIVE-16782] - Flaky Test: TestMiniLlapLocalCliDriver[subquery_scalar]
- * [HIVE-16785] - Ensure replication actions are idempotent if any series of events are applied again.
- * [HIVE-16797] - Enhance HiveFilterSetOpTransposeRule to remove union branches
- * [HIVE-16813] - Incremental REPL LOAD should load the events in the same sequence as it is dumped.
- * [HIVE-16827] - Merge stats task and column stats task into a single task
- * [HIVE-16837] - MetadataOnly optimizer conflicts with count distinct rewrite
- * [HIVE-16838] - Improve plans for subqueries with non-equi co-related predicates
- * [HIVE-16848] - NPE during CachedStore refresh
- * [HIVE-16892] - Move creation of _files from ReplCopyTask to analysis phase for boostrap replication
- * [HIVE-16893] - move replication dump related work in semantic analysis phase to execution phase using a task
- * [HIVE-16895] - Multi-threaded execution of bootstrap dump of partitions
- * [HIVE-16896] - move replication load related work in semantic analysis phase to execution phase using a task
- * [HIVE-16901] - Distcp optimization - One distcp per ReplCopyTask
- * [HIVE-16912] - Improve table validator's performance against Oracle
- * [HIVE-16926] - LlapTaskUmbilicalExternalClient should not start new umbilical server for every fragment request
- * [HIVE-16974] - Change the sort key for the schema tool validator to be <ID>
- * [HIVE-16981] - hive.optimize.bucketingsorting should compare the schema before removing RS
- * [HIVE-16990] - REPL LOAD should update last repl ID only after successful copy of data files.
- * [HIVE-16992] - LLAP: monitoring and better default lambda for LRFU policy
- * [HIVE-16996] - Add HLL as an alternative to FM sketch to compute stats
- * [HIVE-16997] - Extend object store to store and use bit vectors
- * [HIVE-16998] - Add config to enable HoS DPP only for map-joins
- * [HIVE-17005] - Ensure REPL DUMP and REPL LOAD are authorized properly
- * [HIVE-17021] - Support replication of concatenate operation.
- * [HIVE-17087] - Remove unnecessary HoS DPP trees during map-join conversion
- * [HIVE-17091] - "Timed out getting readerEvents" error from external LLAP client
- * [HIVE-17100] - Improve HS2 operation logs for REPL commands.
- * [HIVE-17112] - Reduce logging in HiveSparkClientFactory and RemoteHiveSparkClient
- * [HIVE-17132] - Add InterfaceAudience and InterfaceStability annotations for UDF APIs
- * [HIVE-17137] - Fix javolution conflict
- * [HIVE-17153] - Flaky test: TestMiniSparkOnYarnCliDriver[spark_dynamic_partition_pruning]
- * [HIVE-17157] - Add InterfaceAudience and InterfaceStability annotations for ObjectInspector APIs
- * [HIVE-17167] - Create metastore specific configuration tool
- * [HIVE-17168] - Create separate module for stand alone metastore
- * [HIVE-17170] - Move thrift generated code to stand alone metastore
- * [HIVE-17178] - Spark Partition Pruning Sink Operator can't target multiple Works
- * [HIVE-17183] - Disable rename operations during bootstrap dump
- * [HIVE-17185] - TestHiveMetaStoreStatsMerge.testStatsMerge is failing
- * [HIVE-17195] - Long chain of tasks created by REPL LOAD shouldn't cause stack corruption.
- * [HIVE-17196] - CM: ReplCopyTask should retain the original file names even if copied from CM path.
- * [HIVE-17205] - add functional support for unbucketed tables
- * [HIVE-17212] - Dynamic add partition by insert shouldn't generate INSERT event.
- * [HIVE-17214] - check/fix conversion of unbucketed non-acid to acid
- * [HIVE-17215] - Streaming Ingest API writing unbucketed tables
- * [HIVE-17216] - Additional qtests for HoS DPP
- * [HIVE-17224] - Move JDO classes to standalone metastore
- * [HIVE-17225] - HoS DPP pruning sink ops can target parallel work objects
- * [HIVE-17241] - Change metastore classes to not use the shims
- * [HIVE-17247] - HoS DPP: UDFs on the partition column side does not evaluate correctly
- * [HIVE-17256] - add a notion of a guaranteed task to LLAP
- * [HIVE-17289] - EXPORT and IMPORT shouldn't perform distcp with doAs privileged user.
- * [HIVE-17292] - Change TestMiniSparkOnYarnCliDriver test configuration to use the configured cores
- * [HIVE-17297] - allow AM to use LLAP guaranteed tasks
- * [HIVE-17307] - Change the metastore to not use the metrics code in hive/common
- * [HIVE-17316] - Use String.startsWith for the hidden configuration variables
- * [HIVE-17318] - Make Hikari CP configurable using hive properties in hive-site.xml
- * [HIVE-17319] - Make BoneCp configurable using hive properties in hive-site.xml
- * [HIVE-17330] - refactor TezSessionPoolManager to separate its multiple functions
- * [HIVE-17346] - TestMiniSparkOnYarnCliDriver[spark_dynamic_partition_pruning] is failing every time
- * [HIVE-17347] - TestMiniSparkOnYarnCliDriver[spark_dynamic_partition_pruning_mapjoin_only] is failing every time
- * [HIVE-17359] - Deal with TypeInfo dependencies in the metastore
- * [HIVE-17371] - Move tokenstores to metastore module
- * [HIVE-17375] - stddev_samp,var_samp standard compliance
- * [HIVE-17380] - refactor LlapProtocolClientProxy to be usable with other protocols
- * [HIVE-17381] - When we enable Parquet Writer Version V2, hive throws an exception: Unsupported encoding: DELTA_BYTE_ARRAY.
- * [HIVE-17382] - Change startsWith relation introduced in HIVE-17316
- * [HIVE-17387] - implement Tez AM registry in Hive
- * [HIVE-17405] - HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
- * [HIVE-17409] - refactor LLAP ZK registry to make the ZK-registry part reusable
- * [HIVE-17414] - HoS DPP + Vectorization generates invalid explain plan due to CombineEquivalentWorkResolver
- * [HIVE-17428] - REPL LOAD of ALTER_PARTITION event doesn't create import tasks if the partition doesn't exist during analyze phase.
- * [HIVE-17455] - External LLAP client: connection to HS2 should be kept open until explicitly closed
- * [HIVE-17456] - Set current database for external LLAP interface
- * [HIVE-17473] - implement workload management pools
- * [HIVE-17482] - External LLAP client: acquire locks for tables queried directly by LLAP
- * [HIVE-17488] - Move first set of classes to standalone metastore
- * [HIVE-17494] - Bootstrap REPL DUMP throws exception if a partitioned table is dropped while reading partitions.
- * [HIVE-17495] - CachedStore: prewarm improvement (avoid multiple sql calls to read partition column stats), refactoring and caching some aggregate stats
- * [HIVE-17506] - Fix standalone-metastore pom.xml to not depend on hive's main pom
- * [HIVE-17508] - Implement global execution triggers based on counters
- * [HIVE-17514] - Use SHA-256 for cookie signer to improve security
- * [HIVE-17515] - Use SHA-256 for GenericUDFMaskHash to improve security
- * [HIVE-17527] - Support replication for rename/move table across database
- * [HIVE-17528] - Add more q-tests for Hive-on-Spark with Parquet vectorized reader
- * [HIVE-17534] - Add a config to turn off parquet vectorization
- * [HIVE-17537] - Move Warehouse class to standalone metastore
- * [HIVE-17541] - Move testing related methods from MetaStoreUtils to some testing related utility
- * [HIVE-17566] - Create schema required for workload management.
- * [HIVE-17581] - Replace some calcite dependencies with native ones
- * [HIVE-17607] - remove ColumnStatsDesc usage from columnstatsupdatetask
- * [HIVE-17608] - REPL LOAD should overwrite the data files if exists instead of duplicating it
- * [HIVE-17617] - Rollup of an empty resultset should contain the grouping of the empty grouping set
- * [HIVE-17629] - CachedStore - wait for prewarm at use time, not init time
- * [HIVE-17645] - MM tables patch conflicts with HIVE-17482 (Spark/Acid integration)
- * [HIVE-17647] - DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
- * [HIVE-17651] - TableScanOperator might miss vectorization on flag
- * [HIVE-17652] - retire ANALYZE TABLE ... PARTIALSCAN
- * [HIVE-17661] - DBTxnManager.acquireLocks() - MM tables should use shared lock for Insert
- * [HIVE-17671] - TableScanDesc.isAcidTable is restricted to FullAcid tables
- * [HIVE-17681] - Need to log bootstrap dump progress state property to HS2 logs.
- * [HIVE-17692] - Block HCat on Acid tables
- * [HIVE-17696] - Vectorized reader does not seem to be pushing down projection columns in certain code paths
- * [HIVE-17698] - FileSinkDesk.getMergeInputDirName() uses stmtId=0
- * [HIVE-17708] - Upgrade surefire to 2.20.1
- * [HIVE-17728] - TestHCatClient should use hive.metastore.transactional.event.listeners as per recommendation.
- * [HIVE-17733] - Move RawStore to standalone metastore
- * [HIVE-17743] - Add InterfaceAudience and InterfaceStability annotations for Thrift generated APIs
- * [HIVE-17748] - ReplCopyTask doesn't support multi-file CopyWork
- * [HIVE-17750] - add a flag to automatically create most tables as MM
- * [HIVE-17756] - Enable subquery related Qtests for Hive on Spark
- * [HIVE-17757] - REPL LOAD need to use customised configurations to execute distcp/remote copy.
- * [HIVE-17771] - Implement commands to manage resource plan
- * [HIVE-17778] - Add support for custom counters in trigger expression
- * [HIVE-17809] - Implement per pool trigger validation and move sessions across pools
- * [HIVE-17812] - Move remaining classes that HiveMetaStore depends on
- * [HIVE-17835] - HS2 Logs print unnecessary stack trace when HoS query is cancelled
- * [HIVE-17837] - Explicitly check if the HoS Remote Driver has been lost in the RemoteSparkJobMonitor
- * [HIVE-17841] - implement applying the resource plan
- * [HIVE-17842] - Run checkstyle on ptest2 module with proper configuration
- * [HIVE-17850] - can VectorizedOrcAcidRowReader be removed once HIVE-17458 is done?
- * [HIVE-17856] - MM tables - IOW is not ACID compliant
- * [HIVE-17858] - MM - some union cases are broken
- * [HIVE-17874] - Parquet vectorization fails on tables with complex columns when there are no projected columns
- * [HIVE-17884] - Implement create, alter and drop workload management triggers
- * [HIVE-17887] - Incremental REPL LOAD with Drop partition event on timestamp type partition column fails.
- * [HIVE-17888] - Display the reason for query cancellation
- * [HIVE-17897] - "repl load" in bootstrap phase fails when partitions have whitespace
- * [HIVE-17902] - add notions of default pool and start adding unmanaged mapping
- * [HIVE-17904] - handle internal Tez AM restart in registry and WM
- * [HIVE-17905] - propagate background LLAP cluster changes to WM
- * [HIVE-17906] - use kill query mechanics to kill queries in WM
- * [HIVE-17907] - enable and apply resource plan commands in HS2
- * [HIVE-17913] - Cleanup unused methods in Driver
- * [HIVE-17926] - Support triggers for non-pool sessions
- * [HIVE-17929] - Use sessionId for HoS Remote Driver Client id
- * [HIVE-17931] - Implement Parquet vectorization reader for Array type
- * [HIVE-17933] - make antlr output directory to use a top-level sourceset
- * [HIVE-17934] - Merging Statistics are promoted to COMPLETE (most of the time)
- * [HIVE-17945] - Support column projection for index access when using Parquet Vectorization
- * [HIVE-17950] - Implement resource plan fetching from metastore
- * [HIVE-17954] - Implement pool, user, group and trigger to pool management API's.
- * [HIVE-17961] - NPE during initialization of VectorizedParquetRecordReader when input split is null
- * [HIVE-17967] - Move HiveMetaStore class
- * [HIVE-17970] - MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
- * [HIVE-17972] - Implement Parquet vectorization reader for Map type
- * [HIVE-17980] - Move HiveMetaStoreClient plus a few remaining classes.
- * [HIVE-17981] - Create a set of builders for Thrift classes
- * [HIVE-17982] - Move metastore specific itests
- * [HIVE-17983] - Make the standalone metastore generate tarballs etc.
- * [HIVE-17990] - Add Thrift and DB storage for Schema Registry objects
- * [HIVE-17991] - Remove CommandNeedRetryException
- * [HIVE-17995] - Run checkstyle on standalone-metastore module with proper configuration
- * [HIVE-17996] - Fix ASF headers
- * [HIVE-17997] - Add rat plugin and configuration to standalone metastore pom
- * [HIVE-18002] - add group support for pool mappings
- * [HIVE-18003] - add explicit jdbc connection string args for mappings
- * [HIVE-18004] - investigate deriving app name from JDBC connection for pool mapping
- * [HIVE-18005] - Improve size estimation for array() to be not 0
- * [HIVE-18025] - Push resource plan changes to tez/unmanaged sessions
- * [HIVE-18028] - fix WM based on cluster smoke test; add logging
- * [HIVE-18029] - beeline - support proper usernames based on the URL arg
- * [HIVE-18031] - Support replication for Alter Database operation.
- * [HIVE-18034] - Improving logging with HoS executors spend lots of time in GC
- * [HIVE-18036] - Stats: Remove usage of clone() methods
- * [HIVE-18053] - Support different table types for MVs
- * [HIVE-18056] - CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
- * [HIVE-18057] - remove PostExecute / PreExecute hook support
- * [HIVE-18063] - Make CommandProcessorResponse an exception instead of a return class
- * [HIVE-18071] - add HS2 jmx information about pools and current resource plan
- * [HIVE-18072] - fix various WM bugs based on cluster testing - part 2
- * [HIVE-18073] - AM may assert when its guaranteed task count is reduced
- * [HIVE-18075] - verify commands on a cluster
- * [HIVE-18076] - killquery doesn't actually work for non-trigger WM kills
- * [HIVE-18078] - WM getSession needs some retry logic
- * [HIVE-18084] - Upgrade checkstyle version to support lambdas
- * [HIVE-18085] - Run checkstyle on storage-api module with proper configuration
- * [HIVE-18088] - Add WM event traces at query level for debugging
- * [HIVE-18092] - Fix exception on tables handled by HBaseHandler if columnsstats are auto-gathered
- * [HIVE-18093] - Improve logging when HoS application is killed
- * [HIVE-18095] - add a unmanaged flag to triggers (applies to container based sessions)
- * [HIVE-18096] - add a user-friendly show plan command
- * [HIVE-18125] - Support arbitrary file names in input to Load Data
- * [HIVE-18133] - Parametrize TestTxnNoBuckets wrt Vectorization
- * [HIVE-18134] - some alter resource plan fixes
- * [HIVE-18138] - Fix columnstats problem in case schema evolution
- * [HIVE-18141] - Fix StatsUtils.combineRange to combine intervals
- * [HIVE-18149] - Stats: rownum estimation from datasize underestimates in most cases
- * [HIVE-18153] - refactor reopen and file management in TezTask
- * [HIVE-18161] - Remove hive.stats.atomic
- * [HIVE-18163] - Stats: create materialized view should also collect stats
- * [HIVE-18170] - User mapping not initialized correctly on start
- * [HIVE-18179] - Implement validate resource plan (part 1)
- * [HIVE-18187] - Add jamon generated-sources as source folder
- * [HIVE-18190] - Consider looking at ORC file schema rather than using _metadata_acid file
- * [HIVE-18192] - Introduce WriteID per table rather than using global transaction ID
- * [HIVE-18193] - Migrate existing ACID tables to use write id per table rather than global transaction id
- * [HIVE-18202] - Automatically migrate hbase.table.name to hbase.mapreduce.hfileoutputformat.table.name for hbase-based table
- * [HIVE-18203] - change the way WM is enabled and allow dropping the last resource plan
- * [HIVE-18209] - Fix API call in VectorizedListColumnReader to get value from BytesColumnVector
- * [HIVE-18211] - Support to read multiple level definition for Map type in Parquet file
- * [HIVE-18212] - Make sure Yetus check always has a full log
- * [HIVE-18214] - Flaky test: TestSparkClient
- * [HIVE-18222] - Update checkstyle rules to be less peeky
- * [HIVE-18224] - Introduce interface above driver
- * [HIVE-18229] - add the unmanaged mapping command
- * [HIVE-18230] - create plan like plan, and replace plan commands for easy modification
- * [HIVE-18235] - Columnstats gather on mm tables: re-enable disabled test
- * [HIVE-18237] - missing results for insert_only table after DP insert
- * [HIVE-18238] - Driver execution may not have configuration changing sideeffects
- * [HIVE-18245] - clean up acid_vectorization_original.q
- * [HIVE-18257] - implement scheduling policy configuration instead of hardcoding fair scheduling
- * [HIVE-18273] - add LLAP-level counters for WM
- * [HIVE-18274] - add AM level metrics for WM
- * [HIVE-18275] - add HS2-level WM metrics
- * [HIVE-18286] - java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.DoubleColumnVector cannot be cast to org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
- * [HIVE-18288] - merge/concat not supported on Acid table
- * [HIVE-18294] - add switch to make acid table the default
- * [HIVE-18315] - update tests use non-acid tables
- * [HIVE-18317] - Improve error messages in TransactionalValidationListerner
- * [HIVE-18323] - Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet
- * [HIVE-18366] - Update HBaseSerDe to use hbase.mapreduce.hfileoutputformat.table.name instead of hbase.table.name as the table name property
- * [HIVE-18368] - Improve Spark Debug RDD Graph
- * [HIVE-18372] - Create testing infra to test different HMS instances
- * [HIVE-18389] - Print out Spark Web UI URL to the console log
- * [HIVE-18411] - Fix ArrayIndexOutOfBoundsException for VectorizedListColumnReader
- * [HIVE-18418] - clean up plugin between DAGs
- * [HIVE-18437] - use plan parallelism for the default pool if both are present
- * [HIVE-18438] - WM RP: it's impossible to unset things
- * [HIVE-18443] - Ensure git gc finished in ptest prep phase before copying repo
- * [HIVE-18457] - improve show plan output (triggers, mappings)
- * [HIVE-18458] - Workload manager initializes even when interactive queue is not set
- * [HIVE-18468] - Create tests to cover alterPartition and renamePartition methods
- * [HIVE-18478] - Data files deleted from temp table should not be recycled to CM path
- * [HIVE-18479] - Create tests to cover dropPartition methods
- * [HIVE-18480] - Create tests for function related methods
- * [HIVE-18481] - Create tests for table related methods (get, list, exists)
- * [HIVE-18483] - Create tests to cover getPartition(s) methods
- * [HIVE-18484] - Create tests to cover listPartition(s) methods
- * [HIVE-18486] - Create tests to cover add partition methods
- * [HIVE-18489] - Automatically migrate s3n URIs to s3a URIs
- * [HIVE-18495] - JUnit rule to enable Driver level testing
- * [HIVE-18496] - Create tests to cover add/alter/drop index methods
- * [HIVE-18498] - Create tests to cover get and list index methods
- * [HIVE-18509] - Create tests for table manipulation related methods (create, alter, drop)
- * [HIVE-18511] - Fix generated checkstyle errors
- * [HIVE-18536] - IOW + DP is broken for insert-only ACID
- * [HIVE-18541] - Secure HS2 web UI with PAM
- * [HIVE-18542] - Create tests to cover getTableMeta method
- * [HIVE-18544] - Create tests to cover appendPartition methods
- * [HIVE-18550] - Keep the hbase table name property as hbase.table.name
- * [HIVE-18553] - Support schema evolution in Parquet Vectorization reader
- * [HIVE-18566] - Create tests to cover adding partitions from PartitionSpec
- * [HIVE-18580] - Create tests to cover exchange partitions
- * [HIVE-18596] - Synchronize value of hive.spark.client.connect.timeout across unit tests
- * [HIVE-18609] - Results cache invalidation based on ACID table updates
- * [HIVE-18633] - Service discovery for Active/Passive HA mode
- * [HIVE-18635] - Generalize hook dispatch logics in Driver
- * [HIVE-18651] - Expose additional Spark metrics
- * [HIVE-18663] - Logged Spark Job Id contains a UUID instead of the actual id
- * [HIVE-18672] - Printed state in RemoteSparkJobMonitor is ambiguous
- * [HIVE-18673] - ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
- * [HIVE-18677] - SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
- * [HIVE-18679] - create/replicate open transaction event
- * [HIVE-18703] - Make Operator comparision to be based on some primitive
- * [HIVE-18715] - Remove index support from metastore
- * [HIVE-18720] - Replicate Commit Txn operation (without writes)
- * [HIVE-18745] - Fix MetaStore creation in tests, so multiple MetaStores can be started on the same machine
- * [HIVE-18747] - Cleaner for TXN_TO_WRITE_ID table entries using MIN_HISTORY_LEVEL.
- * [HIVE-18749] - Need to replace transactionId with writeId in RecordIdentifier and other relevant contexts.
- * [HIVE-18750] - Exchange partition should be disabled on ACID/Insert-only tables with per table write ID.
- * [HIVE-18751] - ACID table scan through get_splits UDF doesn't receive ValidWriteIdList configuration.
- * [HIVE-18753] - Correct methods and variables names which uses writeId instead of transactionId.
- * [HIVE-18755] - Modifications to the metastore for catalogs
- * [HIVE-18765] - SparkClientImpl swallows exception messages from the RemoteDriver
- * [HIVE-18771] - Refactor tests, so only 1 MetaStore instance will be started per test class and test configuration
- * [HIVE-18781] - Create/Replicate Open, Commit (without writes) and Abort Txn events
- * [HIVE-18805] - Add ConstantPropagate before stats annotation
- * [HIVE-18824] - ValidWriteIdList config should be defined on tables which has to collect stats after insert
- * [HIVE-18830] - RemoteSparkJobMonitor failures are logged twice
- * [HIVE-18832] - Support change management for trashing data files from ACID tables.
- * [HIVE-18840] - CachedStore: Prioritize loading of recently accessed tables during prewarm
- * [HIVE-18846] - Query results cache: Allow queries to refer to the pending results of a query that has not finished yet
- * [HIVE-18855] - Fix unit test TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
- * [HIVE-18861] - druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating classpath problems on hadoop 3.x
- * [HIVE-18864] - ValidWriteIdList snapshot seems incorrect if obtained after allocating writeId by current transaction.
- * [HIVE-18899] - Separate FetchWork required for each query that uses the results cache
- * [HIVE-18909] - Metrics for results cache
- * [HIVE-18926] - Imporve operator-tree matching
- * [HIVE-18946] - Fix columnstats merge NPE
- * [HIVE-18961] - Error in results cache when query has identifiers with spaces
- * [HIVE-18982] - Provide a CLI option to manually trigger failover
- * [HIVE-18988] - Support bootstrap replication of ACID tables
- * [HIVE-18994] - Handle client connections on failover
- * [HIVE-19009] - Retain and use runtime statistics during hs2 lifetime
- * [HIVE-19031] - Mark duplicate configs in HiveConf as deprecated
- * [HIVE-19083] - Make partition clause optional for INSERT
- * [HIVE-19089] - Create/Replicate Allocate write-id event
- * [HIVE-19112] - Support Analyze table for partitioned tables without partition spec
- * [HIVE-19126] - CachedStore: Use memory estimation to limit cache size during prewarm
- * [HIVE-19127] - Concurrency fixes in QueryResultsCache
- * [HIVE-19128] - Update golden files for spark perf tests
- * [HIVE-19129] - Support DEFAULT keyword with MERGE
- * [HIVE-19135] - Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
- * [HIVE-19138] - Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails
- * [HIVE-19141] - TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
- * [HIVE-19144] - TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
- * [HIVE-19145] - Stabilize statsoptimizer.q test
- * [HIVE-19146] - Delete dangling q.out
- * [HIVE-19147] - Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change
- * [HIVE-19153] - Update golden files for few tests
- * [HIVE-19154] - Poll notification events to invalidate the results cache
- * [HIVE-19156] - TestMiniLlapLocalCliDriver.vectorized_dynamic_semijoin_reduction.q is broken
- * [HIVE-19159] - TestMTQueries.testMTQueries1 failure
- * [HIVE-19164] - TestMetastoreVersion failures
- * [HIVE-19171] - Persist runtime statistics in metastore
- * [HIVE-19175] - TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
- * [HIVE-19178] - TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
- * [HIVE-19193] - TestActivePassiveHA fails
- * [HIVE-19194] - TestDruidStorageHandler fails
- * [HIVE-19195] - Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.
- * [HIVE-19196] - TestTriggersMoveWorkloadManager is flaky
- * [HIVE-19197] - TestReplicationScenarios is flaky
- * [HIVE-19206] - Automatic memory management for open streaming writers
- * [HIVE-19209] - Streaming ingest record writers should accept input stream
- * [HIVE-19210] - Create separate module for streaming ingest
- * [HIVE-19211] - New streaming ingest API and support for dynamic partitioning
- * [HIVE-19214] - High throughput ingest ORC format
- * [HIVE-19222] - TestNegativeCliDriver tests are failing due to "java.lang.OutOfMemoryError: GC overhead limit exceeded"
- * [HIVE-19232] - results_cache_invalidation2 is failing
- * [HIVE-19274] - Add an OpTreeSignature persistence checker hook
- * [HIVE-19319] - RuntimeStats fixes
- * [HIVE-19322] - broken test: TestNegativeMinimrCliDriver#testCliDriver[minimr_broken_pipe]
- * [HIVE-19335] - Disable runtime filtering (semijoin reduction opt with bloomfilter) for external tables
- * [HIVE-19346] - TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5] failling
- * [HIVE-19347] - TestTriggersWorkloadManager tests are failing consistently
- * [HIVE-19348] - org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
- * [HIVE-19371] - Add table ownerType to HMS thrift API
- * [HIVE-19372] - Add table ownerType to JDO/SQL and ObjectStore
- * [HIVE-19374] - Parse and process ALTER TABLE SET OWNER command syntax
- * [HIVE-19400] - Adjust Hive 1.0 to 2.0 conversion utility to the upgrade
- * [HIVE-19471] - bucket_map_join_tez1 and bucket_map_join_tez2 are failing
- * [HIVE-19472] - HiveStreamingConnection swallows exception on partition creation
- * [HIVE-19494] - Accept shade prefix during reflective instantiation of output format
-
-
-** Bug
- * [HIVE-4577] - hive CLI can't handle hadoop dfs command with space and quotes.
- * [HIVE-6348] - Order by/Sort by in subquery
- * [HIVE-6590] - Hive does not work properly with boolean partition columns (wrong results and inserts to incorrect HDFS path)
- * [HIVE-6990] - Direct SQL fails when the explicit schema setting is different from the default one
- * [HIVE-8937] - fix description of hive.security.authorization.sqlstd.confwhitelist.* params
- * [HIVE-9815] - Metastore column"SERDE_PARAMS"."PARAM_VALUE" limited to 4000 bytes
- * [HIVE-10616] - TypeInfoUtils doesn't handle DECIMAL with just precision specified
- * [HIVE-10865] - Beeline needs to support DELIMITER command
- * [HIVE-11064] - ALTER TABLE CASCADE ERROR unbalanced calls to openTransaction/commitTransaction
- * [HIVE-11266] - count(*) wrong result based on table statistics for external tables
- * [HIVE-11297] - Combine op trees for partition info generating tasks
- * [HIVE-11609] - Capability to add a filter to hbase scan via composite key doesn't work
- * [HIVE-12408] - SQLStdAuthorizer should not require external table creator to be owner of directory, in addition to rw permissions
- * [HIVE-12425] - OrcRecordUpdater.close(true) leaves the file open
- * [HIVE-12631] - LLAP IO: support ORC ACID tables
- * [HIVE-12719] - As a hive user, I am facing issues using permanent UDAF's.
- * [HIVE-12734] - Remove redundancy in HiveConfs serialized to UDFContext
- * [HIVE-13000] - Hive returns useless parsing error
- * [HIVE-13652] - Import table change order of dynamic partitions
- * [HIVE-14032] - INSERT OVERWRITE command failed with case sensitive partition key names
- * [HIVE-14052] - Cleanup structures when external clients use LLAP
- * [HIVE-14077] - add implicit decimal arithmetic q test, fix issues if found
- * [HIVE-14455] - upgrade httpclient, httpcore to match updated hadoop dependency
- * [HIVE-14560] - Support exchange partition between s3 and hdfs tables
- * [HIVE-14564] - Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsException.
- * [HIVE-14678] - Hive-on-MR deprecation warning is not diplayed when engine is set to capital letter 'MR'
- * [HIVE-14731] - Use Tez cartesian product edge in Hive (unpartitioned case only)
- * [HIVE-14792] - AvroSerde reads the remote schema-file at least once per mapper, per table reference.
- * [HIVE-14813] - Make TransactionBatchImpl.toString() include state of each txn: commit/abort
- * [HIVE-14988] - Support INSERT OVERWRITE into a partition on transactional tables
- * [HIVE-15077] - Acid LockManager is unfair
- * [HIVE-15104] - Hive on Spark generate more shuffle data than hive on mr
- * [HIVE-15144] - JSON.org license is now CatX
- * [HIVE-15160] - Can't order by an unselected column
- * [HIVE-15176] - Small typo in hiveserver2 webui
- * [HIVE-15249] - HIve 2.1.0 is throwing InvalidObjectException(message:Invalid column type name is too long
- * [HIVE-15267] - Make query length calculation logic more accurate in TxnUtils.needNewQuery()
- * [HIVE-15343] - Spelling errors in logging and exceptions for beeline, common, hbase-handler, hcatalog, llap-server, orc, serde and shims
- * [HIVE-15344] - Spelling errors in logging and exceptions for metastore and service directories
- * [HIVE-15442] - Driver.java has a redundancy code
- * [HIVE-15483] - Database and table name is case sensitive when used in show grant
- * [HIVE-15504] - ArrayIndexOutOfBoundsException in GenericUDFTrunc::initialize
- * [HIVE-15515] - Remove the docs directory
- * [HIVE-15552] - Unable to coalesce DATE and TIMESTAMP types
- * [HIVE-15630] - add operation handle before operation.run instead of after operation.run
- * [HIVE-15632] - Hive/Druid integration: Incorrect result - Limit on timestamp disappears
- * [HIVE-15635] - Hive/Druid integration: timeseries query shows all days, even if no data
- * [HIVE-15636] - Hive/Druid integration: wrong semantics of topN query limit with granularity
- * [HIVE-15637] - Hive/Druid integration: wrong semantics of groupBy query limit with granularity
- * [HIVE-15639] - Hive/Druid integration: wrong semantics for ordering within groupBy queries
- * [HIVE-15680] - Incorrect results when hive.optimize.index.filter=true and same ORC table is referenced twice in query
- * [HIVE-15724] - getPrimaryKeys and getForeignKeys in metastore does not normalize db and table name
- * [HIVE-15739] - Incorrect exception message in PartExprEvalUtils
- * [HIVE-15761] - ObjectStore.getNextNotification could return an empty NotificationEventResponse causing TProtocolException
- * [HIVE-15767] - Hive On Spark is not working on secure clusters from Oozie
- * [HIVE-15829] - LLAP text cache: disable memory tracking on the writer
- * [HIVE-15883] - HBase mapped table in Hive insert fail for decimal
- * [HIVE-15995] - Syncing metastore table with serde schema
- * [HIVE-16007] - When the query does not complie the LogRunnable never stops
- * [HIVE-16025] - Where IN clause throws exception
- * [HIVE-16026] - Generated query will timeout and/or kill the druid cluster.
- * [HIVE-16027] - <timestamp> BETWEEN <string> AND <string> must cast to TIMESTMAP
- * [HIVE-16044] - LLAP: Shuffle Handler keep-alive connections are closed from the server side
- * [HIVE-16053] - Remove newRatio from llap JAVA_OPTS_BASE
- * [HIVE-16057] - SchemaTool ignores --passWord argument if hadoop.security.credential.provider.path is configured
- * [HIVE-16061] - When hive.async.log.enabled is set to true, some output is not printed to the beeline console
- * [HIVE-16077] - UPDATE/DELETE fails with numBuckets > numReducers
- * [HIVE-16113] - PartitionPruner::removeNonPartCols needs to handle AND/OR cases
- * [HIVE-16117] - SortProjectTransposeRule should check for monotonicity preserving CAST
- * [HIVE-16125] - Split work between reducers.
- * [HIVE-16130] - Remove jackson classes from hive-jdbc standalone jar
- * [HIVE-16147] - Rename a partitioned table should not drop its partition columns stats
- * [HIVE-16174] - Update MetricsConstant.WAITING_COMPILE_OPS metric when we aquire lock failed in Driver
- * [HIVE-16177] - non Acid to acid conversion doesn't handle _copy_N files
- * [HIVE-16188] - beeline should block the connection if given invalid database name.
- * [HIVE-16193] - Hive show compactions not reflecting the status of the application
- * [HIVE-16213] - ObjectStore can leak Queries when rollbackTransaction throws an exception
- * [HIVE-16219] - metastore notification_log contains serialized message with non functional fields
- * [HIVE-16222] - add a setting to disable row.serde for specific formats; enable for others
- * [HIVE-16225] - Memory leak in webhcat service (FileSystem CACHE entries)
- * [HIVE-16233] - llap: Query failed with AllocatorOutOfMemoryException
- * [HIVE-16254] - metadata for values temporary tables for INSERTs are getting replicated during bootstrap
- * [HIVE-16275] - Vectorization: Add ReduceSink support for TopN (in specialized native classes)
- * [HIVE-16282] - Semijoin: Disable slow-start for the bloom filter aggregate task
- * [HIVE-16287] - Alter table partition rename with location - moves partition back to hive warehouse
- * [HIVE-16290] - Stats: StatsRulesProcFactory::evaluateComparator estimates are wrong when minValue == filterValue
- * [HIVE-16291] - Hive fails when unions a parquet table with itself
- * [HIVE-16296] - use LLAP executor count to configure reducer auto-parallelism
- * [HIVE-16298] - Add config to specify multi-column joins have correlated columns
- * [HIVE-16299] - MSCK REPAIR TABLE should enforce partition key order when adding unknown partitions
- * [HIVE-16302] - Add junit dependency to hive-shims-common to compile with Hadoop 2.8+
- * [HIVE-16305] - Additional Datanucleus ClassLoaderResolverImpl leaks causing HS2 OOM
- * [HIVE-16307] - add IO memory usage report to LLAP UI
- * [HIVE-16308] - PreExecutePrinter and PostExecutePrinter should log to INFO level instead of ERROR
- * [HIVE-16309] - Hive Test Commands failure should be printed in hive.log in addition to stderr
- * [HIVE-16315] - Describe table doesn't show num of partitions
- * [HIVE-16316] - Prepare master branch for 3.0.0 development.
- * [HIVE-16317] - CASE .. NULL in JOIN condition can trigger SemanticException
- * [HIVE-16318] - LLAP cache: address some issues in 2.2/2.3
- * [HIVE-16319] - LLAP: Better handling of an empty wait queue, should try scheduling checks
- * [HIVE-16321] - Possible deadlock in metastore with Acid enabled
- * [HIVE-16323] - HS2 JDOPersistenceManagerFactory.pmCache leaks after HIVE-14204
- * [HIVE-16324] - Truncate table should not work when EXTERNAL property of table is true
- * [HIVE-16325] - Tez session refresh based on a time interval fails
- * [HIVE-16328] - HoS: more aggressive mapjoin optimization when hive.spark.use.ts.stats.for.mapjoin is true
- * [HIVE-16329] - TopN: use local executor info for LLAP memory checks
- * [HIVE-16333] - remove the redundant symbol "\" to appear red in sublime text 3
- * [HIVE-16335] - Beeline user HS2 connection file should use /etc/hive/conf instead of /etc/conf/hive
- * [HIVE-16336] - Rename hive.spark.use.file.size.for.mapjoin to hive.spark.use.ts.stats.for.mapjoin
- * [HIVE-16341] - Tez Task Execution Summary has incorrect input record counts on some operators
- * [HIVE-16347] - HiveMetastoreChecker should skip listing partitions which are not valid when hive.msck.path.validation is set to skip or ignore
- * [HIVE-16353] - Jetty 9 upgrade breaks hive master LLAP
- * [HIVE-16357] - Failed folder creation when creating a new table is reported incorrectly
- * [HIVE-16363] - QueryLifeTimeHooks should catch parse exceptions
- * [HIVE-16368] - Unexpected java.lang.ArrayIndexOutOfBoundsException from query with LaterView Operation for hive on MR.
- * [HIVE-16369] - Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
- * [HIVE-16380] - removing global test dependency of jsonassert
- * [HIVE-16384] - Remove jdk7 build from travis
- * [HIVE-16385] - StatsNoJobTask could exit early before all partitions have been processed
- * [HIVE-16388] - LLAP: Log rotation for daemon, history and gc files
- * [HIVE-16389] - Allow HookContext to access SQLOperationDisplay
- * [HIVE-16390] - LLAP IO should take job config into account; also LLAP config should load defaults
- * [HIVE-16393] - Fix visibility of CodahaleReporter interface
- * [HIVE-16394] - HoS does not support queue name change in middle of session
- * [HIVE-16396] - Sync storage-api version in pom.xml
- * [HIVE-16399] - create an index for tc_txnid in TXN_COMPONENTS
- * [HIVE-16402] - Upgrade to Hadoop 2.8.0
- * [HIVE-16403] - LLAP UI shows the wrong number of executors
- * [HIVE-16404] - Renaming of public classes in Calcite 12 breeaking druid integration
- * [HIVE-16406] - Remove unwanted interning when creating PartitionDesc
- * [HIVE-16409] - TestEventHandlerFactory has lacked the ASF header
- * [HIVE-16413] - Create table as select does not check ownership of the location
- * [HIVE-16421] - Runtime filtering breaks user-level explain
- * [HIVE-16422] - Should kill running Spark Jobs when a query is cancelled.
- * [HIVE-16425] - Vectorization: unload old hashtables before reloadHashTable
- * [HIVE-16427] - Fix multi-insert query and write qtests
- * [HIVE-16433] - Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
- * [HIVE-16436] - Response times in "Task Execution Summary" at the end of the job is not correct
- * [HIVE-16448] - Vectorization: Vectorized order_null.q fails with deserialize EOF exception below TEZ ReduceRecordSource.processVectorGroup
- * [HIVE-16450] - Some metastore operations are not retried even with desired underlining exceptions
- * [HIVE-16451] - Race condition between HiveStatement.getQueryLog and HiveStatement.runAsyncOnServer
- * [HIVE-16459] - Forward channelInactive to RpcDispatcher
- * [HIVE-16461] - DagUtils checks local resource size on the remote fs
- * [HIVE-16462] - Vectorization: Enabling hybrid grace disables specialization of all reduce side joins
- * [HIVE-16465] - NullPointer Exception when enable vectorization for Parquet file format
- * [HIVE-16468] - BeeLineDriver should be able to run tests against an externally created cluster
- * [HIVE-16471] - Add metrics for "waiting compilation time"
- * [HIVE-16473] - Hive-on-Tez may fail to write to an HBase table
- * [HIVE-16482] - Druid Ser/Des need to use dimension output name
- * [HIVE-16483] - HoS should populate split related configurations to HiveConf
- * [HIVE-16485] - Enable outputName for RS operator in explain formatted
- * [HIVE-16487] - Serious Zookeeper exception is logged when a race condition happens
- * [HIVE-16491] - CBO cant handle join involving complex types in on condition
- * [HIVE-16494] - udaf percentile_approx() may fail on CBO
- * [HIVE-16497] - FileUtils. isActionPermittedForFileHierarchy, isOwnerOfFileHierarchy file system operations should be impersonated
- * [HIVE-16507] - Hive Explain User-Level may print out "Vertex dependency in root stage" twice
- * [HIVE-16510] - Vectorization: Add vectorized PTF tests in preparation for HIVE-16369
- * [HIVE-16511] - CBO looses inner casts on constants of complex type
- * [HIVE-16513] - width_bucket issues
- * [HIVE-16518] - Insert override for druid does not replace all existing segments
- * [HIVE-16519] - Fix exception thrown by checkOutputSpecs
- * [HIVE-16523] - VectorHashKeyWrapper hash code for strings is not so good
- * [HIVE-16524] - Remove the redundant item type in hiveserver2.jsp and QueryProfileTmpl.jamon
- * [HIVE-16533] - Vectorization: Avoid evaluating empty groupby keys
- * [HIVE-16534] - Add capability to tell aborted transactions apart from open transactions in ValidTxnList
- * [HIVE-16538] - TestExecDriver fails if run after TestOperators#testScriptOperator
- * [HIVE-16539] - Add PTF tests for blobstores
- * [HIVE-16545] - LLAP: bug in arena size determination logic
- * [HIVE-16546] - LLAP: Fail map join tasks if hash table memory exceeds threshold
- * [HIVE-16547] - LLAP: may not unlock buffers in some cases
- * [HIVE-16553] - Change default value for hive.tez.bigtable.minsize.semijoin.reduction
- * [HIVE-16554] - ACID: Make HouseKeeperService threads daemon
- * [HIVE-16557] - Vectorization: Specialize ReduceSink empty key case
- * [HIVE-16559] - Parquet schema evolution for partitioned tables may break if table and partition serdes differ
- * [HIVE-16562] - Issues with nullif / fetch task
- * [HIVE-16563] - Alter table partition set location should use fully qualified path for non-default FS
- * [HIVE-16572] - Rename a partition should not drop its column stats
- * [HIVE-16573] - In-place update for HoS can't be disabled
- * [HIVE-16576] - Fix encoding of intervals when fetching select query candidates from druid
- * [HIVE-16577] - Syntax error in the metastore init scripts for mssql
- * [HIVE-16578] - Semijoin Hints should use column name, if provided for partition key check
- * [HIVE-16581] - a bug in HIVE-16523
- * [HIVE-16584] - Warning messages should use LogHelper.printInfo instead of printing to the infoStream directly
- * [HIVE-16588] - Resource leak by druid http client
- * [HIVE-16589] - Vectorization: Support Complex Types and GroupBy modes PARTIAL2, FINAL, and COMPLETE for AVG, VARIANCE
- * [HIVE-16590] - Make initializing dag names in SparkWork thread safe for parallel compilation (HIVE-13512)
- * [HIVE-16592] - Vectorization: Long hashCodes should bit-mix into lower bits
- * [HIVE-16593] - SparkClientFactory.stop may prevent JVM from exiting
- * [HIVE-16598] - LlapServiceDriver - create directories and warn of errors
- * [HIVE-16599] - NPE in runtime filtering cost when handling SMB Joins
- * [HIVE-16603] - Enforce foreign keys to refer to primary keys or unique keys
- * [HIVE-16607] - ColumnStatsAutoGatherContext regenerates HiveConf.HIVEQUERYID
- * [HIVE-16609] - col='__HIVE_DEFAULT_PARTITION__' condition in select statement may produce wrong result
- * [HIVE-16610] - Semijoin Hint : Should be able to handle more than one hint per alias
- * [HIVE-16613] - SaslClientHandler.sendHello is eating exceptions
- * [HIVE-16625] - Extra '\0' characters in the output, when SeparatedValuesOutputFormat is used and the quoting is disabled
- * [HIVE-16633] - username for ATS data shall always be the uid who submit the job
- * [HIVE-16634] - LLAP Use a pool of connections to a single AM from a daemon
- * [HIVE-16640] - The ASF Headers have some errors in some class
- * [HIVE-16645] - Commands.java has missed the catch statement and has some code format errors
- * [HIVE-16646] - Alias in transform ... as clause shouldn't be case sensitive
- * [HIVE-16654] - Optimize a combination of avg(), sum(), count(distinct) etc
- * [HIVE-16658] - TestTimestampTZ.java has missed the ASF header
- * [HIVE-16659] - Query plan should reflect hive.spark.use.groupby.shuffle
- * [HIVE-16660] - Not able to add partition for views in hive when sentry is enabled
- * [HIVE-16665] - Race condition in Utilities.GetInputPathsCallable --> createDummyFileForEmptyPartition
- * [HIVE-16667] - PostgreSQL metastore handling of CLOB types for COLUMNS_V2.TYPE_NAME and other field is incorrect
- * [HIVE-16671] - LLAP IO: BufferUnderflowException may happen in very rare(?) cases due to ORC end-of-CB estimation
- * [HIVE-16675] - Fix ConcurrentModificationException in SparkClientImpl#startDriver
- * [HIVE-16677] - CTAS with no data fails in Druid
- * [HIVE-16678] - Truncate on temporary table fails with "table not found" error.
- * [HIVE-16679] - Missing ASF header on properties file in ptest2 project
- * [HIVE-16689] - Correlated scalar subquery with comparison to constant in predicate fails
- * [HIVE-16692] - LLAP: Keep alive connection in shuffle handler should not be closed until entire data is flushed out
- * [HIVE-16693] - beeline "source" command freezes if you have a comment in it?
- * [HIVE-16696] - Fix JoinCondDesc explain string
- * [HIVE-16698] - HoS should avoid mapjoin optimization in case of union and using table stats
- * [HIVE-16703] - Hive may add the same file to the session and vertex in Tez
- * [HIVE-16708] - Exception while renewing a Delegation Token
- * [HIVE-16721] - Inconsistent behavior in dealing with Timestamp stats
- * [HIVE-16724] - increase session timeout for LLAP ZK token manager
- * [HIVE-16730] - Vectorization: Schema Evolution for Text Vectorization / Complex Types
- * [HIVE-16731] - Vectorization: Make "CASE WHEN (day_name='Sunday') THEN column1 ELSE null end" that involves a column name or expression THEN or ELSE vectorize
- * [HIVE-16732] - Transactional tables should block LOAD DATA
- * [HIVE-16737] - LLAP: Shuffle handler TCP listen queue overflows
- * [HIVE-16738] - Notification ID generation in DBNotification might not be unique across HS2 instances.
- * [HIVE-16742] - cap the number of reducers for LLAP at the configured value
- * [HIVE-16743] - BitSet set() is incorrectly used in TxnUtils.createValidCompactTxnList()
- * [HIVE-16744] - LLAP index update may be broken after ORC switch
- * [HIVE-16745] - Syntax error in 041-HIVE-16556.mysql.sql script
- * [HIVE-16746] - Reduce number of index lookups for same table in IndexWhereTaskDispatcher
- * [HIVE-16751] - Support different types for grouping columns in GroupBy Druid queries
- * [HIVE-16755] - LLAP IO: incorrect assert may trigger in tests
- * [HIVE-16756] - Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
- * [HIVE-16757] - Use of deprecated getRows() instead of new estimateRowCount(RelMetadataQuery..) has serious performance impact
- * [HIVE-16761] - LLAP IO: SMB joins fail elevator
- * [HIVE-16769] - Possible hive service startup due to the existing file /tmp/stderr
- * [HIVE-16776] - Strange cast behavior for table backed by druid
- * [HIVE-16777] - LLAP: Use separate tokens and UGI instances when an external client is used
- * [HIVE-16778] - LLAP IO: better refcount management
- * [HIVE-16780] - Case "multiple sources, single key" in spark_dynamic_pruning.q fails
- * [HIVE-16784] - Missing lineage information when hive.blobstore.optimizations.enabled is true
- * [HIVE-16788] - ODBC call SQLForeignKeys leads to NPE if you use PK arguments rather than FK arguments
- * [HIVE-16793] - Scalar sub-query: sq_count_check not required if gby keys are constant
- * [HIVE-16801] - Vectorization: throwExpandError should be an immediate fatal
- * [HIVE-16803] - Alter table change column comment should not try to get column stats for update
- * [HIVE-16804] - Semijoin hint : Needs support for target table.
- * [HIVE-16808] - WebHCat statusdir parameter doesn't properly handle Unicode characters when using relative path
- * [HIVE-16820] - TezTask may not shut down correctly before submit
- * [HIVE-16821] - Vectorization: support Explain Analyze in vectorized mode
- * [HIVE-16824] - PrimaryToReplicaResourceFunctionTest.java has missed the ASF header
- * [HIVE-16826] - Improvements for SeparatedValuesOutputFormat
- * [HIVE-16828] - With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
- * [HIVE-16832] - duplicate ROW__ID possible in multi insert into transactional table
- * [HIVE-16835] - Addendum to HIVE-16745
- * [HIVE-16844] - Fix Connection leak in ObjectStore when new Conf object is used
- * [HIVE-16845] - INSERT OVERWRITE a table with dynamic partitions on S3 fails with NPE
- * [HIVE-16846] - TestJdbcWithMiniHS2#testHttpHeaderSize test case is not testing in HTTP mode
- * [HIVE-16847] - LLAP queue order issue
- * [HIVE-16851] - Scalar subquery with group by missing sq_count_check UDF
- * [HIVE-16854] - SparkClientFactory is locked too aggressively
- * [HIVE-16864] - add validation to stream position search in LLAP IO
- * [HIVE-16869] - Hive returns wrong result when predicates on non-existing columns are pushed down to Parquet reader
- * [HIVE-16871] - CachedStore.get_aggr_stats_for has side affect
- * [HIVE-16875] - Query against view with partitioned child on HoS fails with privilege exception.
- * [HIVE-16876] - HoS: Make Rpc configs immutable at runtime
- * [HIVE-16877] - NPE when issue query like alter table ... cascade onto non-partitioned table
- * [HIVE-16886] - HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently
- * [HIVE-16888] - Upgrade Calcite to 1.13 and Avatica to 1.10
- * [HIVE-16898] - Validation of source file after distcp in repl load
- * [HIVE-16902] - investigate "failed to remove operation log" errors
- * [HIVE-16903] - LLAP: Fix config name issue in SHUFFLE_MANAGE_OS_CACHE
- * [HIVE-16908] - Failures in TestHcatClient due to HIVE-16844
- * [HIVE-16910] - RpcConfiguration - Improper Cast From Long To Int
- * [HIVE-16915] - partition column count is not determined correctly in LLAP IO non-vectorized wrapper
- * [HIVE-16918] - Skip ReplCopyTask distcp for _metadata copying. Also enable -pb for distcp
- * [HIVE-16920] - remove useless uri.getScheme() from EximUtil
- * [HIVE-16922] - Typo in serde.thrift: COLLECTION_DELIM = "colelction.delim"
- * [HIVE-16927] - LLAP: Slider takes down all daemons when some daemons fail repeatedly
- * [HIVE-16930] - HoS should verify the value of Kerberos principal and keytab file before adding them to spark-submit command parameters
- * [HIVE-16935] - Hive should strip comments from input before choosing which CommandProcessor to run.
- * [HIVE-16937] - INFORMATION_SCHEMA usability: everything is currently a string
- * [HIVE-16938] - INFORMATION_SCHEMA usability: difficult to access # of table records
- * [HIVE-16939] - metastore error: 'export: -Dproc_metastore : not a valid identifier'
- * [HIVE-16942] - INFORMATION_SCHEMA: schematool for setting it up is not idempotent
- * [HIVE-16943] - MoveTask should separate src FileSystem from dest FileSystem
- * [HIVE-16947] - Semijoin Reduction : Task cycle created due to multiple semijoins in conjunction with hashjoin
- * [HIVE-16948] - Invalid explain when running dynamic partition pruning query in Hive On Spark
- * [HIVE-16949] - Leak of threads from Get-Input-Paths and Get-Input-Summary thread pool
- * [HIVE-16954] - LLAP IO: better debugging
- * [HIVE-16958] - Setting hive.merge.sparkfiles=true will retrun an error when generating parquet databases
- * [HIVE-16960] - Hive throws an ugly error exception when HDFS sticky bit is set
- * [HIVE-16961] - Hive on Spark leaks spark application in case user cancels query and closes session
- * [HIVE-16964] - _orc_acid_version file is missing
- * [HIVE-16965] - SMB join may produce incorrect results
- * [HIVE-16973] - Fetching of Delegation tokens (Kerberos) for AccumuloStorageHandler fails in HS2
- * [HIVE-16975] - Vectorization: Fully vectorize CAST date as TIMESTAMP so VectorUDFAdaptor is now used
- * [HIVE-16978] - HoS: add current thread ID to the log redirector for the RemoteDriver
- * [HIVE-16982] - WebUI "Show Query" tab prints "UNKNOWN" instead of explaining configuration option
- * [HIVE-16985] - LLAP IO: enable SMB join in elevator after the former is fixed
- * [HIVE-16991] - HiveMetaStoreClient needs a 2-arg constructor for backwards compatibility
- * [HIVE-17002] - decimal (binary) is not working when creating external table for hbase
- * [HIVE-17006] - LLAP: Parquet caching v1
- * [HIVE-17007] - NPE introduced by HIVE-16871
- * [HIVE-17008] - Fix boolean flag switchup in DropTableEvent
- * [HIVE-17010] - Fix the overflow problem of Long type in SetSparkReducerParallelism
- * [HIVE-17013] - Delete request with a subquery based on select over a view
- * [HIVE-17050] - Multiline queries that have comment in middle fail when executed via "beeline -e"
- * [HIVE-17052] - Remove logging of predicate filters
- * [HIVE-17066] - Query78 filter wrong estimatation is generating bad plan
- * [HIVE-17067] - LLAP: Add http endpoint to provide system level configurations
- * [HIVE-17069] - Refactor OrcRawRecrodMerger.ReaderPair
- * [HIVE-17070] - remove .orig files from src
- * [HIVE-17073] - Incorrect result with vectorization and SharedWorkOptimizer
- * [HIVE-17076] - typo in itests/src/test/resources/testconfiguration.properties
- * [HIVE-17079] - LLAP: Use FQDN by default for work submission
- * [HIVE-17083] - DagUtils overwrites any credentials already added
- * [HIVE-17085] - ORC file merge/concatenation should do full schema check
- * [HIVE-17086] - LLAP: JMX Metric for max file descriptors used so far
- * [HIVE-17088] - HS2 WebUI throws a NullPointerException when opened
- * [HIVE-17090] - spark.only.query.files are not being run by ptest
- * [HIVE-17093] - LLAP ssl configs need to be localized to talk to a wire encrypted hdfs
- * [HIVE-17095] - Long chain repl loads do not complete in a timely fashion
- * [HIVE-17097] - Fix SemiJoinHint parsing in SemanticAnalyzer
- * [HIVE-17098] - Race condition in Hbase tables
- * [HIVE-17099] - Update golden files for spark.only.query.files
- * [HIVE-17109] - Remove calls to RelMetadataQuery.instance() after Calcite 1.13 upgrade
- * [HIVE-17110] - BucketCodec should enforce value ranges
- * [HIVE-17111] - Add TestLocalSparkCliDriver
- * [HIVE-17113] - Duplicate bucket files can get written to table by runaway task
- * [HIVE-17114] - HoS: Possible skew in shuffling when data is not really skewed
- * [HIVE-17115] - MetaStoreUtils.getDeserializer doesn't catch the java.lang.ClassNotFoundException
- * [HIVE-17116] - Vectorization: Add infrastructure for vectorization of ROW__ID struct
- * [HIVE-17117] - Metalisteners are not notified when threadlocal metaconf is cleanup
- * [HIVE-17128] - Operation Logging leaks file descriptors as the log4j Appender is never closed
- * [HIVE-17144] - export of temporary tables not working and it seems to be using distcp rather than filesystem copy
- * [HIVE-17147] - Vectorization: Add code for testing MapJoin operator in isolation and measuring its performance with JMH
- * [HIVE-17148] - Incorrect result for Hive join query with COALESCE in WHERE condition
- * [HIVE-17149] - Hdfs directory is not cleared if partition creation failed on HMS
- * [HIVE-17150] - CREATE INDEX execute HMS out-of-transaction listener
<TRUNCATED>
[2/2] hive git commit: RELEASE_NOTES.txt for release 3.1.1
Posted by da...@apache.org.
RELEASE_NOTES.txt for release 3.1.1
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/436d01a3
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/436d01a3
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/436d01a3
Branch: refs/heads/branch-3.1
Commit: 436d01a36d792078c66b7240b362368d34a4836a
Parents: a01c382
Author: Daniel Dai <da...@gmail.com>
Authored: Tue Oct 23 16:47:12 2018 -0700
Committer: Daniel Dai <da...@gmail.com>
Committed: Tue Oct 23 16:47:12 2018 -0700
----------------------------------------------------------------------
RELEASE_NOTES.txt | 2151 +-----------------------------------------------
1 file changed, 3 insertions(+), 2148 deletions(-)
----------------------------------------------------------------------