You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/11/01 04:16:00 UTC
[jira] [Work logged] (HIVE-25659) Divide IN/(NOT IN) queries based
on number of max parameters SQL engine can support
[ https://issues.apache.org/jira/browse/HIVE-25659?focusedWorklogId=672532&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-672532 ]
ASF GitHub Bot logged work on HIVE-25659:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Nov/21 04:15
Start Date: 01/Nov/21 04:15
Worklog Time Spent: 10m
Work Description: guptanikhil007 commented on a change in pull request #2758:
URL: https://github.com/apache/hive/pull/2758#discussion_r739430648
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##########
@@ -680,6 +680,9 @@ public static ConfVars getMetaConf(String name) {
DIRECT_SQL_MAX_ELEMENTS_VALUES_CLAUSE("metastore.direct.sql.max.elements.values.clause",
"hive.direct.sql.max.elements.values.clause",
1000, "The maximum number of values in a VALUES clause for INSERT statement."),
+ DIRECT_SQL_MAX_PARAMETERS("metastore.direct.sql.max.parameters",
+ "hive.direct.sql.max.parameters", 1000, "The maximum parameters the\n" +
Review comment:
MSSQL: 2100
Oracle: 64,000. Source
MySQL: The MySQL has a limit of 65,535 parameters (because num_params has to fit in two bytes).
PostgreSql: 34464 for a query and 100 for a function
SqlLite: 999 (SQLITE_MAX_VARIABLE_NUMBER, which defaults to 999, but can be lowered at runtime) - And for functions default is 100 parameters.
Source:
[StackOverflow](https://stackoverflow.com/questions/6581573/what-are-the-max-number-of-allowable-parameters-per-database-provider-type/49379324#:~:text=Thus%20the%20PostgreSQL%20protocol%20doesn,of%20which%20has%2065535%20parameters.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 672532)
Time Spent: 50m (was: 40m)
> Divide IN/(NOT IN) queries based on number of max parameters SQL engine can support
> -----------------------------------------------------------------------------------
>
> Key: HIVE-25659
> URL: https://issues.apache.org/jira/browse/HIVE-25659
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
> Affects Versions: 3.1.0, 4.0.0
> Reporter: Nikhil Gupta
> Assignee: Nikhil Gupta
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Function org.apache.hadoop.hive.metastore.txn.TxnUtils#buildQueryWithINClauseStrings can generate queries with huge number of parameters with very small value of DIRECT_SQL_MAX_ELEMENTS_IN_CLAUSE and DIRECT_SQL_MAX_QUERY_LENGTH while generating delete query for completed_compactions table
> Example:
> {code:java}
> DIRECT_SQL_MAX_ELEMENTS_IN_CLAUSE = 100
> DIRECT_SQL_MAX_QUERY_LENGTH = 10 (10 KB)
> Number of parameters in a single query = 4759
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)