[jira] [Created] (SPARK-20248) Spark SQL add limit parameter to enhance the reliability.

["shaolinliu (JIRA)" <ji...@apache.org>]

shaolinliu created SPARK-20248:
----------------------------------

             Summary: Spark SQL add limit parameter to enhance the reliability.
                 Key: SPARK-20248
                 URL: https://issues.apache.org/jira/browse/SPARK-20248
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.1.0
         Environment: 2.1.0
            Reporter: shaolinliu
            Priority: Minor


  When we using thrift server, it is difficult to constrain the user's sql statement;
  When the user query a large table without limit, this will lead to thrift server process memory occupancy lead to service instability;
  In general, the user is not used correctly, because if you really need to return the whole table:
  1, if you use this date to compute , you can complete the computation in the cluster and then return
  2, if you want obtain the data, you can store it in hdfs

  For the above scene, it is recommended to add a "spark.sql.thriftServer.retainedResults" parameter,
  1, when it is 0, we don not restrict user's operation
  2, when it is greater than 0, if user query with limit, we use user's limit;if not we use this to limit query's result
  Priority user's limit is because, if the user consider the limit, in general, the user is aware of the exact meaning of this query



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org