You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "wesharn (Jira)" <ji...@apache.org> on 2022/04/21 02:13:00 UTC
[jira] [Updated] (SPARK-38975) support jdbc about greenplum more
[ https://issues.apache.org/jira/browse/SPARK-38975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
wesharn updated SPARK-38975:
----------------------------
Description:
the code as:
Dataset<Row> invoke = spark.read().format("jdbc").option("url", url)
.option("dbtable", query)
.option("partitionColumn", "id")
.option("numPartitions", "5")
.option("lowerBound", Long.MIN_VALUE)
.option("upperBound", Long.MAX_VALUE)
.load();
when the gp table is a large table,oom will occur
how can handle the large table?
> support jdbc about greenplum more
> ---------------------------------
>
> Key: SPARK-38975
> URL: https://issues.apache.org/jira/browse/SPARK-38975
> Project: Spark
> Issue Type: Improvement
> Components: Java API
> Affects Versions: 3.2.1
> Reporter: wesharn
> Priority: Major
>
> the code as:
> Dataset<Row> invoke = spark.read().format("jdbc").option("url", url)
> .option("dbtable", query)
> .option("partitionColumn", "id")
> .option("numPartitions", "5")
> .option("lowerBound", Long.MIN_VALUE)
> .option("upperBound", Long.MAX_VALUE)
> .load();
> when the gp table is a large table,oom will occur
> how can handle the large table?
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org