[jira] [Resolved] (SPARK-12521) DataFrame Partitions in java does not work

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-12521.
-------------------------------
    Resolution: Not A Problem

It's already described as an arg that controls partition stride. It wouldn't make sense to specify filters separately outside the WHERE clause here.

> DataFrame Partitions in java does not work
> ------------------------------------------
>
>                 Key: SPARK-12521
>                 URL: https://issues.apache.org/jira/browse/SPARK-12521
>             Project: Spark
>          Issue Type: Bug
>          Components: Java API, SQL
>    Affects Versions: 1.5.2
>            Reporter: Sergey Podolsky
>
> Hello,
> Partition does not work in Java interface of the DataFrame:
> {code}
> SQLContext sqlContext = new SQLContext(sc);
> Map<String, String> options = new HashMap<>();
> options.put("driver", ORACLE_DRIVER);
> options.put("url", ORACLE_CONNECTION_URL);
> options.put("dbtable",
>     "(SELECT * FROM JOBS WHERE ROWNUM < 10000) tt");
> options.put("lowerBound", "2704225000");
> options.put("upperBound", "2704226000");
> options.put("partitionColumn", "ID");
> options.put("numPartitions", "10");
> DataFrame jdbcDF = sqlContext.load("jdbc", options);
> List<Row> jobsRows = jdbcDF.collectAsList();
> System.out.println(jobsRows.size());
> {code}
> gives 9999 while expected 1000. Is it because of big decimal of boundaries or partitioins does not work at all in Java?
> Thanks.
> Sergey



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org