[jira] [Resolved] (SPARK-12927) Dataframe doesn't display the current status on Aerospike.

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-12927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-12927.
-------------------------------
    Resolution: Not A Problem

You have copied the contents of a DataFrame to the driver with collect(). That copy won't change just because an external system changes. The results are immutable, though, in theory might change if the resulting DataFrame is recomputed and the underlying data has changed. However, you don't know if/when this happens. Really, you need to make a new DataFrame.

> Dataframe doesn't display the current status on Aerospike.
> ----------------------------------------------------------
>
>                 Key: SPARK-12927
>                 URL: https://issues.apache.org/jira/browse/SPARK-12927
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 1.5.1
>            Reporter: Sasi
>
> Hi,
> I'm facing with issue on my environment when my dataframe doesn't display the real status of my Aerospike bin.
> I have the following code:
> {code}
>  SparkContextFunctions sparkContextFunctions = new SparkContextFunctions(spark_context); 
>             String aeroSpikeAddress = host+port;
>             String query = "select * from bin";
>             dataFrame = sparkContextFunctions.aeroSInput(aeroSpikeAddress, query, sqlCtx, 6);
>             dataFrame .registerTempTable("testTbl");
>             dataFrame .persist(StorageLevel.MEMORY_ONLY());
> {code}
> After create the tmpTbl I do the following:
> {code}
> subscribersDataFrame.where(whereClause).collect();
> {code}
> At the first time I get the same values of the Aerospike bin, but if I add or delete row I don't get the same status.
> If I drop the table and create it again then I get the right status.
> Thanks.
> Sasi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org