You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Burak Yavuz (JIRA)" <ji...@apache.org> on 2016/03/31 06:05:25 UTC

[jira] [Created] (SPARK-14287) isStreaming method for Dataset

Burak Yavuz created SPARK-14287:
-----------------------------------

             Summary: isStreaming method for Dataset
                 Key: SPARK-14287
                 URL: https://issues.apache.org/jira/browse/SPARK-14287
             Project: Spark
          Issue Type: Sub-task
          Components: SQL, Streaming
            Reporter: Burak Yavuz


With the addition of StreamExecution (ContinuousQuery) to Datasets, data will become unbounded. With unbounded data, the execution of some methods and operations will not make sense, e.g. Dataset.count().

A simple API is required to check whether the data in a Dataset is bounded or unbounded. This will allow users to check whether their Dataset is in streaming mode or not. ML algorithms may check if the data is unbounded and throw an exception for example.

The implementation of this method is simple, however naming it is the challenge. Some possible names for this method are:
 - isStreaming
 - isContinuous
 - isBounded
 - isUnbounded



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org