You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/11/13 05:09:47 UTC
[GitHub] [spark] itholic commented on a change in pull request #30346: [SPARK-33253][PYTHON][DOCS] Migration to NumPy documentation style in Streaming (pyspark.streaming.*)
itholic commented on a change in pull request #30346:
URL: https://github.com/apache/spark/pull/30346#discussion_r522650964
##########
File path: python/pyspark/streaming/kinesis.py
##########
@@ -43,38 +43,59 @@ def createStream(ssc, kinesisAppName, streamName, endpointUrl, regionName,
Create an input stream that pulls messages from a Kinesis stream. This uses the
Kinesis Client Library (KCL) to pull messages from Kinesis.
- .. note:: The given AWS credentials will get saved in DStream checkpoints if checkpointing
- is enabled. Make sure that your checkpoint directory is secure.
+ Parameters
+ ----------
+ ssc : :class:`StreamingContext`
+ StreamingContext object
+ kinesisAppName : str
+ Kinesis application name used by the Kinesis Client Library (KCL) to
+ update DynamoDB
+ streamName : str
+ Kinesis stream name
+ endpointUrl : str
+ Url of Kinesis service (e.g., https://kinesis.us-east-1.amazonaws.com)
+ regionName : str
+ Name of region used by the Kinesis Client Library (KCL) to update
+ DynamoDB (lease coordination and checkpointing) and CloudWatch (metrics)
+ initialPositionInStream : int
+ In the absence of Kinesis checkpoint info, this is the
+ worker's initial starting position in the stream. The
+ values are either the beginning of the stream per Kinesis'
+ limit of 24 hours (InitialPositionInStream.TRIM_HORIZON) or
+ the tip of the stream (InitialPositionInStream.LATEST).
+ checkpointInterval : int
+ Checkpoint interval for Kinesis checkpointing. See the Kinesis
+ Spark Streaming documentation for more details on the different
+ types of checkpoints.
+ storageLevel : :class:`StorageLevel`, optional
Review comment:
Cool. Thanks!! 👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org