You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/09/11 01:12:00 UTC
[jira] [Resolved] (SPARK-25157) Streaming of image files from
directory
[ https://issues.apache.org/jira/browse/SPARK-25157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-25157.
----------------------------------
Resolution: Duplicate
> Streaming of image files from directory
> ---------------------------------------
>
> Key: SPARK-25157
> URL: https://issues.apache.org/jira/browse/SPARK-25157
> Project: Spark
> Issue Type: New Feature
> Components: ML, Structured Streaming
> Affects Versions: 2.3.1
> Reporter: Amit Baghel
> Priority: Major
>
> We are doing video analytics for video streams using Spark. At present there is no direct way to stream video frames or image files to Spark and process them using Structured Streaming and Dataset. We are using Kafka to stream images and then doing processing at spark. We need a method in Spark to stream images from directory. Currently *{{DataStreamReader}}* doesn't support Image files. With the introduction of *org.apache.spark.ml.image.ImageSchema* class, we think streaming capabilities can be added for image files. It is fine if it won't support some of the structured streaming features as it is a binary file. This method could be similar to *mmlspark* *streamImages* method. [https://github.com/Azure/mmlspark/blob/4413771a8830e4760f550084da60ea0616bf80b9/src/io/image/src/main/python/ImageReader.py]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org