You are viewing a plain text version of this content. The canonical link for it is here.
- Re: Spark structured streaming Stuck on Batch = 0 on spark 3.1.1, Dataproc cluster - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/01 15:25:18 UTC, 0 replies.
- What is the latest stable release of Spark - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/01 16:32:07 UTC, 8 replies.
- Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 - posted by shiva <sh...@gmail.com> on 2021/03/01 17:25:00 UTC, 5 replies.
- [Virtual meetup] March 11th 10am PST - posted by Alma Maria Rinasz <al...@sg.com.mx> on 2021/03/01 23:14:34 UTC, 0 replies.
- Spark Version 3.0.1 Gui Display Query - posted by Ranju Jain <Ra...@ericsson.com.INVALID> on 2021/03/02 05:31:35 UTC, 19 replies.
- Spark job crashing - Spark Structured Streaming with Kafka - posted by Sachit Murarka <co...@gmail.com> on 2021/03/02 08:25:55 UTC, 4 replies.
- [Spark SQL, intermediate+] possible bug or weird behavior of insertInto - posted by Oldrich Vlasic <ol...@datasentics.com> on 2021/03/02 11:37:22 UTC, 5 replies.
- Re: Structured Streaming With Kafka - processing each event - posted by Sachit Murarka <co...@gmail.com> on 2021/03/02 12:55:34 UTC, 1 replies.
- Please update this notification on Spark download Site - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/02 15:09:38 UTC, 1 replies.
- [ANNOUNCE] Announcing Apache Spark 3.1.1 - posted by Hyukjin Kwon <gu...@gmail.com> on 2021/03/03 01:41:00 UTC, 13 replies.
- Spark 3.1.1 Preliminary results (mainly to do with Spark Structured Streaming) - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/03 09:35:33 UTC, 0 replies.
- Yaml for google spark kubernetes configmap - posted by rajat kumar <ku...@gmail.com> on 2021/03/03 14:32:12 UTC, 0 replies.
- Spark structured streaming seems to work on local mode only - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/03 17:21:20 UTC, 3 replies.
- Possible upgrade path from Spark 3.1.1-RC2 to Spark 3.1.1 GA - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/04 14:15:25 UTC, 2 replies.
- 退订 - posted by 吃完药感觉自己萌萌哒 <13...@qq.com> on 2021/03/05 07:08:35 UTC, 4 replies.
- Structured Streaming Microbatch Semantics - posted by "Dipl.-Inf. Rico Bergmann" <in...@ricobergmann.de> on 2021/03/05 08:06:06 UTC, 8 replies.
- (无主题) - posted by Sophia <sl...@163.com> on 2021/03/05 11:10:40 UTC, 2 replies.
- Spark streaming with multiple Kafka topics - posted by lalitha bandaru <la...@gmail.com> on 2021/03/05 19:28:57 UTC, 0 replies.
- Fwd: [jira] [Commented] (SPARK-34648) Reading Parquet Files in Spark Extremely Slow for Large Number of Files? - posted by Pankaj Bhootra <pa...@gmail.com> on 2021/03/06 17:22:36 UTC, 3 replies.
- - posted by Sandeep Varma <sa...@zs.com> on 2021/03/07 11:40:11 UTC, 0 replies.
- Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage - posted by Ranju Jain <Ra...@ericsson.com.INVALID> on 2021/03/07 16:22:31 UTC, 0 replies.
- com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device\n\t - posted by Sachit Murarka <co...@gmail.com> on 2021/03/08 08:01:35 UTC, 4 replies.
- Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage - posted by Jacek Laskowski <ja...@japila.pl> on 2021/03/08 10:43:58 UTC, 10 replies.
- Re: How to control count / size of output files for - posted by m li <xi...@gmail.com> on 2021/03/08 15:58:40 UTC, 2 replies.
- Single executor processing all tasks in spark structured streaming kafka - posted by Sachit Murarka <co...@gmail.com> on 2021/03/08 17:26:21 UTC, 2 replies.
- Call for Presentations for ApacheCon 2021 now open - posted by Rich Bowen <rb...@apache.org> on 2021/03/08 21:01:34 UTC, 0 replies.
- Creating spark context outside of the driver throws error - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/08 21:02:14 UTC, 2 replies.
- Detecting latecomer events in Spark structured streaming - posted by Sergey Oboguev <ob...@gmail.com> on 2021/03/08 21:14:45 UTC, 1 replies.
- Spark Streaming - Routing rdd to Executor based on Key - posted by forece85 <fo...@gmail.com> on 2021/03/09 07:50:01 UTC, 5 replies.
- Sounds like Structured streaming with foreach, can only run on one executor - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/09 22:41:14 UTC, 2 replies.
- Speed up Spark writes to Google Cloud storage - posted by SRK <sw...@gmail.com> on 2021/03/09 23:40:40 UTC, 0 replies.
- spark 3.1.1 support hive 1.2 - posted by jiahong li <mo...@gmail.com> on 2021/03/10 02:49:56 UTC, 2 replies.
- [spark-core] docker-image-tool.sh question... - posted by Muthu Jayakumar <ba...@gmail.com> on 2021/03/10 04:43:32 UTC, 2 replies.
- compile spark 3.1.1 error - posted by jiahong li <mo...@gmail.com> on 2021/03/10 10:26:47 UTC, 6 replies.
- FlatMapGroupsWithStateFunction is called thrice - Production use case. - posted by Kuttaiah Robin <ku...@gmail.com> on 2021/03/11 07:54:41 UTC, 2 replies.
- Spark on Kubernetes | 3.0.1 | Shared Volume or NFS - posted by Ranju Jain <Ra...@ericsson.com.INVALID> on 2021/03/11 11:28:20 UTC, 6 replies.
- spark on k8s driver pod exception - posted by yxl040840219 <yx...@126.com> on 2021/03/11 12:05:56 UTC, 4 replies.
- How to upgrade kafka client in spark_streaming_kafka 2.2 - posted by Renu Yadav <yr...@gmail.com> on 2021/03/12 08:44:50 UTC, 5 replies.
- Issue while consuming message in kafka using structured streaming - posted by Sachit Murarka <co...@gmail.com> on 2021/03/12 11:28:07 UTC, 4 replies.
- Using Spark as a fail-over platform for Java app - posted by Sergey Oboguev <ob...@gmail.com> on 2021/03/12 19:43:13 UTC, 2 replies.
- pyspark - posted by Antoine Morales <an...@yahoo.fr.INVALID> on 2021/03/14 01:31:05 UTC, 0 replies.
- DB Config data update across multiple Spark Streaming Jobs - posted by forece85 <fo...@gmail.com> on 2021/03/14 06:15:12 UTC, 1 replies.
- Spark Structured Streaming and Kafka message schema evolution - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/15 12:25:30 UTC, 2 replies.
- How to make bucket listing faster while using S3 with wholeTextFile - posted by Alchemist <al...@gmail.com> on 2021/03/15 16:31:02 UTC, 7 replies.
- Spark Structured Streaming from GCS files - posted by Gowrishankar Sunder <sh...@gmail.com> on 2021/03/15 18:56:07 UTC, 2 replies.
- [k8s] PersistentVolumeClaim support in 3.1.1 on minikube - posted by Jacek Laskowski <ja...@japila.pl> on 2021/03/15 19:36:26 UTC, 1 replies.
- How default partitioning in spark is deployed - posted by Renganathan Mutthiah <re...@gmail.com> on 2021/03/16 04:34:07 UTC, 7 replies.
- Spark streaming giving error for version 2.4 - posted by Renu Yadav <yr...@gmail.com> on 2021/03/16 05:02:44 UTC, 2 replies.
- Submitting insert query from beeline failing on executor server with java 11 - posted by kaki mahesh raja <ka...@nokia.com> on 2021/03/16 07:16:53 UTC, 4 replies.
- Using Spark Structured Streaming as an ETL tool - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/17 16:54:34 UTC, 0 replies.
- Is Spark rdd.toDF() thread-safe? - posted by "yujhe.li" <li...@gmail.com> on 2021/03/18 01:15:04 UTC, 0 replies.
- spark 3.1.1 combine hadoop(version 2.6.0-cdh5.13.1) compile error - posted by jiahong li <mo...@gmail.com> on 2021/03/18 03:47:46 UTC, 0 replies.
- Spark 3.1.1 availability in Google Cloud - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/18 14:10:19 UTC, 0 replies.
- Spark version verification - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/18 15:08:46 UTC, 12 replies.
- ERROR org.apache.spark.scheduler.AsyncEventQueue: Listener EventLoggingListener threw an exception - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/18 17:02:26 UTC, 4 replies.
- Coalesce vs reduce operation parameter - posted by Pedro Tuero <tu...@gmail.com> on 2021/03/18 21:46:52 UTC, 3 replies.
- Can JVisual VM monitoring tool be used to Monitor Spark Executor Memory and CPU - posted by Ranju Jain <Ra...@ericsson.com.INVALID> on 2021/03/20 16:05:30 UTC, 6 replies.
- Parallelism parameter in cross validation - posted by Arshee Siddiqui <si...@gmail.com> on 2021/03/20 16:25:08 UTC, 0 replies.
- Spark saveAsTextFile Disk Recommendation - posted by Ranju Jain <Ra...@ericsson.com.INVALID> on 2021/03/21 02:39:48 UTC, 3 replies.
- In built Optimizer on Spark - posted by Felix Kizhakkel Jose <fe...@gmail.com> on 2021/03/21 13:53:02 UTC, 1 replies.
- [Spark SQL]: Can complex oracle views be created using Spark SQL - posted by Gaurav Singh <ga...@gmail.com> on 2021/03/22 05:26:04 UTC, 2 replies.
- Invite Spark community as Pulsar Summit NA 2021 Community Partner - posted by Dianjin Wang <dj...@streamnative.io.INVALID> on 2021/03/22 07:13:48 UTC, 0 replies.
- Bucketing 3.1.1 - posted by German Schiavon <gs...@gmail.com> on 2021/03/22 07:52:18 UTC, 2 replies.
- Why code is failing to connect to Oracle DB in 3.1.1 through JDBC with Scala - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/22 09:13:42 UTC, 1 replies.
- unit testing for spark code - posted by Amit Sharma <re...@gmail.com> on 2021/03/22 13:32:21 UTC, 3 replies.
- connecting to an Apache Spark on AWS and port 22 - posted by Bogdan Tanasa <ta...@gmail.com> on 2021/03/22 16:53:51 UTC, 3 replies.
- Repartition or Coalesce not working - posted by KhajaAsmath Mohammed <md...@gmail.com> on 2021/03/22 17:14:43 UTC, 2 replies.
- Question about how hadoop configurations populated in driver/executor pod - posted by Yue Peng <yu...@microsoft.com.INVALID> on 2021/03/22 18:29:26 UTC, 0 replies.
- Spark History Server log files questions - posted by Hung Vu <hv...@snapchat.com.INVALID> on 2021/03/22 22:49:35 UTC, 1 replies.
- Spark learning for beginner and certification - posted by Kishore Kumar <k....@gmail.com> on 2021/03/23 15:48:37 UTC, 0 replies.
- Spark on your Oracle Data Warehouse - posted by Harish Butani <rh...@gmail.com> on 2021/03/23 15:51:11 UTC, 1 replies.
- Rdd - zip with index - posted by KhajaAsmath Mohammed <md...@gmail.com> on 2021/03/24 01:18:26 UTC, 15 replies.
- Is it enable to use Multiple UGIs in One Spark Context? - posted by Kwangsun Noh <no...@gmail.com> on 2021/03/25 13:12:45 UTC, 3 replies.
- FW: Email to Spark Org please - posted by "Williams, David (Risk Value Stream)" <Da...@Lloydsbanking.com.INVALID> on 2021/03/25 16:21:53 UTC, 5 replies.
- Re: Application Timeout - posted by Brett Spark <bl...@gmail.com> on 2021/03/25 20:29:31 UTC, 0 replies.
- Spark Views Functioning - posted by Kushagra Deep <Ku...@mobileum.com> on 2021/03/26 06:54:05 UTC, 0 replies.
- convert java dataframe to pyspark dataframe - posted by Aditya Singh <ad...@gmail.com> on 2021/03/26 08:35:21 UTC, 6 replies.
- Re: Spark Views Functioning - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/26 08:37:57 UTC, 2 replies.
- The trigger interval in spark structured streaming - posted by Mich Talebzadeh <mi...@gmail.com> on 2021/03/26 17:43:47 UTC, 3 replies.
- Spark Structured Streaming Continuous Trigger Mode React to an External Trigger - posted by shahrajesh2006 <sh...@gmail.com> on 2021/03/27 12:29:44 UTC, 3 replies.
- Source.getBatch and schema vs qe.analyzed.schema? - posted by Jacek Laskowski <ja...@japila.pl> on 2021/03/29 11:07:08 UTC, 1 replies.
- Error Message Suggestion - posted by Josh Herzberg <jo...@audantic.com> on 2021/03/29 15:29:38 UTC, 1 replies.
- How to gracefully shutdown spark job on kubernetes - posted by Sachit Murarka <co...@gmail.com> on 2021/03/29 17:52:55 UTC, 0 replies.
- Ubuntu 18.04: Docker: start-master.sh: command not found - posted by GUINKO Ferdinand <to...@guinko.net> on 2021/03/29 18:17:43 UTC, 12 replies.
- Re: Spark thrift server ldap - posted by Pavel Solomin <p....@gmail.com> on 2021/03/31 08:56:47 UTC, 0 replies.
- Re: Introducing Gallia: a Scala+Spark library for data manipulation - posted by galliaproject <co...@gmail.com> on 2021/03/31 13:41:09 UTC, 0 replies.