dev@spark.apache.org, 2015-03

You are viewing a plain text version of this content. The canonical link for it is here.

- spark-ec2 default to Hadoop 2 - posted by Nicholas Chammas <ni...@gmail.com> on 2015/03/01 23:59:56 UTC, 5 replies.
- Is SparkSQL optimizer aware of the needed data after the query? - posted by Wail <w....@cces-kacst-mit.org> on 2015/03/02 10:55:56 UTC, 0 replies.
- Re: How to create a Row from a List or Array in Spark using Scala - posted by Dirceu Semighini Filho <di...@gmail.com> on 2015/03/02 14:16:50 UTC, 0 replies.
- Re: Using CUDA within Spark / boosting linear algebra - posted by Xiangrui Meng <me...@gmail.com> on 2015/03/02 20:42:06 UTC, 34 replies.
- PSA: Link to files at fixed version - posted by Nicholas Chammas <ni...@gmail.com> on 2015/03/02 23:39:26 UTC, 0 replies.
- Deploying master and worker programatically in java - posted by Niranda Perera <ni...@gmail.com> on 2015/03/03 11:43:13 UTC, 0 replies.
- ideas for MLlib development - posted by Robert Dodier <ro...@gmail.com> on 2015/03/04 02:51:25 UTC, 2 replies.
- [RESULT] [VOTE] Release Apache Spark 1.3.0 (RC1) - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/04 05:14:34 UTC, 0 replies.
- [VOTE] Release Apache Spark 1.3.0 (RC2) - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/04 05:19:19 UTC, 8 replies.
- Sharing SparkContext across multiple Unit Test Scala files - posted by spotvenky <sp...@gmail.com> on 2015/03/04 07:15:36 UTC, 0 replies.
- Spark Streaming and SchemaRDD usage - posted by Haopu Wang <HW...@qilinsoft.com> on 2015/03/04 11:04:24 UTC, 0 replies.
- Re: Google Summer of Code - Quick Query - posted by Ulrich Stärk <ul...@spielviel.de> on 2015/03/04 13:10:32 UTC, 0 replies.
- short jenkins 7am downtime tomorrow morning (3-5-15) - posted by shane knapp <sk...@berkeley.edu> on 2015/03/04 22:06:50 UTC, 2 replies.
- Task result is serialized twice by serializer and closure serializer - posted by Mingyu Kim <mk...@palantir.com> on 2015/03/05 01:01:16 UTC, 4 replies.
- enum-like types in Spark - posted by Xiangrui Meng <me...@gmail.com> on 2015/03/05 02:11:01 UTC, 28 replies.
- Fwd: Unable to Read/Write Avro RDD on cluster. - posted by ๏̯͡๏ <ÐΞ€ρ@Ҝ>, de...@gmail.com on 2015/03/05 05:57:04 UTC, 3 replies.
- Re: Which OutputCommitter to use for S3? - posted by Pei-Lun Lee <pl...@appier.com> on 2015/03/05 09:28:40 UTC, 3 replies.
- over 10000 commits! - posted by Reynold Xin <rx...@databricks.com> on 2015/03/05 22:26:46 UTC, 2 replies.
- [RESULT] [VOTE] Release Apache Spark 1.3.0 (RC2) - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/06 03:52:19 UTC, 0 replies.
- [VOTE] Release Apache Spark 1.3.0 (RC3) - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/06 03:52:34 UTC, 25 replies.
- Fwd: SparkSpark-perf terasort WIP branch - posted by Ewan Higgs <ew...@ugent.be> on 2015/03/06 10:41:14 UTC, 1 replies.
- Loading previously serialized object to Spark - posted by "Ulanov, Alexander" <al...@hp.com> on 2015/03/06 21:55:18 UTC, 7 replies.
- Block Transfer Service encryption support - posted by turp1twin <tu...@gmail.com> on 2015/03/07 01:20:09 UTC, 8 replies.
- Release Scala version vs Hadoop version (was: [VOTE] Release Apache Spark 1.3.0 (RC3)) - posted by Sean Owen <so...@cloudera.com> on 2015/03/08 22:56:31 UTC, 7 replies.
- GSoC 2015 - posted by "David J. Manglano" <da...@gmail.com> on 2015/03/09 04:53:58 UTC, 1 replies.
- missing explanation of cache in the documentation of cluster overview - posted by Hui WANG <he...@gmail.com> on 2015/03/09 10:24:05 UTC, 1 replies.
- How to implement unsupervised or reinforcement algorithm in new org.apache.spark.ml - posted by Egor Pahomov <pa...@gmail.com> on 2015/03/09 13:21:42 UTC, 1 replies.
- Cross cutting internal changes to launch scripts - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/09 20:53:00 UTC, 0 replies.
- SparkSQL 1.3.0 (RC3) failed to read parquet file generated by 1.1.1 - posted by Pei-Lun Lee <pl...@appier.com> on 2015/03/10 10:06:16 UTC, 4 replies.
- [SparkSQL] Reuse HiveContext to different Hive warehouse? - posted by Haopu Wang <HW...@qilinsoft.com> on 2015/03/10 11:37:34 UTC, 3 replies.
- Spark 1.3 SQL Type Parser Changes? - posted by Nitay Joffe <ni...@actioniq.co> on 2015/03/10 21:51:03 UTC, 2 replies.
- Spark tests hang on local machine due to "testGuavaOptional" in JavaAPISuite - posted by "Ganelin, Ilya" <Il...@capitalone.com> on 2015/03/11 01:09:47 UTC, 1 replies.
- GitHub Syncing Down - posted by Michael Armbrust <mi...@databricks.com> on 2015/03/11 02:54:13 UTC, 2 replies.
- [RESULT] [VOTE] Release Apache Spark 1.3.0 (RC3) - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/11 06:24:34 UTC, 0 replies.
- Spark Streaming - received block allocation to batch - posted by Zoltán Zvara <zo...@gmail.com> on 2015/03/11 14:58:27 UTC, 1 replies.
- Using Log4j2 in spark executors - posted by "lior.c" <li...@taboola.com> on 2015/03/11 16:40:03 UTC, 0 replies.
- Apache Spark GSOC 2015 - posted by Tamer TAS <ta...@outlook.com> on 2015/03/11 21:22:39 UTC, 0 replies.
- is there any api in spark like getInstance(className:String):AnyRef - posted by "Zhang, Liyun" <li...@intel.com> on 2015/03/12 07:47:30 UTC, 0 replies.
- Is this a bug in MLlib.stat.test ? About the mapPartitions API used in Chi-Squared test - posted by Chunnan Yao <ya...@gmail.com> on 2015/03/12 08:13:55 UTC, 1 replies.
- Re: adding some temporary jenkins worker nodes... - posted by shane knapp <sk...@berkeley.edu> on 2015/03/12 23:10:19 UTC, 0 replies.
- Spilling when not expected - posted by Tom Hubregtsen <th...@gmail.com> on 2015/03/13 00:09:11 UTC, 3 replies.
- Profiling Spark: MemoryStore - posted by "Ulanov, Alexander" <al...@hp.com> on 2015/03/13 01:34:40 UTC, 1 replies.
- toLocalIterator creates as many jobs as # of partitions, and it ends up spamming Spark UI - posted by Mingyu Kim <mk...@palantir.com> on 2015/03/13 08:04:32 UTC, 0 replies.
- May we merge into branch-1.3 at this point? - posted by Sean Owen <so...@cloudera.com> on 2015/03/13 11:06:39 UTC, 4 replies.
- Spark config option 'expression language' feedback request - posted by Dale Richardson <da...@hotmail.com> on 2015/03/13 11:07:57 UTC, 8 replies.
- [ANNOUNCE] Announcing Spark 1.3! - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/13 18:00:14 UTC, 1 replies.
- extended jenkins downtime monday, march 16th, plus some hints at the future - posted by shane knapp <sk...@berkeley.edu> on 2015/03/13 18:12:30 UTC, 3 replies.
- PR Builder timing out due to ivy cache lock - posted by Hari Shreedharan <hs...@cloudera.com> on 2015/03/13 19:53:16 UTC, 3 replies.
- jenkins httpd being flaky - posted by shane knapp <sk...@berkeley.edu> on 2015/03/13 20:04:17 UTC, 3 replies.
- Spark ThriftServer encounter java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth] - posted by Andrew Lee <al...@hotmail.com> on 2015/03/13 20:43:44 UTC, 1 replies.
- [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3? - posted by Yu Ishikawa <yu...@gmail.com> on 2015/03/15 08:45:22 UTC, 7 replies.
- Wrong version on the Spark documentation page - posted by Cheng Lian <li...@gmail.com> on 2015/03/15 18:40:07 UTC, 3 replies.
- broadcast hang out - posted by lonely Feb <lo...@gmail.com> on 2015/03/16 04:08:14 UTC, 6 replies.
- SparkSQL 1.3.0 cannot read parquet files from different file system - posted by Pei-Lun Lee <pl...@appier.com> on 2015/03/16 09:03:41 UTC, 3 replies.
- problems with Parquet in Spark 1.3.0 - posted by Gil Vernik <GI...@il.ibm.com> on 2015/03/16 13:09:44 UTC, 1 replies.
- Typo in 1.3.0 release notes: s/extended renamed/renamed/ - posted by Joe Halliwell <jo...@gmail.com> on 2015/03/16 14:18:29 UTC, 1 replies.
- Using Spark with a SOCKS proxy - posted by "Kelly, Jonathan" <jo...@amazon.com> on 2015/03/18 01:15:16 UTC, 1 replies.
- InvalidAuxServiceException in dynamicAllocation - posted by Sea <26...@qq.com> on 2015/03/18 03:07:05 UTC, 1 replies.
- Fixed worker ports in the spark worker - posted by Niranda Perera <ni...@gmail.com> on 2015/03/18 06:40:21 UTC, 3 replies.
- parquet support - some questions about code - posted by Gil Vernik <GI...@il.ibm.com> on 2015/03/18 14:46:21 UTC, 1 replies.
- Which linear algebra interface to use within Spark MLlib? - posted by "Ulanov, Alexander" <al...@hp.com> on 2015/03/19 04:09:17 UTC, 7 replies.
- SparkSQL 1.3.0 JDBC data source issues - posted by Pei-Lun Lee <pl...@appier.com> on 2015/03/19 05:20:59 UTC, 1 replies.
- Spark scheduling, data locality - posted by Zoltán Zvara <zo...@gmail.com> on 2015/03/19 11:20:25 UTC, 1 replies.
- Spark SQL ExternalSorter not stopped - posted by Michael Allman <mi...@videoamp.com> on 2015/03/19 18:51:14 UTC, 1 replies.
- Exception using the new createDirectStream util method - posted by Alberto Rodriguez <ar...@gmail.com> on 2015/03/19 20:10:18 UTC, 6 replies.
- Add Char support in SQL dataTypes - posted by "A.M.Chan" <ka...@163.com> on 2015/03/20 02:56:10 UTC, 1 replies.
- Contributing to Spark - posted by "vikas.v.iitr@gmail.com" <vi...@gmail.com> on 2015/03/20 08:50:18 UTC, 0 replies.
- 答复: Contributing to Spark - posted by Tanyinyan <ta...@huawei.com> on 2015/03/20 09:42:22 UTC, 1 replies.
- Connecting a worker to the master after a spark context is made - posted by Niranda Perera <ni...@gmail.com> on 2015/03/20 10:13:29 UTC, 0 replies.
- Storage of RDDs created via sc.parallelize - posted by Karlson <ks...@siberie.de> on 2015/03/20 20:09:07 UTC, 1 replies.
- Minor Edit in the programming guide - posted by "Muttineni, Vinay" <vm...@ebay.com> on 2015/03/20 22:53:38 UTC, 1 replies.
- Directly broadcasting (sort of) RDDs - posted by Guillaume Pitel <gu...@exensa.com> on 2015/03/20 23:24:40 UTC, 6 replies.
- Re: Error: 'SparkContext' object has no attribute 'getActiveStageIds' - posted by xing <eh...@gmail.com> on 2015/03/20 23:59:09 UTC, 1 replies.
- Filesystem closed Exception - posted by Sea <26...@qq.com> on 2015/03/21 06:35:01 UTC, 2 replies.
- 回复： Filesystem closed Exception - posted by Sea <26...@qq.com> on 2015/03/21 16:38:34 UTC, 0 replies.
- lower&upperBound not working/spark 1/3 - posted by Marek Wiewiorka <ma...@gmail.com> on 2015/03/22 16:39:51 UTC, 3 replies.
- Spark Sql with python udf fail - posted by lonely Feb <lo...@gmail.com> on 2015/03/23 08:43:59 UTC, 0 replies.
- Starting sparkthrift server - posted by Neil Dev <ne...@gmail.com> on 2015/03/23 16:37:19 UTC, 5 replies.
- Spark-thriftserver Issue - posted by Neil Dev <ne...@gmail.com> on 2015/03/23 20:01:29 UTC, 3 replies.
- Fwd: hadoop input/output format advanced control - posted by Koert Kuipers <ko...@tresata.com> on 2015/03/23 21:52:10 UTC, 13 replies.
- Spark Executor resources - posted by Zoltán Zvara <zo...@gmail.com> on 2015/03/23 22:08:33 UTC, 4 replies.
- Shuffle Spill Memory and Shuffle Spill Disk - posted by Bijay Pathak <bi...@cloudwick.com> on 2015/03/23 22:25:11 UTC, 1 replies.
- Review request for SPARK-6112:Provide OffHeap support through HDFS RAM_DISK - posted by Zhan Zhang <zz...@hortonworks.com> on 2015/03/23 23:47:51 UTC, 2 replies.
- Optimize the first map reduce of DStream - posted by Bin Wang <wb...@gmail.com> on 2015/03/24 09:08:26 UTC, 4 replies.
- Any guidance on when to back port and how far? - posted by Sean Owen <so...@cloudera.com> on 2015/03/24 14:13:11 UTC, 2 replies.
- Understanding shuffle file name conflicts - posted by Kannan Rajah <kr...@maprtech.com> on 2015/03/24 18:56:26 UTC, 10 replies.
- Experience using binary packages on various Hadoop distros - posted by Patrick Wendell <pw...@gmail.com> on 2015/03/24 20:05:03 UTC, 4 replies.
- Spark SQL(1.3.0) "import sqlContext.implicits._" seems not work for converting a case class RDD to DataFrame - posted by Zhiwei Chan <z....@gmail.com> on 2015/03/25 04:46:24 UTC, 2 replies.
- Can't assembly YARN project with SBT - posted by Zoltán Zvara <zo...@gmail.com> on 2015/03/25 09:45:25 UTC, 1 replies.
- functools.partial as UserDefinedFunction - posted by Karlson <ks...@siberie.de> on 2015/03/25 13:42:32 UTC, 2 replies.
- Re: mllib.recommendation Design - posted by Debasish Das <de...@gmail.com> on 2015/03/25 15:59:59 UTC, 2 replies.
- LogisticGradient Design - posted by Debasish Das <de...@gmail.com> on 2015/03/25 16:15:09 UTC, 4 replies.
- jenkins upgraded to 1.606.... - posted by shane knapp <sk...@berkeley.edu> on 2015/03/25 18:02:57 UTC, 0 replies.
- Jira Issues - posted by Igor Costa <ig...@apache.org> on 2015/03/25 21:51:31 UTC, 4 replies.
- Haskell language Spark support - posted by danilo2 <wo...@gmail.com> on 2015/03/26 03:35:36 UTC, 0 replies.
- Can I call aggregate UDF in DataFrame? - posted by Haopu Wang <HW...@qilinsoft.com> on 2015/03/26 08:37:17 UTC, 0 replies.
- Storing large data for MLlib machine learning - posted by "Ulanov, Alexander" <al...@hp.com> on 2015/03/26 22:16:26 UTC, 7 replies.
- RDD.map does not allowed to preservesPartitioning? - posted by Zhan Zhang <zz...@hortonworks.com> on 2015/03/26 22:44:23 UTC, 5 replies.
- Re: Building spark 1.2 from source requires more dependencies - posted by Pala M Muthaia <mc...@rocketfuelinc.com> on 2015/03/26 23:48:20 UTC, 6 replies.
- Support for Hive 0.14 in secure mode on hadoop 2.6.0 - posted by Doug Balog <do...@dugos.com> on 2015/03/27 09:48:56 UTC, 3 replies.
- Iterative pyspark / scala codebase development - posted by Stephen Boesch <ja...@gmail.com> on 2015/03/27 18:02:27 UTC, 7 replies.
- RDD.count - posted by jimfcarroll <ji...@gmail.com> on 2015/03/28 01:54:58 UTC, 6 replies.
- Lazy casting with Catalyst - posted by Patrick Woody <pa...@gmail.com> on 2015/03/28 16:26:44 UTC, 4 replies.
- One corrupt gzip in a directory of 100s - posted by Dale Richardson <da...@hotmail.com> on 2015/03/29 22:47:14 UTC, 0 replies.
- What is the meaning to of 'STATE' in a worker/ an executor? - posted by Niranda Perera <ni...@gmail.com> on 2015/03/30 06:09:20 UTC, 1 replies.
- [sql] How to uniquely identify Dataframe? - posted by Peter Rudenko <pe...@gmail.com> on 2015/03/30 14:02:37 UTC, 2 replies.
- Problems with cleanup throughout code base - posted by "Ganelin, Ilya" <Il...@capitalone.com> on 2015/03/31 01:40:31 UTC, 0 replies.
- Stochastic gradient descent performance - posted by "Ulanov, Alexander" <al...@hp.com> on 2015/03/31 01:59:49 UTC, 0 replies.
- How to get removed RDD from windows？ - posted by "wyphao.2007" <wy...@163.com> on 2015/03/31 04:46:27 UTC, 1 replies.