You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Josh Rosen <ro...@gmail.com> on 2013/10/24 05:57:25 UTC
Documentation of Java API and PySpark internals
I've created two new pages on the Spark wiki to document the internals of
the Java and Python APIs:
https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
These are only rough drafts; please let me know if there's anything that
you'd like me to document (or feel free to add it yourself!).
- Josh
Re: Documentation of Java API and PySpark internals
Posted by Patrick Wendell <pw...@gmail.com>.
+1 - josh this is awesome.
On Wed, Oct 23, 2013 at 10:15 PM, Reynold Xin <rx...@apache.org> wrote:
> Thanks, Josh. These are very useful for people to understand the APIs and
> to write new language bindings.
>
>
> On Wed, Oct 23, 2013 at 8:57 PM, Josh Rosen <ro...@gmail.com> wrote:
>
> > I've created two new pages on the Spark wiki to document the internals of
> > the Java and Python APIs:
> >
> > https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
> > https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
> >
> > These are only rough drafts; please let me know if there's anything that
> > you'd like me to document (or feel free to add it yourself!).
> >
> > - Josh
> >
>
Re: Documentation of Java API and PySpark internals
Posted by Reynold Xin <rx...@apache.org>.
Thanks, Josh. These are very useful for people to understand the APIs and
to write new language bindings.
On Wed, Oct 23, 2013 at 8:57 PM, Josh Rosen <ro...@gmail.com> wrote:
> I've created two new pages on the Spark wiki to document the internals of
> the Java and Python APIs:
>
> https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
> https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
>
> These are only rough drafts; please let me know if there's anything that
> you'd like me to document (or feel free to add it yourself!).
>
> - Josh
>
Re: Documentation of Java API and PySpark internals
Posted by Christopher Nguyen <ct...@adatao.com>.
+1 thanks, I learned a lot more about Spark's JavaAPI motivations from this
documentation. We've run into many of these issues in our own mixed code
base, particularly with implicit manifests and JVM type erasure.
Sent while mobile. Pls excuse typos etc.
On Oct 23, 2013 8:57 PM, "Josh Rosen" <ro...@gmail.com> wrote:
> I've created two new pages on the Spark wiki to document the internals of
> the Java and Python APIs:
>
> https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
> https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
>
> These are only rough drafts; please let me know if there's anything that
> you'd like me to document (or feel free to add it yourself!).
>
> - Josh
>
Re: Documentation of Java API and PySpark internals
Posted by Matei Zaharia <ma...@gmail.com>.
Yeah, very cool, thanks for writing these up.
On Oct 23, 2013, at 10:33 PM, Henry Saputra <he...@gmail.com> wrote:
> Thanks Josh, this is really great!
>
> +1
>
> - Henry
>
> On Wed, Oct 23, 2013 at 8:57 PM, Josh Rosen <ro...@gmail.com> wrote:
>> I've created two new pages on the Spark wiki to document the internals of
>> the Java and Python APIs:
>>
>> https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
>> https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
>>
>> These are only rough drafts; please let me know if there's anything that
>> you'd like me to document (or feel free to add it yourself!).
>>
>> - Josh
Re: Documentation of Java API and PySpark internals
Posted by Henry Saputra <he...@gmail.com>.
Thanks Josh, this is really great!
+1
- Henry
On Wed, Oct 23, 2013 at 8:57 PM, Josh Rosen <ro...@gmail.com> wrote:
> I've created two new pages on the Spark wiki to document the internals of
> the Java and Python APIs:
>
> https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
> https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
>
> These are only rough drafts; please let me know if there's anything that
> you'd like me to document (or feel free to add it yourself!).
>
> - Josh