You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by MadDoxX <jo...@outlook.fr> on 2020/03/15 20:59:04 UTC
Spark 2.4.x and 3.x datasourcev2 api documentation & references
Hi,
i have been following closely the new proposed api, and seems until now some
lack of documentation... specially for people trying to implement them in
JAVA.
Anyway to give out some reference as starting point for java developpers as
tests classes ?
(anyway new api looks promising) !
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Spark 2.4.x and 3.x datasourcev2 api documentation & references
Posted by MadDoxX <jo...@outlook.fr>.
Thanks for the link !
i was able to code my first datasources in microbatch and continuous by
using the v2 api...
i was wondering if there are some equivalent to SparkListener for
microbatching & continuous stream (where one could use the
spark.extra.listener) ~ monitoring :)
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Spark 2.4.x and 3.x datasourcev2 api documentation & references
Posted by Wenchen Fan <cl...@gmail.com>.
For now you can take a look at `DataSourceV2Suite`, which contains both
Java/Scala implementations. There is also an ongoing PR to implement
catalog APIs for JDBC: https://github.com/apache/spark/pull/27345
We are still working on the user guide.
On Mon, Mar 16, 2020 at 4:59 AM MadDoxX <jo...@outlook.fr>
wrote:
> Hi,
> i have been following closely the new proposed api, and seems until now
> some
> lack of documentation... specially for people trying to implement them in
> JAVA.
>
> Anyway to give out some reference as starting point for java developpers as
> tests classes ?
>
> (anyway new api looks promising) !
>
>
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>