You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Flavio Pompermaier <po...@okkam.it> on 2018/12/03 17:01:33 UTC
CKAN inputFormat (batch)
Hi to all,
we've just published an example of a simple CKAN input format that
downloads a CKAN resource (in parallel) from a CKAN catalog and produce
a DataSet<Row>.
This can be very helpful in setting up a Flink demo using an OpenData
dataset available online (see [1] for a list of available catalogs).
An example of its usage can be found here: [2].
Best,
Flavio
[1] https://ckan.org/about/instances/
[2]
https://github.com/okkam-it/flink-examples/blob/master/src/main/java/it/okkam/flink/CkanDatasetImporter.java
Re: CKAN inputFormat (batch)
Posted by Dawid Wysakowicz <dw...@apache.org>.
Hi Flavio,
Thank you for the example. It is definitely gonna be helpful for some
people!
Best,
Dawid
On 04/12/2018 09:05, Flavio Pompermaier wrote:
> Yesterday it was working...alternatively you can look at
> https://github.com/ckan/ckan-instances/blob/gh-pages/config/instances.json
>
> The purpose of this code is to share with the community an example of
> a useful input format that coul be interesting for people working with
> open data (indeed there are lots of useful datasets in Ckan catalogues
> in the web).
>
>
> On Tue, 4 Dec 2018, 07:10 vino yang <yanghua1127@gmail.com
> <ma...@gmail.com> wrote:
>
> Hi Flavio,
>
> I can not open the first link[1] you provided.
> And what is your purpose? Introduce your CKAN input format to the
> community?
>
> Thanks, vino.
>
> [1]: https://ckan.org/about/instances/
>
> Flavio Pompermaier <pompermaier@okkam.it
> <ma...@okkam.it>> 于2018年12月4日周二 上午1:09写道：
>
> Hi to all,
> we've just published an example of a simple CKAN input format
> that downloads a CKAN resource (in parallel) from a CKAN
> catalog and produce a DataSet<Row>.
> This can be very helpful in setting up a Flink demo using an
> OpenData dataset available online (see [1] for a list of
> available catalogs).
>
> An example of its usage can be found here: [2].
>
> Best,
> Flavio
>
> [1] https://ckan.org/about/instances/
> [2]
> https://github.com/okkam-it/flink-examples/blob/master/src/main/java/it/okkam/flink/CkanDatasetImporter.java
>
Re: CKAN inputFormat (batch)
Posted by Flavio Pompermaier <po...@okkam.it>.
Yesterday it was working...alternatively you can look at
https://github.com/ckan/ckan-instances/blob/gh-pages/config/instances.json
The purpose of this code is to share with the community an example of a
useful input format that coul be interesting for people working with open
data (indeed there are lots of useful datasets in Ckan catalogues in the
web).
On Tue, 4 Dec 2018, 07:10 vino yang <yanghua1127@gmail.com wrote:
> Hi Flavio,
>
> I can not open the first link[1] you provided.
> And what is your purpose? Introduce your CKAN input format to the
> community?
>
> Thanks, vino.
>
> [1]: https://ckan.org/about/instances/
>
> Flavio Pompermaier <po...@okkam.it> 于2018年12月4日周二 上午1:09写道：
>
>> Hi to all,
>> we've just published an example of a simple CKAN input format that
>> downloads a CKAN resource (in parallel) from a CKAN catalog and produce
>> a DataSet<Row>.
>> This can be very helpful in setting up a Flink demo using an OpenData
>> dataset available online (see [1] for a list of available catalogs).
>>
>> An example of its usage can be found here: [2].
>>
>> Best,
>> Flavio
>>
>> [1] https://ckan.org/about/instances/
>> [2]
>> https://github.com/okkam-it/flink-examples/blob/master/src/main/java/it/okkam/flink/CkanDatasetImporter.java
>>
>
Re: CKAN inputFormat (batch)
Posted by vino yang <ya...@gmail.com>.
Hi Flavio,
I can not open the first link[1] you provided.
And what is your purpose? Introduce your CKAN input format to the community?
Thanks, vino.
[1]: https://ckan.org/about/instances/
Flavio Pompermaier <po...@okkam.it> 于2018年12月4日周二 上午1:09写道：
> Hi to all,
> we've just published an example of a simple CKAN input format that
> downloads a CKAN resource (in parallel) from a CKAN catalog and produce
> a DataSet<Row>.
> This can be very helpful in setting up a Flink demo using an OpenData
> dataset available online (see [1] for a list of available catalogs).
>
> An example of its usage can be found here: [2].
>
> Best,
> Flavio
>
> [1] https://ckan.org/about/instances/
> [2]
> https://github.com/okkam-it/flink-examples/blob/master/src/main/java/it/okkam/flink/CkanDatasetImporter.java
>