You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@superset.apache.org by Priyanka Sinha <pr...@zenatix.com> on 2023/01/27 09:35:42 UTC
Ingesting Data from REST API endpoint
Hello Team,
In our Superset installation we have connected with multiple Database
connectors including cloud connectors. I was wondering if it is possible to
create a dataset which ingests from a REST API endpoint.
More specifically, we are trying to create a weather widget in our
dashboard where the data for the Temperature is obtained from an external
weather service REST API endpoint. It could be ingested in say a Big Number
chart.
Thanks and Regards,
Priyanka
Re: Ingesting Data from REST API endpoint
Posted by Beto Dealmeida <ro...@dealmeida.net.INVALID>.
Thanks, Srini!
I also add an adapter to generic JSON APIs, recently:
https://shillelagh.readthedocs.io/en/latest/adapters.html#generic-json-apis
The generic JSON works with any endpoint, at the cost of efficiency (it
doesn't know how to limit results on the server side, so any predicates
are applied on the client side). But if you need performance it's
relatively easy to create a custom adapter to a specific API:
https://shillelagh.readthedocs.io/en/latest/development.html
--Beto
On 1/27/23 7:48 AM, Srini Kadamati wrote:
> Nitesh and Priyanka,
>
> Hey! So the *shillelagh* library by Beto (a Superset PMC member) will
> let you do just this in Superset:
>
> https://shillelagh.readthedocs.io/en/latest/development.html
>
> In fact Beto has a weather API example!
>
> Screen Shot 2023-01-27 at 10.46.03 AM.png
>
> Srini
>
>
> On Fri, Jan 27, 2023 at 10:39 AM Nitesh Baranwal
> <ni...@veeva.com.invalid> wrote:
>
> We would LOVE to have something like this as well! Definitely not
> a very
> easy and generic connector to build :)
> Thanks,
> Nitesh
>
> --
> *Nitesh Baranwal*
> Director, Product Management
> Veeva Nitro
>
> 224.420.2045
> nitesh.baranwal@veeva.com
>
>
> On Fri, Jan 27, 2023 at 4:36 AM Priyanka Sinha
> <pr...@zenatix.com>
> wrote:
>
> > Hello Team,
> >
> > In our Superset installation we have connected with multiple
> Database
> > connectors including cloud connectors. I was wondering if it is
> possible to
> > create a dataset which ingests from a REST API endpoint.
> >
> > More specifically, we are trying to create a weather widget in our
> > dashboard where the data for the Temperature is obtained from an
> external
> > weather service REST API endpoint. It could be ingested in say a
> Big Number
> > chart.
> >
> > Thanks and Regards,
> > Priyanka
> >
>
>
>
> --
> Srini Kadamati
> skadamat@gmail.com
> LinkedIn.com/in/SrinivasaKadamati
> <http://LinkedIn.com/in/SrinivasaKadamati>
>
Re: Ingesting Data from REST API endpoint
Posted by Charles Givre <cg...@gmail.com>.
Hi All,
If you want something that's a little more powerful, that can do things liek OAuth, Pagination etc. you should check out Drill's integration for this:
https://github.com/apache/drill/tree/master/contrib/storage-http
drill/contrib/storage-http at master · apache/drill
github.com
Drill can connect to Superset and you can thus query APIs.. much more.
-- C
> On Jan 27, 2023, at 10:48 AM, Srini Kadamati <sk...@gmail.com> wrote:
>
> Nitesh and Priyanka,
>
> Hey! So the shillelagh library by Beto (a Superset PMC member) will let you do just this in Superset:
>
> https://shillelagh.readthedocs.io/en/latest/development.html
>
> In fact Beto has a weather API example!
>
> <Screen Shot 2023-01-27 at 10.46.03 AM.png>
>
> Srini
>
>
> On Fri, Jan 27, 2023 at 10:39 AM Nitesh Baranwal <ni...@veeva.com.invalid> wrote:
>> We would LOVE to have something like this as well! Definitely not a very
>> easy and generic connector to build :)
>> Thanks,
>> Nitesh
>>
>> --
>> *Nitesh Baranwal*
>> Director, Product Management
>> Veeva Nitro
>>
>> 224.420.2045
>> nitesh.baranwal@veeva.com <ma...@veeva.com>
>>
>>
>> On Fri, Jan 27, 2023 at 4:36 AM Priyanka Sinha <priyanka.sinha@zenatix.com <ma...@zenatix.com>>
>> wrote:
>>
>> > Hello Team,
>> >
>> > In our Superset installation we have connected with multiple Database
>> > connectors including cloud connectors. I was wondering if it is possible to
>> > create a dataset which ingests from a REST API endpoint.
>> >
>> > More specifically, we are trying to create a weather widget in our
>> > dashboard where the data for the Temperature is obtained from an external
>> > weather service REST API endpoint. It could be ingested in say a Big Number
>> > chart.
>> >
>> > Thanks and Regards,
>> > Priyanka
>> >
>
>
> --
> Srini Kadamati
> skadamat@gmail.com <ma...@gmail.com>
> LinkedIn.com/in/SrinivasaKadamati <http://linkedin.com/in/SrinivasaKadamati>
>
Re: Ingesting Data from REST API endpoint
Posted by Srini Kadamati <sk...@gmail.com>.
Nitesh and Priyanka,
Hey! So the *shillelagh* library by Beto (a Superset PMC member) will let
you do just this in Superset:
https://shillelagh.readthedocs.io/en/latest/development.html
In fact Beto has a weather API example!
[image: Screen Shot 2023-01-27 at 10.46.03 AM.png]
Srini
On Fri, Jan 27, 2023 at 10:39 AM Nitesh Baranwal
<ni...@veeva.com.invalid> wrote:
> We would LOVE to have something like this as well! Definitely not a very
> easy and generic connector to build :)
> Thanks,
> Nitesh
>
> --
> *Nitesh Baranwal*
> Director, Product Management
> Veeva Nitro
>
> 224.420.2045
> nitesh.baranwal@veeva.com
>
>
> On Fri, Jan 27, 2023 at 4:36 AM Priyanka Sinha <priyanka.sinha@zenatix.com
> >
> wrote:
>
> > Hello Team,
> >
> > In our Superset installation we have connected with multiple Database
> > connectors including cloud connectors. I was wondering if it is possible
> to
> > create a dataset which ingests from a REST API endpoint.
> >
> > More specifically, we are trying to create a weather widget in our
> > dashboard where the data for the Temperature is obtained from an external
> > weather service REST API endpoint. It could be ingested in say a Big
> Number
> > chart.
> >
> > Thanks and Regards,
> > Priyanka
> >
>
--
Srini Kadamati
skadamat@gmail.com
LinkedIn.com/in/SrinivasaKadamati
Re: Ingesting Data from REST API endpoint
Posted by Nitesh Baranwal <ni...@veeva.com.INVALID>.
We would LOVE to have something like this as well! Definitely not a very
easy and generic connector to build :)
Thanks,
Nitesh
--
*Nitesh Baranwal*
Director, Product Management
Veeva Nitro
224.420.2045
nitesh.baranwal@veeva.com
On Fri, Jan 27, 2023 at 4:36 AM Priyanka Sinha <pr...@zenatix.com>
wrote:
> Hello Team,
>
> In our Superset installation we have connected with multiple Database
> connectors including cloud connectors. I was wondering if it is possible to
> create a dataset which ingests from a REST API endpoint.
>
> More specifically, we are trying to create a weather widget in our
> dashboard where the data for the Temperature is obtained from an external
> weather service REST API endpoint. It could be ingested in say a Big Number
> chart.
>
> Thanks and Regards,
> Priyanka
>