You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Karthik Nandagiri <ka...@costrategix.com> on 2021/09/01 06:54:36 UTC
Data loading from csv, excel and json
Hi Team
We have a requirement where a client(user) will provide me data in any of
the formats, csv, excel or json.
Now, we need to load this data into ignite table(table is created). For csv
we are using sqlline tool with copy from command.
Is there a mechanism to load excel and json data as well? As we won't be
having access to clients' RDBMS, we are provided the data in any of the
file formats mentioned above.
Regards
Karthik N
Re: Data loading from csv, excel and json
Posted by Stephen Darlington <st...@gridgain.com>.
I found this for opening Excel files in Python: https://www.pythonexcel.com
And here is the Python documentation: https://ignite.apache.org/docs/latest/thin-clients/python-thin-client
> On 1 Sep 2021, at 11:23, Karthik Nandagiri <ka...@costrategix.com> wrote:
>
>
> Thank you Stephen for the inputs.
>
> Please provide me with some examples to load excel data to ignite with python client, this will help to speed up my development
>
> The way we use ignite is, start it as an external service with ignite.sh and provide the xml config path, and then create tables and load data .
>
>
> Thank you
> Regards
> Karthik N
>
> On Wed, Sep 1, 2021 at 1:49 PM Stephen Darlington <stephen.darlington@gridgain.com <ma...@gridgain.com>> wrote:
> I would look at using the Python thin client. There are libraries for reading Excel and parsing JSON. There’s a good chance that there are Java libraries, too, but I’ve not used them.
>
> > On 1 Sep 2021, at 07:54, Karthik Nandagiri <karthik.nandagiri@costrategix.com <ma...@costrategix.com>> wrote:
> >
> > Hi Team
> >
> > We have a requirement where a client(user) will provide me data in any of the formats, csv, excel or json.
> > Now, we need to load this data into ignite table(table is created). For csv we are using sqlline tool with copy from command.
> > Is there a mechanism to load excel and json data as well? As we won't be having access to clients' RDBMS, we are provided the data in any of the file formats mentioned above.
> >
> > Regards
> > Karthik N
>
>
Re: Data loading from csv, excel and json
Posted by Karthik Nandagiri <ka...@costrategix.com>.
Thank you Stephen for the inputs.
Please provide me with some examples to load excel data to ignite with
python client, this will help to speed up my development
The way we use ignite is, start it as an external service with ignite.sh
and provide the xml config path, and then create tables and load data .
Thank you
Regards
Karthik N
On Wed, Sep 1, 2021 at 1:49 PM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> I would look at using the Python thin client. There are libraries for
> reading Excel and parsing JSON. There’s a good chance that there are Java
> libraries, too, but I’ve not used them.
>
> > On 1 Sep 2021, at 07:54, Karthik Nandagiri <
> karthik.nandagiri@costrategix.com> wrote:
> >
> > Hi Team
> >
> > We have a requirement where a client(user) will provide me data in any
> of the formats, csv, excel or json.
> > Now, we need to load this data into ignite table(table is created). For
> csv we are using sqlline tool with copy from command.
> > Is there a mechanism to load excel and json data as well? As we won't be
> having access to clients' RDBMS, we are provided the data in any of the
> file formats mentioned above.
> >
> > Regards
> > Karthik N
>
>
>
Re: Data loading from csv, excel and json
Posted by Karthik Nandagiri <ka...@costrategix.com>.
Thank you Stephen for the inputs.
Please provide me with some examples to load excel data to ignite with
python client, this will help to speed up my development
The way we use ignite is, start it as an external service with ignite.sh
and provide the xml config path, and then create tables and load data .
Thank you
Regards
Karthik N
On Wed, Sep 1, 2021 at 1:49 PM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> I would look at using the Python thin client. There are libraries for
> reading Excel and parsing JSON. There’s a good chance that there are Java
> libraries, too, but I’ve not used them.
>
> > On 1 Sep 2021, at 07:54, Karthik Nandagiri <
> karthik.nandagiri@costrategix.com> wrote:
> >
> > Hi Team
> >
> > We have a requirement where a client(user) will provide me data in any
> of the formats, csv, excel or json.
> > Now, we need to load this data into ignite table(table is created). For
> csv we are using sqlline tool with copy from command.
> > Is there a mechanism to load excel and json data as well? As we won't be
> having access to clients' RDBMS, we are provided the data in any of the
> file formats mentioned above.
> >
> > Regards
> > Karthik N
>
>
>
Re: Data loading from csv, excel and json
Posted by Stephen Darlington <st...@gridgain.com>.
I would look at using the Python thin client. There are libraries for reading Excel and parsing JSON. There’s a good chance that there are Java libraries, too, but I’ve not used them.
> On 1 Sep 2021, at 07:54, Karthik Nandagiri <ka...@costrategix.com> wrote:
>
> Hi Team
>
> We have a requirement where a client(user) will provide me data in any of the formats, csv, excel or json.
> Now, we need to load this data into ignite table(table is created). For csv we are using sqlline tool with copy from command.
> Is there a mechanism to load excel and json data as well? As we won't be having access to clients' RDBMS, we are provided the data in any of the file formats mentioned above.
>
> Regards
> Karthik N