You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by ishan-jain <ja...@gmail.com> on 2017/06/07 04:10:32 UTC

IGFS Question

I need to load data from IGFS into a seperate ignite cache which can be
accessed remotely. How to do it in a fast way?




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by Yermek <ye...@gmail.com>.
ishan-jain wrote
> But for this i have to open ignite with IGFS config. How will i use ignite
> then with normal config?
> 
> On Wed, Jun 7, 2017 at 3:55 PM, Yermek &lt;

> yermek.kumarbekov@

> &gt; wrote:
> 
>> Hi, maybe like this:
>>
>> try (
>>          InputStream in = fs.open(new IgfsPath(dataPath));
>>          BufferedReader buffer = new BufferedReader(new
>> InputStreamReader(in));
>>          IgniteDataStreamer&lt;Integer,String&gt; dsStreamer =
>> ignite.dataStreamer(cache.getName())
>>      )
>>        {
>>             AtomicInteger i = new AtomicInteger();
>>             buffer.lines().forEach(s ->
>> dsStreamer.add(i.getAndIncrement(),s));
>>         }
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.
>> 70518.x6.nabble.com/IGFS-Question-tp13447p13463.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>

Why don't you use config file? Just copy it to client machine and change
Client_mode to true. 
Or you should set configuration direct in your program without config file.
How to do it you can find on official Ignite site.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p13514.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by ishan-jain <ja...@gmail.com>.
Thankyou all :)



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p13549.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by vkulichenko <va...@gmail.com>.
If you're reloading data from IGFS to a regular cache, why do you need IGFS
in the first place? Why not load data directly in the cache?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p14280.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by Ishan Jain <ja...@gmail.com>.
By starting another ignite node i mean an ignite node with different
config. As seen in ignite data fabric example of stream words from file we
have to use some event types also .

On Wed, Jun 14, 2017 at 10:38 AM, Ishan Jain <ja...@gmail.com> wrote:

> Okay i start using file system api which ignite file system has. I start
> an ignite node and start loading data from it.
> Now i have to put data into normal ignite cache for using sql queries on
> it. Ignite node has been already started from the program in IGFS mode. How
> will i create a normal ignite cache which can be accessed remotely.
> Shouldn't i have to start another ignite node from the same program?This is
> really confusing. Please revert
>
> On Fri, Jun 9, 2017 at 11:00 AM, Yermek <ye...@gmail.com>
> wrote:
>
>> Yes, of course, you're right. I used a little wrong phrase, only by using
>> FileSystem API
>>
>> vkulichenko wrote
>> > IGFS cache can't and should not be accessed directly. Use FileSystem API
>> > for this.
>> >
>> > -Val
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/IGFS-Question-tp13447p13547.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: IGFS Question

Posted by Ishan Jain <ja...@gmail.com>.
Okay i start using file system api which ignite file system has. I start an
ignite node and start loading data from it.
Now i have to put data into normal ignite cache for using sql queries on
it. Ignite node has been already started from the program in IGFS mode. How
will i create a normal ignite cache which can be accessed remotely.
Shouldn't i have to start another ignite node from the same program?This is
really confusing. Please revert

On Fri, Jun 9, 2017 at 11:00 AM, Yermek <ye...@gmail.com> wrote:

> Yes, of course, you're right. I used a little wrong phrase, only by using
> FileSystem API
>
> vkulichenko wrote
> > IGFS cache can't and should not be accessed directly. Use FileSystem API
> > for this.
> >
> > -Val
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IGFS-Question-tp13447p13547.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: IGFS Question

Posted by Yermek <ye...@gmail.com>.
Yes, of course, you're right. I used a little wrong phrase, only by using
FileSystem API 

vkulichenko wrote
> IGFS cache can't and should not be accessed directly. Use FileSystem API
> for this.
> 
> -Val





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p13547.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by vkulichenko <va...@gmail.com>.
IGFS cache can't and should not be accessed directly. Use FileSystem API for
this.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p13536.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by Yermek <ye...@gmail.com>.
IGFS works as usual caches and you can get it remotely too without any
problem. 
Before accessing set Network configuration, so your program will be able to
connect to remote node.


ishan-jain wrote
> And IGFS cache is system cache so cannot be remotely accessed.
> 
> On Thu, Jun 8, 2017 at 8:51 AM, Ishan Jain &lt;

> jain.ishan95@

> &gt; wrote:
> 
>> But for this i have to open ignite with IGFS config. How will i use
>> ignite
>> then with normal config?
>>
>> On Wed, Jun 7, 2017 at 3:55 PM, Yermek &lt;

> yermek.kumarbekov@

> &gt;
>> wrote:
>>
>>> Hi, maybe like this:
>>>
>>> try (
>>>          InputStream in = fs.open(new IgfsPath(dataPath));
>>>          BufferedReader buffer = new BufferedReader(new
>>> InputStreamReader(in));
>>>          IgniteDataStreamer&lt;Integer,String&gt; dsStreamer =
>>> ignite.dataStreamer(cache.getName())
>>>      )
>>>        {
>>>             AtomicInteger i = new AtomicInteger();
>>>             buffer.lines().forEach(s ->
>>> dsStreamer.add(i.getAndIncrement(),s));
>>>         }
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/IGFS-Question-tp13447p13463.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p13515.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS Question

Posted by Ishan Jain <ja...@gmail.com>.
And IGFS cache is system cache so cannot be remotely accessed.

On Thu, Jun 8, 2017 at 8:51 AM, Ishan Jain <ja...@gmail.com> wrote:

> But for this i have to open ignite with IGFS config. How will i use ignite
> then with normal config?
>
> On Wed, Jun 7, 2017 at 3:55 PM, Yermek <ye...@gmail.com>
> wrote:
>
>> Hi, maybe like this:
>>
>> try (
>>          InputStream in = fs.open(new IgfsPath(dataPath));
>>          BufferedReader buffer = new BufferedReader(new
>> InputStreamReader(in));
>>          IgniteDataStreamer<Integer,String> dsStreamer =
>> ignite.dataStreamer(cache.getName())
>>      )
>>        {
>>             AtomicInteger i = new AtomicInteger();
>>             buffer.lines().forEach(s ->
>> dsStreamer.add(i.getAndIncrement(),s));
>>         }
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/IGFS-Question-tp13447p13463.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: IGFS Question

Posted by Ishan Jain <ja...@gmail.com>.
But for this i have to open ignite with IGFS config. How will i use ignite
then with normal config?

On Wed, Jun 7, 2017 at 3:55 PM, Yermek <ye...@gmail.com> wrote:

> Hi, maybe like this:
>
> try (
>          InputStream in = fs.open(new IgfsPath(dataPath));
>          BufferedReader buffer = new BufferedReader(new
> InputStreamReader(in));
>          IgniteDataStreamer<Integer,String> dsStreamer =
> ignite.dataStreamer(cache.getName())
>      )
>        {
>             AtomicInteger i = new AtomicInteger();
>             buffer.lines().forEach(s ->
> dsStreamer.add(i.getAndIncrement(),s));
>         }
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IGFS-Question-tp13447p13463.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: IGFS Question

Posted by Yermek <ye...@gmail.com>.
Hi, maybe like this:

try (
         InputStream in = fs.open(new IgfsPath(dataPath));
         BufferedReader buffer = new BufferedReader(new
InputStreamReader(in));
	 IgniteDataStreamer<Integer,String> dsStreamer =
ignite.dataStreamer(cache.getName())
     )
       {
            AtomicInteger i = new AtomicInteger();
            buffer.lines().forEach(s ->
dsStreamer.add(i.getAndIncrement(),s));
	}
		




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-Question-tp13447p13463.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.