You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Preeti Nidgunde <pr...@gmail.com> on 2020/02/21 06:28:22 UTC
How to access IGFS file written one node from other node in cluster
??
I have written IGFS java application. I want to write shared file such that
if I write file from one node then it is accessible to all other node in
cluster. How to do that ??
I referred stack overflow and configured discovery spi to shared file system
but then also it is not working. The program written is accessible to only
the node who written that file not to other node (Other terminal).
When I tried to read by giving IGFS path to the file then I received
IGFS file not found exception. Where IGFS store this file.
public class FileExample
{
public static void main(String[] args) throws Exception
{
Ignite ignite =
Ignition.start("/root/apache-ignite-fabric-2.6.0-bin/examples/config/filesystem/example-igfs.xml");
System.out.println("\n");
System.out.println("IGFS example started.....");
IgniteFileSystem fs = ignite.fileSystem("myFileSystem");
IgfsPath dir = new
IgfsPath("myFileSystem://192.168.1.5:9060/Preeti");
fs.mkdirs(dir);
IgfsPath file = new IgfsPath(dir, "myFile.txt");
System.out.println(fs.info(file));
try (OutputStream out = fs.create(file, true))
{
OutputStreamWriter outputStreamWriter = new
OutputStreamWriter(out);
outputStreamWriter.write("This is Apache ignite file
system example .... Preeri Nidgunde ......Veriats Infoscale .... VXVM");
System.out.println("Done .....");
outputStreamWriter.close();
}catch(Exception e){}
try (InputStream in = fs.open(file))
{
Reader inputStreamReader = new
InputStreamReader(in);
int data = inputStreamReader.read();
while(data != -1)
{
char theChar = (char) data;
System.out.print(theChar);
data = inputStreamReader.read();
}
inputStreamReader.close();
}catch(Exception e){}
System.out.println("Read data from file");
}
}
On other node I am trying to read file like
public class ReadFile
{
public static void main(String[] args) throws Exception
{
Ignite ignite =
Ignition.start("/root/apache-ignite-fabric-2.6.0-bin/examples/config/filesystem/example-igfs.xml");
System.out.println("\n");
System.out.println("IGFS Read example started.....");
IgniteFileSystem fs = ignite.fileSystem("myFileSystem");
try (InputStream in = fs.open(new
IgfsPath("myFileSystem://192.168.1.5:9060/Preeti/myFile.txt")))
{
Reader inputStreamReader = new
InputStreamReader(in);
int data = inputStreamReader.read();
while(data != -1)
{
char theChar = (char) data;
System.out.print(theChar);
data = inputStreamReader.read();
}
inputStreamReader.close();
}catch(Exception e){e.printStackTrace();}
System.out.println("Read data from file");
}
}
But it is not working.
I have written file on one node and I want to access that written file from
other node.
Please help me.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to access IGFS file written one node from other node in
cluster ??
Posted by aealexsandrov <ae...@gmail.com>.
As was mentioned above IGFS was deprecated and not recommended for use.
Just use native HDFS API for working with files:
https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/FileSystem.html
In case if you have some SQL over HDFS (Hive, Impala) then you can set up
the Ignite cache store.
https://apacheignite.readme.io/docs/3rd-party-store
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to access IGFS file written one node from other node in
cluster ??
Posted by Preetiiii <pr...@gmail.com>.
If I configure Ignite File system configuration to spark or Hadoop then am I
able to write and read files from other node using normal java application
which I have written ?? If I source code need to modify just tell where to
modify as I am new to Ignite. Thanks in advance.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to access IGFS file written one node from other node in
cluster ??
Posted by Andrei Aleksandrov <ae...@gmail.com>.
Hi,
I can suggest to use the cache store implementation. For example current
guide
<https://www.gridgain.com/docs/latest/integrations/datalake-accelerator/load-sync-hive#schema-importing>
shows how Hive schema can be imported using web console.
In case if you require to work with files (not tables) then please use
HDFS or Spark API directly. Ignite provides good Spark integration:
https://apacheignite-fs.readme.io/docs/ignite-data-frame
BR,
Andrei
2/21/2020 6:52 PM, Preetiiii пишет:
> Then how to create shared file system or is there any way to access/modify
> file written by one node in cluster by other node ??
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to access IGFS file written one node from other node in
cluster ??
Posted by Preetiiii <pr...@gmail.com>.
Then how to create shared file system or is there any way to access/modify
file written by one node in cluster by other node ??
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to access IGFS file written one node from other node in
cluster ??
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
We do not recommend developing new IGFS applications because we are
removing this feature.
Regards,
--
Ilya Kasnacheev
пт, 21 февр. 2020 г. в 09:28, Preeti Nidgunde <pr...@gmail.com>:
> I have written IGFS java application. I want to write shared file such
> that
> if I write file from one node then it is accessible to all other node in
> cluster. How to do that ??
> I referred stack overflow and configured discovery spi to shared file
> system
> but then also it is not working. The program written is accessible to only
> the node who written that file not to other node (Other terminal).
> When I tried to read by giving IGFS path to the file then I
> received
> IGFS file not found exception. Where IGFS store this file.
>
>
> public class FileExample
> {
> public static void main(String[] args) throws Exception
> {
> Ignite ignite =
>
> Ignition.start("/root/apache-ignite-fabric-2.6.0-bin/examples/config/filesystem/example-igfs.xml");
>
>
> System.out.println("\n");
> System.out.println("IGFS example started.....");
>
> IgniteFileSystem fs = ignite.fileSystem("myFileSystem");
> IgfsPath dir = new
> IgfsPath("myFileSystem://192.168.1.5:9060/Preeti");
> fs.mkdirs(dir);
>
> IgfsPath file = new IgfsPath(dir, "myFile.txt");
>
> System.out.println(fs.info(file));
>
>
> try (OutputStream out = fs.create(file, true))
> {
> OutputStreamWriter outputStreamWriter = new
> OutputStreamWriter(out);
> outputStreamWriter.write("This is Apache ignite
> file
> system example .... Preeri Nidgunde ......Veriats Infoscale .... VXVM");
> System.out.println("Done .....");
> outputStreamWriter.close();
> }catch(Exception e){}
>
>
> try (InputStream in = fs.open(file))
> {
> Reader inputStreamReader = new
> InputStreamReader(in);
> int data = inputStreamReader.read();
> while(data != -1)
> {
> char theChar = (char) data;
> System.out.print(theChar);
> data = inputStreamReader.read();
> }
>
> inputStreamReader.close();
> }catch(Exception e){}
>
> System.out.println("Read data from file");
> }
> }
>
> On other node I am trying to read file like
>
> public class ReadFile
> {
> public static void main(String[] args) throws Exception
> {
> Ignite ignite =
>
> Ignition.start("/root/apache-ignite-fabric-2.6.0-bin/examples/config/filesystem/example-igfs.xml");
>
> System.out.println("\n");
> System.out.println("IGFS Read example started.....");
>
> IgniteFileSystem fs = ignite.fileSystem("myFileSystem");
>
> try (InputStream in = fs.open(new
> IgfsPath("myFileSystem://192.168.1.5:9060/Preeti/myFile.txt")))
> {
> Reader inputStreamReader = new
> InputStreamReader(in);
> int data = inputStreamReader.read();
> while(data != -1)
> {
> char theChar = (char) data;
> System.out.print(theChar);
> data = inputStreamReader.read();
> }
>
> inputStreamReader.close();
> }catch(Exception e){e.printStackTrace();}
>
> System.out.println("Read data from file");
> }
> }
> But it is not working.
>
> I have written file on one node and I want to access that written file from
> other node.
>
> Please help me.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>