You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Fengyun RAO <ra...@gmail.com> on 2013/12/06 03:04:27 UTC
Writing to remote HDFS using C# on Windows
Hi, All
Is there a way to write files into remote HDFS on Linux using C# on
Windows? We want to use HDFS as data storage.
We know there is HDFS java API, but not C#. We tried SAMBA for file sharing
and FUSE for mounting HDFS. It worked if we simply copy files to HDFS, but
if we open a filestream and write to it, it always throws exceptions.
Best regards!
RE: Writing to remote HDFS using C# on Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
If you using HDFS on 2.2, you could use a nfs. More recent versions of Windows include an NFS client. The exclamation mark is used to refer to the root directory of HDFS. For example:
net use * \\hostname_or_ip\<file:///\\hostname_or_ip\>!
The downside of using the NFS gateway, is that you may only use NFS for communicating with Hadoop and not any of the real directories.
From: Marco Shaw [mailto:marco.shaw@gmail.com]
Sent: Friday, December 06, 2013 11:34 AM
To: user
Subject: Re: Writing to remote HDFS using C# on Windows
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com>> wrote:
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>>
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>> wrote:
Hi, All
Is there a way to write files into remote HDFS on Linux using C# on Windows? We want to use HDFS as data storage.
We know there is HDFS java API, but not C#. We tried SAMBA for file sharing and FUSE for mounting HDFS. It worked if we simply copy files to HDFS, but if we open a filestream and write to it, it always throws exceptions.
Best regards!
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
thanks, Marco,
the example you showed still operate on file, not filestream.
2013/12/7 Marco Shaw <ma...@gmail.com>
> Does this help?
>
> http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
>
> I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
> cluster...
>
> Marco
>
>
>
> On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Thanks!
>>
>> I tried WebHDFS, which also work well if I copy local files to HDFS, but
>> still can't find a way to open a filestream, and write to it.
>>
>>
>>
>> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>>
>>> You can try using WebHDFS.
>>>
>>> Thanks,
>>> +Vinod
>>>
>>>
>>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>wrote:
>>>
>>>> Hi, All
>>>>
>>>> Is there a way to write files into remote HDFS on Linux using C# on
>>>> Windows? We want to use HDFS as data storage.
>>>>
>>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>>> HDFS, but if we open a filestream and write to it, it always throws
>>>> exceptions.
>>>>
>>>> Best regards!
>>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
RE: Writing to remote HDFS using C# on Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
If you using HDFS on 2.2, you could use a nfs. More recent versions of Windows include an NFS client. The exclamation mark is used to refer to the root directory of HDFS. For example:
net use * \\hostname_or_ip\<file:///\\hostname_or_ip\>!
The downside of using the NFS gateway, is that you may only use NFS for communicating with Hadoop and not any of the real directories.
From: Marco Shaw [mailto:marco.shaw@gmail.com]
Sent: Friday, December 06, 2013 11:34 AM
To: user
Subject: Re: Writing to remote HDFS using C# on Windows
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com>> wrote:
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>>
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>> wrote:
Hi, All
Is there a way to write files into remote HDFS on Linux using C# on Windows? We want to use HDFS as data storage.
We know there is HDFS java API, but not C#. We tried SAMBA for file sharing and FUSE for mounting HDFS. It worked if we simply copy files to HDFS, but if we open a filestream and write to it, it always throws exceptions.
Best regards!
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: Writing to remote HDFS using C# on Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
If you using HDFS on 2.2, you could use a nfs. More recent versions of Windows include an NFS client. The exclamation mark is used to refer to the root directory of HDFS. For example:
net use * \\hostname_or_ip\<file:///\\hostname_or_ip\>!
The downside of using the NFS gateway, is that you may only use NFS for communicating with Hadoop and not any of the real directories.
From: Marco Shaw [mailto:marco.shaw@gmail.com]
Sent: Friday, December 06, 2013 11:34 AM
To: user
Subject: Re: Writing to remote HDFS using C# on Windows
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com>> wrote:
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>>
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>> wrote:
Hi, All
Is there a way to write files into remote HDFS on Linux using C# on Windows? We want to use HDFS as data storage.
We know there is HDFS java API, but not C#. We tried SAMBA for file sharing and FUSE for mounting HDFS. It worked if we simply copy files to HDFS, but if we open a filestream and write to it, it always throws exceptions.
Best regards!
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: Writing to remote HDFS using C# on Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
If you using HDFS on 2.2, you could use a nfs. More recent versions of Windows include an NFS client. The exclamation mark is used to refer to the root directory of HDFS. For example:
net use * \\hostname_or_ip\<file:///\\hostname_or_ip\>!
The downside of using the NFS gateway, is that you may only use NFS for communicating with Hadoop and not any of the real directories.
From: Marco Shaw [mailto:marco.shaw@gmail.com]
Sent: Friday, December 06, 2013 11:34 AM
To: user
Subject: Re: Writing to remote HDFS using C# on Windows
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com>> wrote:
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>>
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>> wrote:
Hi, All
Is there a way to write files into remote HDFS on Linux using C# on Windows? We want to use HDFS as data storage.
We know there is HDFS java API, but not C#. We tried SAMBA for file sharing and FUSE for mounting HDFS. It worked if we simply copy files to HDFS, but if we open a filestream and write to it, it always throws exceptions.
Best regards!
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
thanks, Marco,
the example you showed still operate on file, not filestream.
2013/12/7 Marco Shaw <ma...@gmail.com>
> Does this help?
>
> http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
>
> I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
> cluster...
>
> Marco
>
>
>
> On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Thanks!
>>
>> I tried WebHDFS, which also work well if I copy local files to HDFS, but
>> still can't find a way to open a filestream, and write to it.
>>
>>
>>
>> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>>
>>> You can try using WebHDFS.
>>>
>>> Thanks,
>>> +Vinod
>>>
>>>
>>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>wrote:
>>>
>>>> Hi, All
>>>>
>>>> Is there a way to write files into remote HDFS on Linux using C# on
>>>> Windows? We want to use HDFS as data storage.
>>>>
>>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>>> HDFS, but if we open a filestream and write to it, it always throws
>>>> exceptions.
>>>>
>>>> Best regards!
>>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
thanks, Marco,
the example you showed still operate on file, not filestream.
2013/12/7 Marco Shaw <ma...@gmail.com>
> Does this help?
>
> http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
>
> I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
> cluster...
>
> Marco
>
>
>
> On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Thanks!
>>
>> I tried WebHDFS, which also work well if I copy local files to HDFS, but
>> still can't find a way to open a filestream, and write to it.
>>
>>
>>
>> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>>
>>> You can try using WebHDFS.
>>>
>>> Thanks,
>>> +Vinod
>>>
>>>
>>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>wrote:
>>>
>>>> Hi, All
>>>>
>>>> Is there a way to write files into remote HDFS on Linux using C# on
>>>> Windows? We want to use HDFS as data storage.
>>>>
>>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>>> HDFS, but if we open a filestream and write to it, it always throws
>>>> exceptions.
>>>>
>>>> Best regards!
>>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
thanks, Marco,
the example you showed still operate on file, not filestream.
2013/12/7 Marco Shaw <ma...@gmail.com>
> Does this help?
>
> http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
>
> I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
> cluster...
>
> Marco
>
>
>
> On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Thanks!
>>
>> I tried WebHDFS, which also work well if I copy local files to HDFS, but
>> still can't find a way to open a filestream, and write to it.
>>
>>
>>
>> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>>
>>> You can try using WebHDFS.
>>>
>>> Thanks,
>>> +Vinod
>>>
>>>
>>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com>wrote:
>>>
>>>> Hi, All
>>>>
>>>> Is there a way to write files into remote HDFS on Linux using C# on
>>>> Windows? We want to use HDFS as data storage.
>>>>
>>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>>> HDFS, but if we open a filestream and write to it, it always throws
>>>> exceptions.
>>>>
>>>> Best regards!
>>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Marco Shaw <ma...@gmail.com>.
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
> Thanks!
>
> I tried WebHDFS, which also work well if I copy local files to HDFS, but
> still can't find a way to open a filestream, and write to it.
>
>
>
> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>
>> You can try using WebHDFS.
>>
>> Thanks,
>> +Vinod
>>
>>
>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>>
>>> Hi, All
>>>
>>> Is there a way to write files into remote HDFS on Linux using C# on
>>> Windows? We want to use HDFS as data storage.
>>>
>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>> HDFS, but if we open a filestream and write to it, it always throws
>>> exceptions.
>>>
>>> Best regards!
>>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Marco Shaw <ma...@gmail.com>.
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
> Thanks!
>
> I tried WebHDFS, which also work well if I copy local files to HDFS, but
> still can't find a way to open a filestream, and write to it.
>
>
>
> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>
>> You can try using WebHDFS.
>>
>> Thanks,
>> +Vinod
>>
>>
>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>>
>>> Hi, All
>>>
>>> Is there a way to write files into remote HDFS on Linux using C# on
>>> Windows? We want to use HDFS as data storage.
>>>
>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>> HDFS, but if we open a filestream and write to it, it always throws
>>> exceptions.
>>>
>>> Best regards!
>>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Marco Shaw <ma...@gmail.com>.
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
> Thanks!
>
> I tried WebHDFS, which also work well if I copy local files to HDFS, but
> still can't find a way to open a filestream, and write to it.
>
>
>
> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>
>> You can try using WebHDFS.
>>
>> Thanks,
>> +Vinod
>>
>>
>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>>
>>> Hi, All
>>>
>>> Is there a way to write files into remote HDFS on Linux using C# on
>>> Windows? We want to use HDFS as data storage.
>>>
>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>> HDFS, but if we open a filestream and write to it, it always throws
>>> exceptions.
>>>
>>> Best regards!
>>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Marco Shaw <ma...@gmail.com>.
Does this help?
http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx
I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight
cluster...
Marco
On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO <ra...@gmail.com> wrote:
> Thanks!
>
> I tried WebHDFS, which also work well if I copy local files to HDFS, but
> still can't find a way to open a filestream, and write to it.
>
>
>
> 2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
>
>> You can try using WebHDFS.
>>
>> Thanks,
>> +Vinod
>>
>>
>> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>>
>>> Hi, All
>>>
>>> Is there a way to write files into remote HDFS on Linux using C# on
>>> Windows? We want to use HDFS as data storage.
>>>
>>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>>> HDFS, but if we open a filestream and write to it, it always throws
>>> exceptions.
>>>
>>> Best regards!
>>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but
still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
> You can try using WebHDFS.
>
> Thanks,
> +Vinod
>
>
> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Hi, All
>>
>> Is there a way to write files into remote HDFS on Linux using C# on
>> Windows? We want to use HDFS as data storage.
>>
>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>> HDFS, but if we open a filestream and write to it, it always throws
>> exceptions.
>>
>> Best regards!
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but
still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
> You can try using WebHDFS.
>
> Thanks,
> +Vinod
>
>
> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Hi, All
>>
>> Is there a way to write files into remote HDFS on Linux using C# on
>> Windows? We want to use HDFS as data storage.
>>
>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>> HDFS, but if we open a filestream and write to it, it always throws
>> exceptions.
>>
>> Best regards!
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but
still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
> You can try using WebHDFS.
>
> Thanks,
> +Vinod
>
>
> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Hi, All
>>
>> Is there a way to write files into remote HDFS on Linux using C# on
>> Windows? We want to use HDFS as data storage.
>>
>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>> HDFS, but if we open a filestream and write to it, it always throws
>> exceptions.
>>
>> Best regards!
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Fengyun RAO <ra...@gmail.com>.
Thanks!
I tried WebHDFS, which also work well if I copy local files to HDFS, but
still can't find a way to open a filestream, and write to it.
2013/12/6 Vinod Kumar Vavilapalli <vi...@hortonworks.com>
> You can try using WebHDFS.
>
> Thanks,
> +Vinod
>
>
> On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
>
>> Hi, All
>>
>> Is there a way to write files into remote HDFS on Linux using C# on
>> Windows? We want to use HDFS as data storage.
>>
>> We know there is HDFS java API, but not C#. We tried SAMBA for file
>> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
>> HDFS, but if we open a filestream and write to it, it always throws
>> exceptions.
>>
>> Best regards!
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
> Hi, All
>
> Is there a way to write files into remote HDFS on Linux using C# on
> Windows? We want to use HDFS as data storage.
>
> We know there is HDFS java API, but not C#. We tried SAMBA for file
> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
> HDFS, but if we open a filestream and write to it, it always throws
> exceptions.
>
> Best regards!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
> Hi, All
>
> Is there a way to write files into remote HDFS on Linux using C# on
> Windows? We want to use HDFS as data storage.
>
> We know there is HDFS java API, but not C#. We tried SAMBA for file
> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
> HDFS, but if we open a filestream and write to it, it always throws
> exceptions.
>
> Best regards!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
> Hi, All
>
> Is there a way to write files into remote HDFS on Linux using C# on
> Windows? We want to use HDFS as data storage.
>
> We know there is HDFS java API, but not C#. We tried SAMBA for file
> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
> HDFS, but if we open a filestream and write to it, it always throws
> exceptions.
>
> Best regards!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Writing to remote HDFS using C# on Windows
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
You can try using WebHDFS.
Thanks,
+Vinod
On Thu, Dec 5, 2013 at 6:04 PM, Fengyun RAO <ra...@gmail.com> wrote:
> Hi, All
>
> Is there a way to write files into remote HDFS on Linux using C# on
> Windows? We want to use HDFS as data storage.
>
> We know there is HDFS java API, but not C#. We tried SAMBA for file
> sharing and FUSE for mounting HDFS. It worked if we simply copy files to
> HDFS, but if we open a filestream and write to it, it always throws
> exceptions.
>
> Best regards!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.