You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Ted Dunning <td...@veoh.com> on 2008/01/30 22:30:21 UTC
Low complexity way to write a file to hdfs?
I am looking for a way for scripts to write data to HDFS without having to
install anything.
The /data and /listPaths URL's on the nameserver are ideal for reading
files, but I can't find anything comparable to write files.
Am I missing something?
If not, I think I will file a JIRA and make /data accept POST events. If
anybody has an opinion about that, please let me know (or put a comment on
the JIRA, if and when).
Re: Low complexity way to write a file to hdfs?
Posted by Doug Cutting <cu...@apache.org>.
Ted Dunning wrote:
> Don't know.
>
> I just build a simple patch against the trunk and cloned the UGI stuff from
> the doGet method.
>
> Will that work?
It should. You'll have to specify the user & groups in the query
string. Looking at the code, it looks like this should be something
like "&ugi=user,group1,group2;".
Doug
Re: Low complexity way to write a file to hdfs?
Posted by Ted Dunning <td...@veoh.com>.
On 1/30/08 3:04 PM, "Arun C Murthy" <ac...@yahoo-inc.com> wrote:
>
> On Jan 30, 2008, at 1:30 PM, Ted Dunning wrote:
>
>>
>> Am I missing something?
>>
>
> Umm... how does it interact with HDFS permissions coming in 0.16.0?
Don't know.
I just build a simple patch against the trunk and cloned the UGI stuff from
the doGet method.
Will that work?
Re: Low complexity way to write a file to hdfs?
Posted by Arun C Murthy <ac...@yahoo-inc.com>.
On Jan 30, 2008, at 1:30 PM, Ted Dunning wrote:
>
> I am looking for a way for scripts to write data to HDFS without
> having to
> install anything.
>
> The /data and /listPaths URL's on the nameserver are ideal for reading
> files, but I can't find anything comparable to write files.
>
> Am I missing something?
>
> If not, I think I will file a JIRA and make /data accept POST
> events. If
> anybody has an opinion about that, please let me know (or put a
> comment on
> the JIRA, if and when).
>
Umm... how does it interact with HDFS permissions coming in 0.16.0?
Arun
>
Re: Low complexity way to write a file to hdfs?
Posted by Jason Venner <ja...@attributor.com>.
As of the last version, of this great tool, that I have loaded, file
write was not yet enabled.
Read works well.
Raghu Angadi wrote:
>
> You could take look at fuse plugin in for HDFS. Then client does not
> even need a web browser.
>
> Raghu.
>
> Ted Dunning wrote:
>> I am looking for a way for scripts to write data to HDFS without
>> having to
>> install anything.
>>
>> The /data and /listPaths URL's on the nameserver are ideal for reading
>> files, but I can't find anything comparable to write files.
>>
>> Am I missing something?
>>
>> If not, I think I will file a JIRA and make /data accept POST
>> events. If
>> anybody has an opinion about that, please let me know (or put a
>> comment on
>> the JIRA, if and when).
>>
>>
>
--
Jason Venner
Attributor - Publish with Confidence <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers, contact if interested
Re: Low complexity way to write a file to hdfs?
Posted by Raghu Angadi <ra...@yahoo-inc.com>.
You could take look at fuse plugin in for HDFS. Then client does not
even need a web browser.
Raghu.
Ted Dunning wrote:
> I am looking for a way for scripts to write data to HDFS without having to
> install anything.
>
> The /data and /listPaths URL's on the nameserver are ideal for reading
> files, but I can't find anything comparable to write files.
>
> Am I missing something?
>
> If not, I think I will file a JIRA and make /data accept POST events. If
> anybody has an opinion about that, please let me know (or put a comment on
> the JIRA, if and when).
>
>
Re: Low complexity way to write a file to hdfs?
Posted by Michael Bieniosek <mi...@powerset.com>.
There is a webdav servlet in HADOOP-496 that works for read/write/delete. I've only tested it with the Mac OSX Finder client though.
-Michael
On 1/30/08 2:18 PM, "Ted Dunning" <td...@veoh.com> wrote:
Might work.
On 1/30/08 2:00 PM, "Vadim Zaliva" <kr...@gmail.com> wrote:
> On Jan 30, 2008, at 13:57, Jason Venner wrote:
>
> I think somebody mentioned WebDAV support. That would work for me,
> so I can PUT files.
>
> Vadim
>
>> I suppose we could add a feature to the hdfs web ui to allow
>> uploading files.
>>
>> Ted Dunning wrote:
>>> I am looking for a way for scripts to write data to HDFS without
>>> having to
>>> install anything.
>>>
>>> The /data and /listPaths URL's on the nameserver are ideal for
>>> reading
>>> files, but I can't find anything comparable to write files.
>>>
>>> Am I missing something?
>>>
>>> If not, I think I will file a JIRA and make /data accept POST
>>> events. If
>>> anybody has an opinion about that, please let me know (or put a
>>> comment on
>>> the JIRA, if and when).
>>>
>>>
>>>
>>
>> --
>> Jason Venner
>> Attributor - Publish with Confidence <http://www.attributor.com/><http://www.attributor.com/>
>> Attributor is hiring Hadoop Wranglers, contact if interested
>
Re: Low complexity way to write a file to hdfs?
Posted by Ted Dunning <td...@veoh.com>.
Might work.
On 1/30/08 2:00 PM, "Vadim Zaliva" <kr...@gmail.com> wrote:
> On Jan 30, 2008, at 13:57, Jason Venner wrote:
>
> I think somebody mentioned WebDAV support. That would work for me,
> so I can PUT files.
>
> Vadim
>
>> I suppose we could add a feature to the hdfs web ui to allow
>> uploading files.
>>
>> Ted Dunning wrote:
>>> I am looking for a way for scripts to write data to HDFS without
>>> having to
>>> install anything.
>>>
>>> The /data and /listPaths URL's on the nameserver are ideal for
>>> reading
>>> files, but I can't find anything comparable to write files.
>>>
>>> Am I missing something?
>>>
>>> If not, I think I will file a JIRA and make /data accept POST
>>> events. If
>>> anybody has an opinion about that, please let me know (or put a
>>> comment on
>>> the JIRA, if and when).
>>>
>>>
>>>
>>
>> --
>> Jason Venner
>> Attributor - Publish with Confidence <http://www.attributor.com/>
>> Attributor is hiring Hadoop Wranglers, contact if interested
>
Re: Low complexity way to write a file to hdfs?
Posted by Vadim Zaliva <kr...@gmail.com>.
On Jan 30, 2008, at 13:57, Jason Venner wrote:
I think somebody mentioned WebDAV support. That would work for me,
so I can PUT files.
Vadim
> I suppose we could add a feature to the hdfs web ui to allow
> uploading files.
>
> Ted Dunning wrote:
>> I am looking for a way for scripts to write data to HDFS without
>> having to
>> install anything.
>>
>> The /data and /listPaths URL's on the nameserver are ideal for
>> reading
>> files, but I can't find anything comparable to write files.
>>
>> Am I missing something?
>>
>> If not, I think I will file a JIRA and make /data accept POST
>> events. If
>> anybody has an opinion about that, please let me know (or put a
>> comment on
>> the JIRA, if and when).
>>
>>
>>
>
> --
> Jason Venner
> Attributor - Publish with Confidence <http://www.attributor.com/>
> Attributor is hiring Hadoop Wranglers, contact if interested
Re: Low complexity way to write a file to hdfs?
Posted by Ted Dunning <td...@veoh.com>.
That's what I am about to do.
On 1/30/08 1:57 PM, "Jason Venner" <ja...@attributor.com> wrote:
> I suppose we could add a feature to the hdfs web ui to allow uploading
> files.
>
> Ted Dunning wrote:
>> I am looking for a way for scripts to write data to HDFS without having to
>> install anything.
>>
>> The /data and /listPaths URL's on the nameserver are ideal for reading
>> files, but I can't find anything comparable to write files.
>>
>> Am I missing something?
>>
>> If not, I think I will file a JIRA and make /data accept POST events. If
>> anybody has an opinion about that, please let me know (or put a comment on
>> the JIRA, if and when).
>>
>>
>>
Re: Low complexity way to write a file to hdfs?
Posted by Jason Venner <ja...@attributor.com>.
I suppose we could add a feature to the hdfs web ui to allow uploading
files.
Ted Dunning wrote:
> I am looking for a way for scripts to write data to HDFS without having to
> install anything.
>
> The /data and /listPaths URL's on the nameserver are ideal for reading
> files, but I can't find anything comparable to write files.
>
> Am I missing something?
>
> If not, I think I will file a JIRA and make /data accept POST events. If
> anybody has an opinion about that, please let me know (or put a comment on
> the JIRA, if and when).
>
>
>
--
Jason Venner
Attributor - Publish with Confidence <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers, contact if interested
Re: Low complexity way to write a file to hdfs?
Posted by Ted Dunning <td...@veoh.com>.
http://<namenode-and-port/data/<file-path-in-hadoop>
I also have code written to allow posting to the same URL for file creation,
but haven't had time to get it to actually work (the posting to the URL
doesn't call doPost for some reason).
If somebody else has time to track down the (probably obvious to anybody but
me) issue, I would be happy to file a Jira and post a patch against 15.1 and
trunk for their reference.
On 2/6/08 9:52 AM, "C G" <pa...@yahoo.com> wrote:
> Ted:
>
> I am curious about how you read files without installing anything. Can you
> share your wisdom?
>
> Thanks,
> C G
>
> Ted Dunning <td...@veoh.com> wrote:
>
> I am looking for a way for scripts to write data to HDFS without having to
> install anything.
>
> The /data and /listPaths URL's on the nameserver are ideal for reading
> files, but I can't find anything comparable to write files.
>
> Am I missing something?
>
> If not, I think I will file a JIRA and make /data accept POST events. If
> anybody has an opinion about that, please let me know (or put a comment on
> the JIRA, if and when).
>
>
>
>
>
> ---------------------------------
> Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it
> now.
Re: Low complexity way to write a file to hdfs?
Posted by C G <pa...@yahoo.com>.
Ted:
I am curious about how you read files without installing anything. Can you share your wisdom?
Thanks,
C G
Ted Dunning <td...@veoh.com> wrote:
I am looking for a way for scripts to write data to HDFS without having to
install anything.
The /data and /listPaths URL's on the nameserver are ideal for reading
files, but I can't find anything comparable to write files.
Am I missing something?
If not, I think I will file a JIRA and make /data accept POST events. If
anybody has an opinion about that, please let me know (or put a comment on
the JIRA, if and when).
---------------------------------
Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now.