You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by 小学园PHP <xx...@qq.com> on 2013/04/08 08:02:57 UTC

I want to call HDFS REST api to upload a file using httplib.

I want to call HDFS REST api to upload a file using httplib.
 
My program created the file, but no content is in it.
 
=====================================================
 
Here is my code:
 import httplib conn=httplib.HTTPConnection("localhost:50070") conn.request("PUT","/webhdfs/v1/levi/4?op=CREATE") res=conn.getresponse() print res.status,res.reason conn.close() conn=httplib.HTTPConnection("localhost:50075") conn.connect() conn.putrequest("PUT","/webhdfs/v1/levi/4?op=CREATE&user.name=levi") conn.endheaders() a_file=open("/home/levi/4","rb") a_file.seek(0) data=a_file.read() conn.send(data) res=conn.getresponse() print res.status,res.reason conn.close() 
==================================================
 
Here is the return:
    
307 TEMPORARY_REDIRECT   201 Created
  
=========================================================
 
OK, the file was created, but no content was sent.
 
When I comment the #conn.send(data), the result is the same, still no content.
 
Maybe the file read or the send is wrong, not sure.
 
Do you know how this happened?

RES: I want to call HDFS REST api to upload a file using httplib.

Posted by MARCOS MEDRADO RUBINELLI <ma...@buscapecompany.com>.
On your first call, Hadoop will return a URL pointing to a datanode in the Location header of the 307 response. On your second call, you have to use that URL instead of constructing your own. You can see the specific documentation here:
http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE

Regards,
Marcos
________________________________



I want to call HDFS REST api to upload a file using httplib.

My program created the file, but no content is in it.

=====================================================

Here is my code:

import httplib

conn=httplib.HTTPConnection("localhost:50070")
conn.request("PUT","/webhdfs/v1/levi/4?op=CREATE")
res=conn.getresponse()
print res.status,res.reason
conn.close()

conn=httplib.HTTPConnection("localhost:50075")
conn.connect()
conn.putrequest("PUT","/webhdfs/v1/levi/4?op=CREATE&user.name=levi")
conn.endheaders()
a_file=open("/home/levi/4","rb")
a_file.seek(0)
data=a_file.read()
conn.send(data)
res=conn.getresponse()
print res.status,res.reason
conn.close()

==================================================

Here is the return:

307 TEMPORARY_REDIRECT 201 Created

=========================================================

OK, the file was created, but no content was sent.

When I comment the #conn.send(data), the result is the same, still no content.

Maybe the file read or the send is wrong, not sure.

Do you know how this happened?

RES: I want to call HDFS REST api to upload a file using httplib.

Posted by MARCOS MEDRADO RUBINELLI <ma...@buscapecompany.com>.
On your first call, Hadoop will return a URL pointing to a datanode in the Location header of the 307 response. On your second call, you have to use that URL instead of constructing your own. You can see the specific documentation here:
http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE

Regards,
Marcos
________________________________



I want to call HDFS REST api to upload a file using httplib.

My program created the file, but no content is in it.

=====================================================

Here is my code:

import httplib

conn=httplib.HTTPConnection("localhost:50070")
conn.request("PUT","/webhdfs/v1/levi/4?op=CREATE")
res=conn.getresponse()
print res.status,res.reason
conn.close()

conn=httplib.HTTPConnection("localhost:50075")
conn.connect()
conn.putrequest("PUT","/webhdfs/v1/levi/4?op=CREATE&user.name=levi")
conn.endheaders()
a_file=open("/home/levi/4","rb")
a_file.seek(0)
data=a_file.read()
conn.send(data)
res=conn.getresponse()
print res.status,res.reason
conn.close()

==================================================

Here is the return:

307 TEMPORARY_REDIRECT 201 Created

=========================================================

OK, the file was created, but no content was sent.

When I comment the #conn.send(data), the result is the same, still no content.

Maybe the file read or the send is wrong, not sure.

Do you know how this happened?

RES: I want to call HDFS REST api to upload a file using httplib.

Posted by MARCOS MEDRADO RUBINELLI <ma...@buscapecompany.com>.
On your first call, Hadoop will return a URL pointing to a datanode in the Location header of the 307 response. On your second call, you have to use that URL instead of constructing your own. You can see the specific documentation here:
http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE

Regards,
Marcos
________________________________



I want to call HDFS REST api to upload a file using httplib.

My program created the file, but no content is in it.

=====================================================

Here is my code:

import httplib

conn=httplib.HTTPConnection("localhost:50070")
conn.request("PUT","/webhdfs/v1/levi/4?op=CREATE")
res=conn.getresponse()
print res.status,res.reason
conn.close()

conn=httplib.HTTPConnection("localhost:50075")
conn.connect()
conn.putrequest("PUT","/webhdfs/v1/levi/4?op=CREATE&user.name=levi")
conn.endheaders()
a_file=open("/home/levi/4","rb")
a_file.seek(0)
data=a_file.read()
conn.send(data)
res=conn.getresponse()
print res.status,res.reason
conn.close()

==================================================

Here is the return:

307 TEMPORARY_REDIRECT 201 Created

=========================================================

OK, the file was created, but no content was sent.

When I comment the #conn.send(data), the result is the same, still no content.

Maybe the file read or the send is wrong, not sure.

Do you know how this happened?

RES: I want to call HDFS REST api to upload a file using httplib.

Posted by MARCOS MEDRADO RUBINELLI <ma...@buscapecompany.com>.
On your first call, Hadoop will return a URL pointing to a datanode in the Location header of the 307 response. On your second call, you have to use that URL instead of constructing your own. You can see the specific documentation here:
http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE

Regards,
Marcos
________________________________



I want to call HDFS REST api to upload a file using httplib.

My program created the file, but no content is in it.

=====================================================

Here is my code:

import httplib

conn=httplib.HTTPConnection("localhost:50070")
conn.request("PUT","/webhdfs/v1/levi/4?op=CREATE")
res=conn.getresponse()
print res.status,res.reason
conn.close()

conn=httplib.HTTPConnection("localhost:50075")
conn.connect()
conn.putrequest("PUT","/webhdfs/v1/levi/4?op=CREATE&user.name=levi")
conn.endheaders()
a_file=open("/home/levi/4","rb")
a_file.seek(0)
data=a_file.read()
conn.send(data)
res=conn.getresponse()
print res.status,res.reason
conn.close()

==================================================

Here is the return:

307 TEMPORARY_REDIRECT 201 Created

=========================================================

OK, the file was created, but no content was sent.

When I comment the #conn.send(data), the result is the same, still no content.

Maybe the file read or the send is wrong, not sure.

Do you know how this happened?