You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "Kevin Risden (Jira)" <ji...@apache.org> on 2020/02/27 19:43:00 UTC
[jira] [Comment Edited] (KNOX-2139) Can not handle 8GB file when
using webhdfs
[ https://issues.apache.org/jira/browse/KNOX-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046935#comment-17046935 ]
Kevin Risden edited comment on KNOX-2139 at 2/27/20 7:42 PM:
-------------------------------------------------------------
I wasn't able to reproduce this using the latest builds of CDH and Knox. You can see the full debug output below. I took the exact dd command you shared. [~seanlook] let me know if there is something I missed.
Here is the output from generating the file, uploading with curl, and then checking the file:
{code:java}
[root@krisden-1 ~]# dd if=/dev/zero of=data4gb.bin bs=1024 count=4194304
[root@krisden-1 ~]# ls -l data4gb.bin
-rw-r--r-- 1 root root 4294967296 Feb 27 19:30 data4gb.bin
[root@krisden-1 ~]# time curl -v -L --location-trusted -iku systest:cloudera -X PUT -T data4gb.bin "https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true"
* About to connect() to krisden-1.krisden.root.hwx.site port 8443 (#0)
* Trying 172.27.157.134...
* Connected to krisden-1.krisden.root.hwx.site (172.27.157.134) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
* Server certificate:
* subject: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Feb 27 19:08:52 2020 GMT
* expire date: Feb 26 19:08:52 2021 GMT
* common name: localhost
* issuer: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* Server auth using Basic with user 'systest'
> PUT /gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true HTTP/1.1
> Authorization: Basic c3lzdGVzdDpjbG91ZGVyYQ==
> User-Agent: curl/7.29.0
> Host: krisden-1.krisden.root.hwx.site:8443
> Accept: */*
> Content-Length: 4294967296
> Expect: 100-continue
>
< HTTP/1.1 307 Temporary Redirect
HTTP/1.1 307 Temporary Redirect
< Date: Thu, 27 Feb 2020 19:33:59 GMT
Date: Thu, 27 Feb 2020 19:33:59 GMT
< Set-Cookie: KNOXSESSIONID=node088lkub8iek2zntuskte28b0l7.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
Set-Cookie: KNOXSESSIONID=node088lkub8iek2zntuskte28b0l7.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:01 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:01 GMT
< Date: Thu, 27 Feb 2020 19:34:01 GMT
Date: Thu, 27 Feb 2020 19:34:01 GMT
< Cache-Control: no-cache
Cache-Control: no-cache
< Expires: Thu, 27 Feb 2020 19:34:01 GMT
Expires: Thu, 27 Feb 2020 19:34:01 GMT
< Date: Thu, 27 Feb 2020 19:34:01 GMT
Date: Thu, 27 Feb 2020 19:34:01 GMT
< Pragma: no-cache
Pragma: no-cache
< X-Content-Type-Options: nosniff
X-Content-Type-Options: nosniff
< X-FRAME-OPTIONS: SAMEORIGIN
X-FRAME-OPTIONS: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
X-XSS-Protection: 1; mode=block
< Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o
Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o
< Content-Type: application/octet-stream
Content-Type: application/octet-stream
< Content-Length: 0
Content-Length: 0
< Connection: close
Connection: close
< Server: Jetty(9.4.26.v20200117)
Server: Jetty(9.4.26.v20200117)
<
* Closing connection 0
* Issue another request to this URL: 'https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o'
* About to connect() to krisden-1.krisden.root.hwx.site port 8443 (#1)
* Trying 172.27.157.134...
* Connected to krisden-1.krisden.root.hwx.site (172.27.157.134) port 8443 (#1)
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
* Server certificate:
* subject: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Feb 27 19:08:52 2020 GMT
* expire date: Feb 26 19:08:52 2021 GMT
* common name: localhost
* issuer: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* Server auth using Basic with user 'systest'
> PUT /gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o HTTP/1.1
> Authorization: Basic c3lzdGVzdDpjbG91ZGVyYQ==
> User-Agent: curl/7.29.0
> Host: krisden-1.krisden.root.hwx.site:8443
> Accept: */*
> Content-Length: 4294967296
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 201 Created
HTTP/1.1 201 Created
< Date: Thu, 27 Feb 2020 19:34:01 GMT
Date: Thu, 27 Feb 2020 19:34:01 GMT
< Set-Cookie: KNOXSESSIONID=node0154w9874r0wirsx6wzzdltiaf8.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
Set-Cookie: KNOXSESSIONID=node0154w9874r0wirsx6wzzdltiaf8.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:02 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:02 GMT
< Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin
Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin
< Access-Control-Allow-Origin: *
Access-Control-Allow-Origin: *
< Connection: close
Connection: close
< Server: Jetty(9.4.26.v20200117)
Server: Jetty(9.4.26.v20200117)
<
* Closing connection 1
real 0m57.417s
user 0m50.007s
sys 0m3.412s
[root@krisden-1 ~]# curl -iku systest:cloudera "https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS"
HTTP/1.1 200 OK
Date: Thu, 27 Feb 2020 19:35:25 GMT
Set-Cookie: KNOXSESSIONID=node0exdmqea8q9nvmv95tn4y1l229.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:35:27 GMT
Date: Thu, 27 Feb 2020 19:35:27 GMT
Cache-Control: no-cache
Expires: Thu, 27 Feb 2020 19:35:27 GMT
Date: Thu, 27 Feb 2020 19:35:27 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-FRAME-OPTIONS: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Type: application/json;charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(9.4.26.v20200117)
{"FileStatuses":{"FileStatus":[{"accessTime":1582832043103,"blockSize":134217728,"childrenNum":0,"fileId":16409,"group":"supergroup","length":4294967296,"modificationTime":1582832097041,"owner":"systest","pathSuffix":"","permission":"644","replication":2,"storagePolicy":0,"type":"FILE"}]}}
{code}
Here is the gateway.log and gateway-log.log entries:
{code:java}
tail -f /var/log/knox/gateway/gateway{,-audit}.log
20/02/27 19:33:59 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS||||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|unavailable|Request method: PUT
==> /var/log/knox/gateway/gateway.log <==
2020-02-27 19:34:01,287 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(129)) - Computed roles/groups: [systest] for principal: systest
==> /var/log/knox/gateway/gateway-audit.log <==
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|success|
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|success|Groups: [systest]
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true&doAs=systest|unavailable|Request method: PUT
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true&doAs=systest|success|Response status: 307
20/02/27 19:34:01 |||audit|172.27.157.134|WEBHDFS|systest|||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|success|Response status: 307
20/02/27 19:34:01 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS||||access|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|unavailable|Request method: PUT
==> /var/log/knox/gateway/gateway.log <==
2020-02-27 19:34:02,963 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(129)) - Computed roles/groups: [systest] for principal: systest
==> /var/log/knox/gateway/gateway-audit.log <==
20/02/27 19:34:02 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|success|
20/02/27 19:34:02 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|success|Groups: [systest]
20/02/27 19:34:03 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-1.krisden.root.hwx.site:1006/webhdfs/v1/tmp/data4gb.bin?op=CREATE&delegation=JgAHc3lzdGVzdAdzeXN0ZXN0BGtub3iKAXCIJD1SigFwrDDBUgMCFNTiE-tvHMdSRP-vZGzBuBFFuhl1EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjcuMTcuMTQwOjgwMjA&namenoderpcaddress=krisden-2.krisden.root.hwx.site:8020&createflag&createparent=true&overwrite=true&doAs=systest|unavailable|Request method: PUT
20/02/27 19:34:57 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-1.krisden.root.hwx.site:1006/webhdfs/v1/tmp/data4gb.bin?op=CREATE&delegation=JgAHc3lzdGVzdAdzeXN0ZXN0BGtub3iKAXCIJD1SigFwrDDBUgMCFNTiE-tvHMdSRP-vZGzBuBFFuhl1EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjcuMTcuMTQwOjgwMjA&namenoderpcaddress=krisden-2.krisden.root.hwx.site:8020&createflag&createparent=true&overwrite=true&doAs=systest|success|Response status: 201
20/02/27 19:34:57 |||audit|172.27.157.134|WEBHDFS|systest|||access|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|success|Response status: 201
20/02/27 19:35:25 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS||||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|unavailable|Request method: GET
==> /var/log/knox/gateway/gateway.log <==
2020-02-27 19:35:27,348 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(129)) - Computed roles/groups: [systest] for principal: systest
==> /var/log/knox/gateway/gateway-audit.log <==
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|success|
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|success|Groups: [systest]
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS&doAs=systest|unavailable|Request method: GET
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS&doAs=systest|success|Response status: 200
20/02/27 19:35:27 |||audit|172.27.157.134|WEBHDFS|systest|||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|success|Response status: 200
{code}
here are the HDFS for namenode and datanode logs:
{code:java}
tail -f /var/log/hadoop-hdfs/hadoop-*.log.out
2020-02-27 19:34:03,106 INFO datanode.webhdfs: 172.27.157.134 PUT /webhdfs/v1/tmp/data4gb.bin?op=CREATE&delegation=JgAHc3lzdGVzdAdzeXN0ZXN0BGtub3iKAXCIJD1SigFwrDDBUgMCFNTiE-tvHMdSRP-vZGzBuBFFuhl1EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjcuMTcuMTQwOjgwMjA&namenoderpcaddress=krisden-2.krisden.root.hwx.site%3A8020&createflag&createparent=true&overwrite=true&doAs=systest 201
2020-02-27 19:34:03,119 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741875_1051 src: /172.27.157.134:48742 dest: /172.27.157.134:1004
2020-02-27 19:34:05,148 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48742, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741875_1051, duration(ns): 2022194633
2020-02-27 19:34:05,148 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741875_1051, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:05,153 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741876_1052 src: /172.27.157.134:48820 dest: /172.27.157.134:1004
2020-02-27 19:34:07,687 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Slow BlockReceiver write data to disk cost:1495ms (threshold=300ms), volume=file:/dataroot/ycloud/dfs/dn/, blockId=1073741876
2020-02-27 19:34:08,088 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48820, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741876_1052, duration(ns): 2927140351
2020-02-27 19:34:08,088 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741876_1052, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:08,093 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741877_1053 src: /172.27.157.134:48870 dest: /172.27.157.134:1004
2020-02-27 19:34:09,248 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48870, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741877_1053, duration(ns): 1150253024
2020-02-27 19:34:09,249 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741877_1053, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:09,254 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741878_1054 src: /172.27.157.134:48886 dest: /172.27.157.134:1004
2020-02-27 19:34:10,864 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48886, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741878_1054, duration(ns): 1605156011
2020-02-27 19:34:10,865 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741878_1054, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:10,869 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741879_1055 src: /172.27.157.134:48950 dest: /172.27.157.134:1004
2020-02-27 19:34:12,628 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48950, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741879_1055, duration(ns): 1737380987
2020-02-27 19:34:12,628 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741879_1055, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:12,633 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741880_1056 src: /172.27.157.134:48996 dest: /172.27.157.134:1004
2020-02-27 19:34:14,357 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48996, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741880_1056, duration(ns): 1716139176
2020-02-27 19:34:14,359 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741880_1056, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:14,366 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741881_1057 src: /172.27.157.134:49010 dest: /172.27.157.134:1004
2020-02-27 19:34:16,250 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49010, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741881_1057, duration(ns): 1878801659
2020-02-27 19:34:16,250 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741881_1057, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:16,255 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741882_1058 src: /172.27.157.134:49062 dest: /172.27.157.134:1004
2020-02-27 19:34:17,858 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49062, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741882_1058, duration(ns): 1598023448
2020-02-27 19:34:17,858 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741882_1058, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:17,862 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741883_1059 src: /172.27.157.134:49084 dest: /172.27.157.134:1004
2020-02-27 19:34:19,706 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49084, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741883_1059, duration(ns): 1839957750
2020-02-27 19:34:19,707 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741883_1059, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:19,712 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741884_1060 src: /172.27.157.134:49148 dest: /172.27.157.134:1004
2020-02-27 19:34:21,503 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49148, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741884_1060, duration(ns): 1785462837
2020-02-27 19:34:21,503 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741884_1060, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:21,508 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741885_1061 src: /172.27.157.134:49212 dest: /172.27.157.134:1004
2020-02-27 19:34:23,090 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49212, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741885_1061, duration(ns): 1578039078
2020-02-27 19:34:23,091 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741885_1061, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:23,095 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741886_1062 src: /172.27.157.134:49248 dest: /172.27.157.134:1004
2020-02-27 19:34:24,818 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49248, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741886_1062, duration(ns): 1717976045
2020-02-27 19:34:24,819 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741886_1062, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:24,826 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741887_1063 src: /172.27.157.134:49280 dest: /172.27.157.134:1004
2020-02-27 19:34:25,031 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741888_1064 src: /172.27.19.74:59634 dest: /172.27.157.134:1004
2020-02-27 19:34:25,037 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.19.74:59634, dest: /172.27.157.134:1004, bytes: 56, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1019846644_92, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741888_1064, duration(ns): 3748314
2020-02-27 19:34:25,037 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741888_1064, type=LAST_IN_PIPELINE terminating
2020-02-27 19:34:26,546 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49280, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741887_1063, duration(ns): 1716290970
2020-02-27 19:34:26,547 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741887_1063, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:26,551 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741889_1065 src: /172.27.157.134:49370 dest: /172.27.157.134:1004
2020-02-27 19:34:28,239 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49370, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741889_1065, duration(ns): 1682174223
2020-02-27 19:34:28,240 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741889_1065, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:28,245 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741890_1066 src: /172.27.157.134:49458 dest: /172.27.157.134:1004
2020-02-27 19:34:29,983 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49458, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741890_1066, duration(ns): 1515800730
2020-02-27 19:34:29,984 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741890_1066, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:29,988 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741891_1067 src: /172.27.157.134:49480 dest: /172.27.157.134:1004
2020-02-27 19:34:31,356 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49480, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741891_1067, duration(ns): 1363992759
2020-02-27 19:34:31,357 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741891_1067, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:31,361 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741892_1068 src: /172.27.157.134:49514 dest: /172.27.157.134:1004
2020-02-27 19:34:32,993 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49514, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741892_1068, duration(ns): 1626512624
2020-02-27 19:34:32,993 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741892_1068, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:32,998 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741893_1069 src: /172.27.157.134:49552 dest: /172.27.157.134:1004
2020-02-27 19:34:34,559 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49552, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741893_1069, duration(ns): 1538187653
2020-02-27 19:34:34,559 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741893_1069, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:34,564 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741894_1070 src: /172.27.157.134:49580 dest: /172.27.157.134:1004
2020-02-27 19:34:36,189 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49580, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741894_1070, duration(ns): 1620523026
2020-02-27 19:34:36,190 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741894_1070, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:36,195 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741895_1071 src: /172.27.157.134:49598 dest: /172.27.157.134:1004
2020-02-27 19:34:37,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49598, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741895_1071, duration(ns): 1651915520
2020-02-27 19:34:37,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741895_1071, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:37,856 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741896_1072 src: /172.27.157.134:49620 dest: /172.27.157.134:1004
2020-02-27 19:34:39,562 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49620, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741896_1072, duration(ns): 1700211605
2020-02-27 19:34:39,562 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741896_1072, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:39,567 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741897_1073 src: /172.27.157.134:49652 dest: /172.27.157.134:1004
2020-02-27 19:34:41,145 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49652, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741897_1073, duration(ns): 1573072407
2020-02-27 19:34:41,145 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741897_1073, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:41,149 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741898_1074 src: /172.27.157.134:49684 dest: /172.27.157.134:1004
2020-02-27 19:34:42,678 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49684, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741898_1074, duration(ns): 1523669554
2020-02-27 19:34:42,678 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741898_1074, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:42,730 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741899_1075 src: /172.27.157.134:49702 dest: /172.27.157.134:1004
2020-02-27 19:34:44,376 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49702, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741899_1075, duration(ns): 1641124137
2020-02-27 19:34:44,376 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741899_1075, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:44,382 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741900_1076 src: /172.27.157.134:49716 dest: /172.27.157.134:1004
2020-02-27 19:34:45,915 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49716, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741900_1076, duration(ns): 1526075414
2020-02-27 19:34:45,915 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741900_1076, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:45,921 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741901_1077 src: /172.27.157.134:49742 dest: /172.27.157.134:1004
2020-02-27 19:34:47,464 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49742, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741901_1077, duration(ns): 1536210043
2020-02-27 19:34:47,465 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741901_1077, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:47,471 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741902_1078 src: /172.27.157.134:49754 dest: /172.27.157.134:1004
2020-02-27 19:34:49,034 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49754, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741902_1078, duration(ns): 1557962322
2020-02-27 19:34:49,035 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741902_1078, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:49,039 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741903_1079 src: /172.27.157.134:49784 dest: /172.27.157.134:1004
2020-02-27 19:34:50,607 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49784, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741903_1079, duration(ns): 1562231251
2020-02-27 19:34:50,607 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741903_1079, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:50,612 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741904_1080 src: /172.27.157.134:49822 dest: /172.27.157.134:1004
2020-02-27 19:34:52,295 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49822, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741904_1080, duration(ns): 1677649256
2020-02-27 19:34:52,295 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741904_1080, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:52,299 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741905_1081 src: /172.27.157.134:49858 dest: /172.27.157.134:1004
2020-02-27 19:34:53,831 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49858, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741905_1081, duration(ns): 1526820834
2020-02-27 19:34:53,832 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741905_1081, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:53,836 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741906_1082 src: /172.27.157.134:49884 dest: /172.27.157.134:1004
2020-02-27 19:34:55,350 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49884, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741906_1082, duration(ns): 1509678275
2020-02-27 19:34:55,351 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741906_1082, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:55,355 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741907_1083 src: /172.27.157.134:49896 dest: /172.27.157.134:1004
2020-02-27 19:34:57,039 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49896, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741907_1083, duration(ns): 1677542303
2020-02-27 19:34:57,040 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741907_1083, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:35:55,043 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741908_1084 src: /172.27.19.74:33438 dest: /172.27.157.134:1004
2020-02-27 19:35:55,050 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.19.74:33438, dest: /172.27.157.134:1004, bytes: 56, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1397610945_92, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741908_1084, duration(ns): 4351484
2020-02-27 19:35:55,050 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741908_1084, type=LAST_IN_PIPELINE terminating
2020-02-27 19:37:25,021 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741909_1085 src: /172.27.19.74:35442 dest: /172.27.157.134:1004
{code}
was (Author: risdenk):
I wasn't able to reproduce this:
Here is the output from generating the file, uploading with curl, and then checking the file:
{code:java}
[root@krisden-1 ~]# dd if=/dev/zero of=data4gb.bin bs=1024 count=4194304
[root@krisden-1 ~]# ls -l data4gb.bin
-rw-r--r-- 1 root root 4294967296 Feb 27 19:30 data4gb.bin
[root@krisden-1 ~]# time curl -v -L --location-trusted -iku systest:cloudera -X PUT -T data4gb.bin "https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true"
* About to connect() to krisden-1.krisden.root.hwx.site port 8443 (#0)
* Trying 172.27.157.134...
* Connected to krisden-1.krisden.root.hwx.site (172.27.157.134) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
* Server certificate:
* subject: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Feb 27 19:08:52 2020 GMT
* expire date: Feb 26 19:08:52 2021 GMT
* common name: localhost
* issuer: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* Server auth using Basic with user 'systest'
> PUT /gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true HTTP/1.1
> Authorization: Basic c3lzdGVzdDpjbG91ZGVyYQ==
> User-Agent: curl/7.29.0
> Host: krisden-1.krisden.root.hwx.site:8443
> Accept: */*
> Content-Length: 4294967296
> Expect: 100-continue
>
< HTTP/1.1 307 Temporary Redirect
HTTP/1.1 307 Temporary Redirect
< Date: Thu, 27 Feb 2020 19:33:59 GMT
Date: Thu, 27 Feb 2020 19:33:59 GMT
< Set-Cookie: KNOXSESSIONID=node088lkub8iek2zntuskte28b0l7.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
Set-Cookie: KNOXSESSIONID=node088lkub8iek2zntuskte28b0l7.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:01 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:01 GMT
< Date: Thu, 27 Feb 2020 19:34:01 GMT
Date: Thu, 27 Feb 2020 19:34:01 GMT
< Cache-Control: no-cache
Cache-Control: no-cache
< Expires: Thu, 27 Feb 2020 19:34:01 GMT
Expires: Thu, 27 Feb 2020 19:34:01 GMT
< Date: Thu, 27 Feb 2020 19:34:01 GMT
Date: Thu, 27 Feb 2020 19:34:01 GMT
< Pragma: no-cache
Pragma: no-cache
< X-Content-Type-Options: nosniff
X-Content-Type-Options: nosniff
< X-FRAME-OPTIONS: SAMEORIGIN
X-FRAME-OPTIONS: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
X-XSS-Protection: 1; mode=block
< Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o
Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o
< Content-Type: application/octet-stream
Content-Type: application/octet-stream
< Content-Length: 0
Content-Length: 0
< Connection: close
Connection: close
< Server: Jetty(9.4.26.v20200117)
Server: Jetty(9.4.26.v20200117)
<
* Closing connection 0
* Issue another request to this URL: 'https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o'
* About to connect() to krisden-1.krisden.root.hwx.site port 8443 (#1)
* Trying 172.27.157.134...
* Connected to krisden-1.krisden.root.hwx.site (172.27.157.134) port 8443 (#1)
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
* Server certificate:
* subject: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Feb 27 19:08:52 2020 GMT
* expire date: Feb 26 19:08:52 2021 GMT
* common name: localhost
* issuer: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* Server auth using Basic with user 'systest'
> PUT /gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o HTTP/1.1
> Authorization: Basic c3lzdGVzdDpjbG91ZGVyYQ==
> User-Agent: curl/7.29.0
> Host: krisden-1.krisden.root.hwx.site:8443
> Accept: */*
> Content-Length: 4294967296
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 201 Created
HTTP/1.1 201 Created
< Date: Thu, 27 Feb 2020 19:34:01 GMT
Date: Thu, 27 Feb 2020 19:34:01 GMT
< Set-Cookie: KNOXSESSIONID=node0154w9874r0wirsx6wzzdltiaf8.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
Set-Cookie: KNOXSESSIONID=node0154w9874r0wirsx6wzzdltiaf8.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:02 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:34:02 GMT
< Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin
Location: https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin
< Access-Control-Allow-Origin: *
Access-Control-Allow-Origin: *
< Connection: close
Connection: close
< Server: Jetty(9.4.26.v20200117)
Server: Jetty(9.4.26.v20200117)
<
* Closing connection 1
real 0m57.417s
user 0m50.007s
sys 0m3.412s
[root@krisden-1 ~]# curl -iku systest:cloudera "https://krisden-1.krisden.root.hwx.site:8443/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS"
HTTP/1.1 200 OK
Date: Thu, 27 Feb 2020 19:35:25 GMT
Set-Cookie: KNOXSESSIONID=node0exdmqea8q9nvmv95tn4y1l229.node0; Path=/gateway/cdp-proxy-api; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/cdp-proxy-api; Max-Age=0; Expires=Wed, 26-Feb-2020 19:35:27 GMT
Date: Thu, 27 Feb 2020 19:35:27 GMT
Cache-Control: no-cache
Expires: Thu, 27 Feb 2020 19:35:27 GMT
Date: Thu, 27 Feb 2020 19:35:27 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-FRAME-OPTIONS: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Type: application/json;charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(9.4.26.v20200117)
{"FileStatuses":{"FileStatus":[{"accessTime":1582832043103,"blockSize":134217728,"childrenNum":0,"fileId":16409,"group":"supergroup","length":4294967296,"modificationTime":1582832097041,"owner":"systest","pathSuffix":"","permission":"644","replication":2,"storagePolicy":0,"type":"FILE"}]}}
{code}
Here is the gateway.log and gateway-log.log entries:
{code:java}
tail -f /var/log/knox/gateway/gateway{,-audit}.log
20/02/27 19:33:59 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS||||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|unavailable|Request method: PUT
==> /var/log/knox/gateway/gateway.log <==
2020-02-27 19:34:01,287 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(129)) - Computed roles/groups: [systest] for principal: systest
==> /var/log/knox/gateway/gateway-audit.log <==
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|success|
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|success|Groups: [systest]
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true&doAs=systest|unavailable|Request method: PUT
20/02/27 19:34:01 ||743bcba5-87f7-4a50-be96-8c60125a2cf4|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true&doAs=systest|success|Response status: 307
20/02/27 19:34:01 |||audit|172.27.157.134|WEBHDFS|systest|||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=CREATE&overwrite=true|success|Response status: 307
20/02/27 19:34:01 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS||||access|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|unavailable|Request method: PUT
==> /var/log/knox/gateway/gateway.log <==
2020-02-27 19:34:02,963 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(129)) - Computed roles/groups: [systest] for principal: systest
==> /var/log/knox/gateway/gateway-audit.log <==
20/02/27 19:34:02 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|success|
20/02/27 19:34:02 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|success|Groups: [systest]
20/02/27 19:34:03 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-1.krisden.root.hwx.site:1006/webhdfs/v1/tmp/data4gb.bin?op=CREATE&delegation=JgAHc3lzdGVzdAdzeXN0ZXN0BGtub3iKAXCIJD1SigFwrDDBUgMCFNTiE-tvHMdSRP-vZGzBuBFFuhl1EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjcuMTcuMTQwOjgwMjA&namenoderpcaddress=krisden-2.krisden.root.hwx.site:8020&createflag&createparent=true&overwrite=true&doAs=systest|unavailable|Request method: PUT
20/02/27 19:34:57 ||64aa844f-9bf8-442f-9769-6ec696ecffa0|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-1.krisden.root.hwx.site:1006/webhdfs/v1/tmp/data4gb.bin?op=CREATE&delegation=JgAHc3lzdGVzdAdzeXN0ZXN0BGtub3iKAXCIJD1SigFwrDDBUgMCFNTiE-tvHMdSRP-vZGzBuBFFuhl1EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjcuMTcuMTQwOjgwMjA&namenoderpcaddress=krisden-2.krisden.root.hwx.site:8020&createflag&createparent=true&overwrite=true&doAs=systest|success|Response status: 201
20/02/27 19:34:57 |||audit|172.27.157.134|WEBHDFS|systest|||access|uri|/gateway/cdp-proxy-api/webhdfs/data/v1/webhdfs/v1/tmp/data4gb.bin?_=AAAACAAAABAAAAFAatQtHK8HEcpSPDwp1-h7AwwEzALze5sjuzRhWSvgE0TlN8buceW74_Tp8DQjTo-ZR28wa3uNKMloFIvktGANHmCWXxtyjalGCm0K1k7bTR2K8TYeSEo6YhVOJ8ARgrEidawb-umTuSGXiFSf0rh1Fd15ngNVR9FOcoXVtN6XHBKS8eA7BT40F01n5EERe5Qtwm4hxgYfxrLNKwbSKvTROIZsRb6o4lzSIXdcJGouFk7tKXxGuQ_l7a-NS3wlMMSJze6ur6AlJQT-myABo1I6vAPE59P49705W-bDRd1IAhgC8d89NqhIGcwvD21UfojnW8grTEnlkcOy2-4yjq7KC-ndxc1q1TgvJKAxEKdapDCsrir-pgBjSyPRCXzVOH1oAsOa0skU28atPD76RvNu8sGjAOqQeSCKbd6QXtNwu49E1a29bv4TvLFkanht7JKQLi6kR8knx_o|success|Response status: 201
20/02/27 19:35:25 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS||||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|unavailable|Request method: GET
==> /var/log/knox/gateway/gateway.log <==
2020-02-27 19:35:27,348 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(129)) - Computed roles/groups: [systest] for principal: systest
==> /var/log/knox/gateway/gateway-audit.log <==
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|success|
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||authentication|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|success|Groups: [systest]
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS&doAs=systest|unavailable|Request method: GET
20/02/27 19:35:27 ||f2e2d7cd-e619-4e73-b6dc-3217d6f63935|audit|172.27.157.134|WEBHDFS|systest|||dispatch|uri|http://krisden-2.krisden.root.hwx.site:20101/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS&doAs=systest|success|Response status: 200
20/02/27 19:35:27 |||audit|172.27.157.134|WEBHDFS|systest|||access|uri|/gateway/cdp-proxy-api/webhdfs/v1/tmp/data4gb.bin?op=LISTSTATUS|success|Response status: 200
{code}
here are the HDFS for namenode and datanode logs:
{code:java}
tail -f /var/log/hadoop-hdfs/hadoop-*.log.out
2020-02-27 19:34:03,106 INFO datanode.webhdfs: 172.27.157.134 PUT /webhdfs/v1/tmp/data4gb.bin?op=CREATE&delegation=JgAHc3lzdGVzdAdzeXN0ZXN0BGtub3iKAXCIJD1SigFwrDDBUgMCFNTiE-tvHMdSRP-vZGzBuBFFuhl1EldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjcuMTcuMTQwOjgwMjA&namenoderpcaddress=krisden-2.krisden.root.hwx.site%3A8020&createflag&createparent=true&overwrite=true&doAs=systest 201
2020-02-27 19:34:03,119 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741875_1051 src: /172.27.157.134:48742 dest: /172.27.157.134:1004
2020-02-27 19:34:05,148 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48742, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741875_1051, duration(ns): 2022194633
2020-02-27 19:34:05,148 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741875_1051, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:05,153 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741876_1052 src: /172.27.157.134:48820 dest: /172.27.157.134:1004
2020-02-27 19:34:07,687 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Slow BlockReceiver write data to disk cost:1495ms (threshold=300ms), volume=file:/dataroot/ycloud/dfs/dn/, blockId=1073741876
2020-02-27 19:34:08,088 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48820, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741876_1052, duration(ns): 2927140351
2020-02-27 19:34:08,088 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741876_1052, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:08,093 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741877_1053 src: /172.27.157.134:48870 dest: /172.27.157.134:1004
2020-02-27 19:34:09,248 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48870, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741877_1053, duration(ns): 1150253024
2020-02-27 19:34:09,249 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741877_1053, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:09,254 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741878_1054 src: /172.27.157.134:48886 dest: /172.27.157.134:1004
2020-02-27 19:34:10,864 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48886, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741878_1054, duration(ns): 1605156011
2020-02-27 19:34:10,865 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741878_1054, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:10,869 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741879_1055 src: /172.27.157.134:48950 dest: /172.27.157.134:1004
2020-02-27 19:34:12,628 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48950, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741879_1055, duration(ns): 1737380987
2020-02-27 19:34:12,628 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741879_1055, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:12,633 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741880_1056 src: /172.27.157.134:48996 dest: /172.27.157.134:1004
2020-02-27 19:34:14,357 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:48996, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741880_1056, duration(ns): 1716139176
2020-02-27 19:34:14,359 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741880_1056, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:14,366 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741881_1057 src: /172.27.157.134:49010 dest: /172.27.157.134:1004
2020-02-27 19:34:16,250 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49010, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741881_1057, duration(ns): 1878801659
2020-02-27 19:34:16,250 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741881_1057, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:16,255 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741882_1058 src: /172.27.157.134:49062 dest: /172.27.157.134:1004
2020-02-27 19:34:17,858 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49062, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741882_1058, duration(ns): 1598023448
2020-02-27 19:34:17,858 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741882_1058, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:17,862 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741883_1059 src: /172.27.157.134:49084 dest: /172.27.157.134:1004
2020-02-27 19:34:19,706 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49084, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741883_1059, duration(ns): 1839957750
2020-02-27 19:34:19,707 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741883_1059, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:19,712 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741884_1060 src: /172.27.157.134:49148 dest: /172.27.157.134:1004
2020-02-27 19:34:21,503 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49148, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741884_1060, duration(ns): 1785462837
2020-02-27 19:34:21,503 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741884_1060, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:21,508 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741885_1061 src: /172.27.157.134:49212 dest: /172.27.157.134:1004
2020-02-27 19:34:23,090 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49212, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741885_1061, duration(ns): 1578039078
2020-02-27 19:34:23,091 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741885_1061, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:23,095 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741886_1062 src: /172.27.157.134:49248 dest: /172.27.157.134:1004
2020-02-27 19:34:24,818 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49248, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741886_1062, duration(ns): 1717976045
2020-02-27 19:34:24,819 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741886_1062, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:24,826 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741887_1063 src: /172.27.157.134:49280 dest: /172.27.157.134:1004
2020-02-27 19:34:25,031 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741888_1064 src: /172.27.19.74:59634 dest: /172.27.157.134:1004
2020-02-27 19:34:25,037 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.19.74:59634, dest: /172.27.157.134:1004, bytes: 56, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1019846644_92, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741888_1064, duration(ns): 3748314
2020-02-27 19:34:25,037 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741888_1064, type=LAST_IN_PIPELINE terminating
2020-02-27 19:34:26,546 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49280, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741887_1063, duration(ns): 1716290970
2020-02-27 19:34:26,547 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741887_1063, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:26,551 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741889_1065 src: /172.27.157.134:49370 dest: /172.27.157.134:1004
2020-02-27 19:34:28,239 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49370, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741889_1065, duration(ns): 1682174223
2020-02-27 19:34:28,240 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741889_1065, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:28,245 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741890_1066 src: /172.27.157.134:49458 dest: /172.27.157.134:1004
2020-02-27 19:34:29,983 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49458, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741890_1066, duration(ns): 1515800730
2020-02-27 19:34:29,984 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741890_1066, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:29,988 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741891_1067 src: /172.27.157.134:49480 dest: /172.27.157.134:1004
2020-02-27 19:34:31,356 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49480, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741891_1067, duration(ns): 1363992759
2020-02-27 19:34:31,357 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741891_1067, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:31,361 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741892_1068 src: /172.27.157.134:49514 dest: /172.27.157.134:1004
2020-02-27 19:34:32,993 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49514, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741892_1068, duration(ns): 1626512624
2020-02-27 19:34:32,993 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741892_1068, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:32,998 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741893_1069 src: /172.27.157.134:49552 dest: /172.27.157.134:1004
2020-02-27 19:34:34,559 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49552, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741893_1069, duration(ns): 1538187653
2020-02-27 19:34:34,559 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741893_1069, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:34,564 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741894_1070 src: /172.27.157.134:49580 dest: /172.27.157.134:1004
2020-02-27 19:34:36,189 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49580, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741894_1070, duration(ns): 1620523026
2020-02-27 19:34:36,190 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741894_1070, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:36,195 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741895_1071 src: /172.27.157.134:49598 dest: /172.27.157.134:1004
2020-02-27 19:34:37,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49598, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741895_1071, duration(ns): 1651915520
2020-02-27 19:34:37,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741895_1071, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:37,856 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741896_1072 src: /172.27.157.134:49620 dest: /172.27.157.134:1004
2020-02-27 19:34:39,562 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49620, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741896_1072, duration(ns): 1700211605
2020-02-27 19:34:39,562 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741896_1072, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:39,567 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741897_1073 src: /172.27.157.134:49652 dest: /172.27.157.134:1004
2020-02-27 19:34:41,145 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49652, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741897_1073, duration(ns): 1573072407
2020-02-27 19:34:41,145 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741897_1073, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:41,149 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741898_1074 src: /172.27.157.134:49684 dest: /172.27.157.134:1004
2020-02-27 19:34:42,678 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49684, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741898_1074, duration(ns): 1523669554
2020-02-27 19:34:42,678 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741898_1074, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:42,730 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741899_1075 src: /172.27.157.134:49702 dest: /172.27.157.134:1004
2020-02-27 19:34:44,376 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49702, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741899_1075, duration(ns): 1641124137
2020-02-27 19:34:44,376 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741899_1075, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:44,382 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741900_1076 src: /172.27.157.134:49716 dest: /172.27.157.134:1004
2020-02-27 19:34:45,915 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49716, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741900_1076, duration(ns): 1526075414
2020-02-27 19:34:45,915 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741900_1076, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:45,921 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741901_1077 src: /172.27.157.134:49742 dest: /172.27.157.134:1004
2020-02-27 19:34:47,464 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49742, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741901_1077, duration(ns): 1536210043
2020-02-27 19:34:47,465 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741901_1077, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:47,471 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741902_1078 src: /172.27.157.134:49754 dest: /172.27.157.134:1004
2020-02-27 19:34:49,034 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49754, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741902_1078, duration(ns): 1557962322
2020-02-27 19:34:49,035 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741902_1078, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:49,039 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741903_1079 src: /172.27.157.134:49784 dest: /172.27.157.134:1004
2020-02-27 19:34:50,607 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49784, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741903_1079, duration(ns): 1562231251
2020-02-27 19:34:50,607 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741903_1079, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:50,612 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741904_1080 src: /172.27.157.134:49822 dest: /172.27.157.134:1004
2020-02-27 19:34:52,295 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49822, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741904_1080, duration(ns): 1677649256
2020-02-27 19:34:52,295 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741904_1080, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:52,299 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741905_1081 src: /172.27.157.134:49858 dest: /172.27.157.134:1004
2020-02-27 19:34:53,831 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49858, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741905_1081, duration(ns): 1526820834
2020-02-27 19:34:53,832 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741905_1081, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:53,836 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741906_1082 src: /172.27.157.134:49884 dest: /172.27.157.134:1004
2020-02-27 19:34:55,350 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49884, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741906_1082, duration(ns): 1509678275
2020-02-27 19:34:55,351 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741906_1082, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:34:55,355 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741907_1083 src: /172.27.157.134:49896 dest: /172.27.157.134:1004
2020-02-27 19:34:57,039 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.157.134:49896, dest: /172.27.157.134:1004, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1756780349_78, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741907_1083, duration(ns): 1677542303
2020-02-27 19:34:57,040 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741907_1083, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=1:[172.27.19.74:1004] terminating
2020-02-27 19:35:55,043 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741908_1084 src: /172.27.19.74:33438 dest: /172.27.157.134:1004
2020-02-27 19:35:55,050 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /172.27.19.74:33438, dest: /172.27.157.134:1004, bytes: 56, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1397610945_92, offset: 0, srvID: 244ead31-a3bb-4a2e-8c73-820bbad3b024, blockid: BP-507181519-172.27.17.140-1582830548203:blk_1073741908_1084, duration(ns): 4351484
2020-02-27 19:35:55,050 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-507181519-172.27.17.140-1582830548203:blk_1073741908_1084, type=LAST_IN_PIPELINE terminating
2020-02-27 19:37:25,021 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-507181519-172.27.17.140-1582830548203:blk_1073741909_1085 src: /172.27.19.74:35442 dest: /172.27.157.134:1004
{code}
> Can not handle 8GB file when using webhdfs
> ------------------------------------------
>
> Key: KNOX-2139
> URL: https://issues.apache.org/jira/browse/KNOX-2139
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.1.0, 1.2.0
> Reporter: Sean Chow
> Priority: Critical
>
> I use knox with webhdfs for a long time, andI uprade my knox version from 0.8 to 1.2 recent days. It's really strange that knox can't handle file with size *8589934592 bytes* when I upload my splited file to hdfs.
> It's easy to reproduce and both knox1.1 and 1.2 have this issue. But is works fine in knox0.8.
> Any error log found in gateway.log? No, all logs is clean. From the client side (curl), I saw the the url is redirected correctly and failed with {{curl: (55) Send failure: Connection reset by peer}} or {{curl: (55) Send failure: Broken pipe}}
> I'm sure my network is ok. Any files with other size(smaller or larger) can be upload successfully.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)