You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Vyacheslav Tutrinov (Jira)" <ji...@apache.org> on 2023/08/22 07:44:00 UTC
[jira] [Created] (HDDS-9196) OzoneManager: NPE on ACLs check in case of multipart upload to EC-bucket
Vyacheslav Tutrinov created HDDS-9196:
-----------------------------------------
Summary: OzoneManager: NPE on ACLs check in case of multipart upload to EC-bucket
Key: HDDS-9196
URL: https://issues.apache.org/jira/browse/HDDS-9196
Project: Apache Ozone
Issue Type: Task
Components: EC, Ozone Manager, Security
Affects Versions: 1.4.0
Reporter: Vyacheslav Tutrinov
ECKeyOutputStream tries to flush data (and allocateBlock) on closing the stream for file's part on multipart uploading. But the user for AllocateBlock request is empty and that leads to NPE on the OM side.
How to reproduce:
{code:bash}
# start cluster
cd ./hadoop-ozone/dist/target/ozone-${version}/compose/ozonesecure
docker-compose up -d --scale datanode=10
#initialize krb ticket
docker exec ozonesecure-httpfs-1 kinit -kt /etc/security/keytabs/testuser.keytab testuser/scm@EXAMPLE.COM
#create bucket
docker exec ozonesecure-httpfs-1 ozone sh bucket create --replication rs-3-2-1024k --type EC s3v/bucket3
#generate file for multipart uploading
docker exec ozonesecure-httpfs-1 bash -c 'head -c 10MB </dev/urandom > /tmp/big_file'
#get s3 secret
docker exec ozonesecure-httpfs-1 ozone s3 getsecret -e
#output of the command to get s3 secret
export AWS_ACCESS_KEY_ID='testuser/scm@EXAMPLE.COM'
export AWS_SECRET_ACCESS_KEY='c358d26fb7201496653c4127250f998df343ac94ffb4d3ce81e658f0aa43160e'
#upload generated file with the secret above
docker exec ozonesecure-httpfs-1 bash -c 'AWS_ACCESS_KEY_ID="testuser/scm@EXAMPLE.COM" AWS_SECRET_ACCESS_KEY="c358d26fb7201496653c4127250f998df343ac94ffb4d3ce81e658f0aa43160e" aws --debug s3 --endpoint http://s3g:9878 cp /tmp/big_file s3://bucket3/big_file_001'
{code}
OM log:
{code:bash}
ozonesecure-om-1 | 2023-08-22 07:27:30,754 [OM StateMachine ApplyTransaction Thread - 0] ERROR ratis.OzoneManagerStateMachine: Terminating with exit status 1: Request cmdType: AllocateBlock
ozonesecure-om-1 | traceID: ""
ozonesecure-om-1 | clientId: "client-C23E4F9517F3"
ozonesecure-om-1 | userInfo {
ozonesecure-om-1 | }
ozonesecure-om-1 | version: 3
ozonesecure-om-1 | allocateBlockRequest {
ozonesecure-om-1 | keyArgs {
ozonesecure-om-1 | volumeName: "s3v"
ozonesecure-om-1 | bucketName: "bucket3"
ozonesecure-om-1 | keyName: "big_file_001"
ozonesecure-om-1 | dataSize: 8388608
ozonesecure-om-1 | type: EC
ozonesecure-om-1 | modificationTime: 1692689250736
ozonesecure-om-1 | ecReplicationConfig {
ozonesecure-om-1 | data: 3
ozonesecure-om-1 | parity: 2
ozonesecure-om-1 | codec: "rs"
ozonesecure-om-1 | ecChunkSize: 1048576
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | clientID: 110932082709233666
ozonesecure-om-1 | excludeList {
ozonesecure-om-1 | }
ozonesecure-om-1 | keyLocation {
ozonesecure-om-1 | blockID {
ozonesecure-om-1 | containerBlockID {
ozonesecure-om-1 | containerID: 2
ozonesecure-om-1 | localID: 111677748019200002
ozonesecure-om-1 | }
ozonesecure-om-1 | blockCommitSequenceId: 0
ozonesecure-om-1 | }
ozonesecure-om-1 | offset: 0
ozonesecure-om-1 | length: 268435456
ozonesecure-om-1 | createVersion: 0
ozonesecure-om-1 | token {
ozonesecure-om-1 | identifier: "\n\002om\022\"conID: 2 locID: 111677748019200002\030\253\253\252\212\2421(\001(\0020\200\200\200\200\001:\025\b\201\206\255\274\267\260\377\343Z\020\244\352\247\263\336\326\247\260\250\001"
ozonesecure-om-1 | password: "qs\002\003G\356U\bmt\024\321\364\312q\3747-_j[\250\266~\032P\035n\277\227\261^"
ozonesecure-om-1 | kind: "HDDS_BLOCK_TOKEN"
ozonesecure-om-1 | service: "conID: 2 locID: 111677748019200002"
ozonesecure-om-1 | }
ozonesecure-om-1 | pipeline {
ozonesecure-om-1 | members {
ozonesecure-om-1 | uuid: "9410a2fa-d984-4e01-ac6c-b19d8c24c505"
ozonesecure-om-1 | ipAddress: "172.28.0.16"
ozonesecure-om-1 | hostName: "ozonesecure-datanode-4.ozonesecure_default"
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "HTTP"
ozonesecure-om-1 | value: 9882
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "CLIENT_RPC"
ozonesecure-om-1 | value: 9864
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "REPLICATION"
ozonesecure-om-1 | value: 9886
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS"
ozonesecure-om-1 | value: 9858
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_ADMIN"
ozonesecure-om-1 | value: 9857
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_SERVER"
ozonesecure-om-1 | value: 9856
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_DATASTREAM"
ozonesecure-om-1 | value: 9855
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "STANDALONE"
ozonesecure-om-1 | value: 9859
ozonesecure-om-1 | }
ozonesecure-om-1 | certSerialId: "163414755675861"
ozonesecure-om-1 | networkName: "9410a2fa-d984-4e01-ac6c-b19d8c24c505"
ozonesecure-om-1 | networkLocation: "/default-rack"
ozonesecure-om-1 | persistedOpStateExpiry: 0
ozonesecure-om-1 | uuid128 {
ozonesecure-om-1 | mostSigBits: -7777537358193996287
ozonesecure-om-1 | leastSigBits: -6022243311481993979
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | members {
ozonesecure-om-1 | uuid: "0803b812-f280-4b19-b594-04e9abd54953"
ozonesecure-om-1 | ipAddress: "172.28.0.15"
ozonesecure-om-1 | hostName: "ozonesecure-datanode-1.ozonesecure_default"
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "HTTP"
ozonesecure-om-1 | value: 9882
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "CLIENT_RPC"
ozonesecure-om-1 | value: 9864
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "REPLICATION"
ozonesecure-om-1 | value: 9886
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS"
ozonesecure-om-1 | value: 9858
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_ADMIN"
ozonesecure-om-1 | value: 9857
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_SERVER"
ozonesecure-om-1 | value: 9856
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_DATASTREAM"
ozonesecure-om-1 | value: 9855
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "STANDALONE"
ozonesecure-om-1 | value: 9859
ozonesecure-om-1 | }
ozonesecure-om-1 | certSerialId: "163413142286488"
ozonesecure-om-1 | networkName: "0803b812-f280-4b19-b594-04e9abd54953"
ozonesecure-om-1 | networkLocation: "/default-rack"
ozonesecure-om-1 | persistedOpStateExpiry: 0
ozonesecure-om-1 | uuid128 {
ozonesecure-om-1 | mostSigBits: 577507568750971673
ozonesecure-om-1 | leastSigBits: -5362655854634645165
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | members {
ozonesecure-om-1 | uuid: "47a1d06f-47bd-422c-917f-ff85e73668f0"
ozonesecure-om-1 | ipAddress: "172.28.0.14"
ozonesecure-om-1 | hostName: "ozonesecure-datanode-6.ozonesecure_default"
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "HTTP"
ozonesecure-om-1 | value: 9882
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "CLIENT_RPC"
ozonesecure-om-1 | value: 9864
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "REPLICATION"
ozonesecure-om-1 | value: 9886
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS"
ozonesecure-om-1 | value: 9858
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_ADMIN"
ozonesecure-om-1 | value: 9857
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_SERVER"
ozonesecure-om-1 | value: 9856
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_DATASTREAM"
ozonesecure-om-1 | value: 9855
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "STANDALONE"
ozonesecure-om-1 | value: 9859
ozonesecure-om-1 | }
ozonesecure-om-1 | certSerialId: "163413226290763"
ozonesecure-om-1 | networkName: "47a1d06f-47bd-422c-917f-ff85e73668f0"
ozonesecure-om-1 | networkLocation: "/default-rack"
ozonesecure-om-1 | persistedOpStateExpiry: 0
ozonesecure-om-1 | uuid128 {
ozonesecure-om-1 | mostSigBits: 5161635824306831916
ozonesecure-om-1 | leastSigBits: -7962364665592911632
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | members {
ozonesecure-om-1 | uuid: "00737d34-3f63-44c9-b07a-820225dd909c"
ozonesecure-om-1 | ipAddress: "172.28.0.10"
ozonesecure-om-1 | hostName: "ozonesecure-datanode-5.ozonesecure_default"
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "HTTP"
ozonesecure-om-1 | value: 9882
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "CLIENT_RPC"
ozonesecure-om-1 | value: 9864
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "REPLICATION"
ozonesecure-om-1 | value: 9886
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS"
ozonesecure-om-1 | value: 9858
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_ADMIN"
ozonesecure-om-1 | value: 9857
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_SERVER"
ozonesecure-om-1 | value: 9856
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_DATASTREAM"
ozonesecure-om-1 | value: 9855
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "STANDALONE"
ozonesecure-om-1 | value: 9859
ozonesecure-om-1 | }
ozonesecure-om-1 | certSerialId: "163414521241922"
ozonesecure-om-1 | networkName: "00737d34-3f63-44c9-b07a-820225dd909c"
ozonesecure-om-1 | networkLocation: "/default-rack"
ozonesecure-om-1 | persistedOpStateExpiry: 0
ozonesecure-om-1 | uuid128 {
ozonesecure-om-1 | mostSigBits: 32507285676967113
ozonesecure-om-1 | leastSigBits: -5730124630138711908
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | members {
ozonesecure-om-1 | uuid: "d21b0395-cc79-4b4d-9073-dbf158184ecf"
ozonesecure-om-1 | ipAddress: "172.28.0.13"
ozonesecure-om-1 | hostName: "ozonesecure-datanode-10.ozonesecure_default"
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "HTTP"
ozonesecure-om-1 | value: 9882
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "CLIENT_RPC"
ozonesecure-om-1 | value: 9864
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "REPLICATION"
ozonesecure-om-1 | value: 9886
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS"
ozonesecure-om-1 | value: 9858
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_ADMIN"
ozonesecure-om-1 | value: 9857
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_SERVER"
ozonesecure-om-1 | value: 9856
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "RATIS_DATASTREAM"
ozonesecure-om-1 | value: 9855
ozonesecure-om-1 | }
ozonesecure-om-1 | ports {
ozonesecure-om-1 | name: "STANDALONE"
ozonesecure-om-1 | value: 9859
ozonesecure-om-1 | }
ozonesecure-om-1 | certSerialId: "163414923048335"
ozonesecure-om-1 | networkName: "d21b0395-cc79-4b4d-9073-dbf158184ecf"
ozonesecure-om-1 | networkLocation: "/default-rack"
ozonesecure-om-1 | persistedOpStateExpiry: 0
ozonesecure-om-1 | uuid128 {
ozonesecure-om-1 | mostSigBits: -3307045559457985715
ozonesecure-om-1 | leastSigBits: -8037839080314614065
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | state: PIPELINE_OPEN
ozonesecure-om-1 | type: EC
ozonesecure-om-1 | id {
ozonesecure-om-1 | id: "c513c8db-ce0d-40fa-82a4-e093303aa6e9"
ozonesecure-om-1 | uuid128 {
ozonesecure-om-1 | mostSigBits: -4245829177299877638
ozonesecure-om-1 | leastSigBits: -9032848029824407831
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | leaderID: ""
ozonesecure-om-1 | creationTimeStamp: 1692689250701
ozonesecure-om-1 | memberReplicaIndexes: 1
ozonesecure-om-1 | memberReplicaIndexes: 2
ozonesecure-om-1 | memberReplicaIndexes: 3
ozonesecure-om-1 | memberReplicaIndexes: 4
ozonesecure-om-1 | memberReplicaIndexes: 5
ozonesecure-om-1 | ecReplicationConfig {
ozonesecure-om-1 | data: 3
ozonesecure-om-1 | parity: 2
ozonesecure-om-1 | codec: "rs"
ozonesecure-om-1 | ecChunkSize: 1048576
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | partNumber: 0
ozonesecure-om-1 | }
ozonesecure-om-1 | }
ozonesecure-om-1 | failed with exception
ozonesecure-om-1 | java.lang.NullPointerException
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.OzoneAclUtils.isOwner(OzoneAclUtils.java:190)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.OzoneAclUtils.checkAllAcls(OzoneAclUtils.java:86)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.request.OMClientRequest.checkAcls(OMClientRequest.java:346)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.request.OMClientRequest.checkAcls(OMClientRequest.java:219)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.request.key.OMKeyRequest.checkKeyAcls(OMKeyRequest.java:391)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.request.key.OMKeyRequest.checkKeyAclsInOpenKeyTable(OMKeyRequest.java:441)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.request.key.OMAllocateBlockRequestWithFSO.validateAndUpdateCache(OMAllocateBlockRequestWithFSO.java:121)
ozonesecure-om-1 | at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:375)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.runCommand(OzoneManagerStateMachine.java:568)
ozonesecure-om-1 | at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.lambda$1(OzoneManagerStateMachine.java:359)
ozonesecure-om-1 | at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
ozonesecure-om-1 | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
ozonesecure-om-1 | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
ozonesecure-om-1 | at java.base/java.lang.Thread.run(Thread.java:829)
ozonesecure-om-1 | 2023-08-22 07:27:30,758 [shutdown-hook-0] INFO om.OzoneManagerStarter: SHUTDOWN_MSG:
ozonesecure-om-1 | /************************************************************
ozonesecure-om-1 | SHUTDOWN_MSG: Shutting down OzoneManager at om/172.28.0.6
ozonesecure-om-1 | ************************************************************/
ozonesecure-om-1 | 2023-08-22 07:27:30,759 [shutdown-hook-0] INFO om.OzoneManager: om1[om:9862]: Stopping Ozone Manager
ozonesecure-om-1 | 2023-08-22 07:27:30,760 [shutdown-hook-0] INFO ipc.Server: Stopping server on 9862
ozonesecure-om-1 | 2023-08-22 07:27:30,766 [IPC Server listener on 9862] INFO ipc.Server: Stopping IPC Server listener on 9862
ozonesecure-om-1 | 2023-08-22 07:27:30,766 [IPC Server Responder] INFO ipc.Server: Stopping IPC Server Responder
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org