You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by Nuwan Galkissa Mudalige <nm...@virtusa.com.INVALID> on 2022/01/27 01:57:38 UTC
Big File Upload to AEM Cloud
Hi Team,
While uploading big file to AEM Cloud (more than 10MB) it will be chunking and send to a blob storage location by multiple PUT requests.
There are 3 main requests for this file upload, 1 . upload initiate, 2. PUT request which share the file to blob storage and 3. Upload completion request
Jmeter script is working fine up to 10MB file since no chunking happen till that and only one PUT request made. For this PUT request follow the URI return by initiate upload request. Like that if the original file to be split initiate requests returns multiple URIs accordingly which have to be used for PUT requests.
But our problem is while uploading big files that will be chunking by client side and send through multiple PUT requests instead single request. We can capture unique URIs for following PUT request but so, we are not able to handle this particular behavior (chunking) at this moment in Jmeter.
It would be a great help if you can share your valuable inputs with us.
Regards,
Nuwan
________________________________
"Virtusa is a worldwide provider of digital business strategy, digital engineering, and information technology (IT) services and solutions, serving Global 2000 companies across multiple industries. We help clients change, disrupt, and unlock new value with unmatched speed."
________________________________
This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
________________________________
Re: Big File Upload to AEM Cloud
Posted by Kiran Badi <ki...@yahoo.com.INVALID>.
Pls Google for resumable uploads. I know gcp/aws provides api for resumable uploads. Guess Adobe should also have one for this. Once you have this then built custom jar and hook it with jmeter to run tests.
Sent from Yahoo Mail for iPad
On Sunday, January 30, 2022, 11:06 PM, Nuwan Galkissa Mudalige <nm...@virtusa.com.INVALID> wrote:
Hi Felix,
Thanks for your response.
Somehow we were able to manage upto a certain level where sending already splitted files from original instead doing real splitting what does browser do as Jmeter doesn't have a kind of logic.
After splitting client send a block size of requests in parallel (6 request per block) so here we use parallel controller to manage those requests. There is another challenge to measure the response time that parallel controller gives you cumulative number instead parallel completion timing. So we need to find the highest response time of particular request in each block then to be calculated sum from all blocks. Since this a sub part of main upload user transaction again we need to calculate final response time considering above calculated value and rest of part response time in the same transaction.
If you see a simple calculation mechanism for the above or a different approach for this entire things would be great. Thank you !!
Regards,
Nuwan
-----Original Message-----
From: Felix Schumacher <fe...@internetallee.de>
Sent: Sunday, January 30, 2022 6:17 AM
To: user@jmeter.apache.org
Subject: Re: Big File Upload to AEM Cloud
Hi Nuwan,
I have no experience with AEM Cloud, so I might not be able to help you, but looking at
https://medium.com/adobetech/introduction-to-aem-as-a-cloud-service-asset-upload-http-api-fc560ffd7221
it seems, that the description is probably for the same use case, as yours.
Sadly, the splitting is not described there, but you said, you are able to capture the requests (from a real client?).
Can you verify, if the splitted files are splitted only, or has there been done some special handling to the data?
Have a look at the headers, how is the file name (part) encoded?
Perhaps you could show the headers for the splitted requests (be sure to obfuscate authentication data).
Felix
Am 27.01.22 um 02:57 schrieb Nuwan Galkissa Mudalige:
> Hi Team,
>
> While uploading big file to AEM Cloud (more than 10MB) it will be chunking and send to a blob storage location by multiple PUT requests.
> There are 3 main requests for this file upload, 1 . upload initiate,
> 2. PUT request which share the file to blob storage and 3. Upload completion request Jmeter script is working fine up to 10MB file since no chunking happen till that and only one PUT request made. For this PUT request follow the URI return by initiate upload request. Like that if the original file to be split initiate requests returns multiple URIs accordingly which have to be used for PUT requests.
>
> But our problem is while uploading big files that will be chunking by client side and send through multiple PUT requests instead single request. We can capture unique URIs for following PUT request but so, we are not able to handle this particular behavior (chunking) at this moment in Jmeter.
>
> It would be a great help if you can share your valuable inputs with us.
>
> Regards,
> Nuwan
>
> ________________________________
>
> "Virtusa is a worldwide provider of digital business strategy, digital engineering, and information technology (IT) services and solutions, serving Global 2000 companies across multiple industries. We help clients change, disrupt, and unlock new value with unmatched speed."
>
> ________________________________
>
> This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
>
> ________________________________
>
________________________________
"Virtusa is a worldwide provider of digital business strategy, digital engineering, and information technology (IT) services and solutions, serving Global 2000 companies across multiple industries. We help clients change, disrupt, and unlock new value with unmatched speed."
________________________________
This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
________________________________
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
RE: Big File Upload to AEM Cloud
Posted by Nuwan Galkissa Mudalige <nm...@virtusa.com.INVALID>.
Hi Felix,
Thanks for your response.
Somehow we were able to manage upto a certain level where sending already splitted files from original instead doing real splitting what does browser do as Jmeter doesn't have a kind of logic.
After splitting client send a block size of requests in parallel (6 request per block) so here we use parallel controller to manage those requests. There is another challenge to measure the response time that parallel controller gives you cumulative number instead parallel completion timing. So we need to find the highest response time of particular request in each block then to be calculated sum from all blocks. Since this a sub part of main upload user transaction again we need to calculate final response time considering above calculated value and rest of part response time in the same transaction.
If you see a simple calculation mechanism for the above or a different approach for this entire things would be great. Thank you !!
Regards,
Nuwan
-----Original Message-----
From: Felix Schumacher <fe...@internetallee.de>
Sent: Sunday, January 30, 2022 6:17 AM
To: user@jmeter.apache.org
Subject: Re: Big File Upload to AEM Cloud
Hi Nuwan,
I have no experience with AEM Cloud, so I might not be able to help you, but looking at
https://medium.com/adobetech/introduction-to-aem-as-a-cloud-service-asset-upload-http-api-fc560ffd7221
it seems, that the description is probably for the same use case, as yours.
Sadly, the splitting is not described there, but you said, you are able to capture the requests (from a real client?).
Can you verify, if the splitted files are splitted only, or has there been done some special handling to the data?
Have a look at the headers, how is the file name (part) encoded?
Perhaps you could show the headers for the splitted requests (be sure to obfuscate authentication data).
Felix
Am 27.01.22 um 02:57 schrieb Nuwan Galkissa Mudalige:
> Hi Team,
>
> While uploading big file to AEM Cloud (more than 10MB) it will be chunking and send to a blob storage location by multiple PUT requests.
> There are 3 main requests for this file upload, 1 . upload initiate,
> 2. PUT request which share the file to blob storage and 3. Upload completion request Jmeter script is working fine up to 10MB file since no chunking happen till that and only one PUT request made. For this PUT request follow the URI return by initiate upload request. Like that if the original file to be split initiate requests returns multiple URIs accordingly which have to be used for PUT requests.
>
> But our problem is while uploading big files that will be chunking by client side and send through multiple PUT requests instead single request. We can capture unique URIs for following PUT request but so, we are not able to handle this particular behavior (chunking) at this moment in Jmeter.
>
> It would be a great help if you can share your valuable inputs with us.
>
> Regards,
> Nuwan
>
> ________________________________
>
> "Virtusa is a worldwide provider of digital business strategy, digital engineering, and information technology (IT) services and solutions, serving Global 2000 companies across multiple industries. We help clients change, disrupt, and unlock new value with unmatched speed."
>
> ________________________________
>
> This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
>
> ________________________________
>
________________________________
"Virtusa is a worldwide provider of digital business strategy, digital engineering, and information technology (IT) services and solutions, serving Global 2000 companies across multiple industries. We help clients change, disrupt, and unlock new value with unmatched speed."
________________________________
This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
________________________________
Re: Big File Upload to AEM Cloud
Posted by Felix Schumacher <fe...@internetallee.de>.
Hi Nuwan,
I have no experience with AEM Cloud, so I might not be able to help you,
but looking at
https://medium.com/adobetech/introduction-to-aem-as-a-cloud-service-asset-upload-http-api-fc560ffd7221
it seems, that the description is probably for the same use case, as yours.
Sadly, the splitting is not described there, but you said, you are able
to capture the requests (from a real client?).
Can you verify, if the splitted files are splitted only, or has there
been done some special handling to the data?
Have a look at the headers, how is the file name (part) encoded?
Perhaps you could show the headers for the splitted requests (be sure to
obfuscate authentication data).
Felix
Am 27.01.22 um 02:57 schrieb Nuwan Galkissa Mudalige:
> Hi Team,
>
> While uploading big file to AEM Cloud (more than 10MB) it will be chunking and send to a blob storage location by multiple PUT requests.
> There are 3 main requests for this file upload, 1 . upload initiate, 2. PUT request which share the file to blob storage and 3. Upload completion request
> Jmeter script is working fine up to 10MB file since no chunking happen till that and only one PUT request made. For this PUT request follow the URI return by initiate upload request. Like that if the original file to be split initiate requests returns multiple URIs accordingly which have to be used for PUT requests.
>
> But our problem is while uploading big files that will be chunking by client side and send through multiple PUT requests instead single request. We can capture unique URIs for following PUT request but so, we are not able to handle this particular behavior (chunking) at this moment in Jmeter.
>
> It would be a great help if you can share your valuable inputs with us.
>
> Regards,
> Nuwan
>
> ________________________________
>
> "Virtusa is a worldwide provider of digital business strategy, digital engineering, and information technology (IT) services and solutions, serving Global 2000 companies across multiple industries. We help clients change, disrupt, and unlock new value with unmatched speed."
>
> ________________________________
>
> This message, including any attachments, contains confidential information intended for a specific individual and purpose, and is intended for the addressee only. Any unauthorized disclosure, use, dissemination, copying, or distribution of this message or any of its attachments or the information contained in this e-mail, or the taking of any action based on it, is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this message.
>
> ________________________________
>