[GitHub] [incubator-dolphinscheduler] Baoqi commented on issue #3643: [Bug][API] OOM occurs when uploading large files

[GitBox <gi...@apache.org>]

Baoqi commented on issue #3643:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3643#issuecomment-686364687


   I think we may be able to do better for this.  for multipart files in springboot & jetty, it should use streaming mode to not directly load the whole file in memory. 
   
   But according to spring boot's documentation:  https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/autoconfigure/web/servlet/MultipartProperties.html
   
          `file-size-threshold specifies the size threshold after which files will be written to disk. The default is 0.`
           
   The default is 0, so, it will parse uploaded multipart file into memory, instead of writing to temp file.
   
   So, I guess set the   `spring.http.multipart.file-size-threshold`   to a value like  10MB.  may avoid the OutOfMemory error
   
   
   spring.http.multipart.file-size-threshold=10MB
   
   
    Can you try whether this will reduce memory usage?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org