[GitHub] [openwhisk] tz70s opened a new pull request #3757: Introduce configurable action blob size limit.

[GitBox <gi...@apache.org>]

tz70s opened a new pull request #3757: Introduce configurable action blob size limit.
URL: https://github.com/apache/openwhisk/pull/3757
 
 
   I think this pr still needs some improvements, but I need some feedback on current implementation.
   
   This pr makes the action blob size configurable with some docs for deployment and system tuning.
   
   I’ve tested for 200MB zip file for execution,
   
   Required changes contain two kind of limitations:
   1. Size limit.
   2. Timeout limit.
   
   Size limit:
   
   1. Nginx: `http.client_max_body_size`
   2. Akka: `http.[client/server].parsing.max-content-length`
   3. Internal size limitation passed by pure config. (remove hardcoded limit)
   4. Database: only support on couchdb currently, -> `httpd.max_http_request_size`
   
   All these can be done with same config —> `limit_action_blob_size`
   
   Timeout:
   1. Akka: http.[client/server].[request-timeout/idle-timeout]
   2. Couchdb: query_server_config.os_process_limit (in sec)
   3. Action invocation timeout.
   
   ### Discussion:
   
   1. To make the size limit configurable, requires some changes in runtimes. [sample-change-in-node](https://github.com/tz70s/incubator-openwhisk-runtime-nodejs/commit/4f1f562c5de9827fd296521972928a228a5d436d). Invoker will pass the size limit as environment variables to it, same as `__OW_API_HOST`. Is this appropriate?
   
   2. I didn't add any timeout related variables passed from ansible. There might be still some efforts for user to configure correctly. But the behavior of timeout depends on system resources, to avoid potential risk, using ad-hoc configuration currently.
   
   3. Due to same config passed into `akka.http.[client/server].xxx`, I think that would be nice to raise them to upper level -> i.e. `akka.http.parsing.max-content-length`. User can override it on either server or client config, but I'm not sure whether there's any risk or not?
   
   4. See the system tuning section: to test with large action blob size, needs lots of changes for system tuning in current travis build. Therefore, no additional tests are provided now.
   
   5. After testing, I think the large action size is still not a recommended way, seems current implementation of OpenWhisk requires lots of optimization to support it. Therefore, I've mentioned this into doc.
   
   6. There's another way to config database's `request-size-limit` via controller side by using remote (http) APIs, might be better by using same abstraction for all artifact store.
   
   Signed-off-by: Tzu-Chiao Yeh <su...@gmail.com>
   
   <!--- Provide a concise summary of your changes in the Title -->
   
   ## Description
   <!--- Provide a detailed description of your changes. -->
   <!--- Include details of what problem you are solving and how your changes are tested. -->
   
   ## Related issue and scope
   <!--- Please include a link to a related issue if there is one. -->
   - [x] Opened issue to propose and discuss this change (#3712)
   
   ## My changes affect the following components
   <!--- Select below all system components are affected by your change. -->
   <!--- Enter an `x` in all applicable boxes. -->
   - [x] API
   - [ ] Controller
   - [ ] Message Bus (e.g., Kafka)
   - [ ] Loadbalancer
   - [x] Invoker
   - [ ] Intrinsic actions (e.g., sequences, conductors)
   - [x] Data stores (e.g., CouchDB)
   - [ ] Tests
   - [x] Deployment
   - [ ] CLI
   - [ ] General tooling
   - [x] Documentation
   
   ## Types of changes
   <!--- What types of changes does your code introduce? Use `x` in all the boxes that apply: -->
   - [ ] Bug fix (generally a non-breaking change which closes an issue).
   - [x] Enhancement or new feature (adds new functionality).
   - [ ] Breaking change (a bug fix or enhancement which changes existing behavior).
   
   ## Checklist:
   <!--- Please review the points below which help you make sure you've covered all aspects of the change you're making. -->
   
   - [x] I signed an [Apache CLA](https://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md).
   - [x] I reviewed the [style guides](https://github.com/apache/incubator-openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :).
   - [ ] I added tests to cover my changes.
   - [ ] My changes require further changes to the documentation.
   - [x] I updated the documentation where necessary.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services