[GitHub] [openwhisk] ecooperman commented on issue #5170: Activations Fail After 5 Min Timeout Occurs

[GitBox <gi...@apache.org>]

ecooperman commented on issue #5170:
URL: https://github.com/apache/openwhisk/issues/5170#issuecomment-954793598


   @rabbah thanks for responding!  I worked with the Adobe I/O support team and got the root cause of the issue, which appears to be a few different things as follows:
   
   dcfcd0681d804dafbcd0681d80edafb6 : (Pod eviction) [Pod ephemeral local storage usage exceeds the total limit of containers 500Mi.]
    
   4163948263e84268a3948263e8b2682e : (the node process may have crashed, or the network was slow for some time)
    
   50f29217fa9a4d6db29217fa9a7d6d54 : (Concurrent activation failed and container was terminated) [Activation f27f18a7347146f0bf18a7347116f003 - {"error":"An error has occurred: Cannot load module '<path to our main.js>': ReferenceError: config is not defined"} ]
   
   To give a little context, all our application does it to make JS calls (using the Axios library), to a commerce system.  The commerce system returns data, and we then transform it into what our CMS expects, and return it to the CMS.  So the application effectively works as a middleware/ESL layer between a CMS and Commerce system in order to integrate them in a way that makes them feel like a single website. That's all the code does in every possible instance.
   
   So I think this is probably not related to anything that's core to OpenWhisk, but more that we're not handling some specific cases that I'm not really sure how to handle.  To answer your question, we're not intentional reusing any containers.  I'm new to this, so maybe I'm doing that by accident?  Is there some way I can force each activation to use a new container or something?
   
   Since we saw this error, I've at least added a timeout to our Axios calls (30s), with the hopes that at least we won't have any long running calls after that.  I'm just not sure exactly what might be happening and what I can do to prevent it.  My understanding here is that each activation is independent of all other activations, and so I'm a little confused as to where to even start.  That being said, I'm trying to escalate the issue with Adobe to try and get some support from them at least.  If you have any suggestions though, I'd love to hear them!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org