[jira] [Updated] (GIRAPH-996) Large requests degrade performance. Print out warnings.

["Dionysios Logothetis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GIRAPH-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dionysios Logothetis updated GIRAPH-996:
----------------------------------------
    Attachment: GIRAPH-996.patch

If message request size exceeds the default maximum size by a factor (e.g. by 2.0), then we print a warning to the log and also to the command line. The factor is configurable and its default value is 2.0.

The check is inserted at SendMessageCache.sendMessageRequest() and SendOneMessageToManyCache.sendMessageToAllRequest(). Some of these requests may end up being local,  so this may just generate some warning for requests that don't really go through netty. But after doRequest() is called there is no way to distinguish between message requests, edge requests, which have a different max request size value. The code will become unnecessarily messy.

To test:

1) Run a job with giraph.msgRequestSize set to 1. This will make all requests big and print the warning. 
e.g. Add conf.set("giraph.msgRequestSize","1") in PageRankComputationTest and run this locally:
 mvn -DfailIfNoTests=false -Dtest=PageRankComputationTest test

2) Do all unit tests.


> Large requests degrade performance. Print out warnings.
> -------------------------------------------------------
>
>                 Key: GIRAPH-996
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-996
>             Project: Giraph
>          Issue Type: Task
>            Reporter: Dionysios Logothetis
>            Assignee: Dionysios Logothetis
>            Priority: Minor
>         Attachments: GIRAPH-996.patch
>
>
> Sometimes performance degrades significantly or a job starts using too much off heap memory when requests larger than expected are being sent through netty. We should print warnings when large requests are sent, and also send warnings to command line to make sure users notice them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)