[jira] [Comment Edited] (FLINK-15942) Improve logging of infinite resource profile

["Xintong Song (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032086#comment-17032086 ] 

Xintong Song edited comment on FLINK-15942 at 2/7/20 3:27 AM:
--------------------------------------------------------------

I wonder whether this logging issue can be solved by simply changing how infinite / extremely large values are printed.
- For {{MemorySize}}, print values larger than {{1L << 50}} (1 PB) as “infinite” in {{toHumanReadableString}}。
- For {{CPUResource}}, we add a similar {{toHumanReadableString}} method, and print values larger than {{1024.0}} (or maybe larger) as "infinite".

I think this is probably the quickest way to fix the log readability issue, say if we still want to give it a try to catch 1.10.0.

Having special infinite values for {{MemorySize}} and {{CPUResource}} and clearly define how the should behave when involved in computations might be the right approach to take care of infinite values. However, it may require more time to discuss what is the proper behavior. E.g., should we allow {{INFINITE + INFINITE = INFINITE}}, and {{INFINITE - non_infinite_value = INFINITE}}? And I would also try to avoid touching the {{MemorySize}} and {{CPUResource}} computation logics at the very end of releasing, given the risk of breaking things that already work well.


was (Author: xintongsong):
I wonder whether this logging issue can be solved by simply changing how infinite / extremely large values are printed.
- For {{MemorySize}}, print values larger than {{1L << 50}} (1 PB) as “infinite” in {{toHumanReadableString}}。
- For {{CPUResource}}, we add a similar {{toHumanReadableString}} method, and print values larger than {{1024.0}} (or maybe larger) as "infinite". I think this is probably the quickest way to fix the log readability issue, say if we still want to give it a try to catch 1.10.0.

Having special infinite values for {{MemorySize}} and {{CPUResource}} and clearly define how the should behave when involved in computations might be the right approach to take care of infinite values. However, it may require more time to discuss what is the proper behavior. E.g., should we allow {{INFINITE + INFINITE = INFINITE}}, and {{INFINITE - non_infinite_value = INFINITE}}? And I would also try to avoid touching the {{MemorySize}} and {{CPUResource}} computation logics at the very end of releasing, given the risk of breaking things that already work well.

> Improve logging of infinite resource profile
> --------------------------------------------
>
>                 Key: FLINK-15942
>                 URL: https://issues.apache.org/jira/browse/FLINK-15942
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Configuration, Runtime / Task
>    Affects Versions: 1.10.0
>            Reporter: Andrey Zagrebin
>            Priority: Major
>
> After we set task memory and CPU to infinity in FLINK-15763, it spoiled the logs:
> {code:java}
> 00:23:49,442 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:0, state:ACTIVE, resource profile: ResourceProfile{cpuCores=44942328371557892500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.0000000000000000, taskHeapMemory=2097152.000tb (2305843009213693951 bytes), taskOffHeapMemory=2097152.000tb (2305843009213693951 bytes), managedMemory=20.000mb (20971520 bytes), networkMemory=16.000mb (16777216 bytes)}, allocationId: 349dacfbf1ac4d0b44a2d11e1976d264, jobId: 689a0cf24b40f16b6f45157f78754c46).
> {code}
> We should treat the infinity as a special case and print it accordingly



--
This message was sent by Atlassian Jira
(v8.3.4#803005)