You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Ilya Kasnacheev <il...@gmail.com> on 2020/01/09 15:12:42 UTC
Re: Improve logging of Data Region configuration
Hello again!
TC passes, review is ready.
Your last chance to act before merge!
Regards,
--
Ilya Kasnacheev
вт, 31 дек. 2019 г. в 16:01, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> I have prepared a patch, please review. It will look like this:
>
> Data Regions Started: 4
> ^-- sysMemPlc region [type=internal, persistence=true, lazyAlloc=false,
> ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%, allocRam=100MB, allocTotal=0MB]
> ^-- default region [type=default, persistence=true, lazyAlloc=true,
> ... initCfg=30MB, maxCfg=30MB, usedRam=0MB, freeRam=100%, allocRam=0MB, allocTotal=0MB]
> ^-- metastoreMemPlc region [type=internal, persistence=true, lazyAlloc=false,
> ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%, allocRam=100MB, allocTotal=0MB]
> ^-- TxLog region [type=internal, persistence=true, lazyAlloc=false,
> ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%, allocRam=100MB, allocTotal=0MB]
> ^-- Ignite persistence [used=0MB]
> ...
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=9e6c8e45, name=internal.GridNodeMetricsLogPdsSelfTest0, uptime=00:00:12.271]
> ^-- Cluster [hosts=1, CPUs=8, servers=2, clients=0, topVer=2, minorTopVer=3]
> ^-- Network [addrs=[127.0.0.1], localHost=127.0.0.1, discoPort=47500, commPort=45010]
> ^-- CPU [CPUs=8, curLoad=2.7%, avgLoad=3.38%, GC=0%]
> ^-- Heap [used=48MB, free=99.32%, comm=564MB]
> ^-- Off-heap memory [used=0MB, free=99.96%, allocated=230MB]
> ^-- Page memory [pages=34]
> ^-- sysMemPlc region [type=internal, persistence=true, lazyAlloc=false,
> ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=99.98%, allocRam=100MB, allocTotal=0MB]
> ^-- default region [type=default, persistence=true, lazyAlloc=true,
> ... initCfg=30MB, maxCfg=30MB, usedRam=0MB, freeRam=99.75%, allocRam=30MB, allocTotal=0MB]
> ^-- metastoreMemPlc region [type=internal, persistence=true, lazyAlloc=false,
> ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=99.96%, allocRam=0MB, allocTotal=0MB]
> ^-- TxLog region [type=internal, persistence=true, lazyAlloc=false,
> ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%, allocRam=100MB, allocTotal=0MB]
> ^-- Ignite persistence [used=0MB]
> ^-- Outbound messages queue [size=0]
> ^-- Public thread pool [active=0, idle=6, qSize=0]
> ^-- System thread pool [active=0, idle=8, qSize=0]
> ^-- Custom executor 0 [active=0, idle=0, qSize=0]
> ^-- Custom executor 1 [active=0, idle=0, qSize=0]
>
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 27 дек. 2019 г. в 15:03, Ilya Kasnacheev <il...@gmail.com>:
>
>> Hello!
>>
>> I have filed a ticket about this improvement, plan to start on coding.
>>
>> https://issues.apache.org/jira/browse/IGNITE-12505
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 26 дек. 2019 г. в 12:50, Ilya Kasnacheev <il...@gmail.com>:
>>
>>> Hello!
>>>
>>> Okay, I will mark default region.
>>>
>>> We already log information about internal memory regions, so removing it
>>> will require a lot of consensus. However, I can clearly map them as system
>>> regions when printing.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> чт, 26 дек. 2019 г. в 12:10, Ivan Pavlukhin <vo...@gmail.com>:
>>>
>>>> Ilya,
>>>>
>>>> Indeed the matters can be improved.
>>>>
>>>> Is not it useful to mark what region is default? Also some doubts
>>>> about internal memory regions. It is not obvious that we should print
>>>> an information about them for every user. If we need to have some
>>>> determinism about offheap memory than I can think about logging
>>>> amounts for internal needs of total ones (a sum for all regions).
>>>>
>>>> вт, 24 дек. 2019 г. в 15:38, Ilya Kasnacheev <il...@apache.org>:
>>>> >
>>>> > Hello!
>>>> >
>>>> > It came to my attention that we output data regions' configurations
>>>> twice
>>>> > when starting node, but we never output list of data regions
>>>> (including
>>>> > system, etc) that were actually started.
>>>> >
>>>> > First we have IgniteConfiguration printed (quiet=false):
>>>> > 2019-07-24 02:33:33.918[INFO
>>>> ][Thread-139][o.a.i.i.IgniteKernal%GridNodeName
>>>> > ] IgniteConfiguration [... dfltDataRegConf=DataRegionConfiguration
>>>> [name=
>>>> > mem_plc, maxSize=635655159808, initSize=268435456, swapPath=null,
>>>> > pageEvictionMode=DISABLED, evictionThreshold=0.9,
>>>> emptyPagesPoolSize=100,
>>>> > metricsEnabled=true, metricsSubIntervalCount=5,
>>>> metricsRateTimeInterval=1000
>>>> > , persistenceEnabled=true, checkpointPageBufSize=17179869184],
>>>> storagePath=/
>>>> > ssd/data, checkpointFreq=30000, lockWaitTime=10000,
>>>> checkpointThreads=4,
>>>> > checkpointWriteOrder=SEQUENTIAL, walHistSize=2147483647,
>>>> walSegments=10,
>>>> > walSegmentSize=1073741824, walPath=/ssd/data/wal, walArchivePath=/sas/
>>>> > wal_archive, metricsEnabled=false, walMode=LOG_ONLY,
>>>> walTlbSize=131072,
>>>> > walBuffSize=5242880, walFlushFreq=2000, walFsyncDelay=1000,
>>>> > walRecordIterBuffSize=67108864, alwaysWriteFullPages=false,
>>>> fileIOFactory=
>>>> > org.apache.ignite.internal.processors.cache.persistence.file.
>>>> > AsyncFileIOFactory@3612c49a, metricsSubIntervalCnt=5,
>>>> > metricsRateTimeInterval=60000, walAutoArchiveAfterInactivity=-1,
>>>> > writeThrottlingEnabled=false, walCompactionEnabled=true,
>>>> walCompactionLevel=
>>>> > 1], ...]
>>>> >
>>>> > Then we have all configured Data Regions printed per IGNITE-8803
>>>> > (quiet=true):
>>>> > [11:30:36] Data Regions Configured:
>>>> > [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>>> > persistence=false, lazyMemoryAllocation=true]
>>>> > [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>>> > persistence=false, lazyMemoryAllocation=true]
>>>> >
>>>> > Then we print number of Data Regions that were initialized as per
>>>> > IGNITE-7196, but not regions themselves (quiet=false):
>>>> > Configured data regions initialized successfully [total=4]
>>>> >
>>>> > I propose to keep the first one (IgniteConfiguration), remove the
>>>> second
>>>> > one (Data Regions Configured), and promote the last one to quiet mode
>>>> while
>>>> > also outputting the regions themselves like this:
>>>> > [11:30:36] Data Regions Initialized Successfully: 4
>>>> > [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>>> > persistence=true, lazyMemoryAllocation=true]
>>>> > [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
>>>> > persistence=true, lazyMemoryAllocation=true]
>>>> > [11:30:36] ^-- sysMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB,
>>>> > persistence=true, lazyMemoryAllocation=false]
>>>> > [11:30:36] ^-- volatileMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB,
>>>> > persistence=false, lazyMemoryAllocation=true]
>>>> >
>>>> > (maybe it will also include information about current usage of region
>>>> in
>>>> > line with IGNITE-9305's "Metrics for local node"
>>>> >
>>>> > WDYT?
>>>> >
>>>> > Regards,
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Ivan Pavlukhin
>>>>
>>>