You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Geoffrey Jacoby (Jira)" <ji...@apache.org> on 2020/06/12 17:46:00 UTC
[jira] [Commented] (PHOENIX-5951) IndexTool output logging for
past-max-lookback rows should be configurable
[ https://issues.apache.org/jira/browse/PHOENIX-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134423#comment-17134423 ]
Geoffrey Jacoby commented on PHOENIX-5951:
------------------------------------------
HadoopQA seems to have crashed midway through the run (in SplitSystemCatalog tests) but the following were the results up to that point. None seem related to this patch, and many are common flappers. The OrphanViewToolIT failures seem to be happening consistently on several runs and are new; I will file a JIRA for those.
{code:java}
17:40:24 [ERROR] Failures:
17:40:24 [ERROR] OrphanViewToolIT.testDeleteBaseTableRows:279
17:40:24 [ERROR] OrphanViewToolIT.testDeleteBaseTableRows:279
17:40:24 [ERROR] OrphanViewToolIT.testDeleteChildParentLinkRows:402->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteChildParentLinkRows:402->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteChildViewRows:315->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteChildViewRows:315->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteGrandchildViewRows:344->verifyOrphanFileLineCounts:256->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteGrandchildViewRows:344->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteParentChildLinkRows:374->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeleteParentChildLinkRows:374->verifyOrphanFileLineCounts:255->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeletePhysicalTableLinks:429->verifyLineCount:209
17:40:24 [ERROR] OrphanViewToolIT.testDeletePhysicalTableLinks:424->verifyCountQuery:218
17:40:24 [ERROR] LocalImmutableNonTxIndexIT>BaseIndexIT.testCreateIndexAfterUpsertStarted:263->BaseIndexIT.testCreateIndexAfterUpsertStarted:338 expected:<4> but was:<3>
17:40:24 [ERROR] LocalImmutableNonTxIndexIT>BaseIndexIT.testCreateIndexAfterUpsertStarted:263->BaseIndexIT.testCreateIndexAfterUpsertStarted:338 expected:<4> but was:<3>
17:40:24 [ERROR] Errors:
17:40:24 [ERROR] OrphanViewToolIT.testCreateTableAndViews:231->verifyOrphanFileLineCounts:255->verifyLineCount:203 » FileNotFound
17:40:24 [ERROR] MutableIndexIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [ERROR] HashJoinGlobalIndexIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [ERROR] HashJoinNoIndexIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [ERROR] SortMergeJoinLocalIndexIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [ERROR] WhereOptimizerForJoinFiltersIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [ERROR] ChunkedResultIteratorIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [ERROR] ParameterizedTransactionIT>ParallelStatsDisabledIT.doSetup:60->BaseTest.setUpTestDriver:516->BaseTest.setUpTestDriver:521->BaseTest.checkClusterInitialized:435->BaseTest.setUpTestCluster:449->BaseTest.initMiniCluster:550 » Runtime
17:40:24 [INFO]
17:40:24 [ERROR] Tests run: 3659, Failures: 14, Errors: 8, Skipped: 3
{code}
> IndexTool output logging for past-max-lookback rows should be configurable
> --------------------------------------------------------------------------
>
> Key: PHOENIX-5951
> URL: https://issues.apache.org/jira/browse/PHOENIX-5951
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Geoffrey Jacoby
> Assignee: Geoffrey Jacoby
> Priority: Major
> Attachments: PHOENIX-5951-4.x.v1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> On a global mutable table with lots of writes, it can be possible for there to be many rows flagged by index verification as invalid that are older than the max lookback age. These are likely due to race conditions between compaction in the base table and index table, but they can possibly show real issues.
> Since heavy output logging to PHOENIX_INDEX_TOOL can cause hotspotting, some operators may want to turn it off for this class of failures, while still wanting output logging for other kinds of invalid rows.
> As part of this change, we should start including in the output row a column with a code identifying the type of error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)