[jira] [Commented] (IMPALA-9122) Ignore FileNotFoundException when loading a table

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16994872#comment-16994872 ] 

ASF subversion and git services commented on IMPALA-9122:
---------------------------------------------------------

Commit c7d2af1bd7320cdbc9b738350cb9a038467dc493 in impala's branch refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c7d2af1 ]

IMPALA-9122 : Ignore FileNotFoundException when loading a table

It is possible that when the file metadata of a table or partition is
being loaded, some temporary files (like the ones in .hive-staging
directory) are deleted by external engines like Hive. This causes a
FileNotFoundException during the load and it fails the reload command.
In general, this should not be a problem since users are careful not to
modify the table from Hive or Spark while Impala is reading them. In
the worst case, currently the refresh command fails which can be
retried by the user. However, this does not go well with when event
processing is turned on. EventProcessor tries to reload the table as
soon as it sees a INSERT_EVENT from metastore. Hive may be still
cleaning up the staging directories when EventProcessor issues a reload
causing it go in error state.

Ideally, we should have some sort of intra-engine synchronization
semantics to avoid such issues, but that is much more complex
architectural change. For now, we should ignore such errors and skip
the deleted file from being loaded.

Testing: Unfortunately, this error is hard to reproduce locally. I
tried creating multiple threads which delete some files while multiple
FileMetadataLoaders are loading concurrently but it didn't fail for me.
Ran TestEventProcessing.test_insert_events in a loop for more than an
hour and didn't see any failure.

Change-Id: Iecf6b193b0d57de27d41ad6ef6e1719005d9e908
Reviewed-on: http://gerrit.cloudera.org:8080/14806
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Ignore FileNotFoundException when loading a table
> -------------------------------------------------
>
>                 Key: IMPALA-9122
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9122
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>            Reporter: Tim Armstrong
>            Assignee: Anurag Mantripragada
>            Priority: Critical
>              Labels: broken-build, flaky
>             Fix For: Impala 3.4.0
>
>
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/8613/
> {noformat}
> custom_cluster.test_event_processing.TestEventProcessing.test_insert_events (from pytest)
> Failing for the past 1 build (Since Failed#8613 )
> Took 1 min 20 sec.
> add description
> Error Message
> assert <bound method TestEventProcessing.wait_for_insert_event_processing of <test_event_processing.TestEventProcessing object at 0x7f7fa86ec6d0>>(18421) is True  +  where <bound method TestEventProcessing.wait_for_insert_event_processing of <test_event_processing.TestEventProcessing object at 0x7f7fa86ec6d0>> = <test_event_processing.TestEventProcessing object at 0x7f7fa86ec6d0>.wait_for_insert_event_processing
> Stacktrace
> custom_cluster/test_event_processing.py:82: in test_insert_events
>     self.run_test_insert_events()
> custom_cluster/test_event_processing.py:143: in run_test_insert_events
>     assert self.wait_for_insert_event_processing(last_synced_event_id) is True
> E   assert <bound method TestEventProcessing.wait_for_insert_event_processing of <test_event_processing.TestEventProcessing object at 0x7f7fa86ec6d0>>(18421) is True
> E    +  where <bound method TestEventProcessing.wait_for_insert_event_processing of <test_event_processing.TestEventProcessing object at 0x7f7fa86ec6d0>> = <test_event_processing.TestEventProcessing object at 0x7f7fa86ec6d0>.wait_for_insert_event_processing
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org