[jira] [Commented] (DRILL-4831) Running refresh table metadata concurrently randomly fails with JsonParseException

["Aman Sinha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15523322#comment-15523322 ] 

Aman Sinha commented on DRILL-4831:
-----------------------------------

Supporting concurrent writers to the metadata cache file will require enhancements to the current implementation.  The underlying file system (in this case MapR-FS but this is not specific to MFS) provides consistency guarantees only at the file system level, not at the application (query) level.  Thus, if there are multiple writers writing to the same byte offsets in the file, the file system will eventually produce a 'serializable' output (i.e the net effect is writes are performed in some sequential manner) but while the writes are happening, the readers may see overlapping writes from different writers.  

In order to address this, there are 2 options:  (a) each writer can write to a temporary unique filename and rename at the very end.  The renaming process is atomic.   (b) create a lock file for the table and have the writers open the lock file in Exclusive mode before writing to the actual file. 

> Running refresh table metadata concurrently randomly fails with JsonParseException
> ----------------------------------------------------------------------------------
>
>                 Key: DRILL-4831
>                 URL: https://issues.apache.org/jira/browse/DRILL-4831
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Metadata
>    Affects Versions: 1.8.0
>            Reporter: Rahul Challapalli
>         Attachments: error.log, l_3level.tgz
>
>
> git.commit.id.abbrev=f476eb5
> Just run the below command concurrently from 10 different JDBC connections. There is a likelihood that you might encounter the below error
> Extracts from the log
> {code}
> Caused By (java.lang.AssertionError) Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#189411:LogicalProject.NONE.ANY([]).[](input=rel#189289:Subset#3.ENUMERABLE.ANY([]).[],l_orderkey=$1,dir0=$2,dir1=$3,dir2=$4,l_shipdate=$5,l_extendedprice=$6,l_discount=$7), rel#189233:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[dfs, metadata_caching_pp, l_3level])]
>     org.apache.calcite.util.Util.newInternal():792
>     org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():251
> .....
> .....
>   java.lang.Thread.run():745
>   Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n, \t) is allowed between tokens
>  at [Source: com.mapr.fs.MapRFsDataInputStream@57a574a8; line: 1, column: 2]
>     org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():95
> {code}  
> Attached the complete log message and the data set



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)