[jira] [Commented] (QUICKSTEP-34) Better error checking for valid num_workers

["Harshad Deshmukh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/QUICKSTEP-34?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357356#comment-15357356 ] 

Harshad Deshmukh commented on QUICKSTEP-34:
-------------------------------------------

HI Rathijit,

Created a PR to address the issue of negative worker threads - https://github.com/apache/incubator-quickstep/pull/48/files

> Better error checking for valid num_workers
> -------------------------------------------
>
>                 Key: QUICKSTEP-34
>                 URL: https://issues.apache.org/jira/browse/QUICKSTEP-34
>             Project: Apache Quickstep
>          Issue Type: Improvement
>          Components: CLI
>         Environment: repo version: 714874ce54e12972285a43f92784ef6954a8b6fd
>            Reporter: Rathijit Sen
>            Priority: Minor
>
> Negative values, e.g., num_workers=-2, should be treated as invalid.
> Starting Quickstep with -2 worker thread(s) and a 125.75 GB buffer pool
> terminate called after throwing an instance of 'std::length_error'
>   what():  vector::_M_fill_insert
> ./run-benchmark.sh: line 82: 29034 Aborted                 $QSEXE < tmp.sql
> With too many workers (>> number of logical processors), performance drops significantly. So the maximum number of workers should perhaps be limited to the number of logical processors on the system. 
> Moreover, with a very large number of workers, occasional failures may happen during I/O. For example: 
> Query 07.sql 
> Starting Quickstep with 1024 worker thread(s) and a 125.75 GB buffer pool 
> Could not create logging file: Too many open files 
> COULD NOT CREATE A LOGGINGFILE 20160628-231706.2986!E0628 23:17:06.323117 3894 FileManagerPosix.cpp:117] Failed to open file /slowdisk/rathijit/store100/qsblk_0001_0000000000000010033.qsb with error: Too many open files Could not create logging file: Too many open files 
> COULD NOT CREATE A LOGGINGFILE 20160628-231706.2986!F0628 23:17:06.323297 3894 StorageManager.cpp:659] Check failed: status Failed to read block from persistent storage: 281474976720689 
> ./run-benchmark.sh: line 82: 2986 Aborted $QSEXE < tmp.sql Quickstep failed on query 07, continuing to next query. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)