You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2021/06/07 20:31:00 UTC
[jira] [Closed] (ARROW-11588) [C++] Add traceable general purpose
queue(s)
[ https://issues.apache.org/jira/browse/ARROW-11588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weston Pace closed ARROW-11588.
-------------------------------
Resolution: Won't Fix
Cleaning up my issues. General queues seems to be more of a pipe dream. Different operators have had different queuing needs and so we are likely to have multiple types. David has already made good work on tracing and I feel this issue could probably be better served by a "execution graph tracing" issue.
> [C++] Add traceable general purpose queue(s)
> --------------------------------------------
>
> Key: ARROW-11588
> URL: https://issues.apache.org/jira/browse/ARROW-11588
> Project: Apache Arrow
> Issue Type: Task
> Components: C++
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Major
>
> There are a few places we queue data/jobs today (e.g. readahead, thread pool). In order to reduce complexity (single responsibility) and add traceability (so we can tell where queues are filling up) we should create general purpose queues (e.g. the boost lockfree one) and add some basic performance counters/events (queue size, queue full, queue starved).
> We will need multiple queue implementations. Some places will need a spsc lock free queue and others will need a multi-producer / multi-consumer queue with proper waits (and potentially more queues than that).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)