You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/02/13 12:40:31 UTC

[GitHub] [incubator-doris] chaoyli opened a new pull request #2895: Import new ThreadPool framework

chaoyli opened a new pull request #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895
 
 
   1. Using a token machanism to receive tasks.
      Task in token can be handled serially or concurrently.
      MemTableFlush in Doirs can use this feature,
      different tasks belongs to different tablet can be flushed concurrently.
      But task belongs one tablet should flushed serially.
   2. Support set min_threads and max_threads in ThreadPool.
      At peek time, ThreadPool can expand threads up to max_threads.
   3. New thread implementation which is better to trace on resource usage
      than C++ library.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli commented on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli commented on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585767725
 
 
   Thread design point:
   1. It is a thin wrapper around pthread that can register itself with the singleton ThreadMgr
   (a private class implemented in thread.cpp entirely, which tracks all live threads so
   that they may be monitored via the debug webpages). This class has a limited subset of
   boost::thread's API. Construction is almost the same, but clients must supply a
   category and a name for each thread so that they can be identified in the debug web
   UI. Otherwise, join() is the only supported method from boost::thread.
   2. Each Thread object knows its operating system thread ID (TID), which can be used to
   attach debuggers to specific threads, to retrieve resource-usage statistics from the
   operating system, and to assign threads to resource control groups.
   3. Threads are shared objects, but in a degenerate way. They may only have
   up to two referents: the caller that created the thread (parent), and
   the thread itself (child). Moreover, the only two methods to mutate state
   (join() and the destructor) are constrained: the child may not join() on
   itself, and the destructor is only run when there's one referent left.
   These constraints allow us to access thread internals without any locks.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.
- **_advantage_**
- only requires a single allocation, and ref count is on the same cache line as the object
- the pointer only requires 8 bytes (since the ref count is within the object)
- you can manually increase or decrease reference counts when more control is required
- you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
- since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
- **_disadvantage_**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases
2. MonoTime/MonoDelta
MonoTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.
3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.
**_advantage_**
- only requires a single allocation, and ref count is on the same cache line as the object
- the pointer only requires 8 bytes (since the ref count is within the object)
- you can manually increase or decrease reference counts when more control is required
- you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
- since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
**_disadvantage_**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases
2. MonoTime/MonoDelta
MonoTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.
3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911
 
 
   This patch imports many basis class.
   
   1. scoped_refptr
       scoped_refptr is used to replace std::shared_ptr,  is generally faster and smaller.
    
       **advantage**
       -  only requires a single allocation, and ref count is on the same cache line as the object
       - the pointer only requires 8 bytes (since the ref count is within the object)
       -  you can manually increase or decrease reference counts when more control is required
       - you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
      - since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
   
       **disadvantage**
       - the referred-to object must inherit from RefCounted
       - does not support the weak_ptr use cases
   
   2. MonoTime/MonoDelta
       MonTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
       MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

**_disadvantage_**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases

2. MonoTime/MonoDelta
MonoTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.
**advantage**
- only requires a single allocation, and ref count is on the same cache line as the object
- the pointer only requires 8 bytes (since the ref count is within the object)
- you can manually increase or decrease reference counts when more control is required
- you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
- since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.

**disadvantage**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases
2. MonoTime
The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
3. MonoDelta
The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911
 
 
   This patch imports many basis class.
   
   1. scoped_refptr
       scoped_refptr is used to replace std::shared_ptr,  is generally faster and smaller.
       
       **advantage**
       -  only requires a single allocation, and ref count is on the same cache line as the object
       - the pointer only requires 8 bytes (since the ref count is within the object)
       -  you can manually increase or decrease reference counts when more control is required
       - you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
      - since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
   
       **disadvantage**
   
       - the referred-to object must inherit from RefCounted
       - does not support the weak_ptr use cases
   
   2. MonoTime/MonoDelta
       MonTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
       MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli closed pull request #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli closed pull request #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585766207
 
 
   Thread pool design point:
   All tasks submitted directly to the thread pool enter a FIFO queue and are
   dispatched to a worker thread when one becomes free. Tasks may also be
   submitted via ThreadPoolTokens. The token wait() and shutdown() functions
   can then be used to block on logical groups of tasks.
   
   A token operates in one of two ExecutionModes, determined at token
   construction time:
   1. SERIAL: submitted tasks are run one at a time.
   2. CONCURRENT: submitted tasks may be run in parallel. This isn't unlike
       tasks submitted without a token, but the logical grouping that tokens
       impart can be useful when a pool is shared by many contexts (e.g. to
       safely shut down one context, to derive context-specific metrics, etc.).
   
   Tasks submitted without a token or via ExecutionMode::CONCURRENT tokens are
   processed in FIFO order. On the other hand, ExecutionMode::SERIAL tokens are
   processed in a round-robin fashion, one task at a time. This prevents them
   from starving one another. However, tokenless (and CONCURRENT token-based)
   tasks can starve SERIAL token-based tasks.
   
   ```
   Usage Example:
   std::unique_ptr<ThreadPool> thread_pool;
   CHECK_OK(
         ThreadPoolBuilder("my_pool")
               .set_min_threads(0)
               .set_max_threads(5)
               .set_max_queue_size(10)
               .set_idle_timeout(MonoDelta::FromMilliseconds(2000))
               .build(&thread_pool));
        thread_pool->submit(shared_ptr<Runnable>(new Task()));
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911
 
 
   This patch imports many basis class.
   
   1. scoped_refptr
       scoped_refptr is used to replace std::shared_ptr,  is generally faster and smaller.
       
       **advantage**
       -  only requires a single allocation, and ref count is on the same cache line as the object
       - the pointer only requires 8 bytes (since the ref count is within the object)
       -  you can manually increase or decrease reference counts when more control is required
       - you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
      - since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
   
       **disadvantage**
       - the referred-to object must inherit from RefCounted
       - does not support the weak_ptr use cases
   
   2. MonoTime/MonoDelta
       MonTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
       MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.

**advantage**
- only requires a single allocation, and ref count is on the same cache line as the object
- the pointer only requires 8 bytes (since the ref count is within the object)
- you can manually increase or decrease reference counts when more control is required
- you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
- since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.

**disadvantage**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases

3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

**disadvantage**
(1) the referred-to object must inherit from RefCounted
(2) does not support the weak_ptr use cases
2. MonoTime
The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
3. MonoDelta
The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

**_disadvantage_**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases
2. MonoTime/MonoDelta
MonoTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.
3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

- **_disadvantage_**

- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases
2. MonoTime/MonoDelta
MonoTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.
3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] lingbin commented on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

lingbin commented on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585807638
 
 
   Could you please split this PR into multiple commits?
   For example:
   1. import some gutil
   2. import some util
   3. import new thread-pool
   4. some refactor on current code

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.

1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.

**disadvantage**
- the referred-to object must inherit from RefCounted
- does not support the weak_ptr use cases

2. MonoTime/MonoDelta
MonTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

3. CountDownLatch
This is a C++ implementation of the Java CountDownLatch

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli commented on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli commented on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585766207
 
 
   Thread pool with a variable number of threads.
   All tasks submitted directly to the thread pool enter a FIFO queue and are
   dispatched to a worker thread when one becomes free. Tasks may also be
   submitted via ThreadPoolTokens. The token Wait() and Shutdown() functions
   can then be used to block on logical groups of tasks.
   
   A token operates in one of two ExecutionModes, determined at token
   construction time:
   1. SERIAL: submitted tasks are run one at a time.
   2. CONCURRENT: submitted tasks may be run in parallel. This isn't unlike
       tasks submitted without a token, but the logical grouping that tokens
       impart can be useful when a pool is shared by many contexts (e.g. to
       safely shut down one context, to derive context-specific metrics, etc.).
   
   Tasks submitted without a token or via ExecutionMode::CONCURRENT tokens are
   processed in FIFO order. On the other hand, ExecutionMode::SERIAL tokens are
   processed in a round-robin fashion, one task at a time. This prevents them
   from starving one another. However, tokenless (and CONCURRENT token-based)
   tasks can starve SERIAL token-based tasks.
   
   ```
   Usage Example:
   std::unique_ptr<ThreadPool> thread_pool;
   CHECK_OK(
         ThreadPoolBuilder("my_pool")
               .set_min_threads(0)
               .set_max_threads(5)
               .set_max_queue_size(10)
               .set_idle_timeout(MonoDelta::FromMilliseconds(2000))
               .build(&thread_pool));
        thread_pool->submit(shared_ptr<Runnable>(new Task()));
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli commented on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli commented on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585760983
 
 
   #2896 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli commented on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli commented on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911
 
 
   This patch imports many basis class.
   1. scoped_refptr
       scoped_refptr is used to replace std::shared_ptr,  is generally faster and smaller.
       And also we can manually increase or decrease reference counts when more control is required.
   2. MonoTime
       The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
   3. MonoDelta
       The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.
1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.
**advantage**

- only requires a single allocation, and ref count is on the same cache line as the object

(2) the pointer only requires 8 bytes (since the ref count is within the object)
(3) you can manually increase or decrease reference counts when more control is required
(4) you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
(5) since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
**disadvantage**
(1) the referred-to object must inherit from RefCounted
(2) does not support the weak_ptr use cases
2. MonoTime
The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
3. MonoDelta
The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911

This patch imports many basis class.
1. scoped_refptr
scoped_refptr is used to replace std::shared_ptr, is generally faster and smaller.
**advantage**
(1) only requires a single allocation, and ref count is on the same cache line as the object
(2) the pointer only requires 8 bytes (since the ref count is within the object)
(3) you can manually increase or decrease reference counts when more control is required
(4) you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
(5) since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
**disadvantage**
(1) the referred-to object must inherit from RefCounted
(2) does not support the weak_ptr use cases
2. MonoTime
The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
3. MonoDelta
The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] chaoyli edited a comment on issue #2895: Import new ThreadPool framework

Posted by GitBox <gi...@apache.org>.

chaoyli edited a comment on issue #2895: Import new ThreadPool framework
URL: https://github.com/apache/incubator-doris/pull/2895#issuecomment-585763911
 
 
   This patch imports many basis class.
   
   1. scoped_refptr
       scoped_refptr is used to replace std::shared_ptr,  is generally faster and smaller.
       
       **advantage**
       -  only requires a single allocation, and ref count is on the same cache line as the object
       - the pointer only requires 8 bytes (since the ref count is within the object)
       -  you can manually increase or decrease reference counts when more control is required
       - you can convert from a raw pointer back to a scoped_refptr safely without worrying about double freeing
      - since we control the implementation, we can implement features, such as debug builds that capture the stack trace of every referent to help debug leaks.
   
   
       **disadvantage**
   
       - the referred-to object must inherit from RefCounted
       - does not support the weak_ptr use cases
   
   2. MonoTime/MonoDelta
       MonTime: The MonoTime represents a particular point in time, relative to some fixed but unspecified reference point.
       MonoDelta: The MonoDelta class represents an elapsed duration of time, the delta between two MonoTime instances.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org