You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by franz1981 <gi...@git.apache.org> on 2017/05/29 07:56:43 UTC

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

GitHub user franz1981 opened a pull request:

    https://github.com/apache/activemq-artemis/pull/1295

    ARTEMIS-1185 Inter-Process Journal Sampler Profiler + CLI command

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/franz1981/activemq-artemis real_time_journal_latency

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/activemq-artemis/pull/1295.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1295
    
----
commit 87b59431dbab8d987d217bb9bcdaac688bd5f120
Author: Francesco Nigro <ni...@gmail.com>
Date:   2017-05-17T08:35:35Z

    ARTEMIS-1185 Inter-Process Journal Sampler Profiler + CLI command

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic but I've run it already concurrently with a server...Simply I've used a different bash console.
    That's how I've collected the data I've used for the doc....
    I've not seen any lock on the server..the only interaction with the server is by reading the configuration and the MulticastBuffer...or there are other points?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    Looks good +1, comments are just comments, I'm happy with as is.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r118992513
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/counters/Profiler.java ---
    @@ -0,0 +1,72 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.core.io.buffer.counters;
    +
    +import java.io.File;
    +import java.io.IOException;
    +
    +import org.apache.activemq.artemis.journal.ActiveMQJournalLogger;
    +
    +/**
    + * Factory class to instantiate {@link FlushProfiler}s classes.
    + */
    +public final class Profiler {
    +
    +   private static final FlushProfiler BLACK_HOLE_PROFILER = new FlushProfiler() {
    --- End diff --
    
    or even remove if go with setting to null and separate guard statements


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    I need to add some docs about the CLI command usage and improves some class names (eg MulticastSample). It can be done in a separated PR too


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    what I am missing:
    
    - doc how to use this...  (is that internal only)?
    - how to read the file...
    
    
    I am a bit confused with the file format...
    
    
    Also, it would be nice to be allowed to configure where the file would be generated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r118983050
  
    --- Diff: artemis-cli/src/main/java/org/apache/activemq/artemis/cli/commands/tools/journal/StatJournal.java ---
    @@ -0,0 +1,288 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.cli.commands.tools.journal;
    +
    +import java.io.File;
    +import java.io.FileOutputStream;
    +import java.io.PrintStream;
    +import java.util.Arrays;
    +import java.util.Comparator;
    +import java.util.concurrent.TimeUnit;
    +import java.util.concurrent.atomic.AtomicLong;
    +import java.util.concurrent.locks.LockSupport;
    +
    +import io.airlift.airline.Command;
    +import io.airlift.airline.Option;
    +import org.apache.activemq.artemis.cli.commands.ActionContext;
    +import org.apache.activemq.artemis.cli.commands.Configurable;
    +import org.apache.activemq.artemis.core.config.impl.FileConfiguration;
    +import org.apache.activemq.artemis.core.io.buffer.counters.FlushSampleFlyweight;
    +import org.apache.activemq.artemis.core.io.buffer.counters.Profiler;
    +import org.apache.activemq.artemis.utils.collections.MulticastSample;
    +
    +@Command(name = "stat", description = "Provide performance statistics around the journal usage")
    --- End diff --
    
    maybe journal-stat?, stat seems a little too generic, e.g. what if other stat tools added for other parts?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r118983292
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java ---
    @@ -93,10 +82,10 @@
        public TimedBuffer(final int size, final int timeout, final boolean logRates) {
           bufferSize = size;
     
    -      this.logRates = logRates;
    -
           if (logRates) {
    -         logRatesTimer = new Timer(true);
    +         this.flushProfiler = Profiler.instrumented();
    +      } else {
    +         this.flushProfiler = Profiler.none();
    --- End diff --
    
    would it not be better to set to null, as so don't then need a Noop implementation, and later do a guard statement so the code can be compiled out if not enabled..


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119024088
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/counters/Profiler.java ---
    @@ -0,0 +1,72 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.core.io.buffer.counters;
    +
    +import java.io.File;
    +import java.io.IOException;
    +
    +import org.apache.activemq.artemis.journal.ActiveMQJournalLogger;
    +
    +/**
    + * Factory class to instantiate {@link FlushProfiler}s classes.
    + */
    +public final class Profiler {
    +
    +   private static final FlushProfiler BLACK_HOLE_PROFILER = new FlushProfiler() {
    --- End diff --
    
    The ``default`` method idea is good IMHO when you've methods not bounded each others, but when you have methods that makes sense to be implemented (and called) together (eg Lock::lock and Lock::unlock), I prefer to use normal methods and force the implementation to have both.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119284924
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java ---
    @@ -93,10 +82,10 @@
        public TimedBuffer(final int size, final int timeout, final boolean logRates) {
           bufferSize = size;
     
    -      this.logRates = logRates;
    -
           if (logRates) {
    --- End diff --
    
    Good point!I'll change the name in something more appropriate :+1: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119040186
  
    --- Diff: artemis-cli/src/main/java/org/apache/activemq/artemis/cli/commands/tools/journal/StatJournal.java ---
    @@ -0,0 +1,288 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.cli.commands.tools.journal;
    +
    +import java.io.File;
    +import java.io.FileOutputStream;
    +import java.io.PrintStream;
    +import java.util.Arrays;
    +import java.util.Comparator;
    +import java.util.concurrent.TimeUnit;
    +import java.util.concurrent.atomic.AtomicLong;
    +import java.util.concurrent.locks.LockSupport;
    +
    +import io.airlift.airline.Command;
    +import io.airlift.airline.Option;
    +import org.apache.activemq.artemis.cli.commands.ActionContext;
    +import org.apache.activemq.artemis.cli.commands.Configurable;
    +import org.apache.activemq.artemis.core.config.impl.FileConfiguration;
    +import org.apache.activemq.artemis.core.io.buffer.counters.FlushSampleFlyweight;
    +import org.apache.activemq.artemis.core.io.buffer.counters.Profiler;
    +import org.apache.activemq.artemis.utils.collections.MulticastSample;
    +
    +@Command(name = "stat", description = "Provide performance statistics around the journal usage")
    --- End diff --
    
    maybe , "profiler" then if you are going to extend it in future.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic Maybe I've not understand your comment...but to capture data I use lock-free inter process communication primitives,hence no lock is involved because locks works only in the same process.
    Please look at the MultiSampleBuffer implementstion: it is the core of the whole process.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @franz1981 How i thought it worked.. you would be able to enable profile on a live server.. and be able to capture stuff from the running server.
    
    if that's not how it works.. perhaps we could change it to be that way!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic yep, let's talk tomorrow. I've just tried now but I've no issues at all so probably there is something I'm missing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119039588
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java ---
    @@ -93,10 +82,10 @@
        public TimedBuffer(final int size, final int timeout, final boolean logRates) {
           bufferSize = size;
     
    -      this.logRates = logRates;
    -
           if (logRates) {
    -         logRatesTimer = new Timer(true);
    +         this.flushProfiler = Profiler.instrumented();
    +      } else {
    +         this.flushProfiler = Profiler.none();
    --- End diff --
    
    agreed this is just different styles :) , as noted in main comment I'm happy :) 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic But the *sampler* is not strictly part of Artemis code, is a CLI command that works as a separated process from the broker. The user can choose every time a different trace file without changing the broker configuration, on the fly, while the broker is running.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r118982819
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java ---
    @@ -93,10 +82,10 @@
        public TimedBuffer(final int size, final int timeout, final boolean logRates) {
           bufferSize = size;
     
    -      this.logRates = logRates;
    -
           if (logRates) {
    --- End diff --
    
    NIT: is there a better name for this, i can't think of one, but name just seems misleading a little, as actually this is enabling a profiler not logging out anything to the std log file. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119039718
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/counters/Profiler.java ---
    @@ -0,0 +1,72 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.core.io.buffer.counters;
    +
    +import java.io.File;
    +import java.io.IOException;
    +
    +import org.apache.activemq.artemis.journal.ActiveMQJournalLogger;
    +
    +/**
    + * Factory class to instantiate {@link FlushProfiler}s classes.
    + */
    +public final class Profiler {
    +
    +   private static final FlushProfiler BLACK_HOLE_PROFILER = new FlushProfiler() {
    --- End diff --
    
    This is just different coding styles, I'm happy :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    `` It is able to capture samples from a running live serve```
    
    ^^ ok that then confirms an issue with your PR. Your CLI is trying to lock the server.. so you wouldn't ever be able to capture anything from a server as the CLI tool will never hold the lock.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    it is really a curse! Anyway thanks for the feedbacks...in particular for coding style and names...are 2 parts that make a big difference in usability of a feature, from the pov of dev/user!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r118982620
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java ---
    @@ -288,7 +266,14 @@ private void flush(final boolean force) {
                 buffer.getBytes(0, bufferToFlush);
     
                 final List<IOCallback> ioCallbacks = callbacks == null ? Collections.emptyList() : callbacks;
    -            bufferObserver.flushBuffer(bufferToFlush, pendingSyncs.get() > 0, ioCallbacks);
    +
    +            final boolean requiredSync = pendingSyncs.get() > 0;
    +            this.flushProfiler.onStartFlush(bufferToFlush.position(), requiredSync);
    --- End diff --
    
    why not a guard statement here? so if disabled the JVM would simply compile out the code when disabled.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @franz1981 you are misunderstanding me.. the CLI won't ever run concurrently with the server.. it's an issue on your CLI.. you are holding the lock on the server.. hence you're not able to capture it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    PR raised to hopefully end the sevntu curse (aka use new version published now in maven central)
    https://github.com/apache/activemq-artemis/pull/1299


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic 
    
    > So, right now the logic for missing measures is inside FlushSampleFlyweight.. while it should / should be at the buffer...
    
    The missing logic is already inside the buffer, while the FlushSampleFlyweight class is a struct-like class to write/read measurements over a ByteBuffer.
    
    > I  have moved the file creationg to MultiCastBuffer.
    
    I'm (re)thinking about putting the file naming/location logic on the MulticastBuffer and maybe is not the best place there...
    To favour the separation of concerns I prefer to leave the buffer without any other logic than what is needed for the data structure to do its job. 
    If different parts of the system will require to put all the counters in one file and/or build something more dynamic to allocate counters on demands, the MulticastBuffer (and unit tests) won't be touched by these changes.
    
    > Or maybe we could make this generic for not just the journal.. .and still have a single File.
    
    :100:  This is a very good idea!
    
    > This  could be extended to other measures.. E.G. Message sending and consuming... etc.. etc...
    
    Related to this thought, I'm thinking to release other perf counters for the broker (eg connection bytes sent/received?), but probably this will require:
    
    -  to build different file location/naming strategies 
    -  to built a manager to allocate/release dinamically the counters
    -  to build at least another type of IPC data structure without the missing recognition features (pretty simiilar to [AtomicCounter](https://github.com/real-logic/Agrona/blob/master/agrona/src/main/java/org/agrona/concurrent/status/AtomicCounter.java))
    
    Probably using directly Agrona's [CountersManager](https://github.com/real-logic/agrona/blob/master/agrona/src/main/java/org/agrona/concurrent/status/CountersManager.java) could be a good idea.
    It's a perfect fit for single Long sized messages where isn't needed the timestamp of the message and is not required to know any accuracy of the sample (aka missing samples).
    When you'll be online we could talk about it (I've a demo to show you about it) :+1: 
    
    
    
    
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r124857265
  
    --- Diff: docs/user-manual/en/tools.md ---
    @@ -220,5 +233,115 @@ COMMANDS
                 With --bindings option, The folder used for bindings (default from
                 broker.xml)
     
    +        perf-journal-stat
    +            Provide sampling of performance counters of a profiled journal
    +
    +            With --separator option, The column separator, whitespace if not
    +            specified
    +
    +            With --broker option, This would override the broker configuration
    +            from the bootstrap
    +
    +            With --verbose option, Adds more information on the execution
    +
    +            With --bytes option, Add bytes flushed column in the CSV output,
    +            false if not defined
    +
    +            With --freq option, Profile at this frequency (Hz) or will use half
    +            of the configured Journal timeout
    +
    +            With --raw-time option, Uses raw timestamps, false if not defined
    +
    +            With --out option, Print the CSV output in a text file or stdout if
    +            not defined
    +
    +            With --in option, The input counter file to be used to sample
    +            profiled data
    +
    +
    +```
    +
    +## The `perf-journal-stat` tool
    +
    +This CLI command allows to sample the journal performance counters events of a broker [configured](configuration-index.md) to emit them.
    +
    +This sampling profiling tool shares most features of Linux's [`perf-tools`](https://perf.wiki.kernel.org/index.php/Main_Page):
    +
    +- (costantly) lightweight while profiling
    +- (negligible) no performance hit while not profiling
    +- constant and very low memory footprint (i.e. just one [OS Page](https://en.wikipedia.org/wiki/Page_(computer_memory)) size in total)
    +- no Network stack involved (i.e. only Inter-Process Communication)
    +- no external dependencies
    +
    +The next simple example will illustrate how to use it, but it could be easily integrated with Nagios or any custom toolchain.
    +
    +### Gnuplot example
    +
    +Run while a proper [configured](configuration-index.md) broker is running:
    +```
    +$ ./perf-journal-stat --verbose --freq 20000 --out ~/Documents/journal_trace.csv
    --- End diff --
    
    typo? wasn't this supposed to be ./artemis perf-journal-state ???
    
    
    
    also: I thought you would run the perf tool against a running broker?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    I'm closing it , waiting to update it before reopening! :+1: 


---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r121005604
  
    --- Diff: artemis-cli/src/main/java/org/apache/activemq/artemis/cli/commands/tools/journal/JournalSamplingProfiler.java ---
    @@ -0,0 +1,307 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.cli.commands.tools.journal;
    +
    +import java.io.File;
    +import java.io.FileOutputStream;
    +import java.io.PrintStream;
    +import java.util.Arrays;
    +import java.util.Comparator;
    +import java.util.concurrent.TimeUnit;
    +import java.util.concurrent.atomic.AtomicLong;
    +import java.util.concurrent.locks.LockSupport;
    +
    +import io.airlift.airline.Command;
    +import io.airlift.airline.Option;
    +import org.apache.activemq.artemis.cli.commands.ActionContext;
    +import org.apache.activemq.artemis.cli.commands.Configurable;
    +import org.apache.activemq.artemis.core.config.impl.FileConfiguration;
    +import org.apache.activemq.artemis.core.io.buffer.counters.FlushSampleFlyweight;
    +import org.apache.activemq.artemis.core.io.buffer.counters.Profiler;
    +import org.apache.activemq.artemis.utils.collections.MulticastSample;
    +
    +@Command(name = "perf-journal-stat", description = "Provide sampling of performance counters of a profiled journal")
    +public class JournalSamplingProfiler extends Configurable {
    --- End diff --
    
    would be too much to ask renaming this to PerfJournalStat ? :)
    
    don't mean to be annoying :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic It is something I think can over-complicate the environment setting for a user and could potentially makes the mechanism less efficient too.
    That's why:
    - the file is not a "real" file on Linux (it uses [/dev/shm](https://www.cyberciti.biz/tips/what-is-devshm-and-its-practical-usage.html)) to avoid potential costs on page swapping/writeback of the dirty region, hence the position/name could not be bound to a fixed configured position
    - the file need to be zeroed on broker start and to make this behaviour safer&simpler to be implemented I prefer to let the file system do it for me, creating a new file with `pid-date of creation` info on the name (+ File::deleteOnExit), hence a fixed file name is not an option here too
    
    wdyt?
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic 
    
    > doc how to use this... (is that internal only)?
    
    :+1: You're right, but what could be a good place where put it?
    It is like the other CLI commands, so it can be used by any user, but I suspect that perf engineers and devOps engineers are the typical audience for it.
    A possible use case is to build a Cron job that during the day run it and use the extracted log to feed a time series database, an Elastic Search repository or anything suitable to collect latency samples.
    After the doc probably most of the aspect of the profiler will be clear, it uses an approach similar to perf tools on *nix.
    
    > how to read the file...
    
    The *profiler* (ie FlushProfiler) produces only one file of 4K fixed size in `/dev/shm` or  in the Artemis configured `tmp` dir: this file is not built to be read by humans or other user-provided tools right now.
    
    > I am a bit confused with the file format...
    
    The *sampler* (ie the CLI command) samples the data on the file written by the *profiler* and produces a CSV US-ASCII text trace with `<timestamp in ns><separator><latency in ns>\n` pairs.
    For example, using the default parameters:
    ```
    0 27729
    4534870 32709
    8443760 27556
    15067045 44260
    16290386 18286
    17146291 14201
    17908462 8243
    19349526 33071
    24171483 79496
    27741532 18992
    ...
    ```
    This trace, if collected in a file, can be used directly with `gnuplot` to plot this:
    ![image](https://cloud.githubusercontent.com/assets/13125299/26622962/72d67788-45ec-11e7-9e23-a6e1c054b75f.png)
    But it won't be saved in a file if not configured properly.
    
    > Also, it would be nice to be allowed to configure where the file would be generated.
    
    Currently the user running the *sampler* could choose where put the trace recorded from the profiler using the `--out` parameter:
    ```
    @Option(name = "--out", description = "The output text csv stat file or stdout if not defined")
    public String fileOut;
    ```
    By default the *sampler* command prints on `stdout` the CSV trace and on `stderr` a real time log of the statistics of the running sampler, in the form:
    ```
    Duration 1003ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1000ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 657 sample - 4 lost - 780 kB - 657 sync
    Duration 1002ms - 3,397 sample - 0 lost - 4,413 kB - 3,397 sync
    Duration 1001ms - 4,328 sample - 14 lost - 5,827 kB - 4,328 sync
    Duration 1001ms - 4,807 sample - 2 lost - 6,513 kB - 4,807 sync
    Duration 1000ms - 2,100 sample - 0 lost - 2,847 kB - 2,100 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 2,994 sample - 0 lost - 4,051 kB - 2,994 sync
    Duration 1001ms - 5,165 sample - 0 lost - 7,001 kB - 5,165 sync
    Duration 1000ms - 5,244 sample - 0 lost - 7,116 kB - 5,244 sync
    Duration 1001ms - 1,647 sample - 0 lost - 2,230 kB - 1,647 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1000ms - 3,237 sample - 0 lost - 4,381 kB - 3,237 sync
    Duration 1001ms - 4,892 sample - 0 lost - 6,616 kB - 4,892 sync
    Duration 1001ms - 4,543 sample - 0 lost - 6,120 kB - 4,543 sync
    Duration 1000ms - 2,468 sample - 0 lost - 3,281 kB - 2,468 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 1,747 sample - 1 lost - 2,374 kB - 1,747 sync
    Duration 1000ms - 5,318 sample - 0 lost - 7,224 kB - 5,318 sync
    Duration 1001ms - 5,327 sample - 0 lost - 7,233 kB - 5,327 sync
    Duration 1001ms - 2,628 sample - 0 lost - 3,565 kB - 2,628 sync
    Duration 1000ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 2,553 sample - 0 lost - 3,451 kB - 2,553 sync
    Duration 1001ms - 5,243 sample - 0 lost - 7,108 kB - 5,243 sync
    Duration 1001ms - 5,162 sample - 0 lost - 7,008 kB - 5,162 sync
    Duration 1000ms - 2,087 sample - 0 lost - 2,831 kB - 2,087 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 3,011 sample - 0 lost - 4,090 kB - 3,011 sync
    Duration 1001ms - 6,720 sample - 0 lost - 9,116 kB - 6,720 sync
    Duration 1000ms - 5,306 sample - 0 lost - 7,193 kB - 5,306 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1000ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1001ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 1000ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Duration 102ms - 0 sample - 0 lost - 0 bytes - 0 sync
    Total Duration 38132ms - 90,581 total sample - 21 total lost
    ```
    Where `lost` represent the number of lost samples due to a low frequency of the *sampler* vs the higher frequency of profiled data.
    
    
    
    
    
    
    
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by michaelandrepearce <gi...@git.apache.org>.
Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r118982923
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/counters/Profiler.java ---
    @@ -0,0 +1,72 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.core.io.buffer.counters;
    +
    +import java.io.File;
    +import java.io.IOException;
    +
    +import org.apache.activemq.artemis.journal.ActiveMQJournalLogger;
    +
    +/**
    + * Factory class to instantiate {@link FlushProfiler}s classes.
    + */
    +public final class Profiler {
    +
    +   private static final FlushProfiler BLACK_HOLE_PROFILER = new FlushProfiler() {
    --- End diff --
    
    Why not make use of java 8 default methods and define in the interface to have default methods, that do nothing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119025093
  
    --- Diff: artemis-cli/src/main/java/org/apache/activemq/artemis/cli/commands/tools/journal/StatJournal.java ---
    @@ -0,0 +1,288 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements. See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.activemq.artemis.cli.commands.tools.journal;
    +
    +import java.io.File;
    +import java.io.FileOutputStream;
    +import java.io.PrintStream;
    +import java.util.Arrays;
    +import java.util.Comparator;
    +import java.util.concurrent.TimeUnit;
    +import java.util.concurrent.atomic.AtomicLong;
    +import java.util.concurrent.locks.LockSupport;
    +
    +import io.airlift.airline.Command;
    +import io.airlift.airline.Option;
    +import org.apache.activemq.artemis.cli.commands.ActionContext;
    +import org.apache.activemq.artemis.cli.commands.Configurable;
    +import org.apache.activemq.artemis.core.config.impl.FileConfiguration;
    +import org.apache.activemq.artemis.core.io.buffer.counters.FlushSampleFlyweight;
    +import org.apache.activemq.artemis.core.io.buffer.counters.Profiler;
    +import org.apache.activemq.artemis.utils.collections.MulticastSample;
    +
    +@Command(name = "stat", description = "Provide performance statistics around the journal usage")
    --- End diff --
    
    It is a weird name...Do you have something in mind?
    It is a journal sampling profiler and the ideas on how it works are taken from:
    - https://linux.die.net/man/1/perf-record
    - https://github.com/jvm-profiling-tools/honest-profiler
    But in the future I'm planning to provide a single CLI command that could collect perf data using a filter of some kind or different CLI commands for macro profiling areas ...it is something I want to discuss on the dev list...



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    these are some suggested changes: https://github.com/clebertsuconic/activemq-artemis/commit/c1490de200d373925f455b79c561d535a62f7248


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r124868500
  
    --- Diff: docs/user-manual/en/tools.md ---
    @@ -220,5 +233,115 @@ COMMANDS
                 With --bindings option, The folder used for bindings (default from
                 broker.xml)
     
    +        perf-journal-stat
    +            Provide sampling of performance counters of a profiled journal
    +
    +            With --separator option, The column separator, whitespace if not
    +            specified
    +
    +            With --broker option, This would override the broker configuration
    +            from the bootstrap
    +
    +            With --verbose option, Adds more information on the execution
    +
    +            With --bytes option, Add bytes flushed column in the CSV output,
    +            false if not defined
    +
    +            With --freq option, Profile at this frequency (Hz) or will use half
    +            of the configured Journal timeout
    +
    +            With --raw-time option, Uses raw timestamps, false if not defined
    +
    +            With --out option, Print the CSV output in a text file or stdout if
    +            not defined
    +
    +            With --in option, The input counter file to be used to sample
    +            profiled data
    +
    +
    +```
    +
    +## The `perf-journal-stat` tool
    +
    +This CLI command allows to sample the journal performance counters events of a broker [configured](configuration-index.md) to emit them.
    +
    +This sampling profiling tool shares most features of Linux's [`perf-tools`](https://perf.wiki.kernel.org/index.php/Main_Page):
    +
    +- (costantly) lightweight while profiling
    +- (negligible) no performance hit while not profiling
    +- constant and very low memory footprint (i.e. just one [OS Page](https://en.wikipedia.org/wiki/Page_(computer_memory)) size in total)
    +- no Network stack involved (i.e. only Inter-Process Communication)
    +- no external dependencies
    +
    +The next simple example will illustrate how to use it, but it could be easily integrated with Nagios or any custom toolchain.
    +
    +### Gnuplot example
    +
    +Run while a proper [configured](configuration-index.md) broker is running:
    +```
    +$ ./perf-journal-stat --verbose --freq 20000 --out ~/Documents/journal_trace.csv
    --- End diff --
    
    The docs is great.. small mistake there.. but I'm having some issues on how to operate this.. lets talk offline.. and update here later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @franz1981 I tried.. cannot run .. because of the lock... lets just talk over tomorrow..


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    I think this could be made a bit more generic.. So I have moved the file creationg to MultiCastBuffer.
    
    This could be extended to other measures.. E.G. Message sending and consuming... etc.. etc... 
    
    So, right now the logic for missing measures is inside FlushSampleFlyweight.. while it should / should be at the buffer...
    
    
    
    I still think we should name this PipedBuffer.. you could create many pipes. .one for the journal, one for messages.. .etc.. etc...   it's a pipe of information.. where you can have multiple receivers...   
    
    
    if you create a chain of information (I'm calling pipe.. we can find other names) for messages.. it could be there... 
    
    
    Or maybe we could make this generic for not just the journal.. .and still have a single File.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by mtaylor <gi...@git.apache.org>.
Github user mtaylor commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic @franz1981 This has been open for some time now.  Can you work together to resolve this, either by merging or closing.  If this need significant refactory/impl work that is going to take some time then I suggest we close and reopen when it's ready.


---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic I've fixed the typo error and about "how it works" we could talk tomorrow about it :+1: 
    I'm happy you like the doc :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic It is able to capture samples from a running live server, but the server must be first started with the proper configuration. 
    It is simply not possible to make it light as it is with other options: that's why I've build the MulticastBuffer to not backpressure the broker when the sampling profiler is not reading data.
    Making it differently will at least add branch mispredictions and JVM compile deoptimizations in the hot paths. 
    Hence you can let a broker be configured to emit performance counters data with no performance impact, making it suitable to be used in production too.
    It is really like most perf profilers work: if you want profiling you need at least some debug info to capture meaningful data.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @franz1981 the only reason I didn't merge this is that this would be a hidden feature... no docs.. 
    
    can you add a doc chapter about this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @franz1981  I was just looking from the POV of the code.. TimedBuffer->profiler (whatever the name is) is instantiating an output file. shouldn't it be configurable for the running server?  the CLI to read it could also get the file from the configuration if you're running from the instance.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 closed the pull request at:

    https://github.com/apache/activemq-artemis/pull/1295


---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    @clebertsuconic I've finally changed the name of MulticastSample into MulticastBuffer and added a doc chapter with an example, but I need a review on it, in particular by someone that doesn't know anything about how the profiler works!wdyt?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis pull request #1295: ARTEMIS-1185 Inter-Process Journal Samp...

Posted by franz1981 <gi...@git.apache.org>.
Github user franz1981 commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1295#discussion_r119023463
  
    --- Diff: artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java ---
    @@ -93,10 +82,10 @@
        public TimedBuffer(final int size, final int timeout, final boolean logRates) {
           bufferSize = size;
     
    -      this.logRates = logRates;
    -
           if (logRates) {
    -         logRatesTimer = new Timer(true);
    +         this.flushProfiler = Profiler.instrumented();
    +      } else {
    +         this.flushProfiler = Profiler.none();
    --- End diff --
    
    About the compiler we're "lucky", because the noop impl will be inlined (is monomorphic with no code inside) and then wiped away (verified using JitWatch), leaving the TimedBuffer clean and without null checks too.
    The guard statement that can be compiled out AFAIK are the ones that rely on final static fields, while normal instance members do not receive the same treatment from the JIT, sadly.
    It is a different coding style: like having a null logger instance member or a noop logger implementation...wdyt?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] activemq-artemis issue #1295: ARTEMIS-1185 Inter-Process Journal Sampler Pro...

Posted by clebertsuconic <gi...@git.apache.org>.
Github user clebertsuconic commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1295
  
    ``Currently the user running the sampler could choose where put the trace recorded from the profiler using the --out parameter:``
    
    There's an option on the configuration. I think it's logRates.. something like that... if you used the config for that, you won't have to configure the --out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---