You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nlpcraft.apache.org by ar...@apache.org on 2020/09/20 06:41:42 UTC
[incubator-nlpcraft-website] branch master updated: WIP on 0.7.0
changes.
This is an automated email from the ASF dual-hosted git repository.
aradzinski pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-nlpcraft-website.git
The following commit(s) were added to refs/heads/master by this push:
new f6c9495 WIP on 0.7.0 changes.
f6c9495 is described below
commit f6c9495bd860a90fb2ed6103442d58e07de639e1
Author: Aaron Radzinski <ar...@datalingvo.com>
AuthorDate: Sat Sep 19 23:41:30 2020 -0700
WIP on 0.7.0 changes.
---
_data/metrics.yml | 6 ++++
server-and-probe.html | 99 +++++++++++++++++++++++++++++++++++++++++++++++++--
using-rest.html | 21 +++++++++++
3 files changed, 123 insertions(+), 3 deletions(-)
diff --git a/_data/metrics.yml b/_data/metrics.yml
index 800441a..93862fa 100644
--- a/_data/metrics.yml
+++ b/_data/metrics.yml
@@ -82,6 +82,12 @@ server:
- name: probe_all
description: <code>/probe/all</code> REST call
+ - name: health
+ description: <code>/health</code> REST call
+
+ - name: model_subsyn
+ description: <code>/model/sugsyn</code> REST call
+
- name: roundtrip
description: server-probe-server rountrip
diff --git a/server-and-probe.html b/server-and-probe.html
index a1b0bbc..33e0735 100644
--- a/server-and-probe.html
+++ b/server-and-probe.html
@@ -474,13 +474,106 @@ nlpcraft {
</p>
<p>
The challenge is that from the Maven build you need to start the server, wait til its fully started and
- initialized, and only then start the data probes or run tests that use embedded probes. When done manually
- (e.g. from IDE) you can visually observe when the server finished its startup and then manually
+ initialized, and only then start issuing REST calls, start data probes or run tests that use <a href="/tools/embedded_probe.html">embedded probes</a>.
+ When done manually (e.g. from IDE) you can visually observe when the server finished its startup and then manually
launch the tests. In Maven, however, you need to use a special plugin to accomplish the same in
automated fashion.
</p>
+ <div class="bq info">
+ <b>Probe Waits For Server</b>
+ <p>
+ Technically, when a data probe starts up it will initialize, load the models, and will automatically wait for the server to get online
+ if it isn't yet (as well as periodically check for it). Once server is online the data probe will automatically connect to it. However,
+ if the unit tests don't use data probe and just issue REST calls then these tests have to somehow wait for the
+ server to get online.
+ </p>
+ <p>
+ To overcome this challenge you can use
+ <a target="github" href="https://github.com/bazaarvoice/maven-process-plugin"><code>process-exec-maven-plugin</code></a>
+ Maven plugin.
+ </p>
+ </div>
<p>
+ To get around this problem NLPCraft uses <a target="github" href="https://github.com/bazaarvoice/maven-process-plugin"><code>process-exec-maven-plugin</code></a>
+ Maven plugin in its own build. This plugin allows to start the external process and use configured URL
+ endpoint to check whether or not the external process has fully started. This works perfect with NLPCraft server
+ <a href="using-rest.html#misc">health check REST call</a>. The plugin can be configured in the following way for your own project
+ (taken directly from NLPCraft <a target="github" href="https://github.com/apache/incubator-nlpcraft/blob/master/nlpcraft/pom.xml"><code>pom.xml</code></a>):
</p>
+ <pre class="brush: xml, highlight: [14, 16, 21]">
+<plugin>
+ <groupId>com.bazaarvoice.maven.plugins</groupId>
+ <artifactId>process-exec-maven-plugin</artifactId>
+ <version>0.9</version>
+ <executions>
+ <execution>
+ <id>pre-integration-test</id>
+ <phase>pre-integration-test</phase>
+ <goals>
+ <goal>start</goal>
+ </goals>
+ <configuration>
+ <name>server</name>
+ <healthcheckUrl>http://localhost:8081/api/v1/health</healthcheckUrl>
+ <waitAfterLaunch>180</waitAfterLaunch>
+ <processLogFile>${project.build.directory}/server.log</processLogFile>
+ <arguments>
+ <argument>java</argument>
+ <argument>-Xmx4G</argument>
+ <argument>-Xms4G</argument>
+ <argument>-DNLPCRAFT_ANSI_COLOR_DISABLED=true</argument>
+ <argument>-jar</argument>
+ <argument>${project.build.directory}/${nlpcraft.all.deps.jar}</argument>
+ <argument>-server</argument>
+ </arguments>
+ </configuration>
+ </execution>
+ <execution>
+ <id>stop-all</id>
+ <phase>post-integration-test</phase>
+ <goals>
+ <goal>stop-all</goal>
+ </goals>
+ </execution>
+ </executions>
+</plugin>
+ </pre>
+ <p>
+ <b>NOTES</b>:
+ </p>
+ <ul>
+ <li>
+ On line 14 we specify the URL endpoint to check whether or not our server is online. We use
+ <code>/health</code> localhost REST call for that.
+ </li>
+ <li>
+ On line 16 we redirect the output from server to a dedicated file to <b>avoid interleaving</b> log
+ from server and log from data probe in the same console (where we are running the Maven build from).
+ Such interleaving will make the combined log unreadable and can cause output problem for the console
+ due to mixed up ANSI escape sequences from server and data probe.
+ </li>
+ <li>
+ Since the server log is piped out to a separate file we disable ANSI coloring for the server
+ on the line 21.
+ </li>
+ </ul>
+ <div class="bq info">
+ <b>Avoid Interleaving Logs</b>
+ <p>
+ When running both server and the data probe(s) from the Maven build it is important to avoid interleaving
+ logs from the server and the probe. Such interleaving will make the combined log in Maven unreadable and
+ can cause console malfunction due to mixed up ANSI escape codes. It is idiomatic in such cases to:
+ </p>
+ <ul>
+ <li>
+ Disable ANSI coloring for the server via <code>-D<b>NLPCRAFT_ANSI_COLOR_DISABLED</b>=true</code>
+ </li>
+ <li>
+ Pipe out the server log into a dedicated file using Maven plugins like
+ <a target="github" href="https://github.com/bazaarvoice/maven-process-plugin"><code>process-exec-maven-plugin</code></a>.
+ </li>
+ </ul>
+ </div>
</section>
</div>
<div class="col-md-2 third-column">
@@ -491,7 +584,7 @@ nlpcraft {
<li><a href="#config">Configuration</a></li>
<li><a href="#override">Custom Configuration</a></li>
<li><a href="#ansi">ANSI Colors</a></li>
- <li><a href="#override">Testing</a></li>
+ <li><a href="#testing">Testing</a></li>
{% include quick-links.html %}
</ul>
</div>
diff --git a/using-rest.html b/using-rest.html
index 85e7d34..5fc4e53 100644
--- a/using-rest.html
+++ b/using-rest.html
@@ -99,6 +99,7 @@ id: rest
<li><a href="#user">User Management</a></li>
<li><a href="#data_probe">Data Probe Management</a></li>
<li><a href="#model">Model Operations</a></li>
+ <li><a href="#misc">Miscellaneous</a></li>
</ul>
<div class="bq info">
<b>Admins</b>
@@ -336,6 +337,26 @@ id: rest
</tbody>
</table>
</section>
+ <section id="misc">
+ <h3 class="section-title">Miscellaneous Operations</h3>
+ <p>
+ This group of operations allows to perform various miscellaneous and utility operations:
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>REST call</th>
+ <th>Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><a target="github" href="https://github.com/apache/incubator-nlpcraft/blob/master/openapi/nlpcraft_swagger.yml"><nobr><code>/health</code></nobr></a></td>
+ <td>Performs a health ping for the server. Can be used by periodic monitoring tools.</td>
+ </tr>
+ </tbody>
+ </table>
+ </section>
<section id="server_errors">
<h2 class="section-title">Server Errors</h2>
<p>