You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streampipes.apache.org by ri...@apache.org on 2022/02/04 22:43:43 UTC
[incubator-streampipes] branch experimental-module-federation-494 updated (29612cb -> e123c80)
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a change to branch experimental-module-federation-494
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes.git.
from 29612cb [hotfix] Show possible values of tags in data explorer query config
new 633b2aa [hotfix] Extract correct instanceId in health check
new fb8e7f7 [hotfix] Remove non-used pipeline template generation code
new 0c46ccc [hotfix] Do not use random topics in pipeline generation
new 5e07c76 [STREAMPIPES-494] Do not show pipeline element status in preview mode
new e123c80 [STREAMPIPES-494] Fix layout bug in pipeline details component
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../backend/StreamPipesResourceConfig.java | 1 -
.../performance/pipeline/PipelineGenerator.java | 112 ---------------
.../manager/health/PipelineHealthCheck.java | 3 +-
.../manager/matching/ProtocolSelector.java | 2 +-
.../streampipes/manager/operations/Operations.java | 25 ----
.../manager/template/PipelineGenerator.java | 143 -------------------
.../template/PipelineTemplateGenerator.java | 157 ---------------------
.../PipelineTemplateInvocationGenerator.java | 84 -----------
.../PipelineTemplateInvocationHandler.java | 101 -------------
.../instances/DashboardPipelineTemplate.java | 45 ------
.../instances/DataLakePipelineTemplate.java | 45 ------
.../instances/DelmeExamplePipelineTemplate.java | 51 -------
.../instances/ElasticsearchPipelineTemplate.java | 45 ------
.../FilterElasticSearchPipelineTemplate.java | 46 ------
.../template/instances/FilterPipelineTemplate.java | 46 ------
.../template/instances/PipelineTemplate.java | 30 ----
.../streampipes/rest/impl/PipelineTemplate.java | 131 -----------------
.../components/pipeline/pipeline.component.html | 2 +-
ui/src/app/editor/services/jsplumb.service.ts | 1 -
.../services/pipeline-positioning.service.ts | 1 -
.../monitoring/pipeline-monitoring.component.html | 4 -
.../preview/pipeline-preview.component.ts | 2 +-
.../pipeline-details.component.html | 14 +-
23 files changed, 13 insertions(+), 1078 deletions(-)
delete mode 100644 streampipes-performance-tests/src/main/java/org/apache/streampipes/performance/pipeline/PipelineGenerator.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineGenerator.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateGenerator.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationGenerator.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationHandler.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DashboardPipelineTemplate.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DataLakePipelineTemplate.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DelmeExamplePipelineTemplate.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/ElasticsearchPipelineTemplate.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterElasticSearchPipelineTemplate.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterPipelineTemplate.java
delete mode 100644 streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/PipelineTemplate.java
delete mode 100644 streampipes-rest/src/main/java/org/apache/streampipes/rest/impl/PipelineTemplate.java
[incubator-streampipes] 05/05: [STREAMPIPES-494] Fix layout bug in pipeline details component
Posted by ri...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch experimental-module-federation-494
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes.git
commit e123c802550374cacfeeb474752d85bb3d46b50d
Author: Dominik Riemer <do...@gmail.com>
AuthorDate: Fri Feb 4 23:43:29 2022 +0100
[STREAMPIPES-494] Fix layout bug in pipeline details component
---
ui/src/app/editor/services/jsplumb.service.ts | 1 -
ui/src/app/editor/services/pipeline-positioning.service.ts | 1 -
.../monitoring/pipeline-monitoring.component.html | 4 ----
.../components/preview/pipeline-preview.component.ts | 2 +-
.../app/pipeline-details/pipeline-details.component.html | 14 ++++++++------
5 files changed, 9 insertions(+), 13 deletions(-)
diff --git a/ui/src/app/editor/services/jsplumb.service.ts b/ui/src/app/editor/services/jsplumb.service.ts
index 2277a16..fb1712c 100644
--- a/ui/src/app/editor/services/jsplumb.service.ts
+++ b/ui/src/app/editor/services/jsplumb.service.ts
@@ -195,7 +195,6 @@ export class JsplumbService {
y: coordinates.y
}
};
- console.log(pipelineElementConfig);
if (!pipelineElementConfig.payload.dom) {
pipelineElementConfig.payload.dom = 'jsplumb_' + this.idCounter + '_' + this.makeId(4);
this.idCounter++;
diff --git a/ui/src/app/editor/services/pipeline-positioning.service.ts b/ui/src/app/editor/services/pipeline-positioning.service.ts
index 478dc1f..2293d0b 100644
--- a/ui/src/app/editor/services/pipeline-positioning.service.ts
+++ b/ui/src/app/editor/services/pipeline-positioning.service.ts
@@ -177,7 +177,6 @@ export class PipelinePositioningService {
pe.payload.connectedTo.forEach((connection, index) => {
source = connection;
target = pe.payload.dom;
-
const sourceEndpointId = 'out-' + connection;
const inTargetEndpointId = 'in-' + index + '-' + pe.payload.dom;
jsPlumbBridge.connect(
diff --git a/ui/src/app/pipeline-details/components/monitoring/pipeline-monitoring.component.html b/ui/src/app/pipeline-details/components/monitoring/pipeline-monitoring.component.html
index c53c3d5..4c27a46 100644
--- a/ui/src/app/pipeline-details/components/monitoring/pipeline-monitoring.component.html
+++ b/ui/src/app/pipeline-details/components/monitoring/pipeline-monitoring.component.html
@@ -26,10 +26,6 @@
</div>
</div>
<div fxFlex="100" fxLayout="column" class="page-container-padding-inner">
- <pipeline-preview [jspcanvas]="'assembly-quickedit'" [pipeline]="pipeline"
- (selectedElementEmitter)="selectElement($event)"
- style="margin-bottom:15px;"></pipeline-preview>
-
<div fxFlex="100" *ngIf="!pipeline.running" fxLayout="column" fxLayoutAlign="center center">
<div class="error-message">(monitoring info is only available for running pipelines)</div>
<button mat-button mat-raised-button color="accent"
diff --git a/ui/src/app/pipeline-details/components/preview/pipeline-preview.component.ts b/ui/src/app/pipeline-details/components/preview/pipeline-preview.component.ts
index 6520b90..564ccc9 100644
--- a/ui/src/app/pipeline-details/components/preview/pipeline-preview.component.ts
+++ b/ui/src/app/pipeline-details/components/preview/pipeline-preview.component.ts
@@ -16,7 +16,7 @@
*
*/
-import { Component, EventEmitter, Input, OnInit, Output } from '@angular/core';
+import { Component, EventEmitter, Input, OnDestroy, OnInit, Output } from '@angular/core';
import { Pipeline } from '@streampipes/platform-services';
import { PipelineElementConfig, PipelineElementUnion } from '../../../editor/model/editor.model';
import { PipelinePositioningService } from '../../../editor/services/pipeline-positioning.service';
diff --git a/ui/src/app/pipeline-details/pipeline-details.component.html b/ui/src/app/pipeline-details/pipeline-details.component.html
index a14dac6..2733ce1 100644
--- a/ui/src/app/pipeline-details/pipeline-details.component.html
+++ b/ui/src/app/pipeline-details/pipeline-details.component.html
@@ -29,15 +29,18 @@
</div>
</div>
</div>
-
+ <div fxFlex="100" fxLayout="column">
+ <pipeline-preview [jspcanvas]="'assembly-preview'"
+ [pipeline]="pipeline"
+ (selectedElementEmitter)="selectElement($event)"
+ style="margin-bottom:15px;"
+ class="md-padding"
+ *ngIf="pipelineAvailable"></pipeline-preview>
<div fxFlex fxLayout="row" fxLayoutAlign="start top" *ngIf="pipelineAvailable && selectedIndex == 0">
<div fxFlex="60" class="md-padding">
<div fxFlex fxLayout="column">
- <pipeline-preview [jspcanvas]="'assembly-preview'" [pipeline]="pipeline" (selectedElementEmitter)="selectElement($event)"
- style="margin-bottom:15px;"></pipeline-preview>
<pipeline-elements [pipeline]="pipeline"
[selectedElement]="selectedElement"></pipeline-elements>
-
</div>
</div>
<div fxFlex="40" class="md-padding">
@@ -54,10 +57,9 @@
<div fxFlex fxLayout="row" fxLayoutAlign="start top" *ngIf="pipelineAvailable && selectedIndex == 3">
<div fxFlex="100" class="md-padding">
<div fxFlex fxLayout="column">
- <pipeline-preview [jspcanvas]="'assembly-quickedit'" [pipeline]="pipeline" (selectedElementEmitter)="selectElement($event)"
- style="margin-bottom:15px;"></pipeline-preview>
<quick-edit [pipeline]="pipeline" [selectedElement]="selectedElement" (reloadPipelineEmitter)="loadPipeline()"></quick-edit>
</div>
</div>
</div>
+ </div>
</div>
[incubator-streampipes] 02/05: [hotfix] Remove non-used pipeline template generation code
Posted by ri...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch experimental-module-federation-494
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes.git
commit fb8e7f7d1ade6d494f002f65f5e3b1a8c0532ec0
Author: Dominik Riemer <do...@gmail.com>
AuthorDate: Fri Feb 4 22:37:45 2022 +0100
[hotfix] Remove non-used pipeline template generation code
---
.../backend/StreamPipesResourceConfig.java | 1 -
.../performance/pipeline/PipelineGenerator.java | 112 ---------------
.../streampipes/manager/operations/Operations.java | 25 ----
.../manager/template/PipelineGenerator.java | 143 -------------------
.../template/PipelineTemplateGenerator.java | 157 ---------------------
.../PipelineTemplateInvocationGenerator.java | 84 -----------
.../PipelineTemplateInvocationHandler.java | 101 -------------
.../instances/DashboardPipelineTemplate.java | 45 ------
.../instances/DataLakePipelineTemplate.java | 45 ------
.../instances/DelmeExamplePipelineTemplate.java | 51 -------
.../instances/ElasticsearchPipelineTemplate.java | 45 ------
.../FilterElasticSearchPipelineTemplate.java | 46 ------
.../template/instances/FilterPipelineTemplate.java | 46 ------
.../template/instances/PipelineTemplate.java | 30 ----
.../streampipes/rest/impl/PipelineTemplate.java | 131 -----------------
15 files changed, 1062 deletions(-)
diff --git a/streampipes-backend/src/main/java/org/apache/streampipes/backend/StreamPipesResourceConfig.java b/streampipes-backend/src/main/java/org/apache/streampipes/backend/StreamPipesResourceConfig.java
index fd3c485..a57ad7d 100644
--- a/streampipes-backend/src/main/java/org/apache/streampipes/backend/StreamPipesResourceConfig.java
+++ b/streampipes-backend/src/main/java/org/apache/streampipes/backend/StreamPipesResourceConfig.java
@@ -88,7 +88,6 @@ public class StreamPipesResourceConfig extends ResourceConfig {
register(PipelineElementPreview.class);
register(PipelineElementRuntimeInfo.class);
register(PipelineMonitoring.class);
- register(PipelineTemplate.class);
register(PipelineResource.class);
register(DataSinkResource.class);
register(DataProcessorResource.class);
diff --git a/streampipes-performance-tests/src/main/java/org/apache/streampipes/performance/pipeline/PipelineGenerator.java b/streampipes-performance-tests/src/main/java/org/apache/streampipes/performance/pipeline/PipelineGenerator.java
deleted file mode 100644
index e5c25ed..0000000
--- a/streampipes-performance-tests/src/main/java/org/apache/streampipes/performance/pipeline/PipelineGenerator.java
+++ /dev/null
@@ -1,112 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-package org.apache.streampipes.performance.pipeline;
-
-import org.apache.streampipes.commons.exceptions.NoMatchingJsonSchemaException;
-import org.apache.streampipes.commons.exceptions.NoSepaInPipelineException;
-import org.apache.streampipes.commons.exceptions.RemoteServerNotAccessibleException;
-import org.apache.streampipes.manager.matching.PipelineVerificationHandler;
-import org.apache.streampipes.model.SpDataStream;
-import org.apache.streampipes.model.client.exception.InvalidConnectionException;
-import org.apache.streampipes.model.pipeline.Pipeline;
-import org.apache.streampipes.model.message.PipelineModificationMessage;
-import org.apache.streampipes.model.graph.DataProcessorDescription;
-import org.apache.streampipes.model.graph.DataProcessorInvocation;
-import org.apache.streampipes.model.graph.DataSinkDescription;
-import org.apache.streampipes.model.graph.DataSinkInvocation;
-import org.apache.streampipes.model.graph.DataSourceDescription;
-import org.apache.streampipes.performance.model.PerformanceTestSettings;
-
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.List;
-
-public class PipelineGenerator {
-
- private PerformanceTestSettings settings;
-
-// private static final String backendUrl = "http://backend:8030";
-// private static final String sourcesUrl = "http://pe-examples-sources:8090";
-// private static final String flinkUrl = "http://pe-flink:8094";
-
- private DataSourceDescription dataSourceDescription;
- private DataProcessorDescription timestampEnrichmentDescription;
- private DataSinkDescription elasticSearchDescription;
-
- public PipelineGenerator(PerformanceTestSettings settings) {
- this.settings = settings;
- this.dataSourceDescription = fetchDataSourceDescription();
- this.timestampEnrichmentDescription = fetchTimestampEnrichmentDescription();
- this.elasticSearchDescription = fetchElasticSearchDecription();
- }
-
- private DataSinkDescription fetchElasticSearchDecription() {
-
- return null;
- }
-
- private DataProcessorDescription fetchTimestampEnrichmentDescription() {
-
- return null;
- }
-
- private DataSourceDescription fetchDataSourceDescription() {
-
- return null;
- }
-
- public Pipeline buildPipeline() throws NoSepaInPipelineException, InvalidConnectionException, RemoteServerNotAccessibleException, NoMatchingJsonSchemaException {
- Pipeline pipeline = new Pipeline();
- pipeline.setStreams(buildStreams());
- pipeline.setActions(new ArrayList<>());
- pipeline.setSepas(new ArrayList<>());
-
- String lastElementId = "stream0";
- for (Integer i = 0; i < settings.getNumberOfTimestampEnrichmentEpas(); i++) {
- DataProcessorInvocation invocation = new DataProcessorInvocation(timestampEnrichmentDescription);
- invocation.setDOM("epa" + i);
- invocation.setConnectedTo(Arrays.asList(lastElementId));
- lastElementId = "epa" + i;
- pipeline.getSepas().add(invocation);
- PipelineModificationMessage message = verifyPipeline(pipeline);
- //pipeline.getSepas().get(i).setStaticProperties(modifyTimestampEnricher(message.getPipelineModifications().get
- // (0)));
- }
-
- DataSinkInvocation dataSinkInvocation = new DataSinkInvocation(elasticSearchDescription);
- dataSinkInvocation.setConnectedTo(Arrays.asList(lastElementId));
- dataSinkInvocation.setDOM("action0");
- pipeline.getActions().add(dataSinkInvocation);
-
- PipelineModificationMessage message = verifyPipeline(pipeline);
- //message.getPipelineModifications().get(0)
- return null;
-
- }
-
- private List<SpDataStream> buildStreams() {
- //Optional<SpDataStream> streamOpt = dataSourceDescription.getSpDataStreams().
- return null;
- }
-
- private PipelineModificationMessage verifyPipeline(Pipeline pipeline) throws NoSepaInPipelineException, InvalidConnectionException, NoMatchingJsonSchemaException, RemoteServerNotAccessibleException {
- return new PipelineVerificationHandler(pipeline).validateConnection()
- .computeMappingProperties().getPipelineModificationMessage();
- }
-
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/operations/Operations.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/operations/Operations.java
index 3b1e5fa..2aaf29c 100644
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/operations/Operations.java
+++ b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/operations/Operations.java
@@ -29,9 +29,6 @@ import org.apache.streampipes.manager.matching.PipelineVerificationHandlerV2;
import org.apache.streampipes.manager.recommender.ElementRecommender;
import org.apache.streampipes.manager.remote.ContainerProvidedOptionsHandler;
import org.apache.streampipes.manager.runtime.PipelineElementRuntimeInfoFetcher;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.manager.template.PipelineTemplateInvocationGenerator;
-import org.apache.streampipes.manager.template.PipelineTemplateInvocationHandler;
import org.apache.streampipes.manager.topic.WildcardTopicGenerator;
import org.apache.streampipes.manager.verification.extractor.TypeExtractor;
import org.apache.streampipes.model.SpDataSet;
@@ -46,8 +43,6 @@ import org.apache.streampipes.model.pipeline.PipelineElementRecommendationMessag
import org.apache.streampipes.model.pipeline.PipelineOperationStatus;
import org.apache.streampipes.model.runtime.RuntimeOptionsRequest;
import org.apache.streampipes.model.runtime.RuntimeOptionsResponse;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.model.template.PipelineTemplateInvocation;
import org.apache.streampipes.storage.management.StorageDispatcher;
import java.util.ArrayList;
@@ -164,26 +159,6 @@ public class Operations {
return new ContainerProvidedOptionsHandler().fetchRemoteOptions(request);
}
- public static List<PipelineTemplateDescription> getAllPipelineTemplates() {
- return new PipelineTemplateGenerator().getAllPipelineTemplates();
- }
-
- public static List<PipelineTemplateDescription> getCompatiblePipelineTemplates(String streamId) {
- return new PipelineTemplateGenerator().getCompatibleTemplates(streamId);
- }
-
- public static PipelineOperationStatus handlePipelineTemplateInvocation(String username, PipelineTemplateInvocation pipelineTemplateInvocation) {
- return new PipelineTemplateInvocationHandler(username, pipelineTemplateInvocation).handlePipelineInvocation();
- }
-
- public static PipelineOperationStatus handlePipelineTemplateInvocation(String username, PipelineTemplateInvocation pipelineTemplateInvocation, PipelineTemplateDescription pipelineTemplateDescription) {
- return new PipelineTemplateInvocationHandler(username, pipelineTemplateInvocation, pipelineTemplateDescription).handlePipelineInvocation();
- }
-
- public static PipelineTemplateInvocation getPipelineInvocationTemplate(SpDataStream dataStream, PipelineTemplateDescription pipelineTemplateDescription) {
- return new PipelineTemplateInvocationGenerator(dataStream, pipelineTemplateDescription).generateInvocation();
- }
-
public static String getRuntimeInfo(SpDataStream spDataStream) throws SpRuntimeException {
return PipelineElementRuntimeInfoFetcher.INSTANCE.getCurrentData(spDataStream);
}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineGenerator.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineGenerator.java
deleted file mode 100644
index ad0ba55..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineGenerator.java
+++ /dev/null
@@ -1,143 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-package org.apache.streampipes.manager.template;
-
-import org.apache.streampipes.commons.exceptions.NoSepaInPipelineException;
-import org.apache.streampipes.manager.matching.DataSetGroundingSelector;
-import org.apache.streampipes.manager.matching.PipelineVerificationHandler;
-import org.apache.streampipes.model.SpDataSet;
-import org.apache.streampipes.model.SpDataStream;
-import org.apache.streampipes.model.base.InvocableStreamPipesEntity;
-import org.apache.streampipes.model.client.exception.InvalidConnectionException;
-import org.apache.streampipes.model.graph.DataProcessorInvocation;
-import org.apache.streampipes.model.graph.DataSinkInvocation;
-import org.apache.streampipes.model.message.DataSetModificationMessage;
-import org.apache.streampipes.model.message.PipelineModificationMessage;
-import org.apache.streampipes.model.pipeline.Pipeline;
-import org.apache.streampipes.model.template.BoundPipelineElement;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.storage.management.StorageManager;
-
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-import java.util.UUID;
-
-public class PipelineGenerator {
-
- private PipelineTemplateDescription pipelineTemplateDescription;
- private String datasetId;
- private Pipeline pipeline;
- private String pipelineName;
-
- private int count = 0;
-
- public PipelineGenerator(String datasetId, PipelineTemplateDescription pipelineTemplateDescription, String pipelineName) {
- this.pipelineTemplateDescription = pipelineTemplateDescription;
- this.datasetId = datasetId;
- this.pipelineName = pipelineName;
- this.pipeline = new Pipeline();
- }
-
- public Pipeline makePipeline() {
-
- pipeline.setName(pipelineName);
- pipeline.setPipelineId(UUID.randomUUID().toString());
-
- pipeline.setStreams(Collections.singletonList(prepareStream(datasetId)));
- pipeline.setSepas(new ArrayList<>());
- pipeline.setActions(new ArrayList<>());
- collectInvocations("domId" + count, pipelineTemplateDescription.getBoundTo());
-
- return pipeline;
- }
-
- private SpDataStream prepareStream(String streamId) {
- SpDataStream stream = getStream(streamId);
- if (stream instanceof SpDataSet) {
- stream = new SpDataSet((SpDataSet) stream);
- DataSetModificationMessage message = new DataSetGroundingSelector((SpDataSet) stream).selectGrounding();
- stream.setEventGrounding(message.getEventGrounding());
- ((SpDataSet) stream).setDatasetInvocationId(message.getInvocationId());
- } else {
- stream = new SpDataStream(stream);
- }
- stream.setDOM(getDom());
- return stream;
- }
-
- private void collectInvocations(String currentDomId, List<BoundPipelineElement> boundPipelineElements) {
- for (BoundPipelineElement pipelineElement : boundPipelineElements) {
- InvocableStreamPipesEntity entity = clonePe(pipelineElement.getPipelineElementTemplate());
- entity.setConnectedTo(Collections.singletonList(currentDomId));
- entity.setDOM(getDom());
- //entity.setConfigured(true);
- // TODO hack
- //entity.setInputStreams(Arrays.asList(inputStream));
- if (entity instanceof DataProcessorInvocation) {
- pipeline.getSepas().add((DataProcessorInvocation) entity);
- try {
- PipelineModificationMessage message = new PipelineVerificationHandler(pipeline).validateConnection().computeMappingProperties().getPipelineModificationMessage();
- //entity.setInputStreams(message.getPipelineModifications().get(0).getInputStreams());
- pipeline.getSepas().remove(entity);
- entity.setConfigured(true);
- entity.setStaticProperties(message.getPipelineModifications().get(0).getStaticProperties());
- pipeline.getSepas().add((DataProcessorInvocation) entity);
- } catch (NoSepaInPipelineException | InvalidConnectionException e) {
- e.printStackTrace();
- }
- if (pipelineElement.getConnectedTo().size() > 0) {
- collectInvocations(entity.getDOM(), pipelineElement.getConnectedTo());
- }
- } else {
- pipeline.getActions().add((DataSinkInvocation) entity);
- try {
- PipelineModificationMessage message = new PipelineVerificationHandler(pipeline).validateConnection().computeMappingProperties().getPipelineModificationMessage();
- pipeline.getActions().remove(entity);
- //entity.setInputStreams(message.getPipelineModifications().get(0).getInputStreams());
- entity.setConfigured(true);
- entity.setStaticProperties(message.getPipelineModifications().get(0).getStaticProperties());
- pipeline.getActions().add((DataSinkInvocation) entity);
- } catch ( NoSepaInPipelineException | InvalidConnectionException e) {
- e.printStackTrace();
- }
- }
- }
- }
-
- private InvocableStreamPipesEntity clonePe(InvocableStreamPipesEntity pipelineElementTemplate) {
- if (pipelineElementTemplate instanceof DataProcessorInvocation) {
- return new DataProcessorInvocation((DataProcessorInvocation) pipelineElementTemplate);
- } else {
- return new DataSinkInvocation((DataSinkInvocation) pipelineElementTemplate);
- }
- }
-
- private SpDataStream getStream(String datasetId) {
- return StorageManager
- .INSTANCE
- .getPipelineElementStorage()
- .getEventStreamById(datasetId);
- }
-
-
- private String getDom() {
- count++;
- return "domId" + count;
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateGenerator.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateGenerator.java
deleted file mode 100644
index 03531d4..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateGenerator.java
+++ /dev/null
@@ -1,157 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-package org.apache.streampipes.manager.template;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.matching.DataSetGroundingSelector;
-import org.apache.streampipes.manager.matching.v2.ElementVerification;
-import org.apache.streampipes.manager.template.instances.*;
-import org.apache.streampipes.model.SpDataSet;
-import org.apache.streampipes.model.SpDataStream;
-import org.apache.streampipes.model.base.InvocableStreamPipesEntity;
-import org.apache.streampipes.model.graph.DataProcessorDescription;
-import org.apache.streampipes.model.graph.DataProcessorInvocation;
-import org.apache.streampipes.model.graph.DataSinkDescription;
-import org.apache.streampipes.model.graph.DataSinkInvocation;
-import org.apache.streampipes.model.message.DataSetModificationMessage;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.storage.api.IPipelineElementDescriptionStorage;
-import org.apache.streampipes.storage.management.StorageManager;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.net.URISyntaxException;
-import java.util.ArrayList;
-import java.util.List;
-
-public class PipelineTemplateGenerator {
-
-
- Logger logger = LoggerFactory.getLogger(PipelineTemplateGenerator.class);
-
- private List<PipelineTemplateDescription> availableDescriptions = new ArrayList<>();
-
- public List<PipelineTemplateDescription> getAllPipelineTemplates() {
-
- List<PipelineTemplate> allPipelineTemplates = new ArrayList<>();
-
-// allPipelineTemplates.add(new DashboardPipelineTemplate());
-// allPipelineTemplates.add(new ElasticsearchPipelineTemplate());
- allPipelineTemplates.add(new DataLakePipelineTemplate());
-// allPipelineTemplates.add(new FilterPipelineTemplate());
-// allPipelineTemplates.add(new FilterElasticSearchPipelineTemplate());
-// allPipelineTemplates.add(new DelmeExamplePipelineTemplate());
-
-
- for (PipelineTemplate pt : allPipelineTemplates) {
- try {
- availableDescriptions.add(pt.declareModel());
- } catch (URISyntaxException e) {
- e.printStackTrace();
- } catch (ElementNotFoundException e) {
- logger.warn("Adapter template can not be used because some elements are not installed", e);
- }
- }
-
- return availableDescriptions;
- }
-
- public List<PipelineTemplateDescription> getCompatibleTemplates(String streamId) {
- List<PipelineTemplateDescription> compatibleTemplates = new ArrayList<>();
- ElementVerification verifier = new ElementVerification();
- SpDataStream streamOffer = null;
-
- try {
- streamOffer = getStream(streamId);
-
- if (streamOffer instanceof SpDataSet) {
- streamOffer = new SpDataSet((SpDataSet) prepareStream((SpDataSet) streamOffer));
- } else {
- streamOffer = new SpDataStream(streamOffer);
- }
- if (streamOffer != null) {
- for(PipelineTemplateDescription pipelineTemplateDescription : getAllPipelineTemplates()) {
- // TODO make this work for 2+ input streams
- InvocableStreamPipesEntity entity = cloneInvocation(pipelineTemplateDescription.getBoundTo().get(0).getPipelineElementTemplate());
- if (verifier.verify(streamOffer, entity)) {
- compatibleTemplates.add(pipelineTemplateDescription);
- }
- }
- }
-
- } catch (ElementNotFoundException e) {
- e.printStackTrace();
- }
-
- return compatibleTemplates;
- }
-
- private InvocableStreamPipesEntity cloneInvocation(InvocableStreamPipesEntity pipelineElementTemplate) {
- if (pipelineElementTemplate instanceof DataProcessorInvocation) {
- return new DataProcessorInvocation((DataProcessorInvocation) pipelineElementTemplate);
- } else {
- return new DataSinkInvocation((DataSinkInvocation) pipelineElementTemplate);
- }
- }
-
- private SpDataStream prepareStream(SpDataSet stream) {
- DataSetModificationMessage message = new DataSetGroundingSelector(stream).selectGrounding();
- stream.setEventGrounding(message.getEventGrounding());
- stream.setDatasetInvocationId(message.getInvocationId());
- return stream;
- }
-
- protected SpDataStream getStream(String streamId) throws ElementNotFoundException {
- SpDataStream result = getStorage()
- .getEventStreamById(streamId);
-
- if (result == null) {
- throw new ElementNotFoundException("Data stream " + streamId + " is not installed!");
- }
-
- return result;
- }
-
- protected DataProcessorDescription getProcessor(String id) throws URISyntaxException, ElementNotFoundException {
- DataProcessorDescription result = getStorage()
- .getDataProcessorByAppId(id);
-
- if (result == null) {
- throw new ElementNotFoundException("Data processor " + id + " is not installed!");
- }
-
- return result;
- }
-
- protected DataSinkDescription getSink(String id) throws URISyntaxException, ElementNotFoundException {
- DataSinkDescription result = getStorage()
- .getDataSinkByAppId(id);
-
- if (result == null) {
- throw new ElementNotFoundException("Data stream " + id + " is not installed!");
- }
-
- return result;
- }
-
- protected IPipelineElementDescriptionStorage getStorage() {
- return StorageManager
- .INSTANCE
- .getPipelineElementStorage();
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationGenerator.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationGenerator.java
deleted file mode 100644
index 089d232..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationGenerator.java
+++ /dev/null
@@ -1,84 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-package org.apache.streampipes.manager.template;
-
-import org.apache.streampipes.model.SpDataStream;
-import org.apache.streampipes.model.pipeline.Pipeline;
-import org.apache.streampipes.model.staticproperty.StaticProperty;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.model.template.PipelineTemplateInvocation;
-
-import java.util.ArrayList;
-import java.util.List;
-import java.util.stream.Collectors;
-
-public class PipelineTemplateInvocationGenerator {
-
- private SpDataStream spDataStream;
- private PipelineTemplateDescription pipelineTemplateDescription;
-
- public PipelineTemplateInvocationGenerator(SpDataStream dataStream, PipelineTemplateDescription pipelineTemplateDescription) {
- this.spDataStream = dataStream;
- this.pipelineTemplateDescription = pipelineTemplateDescription;
- }
-
- public PipelineTemplateInvocation generateInvocation() {
-
- Pipeline pipeline = new PipelineGenerator(spDataStream.getElementId(), pipelineTemplateDescription, "test").makePipeline();
-
- PipelineTemplateInvocation pipelineTemplateInvocation = new PipelineTemplateInvocation();
- pipelineTemplateInvocation.setStaticProperties(collectStaticProperties(pipeline));
- pipelineTemplateInvocation.setDataSetId(spDataStream.getElementId());
- //pipelineTemplateInvocation.setPipelineTemplateDescription(pipelineTemplateDescription);
- pipelineTemplateInvocation.setPipelineTemplateId(pipelineTemplateDescription.getPipelineTemplateId());
- return pipelineTemplateInvocation;
- }
-
- private List<StaticProperty> collectStaticProperties(Pipeline pipeline) {
- List<StaticProperty> staticProperties = new ArrayList<>();
-
- pipeline.getSepas().forEach(pe -> {
- pe.getStaticProperties().forEach(sp -> sp.setInternalName(pe.getDOM() + sp.getInternalName()));
- staticProperties.addAll(pe.getStaticProperties());
- });
- pipeline.getActions().forEach(pe -> {
- pe.getStaticProperties().forEach(sp -> sp.setInternalName(pe.getDOM() + sp.getInternalName()));
- staticProperties.addAll(pe.getStaticProperties());
- });
-
- // Not sure what it does
-// staticProperties
-// .stream()
-// .filter(sp -> sp instanceof MappingPropertyUnary)
-// .forEach(mp -> ((MappingPropertyUnary) mp)
-// .setSelectedProperty(((MappingPropertyUnary) mp)
-// .getMapsFromOptions()
-// .get(0)));
-
- return staticProperties;
- }
-
- private List<StaticProperty> filter(List<StaticProperty> staticProperties) {
- return staticProperties
- .stream()
- // TODO fix (predefined is always true
- //.filter(sp -> !(sp instanceof MappingProperty))
- .filter(sp -> !(sp.isPredefined()))
- .collect(Collectors.toList());
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationHandler.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationHandler.java
deleted file mode 100644
index 3128552..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/PipelineTemplateInvocationHandler.java
+++ /dev/null
@@ -1,101 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-package org.apache.streampipes.manager.template;
-
-import org.apache.streampipes.manager.operations.Operations;
-import org.apache.streampipes.model.base.InvocableStreamPipesEntity;
-import org.apache.streampipes.model.pipeline.Pipeline;
-import org.apache.streampipes.model.pipeline.PipelineOperationStatus;
-import org.apache.streampipes.model.staticproperty.StaticProperty;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.model.template.PipelineTemplateInvocation;
-import org.apache.streampipes.storage.management.StorageDispatcher;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public class PipelineTemplateInvocationHandler {
-
- private PipelineTemplateInvocation pipelineTemplateInvocation;
- private PipelineTemplateDescription pipelineTemplateDescription;
- private String username;
-
- public PipelineTemplateInvocationHandler(String username, PipelineTemplateInvocation pipelineTemplateInvocation) {
- this.username = username;
- this.pipelineTemplateInvocation = pipelineTemplateInvocation;
- this.pipelineTemplateDescription = getTemplateById(pipelineTemplateInvocation.getPipelineTemplateId());
- }
-
- public PipelineTemplateInvocationHandler(String username, PipelineTemplateInvocation pipelineTemplateInvocation, PipelineTemplateDescription pipelineTemplateDescription) {
- this.username = username;
- this.pipelineTemplateInvocation = pipelineTemplateInvocation;
- this.pipelineTemplateDescription = pipelineTemplateDescription;
- }
-
-
- public PipelineOperationStatus handlePipelineInvocation() {
- Pipeline pipeline = new PipelineGenerator(pipelineTemplateInvocation.getDataSetId(), pipelineTemplateDescription, pipelineTemplateInvocation.getKviName()).makePipeline();
- pipeline.setCreatedByUser(username);
- pipeline.setCreatedAt(System.currentTimeMillis());
- replaceStaticProperties(pipeline);
- Operations.storePipeline(pipeline);
- Pipeline storedPipeline = StorageDispatcher.INSTANCE.getNoSqlStore().getPipelineStorageAPI().getPipeline(pipeline.getPipelineId());
- return Operations.startPipeline(storedPipeline);
- }
-
- private void replaceStaticProperties(Pipeline pipeline) {
- pipeline.getSepas().forEach(this::replace);
- pipeline.getActions().forEach(this::replace);
- }
-
- private void replace(InvocableStreamPipesEntity pe) {
- List<StaticProperty> newProperties = new ArrayList<>();
- pe.getStaticProperties().forEach(sp -> {
- if (existsInCustomizedElements(pe.getDOM(), sp)) {
- newProperties.add(getCustomizedElement(pe.getDOM(), pe.getDOM() + sp.getInternalName()));
- } else {
- newProperties.add(sp);
- }
- });
- pe.setStaticProperties(newProperties);
- }
-
-
-
- private StaticProperty getCustomizedElement(String dom, String internalName) {
- StaticProperty staticProperty = pipelineTemplateInvocation
- .getStaticProperties()
- .stream()
- .filter(sp -> sp.getInternalName().equals(internalName)).findFirst().get();
-
- staticProperty.setInternalName(staticProperty.getInternalName().replace(dom, ""));
- return staticProperty;
- }
-
- private boolean existsInCustomizedElements(String dom, StaticProperty staticProperty) {
- return pipelineTemplateInvocation
- .getStaticProperties()
- .stream()
- .anyMatch(sp -> sp.getInternalName().equals(dom +staticProperty.getInternalName()));
- }
-
-
- private PipelineTemplateDescription getTemplateById(String pipelineTemplateId) {
- return new PipelineTemplateGenerator().getAllPipelineTemplates().stream().filter(template -> template.getAppId().equals(pipelineTemplateId)).findFirst().get();
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DashboardPipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DashboardPipelineTemplate.java
deleted file mode 100644
index 0ca6346..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DashboardPipelineTemplate.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.sdk.builder.BoundPipelineElementBuilder;
-import org.apache.streampipes.sdk.builder.PipelineTemplateBuilder;
-
-import java.net.URISyntaxException;
-
-public class DashboardPipelineTemplate extends PipelineTemplateGenerator implements PipelineTemplate {
-
- private static String ID = "org.apache.streampipes.manager.template.instances.DashboardPipelineTemplate";
-
- @Override
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException {
- return new PipelineTemplateDescription(PipelineTemplateBuilder.create("http://streampipes.org/DashboardPipelineTemplate","Dashboard",
- "All events can be displayed on the live dashboard.")
- .setAppId(ID)
- .boundPipelineElementTemplate(
- BoundPipelineElementBuilder
- .create(getSink("org.apache.streampipes.sinks.internal.jvm.dashboard"))
- .build())
- .build());
-
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DataLakePipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DataLakePipelineTemplate.java
deleted file mode 100644
index 4912b02..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DataLakePipelineTemplate.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.sdk.builder.BoundPipelineElementBuilder;
-import org.apache.streampipes.sdk.builder.PipelineTemplateBuilder;
-
-import java.net.URISyntaxException;
-
-public class DataLakePipelineTemplate extends PipelineTemplateGenerator implements PipelineTemplate {
-
- private static String ID = "org.apache.streampipes.manager.template.instances.DataLakePipelineTemplate";
-
- @Override
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException {
- return new PipelineTemplateDescription(PipelineTemplateBuilder.create("http://streampipes.org/DataLakePipelineTemplate","DataLake",
- "")
- .setAppId(ID)
- .boundPipelineElementTemplate(
- BoundPipelineElementBuilder
- .create(getSink("org.apache.streampipes.sinks.internal.jvm.datalake"))
- .build())
- .build());
-
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DelmeExamplePipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DelmeExamplePipelineTemplate.java
deleted file mode 100644
index 09cb161..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/DelmeExamplePipelineTemplate.java
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.sdk.builder.BoundPipelineElementBuilder;
-import org.apache.streampipes.sdk.builder.PipelineTemplateBuilder;
-
-import java.net.URISyntaxException;
-
-public class DelmeExamplePipelineTemplate extends PipelineTemplateGenerator implements PipelineTemplate {
-
- private static String ID = "org.apache.streampipes.manager.template.instances.DelmeExamplePipelineTemplate";
-
- @Override
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException {
- return new PipelineTemplateDescription(PipelineTemplateBuilder.create("distance-kvi","Distance KVI",
- "Calculates the distance between two locations")
- .setAppId(ID)
- .boundPipelineElementTemplate(BoundPipelineElementBuilder
- .create(getProcessor("org.apache.streampipes.processors.geo.jvm.google-routing"))
-// .withPredefinedFreeTextValue("timeWindow", "30")
-// .withPredefinedSelection("operation", Collections.singletonList("Average"))
-// .withOverwrittenLabel("aggregate", "Select a field you'd like to use for the KVI calculation")
- .connectTo(BoundPipelineElementBuilder
-// .create(getSink("http://localhost:8090/sec/dashboard_sink"))
- .create(getSink("org.apache.streampipes.sinks.databases.jvm.couchdb"))
- .withPredefinedFreeTextValue("db_name", "kvi")
- .build())
- .build())
- .build());
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/ElasticsearchPipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/ElasticsearchPipelineTemplate.java
deleted file mode 100644
index fe41988..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/ElasticsearchPipelineTemplate.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.sdk.builder.BoundPipelineElementBuilder;
-import org.apache.streampipes.sdk.builder.PipelineTemplateBuilder;
-
-import java.net.URISyntaxException;
-
-public class ElasticsearchPipelineTemplate extends PipelineTemplateGenerator implements PipelineTemplate {
-
- private static String ID = "org.apache.streampipes.manager.template.instances.ElasticsearchPipelineTemplate";
-
- @Override
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException {
- return new PipelineTemplateDescription(PipelineTemplateBuilder.create("http://streampipes.org/ElasticsearchPipelineTemplate","Elasticsearch",
- "")
- .setAppId(ID)
- .boundPipelineElementTemplate(
- BoundPipelineElementBuilder
- .create(getSink("org.apache.streampipes.sinks.databases.flink.elasticsearch"))
- .build())
- .build());
-
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterElasticSearchPipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterElasticSearchPipelineTemplate.java
deleted file mode 100644
index 4a56edb..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterElasticSearchPipelineTemplate.java
+++ /dev/null
@@ -1,46 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.sdk.builder.BoundPipelineElementBuilder;
-import org.apache.streampipes.sdk.builder.PipelineTemplateBuilder;
-
-import java.net.URISyntaxException;
-
-public class FilterElasticSearchPipelineTemplate extends PipelineTemplateGenerator implements PipelineTemplate {
-
- private static String ID = "org.apache.streampipes.manager.template.instances.FilterElasticSearchPipelineTemplate";
-
- @Override
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException {
- return new PipelineTemplateDescription(PipelineTemplateBuilder.create("http://streampipes.org/FilterElasticKPI","Elastic Filter KPI",
- "Define a threshold value to filter")
- .setAppId(ID)
- .boundPipelineElementTemplate(BoundPipelineElementBuilder
- .create(getProcessor("org.apache.streampipes.processors.filters.jvm.numericalfilter"))
- .connectTo(BoundPipelineElementBuilder
- .create(getSink("org.apache.streampipes.sinks.databases.flink.elasticsearch"))
- .build())
- .build())
- .build());
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterPipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterPipelineTemplate.java
deleted file mode 100644
index 7b23eb7..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/FilterPipelineTemplate.java
+++ /dev/null
@@ -1,46 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.manager.template.PipelineTemplateGenerator;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.sdk.builder.BoundPipelineElementBuilder;
-import org.apache.streampipes.sdk.builder.PipelineTemplateBuilder;
-
-import java.net.URISyntaxException;
-
-public class FilterPipelineTemplate extends PipelineTemplateGenerator implements PipelineTemplate {
-
- private static String ID = "org.apache.streampipes.manager.template.instances.FilterPipelineTemplate";
-
- @Override
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException {
- return new PipelineTemplateDescription(PipelineTemplateBuilder.create("http://streampipes.org/FilterKPI","Filter KPI",
- "Define a threshold value to filter")
- .setAppId(ID)
- .boundPipelineElementTemplate(BoundPipelineElementBuilder
- .create(getProcessor("org.apache.streampipes.processors.filters.jvm.numericalfilter"))
- .connectTo(BoundPipelineElementBuilder
- .create(getSink("org.apache.streampipes.sinks.internal.jvm.dashboard"))
- .build())
- .build())
- .build());
- }
-}
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/PipelineTemplate.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/PipelineTemplate.java
deleted file mode 100644
index 14001d2..0000000
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/template/instances/PipelineTemplate.java
+++ /dev/null
@@ -1,30 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.streampipes.manager.template.instances;
-
-import org.apache.streampipes.commons.exceptions.ElementNotFoundException;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-
-import java.net.URISyntaxException;
-
-public interface PipelineTemplate {
-
- public PipelineTemplateDescription declareModel() throws URISyntaxException, ElementNotFoundException;
-
-}
diff --git a/streampipes-rest/src/main/java/org/apache/streampipes/rest/impl/PipelineTemplate.java b/streampipes-rest/src/main/java/org/apache/streampipes/rest/impl/PipelineTemplate.java
deleted file mode 100644
index 9508d7d..0000000
--- a/streampipes-rest/src/main/java/org/apache/streampipes/rest/impl/PipelineTemplate.java
+++ /dev/null
@@ -1,131 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-package org.apache.streampipes.rest.impl;
-
-import org.apache.streampipes.manager.operations.Operations;
-import org.apache.streampipes.model.SpDataSet;
-import org.apache.streampipes.model.SpDataStream;
-import org.apache.streampipes.model.SpDataStreamContainer;
-import org.apache.streampipes.model.pipeline.PipelineOperationStatus;
-import org.apache.streampipes.model.template.PipelineTemplateDescription;
-import org.apache.streampipes.model.template.PipelineTemplateDescriptionContainer;
-import org.apache.streampipes.model.template.PipelineTemplateInvocation;
-import org.apache.streampipes.rest.core.base.impl.AbstractAuthGuardedRestResource;
-import org.apache.streampipes.rest.shared.util.SpMediaType;
-
-import javax.ws.rs.*;
-import javax.ws.rs.core.MediaType;
-import javax.ws.rs.core.Response;
-import java.util.ArrayList;
-import java.util.List;
-
-@Path("/v2/pipeline-templates")
-public class PipelineTemplate extends AbstractAuthGuardedRestResource {
-
- @GET
- @Path("/streams")
- @Produces(MediaType.APPLICATION_JSON)
- public Response getAvailableDataStreams() {
- List<SpDataStream> sources = getPipelineElementRdfStorage().getAllDataStreams();
- List<SpDataStream> datasets = new ArrayList<>();
-
- sources.stream()
- .filter(stream -> !(stream instanceof SpDataSet))
- .map(SpDataStream::new)
- .forEach(datasets::add);
-
- return ok((new SpDataStreamContainer(datasets)));
- }
-
- @GET
- @Path("/sets")
- @Produces(MediaType.APPLICATION_JSON)
- public Response getAvailableDataSets() {
-
- List<SpDataStream> sources = getPipelineElementRdfStorage().getAllDataStreams();
- List<SpDataStream> datasets = new ArrayList<>();
-
- sources
- .stream()
- .filter(stream -> stream instanceof SpDataSet)
- .map(stream -> new SpDataSet((SpDataSet) stream))
- .forEach(set -> datasets.add((SpDataSet) set));
-
- return ok(new SpDataStreamContainer(datasets));
- }
-
- @GET
- @Produces(SpMediaType.JSONLD)
- public Response getPipelineTemplates(@QueryParam("streamId") String streamId) {
- if (streamId != null) {
- return ok(new PipelineTemplateDescriptionContainer(Operations.getCompatiblePipelineTemplates(streamId)));
- } else {
- PipelineTemplateDescriptionContainer container = new PipelineTemplateDescriptionContainer(Operations.getAllPipelineTemplates());
- return ok(container);
- }
- }
-
- @GET
- @Path("/invocation")
- @Produces(MediaType.APPLICATION_JSON)
- public Response getPipelineTemplateInvocation(@QueryParam("streamId") String streamId,
- @QueryParam("templateId") String pipelineTemplateId) {
- if (pipelineTemplateId != null) {
- SpDataStream dataStream = getDataStream(streamId);
- PipelineTemplateDescription pipelineTemplateDescription = getPipelineTemplateDescription(pipelineTemplateId);
- PipelineTemplateInvocation invocation = Operations.getPipelineInvocationTemplate(dataStream, pipelineTemplateDescription);
- PipelineTemplateInvocation clonedInvocation = new PipelineTemplateInvocation(invocation);
- return ok(new PipelineTemplateInvocation(clonedInvocation));
- } else {
- return fail();
- }
- }
-
- @POST
- @Produces(MediaType.APPLICATION_JSON)
- public Response generatePipeline(PipelineTemplateInvocation pipelineTemplateInvocation) {
-
- PipelineOperationStatus status = Operations
- .handlePipelineTemplateInvocation(getAuthenticatedUsername(), pipelineTemplateInvocation);
-
- return ok(status);
-
- }
-
-
- private PipelineTemplateDescription getPipelineTemplateDescription(String pipelineTemplateId) {
- return Operations
- .getAllPipelineTemplates()
- .stream()
- .filter(pt -> pt.getAppId().equals(pipelineTemplateId))
- .findFirst()
- .get();
- }
-
- private List<SpDataStream> getAllDataStreams() {
- return getPipelineElementRdfStorage().getAllDataStreams();
- }
-
- private SpDataStream getDataStream(String streamId) {
- return getAllDataStreams()
- .stream()
- .filter(sp -> sp.getElementId().equals(streamId))
- .findFirst()
- .get();
- }
-}
[incubator-streampipes] 03/05: [hotfix] Do not use random topics in pipeline generation
Posted by ri...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch experimental-module-federation-494
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes.git
commit 0c46ccc66f5b186bc435d0dabb89b0ebdc957477
Author: Dominik Riemer <do...@gmail.com>
AuthorDate: Fri Feb 4 22:42:24 2022 +0100
[hotfix] Do not use random topics in pipeline generation
---
.../java/org/apache/streampipes/manager/matching/ProtocolSelector.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/matching/ProtocolSelector.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/matching/ProtocolSelector.java
index ec3a891..4527e90 100644
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/matching/ProtocolSelector.java
+++ b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/matching/ProtocolSelector.java
@@ -39,7 +39,7 @@ public class ProtocolSelector extends GroundingSelector {
public ProtocolSelector(NamedStreamPipesEntity source, Set<InvocableStreamPipesEntity> targets) {
super(source, targets);
- this.outputTopic = TopicGenerator.generateRandomTopic();
+ this.outputTopic = TopicGenerator.generateInternalPipelineElementTopic(source.getDOM());
this.prioritizedProtocols =
BackendConfig.INSTANCE.getMessagingSettings().getPrioritizedProtocols();
}
[incubator-streampipes] 01/05: [hotfix] Extract correct instanceId in health check
Posted by ri...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch experimental-module-federation-494
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes.git
commit 633b2aa2ee325c9b1a57fbe83fea2fb59450fd5d
Author: Dominik Riemer <do...@gmail.com>
AuthorDate: Fri Feb 4 22:16:24 2022 +0100
[hotfix] Extract correct instanceId in health check
---
.../org/apache/streampipes/manager/health/PipelineHealthCheck.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/health/PipelineHealthCheck.java b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/health/PipelineHealthCheck.java
index 217b73c..ecdd303 100644
--- a/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/health/PipelineHealthCheck.java
+++ b/streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/health/PipelineHealthCheck.java
@@ -18,6 +18,7 @@
package org.apache.streampipes.manager.health;
+import org.apache.streampipes.commons.constants.InstanceIdExtractor;
import org.apache.streampipes.commons.exceptions.NoServiceEndpointsAvailableException;
import org.apache.streampipes.manager.execution.endpoint.ExtensionsServiceEndpointGenerator;
import org.apache.streampipes.manager.execution.endpoint.ExtensionsServiceEndpointUtils;
@@ -156,7 +157,7 @@ public class PipelineHealthCheck implements Runnable {
}
private String extractInstanceId(InvocableStreamPipesEntity graph) {
- return graph.getElementId().replace(graph.getBelongsTo() + ":", "");
+ return InstanceIdExtractor.extractId(graph.getElementId());
}
[incubator-streampipes] 04/05: [STREAMPIPES-494] Do not show pipeline element status in preview mode
Posted by ri...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch experimental-module-federation-494
in repository https://gitbox.apache.org/repos/asf/incubator-streampipes.git
commit 5e07c76b54121ae82a0b0eac82bcdaca9ac5e034
Author: Dominik Riemer <do...@gmail.com>
AuthorDate: Fri Feb 4 22:49:33 2022 +0100
[STREAMPIPES-494] Do not show pipeline element status in preview mode
---
ui/src/app/editor/components/pipeline/pipeline.component.html | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ui/src/app/editor/components/pipeline/pipeline.component.html b/ui/src/app/editor/components/pipeline/pipeline.component.html
index 39911fb..768997a 100644
--- a/ui/src/app/editor/components/pipeline/pipeline.component.html
+++ b/ui/src/app/editor/components/pipeline/pipeline.component.html
@@ -31,7 +31,7 @@
</div>
<div class="pipeline-element-loading-container sp-fade-opacity"
*ngIf="pipelineElement.settings.loadingStatus"></div>
- <div class="pipeline-element-configuration-status {{pipelineElement.type === 'stream' ? 'pi-stream' : 'pi-processor'}}">
+ <div class="pipeline-element-configuration-status {{pipelineElement.type === 'stream' ? 'pi-stream' : 'pi-processor'}}" *ngIf="!preview">
<i class="material-icons pipeline-element-configuration-invalid-icon" *ngIf="pipelineElement.settings.completed === 3">
warning
</i>