You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by sa...@apache.org on 2022/05/03 14:58:03 UTC
[incubator-heron] branch saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev updated (3875c319dec -> 2a5c30ec7bb)
This is an automated email from the ASF dual-hosted git repository.
saadurrahman pushed a change to branch saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev
in repository https://gitbox.apache.org/repos/asf/incubator-heron.git
omit 3875c319dec [K8s] using Volume Factory to generate PVC.
new 2c0512e4ffb [K8s] using Volume Factory to generate PVC.
new 1aa3289017b [K8s] test hardening.
new b4d7cd3a70e [K8s] generating volume with PVC in Volume Factory.
new 2a5c30ec7bb [K8s] using Volume Factory to create volumes with PVC.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3875c319dec)
\
N -- N -- N refs/heads/saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev (2a5c30ec7bb)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/heron/scheduler/kubernetes/V1Controller.java | 11 +----------
.../org/apache/heron/scheduler/kubernetes/Volumes.java | 15 +++++++++++++++
.../heron/scheduler/kubernetes/V1ControllerTest.java | 1 +
.../apache/heron/scheduler/kubernetes/VolumesTests.java | 16 ++++++++++++++++
4 files changed, 33 insertions(+), 10 deletions(-)
[incubator-heron] 03/04: [K8s] generating volume with PVC in Volume Factory.
Posted by sa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
saadurrahman pushed a commit to branch saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev
in repository https://gitbox.apache.org/repos/asf/incubator-heron.git
commit b4d7cd3a70e56b3791ea488e56cc4b5a3bf1821e
Author: Saad Ur Rahman <sa...@apache.org>
AuthorDate: Tue May 3 10:57:11 2022 -0400
[K8s] generating volume with PVC in Volume Factory.
---
.../org/apache/heron/scheduler/kubernetes/Volumes.java | 15 +++++++++++++++
.../apache/heron/scheduler/kubernetes/VolumesTests.java | 16 ++++++++++++++++
2 files changed, 31 insertions(+)
diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/Volumes.java b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/Volumes.java
index d9e641f78cc..4dcba8fdccb 100644
--- a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/Volumes.java
+++ b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/Volumes.java
@@ -142,6 +142,21 @@ final class Volumes {
return claim;
}
+ /**
+ * Generates a <code>Volume</code> with a <code>Persistent Volume Claim</code> inserted.
+ * @param claimName Name of the <code>Persistent Volume Claim</code>.
+ * @param volumeName Name of the <code>Volume</code> to place the <code>Persistent Volume Claim</code> in.
+ * @return Fully configured <code>Volume</code> with <code>Persistent Volume Claim</code> in it.
+ */
+ V1Volume createPersistentVolumeClaim(String claimName, String volumeName) {
+ return new V1VolumeBuilder()
+ .withName(volumeName)
+ .withNewPersistentVolumeClaim()
+ .withClaimName(claimName)
+ .endPersistentVolumeClaim()
+ .build();
+ }
+
interface IVolumeFactory {
V1Volume create(String volumeName, Map<KubernetesConstants.VolumeConfigKeys, String> configs);
}
diff --git a/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/VolumesTests.java b/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/VolumesTests.java
index 1cd7889741b..c457012e342 100644
--- a/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/VolumesTests.java
+++ b/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/VolumesTests.java
@@ -340,4 +340,20 @@ public class VolumesTests {
.createPersistentVolumeClaim(volumeNameStatic, labels, volStaticConfig);
Assert.assertEquals("Volume static PVC", claimStatic, actualPVCStatic);
}
+
+ @Test
+ public void testVolumeWithPersistentVolumeClaim() {
+ final String claimName = "claim-name";
+ final String volumeName = "volume-name";
+ final V1Volume expected = new V1VolumeBuilder()
+ .withName(volumeName)
+ .withNewPersistentVolumeClaim()
+ .withClaimName(claimName)
+ .endPersistentVolumeClaim()
+ .build();
+
+ final V1Volume actual = Volumes.get().createPersistentVolumeClaim(claimName, volumeName);
+
+ Assert.assertEquals("Volume with Persistent Volume Claim configured", expected, actual);
+ }
}
[incubator-heron] 04/04: [K8s] using Volume Factory to create volumes with PVC.
Posted by sa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
saadurrahman pushed a commit to branch saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev
in repository https://gitbox.apache.org/repos/asf/incubator-heron.git
commit 2a5c30ec7bbbbf9d89e54e99cdba7e6f4da2324c
Author: Saad Ur Rahman <sa...@apache.org>
AuthorDate: Tue May 3 10:57:45 2022 -0400
[K8s] using Volume Factory to create volumes with PVC.
---
.../org/apache/heron/scheduler/kubernetes/V1Controller.java | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java
index 9c93e30cf84..1cd85e5ad6f 100644
--- a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java
+++ b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java
@@ -77,7 +77,6 @@ import io.kubernetes.client.openapi.models.V1StatefulSetSpec;
import io.kubernetes.client.openapi.models.V1Status;
import io.kubernetes.client.openapi.models.V1Toleration;
import io.kubernetes.client.openapi.models.V1Volume;
-import io.kubernetes.client.openapi.models.V1VolumeBuilder;
import io.kubernetes.client.openapi.models.V1VolumeMount;
import io.kubernetes.client.util.PatchUtils;
import io.kubernetes.client.util.Yaml;
@@ -1145,14 +1144,7 @@ public class V1Controller extends KubernetesController {
final String claimName = configs.getValue()
.get(KubernetesConstants.VolumeConfigKeys.claimName);
if (claimName != null && !KubernetesConstants.LABEL_ON_DEMAND.equalsIgnoreCase(claimName)) {
- volumes.add(
- new V1VolumeBuilder()
- .withName(volumeName)
- .withNewPersistentVolumeClaim()
- .withClaimName(claimName)
- .endPersistentVolumeClaim()
- .build()
- );
+ volumes.add(Volumes.get().createPersistentVolumeClaim(claimName, volumeName));
}
volumeMounts.add(Volumes.get().createMount(volumeName, configs.getValue()));
}
[incubator-heron] 01/04: [K8s] using Volume Factory to generate PVC.
Posted by sa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
saadurrahman pushed a commit to branch saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev
in repository https://gitbox.apache.org/repos/asf/incubator-heron.git
commit 2c0512e4ffb49e992f835bfefb82cf2b7916a840
Author: Saad Ur Rahman <sa...@apache.org>
AuthorDate: Tue May 3 10:33:17 2022 -0400
[K8s] using Volume Factory to generate PVC.
---
.../heron/scheduler/kubernetes/V1Controller.java | 39 ++--------------------
1 file changed, 3 insertions(+), 36 deletions(-)
diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java
index 7812ed6aae9..9c93e30cf84 100644
--- a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java
+++ b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java
@@ -64,7 +64,6 @@ import io.kubernetes.client.openapi.models.V1LabelSelector;
import io.kubernetes.client.openapi.models.V1ObjectFieldSelector;
import io.kubernetes.client.openapi.models.V1ObjectMeta;
import io.kubernetes.client.openapi.models.V1PersistentVolumeClaim;
-import io.kubernetes.client.openapi.models.V1PersistentVolumeClaimBuilder;
import io.kubernetes.client.openapi.models.V1PodSpec;
import io.kubernetes.client.openapi.models.V1PodTemplate;
import io.kubernetes.client.openapi.models.V1PodTemplateSpec;
@@ -1120,41 +1119,9 @@ public class V1Controller extends KubernetesController {
continue;
}
- V1PersistentVolumeClaim claim = new V1PersistentVolumeClaimBuilder()
- .withNewMetadata()
- .withName(pvc.getKey())
- .withLabels(getPersistentVolumeClaimLabels(getTopologyName()))
- .endMetadata()
- .withNewSpec()
- .withStorageClassName("")
- .endSpec()
- .build();
-
- // Populate PVC options.
- for (Map.Entry<KubernetesConstants.VolumeConfigKeys, String> option
- : pvc.getValue().entrySet()) {
- String optionValue = option.getValue();
- switch(option.getKey()) {
- case storageClassName:
- claim.getSpec().setStorageClassName(optionValue);
- break;
- case sizeLimit:
- claim.getSpec().setResources(
- new V1ResourceRequirements()
- .putRequestsItem("storage", new Quantity(optionValue)));
- break;
- case accessModes:
- claim.getSpec().setAccessModes(Arrays.asList(optionValue.split(",")));
- break;
- case volumeMode:
- claim.getSpec().setVolumeMode(optionValue);
- break;
- // Valid ignored options not used in a PVC.
- default:
- break;
- }
- }
- listOfPVCs.add(claim);
+ listOfPVCs.add(Volumes.get()
+ .createPersistentVolumeClaim(pvc.getKey(),
+ getPersistentVolumeClaimLabels(getTopologyName()), pvc.getValue()));
}
return listOfPVCs;
}
[incubator-heron] 02/04: [K8s] test hardening.
Posted by sa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
saadurrahman pushed a commit to branch saadurrahman/3821-Remove-Deprecated-Volumes-K8s-dev
in repository https://gitbox.apache.org/repos/asf/incubator-heron.git
commit 1aa3289017b598fe1bc10802b557f325f66a9919
Author: Saad Ur Rahman <sa...@apache.org>
AuthorDate: Tue May 3 10:34:23 2022 -0400
[K8s] test hardening.
Test in <testCreatePersistentVolumeClaims> hardened for size.
---
.../java/org/apache/heron/scheduler/kubernetes/V1ControllerTest.java | 1 +
1 file changed, 1 insertion(+)
diff --git a/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/V1ControllerTest.java b/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/V1ControllerTest.java
index 3e667d81fd6..a039ca980e5 100644
--- a/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/V1ControllerTest.java
+++ b/heron/schedulers/tests/java/org/apache/heron/scheduler/kubernetes/V1ControllerTest.java
@@ -871,6 +871,7 @@ public class V1ControllerTest {
final List<V1PersistentVolumeClaim> actualClaims =
v1ControllerWithPodTemplate.createPersistentVolumeClaims(mapPVCOpts);
+ Assert.assertEquals("Generated claim sizes match", expectedClaims.size(), actualClaims.size());
Assert.assertTrue(expectedClaims.containsAll(actualClaims));
}