[GitHub] [incubator-heron] the-scott-hand opened a new issue #3288: heron helm chart issues and features

[GitBox <gi...@apache.org>]

the-scott-hand opened a new issue #3288: heron helm chart issues and features
URL: https://github.com/apache/incubator-heron/issues/3288
 
 
   Moving the discussion from [this pull](https://github.com/apache/incubator-heron/pull/3219) to here
   
   ## Background
   I have a bare metal kubernetes install which uses rook-ceph for storage and prometheus for metrics. I use helm for many pieces of my cluster. I was previously running heron 0.17.8 without the helm chart, but after the helm chart got updated to use apache/bookkeeper:4.7.3 I wanted to switch to using helm for my heron install.
   
   ## Build
   Since there has not been an official helm chart released since upgrading to apache/bookkeeper:4.7.3(that I know of), I grabbed [the helm files from master](https://github.com/apache/incubator-heron/tree/master/deploy/kubernetes/helm), replaced "VERSION" with 0.17.8 in the Chart and values yaml template files, and removed ".template" from the end. I then helm installed from that directory. 
   
   ## Problems Encountered (and at least temporarily solved)
   ### [bookie.yaml](https://github.com/apache/incubator-heron/blob/master/deploy/kubernetes/helm/templates/bookie.yaml)
   - the new image, apache/bookkeeper, does not appear to have netcat(nc) installed, which causes [the initContainer](https://github.com/apache/incubator-heron/blob/master/deploy/kubernetes/helm/templates/bookie.yaml#L101-L117) to break. Commenting it out was a functional temporary fix.
   - If using "baremetal" or "aws" there is inconsistent volume naming in [these lines of bookie.yaml](https://github.com/apache/incubator-heron/blob/master/deploy/kubernetes/helm/templates/bookie.yaml#L138-L179). Bookie is trying to mount a volume called data-disk, but if you choose these platforms that volume is called ledgers-disk. This results in an error. I opted to just use the minikube "platform" for my bare-metal install instead, which suited my storage needs better.
   ### [tools.yaml](https://github.com/apache/incubator-heron/blob/master/deploy/kubernetes/helm/templates/tools.yaml)
   - Got an error that helm could not find the requested resource. The [variables set at the top of the file](https://github.com/apache/incubator-heron/blob/master/deploy/kubernetes/helm/templates/tools.yaml#L18-L20) use "{{- " and " -}}" to remove white-space, but it seems like it may have removed too much. Ended up looking somthing like this "#  under the License.apiVersion: v1". Removing the first "-" from line 18 fixed the issue.
   - Had various strange errors that were ultimately caused by having a space in between the service name and port instead of a colon. Delinquent lines included [L73](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/tools.yaml#L73),  [L134](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/tools.yaml#L134), and [L138](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/tools.yaml#L138).
   - Had issues with some package names that started with org.apache. Changing them to com.twitter fixed my issues. Modfied the following lines: [L136](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/tools.yaml#L136) and [L148-L149](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/tools.yaml#L148-L149) (although its possible L151-L155 would have similar issues if I tried a different packing option). 
   ## Features Needed 
   - Option to specify kubectl image/version from values.yaml. Currently it is hardcoded to latest [at this line](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/tools.yaml#L161)
   - Option to change storage class from default in [bookie.yaml](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/bookie.yaml#L156-L178) and [zookeeper.yaml](https://github.com/apache/incubator-heron/blob/a5e6ccc16876518a4ed8ea6e7c4cc20aad3dca58/deploy/kubernetes/helm/templates/zookeeper.yaml#L144-L151).
   - Option to use volumeClaimTemplates in bookie as part of baremetal install rather than specifying minikube when I'm not using minikube.
   - Option to enable Prometheus metrics for bookkeeper
   
   #### Thanks again for your help. Hopefully I'll be able to put in a pull request for some of the changes I had to make if that would be helpful

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services