Readiness check doesn't wait for Green state

[mtruneck]

Chart version: 7.8.0

Short description: Readiness probe passes even when the cluster is in RED state because of bash syntax error

Long description:
In the readiness probe in chart/templates (https://github.com/elastic/helm-charts/blob/master/elasticsearch/templates/statefulset.yaml#L262), the check for readiness is:

if http "/_cluster/health?{{ .Values.clusterHealthCheckParams }}" "--fail" ; then

with default value of clusterHealthCheckParams: "wait_for_status=green&timeout=1s", it will render to:

if curl -XGET -s -k --fail ${BASIC_AUTH} http://127.0.0.1:9200/_cluster/health?wait_for_status=green&timeout=1s ; then
Where the url is not enclosed in single quotes and since there is an ampersand in the command, it doesn't work.

Namely, Bash will run curl -XGET -s -k --fail ${BASIC_AUTH} http://127.0.0.1:9200/_cluster/health?wait_for_status=green in the background (because of the ampersand), then will run timeout=1s which is a valid bash operation and obviously, it will return 0 and the check passes even though the cluster is in RED state.

Suggested solution: The solution is easy, add single quotes to the url.

possibly related issues:
#215
#374

[fatmcgav]

@mtruneck Thank-you for opening an issue.

Have you seen logs indicating that there is a quoting issue?

As the readinessProbe handling was reworked in #638 to solve a very similar issue, and now the curl command contains a double-quoted request URL:

helm-charts/elasticsearch/templates/statefulset.yaml

Line 239 in fa72b48

curl --output /dev/null -k "$@" "{{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}${path}"

Are you able to provide the output from the following to help us investigate further:

• helm get <deployment>
• kubectl describe <elasticsearch pod>
• kubectl get events

Cheers

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue has been automatically closed because it has not had recent activity since being marked as stale.