Getting the status of Hosa

A blog post by Heiko W. Rupp

agent | openshift

Getting the status of the Hawkular OpenShift Agent

In a previous blog post I talked about the Hawkular OpenShift Agent and how to use it to monitor Microservices.

While setting up the agent and the config maps is not complicated, it is always possible to make a typo. Hosa luckily has a status endpoint that exposes what it is currently doing.

Status in browser
Figure 1. Status display in browser

The endpoint is secured by basic authentication. We will see below how to set this up.

Exposing the Hosa status endpoint

It is possible to expose the agent’s status endpoint, which is deployed as a pod only, as service and with a route to be able to visit the /status url of it.

When you have checked out the agent source code and deployed the agent via make openshift-deploy you can skip the following, as the make command already did the setup.

We will create a Yaml definition for the service and route and also a secret.

Definition for service and route, hosa-service-route.yml
apiVersion: v1
kind: Service
  name: hawkular-openshift-agent
    metrics-infra: agent
    - protocol: TCP
      port: 8080 (1)
    name: hawkular-openshift-agent
apiVersion: v1
kind: Route
  name: hosa
  namespace: openshift-infra
    metrics-infra: agent
  path: /status (2)
    kind: Service
    name: hawkular-openshift-agent
    weight: 100
apiVersion: v1
kind: Secret
  name: hawkular-openshift-agent-status
      metrics-infra: agent
    username: ZGV2  (3)
    password: Y2hhbmdlbWU= (4)
  1. Expose port 8080

  2. Put the status endpoint in the url, so that it opens when clicking in the OpenShift console

  3. Username dev as base64

  4. Password changeme as base64

You can then deploy this into OpenShift via

$ oc project openshift-infra
$ oc create -f hosa-service-route.yml

OpenShift will then expose a route in the format http://hosa-openshift-infra.<nodeip>

When you navigate to it then you will be prompted for basic authentication. In above example use dev/changeme for the credentials.

Using a different password above

When you want to use a different password in the above, you have to supply it in Base64-encoding

Base64 encoding of the password
$ echo -n "test4hawkular" | base64
dGVzdDRoYXdrdWxhcgo== (1)
  1. Copy this onto the password line of hosa-service-route.yml

Use a different password later on

When you have already installed the secret and want to change the password, you need to delete and re-create the secret with the new values:

Install the new password
$ oc project openshift-infra
$ oc delete secret hawkular-openshift-agent-status
$ oc secret new-basicauth hawkular-openshift-agent-status \ (1)
    --username=dev \
    --password=changeme2 (2)
  1. Name of the secret

  2. New password. No Base64 encoding is needed; OpenShift will take care.

Unfortunately this is not enough as OpenShift does not trigger reloads of pods upon changes of secrets (unlike e.g. for ConfigMaps), so we have to explicitly terminate the running agent to have a new instance pick up the new secret:

Make the agent pick up the new password
$ oc project openshift-infra
$ oc get pods | grep agent
hawkular-openshift-agent-upubo   1/1       Running     0          2m
$ oc delete pod hawkular-openshift-agent-upubo
pod "hawkular-openshift-agent-upubo" deleted

Lets have a look at the output

Now let’s have a quick look at the output of the status endpoint. I will use the curl command for this. The output is similar to what is shown in Figure1 above.

Get the status
$ curl -u dev:changeme2
name: Hawkular OpenShift Agent
version: 1.0.1.Final-SNAPSHOT (1)
commit_hash: 4655b5cb8363b046e80c052c5fe08723770088ea
pods: (2)
  - openshift-infra/hawkular-openshift-agent-omf3r|
  - myproject/obs-demo-3-x9hkg|
endpoints: (3)
  openshift-infra/hawkular-openshift-agent-omf3r| OK.
    Last collection at [Wed, 25 Jan 2017 13:56:58 +0000] gathered [22] metrics in
myproject/obs-demo-3-x9hkg| 'Failed to collect
    metrics from [myproject/obs-demo-3-x9hkg|] at
    [Thu, 25 Jan 2017 13:57:06 +0000]. err=Failed to collect metrics from Jolokia
    endpoint []. err=Post
    x509: cannot validate certificate for because it doesn''t contain any
    IP SANs' (4)
log: (5)
- 'Wed, 25 Jan 2017 11:53:29 +0000: STOP collection: openshift-infra/hawkular-openshift-agent-omf3r|'
- 'Wed, 25 Jan 2017 11:53:29 +0000: START collection: openshift-infra/hawkular-openshift-agent-omf3r|'
  1. Version of the agent (and the commit hash it was built from)

  2. List of pods it is monitoring with the endppoint it talks to

  3. List of endpoints it monitors + result of last collection

  4. This one shows an error because the endpoint is on https, but the certificate check fails and we have not disabled this check

  5. Latest log messages of the agent.

When you have the agent source checked out from git, you can just run make openshift-status instead of the curl command shown above.

Published by Heiko W. Rupp on 27 January 2017


© 2016 | Hawkular is released under Apache License v2.0