Grafana Quickstart Guide

Getting started with Hawkular and Grafana

client | grafana

This project is the Hawkular Datasource plugin for Grafana 3. It works with:

  • Metrics standalone servers as well

  • Hawkular servers, starting from version Alpha13

Installing

If you don’t have Grafana yet, download and install it.

From source

Download the source code and copy the content of dist to hawkular inside Grafana’s plugin directory:

# This is the default for Linux Grafana installs. Change it to match yours, if needed.
GRAFANA_PLUGINS=/var/lib/grafana/plugins
wget https://github.com/hawkular/hawkular-grafana-datasource/archive/release.zip -O hawkular-grafana-datasource-release.zip
unzip hawkular-grafana-datasource-release.zip
mkdir ${GRAFANA_PLUGINS}/hawkular-datasource
cp -R hawkular-grafana-datasource-release/dist/* ${GRAFANA_PLUGINS}/hawkular-datasource

From the Grafana plugin directory

The Hawkular datasource is available on the Grafana plugin directory. You can install it using grafana-cli:

grafana-cli plugins install hawkular-datasource

Configuration

The datasource URL must point to the Hawkular Metrics service, e.g. http://myhost:8080/hawkular/metrics

Access: proxy mode is recommended, since it doesn’t leak any authentication information to the browser.

Both proxy and direct modes should work in most configurations. Some earlier versions of Hawkular Metrics had a bug with CORS headers, that prevented the use of direct mode here. If you want to use direct mode (that is, direct calls from client browser to Hawkular REST API), and if you have setup CORS restrictions in Hawkular, make sure to allow the Grafana server origin in Metrics' configuration. If you’re unsure, just use proxy mode and you should be fine.

Authentication must be set when working with a Hawkular server. Check the 'Basic Auth' box and fill the user and password fields.

Select the tenant. For Hawkular-Services, use hawkular. For Openshift, the tenant is the project name. Custom tenants can also be used if you manage writing metrics to Hawkular’s REST API.

Openshift-Metrics users must provide an authentication token.

Usage

Queries

When adding a Graph panel, the Metrics tab allows you fetch Gauges, Counters and Availability metrics in Hawkular. You can either search by metric id or by tag, assuming your version of Hawkular Metrics is at least 0.20.0. Prior versions only allow searching by id.

To know your version of Hawkular Metrics, check the "status" endpoint. E.g. http://myhost:8080/hawkular/metrics/status

When searching by id, you must provide the exact metric id (or use variables, as discussed later).

search by name

When searching by tag, you must provide the tag key, followed by its value or any pattern recognized by Hawkular Metrics. Check how tagging works in Hawkular.

  • To add more tag filters, hit the "+" button and repeat the process.

  • To remove a tag, click on its key and select "Remove tag".

search by tag
Note that querying by tag may return multiple series for a single query, as illustrated above.

When using a Singlestat panel, some additional options come in. The Hawkular Datasource plugin can perform aggregations on multiple metrics, which usually the Singlestat panel doesn’t. It’s actually a two-steps aggregation: first, multiple series are reduced into a single one (that is either the sum of its parts, or the average). Then, the resulting series is aggregated over time through another folding: min, max, average etc.

single stats aggreg
Note that because the aggregation takes place in Hawkular, the Singlestat panel has nothing to aggregate. Thus in panel options, setting whatever in the value field on the Big value won’t have any effect. However if you don’t want to use the Hawkular aggregation, just set Multiple series aggregation to None.

Templating variables

Grafana allows you to create dashboard templates through the definition of variables. This is documented on Grafana’s site. With the Hawkular Datasource Plugin, the variables of type 'Query' are mapped to the @get (url)/metrics Hawkular Metrics endpoint and can be used to retrieve tenant’s metric names. Use the Query Options text field to pass query parameters, as illustrated below:

query for metrics
For instance, if you have metrics tagged "type:memory" and others tagged "type:cpu", you can write "tags=type:memory" to get only the "memory" ones, or "tags=type:cpu|memory" to get them both.

There is an exception to that rule: if the query string is prefixed with 'tags/', the variable will contain the matching tag names rather than the metric names. In this case, the Hawkular Metrics endpoint @get (url)/metrics/tags/{tags} will be used.

query for tags
For instance, type "tags/type:*" to get all of the available tag values for "type".

Once you have set some variables, you can use them in graph queries: either for row or graph duplication, or to display multiple series in a single graph from a single query. This is especially useful when metric names contain some dynamic parts and thus cannot be known in advance.

Annotations

Annotations are available through the use of 'string' metrics in Hawkular. It’s a Grafana feature that allows to display custom events in timed charts. Here’s how to proceed:

  1. Setup an annotation query in Grafana. In 'Query', put the name of a 'string' metric you want to use to gather these annotations.

annotation setup
  1. Post any event (ie. string + timestamp) to some string metric.

Example, JSON posted on the Hawkular’s REST API to /strings/my.timeline.metric/raw:

  [
    {"timestamp": 1485352477824, "value": "Starting my scenario..."}
  ]
  1. Check your charts:

annotation

Building

You need npm and grunt to build the project. Clone the repository from github, then from that directory run:

npm install
grunt

Files are generated under the dist directory. To test your build, copy these files to ${GRAFANA_PLUGINS}/hawkular-datasource and restart grafana-server.

Running with Docker

Docker images are available on docker hub: https://hub.docker.com/r/hawkular/hawkular-grafana-datasource/

To run it:

# This will run the image on http://localhost:3000/
docker run -i -p 3000:3000 --name hawkular-grafana-datasource --rm hawkular/hawkular-grafana-datasource:latest

To build the docker image with the latest git release:

# This will build the image
cd docker
docker build -t hawkular/hawkular-grafana-datasource .

Running on OpenShift

If you are running on an OpenShift environment; several pre-made templates are available. Once you have decided which template fits your requirements, simply process and create the template as shown below:

oc process -f https://raw.githubusercontent.com/hawkular/hawkular-grafana-datasource/master/docker/openshift/openshift-template-ephemeral.yaml | oc create -f -

Troubleshooting

Grafana fails to establish a connection or get data from hawkular

  • Check the URL: [host]/hawkular/metrics. Make sure there’s no ending slash. When you open up this URL in a browser you should see the Hawkular logo, the installed version and a mention that the service is started.

  • Make sure the credentials or token match your installation. In general, if you installed a standalone Hawkular Metrics server without any specific configuration you probably don’t have any authentication information to provide. If you installed Hawkular-Services using its installation guide you will probably have to fill-in the basic auth fields. If you are using Hawkular from OpenShift, you have to provide a Bearer token in the Token field. Tokens can be generated temporarily (go to [OpenShift host]/oauth/token/request) or from a Service account in OpenShift.

  • Check the javascript debugging tool of your browser. If you see an error mentioning issues with CORS, switch to proxy mode in the datasource configuration.

I can’t query by tag, the option is not displayed

Querying by tag was introduced before the plugin was properly versioned, so if you have a version >= 1.0.2 you should have it. However it is only enabled when Grafana talks to Hawkular Metrics >= 0.20.0. To check your version of Hawkular Metrics just open its status page in a browser ([host]/hawkular/metrics or [host]/hawkular/metrics/status).

Connection is OK but I can’t get any metric

Make sure the tenant you’ve configured is exactly the same than the one used to insert data. Beware that it is case sensitive. If you have any doubt about the actual presence of data in Hawkular, you can confirm with a curl command, for instance:

curl -u myUsername:myPassword \
  -X GET "http://myserver/hawkular/metrics/gauges/mymetric/raw" \
  -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"`

Also note that in Hawkular, data has a retention period of 7 days by default (it can be configured). So if no data has been produced since that time, you won’t be able to see anything.

I’m running Hawkular in OpenShift, connection is OK but I can’t get any metric

Check your version of Hawkular Metrics ([host]/hawkular/metrics or [host]/hawkular/metrics/status). Prior to 0.16.0, metric names containing slashes, like in OpenShift, were unfortunately not showing up in Grafana. You can consider upgrading metrics.

redhatlogo-white

© 2016 | Hawkular is released under Apache License v2.0