When the agent starts, it reads its configuration file from disk. The agent then attempts to connect to the Hawkular Server.
The agent will download some additional configuration from the Hawkular Server that it overlays on top of its existing
configuration (this pulls down specific metadata for the types of resources it is to manage).
The agent will also pull down a separate configuration file that is used to configure
the Prometheus JMX Exporter which is started by the agent. This is
called the "Metrics Exporter". Thus you will get an agent that collects inventory and stores it in Hawkular,
but you will also have a metrics exporter that will be used to expose metrics for collection by Hawkular’s Prometheus Server.
The agent determines what configuration files to pull down via the settings in the local agent configuration file. The local agent configuration file is the one referred to in the javaagent command line - it is where you told the agent to find the configuration file. e.g. javaagent:hawkular-javaagent.jar=config=this-config-file.yaml,delay=10:
subsystem:
type-version: WF10
...
metrics-exporter:
config-file: WF10
As an example, if the agent is to monitor EAP6, you would replace "WF10" with "EAP6". As more supported products are
added, additional configuration files are loaded on the Hawkular Server which can then be pulled down by the agent
by simply changing the values of these type-version and config-file settings.
|
Note
|
Note that a WildFly host controller agent must be told to download the metrics exporter
config file of "WF10-DOMAIN" while the type-version should still be "WF10".
See the WildFly+Agent distro (Maven artifact org.hawkular.agent:hawkular-javaagent-wildfly-dist) as an example.
|
When an agent itself is registered in inventory on the Hawkular Server, Hawkular’s Prometheus Server will be told to
scrape the agent’s metrics exporter endpoint, thus the agent’s metrics are collected automatically.
How It Works with WildFly Domain Mode
The agent runs in the host controller. The host controller’s agent can get all of the inventory for the host controller
and its slave servers. There is also a metrics exporter running in the host controller’s agent as well (so
metrics for the host controller can be collected).
An agent runs in each slave server as well, but those agents run in "metrics-only" mode. No inventory is collected
by these agents, but they do start a metrics exporter so metrics from each slave server can be collected.
To configure the Prometheus JMX Exporter in the slave server’s agent, you must turn it on in "slave proxy mode" via these
settings within the local agent configuration file. This is the agent configuration file you told the slave server to use in their -javaagent command line argument. See the host.xml entry you added for the slave servers. e.g.:
<jvms>
<jvm name="default">
<jvm-options>
<option value="-javaagent:${jboss.home.dir}/bin/hawkular-javaagent.jar=config=${jboss.domain.config.dir}/hawkular-javaagent-config-metrics-only.yaml,delay=10"/>
metrics-exporter:
proxy:
mode: slave
data-dir: ${jboss.domain.base.dir}/data/hawkular-metrics-exporter
The host controller’s agent must turn on the Prometheus JMX Exporter in "master proxy mode" in the host controller agent’s local configuration file (this is the configuration file you told the host controller agent to use in its -javaagent command line option within the domain.conf file):
metrics-exporter:
proxy:
mode: master
data-dir: ${jboss.domain.data.dir}/hawkular-metrics-exporter
Note the data directories between the slaves and master must be the same. It is recommended to use a subdirectory under WildFly’s domain/data directory as you see in the examples above.
What happens under the covers is the slave server will write a file to the data directory describing the
metrics exporter endpoint it started. The master will collect this information from all slaves and makes
sure the Hawkular Server will tell the Prometheus Server to scrape those slave endpoints as well as the host controller
agent’s own metrics exporter endpoint.