[rudder-users] tags, comments and best practise

[tim taler]

Hi Benoit
thanks for answering
...
>
> We have the support in the agent and it can be done in a custom technique using the generic method "Variable dict from file type".
> For pre existing technique, the "Variable from JSON file (dict)" technique could easily be extended to support yaml too.
>
> > > One use-case here would be to automate the injection of new
> > > monitoring targets into my prometheus config
> > > (though that might be better done in a propper plugin,
> > > haven't had the time yet, but will also have a look at
> > > your centreon adapter ... but I like prometheus A LOT ;-)
>
> > Hey, we would enjoy to help you (or do with you) a prometheus plugin ! We typically add them based on users/customers need so that the plugin actually matches a real use case. Most likelly a part for Benoit ! (a couple more ideas, and I will have pong everyone :)
>
> What do you want to do with your plugin ? Monitor Rudder or configure the monitoring of something else ?

I like prometheus for monitoring nodes, services, switches and the like.
One of the main reasons I like prometheus is how easy it is to deploy
from the config-management point of view,
since there is not much of configuration to be done at all (compared
to other solutions - though all have their pros and cons ;).
In principal one only needs to add a new target (IP or hostname) to a
yaml snippet:

  - job_name: 'nodes'
    static_configs:
      - targets: [
                  # target_anker
192.168.9.110:9100,
192.168.9.111:9100,
192.168.9.112:9100
]

than prometheus will scrap this adresses (which are exposing their
metrics through html) and store the results.
This data than can be visualized through grafana (not necessarily
part of system-management but system-monitoring)
with quite powerful promql expressions.
Alerting on thresholds is done through the alertmanager (or, quick and
dirty, even through grafana for simple add-hoc cases)

I will have a look in the "variable from json file" next week.
The above snippet would be for a standard node_exporter, similar
snippets can be facilitated to monitor other endpoints, (snmp, mysql
....
see https://prometheus.io/docs/instrumenting/exporters/ for an actual list).

Monitoring rudder itself is an interesting idea!
For that though, it would be best if rudder would export it's own
metrics (maybe with a standalone rudder-exporter which pulls
information through existing rudder-api? and export the data in
prometheus style)
There might be value in this:
- how many nodes are managed, enforced, audited ....
- how regularly does a host pulls it's policies from the server ..
- how many compliance issues are detected ...
once one get his head into prometheus there are a lot of interesting
questions prometheus/grafana could answer I think.
In theory even each node agent could export it's own metrics

But since I haven't seen the rudder reporting module yet, it might
very well be an overkill?
Although the idea of "seeing it all in one place" might be intriguing
from the operations (hands-on) point of view
(while proper reports are more for the company management audience?)

Just brainstorming for now,
will get back on it next week.

Thanks again ;)
best
T