[rudder-users] Compliance reports
Francois Armand
francois.armand at normation.com
Fri Mar 30 11:58:01 CEST 2018
Hey, we would love to know more about your integration with check_mk,
and why not help you build a Rudder plugin for that. Nowadays, we are
trying to expend the number of integration plugins, and along with CMDB
like iTop, monitoring software are the main proponent (we have a working
plugin for Centreon for ex).
In particular, we would love to know if you are missing things in Rudder
integration api to reach your goal - and a compliance hook will most
likelly be one that list (see
https://www.rudder-project.org/redmine/issues/11221 or perhaps even a
full fledge event notification system).
Cheers,
On 30/03/2018 11:44, Rob Pomeroy wrote:
> Hi Francois,
>
> Thank you for your very complete answers! Yes, I see what you mean
> about the semantic/race condition on compliance state.
>
> I'll think about how best to express my change request for an improved
> search interface.
>
> In the meantime I need to stop being lazy and just take the plunge
> with the API! My ultimate aim is to bring Rudder status into Check_MK,
> so I have a single pane of glass for monitoring and alerting. (We then
> integrate Check_MK with PagerDuty so we can have the delight of being
> woken up in the middle of the night when a disk fills.) I may also
> send events into Graylog.
>
> Merci beaucoup!
>
> Rob
>
>
> --
> *Rob Pomeroy*, CISSP <https://www.isc2.org/cissp>, Solicitor
> <http://solicitors.lawsociety.org.uk/person/250541/robert-john-pomeroy>
> *| *rob at pomeroy.me <mailto:rob at pomeroy.me>
> My novel <https://www.smashwords.com/books/view/78386> | LinkedIn
> <http://www.linkedin.com/in/robpomeroy> | Personal blog
> <http://pomeroy.me/> | Geek & Dummy <http://geekanddummy.com/>
>
> On 30 March 2018 at 10:26, Francois Armand
> <francois.armand at normation.com <mailto:francois.armand at normation.com>>
> wrote:
>
> On 29/03/2018 15:59, Rob Pomeroy wrote:
>> Hi,
>>
>> This probably should be raised as a feature request/issue - but I
>> wanted the views of those on this list first? (Also when I looked
>> in the issue log, there are many pages of issues there, so no
>> point adding another one if it wouldn't be a priority.) :-)
>>
>> I'd really like to be able to search by the compliant state of
>> nodes. It'd be handy to be able to pull out just those nodes that
>> require attention. I have dozens and dozens of managed nodes now
>> and it's getting harder to find those that need to be fixed.
>>
>> So two things I'd be interested in:
>>
>> 1. A search/dynamic group based on whether a not a node is in a
>> compliant state.
>> 2. A way of finding nodes based on a period between today's date
>> and the last inventory received. You can of course search
>> based on a fixed date, but I'm not sure what, if any,
>> functions can be used in the search box. Something like last
>> inventory date > now() - 1 day.
>>
>>
>> I couldn't find anything like this in the online documentation or
>> the interface. But perhaps I've overlooked something that already
>> exists?
>>
>> Cheers,
>>
>> Rob
>>
>> -
>
> Hello Rob,
>
> Firstly, just to be sure you didn't miss that: you can sort nodes
> in the "all node" list by compliance (click on the "Compliance"
> column header to do so), so that help put the one with red on top.
>
> Now, for your requests:
>
> 1/ This one is interesting, but it's hard to get right in the
> "group" workflow. First, there is the semantic problem about what
> is the "compliance" number you want to match on. Global compliance
> ? One state in particular ? It can be sorted out, but with the
> current group search UI, our tests make it cumbersome.
> But then, the real problem is in the feedback loop and
> hysteris/instabilities it may introduce in your system. Imagine
> you build such a group and make a rule on them. Now, the rule
> depends on its outcome, with a propagation travel time to take in
> account. Node may enter and get out fast ("fast" as in "not many
> tickes", considering a "tick" at rudder scale to be 5min). So that
> will multiply policy generation, but the time they reach the
> nodes, perhaps the dyn group already change again...
>
> So. To sum up, it is a not obvious problem to sort out cleanly,
> and we are not yet at the point where we can go there.
>
> So, in the meantime, we are solving the much more easier problem
> of getting nice custom reports on compliance (read-only, no
> feedback loop, nice graphs :). It will be through a "compliance
> report" plugin that allows to build custom reports with history of
> compliance evolution on selected nodes/groups/rules/etc. It will
> be ready in the very short term (coming weeks) and part of a
> commercial offer for Rudder. If you are interested to learn more,
> I can put you in touch with the relevant person.
>
> And in a hacky-already-work way of doing things, you can use API +
> jq (https://stedolan.github.io/jq) to do what you want. For
> example, to get all nodes with global compliance < 80%, you can do:
>
> % curl -k -H "X-API-Token: xxxx" -H "Content-Type:
> application/json" -X GET
> 'https://....rudder/api/latest/compliance/nodes?level=1
> <https://....rudder/api/latest/compliance/nodes?level=1>' |*jq
> '.data.nodes | sort_by(.compliance) | map(select(.compliance <
> 80))' *
> [
> {
> "id": "717b63d1-01fe-4d4f-a7e5-cfb7c0d47b4f",
> "name": "debian-8-64.labo.normation.com
> <http://debian-8-64.labo.normation.com>",
> "compliance": 0,
> "mode": "full-compliance",
> "complianceDetails": {
> "unexpectedMissingComponent": 49.14,
> "unexpectedUnknownComponent": 50.86
> }
> }
> {
> "id": "0c846655-cb06-486f-ace4-eaeb11372097",
> "name": "centos-7-64.labo.normation.com
> <http://centos-7-64.labo.normation.com>",
> "compliance": 69.57,
> "mode": "full-compliance",
> "complianceDetails": {
> "successAlreadyOK": 30.43,
> "successNotApplicable": 34.78,
> "error": 30.43,
> "successRepaired": 4.35
> }
> }
> ...
> ]
>
>
> 2/ There is no particular function in the search box appart from
> "is:" (for rule, node, group, parameter, directive, rule) and
> "in:" (for attributes), as explained in the doc here:
> https://orchestrateur-4.labo.normation.com/rudder-doc/search-nodes.html
> <https://orchestrateur-4.labo.normation.com/rudder-doc/search-nodes.html>
> . Having a more complexe search language for the quick search
> would be nice. Perhaps could you mind opening an user story for that?
>
> In the meantime, you can again rely on APIs (yeah, that's our
> swiss army knife ;). It will fullfill what I understand of your
> need, more or less.
> More or less because medling with date format and date difference
> is a nightmare in any language, put it's even worse in the middle
> of a jq directive. So in place of "selected range date", I give
> you "sorted, and select a subsection of the resulting array".
>
> This command will give you the 3 nodes with the oldest last
> inventory date (you can choose any slice in the resulting array,
> of course):
>
> % curl -k -H "X-API-Token: dTxvl4eL8p3YqvwefVbaJLdy8DyEt7Vw"
> -H "Content-Type: application/json" -X GET
> 'https://orchestrateur-4.labo.normation.com/rudder/api/latest/nodes?include=minimal,lastInventoryDate
> <https://orchestrateur-4.labo.normation.com/rudder/api/latest/nodes?include=minimal,lastInventoryDate>'
> | *jq '.data.nodes | sort_by(.lastInventoryDate) | .[0:3]'*
>
> [
> {
> "id": "fc846655-cb06-486f-ace4-eaeb11372097",
> "hostname": "sovma136",
> "status": "accepted",
> "lastInventoryDate": "2016-12-05 15:12"
> },
> {
> "id": "8b168194-c0b4-41ab-b2b5-9571a8906d59",
> "hostname": "debian-5-64.labo.normation.com
> <http://debian-5-64.labo.normation.com>",
> "status": "accepted",
> "lastInventoryDate": "2017-02-01 17:06"
> },
> {
> "id": "94b6d33d-a23e-46b9-b5f8-971751bebcbb",
> "hostname": "ubuntu-16-04-64",
> "status": "accepted",
> "lastInventoryDate": "2018-03-30 02:02"
> }
> ]
>
>
> And here come the link toward jq manual:
> https://stedolan.github.io/jq/manual
> <https://stedolan.github.io/jq/manual>
> jq is globally hard, but you can get a lots of things done with it.
>
> Hope it helps,
>
> --
>
>
> ------------------------------------------------------------------------
> *François ARMAND*
> /Co-founder & CTO/
> Normation <http://www.normation.com>
> ------------------------------------------------------------------------
> *87 rue de Turbigo, 75003 Paris, France*
> Telephone: +33 (0)1 83 62 99 23 <tel:+33%201%2083%2062%2099%2023>
> Mobile: +33 (0)6 63 37 60 55 <tel:+33%206%2063%2037%2060%2055>
> ------------------------------------------------------------------------
>
>
--
------------------------------------------------------------------------
*François ARMAND*
/Co-founder & CTO/
Normation <http://www.normation.com>
------------------------------------------------------------------------
*87 rue de Turbigo, 75003 Paris, France*
Telephone: +33 (0)1 83 62 99 23
Mobile: +33 (0)6 63 37 60 55
------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-users/attachments/20180330/5587f9c6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logo-square.gif
Type: image/gif
Size: 1036 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-users/attachments/20180330/5587f9c6/attachment-0001.gif>
More information about the rudder-users
mailing list