[rudder-dev] Having correct FQDN in (first) inventory
Francois Armand
francois.armand at normation.com
Fri May 13 16:49:14 CEST 2016
On 02/05/2016 22:13, Mattyasovszky János wrote:
> Hi
>
> /Answers Inline.
> /
> Regards,
> Matya
>
> Jonathan Clarke <jonathan.clarke at normation.com
> <mailto:jonathan.clarke at normation.com>> ezt írta (időpont: 2016. máj.
> 2., H, 18:29):
>
> Hi,
>
>
> On 04/13/2016 11:48 AM, Francois wrote:
>> On 13/04/2016 09:49, Nicolas Charles wrote:
>>> Hello Francois,
>>>
>>> Thank you for the nice sum-up. My answers in the text
>
> +1
>
>
>>> Le 07/04/2016 15:33, Francois Armand a écrit :
>>>> On 07/04/2016 15:08, Vincent Membré wrote:
>>>>> Le 31/03/2016 12:37, Francois Armand a écrit :
>>>>>> Hello,
>>>>>>
>>>>>> So, here goes for a summary of
>>>>>> http://www.rudder-project.org/redmine/issues/8022, "Node's
>>>>>> FQDN-Resolution is sometimes invalid" and related tickets.
>>>>>> The problem cover up several sub-cases, which need to be
>>>>>> addressed systematically to achieve some result.
>>>>>> They are:
>>>>>>
>>>>>> * 1/ the node FQDN is used for identifying a node, and then
>>>>>> manage authentication and authorization to access its
>>>>>> promises. If Rudder server, CFEngine promise server, and
>>>>>> the node don't agree on it, the node can't get its
>>>>>> promises. This is a hard problem because:
>>>>>> o FQDN is fragile. It needs a perfectly up to date and
>>>>>> shared DNS environment. But "it's always a DNS
>>>>>> problem", what gives an idea.
>>>>>> o FQDN tools are notoriously broken, and don't always
>>>>>> agree about what is the FQDN of a host
>>>>>> o even in a perfectly working and up-to-date DNS env,
>>>>>> there may have voluntary decision not to have the
>>>>>> same FQDN from the host and the server, as explained
>>>>>> in #8022 ticket.
>>>>>> * 2/ if the node FQDN is not correct in the first sent
>>>>>> inventory, the node configuration is delayed by one day,
>>>>>> because from the node point of view, the inventory was
>>>>>> correctly sent and so the standard frequency for sending
>>>>>> inventories is applied.
>>>>>>
>>>>>>
>>>>>> The long-term solution for 1/ is to use something else that
>>>>>> FQDN to identify the node - we have for example an UUID for
>>>>>> that. The problem here is that it is a hard limitation of the
>>>>>> protocol used by cf-serverd. So to use another identification
>>>>>> scheme (and why not our own authentication/authorization), we
>>>>>> need to either patch cf-serverd or use a different
>>>>>> client-server protocol for promises transfer. Both solutions
>>>>>> are open, but are out of scope.
>
> Actually, it is already possible to do authorizations in
> cf-serverd based on the key hash of connecting agents:
> https://docs.cfengine.com/docs/3.7/reference-promise-types-access.html#admit_keys.
> This has been possible since CFEngine 3.6 (so works on Rudder
> 2.11+ agents).
>
> We could pretty easily implement this, since we already have an
> inventory with nodes CFEngine keys in. It could be opt-in as a
> start, while we ensure it's reliable (as in "more reliable that
> hostnames"). WDYT?
>
>
> /This is the most welcome!/
> /
> /
> /The only thing one should not forget about is that apache is still
> having its allowed IP Subnets, which are also not making one's life
> too easy if you have a ton of subnets your clients connect from./
> /
> /
> /Please also do not forget about
> https://www.rudder-project.org/redmine/issues/7893, which is about
> extending the Rudder API with the possibility to edit the allowed
> subnets for the policy servers, which one currently has to do either
> in the WebGUI or over some direct-brain-surgery out-of-band LDAP
> editing.../
>
>
>>>>>> Meanwhile, we can address at best 1/ and 2/
>>>>>>
>>>>>> *For 1/, *we need to prevent as much as we can to give bad
>>>>>> FQDN and in all case, give the user a possibility to hook
>>>>>> what he knows should be the correct value.
>>>>>> *- Prevent more bad FQDN:**
>>>>>> *=> update perl version:
>>>>>> http://www.rudder-project.org/redmine/issues/8123
>>>>>> I didn't find anything else on that subject, compared to what
>>>>>> we are doing now.
>>>>>>
>>>>>> *- Let the user hook the correct value: **
>>>>>> *=> http://www.rudder-project.org/redmine/issues/8022#note-20
>>>>>> Here, we still need to specify the path convention for
>>>>>> command and file to look for the correct FDQN.
>>>>> I have some remarks/questions here:
>>>>>
>>>>> * If I understand correctly the process will be: Inventory is
>>>>> ran on the Node, then inventory is modified to add data from
>>>>> commands / file ? or will this be used when runnig
>>>>> fusionInventory ?
>>>>
>>>> The idea is to patch fusion inventory plugin for Rudder to use
>>>> that logic.
>>> There are already a way to extend inventories:
>>> http://fusioninventory.org/documentation/agent/additional_content.html
>>> I think it will be much easier and future proof to use the
>>> FusionInventory way rather than reimplementing it ourselves
>>>
>>> We could use this principle to create our own task that would
>>> read data or run external script - but maybe that's what you
>>> were implying ?
>>
>> No, I meant that we should evolve the logic to fill
>> <RUDDER><HOSTNAME>, contribute the evolution to rudder plugin,
>> and patch our own until then.
>> I'm not about using the what you linked to for that, it's really
>> not an extension, but the actual "business" logic we want to
>> implement for Rudder <HOSTNAME>.
>>
>> On the other hand, what you propose seems really nice for 4670, I
>> updated the ticket accordingly.
>>
>
> /#4670 is really something very useful, as this would allow to merge
> other inventory-collecting systems into the one rudder collects with
> fusioninventory, which is currently not providing any interface to get
> any user-defined data, which could be collected on the server (like
> Firmware version of different cards, SAP SID etc.)/
>
>>>>> * In Which entry will they be stored ? RUDDER/HOSTAME? another one
>>>>
>>>> Yes, RUDDER/HOSTNAME
>>>>
>>>>> * Do we want to define Hostname only ? Would it not be better
>>>>> if the solution was much more adaptable and can modify any
>>>>> entry from the inventory ?
>>>>
>>>> It's an other ticket, #4670, linked in #8022. HOSTNAME (FQDN,
>>>> really) is different from the general use case (at least adding
>>>> information into inventory) because of the special importance
>>>> of FQDN in CFengine server/agent identification protocol. So it
>>>> is kind of ok to see #8022 as a bug in the existing versions,
>>>> and #4670 (or an extension of it) as a new feature going to
>>>> next versions.
>>> Yes, let start with HOSTNAME in released version
>>>>
>>>>>
>>>>>
>>>>> About the path, I suggest we should put them under
>>>>> /var/rudder/inventories :
>>>>>
>>>>> * If we modify only hostname (RUDDER/HOSTNAME):
>>>>> o /var/rudder/inventories/hostname-command: Command to
>>>>> execute the get the correct hostname
>>>>> o /var/rudder/inventories/hostname-file: File containing
>>>>> the path the file containing the correct hostname
>>>>>
>>>>
>>>> No specific feeling on that... Any idea, other ?
>>> I'm not sure it should be in /var - if these are scripts to run
>>> to get data, or data user should put, I feel they would live in
>>> other folder.
>
> It is pretty unusual to configure anything in /var. The right
> place for configs is /etc. (yes, policy_server.dat is a nasty
> exception to this rule). How about
> /opt/rudder/etc/hooks/inventory.d/ ? This would be a good start to
> having extensible inventory.
>
> (and yes, it would be desirable to have configs in /etc, without
> the /opt/rudder prefix, but we're talking about a bug fix in
> existing versions here, so that it clearly out of scope)
>
>
>
> /Well, you already have stuff in /opt/rudder/etc, so adding more stuff
> there would actually make more sense then opening a third location
> (well, there *is* actually /etc/sysconfig/rudder-apache, which is
> already a config below /etc, so you have already stuff laying in there.../
>
>>> Plus, if we want user to preseed the system with these
>>> scripts/data, before installing agent, /var/rudder will not exist
>
> This is irrelevant, I think. If you need to install a file in a
> directory, it's pretty easy to create - in fact, just good practice.
>
>
>>>>>> *For 2/, *we need to prevent the sending of inventory that
>>>>>> will be rejected by the server for sure. *
>>>>>> *=> http://www.rudder-project.org/redmine/issues/8127
>
> FYI, Benoît is working on this this week. We will implement simple
> checks on the most important fields using a Perl script (since the
> inventory is already run in Perl, we know it's available).
>
> Jon
> _______________________________________________
>
So, we sum up everything we have on the subject, and we have (finally)
something coherent with a consistent long term vision. So let's start by
that, it will make other decision clearer.
In the long term (i.e, not in a bug correction releaes, but can and
should go in next major):
* nodes don't get a set of complexe initial promise. They have only a
really small one, which mostly like say "go searching your initial
promises from the policy server at that place"
o this system can be use to also download a set of initial
configuration files, among which can be a set of inventory hooks,
* the inventory processing is done in three steps :
o 1/ produce an inventory file from fusion,
o 2/ run hooks from */var/rudder/hooks/inventory.d/**
+ each hook produce a json file, merged with the previous json
+ values are overloaded is a newer hooks define the same key
+ the resulting JSON file is check for the existence os a
"hostname" key. If it exists, it is extracted (removed) and
its value is used to overwrite <RUDDER><HOSTNAME>
+ the resulting final JSON file is added in the inventory
file, somewhere in <RUDDER> - it could make sense to send it
apport inventory.xml (like the key part) but the gain,
appart for not yielling because there is JSON in XML, does
not seems obvious
+ REMARQUE: it would very much make sense that the resulting
JSON file is left on the node, and can be used as a
datasource for node properties in directives etc. Or it will
make sense that along with promise download, the node
download such a JSON file, sourced for data for the
remaining of the run.
o 3/ the final inventory final is sanity check so that is
necessary data are missing (empty hostname, etc), it is not sent
to the policy server and an error is logged.
* all the content from the json file is added as properties of the
node (and will be searchable, usable to define group, usable as
variable in directives, etc).
This solve nicelly all the bug related to 8022:
* for 8022 itself, it means :
o that we can had a defaults"hostname" hook, with the logic "if
RUDDER_FQDN is defined non empty"
o users are free to add other hooks to overide it ("if RUDDER_FQDN
key is defined in file ...., use that" etc)
o and that can be done since Rudder 3.1 without problem, because
it won't change current behaviour
* for 8127 (check inventory before sending them), it changes nothing :
it's step 3
* we also checked 8123 (update perl version), and the amount of work
to get there is huge and risky. So 8123 will go in next major, and
we are going to try to solve 8283 ASAP
* finally, most of 4670 will be done: all the agent logic be there,
and rudder will be extended to actually use the data in next major.
Hope everhting is clear !
--
------------------------------------------------------------------------
*François ARMAND*
/Co-founder & CTO/
Normation <http://www.normation.com>
------------------------------------------------------------------------
*87 rue de Turbigo, 75003 Paris, France*
Telephone: +33 (0)1 83 62 99 23
Mobile: +33 (0)6 63 37 60 55
------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20160513/c9036f1f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logo-square.gif
Type: image/gif
Size: 1036 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20160513/c9036f1f/attachment-0001.gif>
More information about the rudder-dev
mailing list