Rudder

=== Technique reference

A technique is described by a XML file that lists:

* the template files

* the sections of the technique

* the variables that must be defined

* the compatibility list

==== Files organisation

The techniques are ordered in Categories. A Category is described by a category.xml file, that defines the name and description of a category. A Category can contain other Categories, or Techniques. A Technique is versioned, and can exist in several versions. The description of a Technique is the metadata.xml file.

----

techniques

+--- category.xml

+--- fileConfiguration

| +--- category.xml

| +--- security

| | +--- filesPermissions

| | | +--- 1.0

| | | | +--- permlist.st

| | | | +--- metadata.xml

| | | | +--- filesPermissions.st

| | +--- category.xml

| | +--- sudoCheck

| | | +--- 2.0

| | | | +--- metadata.xml

| | | | +--- sudoCheck.st

| | | +--- 1.0

| | | | +--- metadata.xml

| | | | +--- sudoCheck.st

----

===== metadata.xml and CFEngine templates (*.st)

These files must reside in a folder with a version number. For each Technique, there can be several versions, Rudder will let you choose the version when creating a Directive.

===== Version number formating

The version number follows a formating "a la Debian" as described here: http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version, (without the debian_revision version)

==== General Rules

All the tag name in the .xml are in upper case, all the attributes are in camel case:

----

<SECTION name="example" component="true" componentKey="variable_name">

----

==== Details of the metadata.xml file

----

<TECHNIQUE id="technique_unique_id" name="human_name_of_the_technique">

<DESCRIPTION>Description of the Technique</DESCRIPTION>

<LONG_DESCRIPTION>Long description of the technique</LONG_DESCRIPTION>

<DEPRECATED>Deprecation message</DEPRECATED> <!-- Mark the Technique as deprecated, deprecation message is mandatory, Only available since Rudder 3.0 -->

<DISPLAY>true/false</DISPLAY> <!-- Define if the Technique is displayed in the interface or not. Default value : true -->

<COMPATIBLE> <!-- Optional, describe the version of the OS and CFEngine Agent the Technique has been tested on. Only for information purpose -->

<OS version=">=2.5">OS Name</OS> <!-- Optional; OS Name and version on which the Technique has been tested -->

<AGENT version=">=3.6">cfengine-community</AGENT> <!-- Optional; Agent name and version on which the Technique has been tested -->

</COMPATIBLE>

<MULTIINSTANCE>true/false</MULTIINSTANCE> <!-- Optional; defines if several instances of this template with differents variables can be deployed on a node; default value : false -->

<SYSTEM>true/false</SYSTEM> <!-- Optional, defines if this Technique is a system Technique (internal Rudder usage); default value : false -->

<BUNDLES> <!-- List of the bundles that must be included in the bundlesequence -->

<NAME>BundleName</NAME>

</BUNDLES>

<TMLS> <!-- List of all the templates defined by this Technique -->

<TML name="tmlName"> <!-- Container for a TML (without the trailing .st -->

<OUTPATH>relativ/path/of/file</OUTPATH> <!-- Optional; defines the relative path for the generated file for this template; default : techniqueName/version/tmlName.cf -->

<INCLUDED>true/false</INCLUDED> <!-- Optional; defines if the template must be in the inputs list of the generated promises; default : true -->

</TML>

</TMLS>

<TRACKINGVARIABLE> <!-- Defines a special system variable TRACKINGKEY that contains all the necessary information to track which Directive generated the promises -->

<SAMESIZEAS>VariableName</SAMESIZEAS> <!-- Optional; defines the cardinality of this variable based on the cardinality of the VariableName -->

</TRACKINGVARIABLE>

<SECTIONS> <!-- Lists all the sections of the promises -->

<SECTION name="sectionName"> <!-- Container of a section (see below) -->

</SECTION>

</SECTIONS>

</TECHNIQUE>

----

===== The <SECTION> tag

In a metadata.xml, there can be only one SECTIONS tag, that encloses one or several SECTION tags. A SECTION tag contains variables declaration and subsections. A SECTION can contains Variables definitions and SECTION.

----

<SECTION name="sectionName" multivalued="true/false" component="true/false" componentKey="variableName/None">

----

A SECTION has the following attributes:

* name : mandatory, the name of the section

* multivalued : optional, default false, defines if the section is repetable or not. If so, the Web Interface will display a "Add another" and "Delete" button for this section

* component : optional, default false; defines if the section is a component (for the reporting)

* componentKey: optional, default None; defines the variable that is the key of the component. Note that the componentKey can only be defines if component is true

* displayPriority: optional, default high; defines if the section is displayed by default (high) or hidden by default (low) (2.6 and higher only)

NOTE : A multivalued section can only contain variable, and cannot contain section (for the moment)

===== Variables definitions in the <SECTION> tags

There are three tags to create a variable:

* SELECT1: Can select only one value out of several. If there are less than 3 possible values, displays radio buttons, otherwise a select field.

* SELECT: Can select several values out of al the possibles. Displays checkboxes.

* INPUT: Displays an input field (that can be tuned)

----

<SELECT1/SELECT/INPUT> <!-- Depend on the display and behaviour needed -->

<NAME>variableName</NAME>

<DESCRIPTION>variableDescription</DESCRIPTION>

<LONGDESCRIPTION>longDescription</LONGDESCRIPTION> <!-- Optional, set the text in the tooltips -->

<UNIQUEVARIABLE>true/false</UNIQUEVARIABLE> <!-- Optional, default false; if true, this variable will have the same value over all the instance of this template for a given node -->

<ITEM> <!-- Only for SELECT and SELECT1, list of selectable values -->

<VALUE>value</VALUE> <!-- value that will be put in the template-->

<LABEL>humanReadableText</LABEL> <!-- value displayed in the web interface -->

</ITEM>

<CONSTRAINT> <!-- Optional, defines some constraints on values -->

<DEFAULT>defaultValue</DEFAULT> <!-- Optional; Defines a default value -->

<TYPE>variableType</TYPE> <!-- Optional; default string; variable type -->

<MAYBEEMPTY>true/false</MAYBEEMPTY> <!-- Optional; default false; defines if the variable is optional or not; only for the INPUT variable -->

<REGEX error="errorMsg">regex</REGEX> <!-- Optional; only for the INPUT variable; efine a regular expression the variable should match, and an optional error message -->

<PASSWORDHASH>hashtype</PASSWORDHASH> <!-- Optional; only for the password TYPE variable; define the way a password will be handled (hashed or not, hash types allowed ...) -->

</CONSTRAINT>

</SELECT1/SELECT/INPUT>

----

Note: It is possible to inline LABEL and VALUE in the ITEM tag

----

<ITEM label="Red" value="red"/>

----

is equivalent to

----

<ITEM>

<LABEL>Red</LABEL>

<VALUE>red</VALUE>

</ITEM>

----

NOTE: INPUT fields are automatically escaped, meaning any quote will be written in the policies as \" ; and any backslash will be written as \\

===== Available types for an INPUT variable

* *string* : any string is accepted (no specific displayer)

* *textarea* : accept any strings, but use a textarea in place of the input text.

* *perm* : display a matrix of read/write/execute by user/group/all

* *integer* : only accept integers

* *datetime* : display a JQuery calendar and check date format

* *boolean* : display a checkbox

* *mail* : only accept emails

* *ip* : only accept ips. Before Rudder 3.1.14, 3.2.7 and 4.0.0, "ip" was accepting only IPv4 ip. Since these releases, it accepts both IPv4 and IPv6 format. <br />

* *ipv4* [since Rudder 3.1.14, 3.2.7, 4.0.0]: only accept IPv4 formatedt IPs

* *ipv6* [since Rudder 3.1.14, 3.2.7, 4.0.0]: only accept IPv6 formatted IPs

* *size-<unit>* : (size-b, size-kb, size-mb, size-gb ou size-tb)

* *raw* : the content of this field will not be escaped when written in the promises (Rudder >= 2.6)

* *password* : the content of this field will be handled as a password, and thus be hidden and transformed if necessary (see "Password handling" below) (Rudder >= 2.6)

==== Examples

===== Multivalued sections

In the "NFS Client settings" Technique, there is a multivalued section with several entries. Here is a partial extract from it, with

* A multivalued section, named NFS mountpoint, that is multivalued and is a component. The variable reference for this component (the key) is NFS_CLIENT_LOCAL_PATH

* One SELECT1 field, that will show two radio buttons, Mount and Unmount, with the default value to Mount

* One INPUT field, named NFS_CLIENT_LOCAL_PATH, that is a text

----

<SECTION name="NFS mountpoint" multivalued="true" component="true" componentKey="NFS_CLIENT_LOCAL_PATH">

<SELECT1>

<NAME>NFS_CLIENT_UMOUNT</NAME>

<DESCRIPTION>Which operation should be done on this mountpoint</DESCRIPTION>

<ITEM>

<LABEL>Mount</LABEL>

<VALUE>no</VALUE>

</ITEM>

<ITEM>

<LABEL>Unmount</LABEL>

<VALUE>yes</VALUE>

</ITEM>

<CONSTRAINT>

<DEFAULT>no</DEFAULT>

</CONSTRAINT>

</SELECT1>

<INPUT>

<NAME>NFS_CLIENT_LOCAL_PATH</NAME>

<DESCRIPTION>Local path to mount the remote on</DESCRIPTION>

</INPUT>

...

</SECTION>

----

===== Unique variable across several instance

This variable can have only one value, over all the instances of this Technique, on a node

----

<SECTIONS>

<INPUT>

<NAME>UNIQUE</NAME>

<DESCRIPTION>Unique variable</DESCRIPTION>

<CONSTRAINT>

<TYPE>string</TYPE>

<CONSTRAINT>

<UNIQUEVARIABLE>true</UNIQUEVARIABLE>

</INPUT>

</SECTIONS>

----

===== Password handling

The password type allows to show an input text field whose content will be hashed when the form is submitted so that the password is never store in clear text.

image:./images/password-field.png[Directive Password Field]

*Available hash formats*

For now, the password field support these hash algorithms :

* *PLAIN* : that is not an hash algorithm, it just save the password in plain text, as inputed by the user.

* *MD5, SHA1, SHA256, SHA512* : uses the matching hash algorithm

* *LINUX-SHADOW-MD5, LINUX-SHADOW-SHA256, LINUX-SHADOW-SHA512* : build a string compatible with the Linux /etc/shadow format, as "specified" in http://man7.org/linux/man-pages/man3/crypt.3.html

*Technique metatdata content*

To configure a password, you must specify two things in the `<CONSTRAINT>` section of the field:

* `<TYPE>password</TYPE>` : use the password type

* `<PASSWORDHASH>comma,separated,list,of,hash</PASSWORDHASH>` : specify the list of hash algo from witch the user will be allowed to choose.

* Available algorithm names are the ones from the section above (case insensitive).

* Choices are presented in order given by the list, the first being the default one.

* If the list contains only one algo, the drop down select if change to a phrase saying to the user that the given algo will be used.

* The list can not be empty. Moreover, if the `<MAYBEEMPTY>` contraint is set to false, the "None" option is not displayed to the user.

*Password field definition example*

----

<SECTION name="Password" component="true" componentKey="USERGROUP_USER_LOGIN">

<INPUT>

<NAME>USERGROUP_USER_PASSWORD</NAME>

<DESCRIPTION>Password for this account</DESCRIPTION>

<CONSTRAINT>

<MAYBEEMPTY>true</MAYBEEMPTY>

<TYPE>password</TYPE>

<PASSWORDHASH>linux-shadow-md5,linux-shadow-sha256,linux-shadow-sha512</PASSWORDHASH>

</CONSTRAINT>

</INPUT>

</SECTION>

----

Other examples are available http://www.rudder-project.org/redmine/projects/clerk/repository/revisions/master/entry/src/test/resources/testVariable.xml[here].

==== Known limitations

There are several known limitations at the moment, that are acknowleged, and will be solved in a "not too distant" future:

===== Can't put a multivalued section in a multivalued section

It is not possible, due to limitation in the format in which the variable's values are stored in the LDAP tree, to put multivalued sections within multivalued sections.

===== Can't have several multivalued sections that are components with keys

For the moment, there is only one TRACKINGKEY, so it is not possible to have several multivalued sections that have keys.

===== Can't have several sections that are components with keys in multivalued Techniques.

It is a side effect of the previous limitation.

=== Reports reference

This page describes the concept behind the reporting in Rudder, and specifically how to write the Techniques to get proper reporting in Rudder

==== Concepts

Each Technique, when converted into a Directive and applied to a Node, must generate reports for Rudder to get proper compliance reports. This reports must contains specific information :

* The Report type, that can be logs for information purpose or result to express a compliance

* The Rule Id (autogenerated)

* The Directive Id (autogenerated)

* The Version Id (revision of the Rule) (autogenerated)

* The name of the component the report is related to

* The value of the key variable in the component (or None if not available)

* The Execution Timestamp, to know in which execution of the agent the promise has been generated

These reports are sent via Syslog to the Rudder Root Server, parsed and put in a database, that is queried to generate the reporting

==== Report format

A report has the following format :

----

@@Technique@@Type@@RuleId@@DirectiveId@@VersionId@@Component@@Key@@ExecutionTimeStamp##NodeId@#HumanReadableMessage

----

* Technique : Human readable Technique name

* Type : type of report (see bellow)

* RuleId : The Id of the Configuration Rule, autogenerated

* DirectiveId : The Id of the Directive, autogenerated

* VersionId : the revision of the ConfigurationRule, autogenerated

* Component : the name of the component this Directive is related to (if no component are defined in the metadata.xml, then the Technique name is used)

* Key : the value of the reference variable. If there is no reference variable, then the value None should be used

* ExecutionTimeStamp : the timestamp of the current CFEngine execution

* NodeId : the id of the node

* HumanReadableMessage : a message than a Human can understand

===== Valid report types

[cols="1,1,1,4", options="header"]

.Report Types

|===

| Name

| Type

| Max number

| Details

| log_trace

| log

| infinity

| Should be used for advanced debuging purpose only

| log_debug

| log

| infinity

| Should be used for debug purpose only

| log_info

| log

| infinity

| Use for standard logging purposes

| log_warn

| log

| infinity

| Used for logging only for the moment. Should be used when something unexpected happens

| log_repaired

| log

| infinity

| Used for logging purposes, to list all that is repaired by the promises

| result_na

| result

| one per component/key

| Defines the status of the Component to Not Applicable (if there are no result_success, result_repaired, result_error) Should be used only when everything is already in the correct state in this component for this key

| result_success

| result

| one per component/key

| Defines the status of the Component to Success (if there are no result_repaired or result_error) Should be used only when everything is already in the correct state in this component for this key

| result_repaired

| result

| one per component/key

| Defines the status of the Component to Repaired (if there are no result_error) Should be used only when something was not in the correct state, but could be corrected

| result_error

| result

| infinity per component/key

| Defines the status of the Component to Error Should be used when something was not in the correct state, and could not be corrected

|===

Variables used to generate the reports

Some facilities have been created to help putting the right values at the right place

* `&TRACKINGKEY&`: this is an auto generated variable, put in the technique file, that Rudder will replace when writing the promises by

----

<pre>RuleId@@DirectiveId@@VersionId

----

* `$(g.execRun)`: this is replaced at runtime by CFEngine 3 to the current execution time

* `$(g.uuid)`: this is replaced at runtime by CFEngine 3 to the Node Id

=== Syntax of the Techniques

==== Generalities

The Techniques use the http://www.stringtemplate.org/[StringTemplate] engine. A Technique *must* have the .st extension to be extended by Rudder (have some variables replaced, some part removed or added given some parameters).

==== Variable remplacement

Note : Rudder use a StringTemplate grammar slighlty different from the default one. Rather than using "$" as a variable identifier, the Techniques use "&" to avoid collision with the CFEngine variables

===== Single-valued variable remplacement

----

&UUID&

----

* Will be remplaced by the value of the variable UUID

===== Remplacement of variable with one or more values

----

&DNS_RESOLVERS: { "&it&" };separator=", "&

----

* Will be remplaced by `"8.8.8.8", "8.8.4.4"`

* Here, `&it&` is an alias for the current item in the list (with no confusion, because there is only one variable)

----

&POLICYCHILDREN, CHILDRENID : {host, uuid |

"/var/rudder/share/&uuid&/"

maproot => { host2ip("&host&"), escape("&host&") },

admit => { host2ip("&host&"), escape("&host&") };

} &

----

* `host` is an alias for the current value of POLICYCHILDREN

* `uuid` is an alias for the current value of CHILDRENID

* Both item are iterated at the same time, so both list must have the same length

===== Remplacement of variable with one or more value, and writing an index all along

----

&FILE_AND_FOLDER_MANAGEMENT_PATH:{path |"file[&i&][path]" string => "&path&";

}&

----

* _i_ is an iterator, starting at 1

The result would be:

----

"file[1][path]" string => "/var";

"file[2][path]" string => "/bin";

----

===== Conditionnal writing of a section

----

&if(NOVA)&

something

&endif&

----

The variable must either be:

* A boolean: If its value is true, then the section will be displayed

* A variable with the parameter `MAYBEEMPTY="true"`: If the value is not set, then the section won't be displayed, otherwise it will be displayed

More information can be found here: http://www.antlr.org/wiki/display/ST/ST+condensed+--+Templates+and+expressions

=== Best Practices for Techniques

==== Naming convention

* The name of bundle and classes should be written with underscore (i.e: this_is_a_good_example) instead of CamelCase (i.e: ThisIsABadExample)

* All variable, class and bundle names should be prefixed by "rudder_"

* The bundle entry point for the Technique should be named rudder_<name_of_the_technique>

* The bundles which makes all the actions should be suffixed by a meaningful name ( "rudder_<name_of_the_Technique>_installation", "rudder_<name_of_the_Technique>_configuration", "rudder_<name_of_the_Technique>_reporting", ..). This rule applies even if there is only one bundle

* The prefix of classes should all be "rudder_<name of the Technique>_"

* The classes defined as an outcome should be named:

* `rudder_<name of the Technique>_<action>_kept`

* `rudder_<name of the Technique>_<action>_repaired`

* `rudder_<name of the Technique>_<action>_failed`

* `rudder_<name of the Technique>_<action>_denied`

* `rudder_<name of the Technique>_<action>_timeout`

* `rudder_<name of the Technique>_<action>_error` (error include failed, denied and timeout)

* The name of the bodies written in the Rudder Library should be prefixed: `rudder_common_`

==== Raising classes

* `rudder_<name of the Technique>_<action>_error` should be raised simultaneously as `rudder_<name of the Technique>_<action>_failed`, `rudder_<name of the Technique>_<action>_denied` or `rudder_<name of the Technique>_<action>_timeout`.

* The body *rudder_common_classes* automatically abide by this rule

==== Writing convention

===== In the Technique

* We try to follow CFEngine conventions but with some exceptions like using brackets "{}" instead of parenthesis "()"

* When defining bundles or bodies, the opening bracket should be on a dedicated line. Exemple:

----

bundle common control

{

bundlesequence => { "exemple" };

}

----

* Indentation should be made by spaces. A incrementation of indentation is equal to two spaces

* The promise type should be indented by two spaces (instead of being at the same indentation level than the bundle name)

* The class expression should be indented by four spaces (two spaces after the promise type)

* The promiser should be indented by six spaces (two spaces after the class expression or four spaces after the promise type if no class expression is defined)

* Attributes of promises should be indented by eight spaces (two spaces after the promiser) and it should be only one attribute by line.

* Attribute's arrows '=>' should all be at the same level, one character after the largest attribute name

----

bundle agent example

{

type:

"promiser"

attribute => "value1";

class::

"promiser2"

attribute2 => "value2";

}

----

* Attributes of promise type "vars" and "classes" should be on only one line except if there are more than one attribute.

* For promise type "vars" and "classes" on one line, attribute names and the arrows should be aligned

* A list should be written multilines if it needs more than 80 characters in one line

* Multilines list should have comma after each element, except the last one.

* Multilines list should begin with only a bracket "{"

----

vars:

"value" slist =>

{

"one",

"two",

"three"

};

----

* The name of the variable in argument of the bundle should be named "params"

* The call of the variables should be made with by using brackets `${var_correctly_called}` instead of parenthesis `$(var_wrongly_called)`

* Alternance of brackets and parenthesis are tolerated when lots of variables are imbricated for more readability: `${var_lv1[$(var_lvl2[${var_lvl3}])]}`

* A Technique should have its bundle wrote with parameters

* All the bundles should have as first argument "prefix" which contains the prefix to use for all the classes made from an outcome. This prefix should never be hardcoded in the bundle.

* Always write comments with # when a promise needs more than 30 seconds of thought.

* If classes should be created in order to iterate for make a workaround of the normal ordering (i.e: "iteration_1", "iteration_2", "iteration_3"), they should always be defined at the end of the promise type "classes".

* The order to the promise type must always be in the order of the normal ordering : http://cfengine.com/manuals/cf3-Reference#Normal-ordering

* StringTemplate variables should always be written in UPPERCASE

* StringTemplate variables should be written with underscore

* StringTemplate variables should always be prefixed by the Technique name in uppecase too. i.e: `CHECK_GENERIC_FILE_FILE_NAME`

===== In the metadata.xml

* Name of sections should always be written in literary English (no CamelCase or underscores).

* The value of variable "Don't change" should always be "dontchange" or "" if the easier.

==== Files convention

* File names in a Technique should not be prefixed by the name of the Technique

* When a Technique needs specific bodies, the bodies should be written in a bodies.st file

* The file containing the bundle which makes all the actions (and containing the bundle "run") should be named "main.cf"

* The file containing all the variables and calling the bundle "run" should be name config.st

* Initialization of a new Technique should always be made from the file "technique-metadata-sample.xml" which is present on the root of the "rudder-techniques" repository

* Rudder standard library should be located in "common" Technique

==== Maintenance

* These rules were introduced after the 2.5 release of Rudder and before the 2.6 release. Therefore, they were enforced as of rudder-techniques-2.6.*.

* Always follow the conventions above when Techniques are updated but only for the lines edited. This rule concerns the Techniques on all the branches of git.

* On any branches that have released versions on them, we only allow minimal modifications. No lines should be modified if not to fix a bug (respecting these best practices is not currently considered a bug).

==== Testing

* There is a test suite in scripts/check-techniques.sh that check metadata.xml and normal ordering in code

* The list of all maintained techniques (techniques and versions) is in maintained-techniques file, and should be updated when new techniques or versions are created.