Table of Contents
- file_check_FIFO_pipe
- file_check_block_device
- file_check_character_device
- file_check_exists
- file_check_hardlink
- file_check_regular
- file_check_socket
- file_check_symlink
- file_check_symlinkto
- file_copy_from_local_source
- file_copy_from_local_source_recursion
- file_copy_from_remote_source
- file_copy_from_remote_source_recursion
- file_create
- file_create_symlink
- file_create_symlink_enforce
- file_create_symlink_force
- file_download
- file_enforce_content
- file_ensure_block_in_section
- file_ensure_block_present
- file_ensure_key_value
- file_ensure_key_value_present_in_ini_section
- file_ensure_keys_values
- Usage
- Example
- file_ensure_line_present_in_ini_section
- file_ensure_line_present_in_xml_tag
- file_ensure_lines_absent
- file_ensure_lines_present
- file_from_string_mustache
- file_from_template
- file_from_template_jinja2
- Setup
- Syntax
- file_from_template_mustache
- Syntax
- file_from_template_type
- Usage
- Template types
- Example
- file_remove
- file_replace_lines
- Syntax
- Example
- file_template_expand
Checks if a file exists and is a FIFO/Pipe
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_FIFO_pipe_${file_name}_{ok, reached, kept}
if the file is a
FIFO, or
file_check_FIFO_pipe_${file_name}_{not_ok, reached, not_kept, failed}
if the file is not a fifo or does not exist
- file_name: File name
file_check_FIFO_pipe_${file_name}_{kept, repaired, not_ok, reached}
Checks if a file exists and is a block device
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_block_device_${file_name}_{ok, reached, kept}
if the file
is a block_device, or
file_check_block_device_${file_name}_{not_ok, reached, not_kept, failed}
if the file is not a block device or does not exist
- file_name: File name
file_check_block_device_${file_name}_{kept, repaired, not_ok, reached}
Checks if a file exists and is a character device
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_character_device_${file_name}_{ok, reached, kept}
if the
file is a character device, or
file_check_character_device_${file_name}_{not_ok, reached, not_kept, failed}
if the file is not a character device or does not exist
- file_name: File name
file_check_character_device_${file_name}_{kept, repaired, not_ok, reached}
Checks if a file exists
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_exists_${file_name}_{ok, reached, kept}
if the file exists,
or file_check_exists_${file_name}_{not_ok, reached, not_kept, failed}
if the file doesn’t exists
- file_name: File name
file_check_exists_${file_name}_{kept, repaired, not_ok, reached}
Checks if two files are the same (hard links)
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_hardlink_${file_name_1}_{ok, reached, kept}
if the two
files ${file_name_1}
and ${file_name_2}
are hard links of each
other, or
file_check_hardlink_${file_name_1}_{not_ok, reached, not_kept, failed}
if if the files are not hard links.
- file_name_1: File name #1
- file_name_2: File name #2
file_check_hardlink_${file_name_1}_{kept, repaired, not_ok, reached}
Checks if a file exists and is a regular file
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_regular_${file_name}_{ok, reached, kept}
if the file is a
regular_file, or
file_check_regular_${file_name}_{not_ok, reached, not_kept, failed}
if
the file is not a regular file or does not exist
- file_name: File name
file_check_regular_${file_name}_{kept, repaired, not_ok, reached}
Checks if a file exists and is a socket
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_socket_${file_name}_{ok, reached, kept}
if the file is a
socket, or
file_check_socket_${file_name}_{not_ok, reached, not_kept, failed}
if
the file is not a socket or does not exist
- file_name: File name
file_check_socket_${file_name}_{kept, repaired, not_ok, reached}
Checks if a file exists and is a symlink
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_symlink_${file_name}_{ok, reached, kept}
if the file is a
symlink, or
file_check_symlink_${file_name}_{not_ok, reached, not_kept, failed}
if
the file is not a symlink or does not exist
- file_name: File name
file_check_symlink_${file_name}_{kept, repaired, not_ok, reached}
Checks if first file is symlink to second file
Compatible with nodes running Rudder 3.1 or higher.
This bundle will define a class
file_check_symlinkto_${target}_{ok, reached, kept}
if the file
${symlink}
is a symbolic link to ${target}
, or
file_check_symlinkto_${target}_{not_ok, reached, not_kept, failed}
if
if it is not a symbolic link, or any of the files does not exist. The
symlink’s path is resolved to the absolute path and checked against the
target file’s path, which must also be an absolute path.
- symlink: Symbolic link (absolute path)
- target: Target file (absolute path)
file_check_symlinkto_${symlink}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a file or directory is copied from a local source
Compatible with nodes running Rudder 3.1 or higher.
- source: Source file
- destination: Destination file
file_copy_from_local_source_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a file or directory is copied from a local source
Compatible with nodes running Rudder 3.1 or higher.
- source: Source file
- destination: Destination file
- recursion: Recursion depth to enforce for this path (0, 1, 2, …, inf)
file_copy_from_local_source_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a file or directory is copied from a remote source
Compatible with nodes running Rudder 3.1 or higher.
Note: This method uses CFEngine file copy protocol, and can only download files from the policy server. To download a file from an external source, you can use HTTP with the file_download method.
This method requires that the policy server is configured to accept copy of the source file from the agents it will be applied to.
You have to write the full path of the file on the policy server, for example:
/home/myuser/myfile
If you are using Rudder, you can download a file from the shared files with:
/var/rudder/configuration-repository/shared-files/PATH_TO_YOUR_FILE
- source: Source file
- destination: Destination file
file_copy_from_remote_source_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a file or directory is copied from a remote source
Compatible with nodes running Rudder 3.1 or higher.
This method requires that the policy server is configured to accept copy of the source file or directory from the agents it will be applied to.
You have to write the full path of the file or directory on the policy server, for example:
/home/myuser/mydirectory
If you are using Rudder, you can download a file from the shared files with:
/var/rudder/configuration-repository/shared-files/PATH_TO_YOUR_DIRECTORY_OR_FILE
- source: Source file
- destination: Destination file
- recursion: Recursion depth to enforce for this path (0, 1, 2, …, inf)
file_copy_from_remote_source_${destination}_{kept, repaired, not_ok, reached}
Create a file if it doesn’t exist
Compatible with nodes running Rudder 3.1 or higher.
- target: File to create
file_create_${target}_{kept, repaired, not_ok, reached}
This is a bundle to create a symlink at a destination path and pointing to a source target except if a file or directory already exists.
Compatible with nodes running Rudder 3.1 or higher.
- source: Source file
- destination: Destination file
file_create_symlink_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to create a symlink at a destination path and pointing to a source target. This is also possible to enforce its creation
Compatible with nodes running Rudder 3.1 or higher.
- source: Source file
- destination: Destination file
- enforce: Force symlink if file already exist (true or false)
file_create_symlink_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to create a symlink at a destination path and pointing to a source target even if a file or directory already exists.
Compatible with nodes running Rudder 3.1 or higher.
- source: Source file
- destination: Destination file
file_create_symlink_${destination}_{kept, repaired, not_ok, reached}
Download a file if it does not exist, using curl with a fallback on wget
Compatible with nodes running Rudder 3.1 or higher.
This method finds a HTTP command-line tool and downloads the given source into the destination.
It tries curl
first, and wget
as fallback.
- source: URL to download from
- destination: File destination
file_download_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to enfore the content of a file
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- lines: Line(s) to add in the file
- enforce: Enforce the file to contain only line(s) defined (true or false)
file_ensure_lines_present_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a section contains exactly a text block
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- section_start: Start of the section
- section_end: End of the section
- block: Block representing the content of the section
file_ensure_block_in_section_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a text block is present in a specific location
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- block: Block(s) to add in the file
file_ensure_block_present_${file}_{kept, repaired, not_ok, reached}
Ensure that the file contains a pair of "key separator value"
Compatible with nodes running Rudder 3.1 or higher.
Edit (or create) the file, and ensure it contains an entry key → value with arbitrary separator between the key and its value. If the key is already present, the method will change the value associated with this key.
- file: File name to edit
- key: Key to define
- value: Value to define
- separator: Separator between key and value (for example "=" or " ")
file_ensure_key_value_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a key-value pair is present in a section in a specific location. The objective of this method is to handle INI-style files.
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- section: Name of the INI-style section under which the line should be added or modified (not including the [] brackets)
- name: Name of the key to add or edit
- value: Value of the key to add or edit
file_ensure_key_value_present_in_ini_section_${file}_{kept, repaired, not_ok, reached}
Ensure that the file contains all pairs of "key separator value", with arbitrary separator between each key and its value
Compatible with nodes running Rudder 3.1 or higher.
This method ensures key-value pairs are present in a file.
This method will iterate over the key-value pairs in the dict, and:
- If the key is not defined in the destination, add the key separator + value line.
- If the key is already present in the file, replace the key separator + anything by key + separator + value
This method always ignores spaces and tabs when replacing (which means
for example that key = value
will match the =
separator).
Keys are considered unique (to allow replacing the value), so you should use file_ensure_lines_present if you want to have multiple lines with the same key.
If you have an initial file (/etc/myfile.conf
) containing:
key1 = something key3 = value3
To define key-value pairs, use the variable_dict or variable_dict_from_file methods.
For example, if you use the following content (stored in
/tmp/data.json
):
{ "key1": "value1", "key2": "value2" }
With the following policy:
# Define the `content` variable in the `configuration` prefix from the json file variable_dict_from_file("configuration", "content", "/tmp/data.json") # Enforce the presence of the key-value pairs file_ensure_keys_values("/etc/myfile.conf", "configuration.content", " = ")
The destination file (/etc/myfile.conf
) will contain:
key1 = value1 key3 = value3 key2 = value2
- file: File name to edit
- keys: Dict structure containing the keys (keys of the dict), and values to define (values of the dict)
- separator: Separator between key and value (for example "=" or " ")
file_ensure_keys_values_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a line is present in a section in a specific location. The objective of this method is to handle INI-style files.
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- section: Name of the INI-style section under which lines should be added (not including the [] brackets)
- line: Line to ensure is present inside the section
file_ensure_line_present_in_ini_section_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a line is present in a tag in a specific location. The objective of this method is to handle XML-style files. Note that if the tag is not present in the file, it won’t be added, and the edition will fail.
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- tag: Name of the XML tag under which lines should be added (not including the <> brackets)
- line: Line to ensure is present inside the section
file_ensure_line_present_in_xml_tag_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a line is absent in a specific location
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- lines: Line(s) to remove in the file
file_ensure_lines_absent_${file}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that one or more lines are present in a file
Compatible with nodes running Rudder 3.1 or higher.
- file: File name to edit
- lines: Line(s) to add in the file
file_ensure_lines_present_${file}_{kept, repaired, not_ok, reached}
This is a bundle to build a file from a mustache string
Compatible with nodes running Rudder 3.2 or higher.
- template: String containing a template to be expanded
- destination: Destination file
file_from_string_mustache_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to build a file from a legacy CFEngine template
Compatible with nodes running Rudder 3.1 or higher.
See file_from_template_type for general documentation about templates usage.
- source_template: Source file containing a template to be expanded
- destination: Destination file
file_from_template_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to build a file from a jinja2 template
Compatible with nodes running Rudder 3.1 or higher.
See file_from_template_type for general documentation about templates usage.
This generic method will build a file from a jinja2 template using data (classes and variables) found in the execution context.
It requires to have the jinja2 python module installed on the node, it
can usually be done in ncf with
package_present("python-jinja2", "", "", "")
.
Warning | |
---|---|
If you are using a jinja2 version older than 2.7 trailing newlines will not be preserved in the destination file. |
Jinja2 is a powerful templating language, running in Python. The Jinja2 syntax reference documentation is http://jinja.pocoo.org/docs/dev/templates/ which will likely be useful, as Jinja2 is very rich and allows a lot more that what is explained here.
This section presents some simple cases that cover what can be done with mustache templating, and the way the agent data is provided to the templating engine.
The main specificity of jinja2 templating is the use of two root containers:
-
classes
to access currently defined classes -
vars
to access all currently defined variables
Note: You can add comments in the template, that will not be rendered in
the output file with {# ... #}
.
To display content based on classes definition:
{% if classes.my_class is defined %} display this if defined {% endif %} {% if not classes.my_class is defined %} display this if not defined {% endif %}
Note: You cannot use class expressions here.
Here is how to display a scalar variable value (integer, string, …),
if you have defined
variable_string("variable_prefix", "my_variable", "my_value")
:
{{ vars.variable_prefix.my_variable }}
To iterate over a list, for example defined with:
variable_iterator("variable_prefix", "iterator_name", "a,b,c", ",")
Use the following file:
{% for item in vars.variable_prefix.iterator_name %} {{ item }} is the current iterator_name value {% endfor %}
Which will be expanded as:
a is the current iterator_name value b is the current iterator_name value c is the current iterator_name value
To iterate over a container defined by the following json file, loaded
with variable_dict_from_file("variable_prefix", "dict_name", "path")
:
{ "hosts": [ "host1", "host2" ], "files": [ {"name": "file1", "path": "/path1", "users": [ "user1", "user11" ] }, {"name": "file2", "path": "/path2", "users": [ "user2" ] } ], "properties": { "prop1": "value1", "prop2": "value2" } }
Use the following template:
{% for item in vars.variable_prefix.dict_name.hosts %} {{ item }} is the current hosts value {% endfor %} # will display the name and path of the current file {% for file in vars.variable_prefix.dict_name.files %} {{ file.name }}: {{ file.path }} {% endfor %} # will display the users list of each file {% for file in vars.variable_prefix.dict_name.files %} {{ file.name }}: {{ file.users|join(' ') }} {% endfor %} # will display the current properties key/value pair {% for key, value in vars.variable_prefix.dict_name.properties %} {{ key }} -> {{ value }} {% endfor %}
Which will be expanded as:
host1 is the current hosts value host2 is the current hosts value # will display the name and path of the current file file1: /path1 file2: /path2 # will display the users list of each file file1: user1 user11 file2: user2 # will display the current properties key/value pair prop1 -> value1 prop2 -> value2
- source_template: Source file containing a template to be expanded
- destination: Destination file
file_from_template_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to build a file from a mustache template
Compatible with nodes running Rudder 3.1 or higher.
See file_from_template_type for general documentation about templates usage.
Mustache is a logic-less templating language, available in a lot of languages, and used for file templating in CFEngine. The mustache syntax reference is https://mustache.github.io/mustache.5.html.
We will here describe the way to get agent data into a template. Ass explained in the general templating documentation, we can access various data in a mustache template.
The main specificity compared to standard mustache syntax of prefixes in all expanded values:
-
classes
to access classes -
vars
to access all variables
Here is how to display content depending on classes definition:
{{#classes.my_class}} content when my_class is defined {{/classes.my_class}} {{^classes.my_class}} content when my_class is *not* defined {{/classes.my_class}}
Note: You cannot use class expressions here.
Here is how to display a scalar variable value (integer, string, …),
if you have defined
variable_string("variable_prefix", "my_variable", "my_value")
:
{{{vars.variable_prefix.my_variable}}}
We use the triple {{{ }}}
to avoid escaping html entities.
Iteration is done using a syntax similar to scalar variables, but applied on container variables.
-
Use
{{#vars.container}} content {{/vars.container}}
to iterate -
Use
{{{.}}}
for the current element value in iteration -
Use
{{{.key}}}
for thekey
value in current element -
Use
{{{@}}}
for the current element key in iteration
To iterate over a list, for example defined with:
variable_iterator("variable_prefix", "iterator_name", "a,b,c", ",")
Use the following file:
{{#vars.variable_prefix.iterator_name}} {{{.}}} is the current iterator_name value {{/vars.variable_prefix.iterator_name}}
Which will be expanded as:
a is the current iterator_name value b is the current iterator_name value c is the current iterator_name value
To iterate over a container defined by the following json file, loaded
with variable_dict_from_file("variable_prefix", "dict_name", "path")
:
{ "hosts": [ "host1", "host2" ], "files": [ {"name": "file1", "path": "/path1", "users": [ "user1", "user11" ] }, {"name": "file2", "path": "/path2", "users": [ "user2" ] } ], "properties": { "prop1": "value1", "prop2": "value2" } }
Use the following template:
{{#vars.variable_prefix.dict_name.hosts}} {{{.}}} is the current hosts value {{/vars.variable_prefix.dict_name.hosts}} # will display the name and path of the current file {{#vars.variable_prefix.dict_name.files}} {{{.name}}}: {{{.path}}} {{/vars.variable_prefix.dict_name.files}} # will display the users list of each file {{#vars.variable_prefix.dict_name.files}} {{{.name}}}:{{#users}} {{{.}}}{{/users}} {{/vars.variable_prefix.dict_name.files}} # will display the current properties key/value pair {{#vars.variable_prefix.dict_name.properties}} {{{@}}} -> {{{.}}} {{/vars.variable_prefix.dict_name.properties}}
Which will be expanded as:
host1 is the current hosts value host2 is the current hosts value # will display the name and path of the current file file1: /path1 file2: /path2 # will display the users list of each file file1: user1 user11 file2: user2 # will display the current properties key/value pair prop1 -> value1 prop2 -> value2
Note: Starting from CFEngine 3.7, you can use
{{#-top-}} ... {{/-top-}}
to iterate over the top level container.
- source_template: Source file containing a template to be expanded
- destination: Destination file
file_from_template_${destination}_{kept, repaired, not_ok, reached}
This is a bundle to build a file from a template
Compatible with nodes running Rudder 3.1 or higher.
These methods write a file based on a provided template and the data available to the agent.
To use these methods (file_from_template_*
), you need to have:
- a template file
- data to fill this template
The template file should be somewhere on the local file system, so if you want to use a file shared from the policy server, you need to copy it first (using file_copy_from_remote_source).
It is common to use a specific folder to store those templates after
copy, for example in ${sys.workdir}/templates/
.
The data that will be used while expanding the template is the data available in the agent at the time of expansion. That means:
-
CFEngine's sytem variables (
${sys.*}
, …) and classes (linux
, ``) - data defined during execution (outcome classes of generic methods, …)
-
classes based on
condition_
generic methods -
data defined in ncf using
variable_*
generic methods, which allow for example to load data from local json or yaml files.
ncf currently supports three templating languages:
- mustache templates, which are documented in file_from_template_mustache
- jinja2 templates, which are documented in file_from_template_jinja2
- CFEngine templates, which are a legacy implementation that is here for compatibility, and should not be used for new templates.
Here is a complete example of templating usage:
The (basic) template file, present on the server in
/PATH_TO_MY_FILE/ntp.conf.mustache
(for syntax reference, see
file_from_template_mustache):
{{#classes.linux}} server {{{vars.configuration.ntp.hostname}}} {{/classes.linux}} {{^classes.linux}} server hardcoded.server.example {{/classes.linux}}
And on your local node in /tmp/ntp.json
, the following json file:
{ "hostname": "my.hostname.example" }
And the following policy:
# Copy the file from the policy server file_copy_from_remote_source("/PATH_TO_MY_FILE/ntp.conf.mustache", "${sys.workdir}/templates/ntp.conf.mustache") # Define the `ntp` varibale in the `configuration` prefix from the json file variable_dict_from_file("configuration", "ntp", "/tmp/ntp.json") # Expand yout template file_from_template_type("${sys.workdir}/templates/ntp.conf.mustache", "/etc/ntp.conf", "mustache") # or # file_from_template_mustache("${sys.workdir}/templates/ntp.conf.mustache", "/etc/ntp.conf")
The destination file will contain the expanded content, for example on a Linux node:
server my.hostname.example
- source_template: Source file containing a template to be expanded
- destination: Destination file
- template_type: Template type (cfengine, jinja2 or mustache)
file_from_template_${destination}_{kept, repaired, not_ok, reached}
Remove a file if it exists
Compatible with nodes running Rudder 3.1 or higher.
- target: File to remove
file_remove_${target}_{kept, repaired, not_ok, reached}
This is a bundle to ensure that a line in a file is replaced by another one
Compatible with nodes running Rudder 3.1 or higher.
You can replace lines in a files, based on regular expression and captured pattern
The content to match in the file is a PCRE regular expression, unanchored that you can replace with the content of replacement.
Content can be captured in regular expression, and be reused with the
notation ${match.1}
(for first matched content), ${match.2}
for
second, etc, and the special captured group ${match.0}
for the whole
text.
Here is an example to remove enclosing specific tags
file_replace_lines("/PATH_TO_MY_FILE/file", "<my>(.*)<pattern>", "my ${match.1} pattern")
- file: File name to edit
- line: Line to match in the file
- replacement: Line to add in the file as a replacement
file_replace_lines_${file}_{kept, repaired, not_ok, reached}
This is a bundle to expand a template in a specific location
WARNING: This generic method is deprecated. Use file_from_template instead.
Compatible with nodes running Rudder 3.1 or higher.
- tml_file: File name (with full path within the framework) of the template file
- target_file: File name (with full path) where to expand the template
- mode: Mode of destination file
- owner: Owner of destination file
- group: Froup of destination file
file_template_expand_${target_file}_{kept, repaired, not_ok, reached}