File


file_check_FIFO_pipe

Checks if a file exists and is a FIFO/Pipe

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_FIFO_pipe_${file_name}_{ok, reached, kept} if the file is a FIFO, or file_check_FIFO_pipe_${file_name}_{not_ok, reached, not_kept, failed} if the file is not a fifo or does not exist

Parameters

  • file_name: File name

Classes defined

file_check_FIFO_pipe_${file_name}_{kept, repaired, not_ok, reached}


file_check_block_device

Checks if a file exists and is a block device

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_block_device_${file_name}_{ok, reached, kept} if the file is a block_device, or file_check_block_device_${file_name}_{not_ok, reached, not_kept, failed} if the file is not a block device or does not exist

Parameters

  • file_name: File name

Classes defined

file_check_block_device_${file_name}_{kept, repaired, not_ok, reached}


file_check_character_device

Checks if a file exists and is a character device

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_character_device_${file_name}_{ok, reached, kept} if the file is a character device, or file_check_character_device_${file_name}_{not_ok, reached, not_kept, failed} if the file is not a character device or does not exist

Parameters

  • file_name: File name

Classes defined

file_check_character_device_${file_name}_{kept, repaired, not_ok, reached}


file_check_exists

Checks if a file exists

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_exists_${file_name}_{ok, reached, kept} if the file exists, or file_check_exists_${file_name}_{not_ok, reached, not_kept, failed} if the file doesn’t exists

Parameters

  • file_name: File name

Classes defined

file_check_exists_${file_name}_{kept, repaired, not_ok, reached}


file_check_hardlink

Checks if two files are the same (hard links)

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_hardlink_${file_name_1}_{ok, reached, kept} if the two files ${file_name_1} and ${file_name_2} are hard links of each other, or file_check_hardlink_${file_name_1}_{not_ok, reached, not_kept, failed} if if the files are not hard links.

Parameters

  • file_name_1: File name #1
  • file_name_2: File name #2

Classes defined

file_check_hardlink_${file_name_1}_{kept, repaired, not_ok, reached}


file_check_regular

Checks if a file exists and is a regular file

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_regular_${file_name}_{ok, reached, kept} if the file is a regular_file, or file_check_regular_${file_name}_{not_ok, reached, not_kept, failed} if the file is not a regular file or does not exist

Parameters

  • file_name: File name

Classes defined

file_check_regular_${file_name}_{kept, repaired, not_ok, reached}


file_check_socket

Checks if a file exists and is a socket

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_socket_${file_name}_{ok, reached, kept} if the file is a socket, or file_check_socket_${file_name}_{not_ok, reached, not_kept, failed} if the file is not a socket or does not exist

Parameters

  • file_name: File name

Classes defined

file_check_socket_${file_name}_{kept, repaired, not_ok, reached}


file_check_symlink

Checks if a file exists and is a symlink

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_symlink_${file_name}_{ok, reached, kept} if the file is a symlink, or file_check_symlink_${file_name}_{not_ok, reached, not_kept, failed} if the file is not a symlink or does not exist

Parameters

  • file_name: File name

Classes defined

file_check_symlink_${file_name}_{kept, repaired, not_ok, reached}


file_check_symlinkto

Checks if first file is symlink to second file

Compatible with nodes running Rudder 3.1 or higher.

Usage

This bundle will define a class file_check_symlinkto_${target}_{ok, reached, kept} if the file ${symlink} is a symbolic link to ${target}, or file_check_symlinkto_${target}_{not_ok, reached, not_kept, failed} if if it is not a symbolic link, or any of the files does not exist. The symlink’s path is resolved to the absolute path and checked against the target file’s path, which must also be an absolute path.

Parameters

  • symlink: Symbolic link (absolute path)
  • target: Target file (absolute path)

Classes defined

file_check_symlinkto_${symlink}_{kept, repaired, not_ok, reached}


file_copy_from_local_source

This is a bundle to ensure that a file or directory is copied from a local source

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • source: Source file
  • destination: Destination file

Classes defined

file_copy_from_local_source_${destination}_{kept, repaired, not_ok, reached}


file_copy_from_local_source_recursion

This is a bundle to ensure that a file or directory is copied from a local source

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • source: Source file
  • destination: Destination file
  • recursion: Recursion depth to enforce for this path (0, 1, 2, …, inf)

Classes defined

file_copy_from_local_source_${destination}_{kept, repaired, not_ok, reached}


file_copy_from_remote_source

This is a bundle to ensure that a file or directory is copied from a remote source

Compatible with nodes running Rudder 3.1 or higher.

Usage

Note: This method uses CFEngine file copy protocol, and can only download files from the policy server. To download a file from an external source, you can use HTTP with the file_download method.

This method requires that the policy server is configured to accept copy of the source file from the agents it will be applied to.

You have to write the full path of the file on the policy server, for example:

/home/myuser/myfile

If you are using Rudder, you can download a file from the shared files with:

/var/rudder/configuration-repository/shared-files/PATH_TO_YOUR_FILE

Parameters

  • source: Source file
  • destination: Destination file

Classes defined

file_copy_from_remote_source_${destination}_{kept, repaired, not_ok, reached}


file_copy_from_remote_source_recursion

This is a bundle to ensure that a file or directory is copied from a remote source

Compatible with nodes running Rudder 3.1 or higher.

Usage

This method requires that the policy server is configured to accept copy of the source file or directory from the agents it will be applied to.

You have to write the full path of the file or directory on the policy server, for example:

/home/myuser/mydirectory

If you are using Rudder, you can download a file from the shared files with:

/var/rudder/configuration-repository/shared-files/PATH_TO_YOUR_DIRECTORY_OR_FILE

Parameters

  • source: Source file
  • destination: Destination file
  • recursion: Recursion depth to enforce for this path (0, 1, 2, …, inf)

Classes defined

file_copy_from_remote_source_${destination}_{kept, repaired, not_ok, reached}


file_create

Create a file if it doesn’t exist

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • target: File to create

Classes defined

file_create_${target}_{kept, repaired, not_ok, reached}


file_create_symlink

This is a bundle to create a symlink at a destination path and pointing to a source target except if a file or directory already exists.

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • source: Source file
  • destination: Destination file

Classes defined

file_create_symlink_${destination}_{kept, repaired, not_ok, reached}


file_create_symlink_enforce

This is a bundle to create a symlink at a destination path and pointing to a source target. This is also possible to enforce its creation

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • source: Source file
  • destination: Destination file
  • enforce: Force symlink if file already exist (true or false)

Classes defined

file_create_symlink_${destination}_{kept, repaired, not_ok, reached}


file_create_symlink_force

This is a bundle to create a symlink at a destination path and pointing to a source target even if a file or directory already exists.

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • source: Source file
  • destination: Destination file

Classes defined

file_create_symlink_${destination}_{kept, repaired, not_ok, reached}


file_download

Download a file if it does not exist, using curl with a fallback on wget

Compatible with nodes running Rudder 3.1 or higher.

Usage

This method finds a HTTP command-line tool and downloads the given source into the destination.

It tries curl first, and wget as fallback.

Parameters

  • source: URL to download from
  • destination: File destination

Classes defined

file_download_${destination}_{kept, repaired, not_ok, reached}


file_enforce_content

This is a bundle to enfore the content of a file

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • lines: Line(s) to add in the file
  • enforce: Enforce the file to contain only line(s) defined (true or false)

Classes defined

file_ensure_lines_present_${file}_{kept, repaired, not_ok, reached}


file_ensure_block_in_section

This is a bundle to ensure that a section contains exactly a text block

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • section_start: Start of the section
  • section_end: End of the section
  • block: Block representing the content of the section

Classes defined

file_ensure_block_in_section_${file}_{kept, repaired, not_ok, reached}


file_ensure_block_present

This is a bundle to ensure that a text block is present in a specific location

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • block: Block(s) to add in the file

Classes defined

file_ensure_block_present_${file}_{kept, repaired, not_ok, reached}


file_ensure_key_value

Ensure that the file contains a pair of "key separator value"

Compatible with nodes running Rudder 3.1 or higher.

Usage

Edit (or create) the file, and ensure it contains an entry key → value with arbitrary separator between the key and its value. If the key is already present, the method will change the value associated with this key.

Parameters

  • file: File name to edit
  • key: Key to define
  • value: Value to define
  • separator: Separator between key and value (for example "=" or " ")

Classes defined

file_ensure_key_value_${file}_{kept, repaired, not_ok, reached}


file_ensure_key_value_present_in_ini_section

This is a bundle to ensure that a key-value pair is present in a section in a specific location. The objective of this method is to handle INI-style files.

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • section: Name of the INI-style section under which the line should be added or modified (not including the [] brackets)
  • name: Name of the key to add or edit
  • value: Value of the key to add or edit

Classes defined

file_ensure_key_value_present_in_ini_section_${file}_{kept, repaired, not_ok, reached}


file_ensure_keys_values

Ensure that the file contains all pairs of "key separator value", with arbitrary separator between each key and its value

Compatible with nodes running Rudder 3.1 or higher.

Usage

This method ensures key-value pairs are present in a file.

Usage

This method will iterate over the key-value pairs in the dict, and:

  • If the key is not defined in the destination, add the key separator + value line.
  • If the key is already present in the file, replace the key separator + anything by key + separator + value

This method always ignores spaces and tabs when replacing (which means for example that key = value will match the = separator).

Keys are considered unique (to allow replacing the value), so you should use file_ensure_lines_present if you want to have multiple lines with the same key.

Example

If you have an initial file (/etc/myfile.conf) containing:

key1 = something
key3 = value3

To define key-value pairs, use the variable_dict or variable_dict_from_file methods.

For example, if you use the following content (stored in /tmp/data.json):

{
   "key1": "value1",
   "key2": "value2"
}

With the following policy:

# Define the `content` variable in the `configuration` prefix from the json file
variable_dict_from_file("configuration", "content", "/tmp/data.json")
# Enforce the presence of the key-value pairs
file_ensure_keys_values("/etc/myfile.conf", "configuration.content", " = ")

The destination file (/etc/myfile.conf) will contain:

key1 = value1
key3 = value3
key2 = value2

Parameters

  • file: File name to edit
  • keys: Dict structure containing the keys (keys of the dict), and values to define (values of the dict)
  • separator: Separator between key and value (for example "=" or " ")

Classes defined

file_ensure_keys_values_${file}_{kept, repaired, not_ok, reached}


file_ensure_line_present_in_ini_section

This is a bundle to ensure that a line is present in a section in a specific location. The objective of this method is to handle INI-style files.

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • section: Name of the INI-style section under which lines should be added (not including the [] brackets)
  • line: Line to ensure is present inside the section

Classes defined

file_ensure_line_present_in_ini_section_${file}_{kept, repaired, not_ok, reached}


file_ensure_line_present_in_xml_tag

This is a bundle to ensure that a line is present in a tag in a specific location. The objective of this method is to handle XML-style files. Note that if the tag is not present in the file, it won’t be added, and the edition will fail.

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • tag: Name of the XML tag under which lines should be added (not including the <> brackets)
  • line: Line to ensure is present inside the section

Classes defined

file_ensure_line_present_in_xml_tag_${file}_{kept, repaired, not_ok, reached}


file_ensure_lines_absent

This is a bundle to ensure that a line is absent in a specific location

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • lines: Line(s) to remove in the file

Classes defined

file_ensure_lines_absent_${file}_{kept, repaired, not_ok, reached}


file_ensure_lines_present

This is a bundle to ensure that one or more lines are present in a file

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • file: File name to edit
  • lines: Line(s) to add in the file

Classes defined

file_ensure_lines_present_${file}_{kept, repaired, not_ok, reached}


file_from_string_mustache

This is a bundle to build a file from a mustache string

Compatible with nodes running Rudder 3.2 or higher.

Parameters

  • template: String containing a template to be expanded
  • destination: Destination file

Classes defined

file_from_string_mustache_${destination}_{kept, repaired, not_ok, reached}


file_from_template

This is a bundle to build a file from a legacy CFEngine template

Compatible with nodes running Rudder 3.1 or higher.

Usage

See file_from_template_type for general documentation about templates usage.

Parameters

  • source_template: Source file containing a template to be expanded
  • destination: Destination file

Classes defined

file_from_template_${destination}_{kept, repaired, not_ok, reached}


file_from_template_jinja2

This is a bundle to build a file from a jinja2 template

Compatible with nodes running Rudder 3.1 or higher.

Usage

See file_from_template_type for general documentation about templates usage.

This generic method will build a file from a jinja2 template using data (classes and variables) found in the execution context.

Setup

It requires to have the jinja2 python module installed on the node, it can usually be done in ncf with package_present("python-jinja2", "", "", "").

[Warning]Warning

If you are using a jinja2 version older than 2.7 trailing newlines will not be preserved in the destination file.

Syntax

Jinja2 is a powerful templating language, running in Python. The Jinja2 syntax reference documentation is http://jinja.pocoo.org/docs/dev/templates/ which will likely be useful, as Jinja2 is very rich and allows a lot more that what is explained here.

This section presents some simple cases that cover what can be done with mustache templating, and the way the agent data is provided to the templating engine.

The main specificity of jinja2 templating is the use of two root containers:

  • classes to access currently defined classes
  • vars to access all currently defined variables

Note: You can add comments in the template, that will not be rendered in the output file with {# ... #}.

Classes

To display content based on classes definition:

{% if classes.my_class is defined  %}
   display this if defined
{% endif %}
{% if not classes.my_class is defined %}
   display this if not defined
{% endif %}

Note: You cannot use class expressions here.

Scalar variables

Here is how to display a scalar variable value (integer, string, …), if you have defined variable_string("variable_prefix", "my_variable", "my_value"):

{{ vars.variable_prefix.my_variable }}

Iteration

To iterate over a list, for example defined with:

variable_iterator("variable_prefix", "iterator_name", "a,b,c", ",")

Use the following file:

{% for item in vars.variable_prefix.iterator_name %}
{{ item }} is the current iterator_name value
{% endfor %}

Which will be expanded as:

a is the current iterator_name value
b is the current iterator_name value
c is the current iterator_name value

To iterate over a container defined by the following json file, loaded with variable_dict_from_file("variable_prefix", "dict_name", "path"):

{
   "hosts": [
       "host1",
       "host2"
   ],
   "files": [
       {"name": "file1", "path": "/path1", "users": [ "user1", "user11" ] },
       {"name": "file2", "path": "/path2", "users": [ "user2" ] }
   ],
   "properties": {
       "prop1": "value1",
       "prop2": "value2"
   }
}

Use the following template:

{% for item in vars.variable_prefix.dict_name.hosts %}
{{ item }} is the current hosts value
{% endfor %}

# will display the name and path of the current file
{% for file in vars.variable_prefix.dict_name.files %}
{{ file.name }}: {{ file.path }}
{% endfor %}

# will display the users list of each file
{% for file in vars.variable_prefix.dict_name.files %}
{{ file.name }}: {{ file.users|join(' ') }}
{% endfor %}


# will display the current properties key/value pair
{% for key, value in vars.variable_prefix.dict_name.properties %}
{{ key }} -> {{ value }}
{% endfor %}

Which will be expanded as:

host1 is the current hosts value
host2 is the current hosts value

# will display the name and path of the current file
file1: /path1
file2: /path2

# will display the users list of each file
file1: user1 user11
file2: user2

# will display the current properties key/value pair
prop1 -> value1
prop2 -> value2

Parameters

  • source_template: Source file containing a template to be expanded
  • destination: Destination file

Classes defined

file_from_template_${destination}_{kept, repaired, not_ok, reached}


file_from_template_mustache

This is a bundle to build a file from a mustache template

Compatible with nodes running Rudder 3.1 or higher.

Usage

See file_from_template_type for general documentation about templates usage.

Syntax

Mustache is a logic-less templating language, available in a lot of languages, and used for file templating in CFEngine. The mustache syntax reference is https://mustache.github.io/mustache.5.html.

We will here describe the way to get agent data into a template. Ass explained in the general templating documentation, we can access various data in a mustache template.

The main specificity compared to standard mustache syntax of prefixes in all expanded values:

  • classes to access classes
  • vars to access all variables

Classes

Here is how to display content depending on classes definition:

{{#classes.my_class}}
   content when my_class is defined
{{/classes.my_class}}

{{^classes.my_class}}
   content when my_class is *not* defined
{{/classes.my_class}}

Note: You cannot use class expressions here.

Scalar variable

Here is how to display a scalar variable value (integer, string, …), if you have defined variable_string("variable_prefix", "my_variable", "my_value"):

{{{vars.variable_prefix.my_variable}}}

We use the triple {{{ }}} to avoid escaping html entities.

Iteration

Iteration is done using a syntax similar to scalar variables, but applied on container variables.

  • Use {{#vars.container}} content {{/vars.container}} to iterate
  • Use {{{.}}} for the current element value in iteration
  • Use {{{.key}}} for the key value in current element
  • Use {{{@}}} for the current element key in iteration

To iterate over a list, for example defined with:

variable_iterator("variable_prefix", "iterator_name", "a,b,c", ",")

Use the following file:

{{#vars.variable_prefix.iterator_name}}
{{{.}}} is the current iterator_name value
{{/vars.variable_prefix.iterator_name}}

Which will be expanded as:

a is the current iterator_name value
b is the current iterator_name value
c is the current iterator_name value

To iterate over a container defined by the following json file, loaded with variable_dict_from_file("variable_prefix", "dict_name", "path"):

{
   "hosts": [
       "host1",
       "host2"
   ],
   "files": [
       {"name": "file1", "path": "/path1", "users": [ "user1", "user11" ] },
       {"name": "file2", "path": "/path2", "users": [ "user2" ] }
   ],
   "properties": {
       "prop1": "value1",
       "prop2": "value2"
   }
}

Use the following template:

{{#vars.variable_prefix.dict_name.hosts}}
{{{.}}} is the current hosts value
{{/vars.variable_prefix.dict_name.hosts}}

# will display the name and path of the current file
{{#vars.variable_prefix.dict_name.files}}
{{{.name}}}: {{{.path}}}
{{/vars.variable_prefix.dict_name.files}}

# will display the users list of each file
{{#vars.variable_prefix.dict_name.files}}
{{{.name}}}:{{#users}} {{{.}}}{{/users}}
{{/vars.variable_prefix.dict_name.files}}


# will display the current properties key/value pair
{{#vars.variable_prefix.dict_name.properties}}
{{{@}}} -> {{{.}}}
{{/vars.variable_prefix.dict_name.properties}}

Which will be expanded as:

host1 is the current hosts value
host2 is the current hosts value

# will display the name and path of the current file
file1: /path1
file2: /path2

# will display the users list of each file
file1: user1 user11
file2: user2

# will display the current properties key/value pair
prop1 -> value1
prop2 -> value2

Note: Starting from CFEngine 3.7, you can use {{#-top-}} ... {{/-top-}} to iterate over the top level container.

Parameters

  • source_template: Source file containing a template to be expanded
  • destination: Destination file

Classes defined

file_from_template_${destination}_{kept, repaired, not_ok, reached}


file_from_template_type

This is a bundle to build a file from a template

Compatible with nodes running Rudder 3.1 or higher.

Usage

These methods write a file based on a provided template and the data available to the agent.

Usage

To use these methods (file_from_template_*), you need to have:

  • a template file
  • data to fill this template

The template file should be somewhere on the local file system, so if you want to use a file shared from the policy server, you need to copy it first (using file_copy_from_remote_source).

It is common to use a specific folder to store those templates after copy, for example in ${sys.workdir}/templates/.

The data that will be used while expanding the template is the data available in the agent at the time of expansion. That means:

  • CFEngine's sytem variables (${sys.*}, …) and classes (linux, ``)
  • data defined during execution (outcome classes of generic methods, …)
  • classes based on condition_ generic methods
  • data defined in ncf using variable_* generic methods, which allow for example to load data from local json or yaml files.

Template types

ncf currently supports three templating languages:

  • mustache templates, which are documented in file_from_template_mustache
  • jinja2 templates, which are documented in file_from_template_jinja2
  • CFEngine templates, which are a legacy implementation that is here for compatibility, and should not be used for new templates.

Example

Here is a complete example of templating usage:

The (basic) template file, present on the server in /PATH_TO_MY_FILE/ntp.conf.mustache (for syntax reference, see file_from_template_mustache):

{{#classes.linux}}
server {{{vars.configuration.ntp.hostname}}}
{{/classes.linux}}
{{^classes.linux}}
server hardcoded.server.example
{{/classes.linux}}

And on your local node in /tmp/ntp.json, the following json file:

{ "hostname": "my.hostname.example" }

And the following policy:

# Copy the file from the policy server
file_copy_from_remote_source("/PATH_TO_MY_FILE/ntp.conf.mustache", "${sys.workdir}/templates/ntp.conf.mustache")
# Define the `ntp` varibale in the `configuration` prefix from the json file
variable_dict_from_file("configuration", "ntp", "/tmp/ntp.json")
# Expand yout template
file_from_template_type("${sys.workdir}/templates/ntp.conf.mustache", "/etc/ntp.conf", "mustache")
# or
# file_from_template_mustache("${sys.workdir}/templates/ntp.conf.mustache", "/etc/ntp.conf")

The destination file will contain the expanded content, for example on a Linux node:

server my.hostname.example

Parameters

  • source_template: Source file containing a template to be expanded
  • destination: Destination file
  • template_type: Template type (cfengine, jinja2 or mustache)

Classes defined

file_from_template_${destination}_{kept, repaired, not_ok, reached}


file_remove

Remove a file if it exists

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • target: File to remove

Classes defined

file_remove_${target}_{kept, repaired, not_ok, reached}


file_replace_lines

This is a bundle to ensure that a line in a file is replaced by another one

Compatible with nodes running Rudder 3.1 or higher.

Usage

You can replace lines in a files, based on regular expression and captured pattern

Syntax

The content to match in the file is a PCRE regular expression, unanchored that you can replace with the content of replacement.

Content can be captured in regular expression, and be reused with the notation ${match.1} (for first matched content), ${match.2} for second, etc, and the special captured group ${match.0} for the whole text.

Example

Here is an example to remove enclosing specific tags

file_replace_lines("/PATH_TO_MY_FILE/file", "<my>(.*)<pattern>", "my ${match.1} pattern")

Parameters

  • file: File name to edit
  • line: Line to match in the file
  • replacement: Line to add in the file as a replacement

Classes defined

file_replace_lines_${file}_{kept, repaired, not_ok, reached}


file_template_expand

This is a bundle to expand a template in a specific location

WARNING: This generic method is deprecated. Use file_from_template instead.

Compatible with nodes running Rudder 3.1 or higher.

Parameters

  • tml_file: File name (with full path within the framework) of the template file
  • target_file: File name (with full path) where to expand the template
  • mode: Mode of destination file
  • owner: Owner of destination file
  • group: Froup of destination file

Classes defined

file_template_expand_${target_file}_{kept, repaired, not_ok, reached}