Technique creation

Rudder provides a set of pre-defined Techniques that cover some basic configuration and system administration needs. You can also create your own Techniques, to implement new functionalities or configure new services. This paragraph will walk you through this process.

There is two ways to configure new Techniques, either thanks to the web Technique Editor in Rudder or by coding them by hand.

The use of the Technique Editor (code name: ncf-builder) is the easiest way to create new Techniques and is fully integrated with Rudder. On the other hand, it does not allow the same level of complexity and expressiveness than coding a Technique by hand. Of course, coding new Techniques by hand is a more involved process that needs to learn how the Technique description language and Technique reporting works.

We advice to always start to try to create new Techniques with the Technique Editor and switch to the hand-coding creation only if you discover specific needs not addressed that way.

Recommended solution: Technique Editor

The easiest way to create your own Techniques is to use the Technique editor, a web interface to create and manage Techniques based on the ncf framework.

Creating a technique in the Technique Editor will generate a Technique for Rudder automatically. You can then use that Technique to create a Directive that will be applied on your Nodes thanks to a Rule.

For more information about ncf and the Technique editor, you can visit: http://www.ncf.io/

Using the Technique Editor

The Technique Editor is available in the Directive screen or directly in the Utilities menu. Once on the Technique Editor, creating a Technique simply consist to add desired "Generic Methods" building block and configure them.

When the Technique match your expectations, hitting save will automatically add it to available Technique in the Directive screen of Rudder (in the "User Technique" category).

Logs

In case of any issue with the Technique Editor, the first step should always be to look for its log messages. These logs are sent to Apache system error logs:

  • On Debian, by default: /var/log/apache2/error.log
  • On RHEL, by default: /var/log/httpd/error_log

Understanding how Technique Editor works

In this chapter, we are giving an overview about how the Technique Editor works and how it is integrated with the main Rudder application.

Directory layout

As explained in http://www.ncf.io/, ncf uses a structured directory tree composed of several layers of logic, from internal libraries to Techniques and user services. All the files and logic in these folders will be named "library" for simplicity

ncf directory structure exists in two root folders:

  • /usr/share/ncf/tree

    • This is the standard library installation folder. It is created and updated by the the ncf package. This folder will be completely overwritten when you update ncf package so you should never modify anything here: it will be lost at some point.
  • /var/rudder/configuration-repository/ncf

    • This is were you add your own ncf Generic Methods and Techniques. Techniques created with the Technique Editor will be located here, and both Generic and Techniques in that place will be accessible in the Technique Editor alongside what is provided by the standard library.

Sharing ncf code with nodes

To share those folders to all nodes, Rudder makes a copy of these folders in two places:

  • /var/rudder/ncf, for part common to all nodes - so NOT techniques,

    • /var/rudder/ncf/local is a copy of node-independant directories from /var/rudder/configuration-repository/ncf, so almost everything BUT /var/rudder/configuration-repository/ncf/50_techniques.
    • /var/rudder/ncf/common is a copy /usr/share/ncf/tree
  • /var/rudder/share/xxxx-yyyy-node-id-zzzz/rules/cfengine-community/Technique_Name/1.0/Technique_Name.cf for techniques, with one directory for each technique applied to the node.
  • /var/rudder/share/xxxx-yyyy-node-id-zzzz/rules/cfengine-community/rudder_expected_reports.csv contains information about report expected for all ncf techniques applied to that node.

Files in /var/rudder/ncf are synchronized automatically by the "rudder agent update" command when the agent runs on the server. So any modification done in files in these directories will be lost at the next synchronization.

Files under /var/rudder/share/ are updated during policy generation.

A node updates its ncf local library by copying the content of these two folders during its promise update phase.

From ncf Technique Editor to Rudder Techniques and back

Here we will explain how the Technique Editor integration to Rudder is done to transform ncf techniques into full fledge Rudder one. We will also get the big picture of the web flow and the resulting events triggered on Rudder servier side.

Each action in the Technique Editor interface produces requests to an API defined over ncf.

All of the requests are authenticated thanks to a token passed in the JSESSIONID header. The token is generated when an authenticated user is connected to the Rudder interface (typically thanks to his browser).

That token is shared to the Technique Editor interface, which itself passes the JSESSIONID header to all requests.

If you have authentication issue, check that your Rudder session is not expired.

Get request
Get request will get all Techniques and Generic Methods in a path passed as parameters of the request in the "path" javascript variable:

https://your.rudder.server/ncf-builder/#!?path=/var/rudder/configuration-repository/ncf

Get requests are triggered when accessing Technique editor.

The ncf API will parse all files in the parameter path by running "cf-promises -pjson" on all Techniques, checking that all Techniques are correctly formed.

The ncf API will also look to all Generic Methods description data to build the catalog of available Generic Methods.

The resulting information are sent back to the Technique Editor for displaying.

Post requests
Post requests are issued when a Technique is created, modified or deleted. They will only work on Techniques available in the path given in parameter.

They are triggered when clicking on save/delete button.

The main difference with get requests is that hooks are launched before and after the action is made.

We will see all hooks behavior in the following dedicated hooks section.

Hooks

On each POST request, pre- and post- hooks are executed by the Technique Editor. These hooks are used for the Rudder integration to help transform pure ncf Techniques into Rudder one.

  • pre-hooks are located in: /var/rudder/configuration-repository/ncf/pre-hooks.d
  • post-hooks are located in: /var/rudder/configuration-repository/ncf/post-hooks.d

As of March 2015, we have two post-hooks defined and no pre-hooks:

  • post.write_technique.commit.sh

    • It commits the Technique newly created into Rudder Git configuration repository located in /var/rudder/configuration-repository.
  • post.write_technique.rudderify.sh

    • It generates a valid Rudder Technique from a the newly created Technique and reloads Rudder Technique Library so that updates are taken into account.

If you want to run post hooks by hand, you can use the following command:

/var/rudder/configuration-repository/ncf/post-hooks.d/post.write_technique.commit.sh /var/rudder/configuration-repository bundle_name

Create Technique manually

Prerequisite

To create a Technique, you’ll need a few things:

CFEngine knowledge
Rudder's Techniques are implemented using CFEngine. Rudder takes care of a lot of the work of using CFEngine, but you’ll need to have a reasonable understanding of the CFEngine syntax.
Rudder installation for testing
To be able to test your new Technique, you’ll need a working Rudder installation (at least a server and a node).
Text editor
The only other tool you need is your favorite text editor!

Define your objective

Before starting to create a new Technique, have you checked that it doesn’t already exist in Rudder? The full list of current Techniques is available from GitHub, at GitHub rudder-techniques repository.

OK, now we’ve got that over with, let’s go on.

A Technique should be an abstract configuration. This means that your Technique shouldn’t just configure something one way, but instead it should implement how to configure something, and offer options for users to choose what way they want it configured. Before starting, make sure you’ve thought through what you want to create.

Here’s a quick checklist to help:

  • Do you need to install packages?
  • Do you need to create or edit configuration files?
  • Do you need to copy files from a central location?
  • Do you need to launch processes or check that they’re running?
  • Do you need to run commands to get things working?

Once you’ve made a list of what needs doing, consider what options could be presented in the user interface, when you create a Directive from your new Technique. Intuitively, the more variables there are, the more flexible your Technique will be. However, experience shows that making the Technique too configurable will actually make it harder to use, so a subtle balance comes in to play here.

At this stage, make a list of all the variables that should be presented to users configuring a Directive from your Technique.

Initialize your new Technique

The simplest way to create a new Technique and be able to test it as you work is to start on a Rudder server. Open a terminal and connect to your Rudder server by ssh, and cd into the directory where Techniques are stored:

cd /var/rudder/configuration-repository/techniques

Under this directory, you’ll find a set of categories, and sub-categories. Before creating your Technique, choose a category to put it in, and change to that directory. For example:

cd applications

You can consult the description of each category by looking at the category.xml file in each directory. For this example:

cat category.xml

Will output:

<xml>
    <name>Application management</name>
    <description>This category contains Techniques designed to install,
        configure and manage applications</description>
</xml>

Once you’ve decided on a category, it’s time to create the basic skeleton of your Technique. The technical name for your Technique is it’s directory name, so choose wisely:

mkdir sampleTechnique

All directories under this one are version numbers. Let’s start with a simple 1.0 version. From now on, we’ll work in this directory.

mkdir sampleTechnique/1.0
cd sampleTechnique/1.0

Now, you need a minimum of two files to get your Technique working:

metadata.xml
This file describes the Technique, and configures how it will be displayed in the web interface.
st files
These files are templates for CFEngine configuration files. You need at least one, but can have as many as you like. Rudder processes them to generate .cf files ready to be used by CFEngine.

To get started, copy and paste these sample files, or download them from GitHub:

metadata.xml (original file: technique-metadata-sample.xml)

include::technique-metadata-sample.xml

sample_technique.st (original file: technique-st-sample.xml)

include::technique-st-sample.xml

Define variables

WORK IN PROGRESS Define metadata. Enter the variables in sections in the metadata.xml file. Cf http://www.rudder-project.org/foswiki/Development/PolicyTemplateXML

First test in the Rudder interface

Load the new Technique into Rudder and check that the variables and sections are displayed as you expect.

Implement the behavior

WORK IN PROGRESS Write CFEngine promises to implement the behavior that your Template should have.

Read in the variables from Rudder

WORK IN PROGRESS Using StringTemplate notation… Cf http://www.rudder-project.org/foswiki/Development/Technique

Add reporting

WORK IN PROGRESS The reports format Cf http://www.rudder-project.org/foswiki/Development/ReportsInTechniques