Project

General

Profile

« Previous | Next » 

Revision fb5e79b5

Added by François ARMAND about 7 years ago

Fixes #10269: Doc about copying ncf technique to /var/rudder/ncf/local is false

View differences:

4_advanced_usage/30_technique_creation.txt
Techniques, to implement new functionalities or configure new services. This
paragraph will walk you through this process.
There is two ways to configure new Techniques, either thanks to the web
There is two ways to configure new Techniques, either thanks to the web
Technique Editor in Rudder or by coding them by hand.
The use of the Technique Editor (code name: http://www.ncf.io/pages/ncf-builder.html[ncf-builder])
is the easiest way to create new Techniques and is fully integrated with Rudder. On the other hand,
it does not allow the same level of complexity and expressiveness than coding a Technique by hand.
Of course, coding new Techniques by hand is a more involved process that needs to learn how the
The use of the Technique Editor (code name: http://www.ncf.io/pages/ncf-builder.html[ncf-builder])
is the easiest way to create new Techniques and is fully integrated with Rudder. On the other hand,
it does not allow the same level of complexity and expressiveness than coding a Technique by hand.
Of course, coding new Techniques by hand is a more involved process that needs to learn how the
Technique description language and Technique reporting works.
We advice to always start to try to create new Techniques with the Technique Editor and switch to
We advice to always start to try to create new Techniques with the Technique Editor and switch to
the hand-coding creation only if you discover specific needs not addressed that way.
==== Recommended solution: Technique Editor
......
The easiest way to create your own Techniques is to use the Technique editor,
a web interface to create and manage Techniques based on the ncf framework.
Creating a technique in the Technique Editor will generate a Technique for Rudder automatically.
Creating a technique in the Technique Editor will generate a Technique for Rudder automatically.
You can then use that Technique to create a Directive that will be applied on your Nodes thanks
to a Rule.
to a Rule.
For more information about ncf and the Technique editor, you can visit: http://www.ncf.io/
===== Using the Technique Editor
The Technique Editor is available in the Directive screen or directly in the Utilities menu.
Once on the Technique Editor, creating a Technique simply consist to add desired "Generic Methods"
The Technique Editor is available in the Directive screen or directly in the Utilities menu.
Once on the Technique Editor, creating a Technique simply consist to add desired "Generic Methods"
building block and configure them.
When the Technique match your expectations, hitting save will automatically add it to available
Technique in the Directive screen of Rudder (in the "User Technique" category).
When the Technique match your expectations, hitting save will automatically add it to available
Technique in the Directive screen of Rudder (in the "User Technique" category).
===== Logs
===== Logs
In case of any issue with the Technique Editor, the first step should always be to look for its log messages.
In case of any issue with the Technique Editor, the first step should always be to look for its log messages.
These logs are sent to Apache system error logs:
- On Debian, by default: /var/log/apache2/error.log
- On RHEL, by default: /var/log/httpd/error_log
- On Debian, by default: `/var/log/apache2/error.log`
- On RHEL, by default: `/var/log/httpd/error_log`
==== Understanding how Technique Editor works
In this chapter, we are giving an overview about how the Technique Editor works and how it is
integrated with the main Rudder application.
In this chapter, we are giving an overview about how the Technique Editor works and how it is
integrated with the main Rudder application.
===== Directory layout
===== Directory layout
As explained in http://www.ncf.io/, ncf uses a structured directory tree composed of several layers of logic,
As explained in http://www.ncf.io/, ncf uses a structured directory tree composed of several layers of logic,
from internal libraries to Techniques and user services. All the files and logic in these folders will be named
"library" for simplicity
"library" for simplicity
ncf directory structure exists in two root folders:
* /usr/share/ncf/tree
* `/usr/share/ncf/tree`
** This is the standard library installation folder. It is created and updated by the
the ncf package. This folder will be completely overwritten when you update ncf
package so you should never modify anything here: it will be lost at some point.
the ncf package. This folder will be completely overwritten when you update ncf
package so you should never modify anything here: it will be lost at some point.
* /var/rudder/configuration-repository/ncf
* `/var/rudder/configuration-repository/ncf`
** This is were you add your own ncf Generic Methods and Techniques.
Techniques created with the Technique Editor will be located here, and both Generic Methods
and Techniques in that place will be accessible in the Technique Editor alongside what is
provided by the standard library.
Techniques created with the Technique Editor will be located here, and both
Generic and Techniques in that place will be accessible in the Technique Editor
alongside what is provided by the standard library.
To share those folders to all nodes, Rudder makes a copy of these folders in /var/rudder/ncf:
==== Sharing ncf code with nodes
* /var/rudder/ncf/local is a copy of /var/rudder/configuration-repository/ncf
* /var/rudder/ncf/common is a copy /usr/share/ncf/tree
To share those folders to all nodes, Rudder makes a copy of these folders in two
places:
They are synchronized automatically by the agent running on the server. So any modification done
in files in these directories will be lost at the next synchronization.
* `/var/rudder/ncf`, for part common to all nodes - so NOT techniques,
** `/var/rudder/ncf/local` is a copy of node-independant directories from
`/var/rudder/configuration-repository/ncf`, so almost everything *BUT*
`/var/rudder/configuration-repository/ncf/50_techniques`.
** `/var/rudder/ncf/common` is a copy `/usr/share/ncf/tree`
* `/var/rudder/share/xxxx-yyyy-node-id-zzzz/rules/cfengine-community/Technique_Name/1.0/Technique_Name.cf`
for techniques, with one directory for each technique applied to the node.
* `/var/rudder/share/xxxx-yyyy-node-id-zzzz/rules/cfengine-community/rudder_expected_reports.csv`
contains information about report expected for all ncf techniques applied to that node.
A node updates its ncf local library by copying the content of these two folders during its
promise update phase.
===== Technique Editor integration with Rudder
Here we will explain Technique Editor behavior and what workflow are initialized by different
action on the Technique Editor.
Files in `/var/rudder/ncf` are synchronized automatically by the "rudder agent update"
command when the agent runs on the server. So any modification done in files
in these directories will be lost at the next synchronization.
Files under `/var/rudder/share/` are updated during policy generation.
A node updates its ncf local library by copying the content of these two folders
during its promise update phase.
===== From ncf Technique Editor to Rudder Techniques and back
Here we will explain how the Technique Editor integration to Rudder is done to
transform ncf techniques into full fledge Rudder one. We will also get the
big picture of the web flow and the resulting events triggered on Rudder servier
side.
Each action in the Technique Editor interface produces requests to an API defined over ncf.
All of the requests are authenticated thanks to a token passed in the JSESSIONID header.
The token is generated when an authenticated user is connected to the Rudder interface
(typically thanks to his browser).
All of the requests are authenticated thanks to a token passed in the JSESSIONID header.
The token is generated when an authenticated user is connected to the Rudder interface
(typically thanks to his browser).
That token is shared to the Technique Editor interface, which itself passes the
That token is shared to the Technique Editor interface, which itself passes the
JSESSIONID header to all requests.
If you have authentication issue, check that your Rudder session is not expired.
......
https://your.rudder.server/ncf-builder/#!?path=/var/rudder/configuration-repository/ncf
Get requests are triggered when accessing Technique editor.
Get requests are triggered when accessing Technique editor.
The ncf API will parse all files in the parameter path by running "cf-promises -pjson" on all Techniques,
checking that all Techniques are correctly formed.
......
The ncf API will also look to all Generic Methods description data to build the catalog of available
Generic Methods.
The resulting information are sent back to the Technique Editor for displaying.
The resulting information are sent back to the Technique Editor for displaying.
Post requests::
Post requests are issued when a Technique is created, modified or deleted.
Post requests are issued when a Technique is created, modified or deleted.
They will only work on Techniques available in the path given in parameter.
They are triggered when clicking on save/delete button.
......
===== Hooks
On each POST request, pre- and post- hooks are executed by the Technique Editor.
On each POST request, pre- and post- hooks are executed by the Technique Editor.
These hooks are used for the Rudder integration to help transform pure ncf Techniques into Rudder one.
- pre-hooks are located in: /var/rudder/configuration-repository/ncf/pre-hooks.d
- post-hooks are located in: /var/rudder/configuration-repository/ncf/post-hooks.d
- pre-hooks are located in: `/var/rudder/configuration-repository/ncf/pre-hooks.d`
- post-hooks are located in: `/var/rudder/configuration-repository/ncf/post-hooks.d`
As of March 2015, we have two post-hooks defined and no pre-hooks:
* post.write_technique.commit.sh
* `post.write_technique.commit.sh`
** It commits the Technique newly created into Rudder Git configuration repository
located in /var/rudder/configuration-repository.
* post.write_technique.rudderify.sh
** It generates a valid Rudder Technique from a the newly created Technique and reloads Rudder
Technique Library so that updates are taken into account.
located in `/var/rudder/configuration-repository`.
* `post.write_technique.rudderify.sh`
** It generates a valid Rudder Technique from a the newly created Technique and reloads Rudder
Technique Library so that updates are taken into account.
If you want to run post hooks by hand, you can use the following command:
......
OK, now we've got that over with, let's go on.
A Technique should be an abstract configuration. This means that your Technique
shouldn't just configure something one way, but instead it should implement
shouldn't just configure something one way, but instead it should implement
*how* to configure something, and offer options for users to choose what way
they want it configured. Before starting, make sure you've thought through what
you want to create.

Also available in: Unified diff