<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body smarttemplateinserted="true" bgcolor="#FFFFFF" text="#000000">
Hello everyone,<br>
<br>
When people have a lot of rules or a lots of nodes, they also have a
lot of generated promises.<br>
When you run a relay, those promises can take a lot of time to
synchronize.<br>
To avoid this we devised an rsync based with the help of some of our
best users.<br>
This can reduce synchronization time "a lot".<br>
<br>
But let's see how to do it.<br>
<br>
<u><b>How does it work ?</b></u><u><b><br>
</b></u>- The update promises will be changed to make it run an
rsync command instead of a cfengine download if a specific flag is
set<br>
- rsync will connect to the server via ssh on the rudder user<br>
- The key used to connect will be derived from the cfengine key
(openssl command to be defined)<br>
- The authorized keys will be filled by the webapp on the server
since it knows the public keys of each relai<br>
- A script on the server triggered by ssh remote_command will check
that the remote relay is allow to synchronize the directory it is
asking on each ssh connection<br>
- The relay will get its promises over rsync over ssh which can be
really fast<br>
<br>
<b><u>Prerequisite: the rudder user</u></b><b><br>
</b>First we need a specific rudder user on the server so that we do
not use root to connect to the rudder server.<br>
This in itself is a challenge because today we only have a rudder
group.<br>
<br>
To do this we need to:<br>
- make the package create a new "rudder" user with the "rudder"
group<br>
- change the group of the generated ncf and generated promises to
"rudder"<br>
- change the right of those directories to g+rs, this will make sure
that we will always have read access to those files from the rudder
user<br>
- change the post install script to change existing rights<br>
- test that this properly works with cf-serverd<br>
<br>
This should be sufficient for our current case, but it allows us to
imagine a day where jetty and cf-serverd could run as the "rudder"
user instead of "root".<br>
<br>
We would like to be able to synchronize the shared-files directory
too. But since this directory is under direct control of the user
and its access rights can be used in promises we won't touch its
sgid bit nor synchronize it by default. If the user want to
synchronize it via rsync he will need to make sure he has posix acl
activated on this directory, run a command to add the rudder group
via those acls and then activate the rsync protocol spécifically for
this directory.<br>
<br>
<u><b>Other steps</b></u><u><b><br>
</b></u>- Have the rudder user on the server and the rudder group
on the relays<br>
- Create a technique to manage the authorized_keys file on the
server (it will be based on a system variable that holds uuid/key
pairs from the webapp)<br>
- On the relay, create a technique (or maybe a postinstall script
would do better) that: <br>
* create the ssh key from the cfengine key and put it in a known
place<br>
* call ssh-keyscan to add the server's key to ~/.ssh/known_hosts<br>
- Create an rsync ACL script that takes 1 parameters (the UUID of a
host), it will:<br>
* know that it is running from ssh, so know the remote command
used<br>
* check that this command is authorized by its parameter
(share/<uuid> ncf and shared-files only are authorized)<br>
* <a class="moz-txt-link-freetext" href="https://www.samba.org/ftp/unpacked/rsync/support/rrsync">https://www.samba.org/ftp/unpacked/rsync/support/rrsync</a> can be a
good source of how to do this<br>
- Add a flag file in generated promises to avoid running rsync if
it's not necessary (maybe it can be implemented using a
filter-from-file option in rsync to avoid using cfengine again)<br>
- Add an option in the interface to configure synchronization via
rsync instead of regular synchronization<br>
- Add an option in the interface to synchronize shared-files too<br>
- Be ware that a relay can become a server for another relay so
everything on the server must work on another machine<br>
<br>
Note: <br>
During our meeting we talked about a user on the agent by i can't
remember why.<br>
<br>
<br>
If you see something wrong or if you want to add a comment, just hit
reply !<br>
<br>
Cheers,<br>
<div class="moz-signature">-- <br>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<style type="text/css">
<!--
a.redlink:link { color: #1782E6; }
a.redlink:visited { color: #1782E6; }
.sig { font-family: 'Century Gothic', CenturyGothic, AppleGothic, sans-serif; font-size: small; }
.sigsmall { font-family: 'Century Gothic', CenturyGothic, AppleGothic, sans-serif; font-size: x-small; }
-->
</style>
<table border="0" width="380" cellpadding="0" cellspacing="2">
<tbody>
<tr>
<td colspan="2">
<hr></td>
</tr>
<tr>
<td colspan="2"><b><img alt="Logo Normation"
src="cid:part1.01040506.00000600@normation.com"
align="left" height="50" hspace="10" width="50"> <span
class="sig">Benoît Peccatte</span></b><br>
<span class="sig"><i>Responsable cloud development</i></span><br>
<span class="sig"><a class="redlink"
href="http://www.normation.com">Normation</a></span> </td>
</tr>
<tr>
<td colspan="2">
<hr></td>
</tr>
<tr>
<td colspan="2"><span class="sigsmall"><b>87, Rue de
Turbigo, 75003 Paris, France</b></span></td>
</tr>
<tr>
<td><span class="sigsmall">Phone:</span></td>
<td><span class="sigsmall">+33 (0)1 85 08 48 96</span></td>
</tr>
<tr>
<td colspan="2">
<hr> </td>
</tr>
</tbody>
</table>
</div>
<span class="st4cursor"></span>
</body>
</html>