<div dir="ltr">Hi<div><br></div><div>Answers inline. </div><div><br></div><div>Janos</div><div><br><br><div class="gmail_quote"><div dir="ltr">Benoit Peccatte <<a href="mailto:benoit.peccatte@normation.com" target="_blank">benoit.peccatte@normation.com</a>> ezt írta (időpont: 2016. jún. 14., K 12:26):<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hello everyone,<br>
<br>
When people have a lot of rules or a lots of nodes, they also have a
lot of generated promises.<br>
When you run a relay, those promises can take a lot of time to
synchronize.<br>
To avoid this we devised an rsync based with the help of some of our
best users.<br>
This can reduce synchronization time "a lot".<br>
<br>
But let's see how to do it.<br>
<br>
<u><b>How does it work ?</b></u><u><b><br>
</b></u>- The update promises will be changed to make it run an
rsync command instead of a cfengine download if a specific flag is
set<br>
- rsync will connect to the server via ssh on the rudder user<br></div></blockquote></div></div><div></div><div><br></div><div>I can only suggest to spawn an own instance of the os-provided sshd binary with a custom sshd_config file, to make sure you have full control over the settings. </div><div>I can imagine people tend to customize and secure their sshd config with additional restrictions, since rudder has full control of everything what is enrolled to it, so heavy restrictions might be in place that could conflict with your purpose. </div><div><br></div><div>I could imagine to have a custom config that would consist of something like this:</div><div><br></div><div><div><font face="monospace">Port 60022</font></div><div><font face="monospace">PidFile /var/rudder/run/sshd.pid</font></div><div><font face="monospace">HostKey /opt/rudder/etc/ssh/host_key_*</font></div><div><font face="monospace">Protocol 2</font></div><div><font face="monospace">AllowUsers rudder </font></div><div><font face="monospace">PermitRootLogin no</font></div><div><font face="monospace">UsePAM no</font></div><div><font face="monospace">PasswordAuthentication no</font></div><div><font face="monospace">ChallengeResponseAuthentication no</font></div><div><font face="monospace">PermitEmptyPasswords no</font></div><div><font face="monospace">AllowTcpForwarding no</font></div><div><font face="monospace">AllowAgentForwarding no</font></div><div><font face="monospace">PermitTunnel no</font></div><div><font face="monospace">UsePrivilegeSeparation no</font></div><div><font face="monospace">X11Forwarding no</font></div><div><font face="monospace">PrintMotd no</font></div></div><div><br></div><div>Something like this would make sure you secure your policy server (root or relay) from those where you want to implement the ssh login from.</div><div>This could then create a rudder-sshd init script, that would control your own sshd instance, and you could let it directly run with the rudder user, since you would actually not require root privileges to bind to a high port and you'd also not need to go through the PAM stack, when you only want key-based auth...</div><div><br></div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
- The key used to connect will be derived from the cfengine key
(openssl command to be defined)<br></div></blockquote></div></div><div><br></div><div>It would make sense to use 4k keys (at least for new nodes), since you are going to use them for ssh key derivation. Small change, big gain for future security. </div><div>See <a href="https://www.rudder-project.org/redmine/issues/6253" target="_blank">https://www.rudder-project.org/redmine/issues/6253</a></div><div><br></div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
- The authorized keys will be filled by the webapp on the server
since it knows the public keys of each relai<br>
- A script on the server triggered by ssh remote_command will check
that the remote relay is allow to synchronize the directory it is
asking on each ssh connection</div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
- The relay will get its promises over rsync over ssh which can be
really fast<br>
<br>
<b><u>Prerequisite: the rudder user</u></b><b><br>
</b>First we need a specific rudder user on the server so that we do
not use root to connect to the rudder server.<br>
This in itself is a challenge because today we only have a rudder
group.<br>
<br>
To do this we need to:<br>
- make the package create a new "rudder" user with the "rudder"
group<br>
- change the group of the generated ncf and generated promises to
"rudder"<br>
- change the right of those directories to g+rs, this will make sure
that we will always have read access to those files from the rudder
user<br>
- change the post install script to change existing rights<br>
- test that this properly works with cf-serverd<br>
<br>
This should be sufficient for our current case, but it allows us to
imagine a day where jetty and cf-serverd could run as the "rudder"
user instead of "root".<br></div></blockquote><div><br></div><div>This would actually make sense since AFAIK there is no real need to have them being run as root, no high ports, and FS-Perms can be adjusted to be rudder:rudder...</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<br>
We would like to be able to synchronize the shared-files directory
too. But since this directory is under direct control of the user
and its access rights can be used in promises we won't touch its
sgid bit nor synchronize it by default. If the user want to
synchronize it via rsync he will need to make sure he has posix acl
activated on this directory, run a command to add the rudder group
via those acls and then activate the rsync protocol spécifically for
this directory.<br>
<br>
<u><b>Other steps</b></u><u><b><br>
</b></u>- Have the rudder user on the server and the rudder group
on the relays<br>
- Create a technique to manage the authorized_keys file on the
server (it will be based on a system variable that holds uuid/key
pairs from the webapp)<br>
- On the relay, create a technique (or maybe a postinstall script
would do better) that: <br>
* create the ssh key from the cfengine key and put it in a known
place<br>
* call ssh-keyscan to add the server's key to ~/.ssh/known_hosts<br></div></blockquote><div><br></div><div>If you manage your own sshd, you know about your hostkeys. You can also pre-generate the known_hosts, since you know which policy-server a relay has to connect to (from which it is receiving the base policy to set up the rsync method anyway, so rather trust what is coming than scanning it over network) </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
- Create an rsync ACL script that takes 1 parameters (the UUID of a
host), it will:<br>
* know that it is running from ssh, so know the remote command
used<br>
* check that this command is authorized by its parameter
(share/<uuid> ncf and shared-files only are authorized)<br>
* <a href="https://www.samba.org/ftp/unpacked/rsync/support/rrsync" target="_blank">https://www.samba.org/ftp/unpacked/rsync/support/rrsync</a> can be a
good source of how to do this<br></div></blockquote><div><br></div><div>Since you are highjacking the rsync command being run over ssh with a forced command, you actually don't have to use the rsync command in the format of "<b>rsync server.fqdn:/path/to/what/I/want</b>", but you can use directly "<b>rsync server.fqdn:POLICY /target/dir</b>", "<b>rsync server.fqdn:NCF /target/dir</b>" and "<b>rsync server.fqdn:SHARED /target/dir</b>", and you could have the forced-command wrapper script translate them to the appropriate directory. </div><div><br></div><div>This would also simplify the generation of the rsync-command, since the only variable you need to add to it is the server.fqdn, as the other directories would be resolved on the policy server to the right directory.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
- Add a flag file in generated promises to avoid running rsync if
it's not necessary (maybe it can be implemented using a
filter-from-file option in rsync to avoid using cfengine again)<br></div></blockquote><div><br></div><div>The overhead is negligible, not sure it is worth checking for it, unless you want to use a flag-file to disable sync for some dedicated relays (rudder relay disable $uuid?) which you could put in maintenance mode or if you want to make sure an upgrade / risky rule is not going to break the nodes connected to a prod-relay, and would give you a low-level method to exclude a subtree of nodes from receiving an updated set of policy.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
- Add an option in the interface to configure synchronization via
rsync instead of regular synchronization<br>
- Add an option in the interface to synchronize shared-files too<br>
- Be ware that a relay can become a server for another relay so
everything on the server must work on another machine<br>
<br>
Note: <br>
During our meeting we talked about a user on the agent by i can't
remember why.<br>
<br>
<br>
If you see something wrong or if you want to add a comment, just hit
reply !<br><div><br></div></div>
</blockquote></div></div></div>