<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Le 05/11/2016 à 22:30, Janos
Mattyasovszky a écrit :<br>
</div>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<div>Hi @all,<br>
</div>
<div><br>
</div>
<div>I find the idea very mind-stimulating, I cannot even think
about the possibilities this will enable ;-] <br>
</div>
<div>I had some question / remarks:<br>
</div>
<blockquote type="cite" class="protonmail_quote" style="padding:
0px 0px 0px 1rem; margin: 0px; border-left: 4px solid rgb(229,
229, 229); color: rgb(34, 34, 34); font-family: Arial,
"Helvetica Neue", Helvetica, sans-serif; font-size:
14px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: normal; letter-spacing:
normal; orphans: 2; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 2;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<p>1. Remote-run:<br>
</p>
</blockquote>
<ul>
<li>
<div>As far I have understood, this remote-run api will be
available on the relays themselves but only accessible from
their policy server, right? The root server will then
provide the capability to run the agent on the given Node
(by UUID) by propagating the task to the relay the node(s)
reside on and "proxy" the action?<br>
</div>
</li>
</ul>
</blockquote>
Yes.<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>
<div>Will it behave like the regular api-system with
one-token-all-access, or is there the possibility to somehow
restrict actions, like maybe assign a token to a group
and/or limit the cfengine class to be set? This would allow
to delegate defined actions to "external" parties, who
should not directly have access to the Rudder UI, but like
they are responsible for a group of nodes, and I can
delegate the task of patching the system and reboot it
afterwards by giving them an API and a Key.<br>
</div>
</li>
</ul>
</blockquote>
The rudder server API will be token based like all other ones, which
means it will have the same limits.<br>
Having better right management on them is a goal that will be done
in another task.<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>
<div>On the "async"-kind of run: have you thought about
creating a "Task-ID", so when you trigger the action, get a
token for that run, and then can poll the action with some
delay for it's outcome (return code) and the output? This
would make longer-running tasks (like patching systems)
possible without having to think about http timeouts and
similar stuff...<br>
</div>
</li>
</ul>
</blockquote>
It comes with some new problems too. Where, for how long and for who
do we keep the output ?<br>
Timeout are not really a problem, you can easily have long running
http connections, it is a matter of proper configuration.<br>
I understand that can be a questioning for people having a proxy
with timeout in them but i don't expect agent runs to last longer
than a http timeout.<br>
<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<blockquote type="cite" class="protonmail_quote" style="padding:
0px 0px 0px 1rem; margin: 0px; border-left: 4px solid rgb(229,
229, 229); color: rgb(34, 34, 34); font-family: Arial,
"Helvetica Neue", Helvetica, sans-serif; font-size:
14px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: normal; letter-spacing:
normal; orphans: 2; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 2;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<p>2. Share Files<br>
</p>
</blockquote>
<div>Some suggestions:<br>
</div>
<ul>
<li>
<div>A good idea is to maybe establish the ability to limit /
turn off this behavior on given relays, like in a DMZ, to
limit the possible attach vectors on intrusions.<br>
</div>
</li>
</ul>
</blockquote>
That is a good idea, how would you expect it to be configured ?<br>
A per relay setting like there is a per relay allowed network ?<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<div>Some questions I still have, I could not completely follow
the logic of sharing Node-Alice sharing a file with Node-Bob: <br>
</div>
<ul>
<li>Does the root server propagate the forwarded files to
downstream relays ? If example the topology is N1 ==> R1
==> Root <== R2 <== N2, and N1 wants to share with
N2, then R1 forwards to root, but root then needs to give R2
the file to make if available for N2...</li>
</ul>
</blockquote>
Indeed, and it works because the root server know how to make a file
available for N2. Like withe the share directory, the structure is a
tree with a directoryfor N2 below the directory for R2. So R2 will
download its shared-files-nodes directory as if it was for itself
and put it in the same place where N2 will download it.<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>
<div>The signature of the File is there to authenticate the
node? Is this done by signing the file with the private key
of the node to generate the hash?<br>
</div>
</li>
</ul>
</blockquote>
Yes<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>
<div>Where does the "file_id" come from? Is that the hash of
the file, or something Node-A determines to be used as ID?<br>
</div>
</li>
</ul>
</blockquote>
It is a free identifier provided by the user of ncf. The goal is to
make it easy to share data just by knowing a common identifier on
both sides.<br>
So you share with "my_own_id" on node A and you download "my_own_id"
on node B.<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>What happens if the Server return 404 for an uploaded file?
Best would be if then the relay would discard that file from
being retried, as otherwise you'd be hammering with uploads
until someone removes that by hand.<br>
</li>
</ul>
</blockquote>
Yes. The relay will only retry if there is a network error or a
server error (5xx).<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>Will there be a DELETE behavior to delete files? Would
really make sense. Who should be able to delete those files?
The sender, or also the receiver?<br>
</li>
</ul>
</blockquote>
There is not yet, but there is a TTL argument to the
sharedfile_to_node to make sure a file doesn't stay here forever.<br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<ul>
<li>Do you plan to implement also anything to make these uploads
/ shared files visible in the GUI? Possibly also with a Delete
button for the files? That could be resolved with remote agent
runs on the relay by setting a delete_file_<hash> class
that would result in that particular file being removed from
the relay. It only gets tricky when you have chained up
multiple relays :-)<br>
</li>
</ul>
</blockquote>
The plan is currently to enable communication between nodes, so we
take th shortest path and we don't push up to the server if we don't
need to.<br>
But it will be possible possible one day to transmit info to the
server if it seems useful. <br>
<br>
<blockquote
cite="mid:RbHpNPrF2gHQMFDolkLlqoq72VYO_kcb9kYbIMlcVWKKyfjchcA4f_kAoSuC869688SZfSRdnkAcD-ZOt3ZLgKdkaYHH61TXdONH6CDOzoI=@matya.hu"
type="cite">
<div>So much for the first round of thinking, later things might
follow.<br>
</div>
<div><br>
</div>
<div>Have a nice sunday,<br>
</div>
<div>Matya</div>
<div><br>
</div>
<blockquote class="protonmail_quote" type="cite">
<div>-------- Original Message --------<br>
</div>
<div>Subject: [rudder-dev] Relay API<br>
</div>
<div>Local Time: 4. November 2016 5:38 PM<br>
</div>
<div>UTC Time: 4. November 2016 16:38<br>
</div>
<div>From: <a class="moz-txt-link-abbreviated" href="mailto:benoit.peccatte@normation.com">benoit.peccatte@normation.com</a><br>
</div>
<div>To: <a class="moz-txt-link-abbreviated" href="mailto:rudder-dev@lists.rudder-project.org">rudder-dev@lists.rudder-project.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:rudder-dev@lists.rudder-project.org"><rudder-dev@lists.rudder-project.org></a><br>
</div>
<div><br>
</div>
<div> <br>
</div>
<p>Hello,<br>
</p>
<p>One of Rudder 4.1 new features will be relay APIs. This is
the first attempt to describe it.<br>
</p>
<p><br>
</p>
<div>There are currently 2 API entries we want to add:
remote-run and share-files.<br>
</div>
<div> They will both be under /rudder/relay-api itself under <a
moz-do-not-send="true" href="https://"
class="moz-txt-link-freetext">https://</a><server>:<port>/
like the current api is.<br>
</div>
<div> /rudder is the common root for all rudder service<br>
</div>
<div> /relay-api is different from existing api to avoid
conflicts with them when it will be installed on the server<br>
</div>
<p><br>
</p>
<p><br>
</p>
<p>1. Remote-run:<br>
</p>
<div>The goal is to make a given relay call "rudder remote run"
on one of its attached node<br>
</div>
<div> The API will be under /rudder/relay-api/remote-run <br>
</div>
<div> GET remote-run/node/<node-uuid><br>
</div>
<div> GET remote-run/all<br>
</div>
<div> GET remote-run/nodes<br>
</div>
<div> <br>
</div>
<div> Parameters:<br>
</div>
<div> - output = keep / discard : to keep the output of the
remote-run call or discard its content<br>
</div>
<div> - async = yes / no : yes to ignore the return code of the
call and return immediately, no to wait until the end of the
call and get the return code<br>
</div>
<div> - classes = XXX : list of cfengine classes to set during
the remote call<br>
</div>
<div> - nodes = uuid,... : list of uuid to call in the "/nodes"
case<br>
</div>
<div> <br>
</div>
<div> Behavior:<br>
</div>
<div> - Loop on all nodes<br>
</div>
<div> - Find its hostname from its uuid in a matching file
created by promise generation on the server<br>
</div>
<div> - The call is descending, so we don't care about host that
do not exist<br>
</div>
<div> - The call is descending, so we will only accept calls
from the policy server<br>
</div>
<div> - Call rudder remote<br>
</div>
<div> - prefix the remote output lines with <uuid>: to
make sure the caller can parse output during async call on
multiple nodes<br>
</div>
<div> - surround the output with json format lines and include
return code, duration and stderr (-> we should escape the
output for use within a json string)<br>
</div>
<div> <br>
</div>
<div> <br>
</div>
<div> <br>
</div>
<p>2. Share Files<br>
</p>
<p><br>
</p>
<div>The goal is to share files between an agent and another one
via their policy server.<br>
</div>
<div> The API will be under /rudder/relay-api/shared-files <br>
</div>
<div> POST
shared-files/node/<target_uuid>/<file_id> (share
a file content with the target uuid using a given file id)<br>
</div>
<div> HEAD
shared-files/node/<target_uuid>/<file_id> (ask if
a file with the given file id is already shared with the given
target id)<br>
</div>
<p><br>
</p>
<p><br>
</p>
<div>Parameters:<br>
</div>
<div> - source-uuid: the uuid of the node sharing a file with
the target uuid<br>
</div>
<div> - signature: the file's signature in case of a POST, to
authenticate the source node<br>
</div>
<div> - hash: the file's hash in case of a HEAD to know is the
version already present matches<br>
</div>
<p><br>
</p>
<div>POST Behavior:<br>
</div>
<div> - if the target is known by the local relay<br>
</div>
<div> -> validate the signature<br>
</div>
<div> -> store the file in
/var/rudder/shared-files-nodes/[..<relay-uuid>..]/<target_uuid>/<source_uuid>/<file_id><br>
</div>
<div> -> store the metadata (including, date, hash and
signature) in <the same path>.medata<br>
</div>
<div> - if the target in not known and we are not the root
server<br>
</div>
<div> -> store the file in a temporary directory<br>
</div>
<div> -> try to send the file to the relay server<br>
</div>
<div> -> do nt remove the file and try again as long as
there is a fatal error (code >= 500, network error)<br>
</div>
<div> - if the target in not known and we are the root server<br>
</div>
<div> -> ignore the file and return 404<br>
</div>
<div> <br>
</div>
<div> HEAD behavior:<br>
</div>
<div> - If the file exists in /var/rudder/shared-files-nodes...
with the same hash, return 200<br>
</div>
<div> - If it doesn't, return 404<br>
</div>
<div> <br>
</div>
<div> 3. Using this API from ncf<br>
</div>
<div> We will create 2 new generic methods in ncf:<br>
</div>
<div> - sharedfile_to_node(target_uuid, file_id, file_path,
ttl) where ttl is infinite by default<br>
</div>
<div> - sharedfile_from_node(source_uuid, file_id, file_path) <br>
</div>
<div> <br>
</div>
<div> The first one will call HEAD on the shared-files API and
if is gets a 404, call POST to send the content.<br>
</div>
<div> <br>
</div>
<div> The second one will just download the file using regular
cfengine protocol.<br>
</div>
<div> <br>
</div>
<div> <br>
</div>
<div> <br>
</div>
<div> That's all folks !<br>
</div>
<div> Any comment ?<br>
</div>
<div> <br>
</div>
<div class="moz-signature">
<div>-- <br>
</div>
<div> <br>
</div>
<table border="0" cellpadding="0" cellspacing="2" width="380">
<tbody>
<tr>
<td colspan="2">
<hr><br>
</td>
</tr>
<tr>
<td colspan="2">
<div><img moz-do-not-send="true"
class="proton-embedded"
data-embedded-img="part1.605096A1.65F1782B@normation.com"
src="blob:https://mail.protonmail.com/820670cb-c264-48f6-95aa-57e01fc17cac"
alt="Logo Normation" align="left" height="50"
width="50"> Benoît Peccatte<br>
</div>
<div> Architecte<br>
</div>
<div> <a moz-do-not-send="true"
href="http://www.normation.com" class="redlink">Normation</a><br>
</div>
</td>
</tr>
<tr>
<td colspan="2">
<hr><br>
</td>
</tr>
<tr>
<td colspan="2">87, Rue de Turbigo, 75003 Paris, France<br>
</td>
</tr>
<tr>
<td>Phone:<br>
</td>
<td>+33 (0)1 85 08 48 96<br>
</td>
</tr>
<tr>
<td colspan="2">
<hr><br>
</td>
</tr>
</tbody>
</table>
</div>
</blockquote>
<div><br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
rudder-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:rudder-dev@lists.rudder-project.org">rudder-dev@lists.rudder-project.org</a>
<a class="moz-txt-link-freetext" href="http://www.rudder-project.org/mailman/listinfo/rudder-dev">http://www.rudder-project.org/mailman/listinfo/rudder-dev</a>
</pre>
</blockquote>
<br>
<p><br>
</p>
<div class="moz-signature">-- <br>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<style type="text/css">
<!--
a.redlink:link { color: #1782E6; }
a.redlink:visited { color: #1782E6; }
.sig { font-family: 'Century Gothic', CenturyGothic, AppleGothic, sans-serif; font-size: small; }
.sigsmall { font-family: 'Century Gothic', CenturyGothic, AppleGothic, sans-serif; font-size: x-small; }
-->
</style>
<table border="0" cellpadding="0" cellspacing="2" width="380">
<tbody>
<tr>
<td colspan="2">
<hr></td>
</tr>
<tr>
<td colspan="2"><img alt="Logo Normation"
src="cid:part4.D9887D73.539BE58F@normation.com"
align="left" height="50" hspace="10" width="50"> Benoît
Peccatte<br>
Architecte<br>
<a class="redlink" href="http://www.normation.com">Normation</a>
</td>
</tr>
<tr>
<td colspan="2">
<hr></td>
</tr>
<tr>
<td colspan="2">87, Rue de Turbigo, 75003 Paris, France</td>
</tr>
<tr>
<td>Phone:</td>
<td>+33 (0)1 85 08 48 96</td>
</tr>
<tr>
<td colspan="2">
<hr> </td>
</tr>
</tbody>
</table>
</div>
</body>
</html>