[rudder-dev] Relay API

Sat Nov 5 22:30:36 CET 2016

Hi @all,

I find the idea very mind-stimulating, I cannot even think about the possibilities this will enable ;-]
I had some question / remarks:

1. Remote-run:

-
As far I have understood, this remote-run api will be available on the relays themselves but only accessible from their policy server, right? The root server will then provide the capability to run the agent on the given Node (by UUID) by propagating the task to the relay the node(s) reside on and "proxy" the action?
-
Will it behave like the regular api-system with one-token-all-access, or is there the possibility to somehow restrict actions, like maybe assign a token to a group and/or limit the cfengine class to be set? This would allow to delegate defined actions to "external" parties, who should not directly have access to the Rudder UI, but like they are responsible for a group of nodes, and I can delegate the task of patching the system and reboot it afterwards by giving them an API and a Key.
-
On the "async"-kind of run: have you thought about creating a "Task-ID", so when you trigger the action, get a token for that run, and then can poll the action with some delay for it's outcome (return code) and the output? This would make longer-running tasks (like patching systems) possible without having to think about http timeouts and similar stuff...

2. Share Files

Some suggestions:

-
A good idea is to maybe establish the ability to limit / turn off this behavior on given relays, like in a DMZ, to limit the possible attach vectors on intrusions.

Some questions I still have, I could not completely follow the logic of sharing Node-Alice sharing a file with Node-Bob:

- Does the root server propagate the forwarded files to downstream relays ? If example the topology is N1 ==> R1 ==> Root <== R2 <== N2, and N1 wants to share with N2, then R1 forwards to root, but root then needs to give R2 the file to make if available for N2...
-
The signature of the File is there to authenticate the node? Is this done by signing the file with the private key of the node to generate the hash?
-
Where does the "file_id" come from? Is that the hash of the file, or something Node-A determines to be used as ID?
- What happens if the Server return 404 for an uploaded file? Best would be if then the relay would discard that file from being retried, as otherwise you'd be hammering with uploads until someone removes that by hand.

- Will there be a DELETE behavior to delete files? Would really make sense. Who should be able to delete those files? The sender, or also the receiver?

- Do you plan to implement also anything to make these uploads / shared files visible in the GUI? Possibly also with a Delete button for the files? That could be resolved with remote agent runs on the relay by setting a delete_file_<hash> class that would result in that particular file being removed from the relay. It only gets tricky when you have chained up multiple relays :-)

So much for the first round of thinking, later things might follow.

Have a nice sunday,
Matya

-------- Original Message --------
Subject: [rudder-dev] Relay API
Local Time: 4. November 2016 5:38 PM
UTC Time: 4. November 2016 16:38
From: benoit.peccatte at normation.com
To: rudder-dev at lists.rudder-project.org <rudder-dev at lists.rudder-project.org>

Hello,

One of Rudder 4.1 new features will be relay APIs. This is the first attempt to describe it.

There are currently 2 API entries we want to add: remote-run and share-files.
They will both be under /rudder/relay-api itself under https://<server>:<port>/ like the current api is.
/rudder is the common root for all rudder service
/relay-api is different from existing api to avoid conflicts with them when it will be installed on the server

1. Remote-run:

The goal is to make a given relay call "rudder remote run" on one of its attached node
The API will be under /rudder/relay-api/remote-run
GET remote-run/node/<node-uuid>
GET remote-run/all
GET remote-run/nodes

Parameters:
- output = keep / discard : to keep the output of the remote-run call or discard its content
- async = yes / no : yes to ignore the return code of the call and return immediately, no to wait until the end of the call and get the return code
- classes = XXX : list of cfengine classes to set during the remote call
- nodes = uuid,... : list of uuid to call in the "/nodes" case

Behavior:
- Loop on all nodes
- Find its hostname from its uuid in a matching file created by promise generation on the server
- The call is descending, so we don't care about host that do not exist
- The call is descending, so we will only accept calls from the policy server
- Call rudder remote
- prefix the remote output lines with <uuid>: to make sure the caller can parse output during async call on multiple nodes
- surround the output with json format lines and include return code, duration and stderr (-> we should escape the output for use within a json string)

2. Share Files

The goal is to share files between an agent and another one via their policy server.
The API will be under /rudder/relay-api/shared-files
POST shared-files/node/<target_uuid>/<file_id> (share a file content with the target uuid using a given file id)
HEAD shared-files/node/<target_uuid>/<file_id> (ask if a file with the given file id is already shared with the given target id)

Parameters:
- source-uuid: the uuid of the node sharing a file with the target uuid
- signature: the file's signature in case of a POST, to authenticate the source node
- hash: the file's hash in case of a HEAD to know is the version already present matches

POST Behavior:
- if the target is known by the local relay
-> validate the signature
-> store the file in /var/rudder/shared-files-nodes/[..<relay-uuid>..]/<target_uuid>/<source_uuid>/<file_id>
-> store the metadata (including, date, hash and signature) in <the same path>.medata
- if the target in not known and we are not the root server
-> store the file in a temporary directory
-> try to send the file to the relay server
-> do nt remove the file and try again as long as there is a fatal error (code >= 500, network error)
- if the target in not known and we are the root server
-> ignore the file and return 404

HEAD behavior:
- If the file exists in /var/rudder/shared-files-nodes... with the same hash, return 200
- If it doesn't, return 404

3. Using this API from ncf
We will create 2 new generic methods in ncf:
- sharedfile_to_node(target_uuid, file_id, file_path, ttl) where ttl is infinite by default
- sharedfile_from_node(source_uuid, file_id, file_path)

The first one will call HEAD on the shared-files API and if is gets a 404, call POST to send the content.

The second one will just download the file using regular cfengine protocol.

That's all folks !
Any comment ?

--

------

Logo NormationBenoît Peccatte
Architecte
[Normation](http://www.normation.com)
------

87, Rue de Turbigo, 75003 Paris, France

Phone:
+33 (0)1 85 08 48 96

------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20161105/5d96d1a7/attachment.html>