https://issues.rudder.io/https://issues.rudder.io/themes/rudder7/favicon/favicon.ico?17096450182017-02-13T11:41:59ZIssue TrackerRudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=704842017-02-13T11:41:59ZFrançois ARMANDfrancois.armand@rudder.io
<ul></ul><p>Editing on the root server file: /opt/rudder/share/relay-api/relay_api/remote_run.py to add a "-i" to REMOTE_RUN_COMMAND && restarting apache, I'm now getting:</p>
<pre>
rudder info: ........................................................................
rudder info: Hailing server.rudder.local : 5309
rudder info: ........................................................................
error: TRUST FAILED, server presented untrusted key: MD5=3275d8e38205fada95e6236901099527
error: Failed to connect to host: server.rudder.local
error Rudder agent was interrupted during execution by a fatal error
## Summary #####################################################################
0 components verified in 0 directives
execution time: 0.30s
################################################################################
</pre>
<p>We should have an API option to allow to use that output.</p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=704862017-02-13T11:47:55ZFrançois ARMANDfrancois.armand@rudder.io
<ul></ul><p>And sometimes, I don't get anything at all:</p>
<pre>
== [fanf@luhman16] ==
% curl -k -H "X-API-Token: 5qsPVwcoa99sZfnSn3A6ive9Q7PMUzRx" -X POST 'https://192.168.44.2/rudder/api/latest/nodes/c867b070-0721-43d3-8825-d78c51c2c632/applyPolicy'
</pre> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=705192017-02-14T09:52:32ZFrançois ARMANDfrancois.armand@rudder.io
<ul><li><strong>Assignee</strong> set to <i>Benoît PECCATTE</i></li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=718702017-02-28T08:59:56ZFrançois ARMANDfrancois.armand@rudder.io
<ul><li><strong>Assignee</strong> changed from <i>Benoît PECCATTE</i> to <i>Nicolas CHARLES</i></li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=718772017-02-28T09:16:28ZFrançois ARMANDfrancois.armand@rudder.io
<ul><li><strong>Translation missing: en.field_tag_list</strong> set to <i>Blocking 4.1</i></li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=718962017-02-28T11:03:45ZNicolas CHARLESnicolas.charles@rudder.io
<ul></ul><p>Webapp log show following error:<br /><pre>
java.lang.NumberFormatException: For input string: "Error when trying to contact internal remote-run API: null"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Byte.parseByte(Byte.java:149)
at java.lang.Byte.parseByte(Byte.java:175)
at scala.collection.immutable.StringLike.toByte(StringLike.scala:297)
at scala.collection.immutable.StringLike.toByte$(StringLike.scala:297)
at scala.collection.immutable.StringOps.toByte(StringOps.scala:29)
at com.normation.rudder.web.rest.node.NodeApiService8.runResponse(NodeAPIService8.scala:119)
at com.normation.rudder.web.rest.node.NodeApiService8.$anonfun$runNode$4(NodeAPIService8.scala:155)
at com.normation.rudder.web.rest.node.NodeApiService8.$anonfun$runNode$4$adapted(NodeAPIService8.scala:155)
at net.liftweb.http.LiftServlet.sendResponse(LiftServlet.scala:1040)
at net.liftweb.http.LiftServlet.doService(LiftServlet.scala:451)
at net.liftweb.http.LiftServlet.$anonfun$service$2(LiftServlet.scala:157)
at net.liftweb.util.TimeHelpers.calcTime(TimeHelpers.scala:427)
...
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:369)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:464)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:913)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:975)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
</pre></p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719032017-02-28T12:03:25ZNicolas CHARLESnicolas.charles@rudder.io
<ul></ul><p>Ha, this message is probably more relevant:<br /><pre>
Feb 28 12:02:32 server cf-serverd[1583]: CFEngine(server) rudder 127.0.0.1> Connection was hung up while receiving line:
Feb 28 12:02:32 server cf-serverd[1583]: CFEngine(server) rudder 127.0.0.1> Client closed connection early! He probably does not trust our key...
</pre></p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719052017-02-28T12:04:32ZNicolas CHARLESnicolas.charles@rudder.io
<ul></ul><p>Full verbose is<br /><pre>
rudder verbose: Obtained IP address of '127.0.0.1' on socket 7 from accept
rudder verbose: New connection (from 127.0.0.1, sd 7), spawning new thread...
rudder info: 127.0.0.1> Accepting connection
rudder verbose: 127.0.0.1> Setting socket timeout to 600 seconds.
rudder verbose: 127.0.0.1> Peeked nothing important in TCP stream, considering the protocol as TLS
rudder verbose: 127.0.0.1> TLS version negotiated: TLSv1.2; Cipher: AES256-GCM-SHA384,TLSv1/SSLv3
rudder verbose: 127.0.0.1> TLS session established, checking trust...
rudder verbose: 127.0.0.1> Remote peer terminated TLS session (SSL_read)
error: 127.0.0.1> Connection was hung up while receiving line:
notice: 127.0.0.1> Client closed connection early! He probably does not trust our key..
</pre></p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719062017-02-28T12:08:39ZNicolas CHARLESnicolas.charles@rudder.io
<ul></ul><p>Agent side:<br /><pre>
rudder verbose: Connected to host 192.168.41.2 address 192.168.41.2 port 5309 (socket descriptor 4)
rudder verbose: TLS version negotiated: TLSv1.2; Cipher: AES256-GCM-SHA384,TLSv1/SSLv3
rudder verbose: TLS session established, checking trust...
rudder verbose: Did not find new key format '/var/rudder/cfengine-community/ppkeys/root-MD5=57ccba22df018012132877618ff655f9.pub'
rudder verbose: Trying old style '/var/rudder/cfengine-community/ppkeys/root-192.168.41.2.pub'
rudder verbose: Received key 'MD5=57ccba22df018012132877618ff655f9' not found in ppkeys
error: TRUST FAILED, server presented untrusted key: MD5=57ccba22df018012132877618ff655f9
rudder verbose: Connection to 192.168.41.2 is closed
error: Failed to connect to host: 192.168.41.2
</pre></p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719112017-02-28T12:39:07ZNicolas CHARLESnicolas.charles@rudder.io
<ul></ul>Ok, so after some tests:
<ol>
<li>we cannot remote run on itself ; cf-runagent doesn't seem to support it</li>
<li>remote running a 4.1 node is ok</li>
<li>remote running a 4.0 node fails, as command is not valid<br /><pre>
root@agent1:/home/vagrant# /opt/rudder/bin/rudder agent run -uR -I -Dcfruncommand
Rudder agent 4.0.4.rc1.git201702280322 (CFEngine Core 3.7.4)
Node uuid: e04cdc24-2180-4d2e-b334-0445a13a3a45
ok: Rudder agent promises were updated.
error: Remote execution cannot ignore locks
</pre></li>
<li>remote running a 3.1 node fails, as command /opt/rudder/bin/rudder agent run -uR -I -Dcfruncommand --inform is not valid<br /><pre>
root@agent2:/home/vagrant# /opt/rudder/bin/rudder agent run -uR -Dcfruncommand --inform
/opt/rudder/share/commands/agent-run : option non permise -- u
/opt/rudder/share/commands/agent-run : option non permise -- -
/opt/rudder/share/commands/agent-run : option non permise -- n
/opt/rudder/share/commands/agent-run : option non permise -- o
Rudder agent 3.1.19.rc1.git201702210714 (CFEngine Core 3.6.5)
Node uuid: 791a6ebe-cfb1-4f54-b9a2-48ca162f64b6
2017-02-28T12:38:05+0000 error: Remote execution cannot ignore locks
</pre></li>
</ol>
<p>So, a remote API should not try to remote run on local system, and we need a fix for 4.0 and 3.1 compatibility</p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719142017-02-28T13:43:07ZFrançois ARMANDfrancois.armand@rudder.io
<ul><li><strong>Translation missing: en.field_tag_list</strong> deleted (<del><i>Blocking 4.1</i></del>)</li><li><strong>Category</strong> set to <i>Relay server or API</i></li><li><strong>Assignee</strong> changed from <i>Nicolas CHARLES</i> to <i>Benoît PECCATTE</i></li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719152017-02-28T16:16:29ZFrançois ARMANDfrancois.armand@rudder.io
<ul></ul><p>I'm letting that ticket open to change Relay API and do the correct call to rudder agent. I'm opening a subticket to correct the null pointer exception on rudder side that should not happen.</p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719162017-02-28T16:17:27ZAlexis Moussetalexis.mousset@rudder.io
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In progress</i></li><li><strong>Assignee</strong> changed from <i>Benoît PECCATTE</i> to <i>Alexis Mousset</i></li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719172017-02-28T16:23:19ZAlexis Moussetalexis.mousset@rudder.io
<ul><li><strong>Status</strong> changed from <i>In progress</i> to <i>Pending technical review</i></li><li><strong>Assignee</strong> changed from <i>Alexis Mousset</i> to <i>Benoît PECCATTE</i></li><li><strong>Pull Request</strong> set to <i>https://github.com/Normation/rudder-packages/pull/1270</i></li></ul><p>PR <a class="external" href="https://github.com/Normation/rudder-packages/pull/1270">https://github.com/Normation/rudder-packages/pull/1270</a></p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719302017-03-01T09:43:50ZAlexis Moussetalexis.mousset@rudder.io
<ul><li><strong>Status</strong> changed from <i>Pending technical review</i> to <i>Pending release</i></li></ul><p>Applied in changeset <a class="changeset" title="Fixes #10185: Remote-run exec for root and nodes behind relays fail with \"rudder agent was inter..." href="https://issues.rudder.io/projects/rudder/repository/rudder-packages/revisions/b6ca5a9f94c4c33be1a2649e5489dbf9e1f53dff">rudder-packages|b6ca5a9f94c4c33be1a2649e5489dbf9e1f53dff</a>.</p> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=719602017-03-01T14:29:47ZNicolas CHARLESnicolas.charles@rudder.io
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-6 priority-16 priority-default closed" href="/issues/10314">User story #10314</a>: Document remote-run exec compatibility</i> added</li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=726672017-03-06T10:53:15ZFrançois ARMANDfrancois.armand@rudder.io
<ul><li><strong>Subject</strong> changed from <i>Remote-run exec for root and nodes behind relays fail with "rudder agent was interrupted"</i> to <i>Remote-run exec for root fail with "rudder agent was interrupted"</i></li></ul> Rudder - Bug #10185: Remote-run exec for root fail with "rudder agent was interrupted"https://issues.rudder.io/issues/10185?journal_id=728452017-03-07T15:28:47ZVincent MEMBRÉvme@rudder.io
<ul><li><strong>Status</strong> changed from <i>Pending release</i> to <i>Released</i></li></ul><p>This bug has been fixed in Rudder 4.1.0~rc1 which was released today.</p>
<ul>
<li>4.1.0~rc1: <a href="http://www.rudder-project.org/pipermail/rudder-announce/2017-March/thread.html000224.html" class="external">Announce</a> <a href="http://www.rudder-project.org/changelog-4.1" class="external">Changelog</a></li>
<li>Download: <a class="external" href="https://www.rudder-project.org/site/get-rudder/downloads/">https://www.rudder-project.org/site/get-rudder/downloads/</a></li>
</ul>