Project

General

Profile

Bug #11416

remote execution is cut at 30 secondes

Added by Nicolas CHARLES 12 months ago. Updated 7 months ago.

Status:
Released
Priority:
N/A
Category:
System techniques
Target version:
Target version (plugin):
Severity:
Major - prevents use of part of Rudder | no simple workaround
User visibility:
Operational - other Techniques | Technique editor | Rudder settings
Effort required:
Small
Priority:
63

Description

We do not define a default timeout for remote execution, and it cuts execution at execution time: 30.61s
Timeout is defined in https://docs.cfengine.com/docs/master/reference-components-cf-runagent.html, but if it is not defined, then default_timeout apply, which default to 30s (see https://docs.cfengine.com/docs/master/reference-components-cf-agent.html#default_timeout )

we should set a default timeout for remote execution

Associated revisions

Revision faf1c876 (diff)
Added by Nicolas CHARLES 9 months ago

Fixes #11416: remote execution is cut at 30 secondes

History

#1 Updated by Janos Mattyasovszky 12 months ago

Timeout manifests as "was interrupted" error from agent run:

policy_server # rudder remote run ruddernode
error    Rudder agent was interrupted during execution by a fatal error
         Run with -i to see log messages.

## Summary #####################################################################
0 components verified in 1 directives
execution time: 30.61s
################################################################################

#2 Updated by François ARMAND 12 months ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround
  • User visibility set to Operational - other Techniques | Technique editor | Rudder settings
  • Priority changed from 0 to 52

#3 Updated by Vincent MEMBRÉ 11 months ago

  • Target version changed from 4.1.8 to 4.1.9

#4 Updated by Benoît PECCATTE 11 months ago

  • Priority changed from 52 to 51

#5 Updated by Benoît PECCATTE 11 months ago

  • Effort required set to Small
  • Priority changed from 51 to 67

#6 Updated by Alexis MOUSSET 10 months ago

  • Priority changed from 67 to 66

In fact the documentation (which states it is a connection timeout) is wrong, it is applied as SO_RCVTIMEO on the socket, and hence closes the connection when we see no output for 30 seconds, which may happen in changes only mode for example.

#7 Updated by Vincent MEMBRÉ 10 months ago

  • Target version changed from 4.1.9 to 4.1.10

#8 Updated by Nicolas CHARLES 9 months ago

  • Priority changed from 66 to 65

Interestingly, it's the agent that closes the connection.
Agent side

   error: SSL_read: receive timeout
rudder  verbose: Connection to 192.168.42.3 is closed

then server side

rudder  verbose: 192.168.42.2> MD5=918083b6dad780baf4e93367aa1cfe7f: Client is TRUSTED, public key MATCHES stored one.
rudder  verbose: 192.168.42.2>      Received:    EXEC 
rudder     info: 192.168.42.2> cf-serverd executing cfruncommand: /opt/rudder/bin/rudder agent run -uR
   error: 192.168.42.2> SSL_write: underlying network error (Broken pipe)
rudder     info: 192.168.42.2> Sending failed, aborting EXEC (send: Broken pipe)
   error: 192.168.42.2> SSL_write: (-1 SSL_ERROR_SSL) bad write retry 
rudder  verbose: 192.168.42.2> Unable to reply with terminator. (send: Success)
rudder  verbose: 192.168.42.2> Remote peer terminated TLS session (SSL_read)
rudder     info: 192.168.42.2> Closing connection, terminating thread

I configured the timeout everywhere

#9 Updated by Nicolas CHARLES 9 months ago

simply setting a -t in the command line solves the issue

#10 Updated by Nicolas CHARLES 9 months ago

  • Status changed from New to In progress

#11 Updated by Nicolas CHARLES 9 months ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to Alexis MOUSSET
  • Pull Request set to https://github.com/Normation/rudder-agent/pull/146

#12 Updated by Nicolas CHARLES 9 months ago

  • Status changed from Pending technical review to Pending release

#13 Updated by Vincent MEMBRÉ 7 months ago

  • Status changed from Pending release to Released
  • Priority changed from 65 to 63

This bug has been fixed in Rudder 4.1.10 and 4.2.4 which were released today.

Also available in: Atom PDF