Project

General

Profile

Actions

Architecture #11906

closed

rsyslog handling of received log on rudder server is inefficient

Added by Nicolas CHARLES about 6 years ago. Updated about 6 years ago.

Status:
Rejected
Priority:
N/A
Assignee:
-
Category:
Performance and scalability
Target version:
Effort required:
Name check:
Fix check:
Regression:

Description

On the Rudder server, each time rsyslog receives a message, it writes it in the database, doing a transaction for only one log message
This is highly inefficient, both for rsyslog and database, and may lead to massive slowdown of infra if rsyslog is used in tcp

Some pointer on how to do it there:
http://www.rsyslog.com/doc/v8-stable/concepts/queues.html
http://www.rsyslog.com/doc/v8-stable/rainerscript/queue_parameters.html
http://rsyslog-users.1305293.n2.nabble.com/Rsyslog-Disk-Queue-Flush-Issue-td7580422.html

Open questions are:
  • how can we force the message to get fast to the database, for it to be handled by Rudder and not lag there for 1 hours if there are few nodes ?
  • memory usage of storing in memory logs ?
Actions #1

Updated by Nicolas CHARLES about 6 years ago

apparently, we cannot do multiple insert in database, see https://github.com/rsyslog/rsyslog/issues/399

but we can still use a queue to handle burst of data that might appear

Actions #2

Updated by Nicolas CHARLES about 6 years ago

transaction were implemented in https://github.com/rsyslog/rsyslog/pull/2172

Actions #3

Updated by Vincent MEMBRÉ about 6 years ago

  • Target version changed from 4.3.0~beta1 to 4.3.0~rc1
Actions #4

Updated by Vincent MEMBRÉ about 6 years ago

  • Target version changed from 4.3.0~rc1 to 4.3.0~rc2
Actions #5

Updated by François ARMAND about 6 years ago

This one seems risky (big potential impact) and syslog will disappear in next major. Should we not close it?

Actions #6

Updated by Nicolas CHARLES about 6 years ago

  • Status changed from New to Rejected

Yes, let's close it - the timeframe before new rsyslog-psql will become available in maintained distribution is after we'll stop using syslog

Actions

Also available in: Atom PDF