Project

General

Profile

Bug #6405

Server roles overwritten regularly for unknown reason

Added by Lionel Le Folgoc over 3 years ago. Updated about 2 months ago.

Status:
New
Priority:
1
Assignee:
-
Category:
Server components
Target version:
Target version (plugin):
Severity:
Minor - inconvenience | misleading | easy workaround
User visibility:
Operational - other Techniques | Technique editor | Rudder settings
Effort required:
Pull Request:
Priority:
28

Description

Hi,

In a multiserver setup, the server roles are overwritten with data loss at some regular (?) interval.

Here's an excerpt of the script I use to init the rudderweb instance, based on the one from rudder-tools git repo:

## Inform Rudder about the various roles so installation can proceed successfully
mkdir -p /var/rudder/cfengine-community/inputs
cat >/var/rudder/cfengine-community/inputs/rudder-server-roles.conf <<EOF
rudder-ldap:rudderldap01
rudder-inventory-endpoint:rudderldap01
rudder-db:rudderdb
rudder-front:rudderrelaytop01
rudder-relay-top:rudderrelaytop01
rudder-webapp:rudderweb
rudder-web:rudderweb
EOF

## Install components
yum install -y rudder-agent rudder-webapp rudder-techniques rsyslog rsyslog-pgsql httpd httpd-tools

## Give more RAM to jetty
#<snip>

## Init DB (/opt/rudder/etc/postgresql/reportsSchema.sql from rudder-reports-3.0.2.release-1.EL.6.noarch)
#<snip, this performs the init our rds instance, set the proper psql password in the conf, etc>

## Initialize the server: hostname, demodata?, ldapreset?, initialpromisesreset?, allowednet
sed -i '/^Pause$/d' /opt/rudder/bin/rudder-init # ensure that it won't hang waiting for user input
/opt/rudder/bin/rudder-init yes <our networks>

After that, rudder seems to work fine for some time, at least the UI. But at some point we can't get any report.

Indeed, because /etc/rsyslog.d/rudder.conf contains:

:ompgsql:localhost,rudder,rudder,password;RudderReportsFormat

localhost instead of rudderdb!
rudder.jdbc.* keys in /opt/rudder/etc/rudder-web-properties have the proper values.

Now, if I look at /var/rudder/cfengine-community/inputs/rudder-server-roles.conf, it has been overwritten by something:

# This file contains, for each role, the list of nodes that have this role
# These roles are auto-detected by default, based on inventories of nodes that have role files in /opt/rudder/etc/server-roles.d/
# Format is the following
# role:hostname1,hostname2,hostname3
rudder-ldap:rudderldap01
rudder-inventory-endpoint:rudderldap01
rudder-relay-top:rudderrelaytop01

rudderdb is gone!

If I put it back and run cf-agent, then the rsyslog config is ok.

However, it's somehow overwritten again at some point, because I restored the proper value yesterday evening, and this morning it's broken again.

Thanks.


Related issues

Related to Rudder - User story #6411: Documentation doesn't explain how to configure Rudder in case of splitted arch, when the DB is on Amazon RDSRejected
Related to Rudder - Architecture #8463: Server roles is a BIG mess causing inconsistencies and bugsNew

History

#1 Updated by Nicolas CHARLES over 3 years ago

  • Category set to Server components
  • Status changed from New to Discussion
  • Assignee set to Lionel Le Folgoc

Hello Lionel,

I have several questions for you:
- which version of Rudder do you use ?
- in the file /opt/rudder/etc/rudder-web-properties, you can define roles there. Which role did you define ?
- do you have the package rudder-db installed on the serveur with the web interface ? if so, that may be the cause of your issue, as a new inventory is generated and reconfigure the roles

#2 Updated by Lionel Le Folgoc over 3 years ago

  • Category deleted (Server components)
  • Status changed from Discussion to New
  • Assignee deleted (Lionel Le Folgoc)

Hi,

- Rudder 3.0.2

- See https://github.com/Normation/rudder-tools/blob/master/scripts/rudder-multiserver-setup/rudder-web.sh#L10 and http://www.rudder-project.org/rudder-doc-3.0/rudder-doc.html#multiserver-rudder, there's no mention of this file to set the roles...

- Nope: rudder-agent rudder-webapp rudder-techniques rsyslog rsyslog-pgsql httpd httpd-tools

But I guess the scripts and the documentation are incorrect and it'll work fine once I edit /opt/rudder/etc/rudder-web.properties. =)

#3 Updated by Lionel Le Folgoc over 3 years ago

# ll /opt/rudder/etc/server-roles.d/
total 0
-rw-r--r-- 1 root root 0 Mar  9 16:06 rudder-jetty
-rw-r--r-- 1 root root 0 Mar  9 16:24 rudder-webapp

so if there's "rudder.server-roles.db=autodetect" in /opt/rudder/etc/rudder-web.properties it will put localhost as db server even when the localhost has not the db role...

#4 Updated by Nicolas CHARLES over 3 years ago

  • Category set to Server components
  • Status changed from New to Discussion
  • Assignee set to Lionel Le Folgoc

"rudder.server-roles.db=autodetect" should be the right value

could you check, on the node with the rudder-db role, that you indeed have the file rudder-db in /opt/rudder/etc/server-roles.d ?

Just to be sure, it used to work correctly, and you have problems since the upgrade ?

#5 Updated by Lionel Le Folgoc over 3 years ago

The db is on amazon rds, so I can't really set the role, so rudder does not "know" rudderdb. I guess this is why autodetection fails. So I think in this case forcing "rudder.server-roles.db=rudderdb" will make it work.

#6 Updated by Nicolas CHARLES over 3 years ago

Indeed, if you don't have a node with the rudder-db role, you'll have to specifically define that the rudder-db is on a specific node in the rudder.server-roles.db value

#7 Updated by François ARMAND over 3 years ago

  • Priority changed from N/A to 1
  • Target version set to 2.11.9

Lionel,

Did changing the property in /opt/rudder/etc/rudder-web.properties make the role OK ?

If so, can I rename that ticket to: "Mis-documentation on server role configuration make them be regularly overwritten" so that we correct the documentation to handle your use case ?

#8 Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 2.11.9 to 2.11.10

#9 Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 2.11.10 to 2.11.11

#10 Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 2.11.11 to 2.11.12

#11 Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 2.11.12 to 2.11.13

#12 Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 2.11.13 to 2.11.14

#13 Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 2.11.14 to 2.11.15

#14 Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 2.11.15 to 2.11.16

#15 Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 2.11.16 to 2.11.17

#16 Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 2.11.17 to 2.11.18

#17 Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 2.11.18 to 2.11.19

#18 Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 2.11.19 to 2.11.20

#19 Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 2.11.20 to 2.11.21

#20 Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 2.11.21 to 2.11.22

#21 Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 2.11.22 to 2.11.23

#22 Updated by Vincent MEMBRÉ about 2 years ago

  • Target version changed from 2.11.23 to 2.11.24

#23 Updated by Vincent MEMBRÉ about 2 years ago

  • Target version changed from 2.11.24 to 308

#24 Updated by Vincent MEMBRÉ about 2 years ago

  • Target version changed from 308 to 3.1.14

#25 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 3.1.14 to 3.1.15

#26 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 3.1.15 to 3.1.16

#27 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 3.1.16 to 3.1.17

#28 Updated by François ARMAND almost 2 years ago

  • Related to Architecture #8463: Server roles is a BIG mess causing inconsistencies and bugs added

#29 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 3.1.17 to 3.1.18

#30 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 3.1.18 to 3.1.19

#31 Updated by Benoît PECCATTE over 1 year ago

  • Subject changed from Server roles overwritten regurlarly for unknown reason to Server roles overwritten regularly for unknown reason

#32 Updated by Jonathan CLARKE over 1 year ago

  • Severity set to Minor - inconvenience | misleading | easy workaround
  • User visibility set to Operational - other Techniques | Technique editor | Rudder settings

#33 Updated by Benoît PECCATTE over 1 year ago

  • Priority set to 14

#34 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 3.1.19 to 3.1.20

#35 Updated by Jonathan CLARKE over 1 year ago

  • Status changed from Discussion to New

#36 Updated by Jonathan CLARKE over 1 year ago

  • Assignee deleted (Lionel Le Folgoc)

#37 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 3.1.20 to 3.1.21

#38 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 3.1.21 to 3.1.22

#39 Updated by Benoît PECCATTE about 1 year ago

  • Priority changed from 14 to 27

#40 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 3.1.22 to 3.1.23

#41 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 3.1.23 to 3.1.24

#42 Updated by Vincent MEMBRÉ 11 months ago

  • Target version changed from 3.1.24 to 3.1.25

#43 Updated by Vincent MEMBRÉ 10 months ago

  • Target version changed from 3.1.25 to 387

#44 Updated by Vincent MEMBRÉ 9 months ago

  • Target version changed from 387 to 4.1.10

#45 Updated by Vincent MEMBRÉ 7 months ago

  • Target version changed from 4.1.10 to 4.1.11

#46 Updated by Vincent MEMBRÉ 5 months ago

  • Target version changed from 4.1.11 to 4.1.12

#47 Updated by Vincent MEMBRÉ 4 months ago

  • Target version changed from 4.1.12 to 4.1.13
  • Priority changed from 27 to 28

#48 Updated by Vincent MEMBRÉ 2 months ago

  • Target version changed from 4.1.13 to 4.1.14

#49 Updated by Benoît PECCATTE about 2 months ago

  • Target version changed from 4.1.14 to 4.1.15

Also available in: Atom PDF