User story #3152: Create a migration script to change the stored internal Rudder variables from ${xxx.yyy} to ${rudder.xxxx.yyyy} - Rudder - Issue Tracker

Actions

Copy link

User story #3152

closed

Create a migration script to change the stored internal Rudder variables from ${xxx.yyy} to ${rudder.xxxx.yyyy}

Added by Nicolas CHARLES over 11 years ago. Updated about 9 years ago.

Status:

Released

Priority:

Assignee:

Nicolas CHARLES

Category:

Packaging

Target version:

2.5.0~rc1

Pull Request:

UX impact:

Suggestion strength:

User visibility:

Effort required:

Name check:

Fix check:

Regression:

Description

We are changing the format of our internal Rudder variables that designates others "object" in Rudder (nodes, group, policyserver) from ${xxx.yyy} to ${rudder.xxx.yyy}

Some values already exists in the LDAP tree, and need to be migrated :
- in the user defined directive, some may exists if an exerienced user used them
- in the System directive (they are mandatory there, and always exists)

The migration script should :

check in the system directives DN: directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration if there are values that starts with ${rudder.*}. If so, exit
update the variables ${X} where X doesn't starts by rudder to ${rudder.X} in the DN: techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration subtree, EXCEPT in techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration
update the variables ${X} where X doesn't starts by rudder to ${rudder.X} in the techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

This script should hence be reentrant.

The script will actually be implemented as a Rudder web application boot-time check, i.e. a Scala function executed each time Rudder webapp is started.

Files

migration_test.sh (2.57 KB) migration_test.sh

Matthieu CERDA, 2013-01-14 14:43

Actions

Copy link

Updated by Matthieu CERDA over 11 years ago

Status changed from 8 to In progress
% Done changed from 0 to 10

I will begin with this command, inspired by the existing cpuSpeed conversion rudder-upgrade part:

/opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL

To see if interesting variables are usable

Actions

Copy link

Updated by Matthieu CERDA over 11 years ago

Here is a usable condition test before migrating (for reentrance):

/opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL|grep '${rudder.'|wc -l

Actions

Copy link

Updated by Matthieu CERDA over 11 years ago

% Done changed from 10 to 30

Script snippet beeing worked on right now:

# Convert old style variable format to the new one
LDAP_SYSVAR_NOT_PREFIXED=$(/opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL|grep '${rudder.'|wc -l)

if [ ${LDAP_SYSVAR_NOT_PREFIXED} -eq 0 ]; then
        VARUPDATE_TMPFILE=$(mktemp)
        /opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL directiveVariable > ${VARUPDATE_TMPFILE}
        # Here, a sed snippet to add:
        # changetype: modify
        # replace: directiveVariable
        # Before the big directiveVariable pack of lines
        sed -i 's%${%${rudder.%' ${VARUPDATE_TMPFILE}

        rm ${VARUPDATE_TMPFILE}
        echo "Some old format variables were updated"

Actions

Copy link

Updated by Matthieu CERDA over 11 years ago

% Done changed from 30 to 50

Update:

LDAP_SYSVAR_NOT_PREFIXED=$(/opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL|grep '${rudder.'|wc -l)

if [ ${LDAP_SYSVAR_NOT_PREFIXED} -eq 0 ]; then
        VARUPDATE_TMPFILE=$(mktemp)
#        sed "s%cpuSpeed: \(.*\)\..*%cpuSpeed: \1%g"| \

        # Build LDIF with interesting values
        /opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "directiveId=common-root,activeTechniqueId=common,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL directiveVariable|sed '0,/directiveVariable:/s//change
type: modify\nreplace: directiveVariable\ndirectiveVariable:/' > ${VARUPDATE_TMPFILE}
        /opt/rudder/bin/ldapsearch -H ldap://localhost -x -w ${LDAP_PASSWORD} -D "${LDAP_USER}" -b "techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration" -LLL "(&(directiveVariable=*)(!(techniqueCategoryId=Rudder Internal)))" directiveVariable|sed '0,/directiveVariable:/s//changetype: modify\nrepl
ace: directiveVariable\ndirectiveVariable:/' >> ${VARUPDATE_TMPFILE}

        # Edit it to match the wanted ones, and insert the whole LDIF
        sed -i 's/${/${rudder./' ${VARUPDATE_TMPFILE}
#       /opt/rudder/bin/ldapmodify -H ldap://localhost -x -w ${LDAP_PASSWORD} -D ${LDAP_USER} -f ${VARUPDATE_TMPFILE} >/dev/null 2>&1

        # Clean the mess and exit the if
        rm ${VARUPDATE_TMPFILE}
        echo "Some old format variables were updated" 
fi

Still not functionnal, the sed is not editing all the blocks and the subtree exclusion is not working.

Actions

Copy link

Updated by Matthieu CERDA over 11 years ago

Status changed from In progress to Discussion
Assignee changed from Matthieu CERDA to Jonathan CLARKE

I need a another LDAP expert help here !

Actions

Copy link

Updated by Matthieu CERDA over 11 years ago

File migration_test.sh migration_test.sh added
Assignee changed from Jonathan CLARKE to Nicolas CHARLES

After some more thinking, it has been suggested that it might be more smart to do this in scala (in the webapp). I'll attach the current work to this ticket, and reassign it to someone more competent in this domain ! :)

Actions

Copy link