[rudder-dev] rules and directives gone after upgrade to nightly

[Michael Gliwinski]

Re-sending to the list too :P

Many thanks for helping out guys :)  Replies inline.

On Thursday 13 Sep 2012 17:05:05 you wrote:
> To start with, we are very sorry that the migration failed, and well,
> that comfort my opinion of migration ("it's hard" ;)

No worries, I agree :)

> One first general remark: what you see in the Rudder Web UI for Groups,
> Directives and Rules is what is in the *LDAP*.
> 
> The XML files that are in /var/rudder/configuration- repository/{groups,
> rules, directives} are there to allow archiving and import/export
> feature - so that you could bootstrap a new Rudder instance more quickly
> (by preparing these file and then importing them), or restore a previous
> state of you configuration policy (the files are in a Git repository and
> when you "export" from the administration->archive screen, that creates
> a tag that could be used latter as a snapshot point).
> 
> So, even without the /var/rudder/configuration- repository/{groups,
> rules, directives} directories, Rudder should work fine, and on the
> opposite, it's not because these files are presents that that means that
> Rudder should work.

OK, this is very useful to know, thanks.  I wasn't sure about that.

> /var/rudder/configuration- repository/*techniques* is a completely
> different beast, as it is required for Rudder to work.
> 
> But your case is *really* strange, because your LDAP content seems OK,
> and you still don't have anything on the UI, on three really different
> parts (rules, directives, groups...)
> 
> So, as for now I don't see what could lead to that behavior, I have some
> questions:
> 
>   * what was your first Rudder version installed ? A 2.3 ? A 2.4 ? If
>     2.4, before beta 2 ? And how many update did you do ?
>       o that will allows to know what was the starting point, and what
>         could have failed along.

First version was 2.4 alpha7, then upgraded to beta1 -> beta2 -> beta3 -> 
beta4 and finally to nightly (on 2012-09-11/12).

>   * you don't see any rules or only some of them are missing ?
>       o if no rules at all, are you able to create new ones (and see
>         them afterward) ?

No rules at all (empty table with "No matching rules!" line).

Cannot create a directive as there are no active techniques.  E.g. see the 
attached screenshot of Techniques screen.  When I try to create a new Active 
Technique category, I get an error "The form contains some errors, please 
correct them
An error occurred while fetching the parent category" (even if parent category 
is "Root of active technique's library").

I can create a rule, and it is visible afterward.  I tried to check if the 
resulting entry in LDAP is somehow different from other rules, but it doesn't 
look like it:


dn: ruleId=de422072-3297-44e3-86bb-
d943340dd76e,ou=Rules,ou=Rudder,cn=rudder-c
 onfiguration
ruleId: de422072-3297-44e3-86bb-d943340dd76e
objectClass: rule
objectClass: top
cn: test-rule
isEnabled: TRUE
isSystem: FALSE
serial: 0


>   * you don't see anything in directive screen ? Or at least some
>     categories ? Or some categories and directives ?

Only "Root of active technique's library" with no categories or directives 
under it.

>   * you don't see any group nor category at all ? Or at least some
>     groups and categories ?

There is "Root of the group and group categories" and a couple of entries 
under it (one for each group that was defined) but each one saying "Can not find 
node UUID" in red.

>   * could you check that the LDAP directory is actually up and running
>     when you try to access Rudder (well, ok... but that's just to be sure :)

Checked with /opt/rudder/bin/ldapsearch -H ldap//localhost -x ... (taken one 
check from rudder-upgrade script) and it works.

> * could you set the loglevel of the Webapp to "debug" to see if some more
> relevant information are available ?
>       o That's on the file /opt/rudder/etc/logback.xml, changin <root
>         level="info"> to <root level="debug">, and restarting Rudder

There is only one additional message in webapp/DATE.stderrout.log that 
indicates any kind of error:

[2012-09-14 12:39:14] DEBUG 
com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
- Dynamic group update started at 2012/09/14 12:39:14, ended at 2012/09/14 
12:39:14
[2012-09-14 12:39:14] ERROR 
com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
- Error when updating dynamic group NodeGroupId(db7463eb-
c5ac-432d-9749-39e9503ef7cb) : <- Error when retrieving the entry for 
NodeGroup 'NodeGroupId(db7463eb-c5ac-432d-9749-39e9503ef7cb)'

When I navigate to directives or groups in the web interface I only get the 
same errors as before 
("com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category" and 
"com.normation.rudder.web.snippet.node.Groups - Error while fetching 
Technique category NodeGroupCategoryId(SystemGroups) <- Entry with ID 
'NodeGroupCategoryId(SystemGroups)' was not found").

Nothing new in core/rudder-webapp.log, but I did notice 52 errors at startup 
like:

Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '6' <- Not migrating eventLog with [id: 6] [type:
 SuccessfulDeployment]: no handler for that type.
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '11' <- Not migrating eventLog with [id: 11] [typ
e: SuccessfulDeployment]: no handler for that type.
...
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '120' <- Not migrating eventLog with [id: 120] [t
ype: ExportRules]: no handler for that type.
ype: ExportTechniqueLibrary]: no handler for that type.
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '122' <- Not migrating eventLog with [id: 122] [t
ype: ExportGroups]: no handler for that type.
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '123' <- Not migrating eventLog with [id: 123] [t
ype: ExportFullArchive]: no handler for that type.
...
Sep 14 11:59:34 centostest rudder[migration]: [INFO] Migration from EventLog 
fileFormat from '2' to '3' done, 0 EventLogs migrated

not sure if they are relevant?

>   * could you give us the number of items in the LDAP for each of groups
>     (nodeGroupId), directive (directiveId) and rules (ruleId) ?

Do you mean counts of actual objects (i.e. objectClass=nodeGroup, 
objectClass=directive, objectClass=rule)?  If yes:

There are 6 nodeGroupId, one "nodeGroupId: hasPolicyServer-root" and the rest 
with UUIDs.

22 directives:

directiveId: inventory-all
directiveId: root-distributePolicy
directiveId: common-root
19 other directives with directiveId: UUID

10 rules:

ruleId: inventory-all
ruleId: root-DP
ruleId: hasPolicyServer-root
7 other rules with ruleId: UUID

> And I have some other questions (and some comments) in the following
> 
> body of the email:
> > On Wednesday 12 Sep 2012 16:36:53 Nicolas Perron wrote:
...
> > <nodeGroup fileFormat="2">
> > 
> >    <id>db7463eb-c5ac-432d-9749-39e9503ef7cb</id>
> >    <displayName>linux</displayName>
> >    <description>All GNU/Linux hosts.</description>
> >    <query>
> >    
> >      
{"select":"node","composition":"And","where":
> > 
[{"objectType":"node","attribute":"OS
> > 
","comparator":"eq","value":"Lin
> > ux"}]}> 
> >    </query>
> >    <isDynamic>true</isDynamic>
> >    <nodeIds>
> >    
> >      <id>456601f4-0a41-4d63-898e-51a965fd5f1f</id>
> >      <id>adc79f12-633e-4338-94ca-681bf09c066e</id>
> >      <id>546de2a2-9631-423b-b8d4-987318e9baf8</id>
> >      <id>e472f66e-9001-465e-a83e-82fb1101e8c0</id>
> >      <id>f958c6d2-1cad-45a8-9579-4ff397592880</id>
> >    
> >    </nodeIds>
> >    <isEnabled>true</isEnabled>
> >    <isSystem>false</isSystem>
> > 
> > </nodeGroup>
> 
> Here, you perhaps found one bug in our migration script: the fileFormat
> should be "3". That seems to be an actual problem in our script, not one
> tied to your scenario, and one that should not leads to what you see. It
> should lead to "bad file format" in Event Log screen and if you try to
> restore that archive.

Which script is that?  I checked rudder-upgrade and postinst scripts and 
didn't see any of them trying to update the xml files.

> > and also in LDAP:
> > 
> > 
> > dn: nodeGroupId=db7463eb-
> > c5ac-432d-9749-39e9503ef7cb,groupCategoryId=GroupRoot
> > 
> >   ,ou=Rudder,cn=rudder-configuration
> > 
> > nodeGroupId: db7463eb-c5ac-432d-9749-39e9503ef7cb
> > objectClass: nodeGroup
> > objectClass: top
> > cn: linux
> > description: All GNU/Linux hosts.
> > isEnabled: TRUE
> > isSystem: FALSE
> > isDynamic: TRUE
> > jsonNodeGroupQuery: {"select":"node","composition":"And","where":
> > [{"objectType
> > 
> >   ":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
> > 
> > structuralObjectClass: nodeGroup
> > entryUUID: ac7cf584-62f0-1031-9c67-1508ccbe5302
> > creatorsName: cn=manager,cn=rudder-configuration
> > createTimestamp: 20120715174555Z
> > nodeId: f958c6d2-1cad-45a8-9579-4ff397592880
> > nodeId: 456601f4-0a41-4d63-898e-51a965fd5f1f
> > nodeId: adc79f12-633e-4338-94ca-681bf09c066e
> > nodeId: 546de2a2-9631-423b-b8d4-987318e9baf8
> > nodeId: e472f66e-9001-465e-a83e-82fb1101e8c0
> > entryCSN: 20120816130628.925149Z#000000#000#000000
> > modifiersName: cn=manager,cn=rudder-configuration
> > modifyTimestamp: 20120816130628Z
> 
> Some more random ideas :
> 
>   * could you check that you don't have duplicated names (cn) for
>     groups, directives and rules ? That is not allowed (to not lead to
>     confusion in reports). That should not lead to what you see, but at
>     that point, that could be interesting to check !

Confirmed that there are no duplicates.

>   * the fact that you don't see any directives nor techniques nor
>     categories in the Directive screen seems du to the fact that all
>     root categories are not found by Rudder in the LDAP, as displayed in
>     the logs in your first email. Could you check that they are actually
>     here, and that their DN look like: *techniqueCategoryId=XXX,
>     techn**iqueCategoryId=Active Techniques, ou=Rudder,
>     cn=rudder-configuration*
>       o with XXX being exactly one the missing categories you see in
>         logs (*applications, fileDistribution, etc)
>         *

Yes, they are all there:


dn: techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active 
Techniques,
 ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=applications,techniqueCategoryId=Active 
Techniques,ou=
 Rudder,cn=rudder-configuration

dn: techniqueCategoryId=fileDistribution,techniqueCategoryId=Active 
Techniques
 ,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=systemSettings,techniqueCategoryId=Active 
Techniques,o
 u=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=security,techniqueCategoryId=systemSettings,techniqueC
 ategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=remoteAccess,techniqueCategoryId=systemSettings,techni
 queCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=misc,techniqueCategoryId=systemSettings,techniqueCateg
 oryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=userManagement,techniqueCategoryId=systemSettings,tech
 niqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=process,techniqueCategoryId=systemSettings,techniqueCa
 tegoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=systemManagement,techniqueCategoryId=systemSettings,te
 chniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=networking,techniqueCategoryId=systemSettings,techniqu
 eCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=jobScheduling,techniqueCategoryId=Active 
Techniques,ou
 =Rudder,cn=rudder-configuration

dn: techniqueCategoryId=fileConfiguration,techniqueCategoryId=Active 
Technique
 s,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=fileSecurity,techniqueCategoryId=fileConfiguration,tec
 hniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

> Well... And I don't know what to ask you anymore for now, so that should
> be the end of that mail !

That's a puzzling one, isn't it? :D
**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rudder-active-techniques-empty.png
Type: image/png
Size: 11484 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120915/0c3bf37a/attachment-0001.png>