[rudder-dev] rules and directives gone after upgrade to nightly

[Nicolas Charles]

On 13/09/2012 17:05, Francois Armand wrote:
> Le 13/09/2012 13:55, Michael Gliwinski a écrit :
>
>
> Hello Michael,
>
> To start with, we are very sorry that the migration failed, and well, 
> that comfort my opinion of migration ("it's hard" ;)
>
> One first general remark: what you see in the Rudder Web UI for 
> Groups, Directives and Rules is what is in the *LDAP*.
>
> The XML files that are in /var/rudder/configuration- 
> repository/{groups, rules, directives} are there to allow archiving 
> and import/export feature - so that you could bootstrap a new Rudder 
> instance more quickly (by preparing these file and then importing 
> them), or restore a previous state of you configuration policy (the 
> files are in a Git repository and when you "export" from the 
> administration->archive screen, that creates a tag that could be used 
> latter as a snapshot point).
>
> So, even without the /var/rudder/configuration- repository/{groups, 
> rules, directives} directories, Rudder should work fine, and on the 
> opposite, it's not because these files are presents that that means 
> that Rudder should work.
>
> /var/rudder/configuration- repository/*techniques* is a completely 
> different beast, as it is required for Rudder to work.
>
> But your case is *really* strange, because your LDAP content seems OK, 
> and you still don't have anything on the UI, on three really different 
> parts (rules, directives, groups...)
>
> So, as for now I don't see what could lead to that behavior, I have 
> some questions:
>
>   * what was your first Rudder version installed ? A 2.3 ? A 2.4 ? If
>     2.4, before beta 2 ? And how many update did you do ?
>       o that will allows to know what was the starting point, and what
>         could have failed along.
>   * you don't see any rules or only some of them are missing ?
>       o if no rules at all, are you able to create new ones (and see
>         them afterward) ?
>   * you don't see anything in directive screen ? Or at least some
>     categories ? Or some categories and directives ?
>   * you don't see any group nor category at all ? Or at least some
>     groups and categories ?
>   * could you check that the LDAP directory is actually up and running
>     when you try to access Rudder (well, ok... but that's just to be
>     sure :)
>   * could you set the loglevel of the Webapp to "debug" to see if some
>     more relevant information are available ?
>       o That's on the file /opt/rudder/etc/logback.xml, changin <root
>         level="info"> to <root level="debug">, and restarting Rudder
>   * could you give us the number of items in the LDAP for each of
>     groups (nodeGroupId), directive (directiveId) and rules (ruleId) ?
>
>
> And I have some other questions (and some comments) in the following 
> body of the email:
>
>> On Wednesday 12 Sep 2012 16:36:53 Nicolas Perron wrote:
>>> Ok, then it could be due to LDAP which fail during migration. Do you
>>> have the output from the migration and if so could you send it to us ?
>> Sorry, don't have the output anymore.  I had to re-try upgrading a couple of
>> times, some of the failures were:
>>
>>   - rudder-agent 'Text file busy' error while updating cf-agent (that's #2792 I
>> think), re-trying fixed it
>>   - rudder-server-root - the problem here was that there was an installed but
>> unconfigured version from yesterday, couldn't configure today because newer
>> version of rudder-webapp was already installed; fixed by running `sudo dpkg --
>> configure --force-depends rudder-server-root` and then `sudo apt-get install
>> rudder-server-root`
>>   - rudder-webapp error from git commit (no changes to commit), this was due to
>> deletion of techniques/system/distributePolicy/1.0/logrotate.st, fixed by doing
>> git rm and git commit manually, then re-trying
>>
>>> I've tried to reproduce your problem without success.
>>> Could it be possible to have a dump of your LDAP base with
>>> /opt/rudder/sbin/slapcat , please ?
>> Yeah, I imagine this would be non-reproducible :)  I may have done something
>> wrong during upgrade, just trying to figure out if this can be fixed without
>> starting over.
>>
>> Regarding the LDAP dump, I'd rather not send the entire thing over email
>> (security reasons), is there something specific I can check?
>
>
> Yeah, you are right for the security problem :)
> We will try to ask for more precise things now.
>
>
>> Looking through the LDAP dump I can actually see the rules and directives
>> there, e.g. here's one rule:
>>
>>
>> dn: ruleId=441d550b-4a3a-4c99-
>> a710-14860667dfd0,ou=Rules,ou=Rudder,cn=rudder-c
>>   onfiguration
>> ruleId: 441d550b-4a3a-4c99-a710-14860667dfd0
>> objectClass: rule
>> objectClass: top
>> isEnabled: TRUE
>> isSystem: FALSE
>> structuralObjectClass: rule
>> entryUUID: f934115e-7bfb-1031-88cc-c913f245ae37
>> creatorsName: cn=manager,cn=rudder-configuration
>> createTimestamp: 20120816143948Z
>> ruleTarget: group:beb7373d-0a16-4f14-ae94-4655b9cfc944
>> description: Setup servers for CUPS server role.
>> longDescription: This installs CUPS server packages and configures CUPS to ena
>>   ble remote administration.
>> cn: cups-server
>> directiveId: 5538072f-e853-48e4-b163-160b5223fa9e
>> directiveId: a1924bbe-2cc9-49a0-9c81-396217f77e6e
>> directiveId: fcc8c069-777f-49a5-acb5-1fad13a8bf02
>> directiveId: e186f22d-b3da-414a-bcf9-fc88d073aac6
>> serial: 9
>> entryCSN: 20120831115436.454389Z#000000#000#000000
>> modifiersName: cn=manager,cn=rudder-configuration
>> modifyTimestamp: 20120831115436Z
>>
>>
>> I also checked the referenced group and directive IDs and they are all there.
>>
>> I just noticed this problem also applies to groups, e.g. on 'Node Management -
>>> Groups' I see errors like "Can not find node db7463eb-
>> c5ac-432d-9749-39e9503ef7cb", yet the file /var/rudder/configuration-
>> repository/groups/db7463eb-c5ac-432d-9749-39e9503ef7cb.xml is there with
>> contents:
>
> As explain at the beginning, what you should look for is the LDAP 
> entry with RDN nodeGroupId=beb7373d-0a16-4f14-ae94-4655b9cfc944, as 
> you checked just after.
>
>>
>> <nodeGroup fileFormat="2">
>>    <id>db7463eb-c5ac-432d-9749-39e9503ef7cb</id>
>>    <displayName>linux</displayName>
>>    <description>All GNU/Linux hosts.</description>
>>    <query>
>>      {"select":"node","composition":"And","where":
>> [{"objectType":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
>>    </query>
>>    <isDynamic>true</isDynamic>
>>    <nodeIds>
>>      <id>456601f4-0a41-4d63-898e-51a965fd5f1f</id>
>>      <id>adc79f12-633e-4338-94ca-681bf09c066e</id>
>>      <id>546de2a2-9631-423b-b8d4-987318e9baf8</id>
>>      <id>e472f66e-9001-465e-a83e-82fb1101e8c0</id>
>>      <id>f958c6d2-1cad-45a8-9579-4ff397592880</id>
>>    </nodeIds>
>>    <isEnabled>true</isEnabled>
>>    <isSystem>false</isSystem>
>> </nodeGroup>
>
> Here, you perhaps found one bug in our migration script: the 
> fileFormat should be "3". That seems to be an actual problem in our 
> script, not one tied to your scenario, and one that should not leads 
> to what you see. It should lead to "bad file format" in Event Log 
> screen and if you try to restore that archive.
>
>>
>> and also in LDAP:
>>
>>
>> dn: nodeGroupId=db7463eb-
>> c5ac-432d-9749-39e9503ef7cb,groupCategoryId=GroupRoot
>>   ,ou=Rudder,cn=rudder-configuration
>> nodeGroupId: db7463eb-c5ac-432d-9749-39e9503ef7cb
>> objectClass: nodeGroup
>> objectClass: top
>> cn: linux
>> description: All GNU/Linux hosts.
>> isEnabled: TRUE
>> isSystem: FALSE
>> isDynamic: TRUE
>> jsonNodeGroupQuery: {"select":"node","composition":"And","where":
>> [{"objectType
>>   ":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
>> structuralObjectClass: nodeGroup
>> entryUUID: ac7cf584-62f0-1031-9c67-1508ccbe5302
>> creatorsName: cn=manager,cn=rudder-configuration
>> createTimestamp: 20120715174555Z
>> nodeId: f958c6d2-1cad-45a8-9579-4ff397592880
>> nodeId: 456601f4-0a41-4d63-898e-51a965fd5f1f
>> nodeId: adc79f12-633e-4338-94ca-681bf09c066e
>> nodeId: 546de2a2-9631-423b-b8d4-987318e9baf8
>> nodeId: e472f66e-9001-465e-a83e-82fb1101e8c0
>> entryCSN: 20120816130628.925149Z#000000#000#000000
>> modifiersName: cn=manager,cn=rudder-configuration
>> modifyTimestamp: 20120816130628Z
>
>
> Some more random ideas :
>
>   * could you check that you don't have duplicated names (cn) for
>     groups, directives and rules ? That is not allowed (to not lead to
>     confusion in reports). That should not lead to what you see, but
>     at that point, that could be interesting to check !
>   * the fact that you don't see any directives nor techniques nor
>     categories in the Directive screen seems du to the fact that all
>     root categories are not found by Rudder in the LDAP, as displayed
>     in the logs in your first email. Could you check that they are
>     actually here, and that their DN look like:
>     *techniqueCategoryId=XXX, techn**iqueCategoryId=Active Techniques,
>     ou=Rudder, cn=rudder-configuration*
>       o with XXX being exactly one the missing categories you see in
>         logs (*applications, fileDistribution, etc)
>         *
>
> Well... And I don't know what to ask you anymore for now, so that 
> should be the end of that mail !
>
> Michael, thanks again for time you are taking to report these 
> information, they are very much valuable for us (and hopefully, we 
> will be able to help you).
>
Michael,

Could you also send us a copy of you configuration.properties files 
(which should be in /opt/rudder/etc) ? (and don't forget to remove the 
LDAP and Postgresql password)
Maybe there's something there which could help us understand what is not 
right.

Thank you very much,
Nicolas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120913/4352a3bb/attachment.html>