From Michael.Gliwinski at henderson-group.com  Mon Sep 10 12:40:20 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Mon, 10 Sep 2012 11:40:20 +0100
Subject: [rudder-dev] nightly build for ubuntu precise not up to date
Message-ID: <201209101140.21279.Michael.Gliwinski@henderson-group.com>

Hi All,

I was just trying to switch to 2.4 nightly builds to test a fix for #2680, so 
I updated the file in sources.list.d to point to the repository at 
http://www.rudder-project.org/apt-2.4-nightly/

After that and `apt-get update`, `apt-get dist-upgrade` wants to remove 
rudder-inventory-endpoint rudder-server-root and rudder-webapp.  Guessing this 
may have something to do with dependencies I went to check the repository, and 
at http://www.rudder-project.org/apt-2.4-nightly/pool/main/r/rudder-webapp/ I 
can see (looking at the timestamps) that rudder-webapp hasn't been updated for 
precise since 7th Sep.

Would you be able to check if there are any build failures, etc.?

Thanks,
Michael


-- 
Michael Gliwinski
Henderson Group Information Services
9-11 Hightown Avenue, Newtownabby, BT36 4RT
Phone: 028 9034 3319

**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From nicolas.perron at normation.com  Mon Sep 10 12:57:14 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Mon, 10 Sep 2012 12:57:14 +0200
Subject: [rudder-dev] nightly build for ubuntu precise not up to date
In-Reply-To: <201209101140.21279.Michael.Gliwinski@henderson-group.com>
References: <201209101140.21279.Michael.Gliwinski@henderson-group.com>
Message-ID: <504DC78A.3080800@normation.com>


> Hi All,
>
> I was just trying to switch to 2.4 nightly builds to test a fix for #2680, so 
> I updated the file in sources.list.d to point to the repository at 
> http://www.rudder-project.org/apt-2.4-nightly/
>
> After that and `apt-get update`, `apt-get dist-upgrade` wants to remove 
> rudder-inventory-endpoint rudder-server-root and rudder-webapp.  Guessing this 
> may have something to do with dependencies I went to check the repository, and 
> at http://www.rudder-project.org/apt-2.4-nightly/pool/main/r/rudder-webapp/ I 
> can see (looking at the timestamps) that rudder-webapp hasn't been updated for 
> precise since 7th Sep.
>
> Would you be able to check if there are any build failures, etc.?
>
> Thanks,
> Michael
Hi Michael,

You're right, it has been building failure this Week End. Don't worry,
it only concerns our building environment.

The Rudder dependencies prevent from installing mixed version which
could result in problem like this when build failed. I'm on it and the
problem will be fixed as soon as possible.

Thanks for telling us.

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================


From Michael.Gliwinski at henderson-group.com  Mon Sep 10 13:14:46 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Mon, 10 Sep 2012 12:14:46 +0100
Subject: [rudder-dev] nightly build for ubuntu precise not up to date
In-Reply-To: <504DC78A.3080800@normation.com>
References: <201209101140.21279.Michael.Gliwinski@henderson-group.com> 
	<504DC78A.3080800@normation.com>
Message-ID: <201209101214.46716.Michael.Gliwinski@henderson-group.com>

On Monday 10 Sep 2012 11:57:14 Nicolas Perron wrote:
> > repository, and at
> > http://www.rudder-project.org/apt-2.4-nightly/pool/main/r/rudder-webapp/
> > I can see (looking at the timestamps) that rudder-webapp hasn't been
> > updated for precise since 7th Sep.
> > 
> > Would you be able to check if there are any build failures, etc.?
> 
> You're right, it has been building failure this Week End. Don't worry,
> it only concerns our building environment.
> 
> The Rudder dependencies prevent from installing mixed version which
> could result in problem like this when build failed. I'm on it and the
> problem will be fixed as soon as possible.

Yeah, that's what I thought.  Thanks for looking into it :)


-- 
Michael Gliwinski
Henderson Group Information Services
9-11 Hightown Avenue, Newtownabby, BT36 4RT
Phone: 028 9034 3319

**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From nicolas.perron at normation.com  Mon Sep 10 15:20:22 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Mon, 10 Sep 2012 15:20:22 +0200
Subject: [rudder-dev] nightly build for ubuntu precise not up to date
In-Reply-To: <201209101214.46716.Michael.Gliwinski@henderson-group.com>
References: <201209101140.21279.Michael.Gliwinski@henderson-group.com>
	<504DC78A.3080800@normation.com>
	<201209101214.46716.Michael.Gliwinski@henderson-group.com>
Message-ID: <504DE916.4080404@normation.com>


> On Monday 10 Sep 2012 11:57:14 Nicolas Perron wrote:
>>> repository, and at
>>> http://www.rudder-project.org/apt-2.4-nightly/pool/main/r/rudder-webapp/
>>> I can see (looking at the timestamps) that rudder-webapp hasn't been
>>> updated for precise since 7th Sep.
>>>
>>> Would you be able to check if there are any build failures, etc.?
>> You're right, it has been building failure this Week End. Don't worry,
>> it only concerns our building environment.
>>
>> The Rudder dependencies prevent from installing mixed version which
>> could result in problem like this when build failed. I'm on it and the
>> problem will be fixed as soon as possible.
> Yeah, that's what I thought.  Thanks for looking into it :)
You should be able to reuse the Ubuntu repository, now.

Sorry for the inconvenience !

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================


From Michael.Gliwinski at henderson-group.com  Wed Sep 12 14:09:37 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Wed, 12 Sep 2012 13:09:37 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
Message-ID: <1368984.40l1Cv2rFx@hgis96>

Hi all,

Sorry, I know you're busy with the fixes for 2.4 release.

I just upgraded rudder server from beta4 to nigthly (rudder-server-root 
2.4.0~beta5~git201209120436-precise0) and all rules and directives have 
disappeared from the web interface.  AFAICS all the files are still there in 
/var/rudder/configuration-repository/{directives,rules}/ directories.

I may have messed something up during upgrade, as I had to re-try it several 
times.  Everything is OK now as far as packages are concerned.

Any ideas where to check what might be happening?

Just after upgrade, I've found these in the 
/var/log/rudder/webapp/2012_09_12.stderrout.log:

[2012-09-12 10:08:40] ERROR 
com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
- Error when updating dynamic group NodeGroupId(db7463eb-
c5ac-432d-9749-39e9503ef7cb) : <- Error when retrieving the entry for 
NodeGroup 'NodeGroupId(db7463eb-c5ac-432d-9749-39e9503ef7cb)'
[2012-09-12 10:09:16] ERROR 
com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(Rudder Internal)
[2012-09-12 10:09:16] ERROR 
com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(applications)
[2012-09-12 10:09:16] ERROR 
com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(fileDistribution)
[2012-09-12 10:09:16] ERROR 
com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(systemSettings)
[2012-09-12 10:09:16] ERROR 
com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(jobScheduling)
[2012-09-12 10:09:16] ERROR 
com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(fileConfiguration)
[2012-09-12 10:09:30] ERROR 
com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
Error when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(Rudder Internal)
[2012-09-12 10:09:30] ERROR 
com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
Error when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(applications)
[2012-09-12 10:09:30] ERROR 
com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
Error when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(fileDistribution)
[2012-09-12 10:09:30] ERROR 
com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
Error when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(systemSettings)
[2012-09-12 10:09:30] ERROR 
com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
Error when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(jobScheduling)
[2012-09-12 10:09:30] ERROR 
com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
Error when displaying category: Error while fetching Active Technique category 
ActiveTechniqueCategoryId(fileConfiguration)
[2012-09-12 10:13:40] ERROR 
com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
- Error when updating dynamic group NodeGroupId(db7463eb-
c5ac-432d-9749-39e9503ef7cb) : <- Error when retrieving the entry for 
NodeGroup 'NodeGroupId(db7463eb-c5ac-432d-9749-39e9503ef7cb)'


**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From nicolas.perron at normation.com  Wed Sep 12 14:36:43 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Wed, 12 Sep 2012 14:36:43 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <1368984.40l1Cv2rFx@hgis96>
References: <1368984.40l1Cv2rFx@hgis96>
Message-ID: <505081DB.2030607@normation.com>


> Hi all,
>
> Sorry, I know you're busy with the fixes for 2.4 release.
>
> I just upgraded rudder server from beta4 to nigthly (rudder-server-root 
> 2.4.0~beta5~git201209120436-precise0) and all rules and directives have 
> disappeared from the web interface.  AFAICS all the files are still there in 
> /var/rudder/configuration-repository/{directives,rules}/ directories.
>
> I may have messed something up during upgrade, as I had to re-try it several 
> times.  Everything is OK now as far as packages are concerned.
>
> Any ideas where to check what might be happening?
>
> Just after upgrade, I've found these in the 
> /var/log/rudder/webapp/2012_09_12.stderrout.log:
>
> [2012-09-12 10:08:40] ERROR 
> com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
> - Error when updating dynamic group NodeGroupId(db7463eb-
> c5ac-432d-9749-39e9503ef7cb) : <- Error when retrieving the entry for 
> NodeGroup 'NodeGroupId(db7463eb-c5ac-432d-9749-39e9503ef7cb)'
> [2012-09-12 10:09:16] ERROR 
> com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
> when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(Rudder Internal)
> [2012-09-12 10:09:16] ERROR 
> com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
> when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(applications)
> [2012-09-12 10:09:16] ERROR 
> com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
> when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(fileDistribution)
> [2012-09-12 10:09:16] ERROR 
> com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
> when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(systemSettings)
> [2012-09-12 10:09:16] ERROR 
> com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
> when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(jobScheduling)
> [2012-09-12 10:09:16] ERROR 
> com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
> when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(fileConfiguration)
> [2012-09-12 10:09:30] ERROR 
> com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
> Error when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(Rudder Internal)
> [2012-09-12 10:09:30] ERROR 
> com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
> Error when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(applications)
> [2012-09-12 10:09:30] ERROR 
> com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
> Error when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(fileDistribution)
> [2012-09-12 10:09:30] ERROR 
> com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
> Error when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(systemSettings)
> [2012-09-12 10:09:30] ERROR 
> com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
> Error when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(jobScheduling)
> [2012-09-12 10:09:30] ERROR 
> com.normation.rudder.web.snippet.configuration.TechniqueLibraryManagement - 
> Error when displaying category: Error while fetching Active Technique category 
> ActiveTechniqueCategoryId(fileConfiguration)
> [2012-09-12 10:13:40] ERROR 
> com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
> - Error when updating dynamic group NodeGroupId(db7463eb-
> c5ac-432d-9749-39e9503ef7cb) : <- Error when retrieving the entry for 
> NodeGroup 'NodeGroupId(db7463eb-c5ac-432d-9749-39e9503ef7cb)'
>
>
> **********************************************************************************************
> The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
> If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
> When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
> If you have received this email in error please notify support at henderson-group.com
>
> John Henderson (Holdings) Ltd
> Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
> Registered in Northern Ireland
> Registration Number NI010588
> Vat No.: 814 6399 12
> *********************************************************************************
>
> _______________________________________________
> rudder-dev mailing list
> rudder-dev at lists.rudder-project.org
> http://www.rudder-project.org/mailman/listinfo/rudder-dev

Hi Michael,

Can you give us the output of these command, please ?:

/cd /var/rudder/configuration-repository/ && git log/
/
//find /var/rudder/configuration-repository/ -name "category.xml"/

It could be due to the latest modifications in packaging which involve
the renaming of a folder in //var/rudder/configuration-repository/ .

/Thanks/
/

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120912/5a647927/attachment.html>

From Michael.Gliwinski at henderson-group.com  Wed Sep 12 15:55:27 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Wed, 12 Sep 2012 14:55:27 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <505081DB.2030607@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <505081DB.2030607@normation.com>
Message-ID: <1701591.RfFQ9E4M0L@hgis96>

On Wednesday 12 Sep 2012 14:36:43 Nicolas Perron wrote:
> Can you give us the output of these command, please ?:
> 
> /cd /var/rudder/configuration-repository/ && git log/
> /

The only commits since before starting the upgrade to nightly are:


commit 7896312c9677198b952f83b1e1afa85de2cc56e3
Author: root <root at centostest.(none)>
Date:   Wed Sep 12 10:45:48 2012 +0100

    Propagate deletion of system Techniques - from rudder-
techniques at c164347663460ea2fa0e9d7db1dd43f421ceb996

commit 834d50918591adfcd4acc76192bc2a0eab50408e
Author: root <root at centostest.(none)>
Date:   Tue Sep 11 22:58:19 2012 +0100

    Upgrade system Techniques - automatically done by rudder-upgrade script


Where commit 7896312c9677198b952f83b1e1afa85de2cc56e3 was actually me 
deleting techniques/system/distributePolicy/1.0/logrotate.st as it was causing 
upgrade to fail.

As far as I can see there were no renames in the other commit, only changes to 
do with disable_agent.

> //find /var/rudder/configuration-repository/ -name "category.xml"/

(did a cd and find . for shorter paths)

./techniques/fileConfiguration/fileSecurity/category.xml
./techniques/fileConfiguration/category.xml
./techniques/system/category.xml
./techniques/systemSettings/networking/category.xml
./techniques/systemSettings/process/category.xml
./techniques/systemSettings/security/category.xml
./techniques/systemSettings/category.xml
./techniques/systemSettings/misc/category.xml
./techniques/systemSettings/userManagement/category.xml
./techniques/systemSettings/remoteAccess/category.xml
./techniques/systemSettings/systemManagement/category.xml
./techniques/category.xml
./techniques/fileDistribution/category.xml
./techniques/applications/category.xml
./techniques/jobScheduling/category.xml
./directives/fileConfiguration/fileSecurity/category.xml
./directives/fileConfiguration/category.xml
./directives/systemSettings/networking/category.xml
./directives/systemSettings/process/category.xml
./directives/systemSettings/security/category.xml
./directives/systemSettings/category.xml
./directives/systemSettings/misc/category.xml
./directives/systemSettings/userManagement/category.xml
./directives/systemSettings/remoteAccess/category.xml
./directives/systemSettings/systemManagement/category.xml
./directives/category.xml
./directives/fileDistribution/category.xml
./directives/applications/category.xml
./directives/jobScheduling/category.xml
./groups/category.xml


> It could be due to the latest modifications in packaging which involve
> the renaming of a folder in //var/rudder/configuration-repository/ .
> 
> /Thanks/

Many thanks for helping out Nicolas!  Let me know what else to check.


**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From nicolas.perron at normation.com  Wed Sep 12 16:36:53 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Wed, 12 Sep 2012 16:36:53 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <1701591.RfFQ9E4M0L@hgis96>
References: <1368984.40l1Cv2rFx@hgis96> <505081DB.2030607@normation.com>
	<1701591.RfFQ9E4M0L@hgis96>
Message-ID: <50509E05.3090708@normation.com>


> On Wednesday 12 Sep 2012 14:36:43 Nicolas Perron wrote:
>> Can you give us the output of these command, please ?:
>>
>> /cd /var/rudder/configuration-repository/ && git log/
>> /
> The only commits since before starting the upgrade to nightly are:
>
>
> commit 7896312c9677198b952f83b1e1afa85de2cc56e3
> Author: root <root at centostest.(none)>
> Date:   Wed Sep 12 10:45:48 2012 +0100
>
>     Propagate deletion of system Techniques - from rudder-
> techniques at c164347663460ea2fa0e9d7db1dd43f421ceb996
>
> commit 834d50918591adfcd4acc76192bc2a0eab50408e
> Author: root <root at centostest.(none)>
> Date:   Tue Sep 11 22:58:19 2012 +0100
>
>     Upgrade system Techniques - automatically done by rudder-upgrade script
>
>
> Where commit 7896312c9677198b952f83b1e1afa85de2cc56e3 was actually me 
> deleting techniques/system/distributePolicy/1.0/logrotate.st as it was causing 
> upgrade to fail.
>
> As far as I can see there were no renames in the other commit, only changes to 
> do with disable_agent.
>
>> //find /var/rudder/configuration-repository/ -name "category.xml"/
> (did a cd and find . for shorter paths)
>
> ./techniques/fileConfiguration/fileSecurity/category.xml
> ./techniques/fileConfiguration/category.xml
> ./techniques/system/category.xml
> ./techniques/systemSettings/networking/category.xml
> ./techniques/systemSettings/process/category.xml
> ./techniques/systemSettings/security/category.xml
> ./techniques/systemSettings/category.xml
> ./techniques/systemSettings/misc/category.xml
> ./techniques/systemSettings/userManagement/category.xml
> ./techniques/systemSettings/remoteAccess/category.xml
> ./techniques/systemSettings/systemManagement/category.xml
> ./techniques/category.xml
> ./techniques/fileDistribution/category.xml
> ./techniques/applications/category.xml
> ./techniques/jobScheduling/category.xml
> ./directives/fileConfiguration/fileSecurity/category.xml
> ./directives/fileConfiguration/category.xml
> ./directives/systemSettings/networking/category.xml
> ./directives/systemSettings/process/category.xml
> ./directives/systemSettings/security/category.xml
> ./directives/systemSettings/category.xml
> ./directives/systemSettings/misc/category.xml
> ./directives/systemSettings/userManagement/category.xml
> ./directives/systemSettings/remoteAccess/category.xml
> ./directives/systemSettings/systemManagement/category.xml
> ./directives/category.xml
> ./directives/fileDistribution/category.xml
> ./directives/applications/category.xml
> ./directives/jobScheduling/category.xml
> ./groups/category.xml
>
>
>> It could be due to the latest modifications in packaging which involve
>> the renaming of a folder in //var/rudder/configuration-repository/ .
>>
>> /Thanks/
> Many thanks for helping out Nicolas!  Let me know what else to check.
>
Ok, then it could be due to LDAP which fail during migration. Do you
have the output from the migration and if so could you send it to us ?

I've tried to reproduce your problem without success.
Could it be possible to have a dump of your LDAP base with
/opt/rudder/sbin/slapcat , please ?

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================


From Michael.Gliwinski at henderson-group.com  Thu Sep 13 13:55:32 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Thu, 13 Sep 2012 12:55:32 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <50509E05.3090708@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <1701591.RfFQ9E4M0L@hgis96> 
	<50509E05.3090708@normation.com>
Message-ID: <1565664.ixSq8sTaWe@hgis96>

On Wednesday 12 Sep 2012 16:36:53 Nicolas Perron wrote:
> Ok, then it could be due to LDAP which fail during migration. Do you
> have the output from the migration and if so could you send it to us ?

Sorry, don't have the output anymore.  I had to re-try upgrading a couple of 
times, some of the failures were:

 - rudder-agent 'Text file busy' error while updating cf-agent (that's #2792 I 
think), re-trying fixed it
 - rudder-server-root - the problem here was that there was an installed but 
unconfigured version from yesterday, couldn't configure today because newer 
version of rudder-webapp was already installed; fixed by running `sudo dpkg --
configure --force-depends rudder-server-root` and then `sudo apt-get install 
rudder-server-root`
 - rudder-webapp error from git commit (no changes to commit), this was due to 
deletion of techniques/system/distributePolicy/1.0/logrotate.st, fixed by doing 
git rm and git commit manually, then re-trying

> I've tried to reproduce your problem without success.
> Could it be possible to have a dump of your LDAP base with
> /opt/rudder/sbin/slapcat , please ?

Yeah, I imagine this would be non-reproducible :)  I may have done something 
wrong during upgrade, just trying to figure out if this can be fixed without 
starting over.

Regarding the LDAP dump, I'd rather not send the entire thing over email 
(security reasons), is there something specific I can check?

Looking through the LDAP dump I can actually see the rules and directives 
there, e.g. here's one rule:


dn: ruleId=441d550b-4a3a-4c99-
a710-14860667dfd0,ou=Rules,ou=Rudder,cn=rudder-c
 onfiguration
ruleId: 441d550b-4a3a-4c99-a710-14860667dfd0
objectClass: rule
objectClass: top
isEnabled: TRUE
isSystem: FALSE
structuralObjectClass: rule
entryUUID: f934115e-7bfb-1031-88cc-c913f245ae37
creatorsName: cn=manager,cn=rudder-configuration
createTimestamp: 20120816143948Z
ruleTarget: group:beb7373d-0a16-4f14-ae94-4655b9cfc944
description: Setup servers for CUPS server role.
longDescription: This installs CUPS server packages and configures CUPS to ena
 ble remote administration.
cn: cups-server
directiveId: 5538072f-e853-48e4-b163-160b5223fa9e
directiveId: a1924bbe-2cc9-49a0-9c81-396217f77e6e
directiveId: fcc8c069-777f-49a5-acb5-1fad13a8bf02
directiveId: e186f22d-b3da-414a-bcf9-fc88d073aac6
serial: 9
entryCSN: 20120831115436.454389Z#000000#000#000000
modifiersName: cn=manager,cn=rudder-configuration
modifyTimestamp: 20120831115436Z


I also checked the referenced group and directive IDs and they are all there.

I just noticed this problem also applies to groups, e.g. on 'Node Management -
> Groups' I see errors like "Can not find node db7463eb-
c5ac-432d-9749-39e9503ef7cb", yet the file /var/rudder/configuration-
repository/groups/db7463eb-c5ac-432d-9749-39e9503ef7cb.xml is there with 
contents:


<nodeGroup fileFormat="2">
  <id>db7463eb-c5ac-432d-9749-39e9503ef7cb</id>
  <displayName>linux</displayName>
  <description>All GNU/Linux hosts.</description>
  <query>
    {&quot;select&quot;:&quot;node&quot;,&quot;composition&quot;:&quot;And&quot;,&quot;where&quot;:
[{&quot;objectType&quot;:&quot;node&quot;,&quot;attribute&quot;:&quot;OS&quot;,&quot;comparator&quot;:&quot;eq&quot;,&quot;value&quot;:&quot;Linux&quot;}]}
  </query>
  <isDynamic>true</isDynamic>
  <nodeIds>
    <id>456601f4-0a41-4d63-898e-51a965fd5f1f</id>
    <id>adc79f12-633e-4338-94ca-681bf09c066e</id>
    <id>546de2a2-9631-423b-b8d4-987318e9baf8</id>
    <id>e472f66e-9001-465e-a83e-82fb1101e8c0</id>
    <id>f958c6d2-1cad-45a8-9579-4ff397592880</id>
  </nodeIds>
  <isEnabled>true</isEnabled>
  <isSystem>false</isSystem>
</nodeGroup>


and also in LDAP:


dn: nodeGroupId=db7463eb-
c5ac-432d-9749-39e9503ef7cb,groupCategoryId=GroupRoot
 ,ou=Rudder,cn=rudder-configuration
nodeGroupId: db7463eb-c5ac-432d-9749-39e9503ef7cb
objectClass: nodeGroup
objectClass: top
cn: linux
description: All GNU/Linux hosts.
isEnabled: TRUE
isSystem: FALSE
isDynamic: TRUE
jsonNodeGroupQuery: {"select":"node","composition":"And","where":
[{"objectType
 ":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
structuralObjectClass: nodeGroup
entryUUID: ac7cf584-62f0-1031-9c67-1508ccbe5302
creatorsName: cn=manager,cn=rudder-configuration
createTimestamp: 20120715174555Z
nodeId: f958c6d2-1cad-45a8-9579-4ff397592880
nodeId: 456601f4-0a41-4d63-898e-51a965fd5f1f
nodeId: adc79f12-633e-4338-94ca-681bf09c066e
nodeId: 546de2a2-9631-423b-b8d4-987318e9baf8
nodeId: e472f66e-9001-465e-a83e-82fb1101e8c0
entryCSN: 20120816130628.925149Z#000000#000#000000
modifiersName: cn=manager,cn=rudder-configuration
modifyTimestamp: 20120816130628Z


Thanks again for your help,
Michael


**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From francois.armand at normation.com  Thu Sep 13 17:05:05 2012
From: francois.armand at normation.com (Francois Armand)
Date: Thu, 13 Sep 2012 17:05:05 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <1565664.ixSq8sTaWe@hgis96>
References: <1368984.40l1Cv2rFx@hgis96> <1701591.RfFQ9E4M0L@hgis96>
	<50509E05.3090708@normation.com> <1565664.ixSq8sTaWe@hgis96>
Message-ID: <5051F621.3090401@normation.com>

Le 13/09/2012 13:55, Michael Gliwinski a ?crit :


Hello Michael,

To start with, we are very sorry that the migration failed, and well, 
that comfort my opinion of migration ("it's hard" ;)

One first general remark: what you see in the Rudder Web UI for Groups, 
Directives and Rules is what is in the *LDAP*.

The XML files that are in /var/rudder/configuration- repository/{groups, 
rules, directives} are there to allow archiving and import/export 
feature - so that you could bootstrap a new Rudder instance more quickly 
(by preparing these file and then importing them), or restore a previous 
state of you configuration policy (the files are in a Git repository and 
when you "export" from the administration->archive screen, that creates 
a tag that could be used latter as a snapshot point).

So, even without the /var/rudder/configuration- repository/{groups, 
rules, directives} directories, Rudder should work fine, and on the 
opposite, it's not because these files are presents that that means that 
Rudder should work.

/var/rudder/configuration- repository/*techniques* is a completely 
different beast, as it is required for Rudder to work.

But your case is *really* strange, because your LDAP content seems OK, 
and you still don't have anything on the UI, on three really different 
parts (rules, directives, groups...)

So, as for now I don't see what could lead to that behavior, I have some 
questions:

  * what was your first Rudder version installed ? A 2.3 ? A 2.4 ? If
    2.4, before beta 2 ? And how many update did you do ?
      o that will allows to know what was the starting point, and what
        could have failed along.
  * you don't see any rules or only some of them are missing ?
      o if no rules at all, are you able to create new ones (and see
        them afterward) ?
  * you don't see anything in directive screen ? Or at least some
    categories ? Or some categories and directives ?
  * you don't see any group nor category at all ? Or at least some
    groups and categories ?
  * could you check that the LDAP directory is actually up and running
    when you try to access Rudder (well, ok... but that's just to be sure :)
  * could you set the loglevel of the Webapp to "debug" to see if some
    more relevant information are available ?
      o That's on the file /opt/rudder/etc/logback.xml, changin <root
        level="info"> to <root level="debug">, and restarting Rudder
  * could you give us the number of items in the LDAP for each of groups
    (nodeGroupId), directive (directiveId) and rules (ruleId) ?


And I have some other questions (and some comments) in the following 
body of the email:

> On Wednesday 12 Sep 2012 16:36:53 Nicolas Perron wrote:
>> Ok, then it could be due to LDAP which fail during migration. Do you
>> have the output from the migration and if so could you send it to us ?
> Sorry, don't have the output anymore.  I had to re-try upgrading a couple of
> times, some of the failures were:
>
>   - rudder-agent 'Text file busy' error while updating cf-agent (that's #2792 I
> think), re-trying fixed it
>   - rudder-server-root - the problem here was that there was an installed but
> unconfigured version from yesterday, couldn't configure today because newer
> version of rudder-webapp was already installed; fixed by running `sudo dpkg --
> configure --force-depends rudder-server-root` and then `sudo apt-get install
> rudder-server-root`
>   - rudder-webapp error from git commit (no changes to commit), this was due to
> deletion of techniques/system/distributePolicy/1.0/logrotate.st, fixed by doing
> git rm and git commit manually, then re-trying
>
>> I've tried to reproduce your problem without success.
>> Could it be possible to have a dump of your LDAP base with
>> /opt/rudder/sbin/slapcat , please ?
> Yeah, I imagine this would be non-reproducible :)  I may have done something
> wrong during upgrade, just trying to figure out if this can be fixed without
> starting over.
>
> Regarding the LDAP dump, I'd rather not send the entire thing over email
> (security reasons), is there something specific I can check?


Yeah, you are right for the security problem :)
We will try to ask for more precise things now.


>
> Looking through the LDAP dump I can actually see the rules and directives
> there, e.g. here's one rule:
>
>
> dn: ruleId=441d550b-4a3a-4c99-
> a710-14860667dfd0,ou=Rules,ou=Rudder,cn=rudder-c
>   onfiguration
> ruleId: 441d550b-4a3a-4c99-a710-14860667dfd0
> objectClass: rule
> objectClass: top
> isEnabled: TRUE
> isSystem: FALSE
> structuralObjectClass: rule
> entryUUID: f934115e-7bfb-1031-88cc-c913f245ae37
> creatorsName: cn=manager,cn=rudder-configuration
> createTimestamp: 20120816143948Z
> ruleTarget: group:beb7373d-0a16-4f14-ae94-4655b9cfc944
> description: Setup servers for CUPS server role.
> longDescription: This installs CUPS server packages and configures CUPS to ena
>   ble remote administration.
> cn: cups-server
> directiveId: 5538072f-e853-48e4-b163-160b5223fa9e
> directiveId: a1924bbe-2cc9-49a0-9c81-396217f77e6e
> directiveId: fcc8c069-777f-49a5-acb5-1fad13a8bf02
> directiveId: e186f22d-b3da-414a-bcf9-fc88d073aac6
> serial: 9
> entryCSN: 20120831115436.454389Z#000000#000#000000
> modifiersName: cn=manager,cn=rudder-configuration
> modifyTimestamp: 20120831115436Z
>
>
> I also checked the referenced group and directive IDs and they are all there.
>
> I just noticed this problem also applies to groups, e.g. on 'Node Management -
>> Groups' I see errors like "Can not find node db7463eb-
> c5ac-432d-9749-39e9503ef7cb", yet the file /var/rudder/configuration-
> repository/groups/db7463eb-c5ac-432d-9749-39e9503ef7cb.xml is there with
> contents:

As explain at the beginning, what you should look for is the LDAP entry 
with RDN nodeGroupId=beb7373d-0a16-4f14-ae94-4655b9cfc944, as you 
checked just after.

>
>
> <nodeGroup fileFormat="2">
>    <id>db7463eb-c5ac-432d-9749-39e9503ef7cb</id>
>    <displayName>linux</displayName>
>    <description>All GNU/Linux hosts.</description>
>    <query>
>      {&quot;select&quot;:&quot;node&quot;,&quot;composition&quot;:&quot;And&quot;,&quot;where&quot;:
> [{&quot;objectType&quot;:&quot;node&quot;,&quot;attribute&quot;:&quot;OS&quot;,&quot;comparator&quot;:&quot;eq&quot;,&quot;value&quot;:&quot;Linux&quot;}]}
>    </query>
>    <isDynamic>true</isDynamic>
>    <nodeIds>
>      <id>456601f4-0a41-4d63-898e-51a965fd5f1f</id>
>      <id>adc79f12-633e-4338-94ca-681bf09c066e</id>
>      <id>546de2a2-9631-423b-b8d4-987318e9baf8</id>
>      <id>e472f66e-9001-465e-a83e-82fb1101e8c0</id>
>      <id>f958c6d2-1cad-45a8-9579-4ff397592880</id>
>    </nodeIds>
>    <isEnabled>true</isEnabled>
>    <isSystem>false</isSystem>
> </nodeGroup>

Here, you perhaps found one bug in our migration script: the fileFormat 
should be "3". That seems to be an actual problem in our script, not one 
tied to your scenario, and one that should not leads to what you see. It 
should lead to "bad file format" in Event Log screen and if you try to 
restore that archive.

>
>
> and also in LDAP:
>
>
> dn: nodeGroupId=db7463eb-
> c5ac-432d-9749-39e9503ef7cb,groupCategoryId=GroupRoot
>   ,ou=Rudder,cn=rudder-configuration
> nodeGroupId: db7463eb-c5ac-432d-9749-39e9503ef7cb
> objectClass: nodeGroup
> objectClass: top
> cn: linux
> description: All GNU/Linux hosts.
> isEnabled: TRUE
> isSystem: FALSE
> isDynamic: TRUE
> jsonNodeGroupQuery: {"select":"node","composition":"And","where":
> [{"objectType
>   ":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
> structuralObjectClass: nodeGroup
> entryUUID: ac7cf584-62f0-1031-9c67-1508ccbe5302
> creatorsName: cn=manager,cn=rudder-configuration
> createTimestamp: 20120715174555Z
> nodeId: f958c6d2-1cad-45a8-9579-4ff397592880
> nodeId: 456601f4-0a41-4d63-898e-51a965fd5f1f
> nodeId: adc79f12-633e-4338-94ca-681bf09c066e
> nodeId: 546de2a2-9631-423b-b8d4-987318e9baf8
> nodeId: e472f66e-9001-465e-a83e-82fb1101e8c0
> entryCSN: 20120816130628.925149Z#000000#000#000000
> modifiersName: cn=manager,cn=rudder-configuration
> modifyTimestamp: 20120816130628Z


Some more random ideas :

  * could you check that you don't have duplicated names (cn) for
    groups, directives and rules ? That is not allowed (to not lead to
    confusion in reports). That should not lead to what you see, but at
    that point, that could be interesting to check !
  * the fact that you don't see any directives nor techniques nor
    categories in the Directive screen seems du to the fact that all
    root categories are not found by Rudder in the LDAP, as displayed in
    the logs in your first email. Could you check that they are actually
    here, and that their DN look like: *techniqueCategoryId=XXX,
    techn**iqueCategoryId=Active Techniques, ou=Rudder,
    cn=rudder-configuration*
      o with XXX being exactly one the missing categories you see in
        logs (*applications, fileDistribution, etc)
        *

Well... And I don't know what to ask you anymore for now, so that should 
be the end of that mail !

Michael, thanks again for time you are taking to report these 
information, they are very much valuable for us (and hopefully, we will 
be able to help you).

Cheers,

-- 
------------------------------------------------------------------------
*Fran?ois ARMAND*
/Directeur de la R&D/
Normation <http://www.normation.com>
------------------------------------------------------------------------
*87 rue de Turbigo, 75003 Paris, France*
Telephone: 	+33 (0)1 83 62 99 23
Mobile: 	+33 (0)6 63 37 60 55
------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120913/22733164/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sig-normation-logo-square.png
Type: image/png
Size: 3503 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120913/22733164/attachment-0001.png>

From nicolas.charles at normation.com  Thu Sep 13 17:24:38 2012
From: nicolas.charles at normation.com (Nicolas Charles)
Date: Thu, 13 Sep 2012 17:24:38 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5051F621.3090401@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <1701591.RfFQ9E4M0L@hgis96>
	<50509E05.3090708@normation.com> <1565664.ixSq8sTaWe@hgis96>
	<5051F621.3090401@normation.com>
Message-ID: <5051FAB6.7060307@normation.com>

On 13/09/2012 17:05, Francois Armand wrote:
> Le 13/09/2012 13:55, Michael Gliwinski a ?crit :
>
>
> Hello Michael,
>
> To start with, we are very sorry that the migration failed, and well, 
> that comfort my opinion of migration ("it's hard" ;)
>
> One first general remark: what you see in the Rudder Web UI for 
> Groups, Directives and Rules is what is in the *LDAP*.
>
> The XML files that are in /var/rudder/configuration- 
> repository/{groups, rules, directives} are there to allow archiving 
> and import/export feature - so that you could bootstrap a new Rudder 
> instance more quickly (by preparing these file and then importing 
> them), or restore a previous state of you configuration policy (the 
> files are in a Git repository and when you "export" from the 
> administration->archive screen, that creates a tag that could be used 
> latter as a snapshot point).
>
> So, even without the /var/rudder/configuration- repository/{groups, 
> rules, directives} directories, Rudder should work fine, and on the 
> opposite, it's not because these files are presents that that means 
> that Rudder should work.
>
> /var/rudder/configuration- repository/*techniques* is a completely 
> different beast, as it is required for Rudder to work.
>
> But your case is *really* strange, because your LDAP content seems OK, 
> and you still don't have anything on the UI, on three really different 
> parts (rules, directives, groups...)
>
> So, as for now I don't see what could lead to that behavior, I have 
> some questions:
>
>   * what was your first Rudder version installed ? A 2.3 ? A 2.4 ? If
>     2.4, before beta 2 ? And how many update did you do ?
>       o that will allows to know what was the starting point, and what
>         could have failed along.
>   * you don't see any rules or only some of them are missing ?
>       o if no rules at all, are you able to create new ones (and see
>         them afterward) ?
>   * you don't see anything in directive screen ? Or at least some
>     categories ? Or some categories and directives ?
>   * you don't see any group nor category at all ? Or at least some
>     groups and categories ?
>   * could you check that the LDAP directory is actually up and running
>     when you try to access Rudder (well, ok... but that's just to be
>     sure :)
>   * could you set the loglevel of the Webapp to "debug" to see if some
>     more relevant information are available ?
>       o That's on the file /opt/rudder/etc/logback.xml, changin <root
>         level="info"> to <root level="debug">, and restarting Rudder
>   * could you give us the number of items in the LDAP for each of
>     groups (nodeGroupId), directive (directiveId) and rules (ruleId) ?
>
>
> And I have some other questions (and some comments) in the following 
> body of the email:
>
>> On Wednesday 12 Sep 2012 16:36:53 Nicolas Perron wrote:
>>> Ok, then it could be due to LDAP which fail during migration. Do you
>>> have the output from the migration and if so could you send it to us ?
>> Sorry, don't have the output anymore.  I had to re-try upgrading a couple of
>> times, some of the failures were:
>>
>>   - rudder-agent 'Text file busy' error while updating cf-agent (that's #2792 I
>> think), re-trying fixed it
>>   - rudder-server-root - the problem here was that there was an installed but
>> unconfigured version from yesterday, couldn't configure today because newer
>> version of rudder-webapp was already installed; fixed by running `sudo dpkg --
>> configure --force-depends rudder-server-root` and then `sudo apt-get install
>> rudder-server-root`
>>   - rudder-webapp error from git commit (no changes to commit), this was due to
>> deletion of techniques/system/distributePolicy/1.0/logrotate.st, fixed by doing
>> git rm and git commit manually, then re-trying
>>
>>> I've tried to reproduce your problem without success.
>>> Could it be possible to have a dump of your LDAP base with
>>> /opt/rudder/sbin/slapcat , please ?
>> Yeah, I imagine this would be non-reproducible :)  I may have done something
>> wrong during upgrade, just trying to figure out if this can be fixed without
>> starting over.
>>
>> Regarding the LDAP dump, I'd rather not send the entire thing over email
>> (security reasons), is there something specific I can check?
>
>
> Yeah, you are right for the security problem :)
> We will try to ask for more precise things now.
>
>
>> Looking through the LDAP dump I can actually see the rules and directives
>> there, e.g. here's one rule:
>>
>>
>> dn: ruleId=441d550b-4a3a-4c99-
>> a710-14860667dfd0,ou=Rules,ou=Rudder,cn=rudder-c
>>   onfiguration
>> ruleId: 441d550b-4a3a-4c99-a710-14860667dfd0
>> objectClass: rule
>> objectClass: top
>> isEnabled: TRUE
>> isSystem: FALSE
>> structuralObjectClass: rule
>> entryUUID: f934115e-7bfb-1031-88cc-c913f245ae37
>> creatorsName: cn=manager,cn=rudder-configuration
>> createTimestamp: 20120816143948Z
>> ruleTarget: group:beb7373d-0a16-4f14-ae94-4655b9cfc944
>> description: Setup servers for CUPS server role.
>> longDescription: This installs CUPS server packages and configures CUPS to ena
>>   ble remote administration.
>> cn: cups-server
>> directiveId: 5538072f-e853-48e4-b163-160b5223fa9e
>> directiveId: a1924bbe-2cc9-49a0-9c81-396217f77e6e
>> directiveId: fcc8c069-777f-49a5-acb5-1fad13a8bf02
>> directiveId: e186f22d-b3da-414a-bcf9-fc88d073aac6
>> serial: 9
>> entryCSN: 20120831115436.454389Z#000000#000#000000
>> modifiersName: cn=manager,cn=rudder-configuration
>> modifyTimestamp: 20120831115436Z
>>
>>
>> I also checked the referenced group and directive IDs and they are all there.
>>
>> I just noticed this problem also applies to groups, e.g. on 'Node Management -
>>> Groups' I see errors like "Can not find node db7463eb-
>> c5ac-432d-9749-39e9503ef7cb", yet the file /var/rudder/configuration-
>> repository/groups/db7463eb-c5ac-432d-9749-39e9503ef7cb.xml is there with
>> contents:
>
> As explain at the beginning, what you should look for is the LDAP 
> entry with RDN nodeGroupId=beb7373d-0a16-4f14-ae94-4655b9cfc944, as 
> you checked just after.
>
>>
>> <nodeGroup fileFormat="2">
>>    <id>db7463eb-c5ac-432d-9749-39e9503ef7cb</id>
>>    <displayName>linux</displayName>
>>    <description>All GNU/Linux hosts.</description>
>>    <query>
>>      {&quot;select&quot;:&quot;node&quot;,&quot;composition&quot;:&quot;And&quot;,&quot;where&quot;:
>> [{&quot;objectType&quot;:&quot;node&quot;,&quot;attribute&quot;:&quot;OS&quot;,&quot;comparator&quot;:&quot;eq&quot;,&quot;value&quot;:&quot;Linux&quot;}]}
>>    </query>
>>    <isDynamic>true</isDynamic>
>>    <nodeIds>
>>      <id>456601f4-0a41-4d63-898e-51a965fd5f1f</id>
>>      <id>adc79f12-633e-4338-94ca-681bf09c066e</id>
>>      <id>546de2a2-9631-423b-b8d4-987318e9baf8</id>
>>      <id>e472f66e-9001-465e-a83e-82fb1101e8c0</id>
>>      <id>f958c6d2-1cad-45a8-9579-4ff397592880</id>
>>    </nodeIds>
>>    <isEnabled>true</isEnabled>
>>    <isSystem>false</isSystem>
>> </nodeGroup>
>
> Here, you perhaps found one bug in our migration script: the 
> fileFormat should be "3". That seems to be an actual problem in our 
> script, not one tied to your scenario, and one that should not leads 
> to what you see. It should lead to "bad file format" in Event Log 
> screen and if you try to restore that archive.
>
>>
>> and also in LDAP:
>>
>>
>> dn: nodeGroupId=db7463eb-
>> c5ac-432d-9749-39e9503ef7cb,groupCategoryId=GroupRoot
>>   ,ou=Rudder,cn=rudder-configuration
>> nodeGroupId: db7463eb-c5ac-432d-9749-39e9503ef7cb
>> objectClass: nodeGroup
>> objectClass: top
>> cn: linux
>> description: All GNU/Linux hosts.
>> isEnabled: TRUE
>> isSystem: FALSE
>> isDynamic: TRUE
>> jsonNodeGroupQuery: {"select":"node","composition":"And","where":
>> [{"objectType
>>   ":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
>> structuralObjectClass: nodeGroup
>> entryUUID: ac7cf584-62f0-1031-9c67-1508ccbe5302
>> creatorsName: cn=manager,cn=rudder-configuration
>> createTimestamp: 20120715174555Z
>> nodeId: f958c6d2-1cad-45a8-9579-4ff397592880
>> nodeId: 456601f4-0a41-4d63-898e-51a965fd5f1f
>> nodeId: adc79f12-633e-4338-94ca-681bf09c066e
>> nodeId: 546de2a2-9631-423b-b8d4-987318e9baf8
>> nodeId: e472f66e-9001-465e-a83e-82fb1101e8c0
>> entryCSN: 20120816130628.925149Z#000000#000#000000
>> modifiersName: cn=manager,cn=rudder-configuration
>> modifyTimestamp: 20120816130628Z
>
>
> Some more random ideas :
>
>   * could you check that you don't have duplicated names (cn) for
>     groups, directives and rules ? That is not allowed (to not lead to
>     confusion in reports). That should not lead to what you see, but
>     at that point, that could be interesting to check !
>   * the fact that you don't see any directives nor techniques nor
>     categories in the Directive screen seems du to the fact that all
>     root categories are not found by Rudder in the LDAP, as displayed
>     in the logs in your first email. Could you check that they are
>     actually here, and that their DN look like:
>     *techniqueCategoryId=XXX, techn**iqueCategoryId=Active Techniques,
>     ou=Rudder, cn=rudder-configuration*
>       o with XXX being exactly one the missing categories you see in
>         logs (*applications, fileDistribution, etc)
>         *
>
> Well... And I don't know what to ask you anymore for now, so that 
> should be the end of that mail !
>
> Michael, thanks again for time you are taking to report these 
> information, they are very much valuable for us (and hopefully, we 
> will be able to help you).
>
Michael,

Could you also send us a copy of you configuration.properties files 
(which should be in /opt/rudder/etc) ? (and don't forget to remove the 
LDAP and Postgresql password)
Maybe there's something there which could help us understand what is not 
right.

Thank you very much,
Nicolas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120913/4352a3bb/attachment.html>

From nicolas.charles at normation.com  Thu Sep 13 17:31:33 2012
From: nicolas.charles at normation.com (Nicolas Charles)
Date: Thu, 13 Sep 2012 17:31:33 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5051FAB6.7060307@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <1701591.RfFQ9E4M0L@hgis96>
	<50509E05.3090708@normation.com> <1565664.ixSq8sTaWe@hgis96>
	<5051F621.3090401@normation.com> <5051FAB6.7060307@normation.com>
Message-ID: <5051FC55.301@normation.com>

On 13/09/2012 17:24, Nicolas Charles wrote:
> Michael,
>
>
> Could you also send us a copy of you configuration.properties files 
> (which should be in /opt/rudder/etc) ? (and don't forget to remove the 
> LDAP and Postgresql password)
Of course, I meant rudder-web.properties, sorry about that !

Nicolas

From nicolas.perron at normation.com  Fri Sep 14 13:54:42 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Fri, 14 Sep 2012 13:54:42 +0200
Subject: [rudder-dev] Echec de Rudder en 2.4: [Rudder] Build failed in
 Jenkins: Rudder-2.4-branch #274
Message-ID: <50531B02.4060703@normation.com>

Hello les devs,

Rudder a ?chou? lors de son build en 2.4. Vous pourriez regarder de quoi
'il s'agit ?

unning com.normation.rudder.migration.TestManageMigration_10_2
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.298 sec
Running com.normation.rudder.migration.TestManageMigration_2_3
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.689 sec
Running com.normation.rudder.migration.TestXmlMigration_10_3
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.112 sec
Running com.normation.rudder.migration.TestXmlMigration_10_2
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.062 sec

Results :

Failed tests:   A component\u002C with a cfengine keys should::return a component globally repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
  A component\u002C with a cfengine keys should::return a component with both cfengine keys repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
  A component\u002C with distinguishable keys should::return a component globally repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
  A component\u002C with distinguishable keys should::return a component with the cfengine keys repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false

Tests run: 238, Failures: 4, Errors: 0, Skipped: 0



merci

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================


From nicolas.perron at normation.com  Fri Sep 14 15:14:52 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Fri, 14 Sep 2012 15:14:52 +0200
Subject: [rudder-dev] Echec de Rudder en 2.4: [Rudder] Build failed in
 Jenkins: Rudder-2.4-branch #274
In-Reply-To: <50531B02.4060703@normation.com>
References: <50531B02.4060703@normation.com>
Message-ID: <50532DCC.5030409@normation.com>


> Hello les devs,
>
> Rudder a ?chou? lors de son build en 2.4. Vous pourriez regarder de quoi
> 'il s'agit ?
>
> unning com.normation.rudder.migration.TestManageMigration_10_2
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.298 sec
> Running com.normation.rudder.migration.TestManageMigration_2_3
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.689 sec
> Running com.normation.rudder.migration.TestXmlMigration_10_3
> Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.112 sec
> Running com.normation.rudder.migration.TestXmlMigration_10_2
> Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.062 sec
>
> Results :
>
> Failed tests:   A component\u002C with a cfengine keys should::return a component globally repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
>   A component\u002C with a cfengine keys should::return a component with both cfengine keys repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
>   A component\u002C with distinguishable keys should::return a component globally repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
>   A component\u002C with distinguishable keys should::return a component with the cfengine keys repaired(com.normation.rudder.domain.reports.bean.ExecutionBatchTest): the value is false
>
> Tests run: 238, Failures: 4, Errors: 0, Skipped: 0
>
>
>
> merci
>
Sorry, for the mistake.  My bad !

Please, ignore this message.

Thank you

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================


From Michael.Gliwinski at henderson-group.com  Sat Sep 15 13:21:34 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Sat, 15 Sep 2012 12:21:34 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5051FC55.301@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <5051FAB6.7060307@normation.com> 
	<5051FC55.301@normation.com>
Message-ID: <34559479.lI6BenurmB@hgis96>

On Thursday 13 Sep 2012 17:31:33 Nicolas Charles wrote:
> > Could you also send us a copy of you configuration.properties files
> > (which should be in /opt/rudder/etc) ? (and don't forget to remove the
> > LDAP and Postgresql password)
> 
> Of course, I meant rudder-web.properties, sorry about that !

No problem, here it is (I stripped comments for brevity):


base.url=http://CNAME/rudder
rudder.dir.lock=/var/rudder/lock/
rudder.endpoint.cmdb=http://localhost:8080/endpoint/upload/
ldap.host=localhost
ldap.port=389
ldap.authdn=cn=manager,cn=rudder-configuration
ldap.authpw=SECRET
ldap.node.base=cn=rudder-configuration
ldap.rudder.base=ou=Rudder, cn=rudder-configuration
ldap.inventories.software.basedn=ou=Inventories, cn=rudder-configuration
ldap.inventories.accepted.basedn=ou=Accepted Inventories, ou=Inventories, 
cn=rudder-configuration
ldap.inventories.pending.basedn=ou=Pending Inventories, ou=Inventories, 
cn=rudder-configuration
ldap.inventories.removed.basedn=ou=Removed Inventories, ou=Inventories, 
cn=rudder-configuration
rudder.jdbc.driver=org.postgresql.Driver
rudder.jdbc.url=jdbc:postgresql://localhost:5432/rudder
rudder.jdbc.username=rudder
rudder.jdbc.password=SECRET
rudder.webdav.user=rudder
rudder.webdav.password=SECRET
rudder.community.port=5309
rudder.community.checkpromises.command=/var/rudder/cfengine-
community/bin/cf-promises
rudder.nova.checkpromises.command=/bin/true
rudder.dir.licensesFolder=/opt/rudder/etc/licenses
rudder.dir.backup=/var/rudder/backup/
rudder.dir.shared.files.folder=/var/rudder/configuration-repository/shared-files
rudder.dir.techniques=/var/rudder/configuration-repository/techniques
rudder.dir.dependencies=/var/rudder/tools
rudder.batch.techniqueLibrary.updateInterval=5
rudder.techniqueLibrary.git.refs.path=refs/heads/master
rudder.dir.gitRoot=/var/rudder/configuration-repository
rudder.autoArchiveItems=true
rudder.autoDeployOnModification=true
rudder.ui.changeMessage.enabled=true
rudder.ui.changeMessage.mandatory=false
rudder.ui.changeMessage.explanation=Please enter a message explaining the 
reason for this change.
rudder.batch.dyngroup.updateInterval=5
rudder.rest.allowNonAuthenticatedUser=true
history.inventories.rootdir=/var/rudder/inventories/historical
upload.root.directory=/var/rudder/files/
rudder.dir.uploaded.file.sharing=/var/rudder/files/
bin.emergency.stop=/opt/rudder/bin/cfe-red-button.sh
history.inventories.enable=true


**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From Michael.Gliwinski at henderson-group.com  Sat Sep 15 15:16:05 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Sat, 15 Sep 2012 14:16:05 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5051F621.3090401@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <1565664.ixSq8sTaWe@hgis96> 
	<5051F621.3090401@normation.com>
Message-ID: <2453252.OBPoUvhTbU@hgis96>

Re-sending to the list too :P

Many thanks for helping out guys :)  Replies inline.

On Thursday 13 Sep 2012 17:05:05 you wrote:
> To start with, we are very sorry that the migration failed, and well,
> that comfort my opinion of migration ("it's hard" ;)

No worries, I agree :)

> One first general remark: what you see in the Rudder Web UI for Groups,
> Directives and Rules is what is in the *LDAP*.
> 
> The XML files that are in /var/rudder/configuration- repository/{groups,
> rules, directives} are there to allow archiving and import/export
> feature - so that you could bootstrap a new Rudder instance more quickly
> (by preparing these file and then importing them), or restore a previous
> state of you configuration policy (the files are in a Git repository and
> when you "export" from the administration->archive screen, that creates
> a tag that could be used latter as a snapshot point).
> 
> So, even without the /var/rudder/configuration- repository/{groups,
> rules, directives} directories, Rudder should work fine, and on the
> opposite, it's not because these files are presents that that means that
> Rudder should work.

OK, this is very useful to know, thanks.  I wasn't sure about that.

> /var/rudder/configuration- repository/*techniques* is a completely
> different beast, as it is required for Rudder to work.
> 
> But your case is *really* strange, because your LDAP content seems OK,
> and you still don't have anything on the UI, on three really different
> parts (rules, directives, groups...)
> 
> So, as for now I don't see what could lead to that behavior, I have some
> questions:
> 
>   * what was your first Rudder version installed ? A 2.3 ? A 2.4 ? If
>     2.4, before beta 2 ? And how many update did you do ?
>       o that will allows to know what was the starting point, and what
>         could have failed along.

First version was 2.4 alpha7, then upgraded to beta1 -> beta2 -> beta3 -> 
beta4 and finally to nightly (on 2012-09-11/12).

>   * you don't see any rules or only some of them are missing ?
>       o if no rules at all, are you able to create new ones (and see
>         them afterward) ?

No rules at all (empty table with "No matching rules!" line).

Cannot create a directive as there are no active techniques.  E.g. see the 
attached screenshot of Techniques screen.  When I try to create a new Active 
Technique category, I get an error "The form contains some errors, please 
correct them
An error occurred while fetching the parent category" (even if parent category 
is "Root of active technique's library").

I can create a rule, and it is visible afterward.  I tried to check if the 
resulting entry in LDAP is somehow different from other rules, but it doesn't 
look like it:


dn: ruleId=de422072-3297-44e3-86bb-
d943340dd76e,ou=Rules,ou=Rudder,cn=rudder-c
 onfiguration
ruleId: de422072-3297-44e3-86bb-d943340dd76e
objectClass: rule
objectClass: top
cn: test-rule
isEnabled: TRUE
isSystem: FALSE
serial: 0


>   * you don't see anything in directive screen ? Or at least some
>     categories ? Or some categories and directives ?

Only "Root of active technique's library" with no categories or directives 
under it.

>   * you don't see any group nor category at all ? Or at least some
>     groups and categories ?

There is "Root of the group and group categories" and a couple of entries 
under it (one for each group that was defined) but each one saying "Can not find 
node UUID" in red.

>   * could you check that the LDAP directory is actually up and running
>     when you try to access Rudder (well, ok... but that's just to be sure :)

Checked with /opt/rudder/bin/ldapsearch -H ldap//localhost -x ... (taken one 
check from rudder-upgrade script) and it works.

> * could you set the loglevel of the Webapp to "debug" to see if some more
> relevant information are available ?
>       o That's on the file /opt/rudder/etc/logback.xml, changin <root
>         level="info"> to <root level="debug">, and restarting Rudder

There is only one additional message in webapp/DATE.stderrout.log that 
indicates any kind of error:

[2012-09-14 12:39:14] DEBUG 
com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
- Dynamic group update started at 2012/09/14 12:39:14, ended at 2012/09/14 
12:39:14
[2012-09-14 12:39:14] ERROR 
com.normation.rudder.batch.UpdateDynamicGroups$LAUpdateDyngroupManager 
- Error when updating dynamic group NodeGroupId(db7463eb-
c5ac-432d-9749-39e9503ef7cb) : <- Error when retrieving the entry for 
NodeGroup 'NodeGroupId(db7463eb-c5ac-432d-9749-39e9503ef7cb)'

When I navigate to directives or groups in the web interface I only get the 
same errors as before 
("com.normation.rudder.web.snippet.configuration.DirectiveManagement - Error 
when displaying category: Error while fetching Active Technique category" and 
"com.normation.rudder.web.snippet.node.Groups - Error while fetching 
Technique category NodeGroupCategoryId(SystemGroups) <- Entry with ID 
'NodeGroupCategoryId(SystemGroups)' was not found").

Nothing new in core/rudder-webapp.log, but I did notice 52 errors at startup 
like:

Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '6' <- Not migrating eventLog with [id: 6] [type:
 SuccessfulDeployment]: no handler for that type.
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '11' <- Not migrating eventLog with [id: 11] [typ
e: SuccessfulDeployment]: no handler for that type.
...
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '120' <- Not migrating eventLog with [id: 120] [t
ype: ExportRules]: no handler for that type.
ype: ExportTechniqueLibrary]: no handler for that type.
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '122' <- Not migrating eventLog with [id: 122] [t
ype: ExportGroups]: no handler for that type.
Sep 14 11:59:34 centostest rudder[migration]: [ERROR] Error when trying to 
migrate event log with id '123' <- Not migrating eventLog with [id: 123] [t
ype: ExportFullArchive]: no handler for that type.
...
Sep 14 11:59:34 centostest rudder[migration]: [INFO] Migration from EventLog 
fileFormat from '2' to '3' done, 0 EventLogs migrated

not sure if they are relevant?

>   * could you give us the number of items in the LDAP for each of groups
>     (nodeGroupId), directive (directiveId) and rules (ruleId) ?

Do you mean counts of actual objects (i.e. objectClass=nodeGroup, 
objectClass=directive, objectClass=rule)?  If yes:

There are 6 nodeGroupId, one "nodeGroupId: hasPolicyServer-root" and the rest 
with UUIDs.

22 directives:

directiveId: inventory-all
directiveId: root-distributePolicy
directiveId: common-root
19 other directives with directiveId: UUID

10 rules:

ruleId: inventory-all
ruleId: root-DP
ruleId: hasPolicyServer-root
7 other rules with ruleId: UUID

> And I have some other questions (and some comments) in the following
> 
> body of the email:
> > On Wednesday 12 Sep 2012 16:36:53 Nicolas Perron wrote:
...
> > <nodeGroup fileFormat="2">
> > 
> >    <id>db7463eb-c5ac-432d-9749-39e9503ef7cb</id>
> >    <displayName>linux</displayName>
> >    <description>All GNU/Linux hosts.</description>
> >    <query>
> >    
> >      
{&quot;select&quot;:&quot;node&quot;,&quot;composition&quot;:&quot;And&quot;,&quot;where&quot;:
> > 
[{&quot;objectType&quot;:&quot;node&quot;,&quot;attribute&quot;:&quot;OS
> > 
&quot;,&quot;comparator&quot;:&quot;eq&quot;,&quot;value&quot;:&quot;Lin
> > ux&quot;}]}> 
> >    </query>
> >    <isDynamic>true</isDynamic>
> >    <nodeIds>
> >    
> >      <id>456601f4-0a41-4d63-898e-51a965fd5f1f</id>
> >      <id>adc79f12-633e-4338-94ca-681bf09c066e</id>
> >      <id>546de2a2-9631-423b-b8d4-987318e9baf8</id>
> >      <id>e472f66e-9001-465e-a83e-82fb1101e8c0</id>
> >      <id>f958c6d2-1cad-45a8-9579-4ff397592880</id>
> >    
> >    </nodeIds>
> >    <isEnabled>true</isEnabled>
> >    <isSystem>false</isSystem>
> > 
> > </nodeGroup>
> 
> Here, you perhaps found one bug in our migration script: the fileFormat
> should be "3". That seems to be an actual problem in our script, not one
> tied to your scenario, and one that should not leads to what you see. It
> should lead to "bad file format" in Event Log screen and if you try to
> restore that archive.

Which script is that?  I checked rudder-upgrade and postinst scripts and 
didn't see any of them trying to update the xml files.

> > and also in LDAP:
> > 
> > 
> > dn: nodeGroupId=db7463eb-
> > c5ac-432d-9749-39e9503ef7cb,groupCategoryId=GroupRoot
> > 
> >   ,ou=Rudder,cn=rudder-configuration
> > 
> > nodeGroupId: db7463eb-c5ac-432d-9749-39e9503ef7cb
> > objectClass: nodeGroup
> > objectClass: top
> > cn: linux
> > description: All GNU/Linux hosts.
> > isEnabled: TRUE
> > isSystem: FALSE
> > isDynamic: TRUE
> > jsonNodeGroupQuery: {"select":"node","composition":"And","where":
> > [{"objectType
> > 
> >   ":"node","attribute":"OS","comparator":"eq","value":"Linux"}]}
> > 
> > structuralObjectClass: nodeGroup
> > entryUUID: ac7cf584-62f0-1031-9c67-1508ccbe5302
> > creatorsName: cn=manager,cn=rudder-configuration
> > createTimestamp: 20120715174555Z
> > nodeId: f958c6d2-1cad-45a8-9579-4ff397592880
> > nodeId: 456601f4-0a41-4d63-898e-51a965fd5f1f
> > nodeId: adc79f12-633e-4338-94ca-681bf09c066e
> > nodeId: 546de2a2-9631-423b-b8d4-987318e9baf8
> > nodeId: e472f66e-9001-465e-a83e-82fb1101e8c0
> > entryCSN: 20120816130628.925149Z#000000#000#000000
> > modifiersName: cn=manager,cn=rudder-configuration
> > modifyTimestamp: 20120816130628Z
> 
> Some more random ideas :
> 
>   * could you check that you don't have duplicated names (cn) for
>     groups, directives and rules ? That is not allowed (to not lead to
>     confusion in reports). That should not lead to what you see, but at
>     that point, that could be interesting to check !

Confirmed that there are no duplicates.

>   * the fact that you don't see any directives nor techniques nor
>     categories in the Directive screen seems du to the fact that all
>     root categories are not found by Rudder in the LDAP, as displayed in
>     the logs in your first email. Could you check that they are actually
>     here, and that their DN look like: *techniqueCategoryId=XXX,
>     techn**iqueCategoryId=Active Techniques, ou=Rudder,
>     cn=rudder-configuration*
>       o with XXX being exactly one the missing categories you see in
>         logs (*applications, fileDistribution, etc)
>         *

Yes, they are all there:


dn: techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active 
Techniques,
 ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=applications,techniqueCategoryId=Active 
Techniques,ou=
 Rudder,cn=rudder-configuration

dn: techniqueCategoryId=fileDistribution,techniqueCategoryId=Active 
Techniques
 ,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=systemSettings,techniqueCategoryId=Active 
Techniques,o
 u=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=security,techniqueCategoryId=systemSettings,techniqueC
 ategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=remoteAccess,techniqueCategoryId=systemSettings,techni
 queCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=misc,techniqueCategoryId=systemSettings,techniqueCateg
 oryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=userManagement,techniqueCategoryId=systemSettings,tech
 niqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=process,techniqueCategoryId=systemSettings,techniqueCa
 tegoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=systemManagement,techniqueCategoryId=systemSettings,te
 chniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: 
techniqueCategoryId=networking,techniqueCategoryId=systemSettings,techniqu
 eCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=jobScheduling,techniqueCategoryId=Active 
Techniques,ou
 =Rudder,cn=rudder-configuration

dn: techniqueCategoryId=fileConfiguration,techniqueCategoryId=Active 
Technique
 s,ou=Rudder,cn=rudder-configuration

dn: techniqueCategoryId=fileSecurity,techniqueCategoryId=fileConfiguration,tec
 hniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuration

> Well... And I don't know what to ask you anymore for now, so that should
> be the end of that mail !

That's a puzzling one, isn't it? :D
**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rudder-active-techniques-empty.png
Type: image/png
Size: 11484 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120915/0c3bf37a/attachment-0001.png>

From nicolas.charles at normation.com  Sat Sep 15 19:06:35 2012
From: nicolas.charles at normation.com (Nicolas Charles)
Date: Sat, 15 Sep 2012 19:06:35 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <2453252.OBPoUvhTbU@hgis96>
References: <1368984.40l1Cv2rFx@hgis96> <1565664.ixSq8sTaWe@hgis96>
	<5051F621.3090401@normation.com> <2453252.OBPoUvhTbU@hgis96>
Message-ID: <5054B59B.5090708@normation.com>

Le 15/09/2012 15:16, Michael Gliwinski a ?crit :
> Re-sending to the list too :P
>
> Many thanks for helping out guys :)  Replies inline.

Hi Michael,

Thank you for the detailed answer.

This is a real puzzler, since your config and LDAP seems to be correct, 
and new and visible entry don't seem to differ from old ones... So I'll 
try a wild guess : we improved the ldap indexes in the latest revision 
(ticket http://www.rudder-project.org/redmine/issues/2751), and you said 
you had errors during the migration procedure, so maybe slapd didn't 
reindexed its entries.

Could you try to stop Rudder (/etc/init.d/rudder-server-root stop), 
reindex your ldap directory (/opt/rudder/sbin/slapindex -f 
/opt/rudder/etc/openldap/slapd.conf), restart Rudder and tell us if it 
solved your issue ?

Many thanks,
Nicolas


From francois.armand at normation.com  Sun Sep 16 23:08:02 2012
From: francois.armand at normation.com (Francois Armand)
Date: Sun, 16 Sep 2012 23:08:02 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5054B59B.5090708@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <1565664.ixSq8sTaWe@hgis96>
	<5051F621.3090401@normation.com> <2453252.OBPoUvhTbU@hgis96>
	<5054B59B.5090708@normation.com>
Message-ID: <50563FB2.50100@normation.com>

Le 15/09/2012 19:06, Nicolas Charles a ?crit :
> Le 15/09/2012 15:16, Michael Gliwinski a ?crit :
>> Re-sending to the list too :P
>>
>> Many thanks for helping out guys :)  Replies inline.
>
> Hi Michael,
>
> Thank you for the detailed answer.
>
> This is a real puzzler, since your config and LDAP seems to be 
> correct, and new and visible entry don't seem to differ from old 
> ones... So I'll try a wild guess : we improved the ldap indexes in the 
> latest revision (ticket 
> http://www.rudder-project.org/redmine/issues/2751), and you said you 
> had errors during the migration procedure, so maybe slapd didn't 
> reindexed its entries.
>
> Could you try to stop Rudder (/etc/init.d/rudder-server-root stop), 
> reindex your ldap directory (/opt/rudder/sbin/slapindex -f 
> /opt/rudder/etc/openldap/slapd.conf), restart Rudder and tell us if it 
> solved your issue ?

Yeh, that seems to be a good idea, or something like that because, as 
far as I can say, the LDAP contains what we want, be we are not able to 
see it from Rudder, and you can see it from ldapsearch... And I just 
can't understand, even with corrupted indexes. But the fact that the new 
Rule appears clearly show that it's not a configuration problem.

Ho, one more idea: could you try to clone a non-working rule, just 
changing its uuid, to see if it appears, to ?

If that does not work, the next step seems to be to make OpenLDAP log 
requests, and see what is ask, what answer it produces, and what it 
should have produce...

That's really a puzzling one...

-- 
------------------------------------------------------------------------
*Fran?ois ARMAND*
/Directeur de la R&D/
Normation <http://www.normation.com>
------------------------------------------------------------------------
*87 rue de Turbigo, 75003 Paris, France*
Telephone: 	+33 (0)1 83 62 99 23
Mobile: 	+33 (0)6 63 37 60 55
------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120916/2be666a6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sig-normation-logo-square.png
Type: image/png
Size: 3503 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120916/2be666a6/attachment.png>

From Michael.Gliwinski at henderson-group.com  Mon Sep 17 11:18:09 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Mon, 17 Sep 2012 10:18:09 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5054B59B.5090708@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <2453252.OBPoUvhTbU@hgis96> 
	<5054B59B.5090708@normation.com>
Message-ID: <2431135.91aUMhcfEy@hgis96>

On Saturday 15 Sep 2012 19:06:35 Nicolas Charles wrote:
> This is a real puzzler, since your config and LDAP seems to be correct,
> and new and visible entry don't seem to differ from old ones... So I'll
> try a wild guess : we improved the ldap indexes in the latest revision
> (ticket http://www.rudder-project.org/redmine/issues/2751), and you said
> you had errors during the migration procedure, so maybe slapd didn't
> reindexed its entries.
> 
> Could you try to stop Rudder (/etc/init.d/rudder-server-root stop),
> reindex your ldap directory (/opt/rudder/sbin/slapindex -f
> /opt/rudder/etc/openldap/slapd.conf), restart Rudder and tell us if it
> solved your issue ?

Ha, you nailed it!  After re-indexing everything (active techniques, 
directives, rules, groups) is visible in the web interface again.

That's brilliant, thanks for help everyone :)


**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From francois.armand at normation.com  Mon Sep 17 11:32:04 2012
From: francois.armand at normation.com (Francois Armand)
Date: Mon, 17 Sep 2012 11:32:04 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <2431135.91aUMhcfEy@hgis96>
References: <1368984.40l1Cv2rFx@hgis96> <2453252.OBPoUvhTbU@hgis96>
	<5054B59B.5090708@normation.com> <2431135.91aUMhcfEy@hgis96>
Message-ID: <5056EE14.6060700@normation.com>

Le 17/09/2012 11:18, Michael Gliwinski a ?crit :
> On Saturday 15 Sep 2012 19:06:35 Nicolas Charles wrote:
>> This is a real puzzler, since your config and LDAP seems to be correct,
>> and new and visible entry don't seem to differ from old ones... So I'll
>> try a wild guess : we improved the ldap indexes in the latest revision
>> (ticket http://www.rudder-project.org/redmine/issues/2751), and you said
>> you had errors during the migration procedure, so maybe slapd didn't
>> reindexed its entries.
>>
>> Could you try to stop Rudder (/etc/init.d/rudder-server-root stop),
>> reindex your ldap directory (/opt/rudder/sbin/slapindex -f
>> /opt/rudder/etc/openldap/slapd.conf), restart Rudder and tell us if it
>> solved your issue ?
> Ha, you nailed it!  After re-indexing everything (active techniques,
> directives, rules, groups) is visible in the web interface again.
>
> That's brilliant, thanks for help everyone :)
>

Ah, yes !!

Thank you so much Nicolas to think to that !
And for the future: http://www.rudder-project.org/redmine/issues/2900

And thank you Michael for all the time and explanation about that bug 
report, it's really valuable for us and Rudder !

-- 
------------------------------------------------------------------------
*Fran?ois ARMAND*
/Directeur de la R&D/
Normation <http://www.normation.com>
------------------------------------------------------------------------
*87 rue de Turbigo, 75003 Paris, France*
Telephone: 	+33 (0)1 83 62 99 23
Mobile: 	+33 (0)6 63 37 60 55
------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120917/67330d5e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sig-normation-logo-square.png
Type: image/png
Size: 3503 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120917/67330d5e/attachment.png>

From Michael.Gliwinski at henderson-group.com  Mon Sep 17 14:54:30 2012
From: Michael.Gliwinski at henderson-group.com (Michael Gliwinski)
Date: Mon, 17 Sep 2012 13:54:30 +0100
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <5056EE14.6060700@normation.com>
References: <1368984.40l1Cv2rFx@hgis96> <2431135.91aUMhcfEy@hgis96> 
	<5056EE14.6060700@normation.com>
Message-ID: <2375574.b7pqnCR8xg@hgis96>

On Monday 17 Sep 2012 11:32:04 Francois Armand wrote:
> Thank you so much Nicolas to think to that !
> And for the future: http://www.rudder-project.org/redmine/issues/2900
> 
> And thank you Michael for all the time and explanation about that bug
> report, it's really valuable for us and Rudder !

No worries.  Still not sure how I ran into this.  Actually, just found that 
apparently indexes are not rebuilt automatically if the change to indexes is 
made in slapd.conf:

http://www.openldap.org/faq/data/cache/136.html

but that seems only more confusing as then this should be easily reproducible.

One more question, unrelated to indexes, but you mentioned a migration might 
have failed because the xml files in /var/rudder/configuration-repository have 
fileFormat="2".  I can see that some of them do have fileFormat="3", but not 
all.  Is there a way to re-run this migration to update the remaining ones?


**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************


From francois.armand at normation.com  Mon Sep 17 17:30:04 2012
From: francois.armand at normation.com (Francois Armand)
Date: Mon, 17 Sep 2012 17:30:04 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <2375574.b7pqnCR8xg@hgis96>
References: <1368984.40l1Cv2rFx@hgis96> <2431135.91aUMhcfEy@hgis96>
	<5056EE14.6060700@normation.com> <2375574.b7pqnCR8xg@hgis96>
Message-ID: <505741FC.6060503@normation.com>

Le 17/09/2012 14:54, Michael Gliwinski a ?crit :
> [..] One more question, unrelated to indexes, but you mentioned a 
> migration might have failed because the xml files in 
> /var/rudder/configuration-repository have fileFormat="2". I can see 
> that some of them do have fileFormat="3", but not all. Is there a way 
> to re-run this migration to update the remaining ones? 

Not now, there is a bug on our side 
(http://www.rudder-project.org/redmine/issues/2894).
We are going to output a beta5 in the next couple of days with that (and 
a lot of other enhancement) included - stay tunned !


-- 
------------------------------------------------------------------------
*Fran?ois ARMAND*
/Directeur de la R&D/
Normation <http://www.normation.com>
------------------------------------------------------------------------
*87 rue de Turbigo, 75003 Paris, France*
Telephone: 	+33 (0)1 83 62 99 23
Mobile: 	+33 (0)6 63 37 60 55
------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120917/36b7d7f7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sig-normation-logo-square.png
Type: image/png
Size: 3503 bytes
Desc: not available
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120917/36b7d7f7/attachment.png>

From nicolas.perron at normation.com  Mon Sep 17 18:33:37 2012
From: nicolas.perron at normation.com (Nicolas Perron)
Date: Mon, 17 Sep 2012 18:33:37 +0200
Subject: [rudder-dev] rules and directives gone after upgrade to nightly
In-Reply-To: <2375574.b7pqnCR8xg@hgis96>
References: <1368984.40l1Cv2rFx@hgis96> <2431135.91aUMhcfEy@hgis96>
	<5056EE14.6060700@normation.com> <2375574.b7pqnCR8xg@hgis96>
Message-ID: <505750E1.7040505@normation.com>


> On Monday 17 Sep 2012 11:32:04 Francois Armand wrote:
>> Thank you so much Nicolas to think to that !
>> And for the future: http://www.rudder-project.org/redmine/issues/2900
>>
>> And thank you Michael for all the time and explanation about that bug
>> report, it's really valuable for us and Rudder !
> No worries.  Still not sure how I ran into this.  Actually, just found that 
> apparently indexes are not rebuilt automatically if the change to indexes is 
> made in slapd.conf:
>
> http://www.openldap.org/faq/data/cache/136.html
>
> but that seems only more confusing as then this should be easily reproducible.
>
Hi Michael,

We admit that the file *slapd.conf* is not updated during migration
(except from a Rudder 2.3) because of user modification (it is still a
configuration file). So, if you have updated it, you're case wasn't
handle by our scripts and it could explain why your LDAP wasn't indexed
properly.

We'll work on it ASAP (as Fran?ois show you, this is the issue
http://www.rudder-project.org/redmine/issues/2900).

Thank you again for report it to us, it will improve Rudder :)

-- 
==========================================
Nicolas PERRON
------------------------------------------
Normation
87 rue de Turbigo, 75003 Paris, France
------------------------------------------
Telephone:  +33 (0)1 83 62 56 79
Mobile:     +33 (0)6 24 79 31 79
------------------------------------------
Web:        http://www.normation.com/
==========================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rudder-project.org/pipermail/rudder-dev/attachments/20120917/88f42e5d/attachment.html>

