Project

General

Profile

Actions

Bug #7285

closed

Only run inventory collection when no other is running

Added by Janos Mattyasovszky over 8 years ago. Updated over 7 years ago.

Status:
Rejected
Priority:
N/A
Category:
System techniques
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
Name check:
Fix check:
Regression:

Description

Please make a lock for running an inventory, so it only runs once, so any hanging / long running inventory will not overlap with a new one.

Otherwise you'd end up with something like this:

/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory --scan-homedirs
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory --scan-homedirs
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null

Times 1500...


Related issues 2 (0 open2 closed)

Related to Rudder - Bug #9626: The locks are removed every 10 minutes by check-rudder-agent when there is no promise updateReleasedNicolas CHARLES2016-11-04Actions
Is duplicate of Rudder - Bug #9622: Fusioninventory is not tracked by check-rudder-healthReleasedNicolas CHARLESActions
Actions #1

Updated by Vincent MEMBRÉ over 8 years ago

  • Category changed from Agent to System techniques
  • Assignee set to Nicolas CHARLES

That would be great indeed to prevent those multiples runs :(

Nicolas, is it possible to check before launching the inventory, that a run-inventory process is not already running ?

Actions #2

Updated by Benoît PECCATTE almost 8 years ago

  • Target version set to 2.11.21
Actions #3

Updated by Vincent MEMBRÉ almost 8 years ago

  • Target version changed from 2.11.21 to 2.11.22
Actions #4

Updated by Vincent MEMBRÉ almost 8 years ago

  • Target version changed from 2.11.22 to 2.11.23
Actions #5

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 2.11.23 to 2.11.24
Actions #6

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 2.11.24 to 308
Actions #7

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 308 to 3.1.14
Actions #8

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 3.1.14 to 3.1.15
Actions #9

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 3.1.15 to 3.1.16
Actions #10

Updated by Nicolas CHARLES over 7 years ago

Investigations on this topic show that:
  1. locks are correctly working: if an inventory is running, then following concurrent runs won't try to run the inventory command (tests with sleep 10000 in the run-inventory command)
  2. BUT purging the cf_lock.lmdb file lift this locking: execution can be concurrents

Promises timeout by default after 2 hours - we need to check if the commands are automatically stopped after 2 hours, even if lock file is removed

Note: cf_lock.lmdb is purged as part of the check_rudder_agent command, when file gets bigger than 10 MB, which apparently can cause bad behaviour :/

Actions #11

Updated by Nicolas CHARLES over 7 years ago

Okn the hypothesis is correct: when we purge the cf_lock.lmdb database, agent looses all track of when was launched something. So things can pile up :(

Actions #12

Updated by Nicolas CHARLES over 7 years ago

workaround could be to check with service_check_running that there are no inventory running - but that will not solve underlying issue that promises don't expire properly when we kill cf_lock, and that we need to purge cf_lock because it grows out of bound

Actions #13

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 3.1.16 to 3.1.17
Actions #14

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 3.1.17 to 3.1.18
Actions #15

Updated by Nicolas CHARLES over 7 years ago

  • Is duplicate of Bug #9622: Fusioninventory is not tracked by check-rudder-health added
Actions #16

Updated by Nicolas CHARLES over 7 years ago

  • Status changed from New to Rejected

Issue should have been fixed in #9622 : when we kill the agent, we didn't kill subtasks, so everything was building up, with no chance of being garbage collected.
With change in related ticket, it should not happen anymore
#9226 fixed also the killing of locks, so agent properly tracks which promise is in progress
Rejected as fixed by others tickets

Actions #17

Updated by Nicolas CHARLES over 7 years ago

  • Related to Bug #9626: The locks are removed every 10 minutes by check-rudder-agent when there is no promise update added
Actions

Also available in: Atom PDF