Project

General

Profile

Actions

Architecture #3007

closed

Update String Template to version 4

Added by François ARMAND over 11 years ago. Updated about 5 years ago.

Status:
Rejected
Priority:
N/A
Category:
Architecture - Dependencies
Effort required:
Large
Name check:
Fix check:
Regression:

Description

Faster, lighter, better: http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4

As explained in the given link, that update is some work, and so it can not be considered as a simple "library update".


Related issues 1 (0 open1 closed)

Has duplicate Rudder - User story #10061: Upgrade to StringTemplate 4 to improve policy generation timeRejectedNicolas CHARLESActions
Actions #1

Updated by François ARMAND over 11 years ago

After looking a little better, it seems that the last version (4.0.6) is not available in maven repos. In fact, the last available version is 4.0.2, almost two year old.

So I'm not sure it is wise to try to update to 4 for now. Or perhaps we could add that in our nexus ?

Actions #2

Updated by François ARMAND over 11 years ago

  • Description updated (diff)
  • Parent task deleted (#3003)
Actions #3

Updated by Nicolas CHARLES over 11 years ago

The 4.0.4 is available : http://mvnrepository.com/artifact/org.antlr/ST4/4.0.4
They changed name of the artefact from stringtemplate to ST
However the git refers to 4.0.4-SNAPSHOT in the pom, hence the unpublished version 4.0.6 ...

Actions #4

Updated by Jonathan CLARKE over 11 years ago

  • Status changed from 2 to New
Actions #5

Updated by François ARMAND over 11 years ago

  • Status changed from New to 8
  • Target version set to Ideas (not version specific)
Actions #6

Updated by Benoît PECCATTE almost 9 years ago

  • Status changed from 8 to New
Actions #7

Updated by Nicolas CHARLES about 7 years ago

  • Has duplicate User story #10061: Upgrade to StringTemplate 4 to improve policy generation time added
Actions #8

Updated by Nicolas CHARLES almost 6 years ago

Ok, I've made some measures
With current implementation

[2018-06-04 12:28:42] DEBUG policy.generation - Promises written in 39543 ms
[2018-06-04 12:28:42] DEBUG policy.generation - Properties written in 138 ms
[2018-06-04 12:28:42] DEBUG policy.generation - Parameters written in 59 ms
[2018-06-04 12:28:42] DEBUG policy.generation - Licenses copied in 0 ms
[2018-06-04 12:28:50] DEBUG policy.generation - Policies moved to their final position in 7514 ms
[2018-06-04 12:28:50] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written on filesystem in 48106 ms
[2018-06-04 12:28:51] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration cached in LDAP in 1174 ms
[2018-06-04 12:28:51] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written in 49282 ms, start to update expected reports.
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node expected reports saved in base in 49282 ms.
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Post-policy-generation hooks ran in 2 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Timing summary:
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen scripts hooks :          0 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen modules hooks :          0 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Fetch all information     :        311 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Historize names           :        973 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build current rule values :         73 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build target configuration:       8180 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Write node configurations :      49282 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Save expected reports     :        904 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run post generation hooks :          2 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Number of nodes updated   :       1597   
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 75630 ms

With StringTemplate4

[2018-06-04 13:56:04] DEBUG policy.generation - Promises written in 22994 ms
[2018-06-04 13:56:04] DEBUG policy.generation - Properties written in 44 ms
[2018-06-04 13:56:04] DEBUG policy.generation - Parameters written in 46 ms
[2018-06-04 13:56:04] DEBUG policy.generation - Licenses copied in 0 ms
[2018-06-04 13:56:13] DEBUG policy.generation - Policies moved to their final position in 9383 ms
[2018-06-04 13:56:13] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written on filesystem in 33251 ms
[2018-06-04 13:56:14] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration cached in LDAP in 1026 ms
[2018-06-04 13:56:14] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written in 34279 ms, start to update expected reports.
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node expected reports saved in base in 34279 ms.
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Post-policy-generation hooks ran in 2 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Timing summary:
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen scripts hooks :          0 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen modules hooks :          0 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Fetch all information     :        319 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Historize names           :        845 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build current rule values :         46 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build target configuration:       7852 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Write node configurations :      34279 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Save expected reports     :        940 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run post generation hooks :          2 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Number of nodes updated   :       1597   
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 57238 ms

So the step "Promises written" that uses a lot of stringtemplate goes from 39s to 23s - a significant improvement !

Alas, StringTemplate 4 is not maintained any more, adapting it to make it work with our template is a pretty signigicant change (we cannot use & as a seperator in stringtemplate), it needs rewritting of techniques, due to change of syntax, plus there are some minor breaking change, especially around }, iteration, and so on.

Actions #9

Updated by Nicolas CHARLES almost 6 years ago

Some possible replacement:

Bottom line: they will all require Technique rewrite, so we'll need both engine at the same time for a period of time

Actions #10

Updated by Nicolas CHARLES almost 6 years ago

  • Effort required set to Large
Actions #11

Updated by Alexis Mousset over 5 years ago

  • Status changed from New to Discussion
  • Assignee set to Nicolas CHARLES

Nicolas, IIRC you said it won't be possible?

Actions #12

Updated by Nicolas CHARLES about 5 years ago

  • Status changed from Discussion to Rejected

I'm rejecting this issue - migrating to stringtemplate 4 is not possible (incompatible syntax, and the project is not rellay maintained), and #13987 did improve a lot performance.

Actions

Also available in: Atom PDF