Project

General

Profile

Actions

Bug #13619

closed

Document that resource file of Techniques are shared to nodes with UTF-8 Encoding *only* (breaks for other encoding)

Added by Nicolas CHARLES over 5 years ago. Updated almost 5 years ago.

Status:
Released
Priority:
N/A
Category:
Documentation
Target version:
Severity:
Major - prevents use of part of Rudder | no simple workaround
UX impact:
User visibility:
Getting started - demo | first install | Technique editor and level 1 Techniques
Effort required:
Very Small
Priority:
89
Name check:
Fix check:
Regression:

Description

somehow, the Rudder-Agent.xml in the generated promises starts with ef bf bd ef bf bd BOM, while the original file starts with ff fe BOM
result is: file is corrupted.

BOM of original file was removed, as a workaround, in #13616, but we need to understand WHY it happened

Hypothesis: scala lib interpreting and adding the BOM, some side effect of #12471, or something else.


Related issues 1 (0 open1 closed)

Related to Rudder - Bug #19222: Policy generation does change technique resource files when it should notReleasedFrançois ARMANDActions
Actions #1

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 4.3-1.11 to 4.3-1.12
Actions #2

Updated by Vincent MEMBRÉ over 5 years ago

  • Project changed from 78 to Rudder
  • Subject changed from UTF-8 encoding BOM added to Rudder-Agent.xml file by rudder dsc plugin to Resource file of Techniques are shared to nodes with UTF-8 Encoding
  • Category changed from System techniques to System techniques
  • Target version changed from 4.3-1.12 to 4.3.5

If we use a Resource file in a Techniques (using a FILE tag in the metadata.xml) the encoding of the file is enforced to UTF-8, even if the file is in an another encoding (ie an xml we want to use on windows ... )

This is due to this function which reads the content of the file and save it with UTF-8 encoding
https://github.com/Normation/rudder/blob/a5ce997aaf34c64fa1919979e87cfb7c56be60cf/rudder-core/src/main/scala/com/normation/rudder/services/policies/write/PolicyWriterService.scala#L717

Actions #3

Updated by Vincent MEMBRÉ over 5 years ago

  • Category changed from System techniques to Web - Config management
Actions #5

Updated by Nicolas CHARLES over 5 years ago

EF BF BD is a replacement character for invalid utf8 character
https://apps.timwhitlock.info/unicode/inspect?s=%EF%BF%BD

Actions #6

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 4.3.5 to 4.3.6
Actions #7

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 4.3.6 to 4.3.7
Actions #8

Updated by François ARMAND over 5 years ago

Here, we need to either document that fact or add an "input encoding" and "output encoding" parameter for resource files.

Actions #9

Updated by François ARMAND over 5 years ago

  • Subject changed from Resource file of Techniques are shared to nodes with UTF-8 Encoding to Resource file of Techniques are shared to nodes with UTF-8 Encoding *only* (breaks for other encoding)
Actions #10

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 4.3.7 to 4.3.8
Actions #11

Updated by Vincent MEMBRÉ about 5 years ago

  • Target version changed from 4.3.8 to 4.3.9
Actions #12

Updated by Alexis Mousset about 5 years ago

  • Target version changed from 4.3.9 to 4.3.10
Actions #13

Updated by François ARMAND about 5 years ago

  • Target version changed from 4.3.10 to 4.3.11
Actions #14

Updated by Vincent MEMBRÉ almost 5 years ago

  • Target version changed from 4.3.11 to 4.3.12
Actions #15

Updated by François ARMAND almost 5 years ago

  • Subject changed from Resource file of Techniques are shared to nodes with UTF-8 Encoding *only* (breaks for other encoding) to Document that resource file of Techniques are shared to nodes with UTF-8 Encoding *only* (breaks for other encoding)
  • Category changed from Web - Config management to Documentation
  • Target version changed from 4.3.12 to 5.0.10
  • Severity changed from Critical - prevents main use of Rudder | no workaround | data loss | security to Major - prevents use of part of Rudder | no simple workaround
  • User visibility set to Getting started - demo | first install | Technique editor and level 1 Techniques
  • Effort required set to Very Small
  • Priority changed from 0 to 90

So, the possibility to add file encoding for all templates is a long term new feature. In the meantime, we need to just clearly document the limitation (it should be ok for most use case, and workaroundable for all even if it supposes a very tedious step of transcoding on the node).

Actions #16

Updated by François ARMAND almost 5 years ago

  • Assignee set to Nicolas CHARLES
Actions #17

Updated by Nicolas CHARLES almost 5 years ago

  • Status changed from New to In progress
Actions #18

Updated by Nicolas CHARLES almost 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to Alexis Mousset
  • Pull Request set to https://github.com/Normation/rudder-doc/pull/578
Actions #19

Updated by Nicolas CHARLES almost 5 years ago

  • Status changed from Pending technical review to Pending release
Actions #20

Updated by Vincent MEMBRÉ almost 5 years ago

  • Status changed from Pending release to Released
  • Priority changed from 90 to 89

This bug has been fixed in Rudder 5.0.10 which was released today.

Actions #21

Updated by François ARMAND 5 months ago

  • Related to Bug #19222: Policy generation does change technique resource files when it should not added
Actions

Also available in: Atom PDF