Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1526494 - Rebase clufter component [RHEL 6.10]
Summary: Rebase clufter component [RHEL 6.10]
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: clufter
Version: 6.10
Hardware: All
OS: Linux
high
medium
Target Milestone: rc
: ---
Assignee: Jan Pokorný [poki]
QA Contact: cluster-qe@redhat.com
Steven J. Levine
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-12-15 16:02 UTC by Jan Pokorný [poki]
Modified: 2018-06-19 05:23 UTC (History)
6 users (show)

Fixed In Version: clufter-0.77.1-1.el6
Doc Type: Rebase: Bug Fixes and Enhancements
Doc Text:
_clufter_ rebased to version 0.77.1 The _clufter_ packages have been upgraded to upstream version 0.77.1, which provides a number of bug fixes, new features, and user experience enhancements over the previous version. Among the notable updates are the following: * When producing `pcs` commands, the `clufter` tool now supports a preferred ability to generate `pcs` commands that will update only the modifications made to a configuration by means of a differential update rather than a pushing a wholesale update of the entire configuration. Likewise when applicable, the `clufter` tool now supports instructing the `pcs` tool to configure user permissions (ACLs). For this to work across the instances of various major versions of the document schemas, `clufter` gained the notion of internal on-demand format upgrades, mirroring the internal mechanics of `pacemaker`. Similarly, `clufter` is now capable of configuring the `bundle` feature. * In any script-like output sequence such as that produced with the `ccs2pcscmd` and `pcs2pcscmd` families of `clufter` commands, the intended shell interpreter is now emitted in a valid form, so that the respective commented line can be honored by the operating system. * When using `clufter` to translate an existing configuration with the `pcs2pcscmd-needle` command in the case where the `corosync.conf` equivalent omits the `cluster_name` option (which is not the case with standard pcs-initiated configurations), the contained `pcs cluster setup` invocation no longer causes cluster misconfiguration with the name of the first given node interpreted as the required cluster name specification. The same invocation will now include the `--encryption 0|1` switch when available, in order to reflect the original configuration accurately. * All `clufter` commands having a sequence of `pcs` commands at the output, meaning they are passed through a post-processing to improve readability (unless disabled with `--noop=cmd-wrap`), no longer have the issue that some characters with special meaning in shell language were not being quoted, which changed their interpretation. * The `clufter` tool now also covers some additional recently added means of configuration as facilitated with `pcs` (heuristics for a quorum device, meta attributes for top-level `bundle` resource units) when producing the sequence of configuring `pcs` commands to reflect existing configurations when applicable. On the `corosync` configuration interfacing side, the format parser no longer misinterprets commented-out lines with spaces or tabulators in front of the respective delimiter, and support for some mechanically introduced options was reconsidered under closer examination of what `pcs` actually handles. For information on the capabilities of *clufter*, see the `clufter(1)` man page or the output of the "clufter -h" command. For examples of *clufter* usage, see the following Red Hat Knowledgebase article: https://access.redhat.com/articles/2810031.
Clone Of: 1367536
Environment:
Last Closed: 2018-06-19 05:23:40 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1923 None None None 2018-06-19 05:23:42 UTC

Description Jan Pokorný [poki] 2017-12-15 16:02:38 UTC
+++ This bug was initially created as a clone of Bug #1367536 +++

+++ This bug was initially created as a clone of Bug #1387424 +++

Upstream development of clufter has progressed  so this asks for
a rebase.

NOTE:
It's understood that under normal circumstances, packages are not
receiving any updates sans the critical fixes, but this component begs
to differ as it also stands for a RHEL 7 migration technology, and RHEL
7 itself is a slightly moving target across minor releases, which is
in turn being gradually accommodated by this tool.  And because it's
reasonable to allow for tailored migration up to the latest RHEL 7 minor
version as of the particular RHEL 6 release (6.10 here), the request
to proceed with the rebase is deemed justified here.

More specifically, clufter received plentiful bug fixes and enhancements
compared to previous RHEL minor release (6.9/clufter-0.59.8) as
currently, clufter is at version 0.77.0.

Comment 8 Jan Pokorný [poki] 2018-04-19 15:21:42 UTC
Regarding changelog items (improving upon the procedure over that
of [bug 1387424 comment 12]):

$ git tag --no-merged v0.59.8
> v0.70.0
> v0.75.0
> v0.76.0
> v0.77.0
> v0.77.1

$ git tag -n9999 --no-merged v0.59.8 | sed 's|^v[^ ]*[ ]*|\n&\n-------\n|'
> 
> v0.70.0         
> -------
> bug fix, enhancement and, foremostly, Python 3 compatibility release
>     
>     - bug fixes:
>       . since v0.59.7, {ccs,pcs}2pcscmd commands were meant to emit
>         hashbangs so as to make it clear which shell (e.g. bash if its
>         extensions are relied upon) the suggested scripts should be run
>         under, but alas, exclamation mark has been accidentally missing
>         [related rhbz#1381531]
>       . invocations like "clufter ccs2pcscmd | bash -x", despite being
>         discouraged (review is needed for any nontrivial case, anyway),
>         if "pcs cluster auth" part interactively prompting credentials
>         was part of the run, they would be obtained from the piped script
>         under interpreter's execution itself due to the way the descriptors
>         are inherited; now there is an explicit stdin/stdout redirect
>         against /dev/tty in place to prevent such undesired side-effect
>     - functional changes:
>       . owing to the internal overhaul, *2pcscmd commands now properly
>         adapt to whether the output is directed to a terminal or not
>         (including piping to another command) with regards to colorizing
>         and text-wrapping for reasonable line lengths unless overridden
>         with explicit switches
>       . commands dealing with XML format (at some internal, not necessarily
>         user-facing phase; covers all *2pcs* ones) are now proofed so as
>         to preserve XML attribute ordering, leading to consistent results
>         (desired way beyond internal test suite)
>       . better diagnostics are now provided for some previously neglected
>         error conditions
>     - feature highlight: Python 3 support (-> minor version bump so high)
>       . beside Python 2.6, clufter project now strives to support 3.2+
>         as well; brand new compatibility with the latter was casually
>         and unit tested (the suite was also adapted to support both),
>         but some corner cases may have not been hit yet, hence this
>         class of bug reports is especially appreciated
>       . for the clufter as a library users: under Python 3, "bytestring"
>         protocol now, due to a strict dichotomy, indeed means "passed as
>         bytes (or compatible) type" (if the contrary is observed, it is
>         a bug to be fixed)
>       . *2pcscmd commands now state also the Python interpreter in the
>         initial informative comment block to ease the troubleshooting
>         in the light of newly introduced dual Python 2/3 support
>       . so-called external plugins are now implicitly searched also in
>         additional, Python interpreter agnostic path intended primarily
>         for plugins maintaining dual Python 2/3 compatibility (in the same
>         vein as core clufter as of this release)
>       . meta-spec prescribing a suggested RPM packaging in a generic way
>         now encompasses the above point as well as it offers python3-clufter
>         subpackage, and in turn factors some shared files to other new
>         subpackages (clufter-bin, clufter-common)
> 
> v0.75.0         
> -------
> enhancement and Python 3 compatibility improving release
>     
>     - enhancements:
>       . when using *2pcscmd commands, one can now specify --set-exec
>         option to make executable; note this is explicitly discouraged
>         as one should always take precautions prior to running anything
>         executable from a semirandom source and this would only make
>         it easier to skip that step and accidentally run the generated
>         script right away
>       . with {cib,pcs}2pcscmd commands, the generated sequence is now
>         adapted to whether pcs is able to natively cope with "external/*"
>         specification of stonith resources where applicable
>       . borrowing directly from pacemaker project, the upgrade between
>         1.x and 2.x lines of the schema for a CIB instance is now
>         available as fmt-cib-1to2 filter, arranged to be implicitly
>         triggered so as to bridge supplied vs. required format
>         discrepancies (see first internal enhancement point below)
>     - feature extensions:
>       . {cib,pcs}2pcscmd* commands are now aware of configured ACL
>         definitions in CIB and able to emit respective configuration
>         commands using pcs tool; thanks to implicit format upgrade
>         arrangement with fmt-cib-1to2 filter (see above), it can cope
>         with "old ACL style" of configuration at the input as well
>         [resolves: rhbz#1440876]
>       . likewise {cib,pcs}2pcscmd* commands now instruct sufficiently
>         recent target pcs to perform push of the CIB changes (accumulated
>         throughout the generated script) as a differential update rather
>         than propagating a complete CIB copy, which remains the default
>         otherwise (both unless push-by-push is applied, despite warnings)
>     - internal enhancements:
>       . format hierarchies were noticeably rethought with a notion of
>         upgradeability in mind -- something natural for the versioned,
>         compatibility-sensitive formats such as CIB, the prime target
>         (see also the mentioned upgrade filter); with a proper
>         arrangement these upgrades are performed implicitly on the
>         format instances being exchanged between the filters, should
>         the target filter impose that on validation failure basis
>       . Python 3 compatibility received further polishing
>       . meta specfile holding the suggested RPM packaging moved "nano"
>         fallback editor dependency over to -cli package, resulting in
>         a bit more lightweight pcs installation, as that itself depends
>         just on python*-clufter
>         [resolves: PGissue#1]
> 
> v0.76.0         
> -------
> bug fix and enhancements (support for pcs commands defining bundles)
>     
>     - bug fixes:
>       . some messages emitted in conversion corner cases are meant to
>         terminate the whole process, which previously may have not been
>         the case
>     - feature extensions:
>       . {cib,pcs}2pcscmd* commands are now aware of configured bundle
>         definitions in CIB and able to emit respective configuration
>         commands using pcs tool, provided that the target versions of
>         relevant packages already support that
>     - internal enhancements:
>       . the logic behind reusing pacemaker's own schemas is now available
>         as a proper script that now also -- rationally -- only extracts the
>         lowest and highest available minor version per a major one of the CIB
>         schema files, which further corresponds to what is now statically
>         carried with the project distribution and, foremost, what decreases
>         the number of the worst-case validations
>       . some warnings possibly emitted during the conversions now share
>         a uniform style
> 
> v0.77.0         
> -------
> bug fix and extension release (qdevice heuristics, bundle meta attrs)
>     
>     - bug fixes:
>       . practically every built-in command accepts "magic files" in place
>         of regular file specifications as arguments (also advertised at
>         the bottom of "clufter COMMAND -h"), but when it denotes the input
>         and happens to be resolved as standard input, a problem would
>         previously arise under Python 3, because this stream is implicitly
>         switched in text mode, whereas the data to be fetched are expected
>         to be raw bytes by the established framework (not enforced in Py2)
>       . pcs2pcscmd-needle command would previously lead to an incorrect
>         "pcs cluster setup" command in the resulting sequence when the
>         "cluster_name" parameter in the "totem" section of the input
>         corosync configuration was not (very legitimately) specified,
>         because the name of the first given node (if present) would
>         be mistakenly used where the cluster name specification was
>         expected as it was effectively skipped in the respective syntax;
>         now an artificial name is supplied instead if need be
>         (currently, pcs cannot create nameless clusters like that)
>       . all commands having sequence of pcs commands on the output,
>         hence getting post-processed (line-wrapped and generally
>         prettified) with the aim to get them human-friendly, might
>         previously suffer from some characters with special meaning
>         in shell language not being quoted (nor escaped for that matter)
>         properly (despite they might have been prior to this post-process),
>         leading to various, possibly subtle misbehaviours when executed
>         (for instance part of the intended standalone command being run in
>         the backgrounded subshell while the rest getting interpreted as
>         as a whole new, likely not fulfillable, command, if the particular
>         unquoted chunk in the case at hand contained '&' character); trivial
>         instances of this discrepancy should no longer occur, and for extra
>         confidence (or as a workaround with older versions), one can always
>         use "--noop=cmd-wrap" to suppress (tiny bit erratic) post-processing
>         (thanks to Madkiss, seconded by lge at #clusterlabs)
>     - feature extensions:
>       . {ccs,cib,pcs}2pcscmd* commands now make use of "--encryption 0|1"
>         switch finally added to "pcs cluster setup" command, incl. dealing
>         with a hiccup of introducing that support unconditionally (0.9.158),
>         only to return back to original default of no encryption, but with
>         a possibility to enable it (0.9.159 and counting), depending
>         on the pcs versions of the target distribution
>       . pcs2pcscmd-needle command is now aware of configured heuristics for
>         a quorum device in corosync.conf-equivalent files and able to emit
>         respective configuration commands sing pcs tool, provided that the
>         target versions of relevant packages already support that
>       . {cib,pcs}2pcscmd* commands are now aware of configured meta
>         attributes for top-level "bundle" encapsulations in CIB and able
>         to emit respective configuration commands using pcs tool, provided
>         that the target versions of relevant packages already support that
>     - internal enhancements:
>       . as pacemaker crossed the bridge and CIB schema versioning is no
>         longer compatible with string- and float-based comparisons, some
>         adaptation was required both code and release automation wise,
>         along with the update of higher-bound (merged-form) schema for CIB
>         up to this 2.10 version, which actually initiated those changes
>         [http://oss.clusterlabs.org/pipermail/developers/2017-July/000478.html]
> 
> v0.77.1         
> -------
> bug fix and slight fine-tuning release
>     
>     - bug fixes:
>       . runnning [cp]cs2pccmd commands in the absence of "defaults"
>         submodule (repository checkout, broken installation) no longer
>         fails
>       . corosync configuration parser, employed e.g. with pcs2pcscmd-needle
>         command, no longer mistreats commented-out lines with spaces or tabs
>         in front of the respective delimiter
>       . some options that were mechanically introduced recently for the
>         purpose of converting existing configuration into the procedural
>         steps leading there with the help pcs tool (i.e. {cib,pcs}2pcscmd
>         belong to affected commands) turned to be not actually suppported
>         by current pcs versions under closer examination, specifically
>         "nodelist.node.name" setting in corosync.conf (added into embedded
>         corosync configuration schema, regardless, in accord with having
>         this value legalized in corosync proper, just as with
>         "resources.watchdog_device"), and "quorum.device.votes" with
>         "quorum.device.net.tls" herein as well
>         [resolves/related: rhbz#1517834]
>         [resolves: rhbz#1552666]
>     - feature extensions:
>       . beyond, Linux top-level system selection (via --sys) of the target
>         the command output is to be taylored to (standing for a whole bunch
>         of parameters that would be too unwieldy to work with individually),
>         there's now also (rather preliminary) support for BSD family,
>         currently comprising only FreeBSD, for a simple fact that some
>         cluster packages are downstreamed there the usual way
>     - internal enhancements:
>       . handling of ambiguous specification of the target distribution plus
>         its version has been improved, also for the input values coming
>         from run-time auto-detection, leading to more widely normalized
>         values internally, hence bringing more reliable outputs
>         (now also with Fedora Rawhide, for instance)


Summarization of the most important items, taking [bug 1509381 comment 4]
and [RHEL7.4relnotes] into account:

---

When producing `pcs` commands, the `clufter` tool now supports
a preferred ability to generate `pcs` commands that will update only
the modifications made to a configuration by means of a differential
update rather than a pushing a wholesale update of the entire
configuration. Likewise when applicable, the `clufter` tool now
supports instructing the `pcs` tool to configure user permissions
(ACLs). For this to work across the instances of various major versions
of the document schemas, `clufter` gained the notion of internal
on-demand format upgrades, mirroring the internal mechanics of
`pacemaker`.  Similarly, `clufter` is now capable of configuring
the `bundle` feature. (BZ#1381531)

In any script-like output sequence such as that produced by the
`ccs2pcscmd` and `pcs2pcscmd` families of `clufter` commands,
the intended shell interpreter is now emitted in a valid form,
so that the respective commented line can finally be honored
by the operating system. (BZ#1381531)

When using `clufter` to translate existing configuration with
`pcs2pcscmd-needle` command whereby `corosync.conf` equivalent
omits `cluster_name` option (not the case with standard `pcs`
initiated configurations), the contained `pcs cluster setup`
invocation no longer causes cluster misconfiguration with the
name of the first given node interpreted as the required cluster
name specification.  The same invocation will now be enriched
with "--encryption 0|1" switch when available, in order to
reflect the original configuration truthfully.

All `clufter` commands having sequence of `pcs` commands at the
output, which are hence passed through a post-processing with the
intention to gain on human friendliness (unless disabled with
`--noop=cmd-wrap`), shall no longer suffer from some characters
with special meaning in shell language not being quoted, thus
changing their interpretation undesirably.

The `clufter` tool now also covers some other recently added
means of configuration as facilitated with `pcs` (heuristics
for a quorum device, meta attributes for top-level `bundle`
encapsulations) when producing sequence of configuring `pcs`
commands to reflect existing configurations when applicable.
On the `corosync` configuration interfacing side, the format
parser no longer mistreats commented-out lines with spaces
or tabulators in front of the respective delimiter, and
support for some mechanically introduced options was
reconsidered under closer examination of what `pcs` actually
handles. (BZ#1517834, BZ#1552666)

---

[RHEL7.4relnotes] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/7.4_release_notes/

Comment 9 Jan Pokorný [poki] 2018-04-19 16:35:26 UTC
re [comment 8]:

"(BZ#1381531)" should be removed at the first item ("When producing ...")
but retained at the second one ("In any script-like").

Comment 15 errata-xmlrpc 2018-06-19 05:23:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1923


Note You need to log in before you can comment on or make changes to this bug.