Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1519904 - [RFE] Allow running pre 2.x DB data validation step independently via cpdb
Summary: [RFE] Allow running pre 2.x DB data validation step independently via cpdb
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Candlepin
Version: 6.2.13
Hardware: Unspecified
OS: Unspecified
urgent vote
Target Milestone: 6.2
Assignee: Chris "Ceiu" Rog
QA Contact: Sanket Jagtap
Depends On: 1519900
Blocks: 1520319 1520326 1533259
TreeView+ depends on / blocked
Reported: 2017-12-01 16:35 UTC by Michael Stead
Modified: 2018-12-06 20:45 UTC (History)
12 users (show)

Fixed In Version: candlepin-
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1519900
: 1530686 (view as bug list)
Last Closed: 2018-02-05 13:55:32 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1520319 None None None Never
Red Hat Product Errata RHSA-2018:0273 normal SHIPPED_LIVE Important: Red Hat Satellite 6 security, bug fix, and enhancement update 2018-02-08 00:35:29 UTC

Internal Links: 1520319

Description Michael Stead 2017-12-01 16:35:23 UTC
+++ This bug was initially created as a clone of Bug #1519900 +++

Data validation should be done before candlepin is upgraded from 0.9.x to 2.x. Currently candlepin does this during the 2.x DB upgrade process via liquibase.

It would be better to be able to perform the validation 'on demand' before any actual update occurs.

cpdb should be updated to enable running the validation step independently, before updating to 2.x, so that any issues can be addressed before the actual upgrade begins.

Comment 2 Lukas Zapletal 2017-12-04 09:13:26 UTC
I have a request, can we assume that there will be something in the cpdb.log file appended we can check in our pre-upgrade tool to verify users finished this check step?

In pre-6.2.14 world, we will ask them to run this:

liquibase --classpath=/usr/share/java/postgresql-jdbc.jar:./cpvalidation --changeLogFile=./cpvalidation/changelog-validate.xml --driver=org.postgresql.Driver --url=jdbc:postgresql:candlepin --username=candlepin --password=$(grep org.quartz.dataSource.myDS.password /etc/candlepin/candlepin.conf | cut -d= -f2) --logLevel=debug migrate 2>&1 | tee -a /var/log/candlepin/cpdb.log

This is leaving:

DEBUG 12/4/17 4:07 AM:liquibase: ./cpvalidation/changelog-validate.xml:

DEBUG 12/4/17 4:07 AM:liquibase: ./cpvalidation/changelog-validate.xml:

which we can grep at the later stage. Can you please make sure this is left when running cpdb? Perhaps by increasing verbosity, or making new verbosity option or adding an extra INFO line because by default there is just "Liquibase Update Successful" which is not enough, thanks.

Comment 3 Lukas Zapletal 2017-12-04 09:19:13 UTC
Please disregard comment 2, this is not relevant anymore. But we'd appreciate increased verbosity for sure, if that helps you to debug possible issues during calls.

Comment 4 Lukas Zapletal 2017-12-04 09:21:38 UTC
Last thing - we do assume that cpdb returns non-zero shell code when validation fails. When looking into 6.2.13 codebase, I don't see explicit exit calls there, just want to make sure. Thanks.

Comment 5 Chris "Ceiu" Rog 2017-12-04 14:55:58 UTC
Additional logging changes were also made as part of this patch. By default, cpdb now executes the liquibase command with --logLevel=SEVERE (up from none); and if you want all of the logging as requested above, cpdb now has a --verbose flag, which changes the liquibase command's log level to DEBUG.

When the liquibase command returns non-zero, cpdb propagates an exception up, causing it to also return a non-zero (but not necessarily the same) error code. With the way the validation bits are written at present, any validation failure causes liquibase to fail in this manner.

Comment 6 Lukas Zapletal 2017-12-05 10:47:12 UTC
Ok please make sure the "--verbose" flag is in 6.2.x series so we can call it. Thanks.

Comment 10 Sanket Jagtap 2018-01-15 13:21:58 UTC
Build: Satellite 6.2.14 snap1

rpm -q candlepin

[root@ibm-x3650-01 candlepin]# ./cpdb --help
Usage: cpdb [options]

  -h, --help            show this help message and exit
  --create              create the Candlepin database; cannot be used with
                        --update or --validate
  --schema-only         assumes database is already created by another tool
                        andapplies schema to database; used with --create
  --validate            validate the current state of the Candlepin database;
                        cannot be used with --create
  --update              update the Candlepin database; cannot be used with
  --drop                drop the existing Candlepin database before creating
  -u DBUSER, --user=DBUSER
                        database user to use
  -d DB, --database=DB  database name to use
                        database password to use
  --dbhost=DBHOST       the database host to use (optional)
  --dbport=DBPORT       the database port to use (optional)
  --community           true if used in a community fashion
  --oracle              run against an Oracle installation
                        Oracle DBA user. Defaults to 'sys'
                        Oracle DBA password
  --verbose             enables verbose logging/output

Using verbose increases the log-level on liquibase to debug

[root@ibm-x3650-01 candlepin]# sudo -u postgres -H -- psql -d candlepin -c "update cp_pool set productid='Ohno' where id='0000000000000000000000000000000';"
[root@ibm-x3650-01 candlepin]# ./cpdb --validate -u candlepin -p 1111111111111111111111111111111 -d candlepin --verboseValidating Candlepin database
--driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-validate.xml --url=jdbc:postgresql:candlepin --username=candlepin --password=1111111111111111111111111111111 --logLevel=debug

########## ERROR ############
Error running command: liquibase --driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-validate.xml --url=jdbc:postgresql:candlepin --username=candlepin --password=1111111111111111111111111111111 --logLevel=debug migrate -Dcommunity=False
Status code: 65280
Command output: DEBUG 1/15/18 8:13 AM:liquibase: Connected to candlepin@jdbc:postgresql:candlepin
DEBUG 1/15/18 8:13 AM:liquibase: Setting auto commit to false from true
DEBUG 1/15/18 8:13 AM:liquibase: Executing QUERY database command: select count(*) from public.databasechangeloglock
	at org.candlepin.liquibase.LiquibaseCustomTaskWrapper.execute(
	... 9 more

For more information, use the --logLevel flag
Traceback (most recent call last):
  File "./cpdb", line 297, in <module>
  File "./cpdb", line 70, in validate
  File "./cpdb", line 106, in _run_liquibase
    output = run_command("liquibase %s migrate -Dcommunity=%s" % (liquibase_options,
  File "./cpdb", line 38, in run_command
    error_out(command, status, output)
  File "./cpdb", line 46, in error_out
    raise Exception("Error running command")
Exception: Error running command
[root@ibm-x3650-01 candlepin]# echo $?

The return code when encountered with error
Also the default loglevel increase to severe
./cpdb --validate -u candlepin -p zzzzzzzzz -d candlepin
Validating Candlepin database
--driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-validate.xml --url=jdbc:postgresql:candlepin --username=candlepin --password=zzzzzzz --logLevel=severe
Liquibase Update Successful

Comment 13 errata-xmlrpc 2018-02-05 13:55:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

Comment 14 Lukas Zapletal 2018-02-07 08:07:41 UTC
Hey, I don't understand. There was an error and you flipped to VERIFIED. Can you explain please? This looks like an issue to me. It was the purpose of this BZ to actually introduce the new parameter.

Note You need to log in before you can comment on or make changes to this bug.