Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1063364 - RHQ installation on JDK1.6 fails with 'Cannot obtain client connection to the RHQ app server!!'
Summary: RHQ installation on JDK1.6 fails with 'Cannot obtain client connection to the...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Operations Network
Classification: JBoss
Component: Core Server
Version: JON 3.2
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: DR01
: JON 3.2.2
Assignee: Jirka Kremser
QA Contact: Mike Foley
URL:
Whiteboard:
Depends On: 1058267
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-10 15:03 UTC by Mike Foley
Modified: 2015-04-17 14:57 UTC (History)
4 users (show)

Doc Type: Bug Fix
Doc Text:
Clone Of: 1058267
Environment:
Last Closed: 2014-07-29 00:17:08 UTC
Type: Bug


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1212878 None None None Never

Internal Links: 1212878

Description Mike Foley 2014-02-10 15:03:11 UTC
+++ This bug was initially created as a clone of Bug #1058267 +++

Description of problem:
$Summary

Version-Release number of selected component (if applicable):
rhq-server-4.10.0-SNAPSHOT
4f60fa0d3c7a

How reproducible:
Always

Steps to Reproduce:
1. unzip rhq-server-4.10.0-SNAPSHOT.zip
2. cd rhq-server-4.10.0-SNAPSHOT/bin/
3. ./rhqctl install
4. set jboss.bind.address to 0.0.0.0

Actual results:
RHQ storage is installed correctly but RHQ server installation failed with:
06:46:15,286 INFO  [org.rhq.server.control.command.Install] The RHQ Server must be started to complete its installation. Starting the RHQ server in preparation of running the server installer...
06:46:15,304 INFO  [org.rhq.server.control.command.Install] Waiting for the RHQ Server to start in preparation of running the server installer...
Trying to start the RHQ Server...
RHQ Server                     (pid 6847   ) is ✘ down
Failed to start - make sure the RHQ Server is fully configured properly
06:46:20,850 INFO  [org.jboss.modules] JBoss Modules version 1.2.0.CR1
06:46:20,971 INFO  [org.rhq.enterprise.server.installer.InstallerServiceImpl] The server is preconfigured and ready for auto-install.
06:46:21,055 INFO  [org.xnio] XNIO Version 3.0.7.GA
06:46:21,066 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.7.GA
06:46:21,074 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.14.GA
06:46:31,338 ERROR [org.rhq.enterprise.server.installer.Installer] The installer will now exit due to previous errors: java.lang.Exception: Cannot obtain client connection to the RHQ app server!!
	at org.rhq.enterprise.server.installer.InstallerServiceImpl.testModelControllerClient(InstallerServiceImpl.java:1100) [rhq-installer-util-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at org.rhq.enterprise.server.installer.InstallerServiceImpl.preInstall(InstallerServiceImpl.java:217) [rhq-installer-util-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at org.rhq.enterprise.server.installer.InstallerServiceImpl.test(InstallerServiceImpl.java:142) [rhq-installer-util-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at org.rhq.enterprise.server.installer.Installer.doInstall(Installer.java:90) [rhq-installer-util-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at org.rhq.enterprise.server.installer.Installer.main(Installer.java:57) [rhq-installer-util-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_24]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.6.0_24]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.6.0_24]
	at java.lang.reflect.Method.invoke(Method.java:616) [rt.jar:1.6.0_24]
	at org.jboss.modules.Module.run(Module.java:262) [jboss-modules.jar:1.2.0.CR1]
	at org.jboss.modules.Main.main(Main.java:329) [jboss-modules.jar:1.2.0.CR1]
Caused by: java.io.IOException: java.net.ConnectException: JBAS012144: Could not connect to remote://127.0.0.1:9999. The connection timed out
	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:129) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:81) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.rhq.common.jbossas.client.controller.JBossASClient.execute(JBossASClient.java:270) [rhq-jboss-as-dmr-client-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at org.rhq.common.jbossas.client.controller.CoreJBossASClient.getSystemProperties(CoreJBossASClient.java:103) [rhq-jboss-as-dmr-client-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	at org.rhq.enterprise.server.installer.InstallerServiceImpl.testModelControllerClient(InstallerServiceImpl.java:1051) [rhq-installer-util-4.10.0-SNAPSHOT.jar:4.10.0-SNAPSHOT]
	... 10 more
Caused by: java.net.ConnectException: JBAS012144: Could not connect to remote://127.0.0.1:9999. The connection timed out
	at org.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:130) [jboss-as-protocol-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:256) [jboss-as-protocol-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:70) [jboss-as-protocol-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:176) [jboss-as-protocol-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:144) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:115) [jboss-as-protocol-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:98) [jboss-as-protocol-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:236) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:141) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:127) [jboss-as-controller-client-7.2.0.Alpha1-redhat-4.jar:7.2.0.Alpha1-redhat-4]
	... 14 more

06:46:31,352 ERROR [org.rhq.server.control.command.Install] An error occurred while starting the RHQ server: Process exited with an error: 2 (Exit value: 2)
RHQ Server                     (pid 6847   ) is ✘ down

Expected results:
Installation is succesfull

--- Additional comment from Filip Brychta on 2014-01-31 04:51:40 EST ---

Update:
This issue is related to JDK1.6. Installation works correctly on JDK1.7

Fails on:
[hudson@last-rhq-server bin]$ java -version
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.1) (rhel-1.45.1.11.1.el6-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)


Works on:
[hudson@last-rhq-server bin]$ java -version
java version "1.7.0_03-icedtea"
OpenJDK Runtime Environment (rhel-2.1.el6.7-x86_64)
OpenJDK 64-Bit Server VM (build 22.0-b10, mixed mode)

--- Additional comment from Filip Brychta on 2014-01-31 07:46:18 EST ---

This is the first build with this issue http://hudson.qa.jboss.com/hudson/view/RHQ/job/rhq-master-gwt-locales/967/

--- Additional comment from Jirka Kremser on 2014-01-31 07:57:27 EST ---

Can be replicated also on HotSpot JVM (1.6.0_24).

--- Additional comment from Jirka Kremser on 2014-01-31 08:11:55 EST ---

The build above (comment 2) contains 86 changes, by using git bisect, it is in the worse case  ceil(log_2(86)) = 7 "build-install-check" cycles. 

It would be nice to have some established tool based on foreman/jenkins to do this automatically.

I am investigating the commits, looking for the culprit..

--- Additional comment from Jirka Kremser on 2014-02-03 08:55:30 EST ---

In commit d6af564d06ce769d1f, the JVM param ("-XX:StringTableSize=1000003") was added that causes this issue. 

Java 6 has apparently problem with this parameter. In Java 7 and higher, the pooled strings are stored on heap, while on Java 6 and lower it is on permgen space.

Comment 3 Jirka Kremser 2014-03-26 18:25:35 UTC
This has been resolved in the master. It was a regression coming from the agent footprint reduction. The agent footprint commits have not been cherry-picked to the release branch yet.

Comment 4 Simeon Pinder 2014-05-22 15:15:37 UTC
Via product triage, determined that this bug is to be included for DR01 target milestone.

Comment 5 Simeon Pinder 2014-05-30 02:43:35 UTC
Moving to ON_QA as available for test in latest cumulative patch build(DR01):
http://jon01.mw.lab.eng.bos.redhat.com:8042/dist/release/jon/3.2.2.GA/5-29-2014/

Comment 6 Mike Foley 2014-05-30 19:26:33 UTC
Verified JON 3.2.2 DR1

Comment 7 Larry O'Leary 2014-07-29 00:17:08 UTC
This has been verified and released in Red Hat JBoss Operations Network 3.2 Update 02 (3.2.2) available from the Red Hat Customer Portal[1].



[1]: https://access.redhat.com/jbossnetwork/restricted/softwareDetail.html?softwareId=31783


Note You need to log in before you can comment on or make changes to this bug.