Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1366526

Summary: [GSS](6.4.z) EJB method invocation hangs when a bean is configured with both @Startup annotation and <init-on-startup> in ejb-jar.xml
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Masafumi Miura <mmiura>
Component: EJBAssignee: Fedor Gavrilov <fgavrilo>
Status: CLOSED CURRENTRELEASE QA Contact: Jan Martiska <jmartisk>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.4.9CC: bmaxwell, david.lloyd, fgavrilo, jbilek, jtruhlar, msochure
Target Milestone: CR1   
Target Release: EAP 6.4.11   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-17 13:10:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1310908    
Bug Blocks: 1319040, 1379471, 1361648    
Attachments:
Description Flags
A reproducer based on quickstarts ejb-in-ear application
none
BZ1366526-potential-patch.diff mmiura: review?

Description Masafumi Miura 2016-08-12 08:49:23 UTC
Created attachment 1190303 [details]
A reproducer based on quickstarts ejb-in-ear application

### Description of problem:

EJB method invocation hangs when a bean is configured with both @Startup annotation and <init-on-startup> in ejb-jar.xml


### Version-Release number of selected component (if applicable):

EAP 6.4.9 or later, which incorporates a fix for https://bugzilla.redhat.com/show_bug.cgi?id=1310908


### How reproducible:

Anytime when setting both @Startup annotation and <init-on-startup> in ejb-jar.xml for beans


### Steps to Reproduce:

1. Deploy the attached application, which is based on JBoss quickstarts ejb-in-ear application 
2. Start JBoss EAP 6.4.9
3. Access http://127.0.0.1:8080/jboss-ejb-in-ear/
4. Put a name and click "Greet" button


### Actual results:

No response is not returned and EJB hangs


### Expected results:

A response is returned without hang at EJB


### Additional info:

A thread dump indicates EJB is awaiting forever at org.jboss.as.ee.component.deployers.StartupCountdown#await().

~~~
"http-127.0.0.1:8080-1" #104 daemon prio=5 os_prio=0 tid=0x00007f40e8001000 nid=0x80d9 in Object.wait() [0x00007f40fd7c1000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000af703280> (a org.jboss.as.ee.component.deployers.StartupCountdown)
        at java.lang.Object.wait(Object.java:502)
        at org.jboss.as.ee.component.deployers.StartupCountdown.await(StartupCountdown.java:37)
        - locked <0x00000000af703280> (a org.jboss.as.ee.component.deployers.StartupCountdown)
        at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:21)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
        at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:189)
        at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
        at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
        at org.jboss.as.quickstarts.ear.ejb.GreeterEJB$$$view1.sayHello(Unknown Source)
        at org.jboss.as.quickstarts.ear.controller.Greeter.setName(Greeter.java:58)
        at org.jboss.as.quickstarts.ear.controller.Greeter$Proxy$_$$_WeldClientProxy.setName(Greeter$Proxy$_$$_WeldClientProxy.java)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.el.parser.AstValue.invoke(AstValue.java:258)
        at org.apache.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:278)
        at org.jboss.weld.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
        at org.jboss.weld.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
        at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
        at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87)
        at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:101)
        at javax.faces.component.UICommand.broadcast(UICommand.java:315)
        at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:786)
        at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1251)
        at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
        at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
        at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:593)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:654)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926)
        at java.lang.Thread.run(Thread.java:745)
~~~

Comment 1 Masafumi Miura 2016-08-12 08:54:09 UTC
When both @Startup annotation and <init-on-startup>true</init-on-startup> in ejb-jar.xml are configured for bean,  "description.getModuleDescription().registerStartupBean()" is called twice in StartupMergingProcessor#handleAnnotations() and in StartupMergingProcessor#handleDeploymentDescriptor(). 

This makes the startup countdown (startupBeansCount in EEModuleDescription) have a wrong value (actual bean count + 1). This causes the count down never reaches 0 and EJB invocation awaits forever.

---------------------------------------------------------

 * ejb3/src/main/java/org/jboss/as/ejb3/deployment/processors/merging/AbstractMergingProcessor.java

~~~
 81     private void processComponentConfig(final DeploymentUnit deploymentUnit, final EEApplicationClasses applicationClasses, final Module module, final DeploymentReflectionIndex deploymentReflectionIndex, final T description) throws DeploymentUnitProcessingException {
 82 
 83         final Class<?> componentClass;
 84         try {
 85             componentClass = module.getClassLoader().loadClass(description.getEJBClassName());
 86         } catch (ClassNotFoundException e) {
 87             throw MESSAGES.failToLoadEjbClass(description.getEJBClassName(),e);
 88         }
 89 
 90         if (!MetadataCompleteMarker.isMetadataComplete(deploymentUnit)) {
 91             handleAnnotations(deploymentUnit, applicationClasses, deploymentReflectionIndex, componentClass, description);
 92         }
 93         handleDeploymentDescriptor(deploymentUnit, deploymentReflectionIndex, componentClass, description);
 94     }
~~~

 * ejb3/src/main/java/org/jboss/as/ejb3/deployment/processors/merging/StartupMergingProcessor.java 

~~~
 46     @Override
 47     protected void handleAnnotations(final DeploymentUnit deploymentUnit, final EEApplicationClasses applicationClasses, final DeploymentReflectionIndex deploymentReflectionIndex, final Class<?> componentClass, final SingletonComponentDescription description) throws DeploymentU    nitProcessingException {
 48         EEModuleClassDescription clazz = applicationClasses.getClassByName(componentClass.getName());
 49         if (clazz != null) {
 50             final ClassAnnotationInformation<Startup, Object> data = clazz.getAnnotationInformation(Startup.class);
 51             if (data != null) {
 52                 if (!data.getClassLevelAnnotations().isEmpty()) {
 53                     description.initOnStartup();
 54                     description.getModuleDescription().registerStartupBean();
 55                 }
 56             }
 57         }
 58     }
 59 
 60     @Override
 61     protected void handleDeploymentDescriptor(final DeploymentUnit deploymentUnit, final DeploymentReflectionIndex deploymentReflectionIndex, final Class<?> componentClass, final SingletonComponentDescription description) throws DeploymentUnitProcessingException {
 62         SessionBeanMetaData data = description.getDescriptorData();
 63         if (data instanceof SessionBean31MetaData) {
 64             SessionBean31MetaData singletonBeanMetaData = (SessionBean31MetaData) data;
 65             Boolean initOnStartup = singletonBeanMetaData.isInitOnStartup();
 66             if (initOnStartup != null && initOnStartup) {
 67                 description.initOnStartup();
 68                 description.getModuleDescription().registerStartupBean();
 69             }
 70         }
 71     }
~~~

 * ee/src/main/java/org/jboss/as/ee/component/EEModuleDescription.java

~~~
308     public int getStartupBeansCount() {
309         return this.startupBeansCount;
310     }
311 
312     public int registerStartupBean() {
313         return ++this.startupBeansCount;
314     }
~~~

Comment 2 Masafumi Miura 2016-08-12 08:57:14 UTC
I think StartupMergingProcessor#handleDeploymentDescriptor() should skip a processing if it's already marked as InitOnStartup by StartupMergingProcessor#handleAnnotations().

So, I suggest that StartupMergingProcessor#handleDeploymentDescriptor() should check SingletonComponentDescription#isInitOnStartup() before continuing the processing. 

As far as I did a simple test (running the attached reproducer), the following patch can fix the issue: 

~~~
diff --git a/ejb3/src/main/java/org/jboss/as/ejb3/deployment/processors/merging/StartupMergingProcessor.java b/ejb3/src/main/java/org/jboss/as/ejb3/deployment/processors/merging/StartupMergingProcessor.java
index 5e1937f..7fc0d84 100644
--- a/ejb3/src/main/java/org/jboss/as/ejb3/deployment/processors/merging/StartupMergingProcessor.java
+++ b/ejb3/src/main/java/org/jboss/as/ejb3/deployment/processors/merging/StartupMergingProcessor.java
@@ -59,13 +59,17 @@ public class StartupMergingProcessor extends AbstractMergingProcessor<SingletonC
 
     @Override
     protected void handleDeploymentDescriptor(final DeploymentUnit deploymentUnit, final DeploymentReflectionIndex deploymentReflectionIndex, final Class<?> componentClass, final SingletonComponentDescription description) throws DeploymentUnitProcessingException {
-        SessionBeanMetaData data = description.getDescriptorData();
-        if (data instanceof SessionBean31MetaData) {
-            SessionBean31MetaData singletonBeanMetaData = (SessionBean31MetaData) data;
-            Boolean initOnStartup = singletonBeanMetaData.isInitOnStartup();
-            if (initOnStartup != null && initOnStartup) {
-                description.initOnStartup();
-                description.getModuleDescription().registerStartupBean();
+        if (description.isInitOnStartup()) {
+            // Skip. This is already marked as InitOnStartup by @Startup annotation.
+        } else {
+            SessionBeanMetaData data = description.getDescriptorData();
+            if (data instanceof SessionBean31MetaData) {
+                SessionBean31MetaData singletonBeanMetaData = (SessionBean31MetaData) data;
+                Boolean initOnStartup = singletonBeanMetaData.isInitOnStartup();
+                if (initOnStartup != null && initOnStartup) {
+                    description.initOnStartup();
+                    description.getModuleDescription().registerStartupBean();
+                }
             }
         }
     }
~~~

Comment 3 Masafumi Miura 2016-08-12 08:59:46 UTC
Created attachment 1190307 [details]
BZ1366526-potential-patch.diff

Comment 4 Masafumi Miura 2016-08-12 09:04:01 UTC
Sorry to forget to mention information about a workaround.

### Workaround

 - Don't use both @Startup annotation and <init-on-startup> in ejb-jar.xml to configure a singleton Startup beans
 - Specify either @Startup annotation or <init-on-startup>.

Comment 5 Fedor Gavrilov 2016-08-23 14:20:29 UTC
(In reply to Masafumi Miura from comment #4)
> Sorry to forget to mention information about a workaround.
> 
> ### Workaround
> 
>  - Don't use both @Startup annotation and <init-on-startup> in ejb-jar.xml
> to configure a singleton Startup beans
>  - Specify either @Startup annotation or <init-on-startup>.

Thank you!
This should be fine.

EAP6 PR: https://github.com/jbossas/jboss-eap/pull/2830

Comment 9 Jiří Bílek 2016-10-04 14:25:31 UTC
Verified with EAP 6.4.11.CP.CR1

Comment 10 Petr Penicka 2017-01-17 13:10:41 UTC
Retroactively bulk-closing issues from released EAP 6.4 cummulative patches.