Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1066592 - Document how to configure RTGov+DTGov+SRAMP to be clustered
Summary: Document how to configure RTGov+DTGov+SRAMP to be clustered
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Fuse Service Works 6
Classification: JBoss
Component: Documentation
Version: 6.0.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: DR1
: 6.1.0
Assignee: Petr Penicka
QA Contact: Stefan Bunciak
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-18 16:55 UTC by Stefan Bunciak
Modified: 2015-01-14 10:37 UTC (History)
4 users (show)

Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Build Name: 22586, Installation Guide-6-6 Build Date: 12-02-2014 11:58:00 Topic ID: 26834-576682 [Specified]
Last Closed: 2015-01-14 10:37:14 UTC
Type: Bug


Attachments (Terms of Use)

Description Stefan Bunciak 2014-02-18 16:55:43 UTC
Title: Run-Time Governance and S-RAMP in a Clustered Environment

Describe the issue:

Document how to run SOA Governance applications in clustered environment:
* with shared repository (single node clustered configuration), 
* with remote S-RAMP server (multiple node clustered configuration)
* remote repository provided by JDG is not supported at the moment

Either by pointing to appropriate EAP documentation and/or provide additional FSW specific steps.


Suggestions for improvement:


Additional information:

Governance wars are not distributable by default. This means that user needs to add <distributable/> to all web.xml files. See: https://bugzilla.redhat.com/show_bug.cgi?id=1035296

Modeshape & Infinispan subsystems needs to be configured in following manner:

	<subsystem xmlns="urn:jboss:domain:modeshape:1.0">

            ...
	    <repository name="sramp" cache-name="sramp" cache-container="modeshape" security-domain="overlord-idp" anonymous-roles="readonly" use-anonymous-upon-failed-authentication="false" cluster-name="sramp-cluster" cluster-stack="tcp">
		<indexing rebuild-upon-startup-mode="sync" rebuild-upon-startup-include-system-content="true"/>
                <local-file-index-storage path="modeshape/clustered-repo/${jboss.node.name}_indexes"/>
                <cache-binary-storage data-cache-name="s-ramp-binary-fs" metadata-cache-name="sramp-binary-fs-meta" cache-container="modeshape-binary-cache-container"/>
            </repository>
            ...
	</subsystem>

	<subsystem xmlns="urn:jboss:domain:infinispan:1.4">
            ...
	    <cache-container name="modeshape" module="org.modeshape" start="EAGER">
		<transport lock-timeout="60000"/>
                <replicated-cache name="sramp" mode="SYNC" batching="true">
                    <locking isolation="NONE"/>
                    <transaction mode="NON_XA"/>
                    <string-keyed-jdbc-store datasource="java:jboss/datasources/srampDS" passivation="false" purge="false">
                        <string-keyed-table prefix="ispn_bucket">
                            <id-column name="id" type="VARCHAR(500)"/>
                            <data-column name="datum" type="VARBINARY(60000)"/>
                            <timestamp-column name="version" type="BIGINT"/>
                        </string-keyed-table>
                    </string-keyed-jdbc-store>
                </replicated-cache>
            </cache-container>
	    <cache-container name="modeshape-binary-cache-container" aliases="modeshape-binary-cache" module="org.modeshape">
                <transport lock-timeout="60000"/>
                <replicated-cache name="sramp-binary-fs" mode="SYNC" batching="true">
                    <transaction mode="NON_XA"/>
                    <file-store relative-to="jboss.server.data.dir" path="modeshape/binary-store/clustered-sramp-binary-data-${jboss.node.name}" passivation="false" purge="false"/>
                </replicated-cache>
		<replicated-cache name="sramp-binary-fs-meta" mode="SYNC" batching="true">
                    <transaction mode="NON_XA"/>
                    <file-store relative-to="jboss.server.data.dir" path="modeshape/binary-store/clustered-sramp-binary-metadata-${jboss.node.name}" passivation="false" purge="false"/>
                </replicated-cache>
            </cache-container>
            ...
        </subsystem>

See: https://docs.jboss.org/author/display/MODE/Configuring+ModeShape+in+EAP

Comment 2 Horia Chiorean 2014-10-16 15:51:23 UTC
Another configuration option for the main "sramp" cache is to use invalidation mode:

  <invalidation-cache name="sramp" mode="SYNC" batching="true">
                    <locking isolation="NONE"/>
                    <transaction mode="NON_XA"/>
                    <string-keyed-jdbc-store datasource="java:jboss/datasources/srampDS" passivation="false" purge="false">
                        <string-keyed-table prefix="ispn_bucket">
                            <id-column name="id" type="VARCHAR(500)"/>
                            <data-column name="datum" type="VARBINARY(60000)"/>
                            <timestamp-column name="version" type="BIGINT"/>
                        </string-keyed-table>
                    </string-keyed-jdbc-store>
   </invalidation-cache>

The other binary caches (from above) would remain unchanged.

This would work only if all the nodes in the cluster connect(use) the same datasource to store data and it would be more performant in a read-heavy system.

Additional information about the Infinispan replicated & invalidation cache modes can be found at: https://docs.jboss.org/author/display/ISPN/Clustering+modes

Comment 3 Horia Chiorean 2014-10-17 06:03:37 UTC
Also, the above binary configuration uses <cache-binary-storage/> with replicated file-based stores for the binary values. This has the advantage of locality for each of the nodes, meaning that access (reading) binaries will be fast.
However, depending on the use case, it may be lacking in the following areas:
1) security - binaries are stored on disk under a given path
2) replication of each binary value across a cluster may cause a lot of unnecessary network traffic (for example when binaries are rarely used)

Depending on the usage scenario (i.e. is the system more read-intensive or write-intensive) there is also the option of configuring the binaries to be stored directly in the database (either the same DS as the main S-RAMP DS or another DS altogether).

This can be accomplished by using 
<repository ....>
   <db-binary-storage data-source-jndi-name="java:jboss/datasources/srampDS"/>         
</repository> 

instead of <cache-binary-storage/> in the repository configuration section.

Comment 4 Stefan Bunciak 2014-10-20 12:55:16 UTC
Hi Horia! You recommend the following configuration to be used for S-RAMP (in EAP), right?

        <subsystem xmlns="urn:jboss:domain:modeshape:1.0">
            ...
	    <repository name="sramp" cache-name="sramp" cache-container="modeshape" security-domain="overlord-idp" anonymous-roles="readonly" use-anonymous-upon-failed-authentication="false" cluster-name="sramp-cluster" cluster-stack="tcp">
                <db-binary-storage data-source-jndi-name="java:jboss/datasources/srampDS"/> 
            </repository>
            ...
	 </subsystem>

        <subsystem xmlns="urn:jboss:domain:infinispan:1.4">
            ...
	    <cache-container name="modeshape" module="org.modeshape" start="EAGER">
		<transport lock-timeout="60000"/>
                <invalidation-cache name="sramp" mode="SYNC" batching="true">
                    <locking isolation="NONE"/>
                    <transaction mode="NON_XA"/>
                    <string-keyed-jdbc-store datasource="java:jboss/datasources/srampDS" passivation="false" purge="false">
                        <string-keyed-table prefix="ispn_bucket">
                            <id-column name="id" type="VARCHAR(500)"/>
                            <data-column name="datum" type="VARBINARY(60000)"/>
                            <timestamp-column name="version" type="BIGINT"/>
                        </string-keyed-table>
                    </string-keyed-jdbc-store>
                </invalidation-cache>
            </cache-container>
            ...
        </subsystem>

Comment 5 Horia Chiorean 2014-10-20 13:04:09 UTC
No, not necessarily: the above option is one possible way in which clustering should work.
The other way in which it's possible to cluster is listed at the beginning of this issue, in the first comment.

The above configuration(s) are related to ModeShape/Infinispan. However, the S-RAMP guys should decide on any given scenario which type of configuration makes sense based on: the volume of data, binary usage scenarios etc. There is no "single correct way" of clustering.

Comment 7 Stefan Bunciak 2014-12-19 12:27:59 UTC
Few discrepancies present:
* "Add the <distributable/> entry inside the <jboss-web> element of the following web.xml files:" - there is no jboss-web element in web.xml file, distributable tag needs to be put inside web-app element.
* List of web.xml files is missing overlord-commons-idp.war/WEB-INF/web.xml and jboss-web.xml files overlord-commons-idp.war/WEB-INF/jboss-web.xml

Comment 8 Petr Penicka 2014-12-19 15:17:54 UTC
Thanks Stefan, available for review at the same URL.

* "Add the <distributable/> entry inside the <jboss-web> element of the following web.xml files:" - there is no jboss-web element in web.xml file, distributable tag needs to be put inside web-app element.

- the jboss-web element is in the jboss-web.xml files, I had them swapped, now should be correct

* List of web.xml files is missing overlord-commons-idp.war/WEB-INF/web.xml and jboss-web.xml files overlord-commons-idp.war/WEB-INF/jboss-web.xml

- added

Comment 9 Stefan Bunciak 2015-01-06 12:42:45 UTC
Verified.


Note You need to log in before you can comment on or make changes to this bug.