Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1509552 - [Doc] Incorrect limit regarding maximum number of volumes in CNS 3.6 Deployment Guide
Summary: [Doc] Incorrect limit regarding maximum number of volumes in CNS 3.6 Deployme...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: doc-Container_Native_Storage_with_OpenShift
Version: cns-3.6
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: CNS 3.6 Async
Assignee: Bhavana
QA Contact: storage-qa-internal@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-04 15:05 UTC by khartsoe@redhat.com
Modified: 2018-01-06 17:18 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-01-06 17:18:42 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description khartsoe@redhat.com 2017-11-04 15:05:23 UTC
Description of problem:
Section 5.2.5 Planning Guidelines incorrectly states that the maximum number of volumes is 300: 

"Ensure that the Trusted Storage Pool is not scaled beyond 300 volumes per 3 nodes per 32GB of RAM."

This should state "1,000", not "300".

https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/container-native_storage_for_openshift_container_platform/#Planning_Guidelines

Version-Release number of selected component (if applicable):
CNS 3.6

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
Statement should read as follows:
"Ensure that the Trusted Storage Pool is not scaled beyond 1,000 volumes per 3 nodes per 32GB of RAM."

Additional info:

Comment 5 Sudhir 2017-11-28 08:50:00 UTC
Kindly modify this section as follows:

The following are the guidelines for setting up Red Hat Container Native Storage or Container Ready Storage with OpenShift Container Platform.

Ensure that the Trusted Storage Pool is appropriately sized and you have room for dynamic scaling on demand. This is to ensure you do not scale beyond the maximum limits:

Sizing Guidelines:

•	PV's backed by file interface: Size for 300-500 PV's backed by Files on three node CNS /CRS cluster for normal operation. The max limit of supported PV backed by the file interface is 1000 PVs per three-node cluster of CNS and CRS deployment.  Given nature of micro-services where they can dynamically scale as per demand, it is recommended that initial sizing keep sufficient headroom for the scaling.  If further scaling is needed please add a new three node CNS/CRS cluster to support additional PV’s.

•	PV's backed by Block-based storage: Size for max of 300 persistent volumes per three nodes of CNS or CRS. Please note CNS/CRS 3.6 only supports OCP logging and metrics on block backed PV's.

•	PV's backed by File and Block: Size for 300-500 PV's backed byFiles and 100-200 PV's backed by block. Do not exceed max limits of file or block backed PV and combination of max 1000 PV's per three node CNS/CRS.

•	3-way distributed-replicated volumes is the only supported volume type.

•	Each physical or virtual node that needs to host a Red Hat Gluster CNS/CRS peer:
         *will need a minimum of 8 GB RAM + 30MB per persistent volume.
         *is expected to have the same disk type.
         *by default the heketidb utilises 2 GB distributed replica volume.

•	Deployment guidelines on CNS and CRS
        *In CNS mode you may install the CNS nodes, Heketi, and all provisioners pods on the OCP Infrastructure nodes or OCP Application nodes
        *In CRS mode you may install the Heketi and all provisioners pods on OCP Infrastructure nodes or on OCP Application nodes


Also remove reference to snapshot line - "Red Hat Gluster Storage Container Native with OpenShift Container Platform supports up to 14 snapshots per volume."  Not sure where this came from

Comment 6 Atin Mukherjee 2017-11-28 09:39:31 UTC
(In reply to Sudhir from comment #5)
> Kindly modify this section as follows:
> 
> The following are the guidelines for setting up Red Hat Container Native
> Storage or Container Ready Storage with OpenShift Container Platform.
> 
> Ensure that the Trusted Storage Pool is appropriately sized and you have
> room for dynamic scaling on demand. This is to ensure you do not scale
> beyond the maximum limits:
> 
> Sizing Guidelines:
> 
> •	PV's backed by file interface: Size for 300-500 PV's backed by Files on
> three node CNS /CRS cluster for normal operation. The max limit of supported
> PV backed by the file interface is 1000 PVs per three-node cluster of CNS
> and CRS deployment.  Given nature of micro-services where they can
> dynamically scale as per demand, it is recommended that initial sizing keep
> sufficient headroom for the scaling.  If further scaling is needed please
> add a new three node CNS/CRS cluster to support additional PV’s.
> 
> •	PV's backed by Block-based storage: Size for max of 300 persistent volumes
> per three nodes of CNS or CRS. Please note CNS/CRS 3.6 only supports OCP
> logging and metrics on block backed PV's.
> 
> •	PV's backed by File and Block: Size for 300-500 PV's backed byFiles and
> 100-200 PV's backed by block. Do not exceed max limits of file or block
> backed PV and combination of max 1000 PV's per three node CNS/CRS.
> 
> •	3-way distributed-replicated volumes is the only supported volume type.
> 
> •	Each physical or virtual node that needs to host a Red Hat Gluster CNS/CRS
> peer:
>          *will need a minimum of 8 GB RAM + 30MB per persistent volume.
>          *is expected to have the same disk type.
>          *by default the heketidb utilises 2 GB distributed replica volume.
> 
> •	Deployment guidelines on CNS and CRS
>         *In CNS mode you may install the CNS nodes, Heketi, and all
> provisioners pods on the OCP Infrastructure nodes or OCP Application nodes
>         *In CRS mode you may install the Heketi and all provisioners pods on
> OCP Infrastructure nodes or on OCP Application nodes
> 
> 
> Also remove reference to snapshot line - "Red Hat Gluster Storage Container
> Native with OpenShift Container Platform supports up to 14 snapshots per
> volume."  Not sure where this came from

We need to definitely have a restriction on the number of snapshots per volume, otherwise as technically the snapshots are nothing but the gluster volumes, there'd be a CPU/memory contention with too many volumes in the system. Also to note with USS, for each snapshot we have individual snap daemon running which is going to add up to the overall memory consumption.

Comment 8 Sudhir 2017-11-28 16:06:40 UTC
Atin,
Where does 14 snapshots come from?  I am ok specifying that snapshots will be counted as volume. Is that you meant or there is a technical limit of 14?

If we just leave it as it is, then users will size it for 500 volume and then possibly 14 snapshots each.

Can you tell us

1. What is the technical limit if any on snapshot on each volume?
2. Is it ok to count it as volume and size it based on that?

Lets be prescriptive on this

Comment 9 Atin Mukherjee 2017-11-28 16:55:21 UTC
This number was derived during the initial sizing of the max volumes and snapshots to be supported in 32GB system in CNS setup. As far as I remember this was done during CNS 3.4 or 3.5. Michael might be able to help you with a link to the document where all the discussions were noted and these limits were derived at. I wasn’t part of the discussion though.

Having said this I like your suggestion to highlight about snapshots to be considered as part of max number of volumes to be supported but this needs a sign off from CNS dev & other stakeholders. With that we don’t need to mention about the snapshot limit.

Comment 14 Sudhir 2017-12-08 05:19:54 UTC
@Ken,

Please publish the limits as I indicated above.  Modify the snapshot limits as follows

*Red Hat Gluster Storage Container Native with OpenShift Container Platform supports up to 14 snapshots per volume by default (snap-max-hard-limit =14 in Heketi Template)

Comment 18 Sudhir 2017-12-22 23:05:25 UTC
looks good


Note You need to log in before you can comment on or make changes to this bug.