Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.

Bug 1366636

Summary: [RFE] ceph-installer does not populate public and cluster network values in ceph.conf global section
Product: Red Hat Storage Console Reporter: Shubhendu Tripathi <shtripat>
Component: ceph-installerAssignee: Gregory Meno <gmeno>
Status: CLOSED NOTABUG QA Contact: ceph-qe-bugs <ceph-qe-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 2CC: adeza, aschoen, ceph-eng-bugs, kdreyer, nthomas, sankarshan, shtripat
Target Milestone: ---   
Target Release: 3   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-13 12:49:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Description Flags
failed task output none

Description Shubhendu Tripathi 2016-08-12 13:33:33 UTC
Description of problem:
While create cluster flow using ceph-installer, there is no values set for public_network and cluster_network on global section fo ceph.conf. This results in calamari returning blank values for public_network and cluster_network in /api/v2/cluster/<fsid>/cluster or /api/v2/cluster/<fsid>/sync_object/config.

Due to this issue, expand cluster feature is broken in console for an imported cluster (which was earlier created and forgot from console).

Be it ceph-installer or ceph-ansible both the cases public_network and cluster_network both the values should be set in global section of ceph.conf

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Create a cluster with say 3mon + 2 osd nodes (4 osd in total) in console. Un-manage and forget the cluster from console
Create a cluster using ceph-ansible from command line
2. Import the cluster in console
3. try to add another OSD node with say 4 disks

Actual results:
Expand cluster fails with error "unable to parse address for mon.<node name>" in ceph-installer task

Expected results:
Expand cluster should be successful without any error in ceph-installer task

Additional info:
This happened as there were no public and cluster network values available for nodes of the imported cluster and so the configure osd call to ceph-installer was passed with blank adsresses for mon nodes and so the failure.

Comment 3 Alfredo Deza 2016-08-12 14:52:04 UTC
Please provide the following information:

* versions of the products used
* configuration(s) used to trigger the current issue, specifically the changed values from defaults
* actual ansible commands used (including flags and options)
* full output of the ansible playbook run

If not interacting with ceph-ansible directly, all ansible information is available in the ceph-installer HTTP API:

Comment 4 Shubhendu Tripathi 2016-08-12 15:11:38 UTC
Created attachment 1190460 [details]
failed task output

Comment 5 Shubhendu Tripathi 2016-08-12 15:13:21 UTC
ceph-installer version : ceph-installer-1.0.15-1.el7scon.noarch
ceph-ansible version: ceph-ansible-1.0.5-32.el7scon.noarch

Comment 6 Alfredo Deza 2016-08-12 17:24:06 UTC
The HTTP API is not receiving any values for public_network/cluster_network.

From the attachment, this is how the request looks:

    --> request: {
        "conf":{"global":{"osd crush update on start":false}},

Both cluster_network and public_network are empty strings. The client (USM) needs to pass those if the end result wants them defined. 

A sample request with those values is documented in the OSD configuration section:

Comment 8 Shubhendu Tripathi 2016-08-16 03:53:30 UTC
Correct Alfredo, USM is not able to pass the values of public and cluster network as it takes that from calamari and same is not available because in the very first time while cluster creation these values are not being set in ceph.conf file.

I understand ceph-installer should set these while cluster creation in ceph.conf file.

Comment 9 Ken Dreyer (Red Hat) 2017-03-02 17:30:30 UTC
Discussed with Andrew today. Console has to give us public and cluster network settings from the very first cluster creation operation, or else there is no way to know what values to use here. Where would you expect to obtain this IP information otherwise?

Comment 10 Shubhendu Tripathi 2017-03-13 11:44:12 UTC
I understand that public and cluster network values can be passed while first mon configuration. If thats the case Tendrl now does the same and it should be fine.
Verified in cluster created by latest ceph-installer and I see the values of public and cluster network populated properly under global section of ceph.conf file as below

fsid = 140cd3d5-58e4-4935-a954-d946ceff371d
max open files = 131072
mon initial members = mon-node-1,mon-node-2,mon-node-3
mon host = mon-ip-1,mon-ip-2,mon-ip-3
public network =
cluster network =

Looks good to me.