Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1600871 - Pod memory discrepancies between web console and cmdline [NEEDINFO]
Summary: Pod memory discrepancies between web console and cmdline
Keywords:
Status: NEW
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.6.z
Assignee: Ruben Vargas Palma
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-13 08:55 UTC by Sudarshan Chaudhari
Modified: 2019-03-08 22:43 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
jsanda: needinfo? (suchaudh)


Attachments (Terms of Use)
Memory Metrics of the pod (deleted)
2018-07-13 08:55 UTC, Sudarshan Chaudhari
no flags Details

Description Sudarshan Chaudhari 2018-07-13 08:55:19 UTC
Created attachment 1458686 [details]
Memory Metrics of the pod

Description of problem:

When checking the memory utilization of pods, the memory consumption in Openshift Console Metrics differs from the command line after checking using the command: `oc adm top pod`


For the pod:
[root@master01 ~]# oc get pods
NAME            READY     STATUS    RESTARTS   AGE
httpd-1-qv7mx   1/1       Running   0          9d


From Openshift Console:
Memory is: 15.9Mi 

Check the attached SNAP(Memory usage).

From the Command Line:
[root@master01 ~]# oc adm top pods
NAME            CPU(cores)   MEMORY(bytes)   
httpd-1-qv7mx   2m           10Mi 


In other test, the difference was lot:
From Openshift Console:
  Memory of pod: 2047Mi

From Command Line:
  Memory of Pod: 1795Mi
[elijah_eng@master01 ~]$ oc adm top pod 
NAME                 CPU(cores)   MEMORY(bytes)
pr-v2-8-k3rwb        13m          1799Mi
pr-v2-8-nsv6k        19m          1793Mi
pr-v2-8-cffvv        9m           1800Mi
pr-v2-8-fjp4v        14m          1795Mi


How reproducible:
Always. 

I have reproduced on OCP 3.6 and 3.9

Steps to Reproduce:
1. Check the metrics on Openshift console > Pods
2. run the command
  # oc adm top pod


Actual results:
The Memory usage is different

Expected results:
The metrics should be the same in GUI as well as CLI

Additional info:
Need justification/fix on this.

Comment 1 Sudarshan Chaudhari 2018-07-17 05:48:25 UTC
Hello, 

Is there any update on this?

Thanks.

Comment 2 John Sanda 2018-07-17 13:50:32 UTC
I was out all last week and just saw this ticket for the first time yesterday. The console queries Hawkular Metrics for the values which are aggregated to provide a fixed number of data points for a given time range, e.g., 60 data points over the past two hours. The raw data is collected from cAdvisor on each node by Heapster and then sent to Hawkular Metrics for persistence. I am not sure how `oc adm top pod` works, but I do know that it does *not* fetch values from Hawkular Metrics. It could be displaying raw values. I am not at all surprised that there is some discrepancy.


Note You need to log in before you can comment on or make changes to this bug.