Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1363728 - [RFE] Use separate stomp topic for each vdsm command in jsonrpc
Summary: [RFE] Use separate stomp topic for each vdsm command in jsonrpc
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Infra
Version: 4.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium vote
Target Milestone: ---
: ---
Assignee: Oved Ourfali
QA Contact: Pavel Stehlik
URL:
Whiteboard:
Depends On:
Blocks: 1275351 1298527 1349817
TreeView+ depends on / blocked
 
Reported: 2016-08-03 12:33 UTC by Martin Sivák
Modified: 2017-08-18 08:35 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-18 08:35:04 UTC
oVirt Team: Infra
mperina: ovirt-future?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)

Description Martin Sivák 2016-08-03 12:33:36 UTC
Description of problem:

All JSON-RPC VDSM commands are sent by engine using only two topics:

- jms.topic.vdsm_requests
- jms.topic.vdsm_irs_requests

This feature would convert that to for example:

jms.topic.vdsm_requests.getAllVmStats when calling getAllVmStats.

The response queue would follow the same model.


I already have a patch [1] that adds support for this on the receiving side of VDSM, by treating anything starting with jms.topic.vdsm_requests(.*) and jms.topic.vdsm_irs_requests(.*) as vdsm topic. The response side does not require any change as we use the Reply-To stomp header anyway and the engine controls the actual topic needed. The patch itself does not require any counterpart engine change, messages from old engines are still accepted and the Reply-To header is obeyed as usual.

[1] https://gerrit.ovirt.org/#/c/59638/

The last part missing is support for the extended topic name and response wildcard matching in the engine.

Reason:

This would allow other components (like MOM or hosted engine) to eavesdrop on the STOMP frames using the VDSM's minibroker infrastructure and that will allow us to get some information (VM died, migration finished) without querying vdsm directly. The listening component will of course send no reply to the received message. The constant polling is currently generating serious load when there are big numbers of VMs on a host, this change would remove most of it.

Comment 1 Martin Perina 2016-08-04 11:37:43 UTC
For now moving to ovirt_future, we need to discuss and prioritize 4.x plans around communication improvements

Comment 2 Yaniv Kaul 2017-06-07 19:18:43 UTC
(In reply to Martin Perina from comment #1)
> For now moving to ovirt_future, we need to discuss and prioritize 4.x plans
> around communication improvements

Can you discuss and prioritize now?

Comment 3 Piotr Kliczewski 2017-07-11 09:53:42 UTC
This rfe is very good idea but it would make backward compatibility harder between the versions. In the original design we went with the approach we have today due to the bugs that we found in activemq (we wanted to use it as messaging backbone). There are changes required in the engine which would make us to rethink the flows we have and how to handle subscriptions per verb.

In the original design we wanted to move towards events where we have no subscription limitations so I propose to add some of the events that MOM needs without making bigger redesigns.

Comment 4 Martin Sivák 2017-07-11 10:44:35 UTC
Where is the backwards compatibility issue?

- 4.2 VDSM already has the code that allows it to listen for the sub-topics and we can disable that for 4.1 hosts
- The reply topic name is always sent as a header so vdsm does not need to think about it at all, it will just use whatever came in with the request

Are there any other places where the topic names are important?

Comment 5 Piotr Kliczewski 2017-07-11 10:56:48 UTC
(In reply to Martin Sivák from comment #4)
> Where is the backwards compatibility issue?
> 
> - 4.2 VDSM already has the code that allows it to listen for the sub-topics
> and we can disable that for 4.1 hosts

When we install vdsm first we subscribe to specific queue and later we query vdsm for version information. How should we subscribe to get the response for specific
queue. At this stage we do not know whether vdsm contains [1] patch or not.

> - The reply topic name is always sent as a header so vdsm does not need to
> think about it at all, it will just use whatever came in with the request

When we receive SEND frame we need to know whether to call API.py or just deliver
it to specific queue.

> 
> Are there any other places where the topic names are important?

Comment 6 Martin Sivák 2017-07-11 13:07:09 UTC
> When we install vdsm first we subscribe to specific queue and later we query
> vdsm for version information. How should we subscribe to get the response
> for specific
> queue. At this stage we do not know whether vdsm contains [1] patch or not.

Why not subscribe to both then?


> When we receive SEND frame we need to know whether to call API.py or just
> deliver
> it to specific queue.

That is already handled by the vdsm side. It delivers everything that starts with the known topic root to API.py. You can send the initial message to the root to figure out what version of vdsm you are talking to.

VDSM in 4.2 actually does not care about subtopics. But other components (mom, hosted engine) can subscribe to only some of them and that avoids the overhead  of parsing and dropping all the non-important ones.

Comment 7 Martin Sivák 2017-07-11 13:08:50 UTC
Actually, you do not have to subscribe to both, you can just use the root topic for the Reply-To header during the initial exchange. Basically treating the host as a 4.1 host before you detect newer version. 4.2 VDSM still allows that (it is used like that right now).

Comment 8 Piotr Kliczewski 2017-07-11 14:11:52 UTC
I am not saying it is not possible to do it but there is a bit of code to be written to handle all the compatibility changes.

I see much clearer solution using events. It would be least invasive approach to provide information for MOM.

Comment 9 Martin Sivák 2017-07-11 14:56:39 UTC
MOM (and hosted engine) would have to listen to everything and filter out a huge amount of frames (because of VM status and migrations) or invent a whole new topic and VDSM would have to start sending events to that MOM specific topic. That is not how topics are used..

Hardly less invasive than just using the current method to get the host version (we can even do that based on the cluster compatibility level) and enabling the topic suffixes only for 4.2 and higher.

Comment 10 Piotr Kliczewski 2017-07-11 15:29:58 UTC
I am proposing adding new queues/topics for specific event types which could be consumed by anyone who is interested in receiving those. There would be no need for filtering.

Please note that we would need to detect vdsm version and act accordingly. New 4.2 vdsms would need to work with older versions of engines as well.

Comment 11 Martin Sivák 2017-07-27 12:31:41 UTC
(In reply to Piotr Kliczewski from comment #10)
> I am proposing adding new queues/topics for specific event types which could
> be consumed by anyone who is interested in receiving those. There would be
> no need for filtering.

I am not talking about events (how many do we have? 2?). I am talking about RPC commands that are all sent to a single topic.

> Please note that we would need to detect vdsm version and act accordingly.
> New 4.2 vdsms would need to work with older versions of engines as well.

But they do already! VDSM does not care about the suffix and it just uses whatever you set to the Reply-To header. This is already done, merged and working. No additional code change is needed.

Comment 12 Piotr Kliczewski 2017-07-27 14:35:24 UTC
(In reply to Martin Sivák from comment #11)
> (In reply to Piotr Kliczewski from comment #10)
> > I am proposing adding new queues/topics for specific event types which could
> > be consumed by anyone who is interested in receiving those. There would be
> > no need for filtering.
> 
> I am not talking about events (how many do we have? 2?). I am talking about
> RPC commands that are all sent to a single topic.
>

I see this as improvement if we would add new events.
 
> > Please note that we would need to detect vdsm version and act accordingly.
> > New 4.2 vdsms would need to work with older versions of engines as well.
> 
> But they do already! VDSM does not care about the suffix and it just uses
> whatever you set to the Reply-To header. This is already done, merged and
> working. No additional code change is needed.

This implies that specific commands/verbs are aware of subscriptions which is bad idea.

Comment 13 Martin Perina 2017-08-18 08:35:04 UTC
We are not going to implement this. As described above it's possible to achieve similar result by adding more events, which don't cause any compatibility issues or requiring deep infrastructural changes


Note You need to log in before you can comment on or make changes to this bug.