Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1062480 - Queries in beakerd processing loops are too slow
Summary: Queries in beakerd processing loops are too slow
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Community
Component: scheduler
Version: 0.15
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: 0.15.4
Assignee: Raymond Mancy
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-07 02:55 UTC by Raymond Mancy
Modified: 2018-02-06 00:41 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-02-18 01:38:51 UTC


Attachments (Terms of Use)

Description Raymond Mancy 2014-02-07 02:55:50 UTC
Description of problem:

On large installations, the beakerd processing loops can be quite slow

Version-Release number of selected component (if applicable):


How reproducible:

Always on a large beaker installation

Steps to Reproduce:
1. Set the slow query log to log anything over 15 seconds
2. Queries from beakerd processing loops turn up there
3.

Actual results:

These kind of queries show up:

SELECT count(1) AS count_1 
FROM recipe_set INNER JOIN job ON job.id = recipe_set.job_id 
WHERE job.dirty_version = job.clean_version AND NOT (EXISTS (SELECT 1 
FROM recipe 
WHERE recipe_set.id = recipe.recipe_set_id AND recipe.status != 'Scheduled'));

Expected results:

Not to be there

Additional info:

A simple fix is to add job.deleted IS NULL to the WHERE clause. In my testing, that has brought down the query times from ~15-20 seconds to 4-6 seconds.

Comment 2 Nick Coghlan 2014-02-07 04:37:36 UTC
Target will be to get these queries back to subsecond execution times in 0.16, but a 3x improvement is acceptable for 0.15.4.

Comment 3 Raymond Mancy 2014-02-07 08:02:17 UTC
Just to clarify, those subsecond execution times used to exist when we had top down status updates, and the query was much simpler than then kind of query we now require. 

That's not to say that they cannot be improved upon further than what BZ#1062480 will do, but unfortunately there is no prior query that we can revert back to, to restore the performance we would like.

Comment 4 Raymond Mancy 2014-02-12 10:59:56 UTC
Gerrit review,

http://gerrit.beaker-project.org/#/c/2798/

Comment 7 xjia 2014-02-14 10:18:00 UTC
After smoke test, QE didn't find regression bug for this issue. So move to "VERIFIED"

Comment 8 Dan Callaghan 2014-02-18 01:38:51 UTC
Beaker 0.15.4 has been released.


Note You need to log in before you can comment on or make changes to this bug.