Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1512085 - Error on run_sync_thread: StaleDataError
Summary: Error on run_sync_thread: StaleDataError
Keywords:
Status: CLOSED DUPLICATE of bug 1487783
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-networking-odl
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Mike Kolesnik
QA Contact: Itzik Brown
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-10 19:43 UTC by Sai Sindhur Malleni
Modified: 2018-10-24 12:41 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
N/A
Last Closed: 2018-02-19 11:36:46 UTC


Attachments (Terms of Use)

Description Sai Sindhur Malleni 2017-11-10 19:43:14 UTC
Description of problem:
On running perf/scale tests on an OSP 12 cloud with 3 OpenStack controllers + 3 ODLs + 28 computes, by creating 100s of neutron resources and deleting them we see several StaleDataErrors in the neutron-server logs.  

Version-Release number of selected component (if applicable):
OSP12
Puddle: 2017-10-31.2
ODL RPM: opendaylight-6.2.0-3.el7ost.noarch
python-networking-odl-11.0.1-0.20171017155941.el7ost.noarch

How reproducible:


Steps to Reproduce:
1. Run perf/scale tests using Browbeat+neutron and create large number of neutro nresources
2.
3.

Actual results:
Errors in logs

Expected results:
No errors

Additional info:
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal [req-0ed305a2-9f63-4b3a-bec6-370a271fde5a - - - - -] Error on run_sync_thread: StaleDataError: UPDATE statement on table 'opendaylightjournal' expected to update 1 row(s); 0 were matched.
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal Traceback (most recent call last):
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/networking_odl/journal/journal.py", line 261, in run_sync_thread
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self.sync_pending_entries()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/networking_odl/journal/journal.py", line 279, in sync_pending_entries
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     entry = db.get_oldest_pending_db_row_with_lock(context.session)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 93, in wrapped
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     setattr(e, '_RETRY_EXCEEDED', True)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self.force_reraise()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     six.reraise(self.type_, self.value, self.tb)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 89, in wrapped
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     return f(*args, **kwargs)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 150, in wrapper
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     ectxt.value = e.inner_exc
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self.force_reraise()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     six.reraise(self.type_, self.value, self.tb)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 138, in wrapper
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     return f(*args, **kwargs)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 128, in wrapped
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     LOG.debug("Retry wrapper got retriable exception: %s", e)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self.force_reraise()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     six.reraise(self.type_, self.value, self.tb)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     return f(*dup_args, **dup_kwargs)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/networking_odl/db/db.py", line 86, in get_oldest_pending_db_row_with_lock
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     update_db_row_state(session, row, odl_const.PROCESSING)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib/python2.7/site-packages/networking_odl/db/db.py", line 102, in update_db_row_state
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     session.merge(row)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1839, in merge
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self._autoflush()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1397, in _autoflush
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self.flush()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2171, in flush
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     self._flush(objects)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2291, in _flush
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     transaction.rollback(_capture_exception=True)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     compat.reraise(exc_type, exc_value, exc_tb)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2255, in _flush
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     flush_context.execute()
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     rec.execute(self)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     uow
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     mapper, table, update)
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 760, in _emit_update_statements
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal     (table.description, len(records), rows))
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal StaleDataError: UPDATE statement on table 'opendaylightjournal' expected to update 1 row(s); 0 were matched.
2017-11-10 19:16:02.276 721543 ERROR networking_odl.journal.journal
2017-11-10 19:16:02.277 721543 DEBUG networking_odl.journal.journal [req-0ed305a2-9f63-4b3a-bec6-370a271fde5a - - - - -] Start processing journal entries sync_pending_entries /usr/lib/python2.7/site-packages/networking_odl/journal/journal.py:267

Comment 1 Mike Kolesnik 2018-02-19 11:36:46 UTC

*** This bug has been marked as a duplicate of bug 1487783 ***


Note You need to log in before you can comment on or make changes to this bug.