Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1532391 - mkvtoolnix: FTBFS in Fedora rawhide
Summary: mkvtoolnix: FTBFS in Fedora rawhide
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: mkvtoolnix
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dominik 'Rathann' Mierzejewski
QA Contact: Fedora Extras Quality Assurance
URL: http://apps.fedoraproject.org/koschei...
Whiteboard:
Depends On:
Blocks: 1515518
TreeView+ depends on / blocked
 
Reported: 2018-01-08 21:39 UTC by Dominik 'Rathann' Mierzejewski
Modified: 2018-01-19 08:11 UTC (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-01-19 08:11:59 UTC


Attachments (Terms of Use)

Description Dominik 'Rathann' Mierzejewski 2018-01-08 21:39:45 UTC
Description of problem:
Package mkvtoolnix fails to build from source in Fedora rawhide.

Version-Release number of selected component (if applicable):
19.0.0-1.fc28

Steps to Reproduce:
koji build --scratch f28 mkvtoolnix-19.0.0-1.fc28.src.rpm

Additional info:
This package is tracked by Koschei. See:
http://apps.fedoraproject.org/koschei/package/mkvtoolnix

It looks like this was caused by https://fedoraproject.org/wiki/Changes/Ruby_2.5.

The following can be seen in build log:
[...]
rake aborted!
File exists @ dir_s_mkdir - ./rake.d/dependency.d

Comment 1 Vít Ondruch 2018-01-10 17:59:56 UTC
This appears to be race condition:

~~~
+ drake -j4 V=1 --trace

... snip

** Invoke apps:tools:ebml_validator 
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_common_pch_h_gch.d -o src/common/common_pch.h.gch -x c++-header src/common/common_pch.h
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_terminal_o.d -o src/common/terminal.o -x c++ src/common/terminal.cpp
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_utf8_codecvt_facet_o.d -o src/common/utf8_codecvt_facet.o -x c++ src/common/utf8_codecvt_facet.cpp
** Execute src/common/utf8_codecvt_facet.o
** Execute src/common/win_itaskbarlist3.o
** Execute src/common/common_pch.h.gch
** Execute src/common/terminal.o
rake aborted!
File exists @ dir_s_mkdir - ./rake.d/dependency.d
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:95:in `mkdir'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:95:in `ensure_dir'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `block in create_dependency_dirs'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `each'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `create_dependency_dirs'
/builddir/build/BUILD/mkvtoolnix-19.0.0/Rakefile:261:in `block in <top (required)>'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:229:in `block in execute'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:224:in `each'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:224:in `execute'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/parallel.rb:76:in `block (3 levels) in compute'
/usr/share/gems/gems/comp_tree-1.1.3/lib/comp_tree/node.rb:89:in `compute'
/usr/share/gems/gems/comp_tree-1.1.3/lib/comp_tree/algorithm.rb:25:in `block in new_worker'
~~~

So there are 4 tasks running in parallel, and they are trying to execute this method:

~~~
def ensure_dir dir
  File.unlink(dir) if  FileTest.exist?(dir) && !FileTest.directory?(dir)
  Dir.mkdir(dir)   if !FileTest.exist?(dir)
end
~~~

Although there are 'exists?' checks, if the directory is created in parallel running task meanwhile, then the re-creation must fail.

Not really sure what is the best option here. For you, the simplest option is either not running in parallel (i.e. drop the %{?_smp_mflags} or user rake instead of rake) or pre-create the directories:

~~~
diff --git a/mkvtoolnix.spec b/mkvtoolnix.spec
index 4a9eec4..5c8bddf 100644
--- a/mkvtoolnix.spec
+++ b/mkvtoolnix.spec
@@ -70,6 +70,9 @@ rm -rf rake.d/vendor drake
   --with-boost-libdir=%{_libdir} \
   --with-tools \
   || cat config.log
+
+mkdir -p rake.d/dependency.d/tmp
+
 drake %{?_smp_mflags} V=1
 
 %install
~~~

And for upstream? Simply make the ensure_dir thread safe. May be something like:

~~~
sed -i 's/Dir\.mkdir.*$/FileUtils.mkdir_p(dir)/' rake.d/helpers.rb
~~~

could be enough, but not sure how safe/atomic the FileUtils.mkdir_p is.

Comment 2 Dominik 'Rathann' Mierzejewski 2018-01-10 21:31:34 UTC
Thanks for the analysis. I'll forward it upstream. Any idea why it always worked before Ruby 2.5 update?

Comment 3 Vít Ondruch 2018-01-11 09:35:17 UTC
(In reply to Dominik 'Rathann' Mierzejewski from comment #2)
> Any idea why it always worked before Ruby 2.5 update?

I knew you are going to ask this question ;) Short answer could be "not really", but I think that Ruby usptream is trying to make Ruby more concurrent/parallel. So if they can, the remove the GIL from various places of Ruby.

Actually, I read the upstream ticket and due to my curiosity, I found this upstream commit which is likely the root cause of this issue:

https://github.com/ruby/ruby/commit/f5f6232399b1e4bf6b702b51bef4d9e36b42df1b

Comment 4 Dominik 'Rathann' Mierzejewski 2018-01-11 10:03:46 UTC
Sounds plausible. Thanks!

Also, upstream has fixed that already, so the fix will be present in the next release.

Comment 5 Dominik 'Rathann' Mierzejewski 2018-01-19 08:11:59 UTC
Fixed across all Fedora branches, but really relevant only for rawhide.


Note You need to log in before you can comment on or make changes to this bug.