Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1431255 - dwz applied to a dts-compiled binary complains about section offsets not monotonically increasing
Summary: dwz applied to a dts-compiled binary complains about section offsets not mono...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: binutils
Version: 25
Hardware: Unspecified
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Nick Clifton
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1366145
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-10 17:59 UTC by Michael Petlan
Modified: 2017-12-12 10:48 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1366145
Environment:
Last Closed: 2017-12-12 10:48:58 UTC


Attachments (Terms of Use)

Description Michael Petlan 2017-03-10 17:59:10 UTC
+++ This bug was initially created as a clone of Bug #1366145 +++

When dwz is run on a simple binary compiled with devtoolset-6 toolchain, it complains about section offsets not monotonically increasing:

=======
# cat test.c 
int main() { return 0; }

# rpm -qf `which gcc objdump dwz`
devtoolset-6-gcc-6.1.1-4.1.el7.x86_64
devtoolset-6-binutils-2.27-3.el7.x86_64
devtoolset-6-dwz-0.12-1.el7.x86_64
# 
# gcc test.c -g -o test
# ./test
# 
# md5sum test
dcafda5ca2f21fe103f83f50f5f49622  test
# dwz ./test
dwz: Section offsets in ./test not monotonically increasing
# md5sum test
dcafda5ca2f21fe103f83f50f5f49622  test
#
=======

When this source gets compiled with the baseos toolchain (gcc-4.8.5-9.el7, binutils-2.25.1-20.base.el7), then devtoolset-6-dwz works fine on the resulting binary.

This reminds me the prelink bug 1357016 in some way.  Applies consistently to both rhel-6 and rhel-7, all supported arches.

Although this might not be a dwz bug, I'm filing it against dwz as a start.

--- Additional comment from Jakub Jelinek on 2016-08-12 10:38:55 EDT ---

Looks like binutils bug to me.
If I link with the DTS ld, I get:
...
  [25] .bss              NOBITS          0000000000601024 001024 000004 00  WA  0   0  4
  [26] .comment          PROGBITS        0000000000000000 001024 000059 01  MS  0   0  1
  [27] .debug_aranges    PROGBITS        0000000000000000 00107d 000030 00      0   0  1
  [28] .debug_info       PROGBITS        0000000000000000 0010ad 000054 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 001101 000037 00      0   0  1
  [30] .debug_line       PROGBITS        0000000000000000 001138 000039 00      0   0  1
  [31] .debug_str        PROGBITS        0000000000000000 001171 00005a 01  MS  0   0  1
  [32] .shstrtab         STRTAB          0000000000000000 001a28 000148 00      0   0  1
  [33] .symtab           SYMTAB          0000000000000000 0011d0 000678 18     34  53  8
  [34] .strtab           STRTAB          0000000000000000 001848 0001e0 00      0   0  1
in the test binary (note the sh_offset for .shstrtab being out of order).
Either .shstrtab should be emitted after .strtab instead of .symtab, because it is placed there, or it should use sh_offset 0x11d0 or better 0x11cb, then .symtab 0x1318 and finally .strtab 0x1990.
If I link with the system ld instead, I get sane ordering:
  [25] .bss              NOBITS          0000000000600834 000834 000004 00  WA  0   0  4
  [26] .comment          PROGBITS        0000000000000000 000834 000059 01  MS  0   0  1
  [27] .debug_aranges    PROGBITS        0000000000000000 00088d 00002f 00      0   0  1
  [28] .debug_info       PROGBITS        0000000000000000 0008bc 000042 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 0008fe 000037 00      0   0  1
  [30] .debug_line       PROGBITS        0000000000000000 000935 000039 00      0   0  1
  [31] .debug_str        PROGBITS        0000000000000000 00096e 00005a 01  MS  0   0  1
  [32] .shstrtab         STRTAB          0000000000000000 0009c8 000148 00      0   0  1
  [33] .symtab           SYMTAB          0000000000000000 0013d0 000678 18     34  50  8
  [34] .strtab           STRTAB          0000000000000000 001a48 000234 00      0   0  1
-fuse-ld=gold with DTS binutils also works sanely:
  [26] .bss              NOBITS          0000000000402010 001010 000004 00  WA  0   0  4
  [27] .comment          PROGBITS        0000000000000000 001010 00005a 01  MS  0   0  1
  [28] .debug_info       PROGBITS        0000000000000000 00106a 000054 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 0010be 000037 00      0   0  1
  [30] .debug_aranges    PROGBITS        0000000000000000 0010f5 000030 00      0   0  1
  [31] .debug_line       PROGBITS        0000000000000000 001125 000039 00      0   0  1
  [32] .debug_str        PROGBITS        0000000000000000 00115e 000068 01  MS  0   0  1
  [33] .note.gnu.gold-version NOTE            0000000000000000 0011c8 00001c 00      0   0  4
  [34] .symtab           SYMTAB          0000000000000000 0011e8 0003a8 18     35  22  8
  [35] .strtab           STRTAB          0000000000000000 001590 000234 00      0   0  1
  [36] .shstrtab         STRTAB          0000000000000000 0017c4 00016f 00      0   0  1
dwz isn't the only tool that expects sanely working linker.

--- Additional comment from Jakub Jelinek on 2016-08-12 12:15:07 EDT ---

I think it is 3e19fb8f990e4ce8a08f9cf2817cd9e9398648d5 https://sourceware.org/bugzilla/show_bug.cgi?id=18277 change that broke this.

--- Additional comment from Nick Clifton on 2016-08-12 12:44:09 EDT ---

Thanks for the pointer - I will look into it on Monday

--- Additional comment from Nick Clifton on 2016-08-15 12:15:11 EDT ---

Just to note that technically the linker is not doing anything wrong.  The ELF spec does not require monotonically increasing section offsets, so tools that rely upon such a feature are in fact broken.

Anyway I have tried several patches so far, but they all break something else in the section layout code.  This stuff is a mess...

--- Additional comment from Jakub Jelinek on 2016-08-15 15:21:23 EDT ---

I know it is technically valid ELF, but the sh_offset adjustments are hard enough even when they are monotonically increasing, if all the sections are laid out at random, having all tools cope with all possible section table vs. sh_offset orderings and special cases for .bss-ish and zero size sections is just too complex.  That is why dwz/prelink and other tools just give up early if they are invoked on too weird section layouts.

--- Additional comment from Jakub Jelinek on 2016-08-16 16:03 EDT ---

E.g. following patch IMHO should work (with minor needed adjustments in dozen or so tests that expect particular section order).

--- Additional comment from Nick Clifton on 2016-08-17 06:45:14 EDT ---

Hi Jakub,

> E.g. following patch IMHO should work (with minor needed adjustments in
> dozen or so tests that expect particular section order).

Doh!  That is a much better solution than the one that I was working on.

It does have one deficiency however - it does not reorder relocation sections, so these still appear out of order in the resultant binary.  Eg from the ld-x86-64/pr13947 test:

  % ../binutils/readelf -S tmpdir/dump --wide
There are 16 section headers, starting at offset 0x5b8:

Section Headers:
  [Nr] Name         Type        Address          Off    Size   ES Flg Lk Inf Al
...
  [ 6] .text        PROGBITS    0000000000000210 000210 000005 00  AX  0   0  1
  [ 7] .rela.text   RELA        0000000000000000 000530 000018 18   I 13   6  8
readelf: Warning: [ 8]: Section offsets not in a monotonically increasing order.
  [ 8] .eh_frame    PROGBITS    0000000000000218 000218 000000 00   A  0   0  8

I do not know if this will be a problem for prelink/dwz.  I am investigating to see whether I can safely extend your patch to cover these sections.

Cheers
  Nick

--- Additional comment from Jakub Jelinek on 2016-08-17 07:02:41 EDT ---

dwz/prelink only support non-ET_REL, so for those tools .rel* sections aren't a problem, and unlike the .shstrtab case, it seems as/ld has been doing that since like forever, at least looking at ET_REL files in RHEL4 confirms it.
Of course, if it isn't too hard on the linker side, adjusting even the .rel* sections wouldn't hurt, but it is much lower urgency.
Actually reading the code I wrote many years ago, it seems prelink actually has code to reorder the sections in the section table first so that it is monotonically increasing unless there are overlaps or similar bogosities, and then verifies this invariant before writing a new ELF object.
dwz doesn't have this code though, all linkers just have done a good job (until recently).  In prelink it isn't just sorting, one needs to update all the section references after the sorting.  And the comments indicate also various versions of libelf (the non-elfutils ones) have issues with non-monotonically increasing sh_offset.

--- Additional comment from Nick Clifton on 2016-08-18 03:32:46 EDT ---

Fixed in binutils-2.27-6.el7

--- Additional comment from  on 2016-10-20 09:52:25 EDT ---

Verified,
bug exists in devtoolset-6-binutils-2.27-5.el7,
bug is fixed in devtoolset-6-binutils-2.27-6.el7

Beaker log: https://url.corp.redhat.com/c1fb5a5

Comment 1 Michael Petlan 2017-03-10 18:02:40 UTC
Hit the same issue on Fedora-25, using binutils-2.26.1-1.fc25 and dwz-0.12-2.fc24. Not sure whether it is worth of backporting to Fedora-25 binutils, but filed anyway, in order to track it.

Comment 2 Nick Clifton 2017-03-13 10:52:06 UTC
I have backported the patch.  Please try: binutils-2.26.1-2.fc25

Comment 3 Fedora End Of Life 2017-11-16 19:43:43 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 4 Fedora End Of Life 2017-12-12 10:48:58 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.