Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 448586 - Firefox 3 crashes Xorg at picture.c:1600
Summary: Firefox 3 crashes Xorg at picture.c:1600
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xorg-x11-server
Version: 5.2
Hardware: All
OS: Linux
urgent
high
Target Milestone: rc
: ---
Assignee: Adam Jackson
QA Contact: desktop-bugs@redhat.com
URL:
Whiteboard:
: 453607 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-05-27 18:16 UTC by Richard Ryder
Modified: 2009-09-02 11:42 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-09-02 11:42:21 UTC
Target Upstream Version:


Attachments (Terms of Use)
Xorg core generated by gdb gcore command (deleted)
2008-05-27 18:16 UTC, Richard Ryder
no flags Details
dmesg from sosreport (deleted)
2008-06-06 13:54 UTC, Matěj Cepl
no flags Details
xorg.conf from sosreport (deleted)
2008-06-06 13:54 UTC, Matěj Cepl
no flags Details
Xorg.0.log from sosreport (deleted)
2008-06-06 13:54 UTC, Matěj Cepl
no flags Details
test case (deleted)
2009-02-02 18:46 UTC, Pavel Kankovsky
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2009:1373 normal SHIPPED_LIVE xorg-x11-server bug fix and enhancement update 2009-09-01 11:12:50 UTC
Red Hat Product Errata RHBA-2009:1391 normal SHIPPED_LIVE xorg-x11-drv-i810 bug fix and enhancement update 2009-09-01 11:58:54 UTC

Description Richard Ryder 2008-05-27 18:16:36 UTC
Description of problem:

Visiting http://developer.pidgin.im/ticket/4986 with Firefox 3.0b5 will cause
Xorg to crash.


Version-Release number of selected component (if applicable):
RHEL5.2 i386 and x86_64
kernel-2.6.18-92.el5
xorg-x11-server-Xorg-1.1.1-48.41.el5
firefox-3.0-0.beta5.6.el5

How reproducible:
100%.  Tested on i386 and x86_64

Steps to Reproduce:
1.Install RHEL5.2
2.Open Firefox and vist http://developer.pidgin.im/ticket/4986
3.As the page is loading Xorg will crash
  
Actual results:
Xorg crashes

Expected results:
Xorg doesn't crash. 

Additional info:
Core and sosreport attached.  No NVidia driver, firefox plugins, or extensions
are installed.  Video driver is i810, also duplicated on nv.  Does not cause
crash on F9.

Program terminated with signal 11, Segmentation fault.
#0  0x08147845 in ValidateOnePicture (pPicture=0x963a730) at picture.c:1600
1600            (*ps->ValidatePicture) (pPicture, pPicture->stateChanges);
(gdb) thread apply all bt

Thread 1 (process 2887):
#0  0x08147845 in ValidateOnePicture (pPicture=0x963a730) at picture.c:1600
#1  0x08147881 in ValidatePicture (pPicture=0x963a730) at picture.c:1609
#2  0x08147941 in CompositePicture (op=12 '\f', pSrc=0x96437b8, pMask=0x963a730,
pDst=0x963a588, xSrc=0, ySrc=0, xMask=0, yMask=0, xDst=0, yDst=0, width=1,
height=1) at picture.c:1782
#3  0x08146d68 in miTrapezoids (op=12 '\f', pSrc=0x96437b8, pDst=0x963a588,
maskFormat=<value optimized out>, xSrc=1, ySrc=0, ntrap=0, traps=0x9711294) at
mitrap.c:175
#4  0x0815e1d4 in cwTrapezoids (op=12 '\f', pSrcPicture=0x96437b8,
pDstPicture=0x963a588, maskFormat=0x953b448, xSrc=1, ySrc=0, ntrap=3,
traps=0x971121c) at cw_render.c:365
#5  0x08147cb3 in CompositeTrapezoids (op=12 '\f', pSrc=0x96437b8,
pDst=0x963a588, maskFormat=0x953b448, xSrc=1, ySrc=0, ntrap=3, traps=0x971121c)
at picture.c:1848
#6  0x0814e934 in ProcRenderTrapezoids (client=0x9643668) at render.c:820
#7  0x0814abb5 in ProcRenderDispatch (client=0x7) at render.c:2001
#8  0x0808815a in Dispatch () at dispatch.c:459
#9  0x0806fab5 in main (argc=10, argv=0xbfce6f44, envp=0xff53f0) at main.c:447
(gdb)

Comment 1 Richard Ryder 2008-05-27 18:16:36 UTC
Created attachment 306809 [details]
Xorg core generated by gdb gcore command

Comment 4 Matěj Cepl 2008-06-06 13:54:25 UTC
Created attachment 308531 [details]
dmesg from sosreport

Comment 5 Matěj Cepl 2008-06-06 13:54:34 UTC
Created attachment 308532 [details]
xorg.conf from sosreport

Comment 6 Matěj Cepl 2008-06-06 13:54:45 UTC
Created attachment 308533 [details]
Xorg.0.log from sosreport

Comment 7 Juliano F. Ravasi 2008-07-13 01:28:16 UTC
Please, could you confirm if this URL causes the same crash?
http://en.wikipedia.org/wiki/Special:Allmessages

I'm having similar crashes in Fedora 8 with the above URL, with intel
(open-source) and nvidia (closed-source) drivers, but with a different backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x0044817f in fbRasterizeEdges (buf=0x8695098, bpp=8, width=436, stride=109,
l=0xbffaff54,
    r=0xbffaff2c, t=-2147481464, b=28571510) at fbedge.c:171
171                     ap[lxi] = clip255 (ap[lxi] + N_X_FRAC(8) - lxs);
(gdb) backtrace
#0  0x0044817f in fbRasterizeEdges (buf=0x8695098, bpp=8, width=436, stride=109,
l=0xbffaff54,
    r=0xbffaff2c, t=-2147481464, b=28571510) at fbedge.c:171
#1  0x00440a0f in fbRasterizeTrapezoid (pPicture=0x85fe368, trap=0x864e978,
    x_off=<value optimized out>, y_off=0) at fbtrap.c:143
#2  0x0035534a in _nv000753X () from /usr/lib/xorg/modules/drivers//nvidia_drv.so

Also bug 453607 seems related.

Comment 8 Matěj Cepl 2008-07-15 16:47:05 UTC
This seems to be a duplicate of bug 455209 (which is in Fedora 8).

Comment 9 Matěj Cepl 2008-07-15 16:47:43 UTC
*** Bug 453607 has been marked as a duplicate of this bug. ***

Comment 10 John Perkins 2008-08-07 14:48:23 UTC
We are experiencing this particular error at our site as well.  The wikipedia link above causes the same crash for us.

backtrace informatino varies slightly based on whether teh Composite X extension is enabled.   With Composite enabled, the backgrace looks as such:

Program received signal SIGSEGV, Segmentation fault.
0x08147898 in PictureMatchFormat ()
(gdb) where
#0  0x08147898 in PictureMatchFormat ()
#1  0x081478e1 in ValidatePicture ()
#2  0x081479a1 in CompositePicture ()
#3  0x08146dc8 in miTrapezoids ()
#4  0x00a61837 in _nv002361X ()
   from /usr/lib/xorg/modules/drivers/nvidia_drv.so
#5  0x0000000c in ?? ()
#6  0x088b67a8 in ?? ()
#7  0x08904908 in ?? ()
#8  0x08824798 in ?? ()
#9  0x00000000 in ?? ()
(gdb)

Disabling Composite changes the backtrace slightly:

Program received signal SIGSEGV, Segmentation fault.
0x08147978 in CompositePicture ()
(gdb) where
#0  0x08147978 in CompositePicture ()
#1  0x08146dc8 in miTrapezoids ()
#2  0x00f82837 in _nv002361X ()
   from /usr/lib/xorg/modules/drivers/nvidia_drv.so
#3  0x0000000c in ?? ()
#4  0x09f2aca8 in ?? ()
#5  0x09efdbd0 in ?? ()
#6  0x09e8f2f0 in ?? ()
#7  0x00000000 in ?? ()
(gdb)

I tried using an X11R7.3 X-server, with Composite extensions enabled...this bug did not occur when running with the X11R7.3 X-server.

All our tests were done using the "nvidia" proprietary video device driver.

Comment 11 Pavel Kankovsky 2009-01-30 01:04:01 UTC
The problem lies in the negative (and very large) value of parameter "t" when fbRasterizeEdges() is called. The function draws outside the allocated buffer ("buf") and causes a lot of collateral damage.

The sources of negative values of "t" is RenderSampleCeilY() called from fbRasterizeTrapezoid(). Something overflows and yields a negative result when it is called with a large positive value of its first parameter "y" (afaik >= 2147481463) and this happens when the client asks the server to draw a strange trapezoid very close the edge of the coordinate space:

(gdb) bt 1
#0  fbRasterizeTrapezoid (pPicture=0x94d5dc8, trap=0x9509b1c, x_off=0, y_off=0)
    at fbtrap.c:137
(gdb) p *trap
$36 = {top = 2147483647, bottom = 2147483647, left = {p1 = {x = 0, y = 0},
      p2 = {x = 0, y = 2147483647}}, right = {p1 = {x = 65536, y = 2147483647},
      p2 = {x = 0, y = 2147483647}}}
(gdb) p t
$37 = -2147481464
(gdb) print RenderSampleCeilY(2147483647, 8)
$38 = -2147481464

(And no, this not a duplicate of bug 455209.)

Comment 12 Pavel Kankovsky 2009-02-02 18:46:10 UTC
Created attachment 330662 [details]
test case

$ gcc trapezoid_of_death.c -lX11 -lXext -lXrender
$ DISPLAY=[the display you want to kill] ./a.out

(It is necessary to send two trapezoids, one with saner top/bottom, to get past a check in miTrapezoids().)

Comment 13 RHEL Product and Program Management 2009-03-11 15:39:47 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 15 Adam Jackson 2009-04-22 18:15:02 UTC
Fixing component.

Comment 16 Adam Jackson 2009-04-22 18:44:45 UTC
Built as xorg-x11-server-1.1.1-48.53.el5

MODIFIED

Comment 18 Mark Gordon 2009-06-22 19:34:26 UTC
Fix verified using the 20090608.2 tree.

Comment 20 errata-xmlrpc 2009-09-02 11:42:21 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-1373.html


Note You need to log in before you can comment on or make changes to this bug.