[libav-bugs] [Bug 295] New: Swscale segmentation in libswscale crashes VLC

[bugzilla-daemon at aruru.libav.org]

https://bugzilla.libav.org/show_bug.cgi?id=295

           Summary: Swscale segmentation in libswscale crashes VLC
           Product: Libav
           Version: 0.8
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: Normal
         Component: libswscale
        AssignedTo: bugzilla at libav.org
        ReportedBy: edward.c.wang at compdigitec.com


Downstream bug: http://trac.videolan.org/vlc/ticket/6751

The essentials: When the file
https://www.cresis.ku.edu/sites/default/files/sea-level-rise/anim/east_asia.mov
is played in VLC media player, it causes a segmentation fault in the libswscale
library provided by the Libav project.

This has been reproduced on 32-bit Linux and 32-bit Windows. Probably also
reproduces on 64-bit as well.

Libav version: 0.8.2-6

Stacktrace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xad37bb70 (LWP 8590)]
0xb0e70204 in yuv2rgb32_X_MMX2 (c=c at entry=0x84b8c80, 
    lumFilter=lumFilter at entry=0x84a7de0, lumSrc=lumSrc at entry=0x84c0720, 
    lumFilterSize=lumFilterSize at entry=0, chrFilter=chrFilter at entry=0x84a7fa0, 
    chrUSrc=chrUSrc at entry=0x84c0760, chrVSrc=chrVSrc at entry=0x84c07c0, 
    chrFilterSize=chrFilterSize at entry=0, alpSrc=0x84c07e0, 
    dest=dest at entry=0x84c38c0 "", dstW=dstW at entry=60, dstY=dstY at entry=0)
    at
/build/buildd-libav_0.8.2-2-i386-Ix1Qdn/libav-0.8.2/libswscale/x86/swscale_template.c:299
299   
/build/buildd-libav_0.8.2-2-i386-Ix1Qdn/libav-0.8.2/libswscale/x86/swscale_template.c:
No such file or directory.
(gdb) bt
#0  0xb0e70204 in yuv2rgb32_X_MMX2 (c=c at entry=0x84b8c80, 
    lumFilter=lumFilter at entry=0x84a7de0, lumSrc=lumSrc at entry=0x84c0720, 
    lumFilterSize=lumFilterSize at entry=0, chrFilter=chrFilter at entry=0x84a7fa0, 
    chrUSrc=chrUSrc at entry=0x84c0760, chrVSrc=chrVSrc at entry=0x84c07c0, 
    chrFilterSize=chrFilterSize at entry=0, alpSrc=0x84c07e0, 
    dest=dest at entry=0x84c38c0 "", dstW=dstW at entry=60, dstY=dstY at entry=0)
    at
/build/buildd-libav_0.8.2-2-i386-Ix1Qdn/libav-0.8.2/libswscale/x86/swscale_template.c:299
#1  0xb0e51f77 in swScale (c=0x84b8c80, src=0xad37ab60, srcStride=0xad37ab80, 
    srcSliceY=0, srcSliceH=2, dst=0xad37ab70, dstStride=0xad37ab90)
    at
/build/buildd-libav_0.8.2-2-i386-Ix1Qdn/libav-0.8.2/libswscale/swscale.c:2665
#2  0xb0e58bae in sws_scale (c=c at entry=0x84b8c80, 
    srcSlice=srcSlice at entry=0xad37abf0, srcStride=srcStride at entry=0xad37ac00, 
    srcSliceY=srcSliceY at entry=0, srcSliceH=srcSliceH at entry=2, 
    dst=dst at entry=0xad37ac10, dstStride=dstStride at entry=0xad37ac20)
    at
/build/buildd-libav_0.8.2-2-i386-Ix1Qdn/libav-0.8.2/libswscale/swscale_unscaled.c:1014
#3  0xb31e8781 in Convert (p_filter=p_filter at entry=0x84aadc0, ctx=0x84b8c80, 
    p_dst=p_dst at entry=0x84a7e90, p_src=p_src at entry=0x84c0828, 
    i_height=i_height at entry=2, i_plane_count=i_plane_count at entry=3, 
    b_swap_uvi=false, b_swap_uvo=false, i_plane_start=0)
    at ../../../modules/video_filter/swscale.c:555
---Type <return> to continue, or q <return> to quit---
#4  0xb31e897e in Filter (p_filter=0x84aadc0, p_pic=0x84c0938)
    at ../../../modules/video_filter/swscale.c:606
#5  0xb7f72e15 in FilterChainVideoFilter (f=0x84aadc0, 
    p_pic=p_pic at entry=0x84c0938) at ../../src/misc/filter_chain.c:232
#6  0xb7f73998 in filter_chain_VideoFilter (p_chain=0x84a7a80, 
    p_pic=p_pic at entry=0x84c0938) at ../../src/misc/filter_chain.c:250
#7  0xb7f34bac in vout_FilterDisplay (vd=vd at entry=0x83c8e80, 
    picture=picture at entry=0x84c0938) at ../../src/video_output/display.c:1087
#8  0xb7f3a6fb in ThreadDisplayRenderPicture (is_forced=true, vout=0x83b0120)
    at ../../src/video_output/video_output.c:1020
#9  ThreadDisplayPicture (vout=vout at entry=0x83b0120, now=56, now at entry=false, 
    deadline=deadline at entry=0xad37b2a0)
    at ../../src/video_output/video_output.c:1126
#10 0xb7f3b144 in ThreadManage (postprocessing=0xad37b28c, 
    interlacing=0xad37b2ac, deadline=0xad37b290, vout=0x83b0120)
    at ../../src/video_output/video_output.c:1138
#11 Thread (object=0x83b0120) at ../../src/video_output/video_output.c:1581
#12 0xb7e2a954 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#13 0xb7da69fe in clone () from /lib/i386-linux-gnu/libc.so.6

-- 
Configure bugmail: https://bugzilla.libav.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.