[libav-bugs] [Bug 746] New: avconv crash when using transpose video filter on glibc double link memory access

bugzilla at libav.org bugzilla at libav.org
Wed Sep 24 19:29:04 CEST 2014


https://bugzilla.libav.org/show_bug.cgi?id=746

           Summary: avconv crash when using transpose video filter on
                    glibc double link memory access
           Product: Libav
           Version: 11
          Platform: X86
        OS/Version: Linux
            Status: NEW
          Severity: critical
          Priority: Normal
         Component: general
        AssignedTo: bugzilla at libav.org
        ReportedBy: bsd_2000 at yahoo.fr


I run the avconv binary using:
/opt/build/libav-11/avconv -i /path/to/video/videos/1201 -b:v 900k -strict
experimental -vf transpose=1 /opt/test-avconv.mp4

*** glibc detected *** /opt/build/libav-11/avconv: realloc(): invalid next
size: 0x000000000273b4f0 ***

The program crashed.

I fixed some uninitialized values to get it running into valgrind, and I get:
==9822== Invalid write of size 4    1319kB time=2.15 bitrate=5013.0kbits/s    
ts/s
==9822==    at 0xAFAF0B: quantize_and_encode_band_cost_ESC (put_bits.h:161)
==9822==    by 0xAF809A: quantize_and_encode_band (aaccoder.c:270)
==9822==    by 0x9F8E64: aac_encode_frame (aacenc.c:430)
==9822==    by 0x8C8A29: avcodec_encode_audio2 (utils.c:1371)
==9822==    by 0x455493: poll_filter (avconv.c:425)
==9822==    by 0x45A696: main (avconv.c:747)
==9822==  Address 0x5c00018 is 0 bytes after a block of size 776 alloc'd
==9822==    at 0x4C279EE: malloc (vg_replace_malloc.c:270)
==9822==    by 0x4C27B62: realloc (vg_replace_malloc.c:662)
==9822==    by 0xB5C731: av_buffer_realloc (buffer.c:154)
==9822==    by 0x56E242: av_new_packet (avpacket.c:72)
==9822==    by 0x9F837B: aac_encode_frame (aacenc.c:574)
==9822==    by 0x8C8A29: avcodec_encode_audio2 (utils.c:1371)
==9822==    by 0x455493: poll_filter (avconv.c:425)
==9822==    by 0x45A696: main (avconv.c:747)
==9822==
frame=  113 fps=  0 q=31.0 Lsize=    1918kB time=3.69 bitrate=4255.6kbits/s
video:1813kB audio:102kB other streams:0kB global headers:0kB muxing overhead:
0.196199%
==9822==
==9822== HEAP SUMMARY:
==9822==     in use at exit: 0 bytes in 0 blocks
==9822==   total heap usage: 22,113 allocs, 22,113 frees, 3,228,007,388 bytes
allocated
==9822==
==9822== All heap blocks were freed -- no leaks are possible
==9822==
==9822== For counts of detected and suppressed errors, rerun with: -v
==9822== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 6 from 6)



The same options inside GDB:

GNU gdb (GDB) Red Hat Enterprise Linux (7.2-64.el6_5.2)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /opt/build/libav-11/avconv...done.
[Thread debugging using libthread_db enabled]
avconv version 11, Copyright (c) 2000-2014 the Libav developers
  built on Sep 24 2014 10:08:52 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/path/to/video/videos/1201':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: isom3gp4
    creation_time   : 2014-05-07 17:19:56
    location        : +46.3921+005.9593/
    location-eng    : +46.3921+005.9593/
  Duration: 00:00:03.90, start: 0.000000, bitrate: 12515 kb/s
    Stream #0.0(eng): Video: h264 (Constrained Baseline), yuv420p, 1920x1080,
12192 kb/s, PAR 65536:65536 DAR 16:9, 30.33 fps, 90k tbn (default)
    Metadata:
      creation_time   : 2014-05-07 17:19:56
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0.1(eng): Audio: aac, 48000 Hz, mono, fltp, 96 kb/s (default)
    Metadata:
      creation_time   : 2014-05-07 17:19:56
File '/opt/test-avconv.mp4' already exists. Overwrite ? [y/N] y
[New Thread 0x7ffff7180700 (LWP 10461)]
[New Thread 0x7ffff677f700 (LWP 10462)]
[New Thread 0x7ffff5d7e700 (LWP 10463)]
[New Thread 0x7ffff537d700 (LWP 10464)]
[New Thread 0x7ffff497c700 (LWP 10465)]
[New Thread 0x7ffff3f7b700 (LWP 10466)]
[New Thread 0x7ffff357a700 (LWP 10467)]
[New Thread 0x7ffff2b79700 (LWP 10468)]
[New Thread 0x7ffff2178700 (LWP 10469)]
[New Thread 0x7ffff1777700 (LWP 10470)]
[New Thread 0x7ffff0d76700 (LWP 10471)]
[New Thread 0x7ffff0375700 (LWP 10472)]
Output #0, mp4, to '/opt/test-avconv.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: isom3gp4
    creation_time   : 2014-05-07 17:19:56
    location        : +46.3921+005.9593/
    location-eng    : +46.3921+005.9593/
    encoder         : Lavf56.1.0
    Stream #0.0(eng): Video: mpeg4, yuv420p, 1080x1920 [PAR 1:1 DAR 9:16],
q=2-31, 900 kb/s, 30.33 fps, 91 tbn, 30.33 tbc (default)
    Metadata:
      creation_time   : 2014-05-07 17:19:56
      encoder         : Lavc56.1.0 mpeg4
    Stream #0.1(eng): Audio: aac, 48000 Hz, mono, fltp, 200 kb/s (default)
    Metadata:
      creation_time   : 2014-05-07 17:19:56
      encoder         : Lavc56.1.0 aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press ctrl-c to stop encoding
*** glibc detected *** /opt/build/libav-11/avconv: realloc(): invalid next
size: 0x000000000273b4f0 ***

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff71faa43 in _int_malloc () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
bzip2-libs-1.0.5-7.el6_0.x86_64 glibc-2.12-1.132.el6_5.2.x86_64
zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x00007ffff71faa43 in _int_malloc () from /lib64/libc.so.6
#1  0x00007ffff71fb636 in calloc () from /lib64/libc.so.6
#2  0x00007ffff7de7d2f in _dl_new_object () from /lib64/ld-linux-x86-64.so.2
#3  0x00007ffff7de41be in _dl_map_object_from_fd () from
/lib64/ld-linux-x86-64.so.2
#4  0x00007ffff7de537a in _dl_map_object () from /lib64/ld-linux-x86-64.so.2
#5  0x00007ffff7defa44 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#6  0x00007ffff7deb1b6 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#7  0x00007ffff7def4fa in _dl_open () from /lib64/ld-linux-x86-64.so.2
#8  0x00007ffff72a7e00 in do_dlopen () from /lib64/libc.so.6
#9  0x00007ffff7deb1b6 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#10 0x00007ffff72a7f57 in __libc_dlopen_mode () from /lib64/libc.so.6
#11 0x00007ffff727fb15 in init () from /lib64/libc.so.6
#12 0x00007ffff7521d33 in pthread_once () from /lib64/libpthread.so.0
#13 0x00007ffff727fc44 in backtrace () from /lib64/libc.so.6
#14 0x00007ffff71f184b in __libc_message () from /lib64/libc.so.6
#15 0x00007ffff71f7166 in malloc_printerr () from /lib64/libc.so.6
#16 0x00007ffff71fcc27 in _int_realloc () from /lib64/libc.so.6
#17 0x00007ffff71fcde5 in realloc () from /lib64/libc.so.6
#18 0x0000000000b5c6d0 in av_buffer_realloc (pbuf=0x7fffffffe360, size=731) at
libavutil/buffer.c:187
#19 0x00000000008c8bd3 in avcodec_encode_audio2 (avctx=0x16189e0,
avpkt=0x7fffffffe360, frame=0x2140d80, got_packet_ptr=0x7fffffffe3cc) at
libavcodec/utils.c:1387
#20 0x0000000000455494 in do_audio_out (ost=0x16188c0) at avconv.c:425
#21 poll_filter (ost=0x16188c0) at avconv.c:688
#22 0x000000000045a697 in poll_filters (argc=<value optimized out>, argv=<value
optimized out>) at avconv.c:747
#23 transcode (argc=<value optimized out>, argv=<value optimized out>) at
avconv.c:2495
#24 main (argc=<value optimized out>, argv=<value optimized out>) at
avconv.c:2649
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x7ffff71faa23 to 0x7ffff71faa63:
   0x00007ffff71faa23 <_int_malloc+1971>:    add    %cl,(%rax,%rax,1)
   0x00007ffff71faa26 <_int_malloc+1974>:    add    %al,(%rax)
   0x00007ffff71faa28 <_int_malloc+1976>:    jmpq   0x7ffff71fa349
<_int_malloc+217>
   0x00007ffff71faa2d <_int_malloc+1981>:    mov    0x8(%rbp),%r13
   0x00007ffff71faa31 <_int_malloc+1985>:    mov    0x10(%rbp),%rax
   0x00007ffff71faa35 <_int_malloc+1989>:    mov    0x18(%rbp),%rdx
   0x00007ffff71faa39 <_int_malloc+1993>:    and    $0xfffffffffffffff8,%r13
   0x00007ffff71faa3d <_int_malloc+1997>:    mov    %r13,%r14
   0x00007ffff71faa40 <_int_malloc+2000>:    sub    %r15,%r14
=> 0x00007ffff71faa43 <_int_malloc+2003>:    cmp    0x18(%rax),%rbp
   0x00007ffff71faa47 <_int_malloc+2007>:    jne    0x7ffff71faf64
<_int_malloc+3316>
   0x00007ffff71faa4d <_int_malloc+2013>:    cmp    %rbp,0x10(%rdx)
   0x00007ffff71faa51 <_int_malloc+2017>:    jne    0x7ffff71faf64
<_int_malloc+3316>
   0x00007ffff71faa57 <_int_malloc+2023>:    cmpq   $0x3ff,0x8(%rbp)
   0x00007ffff71faa5f <_int_malloc+2031>:    mov    %rdx,0x18(%rax)
End of assembler dump.
(gdb) info all-registers
rax            0x898989898989898a    -8536140394893047414
rbx            0x7ffff7510e80    140737342672512
rcx            0x80    128
rdx            0x8989898989898989    -8536140394893047415
rsi            0x3    3
rdi            0x3    3
rbp            0x273b7f0    0x273b7f0
rsp            0x7fffffffcb10    0x7fffffffcb10
r8             0x7ffff7510ee8    140737342672616
r9             0x1    1
r10            0x496    1174
r11            0x246    582
r12            0x7ffff7510ed8    140737342672600
r13            0x8a8a8a8a8a8a8a88    -8463800222054970744
r14            0x8a8a8a8a8a8a85e8    -8463800222054971928
r15            0x4a0    1184
rip            0x7ffff71faa43    0x7ffff71faa43 <_int_malloc+2003>
eflags         0x10286    [ PF SF IF RF ]
cs             0x33    51
ss             0x2b    43
ds             0x0    0
es             0x0    0
fs             0x0    0
gs             0x0    0
st0            -nan(0x8686868686868686)    (raw 0xffff8686868686868686)
st1            -nan(0x8686868686868686)    (raw 0xffff8686868686868686)
st2            -nan(0x000000004)    (raw 0xffff0000000000000004)
st3            -nan(0xfb665e0a72f08000)    (raw 0xfffffb665e0a72f08000)
st4            -nan(0x8080808080808080)    (raw 0xffff8080808080808080)
st5            -nan(0x101010100000000)    (raw 0xffff0101010100000000)
st6            -nan(0xfb40000000000000)    (raw 0xfffffb40000000000000)
st7            -inf    (raw 0xffff0000000000000000)
fctrl          0x37f    895
fstat          0x20    32
ftag           0xffff    65535
fiseg          0x7fff    32767
fioff          0xf7b7f487    -138939257
foseg          0x7fff    32767
fooff          0xffff9d68    -25240
fop            0x0    0
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 =
0x00000000000000000000000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x2f <repeats 16 times>}, v8_int16 = {0x2f2f, 0x2f2f, 0x2f2f,
0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f}, v4_int32 = {0x2f2f2f2f, 0x2f2f2f2f,
0x2f2f2f2f, 0x2f2f2f2f}, v2_int64 = {0x2f2f2f2f2f2f2f2f, 0x2f2f2f2f2f2f2f2f},
uint128 = 0x2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 =
0x00000000000000000000000000000000}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 11 times>, 0xff, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0,
0x0, 0x0, 0x0, 0x0, 0xff00, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0xff000000, 0x0},
v2_int64 = {0x0, 0xff000000}, uint128 = 0x00000000ff0000000000000000000000}
xmm4           {v4_float = {0x197eb2, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x97, 0xf5, 0xcb, 0x49, 0x0 <repeats 12 times>}, v8_int16 =
{0xf597, 0x49cb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x49cbf597, 0x0,
0x0, 0x0}, v2_int64 = {0x49cbf597, 0x0}, uint128 =
0x00000000000000000000000049cbf597}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0xff, 0xff, 0xff, 0x7f, 0x0 <repeats 12 times>}, v8_int16 =
{0xffff, 0x7fff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x7fffffff, 0x0,
0x0, 0x0}, v2_int64 = {0x7fffffff, 0x0}, uint128 =
0x0000000000000000000000007fffffff}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 =
0x00000000000000000000000000000000}
xmm7           {v4_float = {0x4000000, 0x1, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0xfc, 0x18, 0x73, 0xd7, 0x12, 0xf2, 0xd9, 0x3f, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x18fc, 0xd773, 0xf212, 0x3fd9, 0x0, 0x0, 0x0,
0x0}, v4_int32 = {0xd77318fc, 0x3fd9f212, 0x0, 0x0}, v2_int64 =
{0x3fd9f212d77318fc, 0x0},
  uint128 = 0x00000000000000003fd9f212d77318fc}
xmm8           {v4_float = {0x13, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0xf0, 0x37, 0x98, 0x41, 0x0 <repeats 12 times>}, v8_int16 =
{0x37f0, 0x4198, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x419837f0, 0x0,
0x0, 0x0}, v2_int64 = {0x419837f0, 0x0}, uint128 =
0x000000000000000000000000419837f0}
xmm9           {v4_float = {0x2ff21a, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x6a, 0xc8, 0x3f, 0x4a, 0x0 <repeats 12 times>}, v8_int16 =
{0xc86a, 0x4a3f, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x4a3fc86a, 0x0,
0x0, 0x0}, v2_int64 = {0x4a3fc86a, 0x0}, uint128 =
0x0000000000000000000000004a3fc86a}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0xfd, 0x44, 0x57, 0x3d, 0x0 <repeats 12 times>}, v8_int16 =
{0x44fd, 0x3d57, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x3d5744fd, 0x0,
0x0, 0x0}, v2_int64 = {0x3d5744fd, 0x0}, uint128 =
0x0000000000000000000000003d5744fd}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 =
0x00000000000000000000000000000000}
xmm12          {v4_float = {0x2000, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0, 0x0, 0x0, 0x46, 0x0 <repeats 12 times>}, v8_int16 = {0x0,
0x4600, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x46000000, 0x0, 0x0, 0x0},
v2_int64 = {0x46000000, 0x0}, uint128 = 0x00000000000000000000000046000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 =
0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 =
0x00000000000000000000000000000000}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
v16_int8 = {0xa0, 0x83, 0x47, 0x3, 0x1d, 0x3c, 0x8a, 0xb5, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x83a0, 0x347, 0x3c1d, 0xb58a, 0x0, 0x0, 0x0,
0x0}, v4_int32 = {0x34783a0, 0xb58a3c1d, 0x0, 0x0}, v2_int64 =
{0xb58a3c1d034783a0, 0x0},
  uint128 = 0x0000000000000000b58a3c1d034783a0}
mxcsr          0x1fbf    [ IE DE ZE OE UE PE IM DM ZM OM UM PM ]
(gdb)


I tried to do the same using -acodec copy, and it is running as expected (well
... no crash)

Hope there is enough information :)

-- 
Configure bugmail: https://bugzilla.libav.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.


More information about the libav-bugs mailing list