[libav-commits] flacenc: ensure the order is within the min/max range in LPC order search

Justin Ruggles git at libav.org
Wed Jan 2 21:44:02 CET 2013


Module: libav
Branch: release/0.8
Commit: 0ba0e3195517e423c35b232384e380b975b88b4e

Author:    Justin Ruggles <justin.ruggles at gmail.com>
Committer: Reinhard Tartler <siretart at tauware.de>
Date:      Wed Nov  7 14:48:28 2012 -0500

flacenc: ensure the order is within the min/max range in LPC order search

This fixes use of uninitialized values when the FLAC encoder uses the
2-level, 4-level, and 8-level search methods. Fixes failure of the
fate-flac-24-comp-8 test when run using valgrind.
(cherry picked from commit 3a2731cbd31d0c5681ddbc7c78edd5c53c4d0032)

Conflicts:

	libavcodec/flacenc.c

Signed-off-by: Reinhard Tartler <siretart at tauware.de>

---

 libavcodec/flacenc.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/libavcodec/flacenc.c b/libavcodec/flacenc.c
index 94e381d..2bd4788 100644
--- a/libavcodec/flacenc.c
+++ b/libavcodec/flacenc.c
@@ -915,14 +915,16 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch)
         omethod == ORDER_METHOD_8LEVEL) {
         int levels = 1 << omethod;
         uint32_t bits[1 << ORDER_METHOD_8LEVEL];
-        int order;
+        int order       = -1;
         int opt_index   = levels-1;
         opt_order       = max_order-1;
         bits[opt_index] = UINT32_MAX;
         for (i = levels-1; i >= 0; i--) {
+            int last_order = order;
             order = min_order + (((max_order-min_order+1) * (i+1)) / levels)-1;
-            if (order < 0)
-                order = 0;
+            order = av_clip(order, min_order - 1, max_order - 1);
+            if (order == last_order)
+                continue;
             encode_residual_lpc(res, smp, n, order+1, coefs[order], shift[order]);
             bits[i] = find_subframe_rice_params(s, sub, order+1);
             if (bits[i] < bits[opt_index]) {



More information about the libav-commits mailing list