[libav-devel] [PATCH 4/5] svq1enc: fix signed multiplication overflow

Kostya Shishkov kostya.shishkov at gmail.com
Sat Nov 26 19:15:26 CET 2011


On Sat, Nov 26, 2011 at 04:52:10PM +0000, Mans Rullgard wrote:
> This multiplication can overflow the signed range but not the
> unsigned.  After right-shifting it will thus fit in the signed
> range again.
> 
> Signed-off-by: Mans Rullgard <mans at mansr.com>
> ---
>  libavcodec/svq1enc.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/libavcodec/svq1enc.c b/libavcodec/svq1enc.c
> index 79aaa6c..01d8b31 100644
> --- a/libavcodec/svq1enc.c
> +++ b/libavcodec/svq1enc.c
> @@ -160,7 +160,7 @@ static int encode_block(SVQ1Context *s, uint8_t *src, uint8_t *ref, uint8_t *dec
>      }
>  
>      best_count=0;
> -    best_score -= ((block_sum[0]*block_sum[0])>>(level+3));
> +    best_score -= (int)(((unsigned)block_sum[0]*block_sum[0])>>(level+3));
>      best_mean= (block_sum[0] + (size>>1)) >> (level+3);
>  
>      if(level<4){
> -- 

should be OK


More information about the libav-devel mailing list