diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2020-02-16 22:11:52 +0300 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2020-07-02 20:55:09 +0300 |
| commit | 64bee9c2f8479f50fcf61b71b1f0970014621228 (patch) | |
| tree | 72ff1015ce3ee532cba6c6df5d5345ef091bdcd4 /libswscale | |
| parent | d33f8f2871d6d3ecdc589cf72a75e04715cc5e33 (diff) | |
swscale/output: Fix integer overflow in yuv2rgb_write_full() with out of range input
Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int'
Fixes: ticket8293
Found-by: Suhwan
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e057e83a4ff4c0eeeb78dffe58e21af951c056b6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libswscale')
| -rw-r--r-- | libswscale/output.c | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/libswscale/output.c b/libswscale/output.c index 6660b14994..37cd124e21 100644 --- a/libswscale/output.c +++ b/libswscale/output.c @@ -1634,9 +1634,9 @@ static av_always_inline void yuv2rgb_write_full(SwsContext *c, Y -= c->yuv2rgb_y_offset; Y *= c->yuv2rgb_y_coeff; Y += 1 << 21; - R = Y + V*c->yuv2rgb_v2r_coeff; - G = Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff; - B = Y + U*c->yuv2rgb_u2b_coeff; + R = (unsigned)Y + V*c->yuv2rgb_v2r_coeff; + G = (unsigned)Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff; + B = (unsigned)Y + U*c->yuv2rgb_u2b_coeff; if ((R | G | B) & 0xC0000000) { R = av_clip_uintp2(R, 30); G = av_clip_uintp2(G, 30); |