diff options
| author | Michael Niedermayer <michael@niedermayer.cc> | 2020-02-16 22:11:52 +0300 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2020-07-01 14:33:44 +0300 |
| commit | 531b3300a7c1d0639dbcacbd55552ff4d56f4f6e (patch) | |
| tree | 6b5bfb6b707a7194745d33c9dabb98f3355ec7fa /libswscale | |
| parent | 81f9e29bc9befca2c1bd86787a6a0e50c1ba17c6 (diff) | |
swscale/output: Fix integer overflow in yuv2rgb_write_full() with out of range input
Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int'
Fixes: ticket8293
Found-by: Suhwan
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e057e83a4ff4c0eeeb78dffe58e21af951c056b6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libswscale')
| -rw-r--r-- | libswscale/output.c | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/libswscale/output.c b/libswscale/output.c index 70fb25c5e6..901c7bd066 100644 --- a/libswscale/output.c +++ b/libswscale/output.c @@ -1662,9 +1662,9 @@ static av_always_inline void yuv2rgb_write_full(SwsContext *c, Y -= c->yuv2rgb_y_offset; Y *= c->yuv2rgb_y_coeff; Y += 1 << 21; - R = Y + V*c->yuv2rgb_v2r_coeff; - G = Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff; - B = Y + U*c->yuv2rgb_u2b_coeff; + R = (unsigned)Y + V*c->yuv2rgb_v2r_coeff; + G = (unsigned)Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff; + B = (unsigned)Y + U*c->yuv2rgb_u2b_coeff; if ((R | G | B) & 0xC0000000) { R = av_clip_uintp2(R, 30); G = av_clip_uintp2(G, 30); |