| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
Variable frame rate (VFR) files have been difficult to work with.
This is because during sequential decoding, spacing between frames is
not always equal, but it was assumed to be equal. This can result in
movie getting out of sync with sound and difference between preview and
rendered image. A way to resolve these issues was to build and use
timecodes which is quite lengthy and resource intensive process. Such
issues are also difficult to communicate through UI because it is not
possible to predict if timecode usage would be needed.
With this patch, double buffer is used to keep previously decoded frame.
If current frame has PTS greater than what we are looking for, it is not
time to display it yet, and previous frame is displayed instead.
Each `AVFrame` has information about it's duration, so in theory double
buffering would not be needed, but in practice this information is
unreliable.
To ensure double buffer is always used, function
`ffmpeg_decode_video_frame_scan` is used for sequential decoding, even
if no scanning is expected.
This approach is similar to D6392, but this implementation does not
require seeking so it is much faster. Currently `AVFrame` is only
referenced, so no data is copied and therefore no overhead is added.
Note: There is one known issue where seeking fails even with double
buffering: Some files may seek too far in stream and miss requested
PTS. These require preseeking or greater negative subframe offset
Fixes: T86361, T72347
Reviewed By: zeddb, sergey
Differential Revision: https://developer.blender.org/D13583
|
|
D14918 from @linux_dr with some other changes included.
|
|
We had forgotten to unref packets after reading them.
This lead to a memory leak inside of ffmpeg.
|
|
In `ffmpeg_read_video_frame` fix assignment used as truth value.
In `ffmpeg_seek_recover_stream_position` loop while return value is
greater or equal to 0.
|
|
Improve readability and reduce indentation levels. No functional changes.
Reviewed By: zeddb
Differential Revision: https://developer.blender.org/D14075
|
|
|
|
In ffmpeg 5.0, several variables were made const to try to prevent bad API usage.
Removed some dead code that wasn't used anymore as well.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D14063
|
|
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
|
|
Caused by integer overflow in `steps_per_frame` calculation.
|
|
Using a negative linesize to flip an image vertically is supported in
ffmpeg but not for every function.
This method treats frames that need and those that do not need alignment
the same. An RGBA frame buffer with alignment that ffmpeg decides is
optimal for the CPU and build options is allocated by ffmpeg.
The `sws_scale` does the colorspace transformation into this RGBA frame
buffer without flipping. Now the image is upside down and aligned.
The combined unaligning and vertical flipping is then done by
`av_image_copy_to_buffer` which seems to handle negative linesize
correctly.
Reviewed By: ISS
Differential Revision: https://developer.blender.org/D13908
|
|
Includes unwanted changes
This reverts commit 46e049d0ce2bce2f53ddc41a0dbbea2969d00a5d.
|
|
This patch implements the vector types (i.e:`float2`) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the `blender::math` namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
####Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others
we currently don't have (uintX, intX). All these variations were
asking for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector
functions should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a
bit of a let down. Most clases are incomplete, out of sync with each
others with different codestyles, and some functions that should be
static are not (i.e: `float3::reflect()`).
####Upsides:
- Still support `.x, .y, .z, .w` for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types
and can be restricted to certain types. Also template specialization
let us define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance
is the same.
####Downsides:
- Might impact debugability. Though I would arge that the bugs are
rarelly caused by the vector class itself (since the operations are
quite trivial) but by the type conversions.
- Might impact compile time. I did not saw a significant impact since
the usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length.
For instance, one can't call `len_squared_v3v3` in
`math::length_squared()` and call it a day.
- Type cast does not work with the template version of the `math::`
vector functions. Meaning you need to manually cast `float *` and
`(float *)[3]` to `float3` for the function calls.
i.e: `math::distance_squared(float3(nearest.co), positions[i]);`
- Some parts might loose in readability:
`float3::dot(v1.normalized(), v2.normalized())`
becoming
`math::dot(math::normalize(v1), math::normalize(v2))`
But I propose, when appropriate, to use
`using namespace blender::math;` on function local or file scope to
increase readability.
`dot(normalize(v1), normalize(v2))`
####Consideration:
- Include back `.length()` method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement. It felt
like too much for what we need and would be difficult to extend / modify
to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches `delaunay_2d.cc` and the intersection code. I would like
to know @howardt opinion on the matter.
- The `noexcept` on the copy constructor of `mpq(2|3)` is being removed.
But according to @JacquesLucke it is not a real problem for now.
I would like to give a huge thanks to @JacquesLucke who helped during this
and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: https://developer.blender.org/D13791
|
|
Reverted because the commit removes a lot of commits.
This reverts commit a2c1c368af48644fa8995ecbe7138cc0d7900c30.
|
|
This patch implements the vector types (i.e:float2) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the blender::math namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others we
currently don't have (uintX, intX). All these variations were asking
for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector functions
should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the BLI_(float|double|mpq)(2|3|4).hh is a bit of a
let down. Most clases are incomplete, out of sync with each others with
different codestyles, and some functions that should be static are not
(i.e: float3::reflect()).
Upsides:
- Still support .x, .y, .z, .w for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types and
can be restricted to certain types. Also template specialization let us
define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance is
the same.
Downsides:
- Might impact debugability. Though I would arge that the bugs are rarelly
caused by the vector class itself (since the operations are quite trivial)
but by the type conversions.
- Might impact compile time. I did not saw a significant impact since the
usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length. For
instance, one can't call len_squared_v3v3 in math::length_squared() and
call it a day.
- Type cast does not work with the template version of the math:: vector
functions. Meaning you need to manually cast float * and (float *)[3] to
float3 for the function calls.
i.e: math::distance_squared(float3(nearest.co), positions[i]);
- Some parts might loose in readability:
float3::dot(v1.normalized(), v2.normalized())
becoming
math::dot(math::normalize(v1), math::normalize(v2))
But I propose, when appropriate, to use
using namespace blender::math; on function local or file scope to
increase readability. dot(normalize(v1), normalize(v2))
Consideration:
- Include back .length() method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement.
It felt like too much for what we need and would be difficult to
extend / modify to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches delaunay_2d.cc and the intersection code. I would like to
know @Howard Trickey (howardt) opinion on the matter.
- The noexcept on the copy constructor of mpq(2|3) is being removed.
But according to @Jacques Lucke (JacquesLucke) it is not a real problem
for now.
I would like to give a huge thanks to @Jacques Lucke (JacquesLucke) who
helped during this and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: http://developer.blender.org/D13791
|
|
Was not actually flipping in the need_aligned_ffmpeg_buffer case.
|
|
When scaling down image, users expect to see background, which doesn't
currently happen in VSE. This is because strips use cross blend mode by
default, because alpha over is much slower. Reason is, because any area
of image can be transparent, and therefore it can't have early out
implemented in a way that cross blend mode can.
Flag images rendered by codecs that don't support transparency as fully
opaque and implement a form of early out for alpha over blend mode.
When rendering image stack, 2-input effects are ignored on the
"way down". Alpha over needs rendered overlay image to decide whether it
will use only overlay or background too. Therefore overlay can be
rendered safely before it is used. Image flags can be checked and it can
be freed if needed. Freeing doesn't cause any performance degradation,
because image is always stored in cache.
This feature does not improve blend mode performance. In summary, it
only allowes for having alpha over blend mode on background images
without suffering from lower performance.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D12914
|
|
Issue was caused by incorrect FFmpeg asynchronous decoding API. In most
cases, decoder returns 1 frame each time it is fed by 1 packet. Here
decoder wanted to return more frames, but our code always expected only
one.
Before sending new packets to decoder, check if there are frames to
receive. If there are, process them, otherwise continue decoding as
usual.
Reviewed By: zeddb, sergey
Differential Revision: https://developer.blender.org/D13079
|
|
|
|
It was using dummy image buffers to indicate if an animation container
could be initialized or not.
Use booleans instead.
|
|
On the blender side this commit fixes importing video files with audio
and video streams that do not share the same start time and duration.
Differential Revision: https://developer.blender.org/D12353
|
|
Bug caused by integer overflow in ffmpeg_generic_seek_workaround().
Function max_ii() was used to limit int_64tvalue.
After fixing the issue there was another issue, where near-infinite loop
was caused by requested_pos being very large and stream being cut in a
way, that it was missing keyframe at beginning.
This was fixed by checking if we are reading beyond file content.
Reviewed By: zeddb
Differential Revision: https://developer.blender.org/D11888
|
|
The duration and start time for audio strips were not correctly read in
audaspace.
Some video files have a "lead in" section of audio that plays before the
video starts playing back. Before this patch, we would play this lead in
audio at the same time as the video started and thus the audio would not
be in sync anymore.
Now the lead in audio is cut off and the duration should be correctly
calculated with this in mind.
If the audio starts after the video, the audio strip is shifted to
account for this, but it will also lead to cut off audio which might not
be wanted. However we don't have a simple way to solve this at this
point.
Differential Revision: http://developer.blender.org/D11917
|
|
The seek pts was not correctly calculated.
In addition to that we were not seeking in the video pts time base.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11921
|
|
The video duration was not read correctly from the video file.
It would use the global duration of the file which does in some cases
not line up with the actual duration of the video stream.
Now we take the video stream duration and start time into account when
calculating the strip duration.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11920
|
|
|
|
If the last decoded frame had the same timestamp as the GOP current
packet, then we would skip over this frame when fast forwarding and we
would seek until the end of the file.
This would could only be triggered reliably in single threaded mode.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11601
|
|
ffmpeg_generic_seek_workaround did work properly and our start pts
calculation was wrong.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11562
|
|
Because of the added sanity checks in rB14508ef100c9 (D11492), seeking
in proxies would not work correctly any more. This is because it wasn't
working as intended before, but in most cases this wouldn't be
noticeable. However now when the sanity checks are tripped it is very
noticeable that something is wrong
The indexer tried to use dts values for time stamps when we used pts in
our decode functions to get the time positions. This would make it
start in the wrong GOP frames when searching. Now that we enforce no
crossing of GOP frames when decoding after seek, this would lead to
issues.
Now we correctly use pts (or dts if pts is not available) and thus we
don't have any seeking issues because of time stamp format missmatch.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11561
|
|
We should only check if the new pts value lies inside the duration of
the current frame.
|
|
Fixed the logic for seeking in ffmpeg video files.
The main fix is that we now apply a small offset in ffmpeg_get_seek_pos
to make sure we don't get the frame in front of the seek position when
seeking backward.
The rest of the changes is general cleanup and untangling code.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11492
|
|
Images with 4:2:2 and 4:4:4 chroma subsampling were blurred when
`SWS_FAST_BILINEAR` interpolation is set for `anim->img_convert_ctx`.
Use `SWS_BILINEAR` interpolation for all movies, as performance is
not impacted by this change.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D11457
|
|
Includes fixes to misspelled function names.
Ref D11280
|
|
There need to be more cleanup for ffmpeg 4.5 (ffmpeg master branch).
However this now compiles on ffmpeg 4.4 without and deprication
warnings.
Reviewed By: Sergey, Richard Antalik
Differential Revision: http://developer.blender.org/D10338
|
|
|
|
|
|
Ref T78710
|
|
This condition was in contradiction with comment for function
`ffmpeg_generic_seek_workaround()`.
I have noticed, that formats that seeked well used this workaround.
Problem was that I misunderstood code from `av_seek_frame()` - formats
with `read_seek()` function stil don't use generic seeking method.
This is defined in `seek_frame_internal()`
|
|
`ffmpeg_generic_seek_workaround()` applied negative offset for seqrched
packet timestamp, but proxies always start from 0 and timestamp can be
negative.
Limit timestamp value to 0, because `av_seek_frame()` doesn't accept
negative timestamps and returns with error. This prevents seeking from
working correctly.
|
|
|
|
`av_seek_frame()` failed to seek to nearest I-frame. This seems to be
a bug or not implemented feature in FFmpeg. Looks like same issue as
ticket https://trac.ffmpeg.org/ticket/1607 on ffmpeg tracker.
If seeking is done using format specific function (`read_seek2`)
field of `AVInputFormat` is set, `see av_seek_frame()`, use
`av_seek_frame()` function. Otherwise use wrapper that actively searches
for I-frame packet.
Searching is flexible and tries to do minimum amount of work. Currently
it is limited to equivalent of 25 frames, which may not be enough for
some files, but there may be files with no I-frames at all, so it is
best to keep this limit as low as possible. Previously this problem was
masked by preseek, which was hard-coded to 25 frames. This was removed
in rB88604b79b7d1.
If this approach would be unnecessary for some formats, in worst case
file would be seeked 2 times which is very fast, so there will be no
visible impact on performance.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10845
|
|
Generalize threading settings in proxy building and use them for encoding
and decoding in general. Check codec capabilities, prefer FF_THREAD_FRAME
threading over FF_THREAD_SLICE and automatic thread count over setting it
explicitly.
ffmpeg-codecs man page suggests that threads option is global and used by
codecs, that supports this option. Form some tests I have done, it seems that
`av_dict_set_int(&codec_opts, "threads", BLI_system_thread_count(), 0)`
has same effect as
```
pCodecCtx->thread_count = BLI_system_thread_count();
pCodecCtx->thread_type = FF_THREAD_FRAME;
```
Looking at `ff_frame_thread_encoder_init()` code, these cases are not
equivalent. It is probably safer to leave threading setup on libavcodec than
setting up each codec threading individually.
From what I have read all over the internet, frame multithreading should be
faster than slice multithreading. Slice multithreading is mainly used for low
latency streaming.
When running Blender with --debug-ffmpeg it complains about
`pCodecCtx->thread_count = BLI_system_thread_count()` that using thread count
above 16 is not recommended. Using too many threads can negatively affect image
quality, but I am not sure if this is the case for decoding as well - see
https://streaminglearningcenter.com/blogs/ffmpeg-command-threads-how-it-affects-quality-and-performance.html
This is fine for proxies but may be undesirable for final renders.
Number of threads is limited by image size, because of size of motion vectors,
so if it is possible let libavcodec determine optimal thread count.
Performance difference:
Proxy building: None
Playback speed: 2x better on 1920x1080 sample h264 file
Scrubbing: Hard to quantify, but it's much more responsive
Rendering speed: None on 1920x1080 sample h264 file, there is improvement with codecs that do support FF_THREAD_FRAME for encoding like MPNG
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10791
|
|
Split seeking section of `ffmpeg_fetchibuf()` function into multiple
smaller functions.
Conditional statements are moved to own funtions with human readable
names, so code flow is more clear.
To remove one branch of seeking, first frame is now decoded by
scanning, which will do only one iteration. So nothing has technically
changed.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10638
|
|
Applying negative offset to seek position before scanning doesnn't have
any effect. This change results in 1.5x faster seeking (random frame,
average value) in sample file with 30 frame GOP length.
If I am not mistaken, B frames can have pts that can be less than
pts of I frame that must be decoded. Even in this case though, B frame
packet will be stored after that I frame.
In addition, preseek value is de facto hardcoded so seeking would fail
if it could. This can be hard to spot though.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10529
|
|
Allow use all system threads for frame encoding/decoding. This is very
straightforward: the value of zero basically disables threading.
Change threading policy to slice when decoding frames. The reason for
this is because decoding happens frame-by-frame, so inter-frame threading
policy will not bring any speedup.
The change for threading policy to slice is less obvious and is based on
benchmark of the demo files from T78986. This gives best performance so
far.
Rendering the following file went down from 190sec down to 160sec.
https://storage.googleapis.com/institute-storage/vse_simplified_example.zip
This change makes both reading and writing faster. The animation render
is just easiest to get actual time metrics.
Differential Revision: https://developer.blender.org/D8627
|
|
Replace `long long` with an explicit `int64_t`. This is also what is
used in the FFmpeg headers.
Fixes clang diagnostics warning about wrong format used in the log.
Should be no functional changes.
|
|
|
|
This commit resolves problem introduced in e1665c3d3190 - it was
difficult to import media at their original resolution.
This is done by using original resolution as reference for scale.
All crop and strip transform values and their animation is converted
form old files.
To make both workflows easy to use, sequencer tool settings have been
created with preset for preffered scaling method. This setting is in
sequencer timeline header and add image or movie strip operator
properties.
Two new operators have been added:
`sequencer.strip_transform_fit` operator with 3 options: Scale To Fit,
Scale to Fill and Stretch To Fill.
Operator can fail if strip image or video is not loaded currently, this
case should be either sanitized or data loaded on demand.
`sequencer.strip_transform_clear` operator with 4 options:
Clear position, scale, rotation and all (previous 3 options combined).
Reviewed By: sergey, fsiddi
Differential Revision: https://developer.blender.org/D9582
|
|
Follow our code style guide by using C-comments for text descriptions.
|
