| Age | Commit message (Collapse) | Author |
|---|
|
This is a bug in the multithreaded task manager in negative value range.
The problem here is that if previter is unsigned, the comparison in the
return statement is unsigned, and works incorrectly if stop < 0 &&
iter >= 0. This in turn can happen if stop is close to 0, because this
code is designed to overrun the stop by chunk_size*num_threads as
the threads terminate.
This probably should go into 2.78 as it prevents a crash.
|
|
|
|
|
|
non-parallelized case.
|
|
Together with the extended loop callback and userdata_chunk, this allows to perform
cumulative tasks (like aggregation) in a lockfree way using local userdata_chunk to store temp data,
and once all workers have finished, to merge those userdata_chunks in the finalize callback
(from calling thread, so no need to lock here either).
Note that this changes how userdata_chunk is handled (now fully from 'main' thread,
which means a given worker thread will always get the same userdata_chunk, without
being re-initialized anymore to init value at start of each iter chunk).
|
|
BLI_task_parallel_range() & co.
|
|
New code is actually much, much better than first version, using 'fetch_and_add' atomic op
here allows us to get rid of the loop etc.
The broken CAS issue remains on windows, to be investigated...
|
|
feature."
There are some serious issues under windows, causing deadlocks somehow (not reproducible under linux so far).
Until further investigation over why this happens, better to revert to previous
spin-locked behavior.
This reverts commits a83bc4f59707ab and 98123ae9168.
|
|
Reading the shared state->iter value after storing it in the 'reference' var could in theory
lead to a race condition setting state->iter value above state->stop, which would be 'deadly'.
This **may** be the cause of T48422, though I was not able to reproduce that issue so far.
|
|
Code by @sergey, with small edits and doc by @mont29.
|
|
This commit makes use of new taskpool feature (instead of allocating own tasks),
and removes the spinlock used to generate chunks (using atomic ops instead).
In best cases (dynamic scheduled loop with light processing func callback), we
get a few percents of speedup, in most cases there is no sensible enhancement.
|
|
Appears mutex was guarateeing number of tasks is not modified at moments
when it's not expected. Removing those mutexes resulted in some hard-to-catch
locks where worker thread were waiting for work by all the tasks were already
done.
This reverts commit a1d8fe052ccd8945f14be5a50bd5af581b89c643.
|
|
It seems using atomic operations here we can avoid having mute without
breaking anything.
Thanks Bastien for double-checking the changes!
|
|
Brain melt here, intention was to reduce number of tasks in case we have not much chunks of data to loop over,
not to increase it!
Note that this only affected dynamic scheduling.
|
|
This commit implements new function BLI_task_pool_push_from_thread()
who's main goal is to have less parasitic load on the CPU bu avoiding
memory allocations as much as possible, making taks pushing cheaper.
This function expects thread ID, which must be 0 for the thread from
which pool is created from (and from which wait_work() is called) and
for other threads it mush be the ID which was sent to the thread working
function.
This reduces allocations quite a bit in the new dependency graph,
hopefully gaining some visible speedup on a fewzillion core machines
(on my own machine can only see benefit in profiler, which shows
significant reduce of time wasted in the memory allocation).
|
|
Majority of the fields are being overwritten anyway, so calloc it
kinda waste of CPU ticks.
|
|
|
|
|
|
Based on usages so far:
- Split callback worker func in two, 'basic' and 'extended' versions. The former goes back
to the simplest verion, while the later keeps the 'userdata_chunk', and gets the thread_id too.
- Add use_threading to simple BLI_task_parallel_range(), turns out we need this pretty much systematically,
and allows to get rid of most usages of BLI_task_parallel_range_ex().
- Now BLI_task_parallel_range() expects 'basic' version of callback, while BLI_task_parallel_range_ex()
expectes 'extended' version of the callback.
All in all, this should make common usage of BLI_task_parallel_range simpler (less verbose), and add
access to advanced callback to thread id, which is mandatory in some (future) cases.
|
|
This can happen quite often in forloops, and would be annoying to have to check for this
in caller code! So now, just return without doing anything in this case.
|
|
use threading or not, instead of threshold.
From recent experience, turns out we often do want to use something else than basic
range of parallelized forloop as control parameter over threads usage, so now BLI func
only takes a boolean, and caller defines best check for its own case.
|
|
iterations that workers.
When called with very small range, `BLI_task_parallel_range_ex()` would generate a zero `chunk_size`,
leading to some infinite looping in `parallel_range_func` due to `parallel_range_next_iter_get` returning
true without actually increasing the counter!
So now, we ensure `chunk_size` and `num_tasks` are always at least 1 (and avoid generating too much tasks too).
|
|
|
|
This mimics OpenMP's 'firstprivate' feature. It is sometimes handy to have some persistent local data during a whole chunk.
Reviewers: sergey
Reviewed By: sergey
Subscribers: campbellbarton
Differential Revision: https://developer.blender.org/D1635
|
|
|
|
Suggested by Sergey, thanks!
|
|
|
|
With current code, in single-threaded context, a pool of task may never be executed
until one calls BLI_task_pool_work_and_wait() on it, this is not acceptable for
asynchronous tasks where you never want to actually lock the main thread.
This commits adds an extra thread in single-threaded case, and a new 'type' of pool,
such that one can create real background pools of tasks. See code for details.
Review: D1565
|
|
Useful in case one needs more complex handling of tasks data than a mere MEM_freeN().
|
|
In previous code, worker would exit in case it gets awoken from a condition_wait() and
task queue is empty. However, there may be spurious wake up (either due to pthread itself,
or to some race condition between workers) that would lead to wrongly exiting a worker before
we actually exit the whole scheduler. See code for more details.
|
|
creation fails.
Trying to MEM_free a single item of a whole MEM_calloc'ated array, tsst...
Luckily looks like POSIX thread creation does not fail often! :P
|
|
The used heuristic of checking the value prior to lock is not totally safe
because assignment is not atomic and check might not give proper result.
|
|
Allocate statistics array dynamically, so increasing max number of threads does
not increase sloppyness of the memory usage.
For the further cleanups: we can try alloca-ing this array, but it's also not
really safe because we can have quite huge number of threads in the future.
Plus statistics will allocate memory for each individual entry, so using alloca
is not going to give anything beneficial here.
|
|
This ensures proper values of currently running tasks in the pool
(previously difference between mutex locks when acquiring new job
and releasing it might in theory give wrong values).
|
|
This way we can have scheduler capable of scheduling tasks on all the CPUs
but in the same time we can limit tasks like baking (in the future) to use
no more than given number of threads.
|
|
It now supports different scheduling schemas: dynamic and static.
Static one is the default and it splits work into equal number of
range iterations.
Dynamic one allocates chunks of 32 iterations which then being
dynamically send to a thread which is currently idling.
This gives slightly better performance. Still some tricks are
possible to have. For example we can use some smarter static scheduling
when one thread might steal tasks from another threads when it runs
out of work to be done.
Also removed unneeded spin lock in the mesh deform evaluation,
on the first glance it seemed to be a reduction involved here but
int fact threads are just adding value to the original vertex
coordinates. No write access to the same element of vertexCos
happens from separate threads.
|
|
|
|
This commit switches meshdeform modifier to use threads to evaluate
the vertices positions using the central task scheduler.
SO now we've got an utility function to help splitting the for loop
into tasks using BLI_task module which is pretty straightforward to
use: it gets range (which is an integer lower and higher bounds) and
the function and userdata to be invoked for each of the iterations.
The only weak point for now is the passing the data to the callback,
this isn't so trivial to improve in pure C.
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D838
|
|
|
|
|
|
|
|
Replaces ThreadedWorker and is gonna to be used
for threaded object update in the future and
some more upcoming changes.
But in general, it's to be used for any task
based subsystem in Blender.
Originally written by Brecht, with some fixes
and tweaks by self.
|
