| Age | Commit message (Collapse) | Author |
|---|
|
When allocating new `CustomData` layers, often we do redundant
initialization of arrays. For example, it's common that values are
allocated, set to their default value, and then set to some other
value. This is wasteful, and it negates the benefits of optimizations
to the allocator like D15082. There are two reasons for this. The
first is array-of-structs storage that makes it annoying to initialize
values manually, and the second is confusing options in the Custom Data
API. This patch addresses the latter.
The `CustomData` "alloc type" options are rearranged. Now, besides
the options that use existing layers, there are two remaining:
* `CD_SET_DEFAULT` sets the default value.
* Usually zeroes, but for colors this is white (how it was before).
* Should be used when you add the layer but don't set all values.
* `CD_CONSTRUCT` refers to the "default construct" C++ term.
* Only necessary or defined for non-trivial types like vertex groups.
* Doesn't do anything for trivial types like `int` or `float3`.
* Should be used every other time, when all values will be set.
The attribute API's `AttributeInit` types are updated as well.
To update code, replace `CD_CALLOC` with `CD_SET_DEFAULT` and
`CD_DEFAULT` with `CD_CONSTRUCT`. This doesn't cause any functional
changes yet. Follow-up commits will change to avoid initializing
new layers where the correctness is clear.
Differential Revision: https://developer.blender.org/D15617
|
|
While fixing T100302 (rBd76583cb4a1) I did not realize that the
change in imported vertex order would actually matter. Turns out, it
does for morph targets / mesh shape keys. So redo the fix in a way
that does not change the vertex order. Fixes T100421.
|
|
span a continuous range
As part of the previous fix (D15410), the importer got code to track
min & max vertex indices used as part of the mesh faces. However, if
faces refer to a "sparse" (i.e. non-contiguous) subset of all vertices,
then the imported mesh would contain all the vertices between min & max
range.
Replace that with proper tracking of actually used vertex indices
for each imported mesh. Fixes T100302.
This does affect import performance a tiny bit, e.g. importing Blender
3.0 splash scene goes 21.7s -> 22.1s, and importing rungholt.obj
goes 2.37s -> 2.48s.
Importer related tests have a bunch of vertex changes in them, since
now vertices are added in the order that the faces are referring
to them. Which incidentally matches the order that the Python based
importer was creating them too.
|
|
|
|
The Python based importer had logic to immediately turn image paths
into relative-to-blender-file paths, if user preference for relative
paths is used (which is on by default). The new importer code did not
have that. Fixes T100076.
|
|
The importer code was written under incorrect assumption that vertex
data (v, vn, vt commands etc.) are grouped by object, i.e. follow the
o command, and that each object has its own vertex data commands. This
is not the case -- all the vertex data in the whole OBJ file is
"global", with no relation to any objects/groups; it's just that the
faces belong to the object, and then they pull in any vertices they
like.
This patch fixes this incorrect assumption in the importer:
- Vertex data is now properly global; no need to track some sort of
"offsets" per object like it was doing before.
- For each object, face definitions track the minimum & maximum vertex
indices referenced by the object, and then all that vertex range is
created in the final Blender object. Note: it might be (unusual, but
possible) that an object does not reference a sequential range of
vertices, e.g. just a single face with vertex indices 1, 10, 100 --
the resulting Blender mesh will have all the 100 vertices (some
"loose" without belonging to a face). It should be possible to track
the used vertices exactly (e.g. with a vector set), but I haven't
done that for performance reasons.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15410
|
|
assignment
The importer parts that were doing assignment of materials to the
imported objects/meshes were essentially having a quadratic complexity
in terms of scene object count. For each material assigned to each
object, they were scanning the whole scene, checking which other
Objects use the same Mesh data, in order to resize their material
arrays to match the size.
Performance details (Windows, Ryzen 5950X):
- Import OBJ Blender 3.0 splash scene (24k objects): 43.0s -> 32.9s
- Import USD Disney Moana scene (260k objects): saves two hours
(~7400s). Note that later on this crashes when trying to render the
imported result; crashes in the same way/place both in master and
this patch.
Implementation details:
The importers were doing "scan the world" basically twice for each
object, for each material: once when creating a new material slot
(assigns an empty material), and then again when assigning the
material.
However, all these importers (USD, Alembic, OBJ) always create one
Object for one Mesh. So that whole quadratic complexity resulting
from "scan the world for possible other users of this obdata" is
completely not needed; it just never finds anything. So add a new
dedicated function BKE_object_material_assign_single_obdata that skips
the expensive part, but should only be used when the caller knows that
the obdata has exactly one user (the passed object).
Reviewed By: Bastien Montagne, Michael Kowalski
Differential Revision: https://developer.blender.org/D15145
|
|
|
|
|
|
The old Python OBJ importer had a (somewhat confusingly named) "Keep
Vertex Order -> Poly Groups" option, that imported OBJ groups as
"vertex groups" on the resulting mesh. All vertices of any face were
assigned the vertex group, with a 1.0 weight.
The new C++ importer did not have this option. It was trying to do
something with vertex groups, but failing to actually achieve
anything :) -- the vertex groups were created on the wrong object
(later on overwritten by "nomain mesh to main mesh" operation);
vertex weights were set to 1.0/vertex_count, and each vertex was only
set to be in one group, even when it belongs to multiple faces from
different groups. End result was that to the user, vertex groups were
not visible/present at all (see T98874).
This patch adds the import option (named "Vertex Groups"), which is
off by default, and fixes the import code logic to actually do the
right thing. Tested on file from T98874; vertex groups are imported
just like with the Python importer.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15200
|
|
Adds support for vertex colors to OBJ I/O.
Importer:
- Supports both "xyzrgb" and "MRGB" vertex color formats.
- Whenever vertex color is present in the file for a model, it is
imported and a Color attribute is created (per-vertex, full float
color data type). Color coming from the file is assumed to be sRGB,
and is converted to linear upon import.
Exporter:
- Option to export the vertex colors. Defaults to "off", since not
all 3rd party software supports vertex colors.
- When the option is "on", if a mesh has a color attribute layer,
the active one is exported in "xyzrgb" form. If the mesh has
per-face-corner colors, they are averaged on the vertices.
Colors are converted from linear to sRGB upon export.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15159
|
|
|
|
Differential Revision: https://developer.blender.org/D14917
|
|
Fix several correctness issues where the new OBJ/MTL importer was not
producing the same results as the old one, mostly because the code for
some reason had slightly different logic. Fixes T97757:
- When .obj file tries to use a material that does not exist, the code
was continuing to use the previous material, instead of creating new
default one, as the previous importer did.
- Previous importer was always searching/parsing "foo.mtl" for a
"foo.obj" file, even if the file itself does not contain
"mtllib foo.mtl" statement. One file from T97757 repros happens to
depend on that, so resurrect that behavior.
- When IOR (Ni) or Alpha (d) are not specified in .mtl file, do not
wrongly set -1 values to the blender material.
- When base (Kd) or emissive (Ke) colors are not specified in the .mtl
file, do not set them on the blender material.
- Roughness and metallic values used by viewport shading were not set
onto blender material.
- The logic for when metallic was set to zero was incorrect; it should
be set to zero when "not using reflection", not when "mtl file does
not contain metallic".
- Do not produce a warning when illum value is not spelled out in .mtl
file, treat as default (1).
- Parse illum as a float just like python importer does, as to not
reintroduce part of T60135.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14822
|
|
Continued improvements to the new C++ based OBJ importer.
Performance: about 2x faster.
- Rungholt.obj (several meshes, 263MB file): Windows 12.7s -> 5.9s, Mac 7.7s -> 3.1s.
- Blender 3.0 splash (24k meshes, 2.4GB file): Windows 97.3s -> 53.6s, Mac 137.3s -> 80.0s.
- "Windows" is VS2022, AMD Ryzen 5950X (32 threads), "Mac" is Xcode/clang 13, M1Max (10 threads).
- Slightly reduced memory usage during import as well.
The performance gains are a combination of several things:
- Replacing `std::stof` / `std::stoi` with C++17 `from_chars`.
- Stop reading input file char-by-char using `std::getline`, and instead read in 64kb chunks, and parse from there (taking care of possibly handling lines split mid-way due to chunk boundaries).
- Removing abstractions for splitting a line by some char,
- Avoid tiny memory allocations: instead of storing a vector of polygon corners in each face, store all the corners in one big array, and per-face only store indices "where do corners start, and how many". Likewise, don't store full string names of material/group names for each face; only store indices into overall material/group names arrays.
- Stop always doing mesh validation, which is slow. Do it just like the Alembic importer does: only do validation if found some invalid faces during import, or if requested by the user via an import setting checkbox (which defaults to off).
- Stop doing "collection sync" for each object being added; instead do the collection sync right after creating all the objects.
Cleanup / Robustness:
This reworking of parser (see "removing abstractions" point above) means that all the functions that were in `parser_string_utils` file are gone, and replaced with different set of functions. However they are not OBJ specific, so as pointed out during review of the previous differential, they are now in `source/blender/io/common` library.
Added gtest coverage for said functions as well; something that was only indirectly covered by obj tests previously.
Rework of some bits of parsing made the parser actually better able to deal with invalid syntax. E.g. previously, if a face corner were a `/123` string, it would have incorrectly treated that as a vertex index (since it would get "hey that's one number" after splitting a string by a slash), instead of properly marking it as invalid syntax.
Added gtest coverage for .mtl parsing; something that was not covered by any tests at all previously.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14586
|
|
- The comment for create_normals was moved into an inline note
as it's not related to the public API.
- Use a colon after parameters.
Ref T92709
|
|
This takes state of soc-2020-io-performance branch as it was at
e9bbfd0c8c7 (2021 Oct 31), merges latest master (2022 Apr 4),
adds a bunch of tests, and fixes a bunch of stuff found by said
tests. The fixes are detailed in the differential.
Timings on my machine (Windows, VS2022 release build, AMD Ryzen
5950X 32 threads):
- Rungholt minecraft level (269MB file, 1 mesh): 54.2s -> 14.2s
(memory usage: 7.0GB -> 1.9GB).
- Blender 3.0 splash scene: "I waited for 90 minutes and gave up"
-> 109s. Now, this time is not great, but at least 20% of the
time is spent assigning unique names for the imported objects
(the scene has 24 thousand objects). This is not specific to obj
importer, but rather a general issue across blender overall.
Test suite file updates done in Subversion tests repository.
Reviewed By: @howardt, @sybren
Differential Revision: https://developer.blender.org/D13958
|
