| Age | Commit message (Collapse) | Author |
|---|
|
any face
The Python based importer had a special case handling of "no faces in
the whole file at all", where it ended up treating the whole file
as essentially a point-cloud-like object (just loose vertices, no
faces or edges). The new importer code was missing this special case.
Fixes T100017. Added gtest coverage that was failing without the fix.
|
|
The importer code was written under incorrect assumption that vertex
data (v, vn, vt commands etc.) are grouped by object, i.e. follow the
o command, and that each object has its own vertex data commands. This
is not the case -- all the vertex data in the whole OBJ file is
"global", with no relation to any objects/groups; it's just that the
faces belong to the object, and then they pull in any vertices they
like.
This patch fixes this incorrect assumption in the importer:
- Vertex data is now properly global; no need to track some sort of
"offsets" per object like it was doing before.
- For each object, face definitions track the minimum & maximum vertex
indices referenced by the object, and then all that vertex range is
created in the final Blender object. Note: it might be (unusual, but
possible) that an object does not reference a sequential range of
vertices, e.g. just a single face with vertex indices 1, 10, 100 --
the resulting Blender mesh will have all the 100 vertices (some
"loose" without belonging to a face). It should be possible to track
the used vertices exactly (e.g. with a vector set), but I haven't
done that for performance reasons.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15410
|
|
Address the issue by re-working line continuation handling: stop
trying to parse sequences like "backslash, newline" (which is the
bug: it should also handle "backslash, possible whitespace, newline")
during parsing. Instead, fixup line continuations after reading chunks
of input file data - turn backslash and the following newline into
spaces. The rest of parsing code does not have to be aware of them
at all then.
Makes the file attached to T99536 load correctly now. Also will extend
one of the test files in subversion tests repo to contain backslashes
followed by newlines.
|
|
As pointed out in a comment on T89421, if a MTL file contained
something like: `map_Ka -o 1 2.png` then it was parsed as having
offset `1 2` and the texture filename just a `.png`. Make it so that
mtl option numbers are parsed in a way where the number is only
accepted only if it's followed by whitespace.
Differential Revision: https://developer.blender.org/D15385
|
|
The old Python OBJ importer had a (somewhat confusingly named) "Keep
Vertex Order -> Poly Groups" option, that imported OBJ groups as
"vertex groups" on the resulting mesh. All vertices of any face were
assigned the vertex group, with a 1.0 weight.
The new C++ importer did not have this option. It was trying to do
something with vertex groups, but failing to actually achieve
anything :) -- the vertex groups were created on the wrong object
(later on overwritten by "nomain mesh to main mesh" operation);
vertex weights were set to 1.0/vertex_count, and each vertex was only
set to be in one group, even when it belongs to multiple faces from
different groups. End result was that to the user, vertex groups were
not visible/present at all (see T98874).
This patch adds the import option (named "Vertex Groups"), which is
off by default, and fixes the import code logic to actually do the
right thing. Tested on file from T98874; vertex groups are imported
just like with the Python importer.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15200
|
|
The new OBJ importer is producing "sharp" edges on some meshes that
should be completely smooth. Only observed on UV-Sphere type meshes
so far (see T97820).
I'm not 100% sure what is the root cause, but my theory was that
maybe due to limited number of float digits that are printed for
vertex normals in the file, the normals that are read in are not
always exactly 1.0 length. And then the Blender's "set custom loop
normals" function (which expects normalized inputs) wrongly marks
some edges as sharp.
Adding explicit normalization for the normals that are read from the
file fixes the wrongly-sharp edges in test cases from T97820. I
have not observed measurable performance impact in importing large
models (e.g. 6-level subdivided Monkey) that contain vertex normals.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15202
|
|
|
|
Some OBJ files out there (see T98782) have face definitions that
contain vertex normal indices, but the files themselves don't
contain any vertex normals. The code was doing a "hey, that's an
invalid index" and skipping these faces. But the old python importer
was silently ignoring these normal indices, so do the same here.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15177
|
|
Adds support for vertex colors to OBJ I/O.
Importer:
- Supports both "xyzrgb" and "MRGB" vertex color formats.
- Whenever vertex color is present in the file for a model, it is
imported and a Color attribute is created (per-vertex, full float
color data type). Color coming from the file is assumed to be sRGB,
and is converted to linear upon import.
Exporter:
- Option to export the vertex colors. Defaults to "off", since not
all 3rd party software supports vertex colors.
- When the option is "on", if a mesh has a color attribute layer,
the active one is exported in "xyzrgb" form. If the mesh has
per-face-corner colors, they are averaged on the vertices.
Colors are converted from linear to sRGB upon export.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15159
|
|
To match the existing Python .obj importer, and to make it easier for
the user to determine which object is which, use the filename for the
default object name instead of "New object".
Differential Revision: https://developer.blender.org/D15133
|
|
The OBJ parser was primarily using StringRef for convenience, with
functions like "skip whitespace" or "parse a number" taking an input
stringref, representing an input line, and returning a new stringref,
representing the remainder of the line. This is convenient, but does
more work than strictly needed -- while parsing, only the "beginning"
of the line ever changes by moving forward; the end of the line
always stays the same. We can change the code to take a pair of
pointers (begin of line, end of line) as input, and make the
functions return the new begin of line pointer. This makes the return
value neatly fit into a processor register, which StringRef did not.
On Windows, this does result in non-trivial speedups in the actual
OBJ file parsing part, due to Windows calling convention where return
values larger than 64 bits are returned via memory. Does not
measurably affect performance on Mac/Linux, because the calling
convention there uses a pair of 64-bit registers to return a
StringRef.
End-to-end times of importing several test files, on Windows
(VS2022 build, Ryzen 5950X):
- Monkey subdivided to level 6, no normals (220MB file): 1.25s -> 0.85s
- Rungholt minecraft level (270MB file): 7.0s -> 5.8s
- Blender 3 splash scene (2.4GB file): 49.1s -> 45.5s
The full import process has a lot of other overhead besides actual
OBJ file parsing (mostly creating actual blender objects out of
parsed data). In pure parsing, in the monkey test scene above, the
parsing part goes 1.0s -> 0.6s.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14936
|
|
As pointed out in https://developer.blender.org/rB213cd39b6db387bd88f12589fd50ff0e6563cf56#341113,
the utilities are quite OBJ specific due to treating backslash as a line
continuation character. It's unlikely that other formats need that.
No functionality changes, just pure code move (and renamed tests so that
their names reflect obj).
Reviewed By: Campbell Barton
Differential Revision: https://developer.blender.org/D14871
|
|
|
|
While possible extra whitespace after all OBJ/MTL keywords was properly
skipped, it was not done for the "f" (face definition) keyword.
While at it, also support indented keywords, i.e. extra whitespace at
the beginning of the line.
There's a tiny bit of performance drop while importing (e.g. importing
blender 3.0 splash scene: 53.38sec -> 54.21sec on my machine). But
correctness is more important.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14854
|
|
|
|
Fixes T97794 (which is a reintroduction of an older issue T67266 that
has been fixed in the python importer, but the fix was not in the C++
one). Some software produces OBJ files with mtllib statements like
mtllib "file name in quotes.mtl", and the new importer was not stripping
the quotes away.
While at it, I noticed that MTLParser constructor was taking a StringRef
and treating it as a zero-terminated string, which is not necessarily
the case. Fixed that by explicitly using a StringRefNull type.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14838
|
|
|
|
importer
- Fix T97793: when a UV coordinate after vt is missing, use zero. While
at it, also use zeroes for positions & normals, since "maximum
possible float" is very likely to cause issues in imported meshes.
- Fix T97795: use 1.0 default if -bm value is missing, instead of zero.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14826
|
|
Fix several correctness issues where the new OBJ/MTL importer was not
producing the same results as the old one, mostly because the code for
some reason had slightly different logic. Fixes T97757:
- When .obj file tries to use a material that does not exist, the code
was continuing to use the previous material, instead of creating new
default one, as the previous importer did.
- Previous importer was always searching/parsing "foo.mtl" for a
"foo.obj" file, even if the file itself does not contain
"mtllib foo.mtl" statement. One file from T97757 repros happens to
depend on that, so resurrect that behavior.
- When IOR (Ni) or Alpha (d) are not specified in .mtl file, do not
wrongly set -1 values to the blender material.
- When base (Kd) or emissive (Ke) colors are not specified in the .mtl
file, do not set them on the blender material.
- Roughness and metallic values used by viewport shading were not set
onto blender material.
- The logic for when metallic was set to zero was incorrect; it should
be set to zero when "not using reflection", not when "mtl file does
not contain metallic".
- Do not produce a warning when illum value is not spelled out in .mtl
file, treat as default (1).
- Parse illum as a float just like python importer does, as to not
reintroduce part of T60135.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14822
|
|
|
|
keywords
Even if available OBJ/MTL format documentations don't explicitly specify
which characters can possibly separate keywords & arguments, turns out
some files out there in the wild use TAB character after the line
keywords. Which is something the new 3.2 importer was not quite
expecting (T97417).
Fix this by factoring out a utility function that checks if line starts
with a keyword followed by any whitespace, and using that across the
importer. Also fix some other "possible whitespace around name-like
parts" of obj/mtl parser as pointed out by the repro files in T97417.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14782
|
|
- Inconsistent parameter names
- Else after return
- Braces around statements
- Qualified auto
- Also (not clang tidy): Pass StringRef by value, unused parameter
|
|
Continued improvements to the new C++ based OBJ importer.
Performance: about 2x faster.
- Rungholt.obj (several meshes, 263MB file): Windows 12.7s -> 5.9s, Mac 7.7s -> 3.1s.
- Blender 3.0 splash (24k meshes, 2.4GB file): Windows 97.3s -> 53.6s, Mac 137.3s -> 80.0s.
- "Windows" is VS2022, AMD Ryzen 5950X (32 threads), "Mac" is Xcode/clang 13, M1Max (10 threads).
- Slightly reduced memory usage during import as well.
The performance gains are a combination of several things:
- Replacing `std::stof` / `std::stoi` with C++17 `from_chars`.
- Stop reading input file char-by-char using `std::getline`, and instead read in 64kb chunks, and parse from there (taking care of possibly handling lines split mid-way due to chunk boundaries).
- Removing abstractions for splitting a line by some char,
- Avoid tiny memory allocations: instead of storing a vector of polygon corners in each face, store all the corners in one big array, and per-face only store indices "where do corners start, and how many". Likewise, don't store full string names of material/group names for each face; only store indices into overall material/group names arrays.
- Stop always doing mesh validation, which is slow. Do it just like the Alembic importer does: only do validation if found some invalid faces during import, or if requested by the user via an import setting checkbox (which defaults to off).
- Stop doing "collection sync" for each object being added; instead do the collection sync right after creating all the objects.
Cleanup / Robustness:
This reworking of parser (see "removing abstractions" point above) means that all the functions that were in `parser_string_utils` file are gone, and replaced with different set of functions. However they are not OBJ specific, so as pointed out during review of the previous differential, they are now in `source/blender/io/common` library.
Added gtest coverage for said functions as well; something that was only indirectly covered by obj tests previously.
Rework of some bits of parsing made the parser actually better able to deal with invalid syntax. E.g. previously, if a face corner were a `/123` string, it would have incorrectly treated that as a vertex index (since it would get "hey that's one number" after splitting a string by a slash), instead of properly marking it as invalid syntax.
Added gtest coverage for .mtl parsing; something that was not covered by any tests at all previously.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14586
|
|
- The comment for create_normals was moved into an inline note
as it's not related to the public API.
- Use a colon after parameters.
Ref T92709
|
|
|
|
|
|
Related to D13958
|
|
This takes state of soc-2020-io-performance branch as it was at
e9bbfd0c8c7 (2021 Oct 31), merges latest master (2022 Apr 4),
adds a bunch of tests, and fixes a bunch of stuff found by said
tests. The fixes are detailed in the differential.
Timings on my machine (Windows, VS2022 release build, AMD Ryzen
5950X 32 threads):
- Rungholt minecraft level (269MB file, 1 mesh): 54.2s -> 14.2s
(memory usage: 7.0GB -> 1.9GB).
- Blender 3.0 splash scene: "I waited for 90 minutes and gave up"
-> 109s. Now, this time is not great, but at least 20% of the
time is spent assigning unique names for the imported objects
(the scene has 24 thousand objects). This is not specific to obj
importer, but rather a general issue across blender overall.
Test suite file updates done in Subversion tests repository.
Reviewed By: @howardt, @sybren
Differential Revision: https://developer.blender.org/D13958
|
