Merged changes in the trunk up to revision 45133.

Conflicts resolved:
source/blender/blenloader/intern/readfile.c
source/blender/blenloader/intern/writefile.c
source/blender/bmesh/intern/bmesh_construct.c
source/blender/bmesh/intern/bmesh_mesh_conv.c
source/blender/bmesh/intern/bmesh_mesh_conv.h
source/blender/editors/interface/interface_templates.c
source/blender/editors/interface/resources.c
source/blender/editors/mesh/bmesh_select.c
source/blender/editors/mesh/bmesh_tools.c
source/blender/editors/space_view3d/drawobject.c
source/blender/render/intern/source/shadeoutput.c

5 files changed, 602 insertions, 143 deletions

diff --git a/doc/python_api/rst/bge.types.rst b/doc/python_api/rst/bge.types.rst
index 3c15362a4ec..bcef816dc9d 100644
--- a/doc/python_api/rst/bge.types.rst
+++ b/doc/python_api/rst/bge.types.rst
@@ -1806,9 +1806,9 @@ Game Types (bge.types)
       object = cont.owner
 
       for mesh in object.meshes:
-         for material in mesh.materials:
-            for v_index in range(mesh.getVertexArrayLength(mat)):
-               vertex = mesh.getVertex(mat, v_index)
+         for m_index in range(len(mesh.materials)):
+            for v_index in range(mesh.getVertexArrayLength(m_index)):
+               vertex = mesh.getVertex(m_index, v_index)
                # Do something with vertex here...
                # ... eg: colour the vertex red.
                vertex.colour = [1.0, 0.0, 0.0, 1.0]
diff --git a/doc/python_api/rst/include__bmesh.rst b/doc/python_api/rst/include__bmesh.rst
new file mode 100644
index 00000000000..a8d299a5894
--- /dev/null
+++ b/doc/python_api/rst/include__bmesh.rst
@@ -0,0 +1,134 @@
+..
+   This document is appended to the auto generated bmesh api doc to avoid clogging up the C files with details.
+   to test this run:
+   ./blender.bin -b -noaudio -P doc/python_api/sphinx_doc_gen.py -- --partial bmesh* ; cd doc/python_api ; sphinx-build sphinx-in sphinx-out ; cd ../../
+
+
+Intro
+-----
+
+This API gives access the blenders internal mesh editing api, featuring geometry connectivity data and
+access to editing operations such as split, separate, collapse and dissolve.
+
+The features exposed closely follow the C API,
+giving python access to the functions used by blenders own mesh editing tools.
+
+For an overview of BMesh data types and how they reference each other see:
+`BMesh Design Document <http://wiki.blender.org/index.php/Dev:2.6/Source/Modeling/BMesh/Design>`_ .
+
+
+.. note::
+
+   **Disk** and **Radial** data is not exposed by the python api since this is for internal use only.
+
+
+.. warning::
+
+   This API is still in development and experimental, while we don't expect to see large changes,
+   many areas are not well tested yet and so its possible changes will be made that break scripts.
+
+   *Campbell Barton, 13, March 2012*
+
+
+.. warning::
+
+   TODO Items Are
+
+   * add access to BMesh **walkers**
+   * add a way to re-tessellate an editmode bmesh.
+   * add deform vert custom-data access.
+
+
+Example Script
+--------------
+
+.. literalinclude:: ../../../release/scripts/templates/bmesh_simple.py
+
+
+Stand-Alone Module
+^^^^^^^^^^^^^^^^^^
+
+The bmesh module is written to be standalone except for :mod:`mathutils`
+which is used for vertex locations and normals.
+
+The only other exception to this are when converting mesh data to and from :class:`bpy.types.Mesh`.
+
+
+Mesh Access
+-----------
+
+There are 2 ways to access BMesh data, you can create a new BMesh by converting a mesh from
+:class:`bpy.types.BlendData.meshes` or by accessing the current edit mode mesh.
+see: :class:`bmesh.types.BMesh.from_mesh` and :mod:`bmesh.from_edit_mesh` respectively.
+
+When explicitly converting from mesh data python **owns** the data, that is to say - that the mesh only exists while
+python holds a reference to it, and the script is responsible for putting it back into a mesh data-block when the edits
+are done.
+
+Note that unlike :mod:`bpy`, a BMesh does not necessarily correspond to data in the currently open blend file,
+a BMesh can be created, edited and freed without the user ever seeing or having access to it.
+Unlike edit mode, the bmesh module can use multiple BMesh instances at once.
+
+Take care when dealing with multiple BMesh instances since the mesh data can use a lot of memory, while a mesh that
+python owns will be freed in when the script holds no references to it,
+its good practice to call :class:`bmesh.types.BMesh.free` which will remove all the mesh data immediately and disable
+further access.
+
+
+CustomData Access
+-----------------
+
+BMesh has a unified way to access mesh attributes such as UV's vertex colors, shape keys, edge crease etc.
+
+This works by having a **layers** property on bmesh data sequences to access the custom data layers which can then be
+used to access the actual data on each vert/edge/face/loop.
+
+Here are some examples ...
+
+.. code-block:: python
+
+   uv_lay = bm.loops.layers.uv.active
+
+   for face in bm.faces:
+       for loop in f.loops:
+           uv = loop[uv_lay]
+           print("Loop UV: %f, %f" % (uv.x, uv.y))
+
+
+.. code-block:: python
+
+   shape_lay = bm.verts.layers.shape["Key.001"]
+
+   for vert in bm.verts:
+		shape = vert[shape_lay]
+        print("Vert Shape: %f, %f, %f" % (shape.x, shape.y, shape.z))
+
+
+Keeping a Correct State
+-----------------------
+
+When modeling in blender there are certain assumptions made about the state of the mesh.
+
+* hidden geometry isn't selected.
+* when an edge is selected, its vertices are selected too.
+* when a face is selected, its edges and vertices are selected.
+* duplicate edges / faces don't exist.
+* faces have at least 3 vertices.
+
+To give developers flexibility these conventions are not enforced,
+however tools must leave the mesh in a valid state else other tools may behave incorrectly.
+
+Any errors that arise from not following these conventions is considered a bug in the script,
+not a bug in blender.
+
+
+Selection / Flushing
+^^^^^^^^^^^^^^^^^^^^
+
+As mentioned above, it is possible to create an invalid selection state
+(by selecting a state and then de-selecting one of its vertices's for example), mostly the best way to solve this is to
+flush the selection after performing a series of edits. this validates the selection state.
+
+
+Module Functions
+----------------
diff --git a/doc/python_api/rst/info_best_practice.rst b/doc/python_api/rst/info_best_practice.rst
index 1f0f81f7d14..481555d412a 100644
--- a/doc/python_api/rst/info_best_practice.rst
+++ b/doc/python_api/rst/info_best_practice.rst
@@ -268,7 +268,7 @@ The **try** statement is useful to save time writing error checking code.
 
 However **try** is significantly slower then an **if** since an exception has to be set each time, so avoid using **try** in areas of your code that execute in a loop and runs many times.
 
-There are cases where using **try** is faster than checking weather the condition will raise an error, so it is worth experimenting.
+There are cases where using **try** is faster than checking whether the condition will raise an error, so it is worth experimenting.
 
 
 Value Comparison
diff --git a/doc/python_api/rst/info_gotcha.rst b/doc/python_api/rst/info_gotcha.rst
index be11e88b2fa..25ef5175976 100644
--- a/doc/python_api/rst/info_gotcha.rst
+++ b/doc/python_api/rst/info_gotcha.rst
@@ -132,6 +132,91 @@ write useful tools in python which are also fast to execute while in edit-mode.
 For the time being this limitation just has to be worked around but we're aware its frustrating needs to be addressed.
 
 
+NGons and Tessellation Faces
+============================
+
+Since 2.63 NGons are supported, this adds some complexity since in some cases you need to access triangles still (some exporters for example).
+
+There are now 3 ways to access faces:
+
+* :class:`bpy.types.MeshPolygon` - this is the data stricture which now stores faces in object mode (access as ``mesh.polygons`` rather then ``mesh.faces``).
+* :class:`bpy.types.MeshTessFace` - the result of triangulating (tessellated) polygons, the main method of face access in 2.62 or older (access as ``mesh.tessfaces``).
+* :class:`bmesh.types.BMFace` - the polygons as used in editmode.
+
+For the purpose of the following documentation, these will be referred to as polygons, tessfaces and bmesh-faces respectively.
+
+5+ sided faces will be referred to as ``ngons``.
+
+Support Overview
+----------------
+
++--------------+------------------------------+--------------------------------+--------------------------------+
+|Usage         |:class:`bpy.types.MeshPolygon`|:class:`bpy.types.MeshTessFace` |:class:`bmesh.types.BMFace`     |
++==============+==============================+================================+================================+
+|Import/Create |Bad (inflexible)              |Fine (supported as upgrade path)|Best                            |
++--------------+------------------------------+--------------------------------+--------------------------------+
+|Manipulate    |Bad (inflexible)              |Bad (loses ngons)               |Best                            |
++--------------+------------------------------+--------------------------------+--------------------------------+
+|Export/Output |Good (ngons)                  |Good (When ngons can't be used) |Good (ngons, memory overhead)   |
++--------------+------------------------------+--------------------------------+--------------------------------+
+
+
+.. note::
+
+   Using the :mod:`bmesh` api is completely separate api from :mod:`bpy`, typically you would would use one or the other based on the level of editing needed, not simply for a different way to access faces.
+
+
+Creating
+--------
+
+All 3 datatypes can be used for face creation.
+
+* polygons are the most efficient way to create faces but the data structure is _very_ rigid and inflexible, you must have all your vertes and faces ready and create them all at once. This is further complicated by the fact that each polygon does not store its own verts (as with tessfaces), rather they reference an index and size in :class:`bpy.types.Mesh.loops` which are a fixed array too.
+* tessfaces ideally should not be used for creating faces since they are really only tessellation cache of polygons, however for scripts upgrading from 2.62 this is by far the most straightforward option. This works by creating tessfaces and when finished - they can be converted into polygons by calling :class:`bpy.types.Mesh.update`. The obvious limitation is ngons can't be created this way.
+* bmesh-faces are most likely the easiest way for new scripts to create faces, since faces can be added one by one and the api has features intended for mesh manipulation. While :class:`bmesh.types.BMesh` uses more memory it can be managed by only operating on one mesh at a time.
+
+
+Editing
+-------
+
+Editing is where the 3 data types vary most.
+
+* polygons are very limited for editing, changing materials and options like smooth works but for anything else they are too inflexible and are only intended for storage.
+* tessfaces should not be used for editing geometry because doing so will cause existing ngons to be tessellated.
+* bmesh-faces are by far the best way to manipulate geometry.
+
+Exporting
+---------
+
+All 3 data types can be used for exporting, the choice mostly depends on whether the target format supports ngons or not.
+
+* polygons are the most direct & efficient way to export providing they convert into the output format easily enough.
+* tessfaces work well for exporting to formats which dont support ngons, in fact this is the only place where their use is encouraged.
+* bmesh-faces can work for exporting too but may not be necessary if polygons can be used since using bmesh gives some overhead because its not the native storage format in object mode.
+
+
+Upgrading Importers from 2.62
+-----------------------------
+
+Importers can be upgraded to work with only minor changes.
+
+The main change to be made is used the tessellation versions of each attribute.
+
+* mesh.faces --> :class:`bpy.types.Mesh.tessfaces`
+* mesh.uv_textures --> :class:`bpy.types.Mesh.tessface_uv_textures`
+* mesh.vertex_colors --> :class:`bpy.types.Mesh.tessface_vertex_colors`
+
+Once the data is created call :class:`bpy.types.Mesh.update` to convert the tessfaces into polygons.
+
+
+Upgrading Exporters from 2.62
+-----------------------------
+
+For exporters the most direct way to upgrade is to use tessfaces as with importing however its important to know that tessfaces may **not** exist for a mesh, the array will be empty as if there are no faces.
+
+So before accessing tessface data call: :class:`bpy.types.Mesh.update` ``(calc_tessface=True)``.
+
+
 EditBones, PoseBones, Bone... Bones
 ===================================
 
diff --git a/doc/python_api/sphinx_doc_gen.py b/doc/python_api/sphinx_doc_gen.py
index b9444f35e12..7cc605fc6e7 100644
--- a/doc/python_api/sphinx_doc_gen.py
+++ b/doc/python_api/sphinx_doc_gen.py
@@ -26,16 +26,17 @@ API dump in RST files
 ---------------------
   Run this script from blenders root path once you have compiled blender
 
-    ./blender.bin -b -noaudio -P doc/python_api/sphinx_doc_gen.py
+    ./blender.bin --background -noaudio --python doc/python_api/sphinx_doc_gen.py
 
   This will generate python files in doc/python_api/sphinx-in/
   providing ./blender.bin is or links to the blender executable
 
   To choose sphinx-in directory:
-    ./blender.bin -b -P doc/python_api/sphinx_doc_gen.py -- -o ../python_api
+    ./blender.bin --background --python doc/python_api/sphinx_doc_gen.py -- --output ../python_api
 
   For quick builds:
-    ./blender.bin -b -P doc/python_api/sphinx_doc_gen.py -- -q
+    ./blender.bin --background --python doc/python_api/sphinx_doc_gen.py -- --partial
+
 
 Sphinx: HTML generation
 -----------------------
@@ -47,6 +48,7 @@ Sphinx: HTML generation
 
   This requires sphinx 1.0.7 to be installed.
 
+
 Sphinx: PDF generation
 ----------------------
   After you have built doc/python_api/sphinx-in (see above),
@@ -73,6 +75,7 @@ import os
 import sys
 import inspect
 import shutil
+import logging
 
 from platform import platform
 PLATFORM = platform().split('-')[0].lower()    # 'linux', 'darwin', 'windows'
@@ -93,32 +96,42 @@ def handle_args():
     )
 
     # optional arguments
-    parser.add_argument("-o", "--output",
-                        dest="output_dir",
+    parser.add_argument("-p", "--partial",
+                        dest="partial",
                         type=str,
-                        default=SCRIPT_DIR,
-                        help="Path of the API docs (default=<script dir>)",
+                        default="",
+                        help="Use a wildcard to only build specific module(s)\n"
+                             "Example: --partial bmesh*\n",
                         required=False)
 
-    parser.add_argument("-B", "--sphinx-build",
-                        dest="sphinx_build",
+    parser.add_argument("-f", "--fullrebuild",
+                        dest="full_rebuild",
                         default=False,
                         action='store_true',
-                        help="Run sphinx-build SPHINX_IN SPHINX_OUT (default=False)",
+                        help="Rewrite all rst files in sphinx-in/ "
+                             "(default=False)",
                         required=False)
 
-    parser.add_argument("-N", "--sphinx-named-output",
-                        dest="sphinx_named_output",
+    parser.add_argument("-b", "--bpy",
+                        dest="bpy",
                         default=False,
                         action='store_true',
-                        help="Add the theme name to the html dir name (default=False)",
+                        help="Write the rst file of the bpy module "
+                             "(default=False)",
+                        required=False)
+
+    parser.add_argument("-o", "--output",
+                        dest="output_dir",
+                        type=str,
+                        default=SCRIPT_DIR,
+                        help="Path of the API docs (default=<script dir>)",
                         required=False)
 
     parser.add_argument("-T", "--sphinx-theme",
                         dest="sphinx_theme",
                         type=str,
                         default='default',
-                        help=
+                        help =
                         # see SPHINX_THEMES below
                         "Sphinx theme (default='default')\n"
                         "Available themes\n"
@@ -132,25 +145,53 @@ def handle_args():
 #                                 'sphinxdoc', 'traditional'],   # sphinx
                         required=False)
 
-    parser.add_argument("-f", "--fullrebuild",
-                        dest="full_rebuild",
+    parser.add_argument("-N", "--sphinx-named-output",
+                        dest="sphinx_named_output",
                         default=False,
                         action='store_true',
-                        help="Rewrite all rst files in sphinx-in/ (default=False)",
+                        help="Add the theme name to the html dir name.\n"
+                             "Example: \"sphinx-out_haiku\" (default=False)",
                         required=False)
 
-    parser.add_argument("-t", "--testdump",
-                        dest="test_dump",
+    parser.add_argument("-B", "--sphinx-build",
+                        dest="sphinx_build",
                         default=False,
                         action='store_true',
-                        help="Dumps a small part of the API (default=False)",
+                        help="Build the html docs by running:\n"
+                             "sphinx-build SPHINX_IN SPHINX_OUT\n"
+                             "(default=False; does not depend on -P)",
                         required=False)
 
-    parser.add_argument("-b", "--bpy",
-                        dest="bpy",
+    parser.add_argument("-P", "--sphinx-build-pdf",
+                        dest="sphinx_build_pdf",
                         default=False,
                         action='store_true',
-                        help="Write the rst file of the bpy module (default=False)",
+                        help="Build the pdf by running:\n"
+                             "sphinx-build -b latex SPHINX_IN SPHINX_OUT_PDF\n"
+                             "(default=False; does not depend on -B)",
+                        required=False)
+
+    parser.add_argument("-R", "--pack-reference",
+                        dest="pack_reference",
+                        default=False,
+                        action='store_true',
+                        help="Pack all necessary files in the deployed dir.\n"
+                             "(default=False; use with -B and -P)",
+                        required=False)
+
+    parser.add_argument("-l", "--log",
+                        dest="log",
+                        default=False,
+                        action='store_true',
+                        help=(
+                        "Log the output of the api dump and sphinx|latex "
+                        "warnings and errors (default=False).\n"
+                        "If given, save logs in:\n"
+                        "* OUTPUT_DIR/.bpy.log\n"
+                        "* OUTPUT_DIR/.sphinx-build.log\n"
+                        "* OUTPUT_DIR/.sphinx-build_pdf.log\n"
+                        "* OUTPUT_DIR/.latex_make.log",
+                        ),
                         required=False)
 
     # parse only the args passed after '--'
@@ -165,15 +206,22 @@ ARGS = handle_args()
 
 # ----------------------------------BPY-----------------------------------------
 
+BPY_LOGGER = logging.getLogger('bpy')
+BPY_LOGGER.setLevel(logging.DEBUG)
+
 """
 # for quick rebuilds
 rm -rf /b/doc/python_api/sphinx-* && \
-./blender.bin --background -noaudio --factory-startup --python  doc/python_api/sphinx_doc_gen.py && \
+./blender.bin -b -noaudio --factory-startup -P doc/python_api/sphinx_doc_gen.py && \
 sphinx-build doc/python_api/sphinx-in doc/python_api/sphinx-out
+
+or
+
+./blender.bin -b -noaudio --factory-startup -P doc/python_api/sphinx_doc_gen.py -- -f -B
 """
 
 # Switch for quick testing so doc-builds don't take so long
-if not ARGS.test_dump:
+if not ARGS.partial:
     # full build
     FILTER_BPY_OPS = None
     FILTER_BPY_TYPES = None
@@ -181,6 +229,7 @@ if not ARGS.test_dump:
     EXCLUDE_MODULES = ()
 
 else:
+    # can manually edit this too:
     FILTER_BPY_OPS = ("import.scene", )  # allow
     FILTER_BPY_TYPES = ("bpy_struct", "Operator", "ID")  # allow
     EXCLUDE_INFO_DOCS = True
@@ -195,9 +244,9 @@ else:
         "bge.types",
         "bgl",
         "blf",
-        #"bmesh",
-        #"bmesh.types",
-        #"bmesh.utils",
+        "bmesh",
+        "bmesh.types",
+        "bmesh.utils",
         "bpy.app",
         "bpy.app.handlers",
         "bpy.context",
@@ -215,10 +264,29 @@ else:
         "Freestyle",
     )
 
+    # ------
+    # Filter
+    #
+    # TODO, support bpy.ops and bpy.types filtering
+    import fnmatch
+    m = None
+    EXCLUDE_MODULES = tuple([m for m in EXCLUDE_MODULES if not fnmatch.fnmatchcase(m, ARGS.partial)])
+
+    EXCLUDE_INFO_DOCS = (not fnmatch.fnmatchcase("info", ARGS.partial))
+
+    del m
+    del fnmatch
+
+    BPY_LOGGER.debug("Partial Doc Build, Skipping: %s\n" % "\n                             ".join(sorted(EXCLUDE_MODULES)))
+
+    #
+    # done filtering
+    # --------------
+
 try:
     __import__("aud")
 except ImportError:
-    print("Warning: Built without 'aud' module, docs incomplete...")
+    BPY_LOGGER.debug("Warning: Built without 'aud' module, docs incomplete...")
     EXCLUDE_MODULES = EXCLUDE_MODULES + ("aud", )
 
 # examples
@@ -229,7 +297,7 @@ for f in os.listdir(EXAMPLES_DIR):
         EXAMPLE_SET.add(os.path.splitext(f)[0])
 EXAMPLE_SET_USED = set()
 
-#rst files dir
+# rst files dir
 RST_DIR = os.path.abspath(os.path.join(SCRIPT_DIR, "rst"))
 
 # extra info, not api reference docs
@@ -248,10 +316,66 @@ RNA_BLACKLIST = {
     "UserPreferencesSystem": {"language", }
     }
 
+MODULE_GROUPING = {
+    "bmesh.types": (
+                    ("Base Mesh Type", '-'),
+                    "BMesh",
+                    ("Mesh Elements", '-'),
+                    "BMVert",
+                    "BMEdge",
+                    "BMFace",
+                    "BMLoop",
+                    ("Sequence Accessors", '-'),
+                    "BMElemSeq",
+                    "BMVertSeq",
+                    "BMEdgeSeq",
+                    "BMFaceSeq",
+                    "BMLoopSeq",
+                    "BMIter",
+                    ("Selection History", '-'),
+                    "BMEditSelSeq",
+                    "BMEditSelIter",
+                    ("Custom-Data Layer Access", '-'),
+                    "BMLayerAccessVert",
+                    "BMLayerAccessEdge",
+                    "BMLayerAccessFace",
+                    "BMLayerAccessLoop",
+                    "BMLayerCollection",
+                    "BMLayerItem",
+                    ("Custom-Data Layer Types", '-'),
+                    "BMLoopUV"
+                    )
+    }
+
+# --------------------configure compile time options----------------------------
+
+# -------------------------------BLENDER----------------------------------------
+
+blender_version_strings = [str(v) for v in bpy.app.version]
+
+# converting bytes to strings, due to #30154
+BLENDER_REVISION = str(bpy.app.build_revision, 'utf_8')
+BLENDER_DATE = str(bpy.app.build_date, 'utf_8')
+
+BLENDER_VERSION_DOTS = ".".join(blender_version_strings)    # '2.62.1'
+if BLENDER_REVISION != "Unknown":
+    BLENDER_VERSION_DOTS += " r" + BLENDER_REVISION         # '2.62.1 r44584'
+
+BLENDER_VERSION_PATH = "_".join(blender_version_strings)    # '2_62_1'
+if bpy.app.version_cycle == "release":
+    BLENDER_VERSION_PATH = "%s%s_release" % ("_".join(blender_version_strings[:2]),
+                                             bpy.app.version_char)   # '2_62_release'
+
+# --------------------------DOWNLOADABLE FILES----------------------------------
+
+REFERENCE_NAME = "blender_python_reference_%s" % BLENDER_VERSION_PATH
+REFERENCE_PATH = os.path.join(ARGS.output_dir, REFERENCE_NAME)
+BLENDER_PDF_FILENAME = "%s.pdf" % REFERENCE_NAME
+BLENDER_ZIP_FILENAME = "%s.zip" % REFERENCE_NAME
 
 # -------------------------------SPHINX-----------------------------------------
 
-SPHINX_THEMES = {'bf': ['blender-org'], # , 'naiad',
+SPHINX_THEMES = {'bf': ['blender-org'],  # , 'naiad',
                  'sphinx': ['agogo',
                             'basic',
                             'default',
@@ -267,37 +391,43 @@ if ARGS.sphinx_theme not in available_themes:
     print ("Please choose a theme among: %s" % ', '.join(available_themes))
     sys.exit()
 
+if ARGS.sphinx_theme in SPHINX_THEMES['bf']:
+    SPHINX_THEME_DIR = os.path.join(ARGS.output_dir, ARGS.sphinx_theme)
+    SPHINX_THEME_SVN_DIR = os.path.join(SCRIPT_DIR, ARGS.sphinx_theme)
+
 SPHINX_IN = os.path.join(ARGS.output_dir, "sphinx-in")
 SPHINX_IN_TMP = SPHINX_IN + "-tmp"
 SPHINX_OUT = os.path.join(ARGS.output_dir, "sphinx-out")
 if ARGS.sphinx_named_output:
     SPHINX_OUT += "_%s" % ARGS.sphinx_theme
 
-if ARGS.sphinx_theme in SPHINX_THEMES['bf']:
-    SPHINX_THEME_DIR = os.path.join(ARGS.output_dir, ARGS.sphinx_theme)
-    SPHINX_THEME_SVN_DIR = os.path.join(SCRIPT_DIR, ARGS.sphinx_theme)
-
-# ------------------------------------------------------------------------------
-
-# configure compile time options
-
-# -------------------------------BLENDER----------------------------------------
-
-'''
-blender version
-'''
-version_strings = [str(v) for v in bpy.app.version]
-
-BLENDER_VERSION_DOTS = ".".join(version_strings)    # '2.62.1'
-if bpy.app.build_revision != b"Unknown":
-    # converting bytes to strings, due to #30154
-    BLENDER_VERSION_DOTS += " r" + str(bpy.app.build_revision, 'utf_8')    # '2.62.1 r44584'
-
-BLENDER_VERSION_PDF = "_".join(version_strings)    # '2_62_1'
-if bpy.app.version_cycle == "release":
-    BLENDER_VERSION_PDF = "%s%s_release" % ("_".join(version_strings[:2]),
-                                            bpy.app.version_char)   # '2_62_release'
-
+# html build
+if ARGS.sphinx_build:
+    SPHINX_BUILD = ["sphinx-build", SPHINX_IN, SPHINX_OUT]
+
+    if ARGS.log:
+        SPHINX_BUILD_LOG = os.path.join(ARGS.output_dir, ".sphinx-build.log")
+        SPHINX_BUILD = ["sphinx-build",
+                        "-w", SPHINX_BUILD_LOG,
+                        SPHINX_IN, SPHINX_OUT]
+
+# pdf build
+if ARGS.sphinx_build_pdf:
+    SPHINX_OUT_PDF = os.path.join(ARGS.output_dir, "sphinx-out_pdf")
+    SPHINX_BUILD_PDF = ["sphinx-build",
+                        "-b", "latex",
+                        SPHINX_IN, SPHINX_OUT_PDF]
+    SPHINX_MAKE_PDF = ["make", "-C", SPHINX_OUT_PDF]
+    SPHINX_MAKE_PDF_STDOUT = None
+
+    if ARGS.log:
+        SPHINX_BUILD_PDF_LOG = os.path.join(ARGS.output_dir, ".sphinx-build_pdf.log")
+        SPHINX_BUILD_PDF = ["sphinx-build", "-b", "latex",
+                            "-w", SPHINX_BUILD_PDF_LOG,
+                            SPHINX_IN, SPHINX_OUT_PDF]
+
+        sphinx_make_pdf_log = os.path.join(ARGS.output_dir, ".latex_make.log")
+        SPHINX_MAKE_PDF_STDOUT = open(sphinx_make_pdf_log, "w", encoding="utf-8")
 
 # --------------------------------API DUMP--------------------------------------
 
@@ -326,13 +456,13 @@ def undocumented_message(module_name, type_name, identifier):
         preloadtitle = '%s.%s' % (module_name, identifier)
     else:
         preloadtitle = '%s.%s.%s' % (module_name, type_name, identifier)
-    message =   "Undocumented (`contribute "\
-                "<http://wiki.blender.org/index.php/"\
-                "Dev:2.5/Py/API/Generating_API_Reference/Contribute/Howto-message"\
-                "?action=edit"\
-                "&section=new"\
-                "&preload=Dev:2.5/Py/API/Documentation/Contribute/Howto-message"\
-                "&preloadtitle=%s>`_)\n\n" % preloadtitle
+    message = ("Undocumented (`contribute "
+               "<http://wiki.blender.org/index.php/"
+               "Dev:2.5/Py/API/Generating_API_Reference/Contribute"
+               "?action=edit"
+               "&section=new"
+               "&preload=Dev:2.5/Py/API/Generating_API_Reference/Contribute/Howto-message"
+               "&preloadtitle=%s>`_)\n\n" % preloadtitle)
     return message
 
 
@@ -374,15 +504,20 @@ def example_extract_docstring(filepath):
     return "\n".join(text), line_no
 
 
-def write_title(fw, text, heading_char):
-    fw("%s\n%s\n\n" % (text, len(text) * heading_char))
+def title_string(text, heading_char, double=False):
+    filler = len(text) * heading_char
+
+    if double:
+        return "%s\n%s\n%s\n\n" % (filler, text, filler)
+    else:
+        return "%s\n%s\n\n" % (text, filler)
 
 
 def write_example_ref(ident, fw, example_id, ext="py"):
     if example_id in EXAMPLE_SET:
 
         # extract the comment
-        filepath = os.path.join(EXAMPLES_DIR, "%s.%s" % (example_id, ext))
+        filepath = os.path.join("..", "examples", "%s.%s" % (example_id, ext))
         filepath_full = os.path.join(os.path.dirname(fw.__self__.name), filepath)
 
         text, line_no = example_extract_docstring(filepath_full)
@@ -398,7 +533,7 @@ def write_example_ref(ident, fw, example_id, ext="py"):
         EXAMPLE_SET_USED.add(example_id)
     else:
         if bpy.app.debug:
-            print("\tskipping example:", example_id)
+            BPY_LOGGER.debug("\tskipping example: " + example_id)
 
     # Support for numbered files bpy.types.Operator -> bpy.types.Operator.1.py
     i = 1
@@ -557,11 +692,34 @@ def pymodule2sphinx(basepath, module_name, module, title):
     if module_all:
         module_dir = module_all
 
+    # TODO - currently only used for classes
+    # grouping support
+    module_grouping = MODULE_GROUPING.get(module_name)
+
+    def module_grouping_index(name):
+        if module_grouping is not None:
+            try:
+                return module_grouping.index(name)
+            except ValueError:
+                pass
+        return -1
+
+    def module_grouping_heading(name):
+        if module_grouping is not None:
+            i = module_grouping_index(name) - 1
+            if i >= 0 and type(module_grouping[i]) == tuple:
+                return module_grouping[i]
+        return None, None
+
+    def module_grouping_sort_key(name):
+        return module_grouping_index(name)
+    # done grouping support
+
     file = open(filepath, "w", encoding="utf-8")
 
     fw = file.write
 
-    write_title(fw, "%s (%s)" % (title, module_name), "=")
+    fw(title_string("%s (%s)" % (title, module_name), "="))
 
     fw(".. module:: %s\n\n" % module_name)
 
@@ -685,7 +843,7 @@ def pymodule2sphinx(basepath, module_name, module, title):
             write_indented_lines("   ", fw, "constant value %s" % repr(value), False)
             fw("\n")
         else:
-            print("\tnot documenting %s.%s of %r type" % (module_name, attribute, value_type.__name__))
+            BPY_LOGGER.debug("\tnot documenting %s.%s of %r type" % (module_name, attribute, value_type.__name__))
             continue
 
         attribute_set.add(attribute)
@@ -693,7 +851,7 @@ def pymodule2sphinx(basepath, module_name, module, title):
     del module_dir_value_type
 
     # TODO, bpy_extras does this already, mathutils not.
-    """
+    '''
     if submodules:
         fw("\n"
            "**********\n"
@@ -704,10 +862,19 @@ def pymodule2sphinx(basepath, module_name, module, title):
         for attribute, submod in submodules:
             fw("* :mod:`%s.%s`\n" % (module_name, attribute))
         fw("\n")
-    """
+    '''
+
+    if module_grouping is not None:
+        classes.sort(key=lambda pair: module_grouping_sort_key(pair[0]))
 
     # write collected classes now
     for (type_name, value) in classes:
+
+        if module_grouping is not None:
+            heading, heading_char = module_grouping_heading(type_name)
+            if heading:
+                fw(title_string(heading, heading_char))
+
         # May need to be its own function
         fw(".. class:: %s\n\n" % type_name)
         if value.__doc__:
@@ -746,8 +913,7 @@ def pycontext2sphinx(basepath):
     filepath = os.path.join(basepath, "bpy.context.rst")
     file = open(filepath, "w", encoding="utf-8")
     fw = file.write
-    fw("Context Access (bpy.context)\n")
-    fw("============================\n\n")
+    fw(title_string("Context Access (bpy.context)", "="))
     fw(".. module:: bpy.context\n")
     fw("\n")
     fw("The context members available depend on the area of blender which is currently being accessed.\n")
@@ -950,7 +1116,7 @@ def pyrna2sphinx(basepath):
         else:
             title = struct_id
 
-        write_title(fw, title, "=")
+        fw(title_string(title, "="))
 
         fw(".. module:: bpy.types\n\n")
 
@@ -1161,7 +1327,7 @@ def pyrna2sphinx(basepath):
             file = open(filepath, "w", encoding="utf-8")
             fw = file.write
 
-            write_title(fw, class_name, "=")
+            fw(title_string(class_name, "="))
 
             fw(".. module:: bpy.types\n")
             fw("\n")
@@ -1214,7 +1380,7 @@ def pyrna2sphinx(basepath):
 
             title = "%s Operators" % op_module_name.replace("_", " ").title()
 
-            write_title(fw, title, "=")
+            fw(title_string(title, "="))
 
             fw(".. module:: bpy.ops.%s\n\n" % op_module_name)
 
@@ -1297,38 +1463,30 @@ def write_rst_contents(basepath):
     file = open(filepath, "w", encoding="utf-8")
     fw = file.write
 
-    fw("%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n")
-    fw(" Blender Documentation contents\n")
-    fw("%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n")
+    fw(title_string("Blender Documentation Contents", "%", double=True))
     fw("\n")
-    fw("Welcome, this document is an API reference for Blender %s. built %s.\n" % (BLENDER_VERSION_DOTS, str(bpy.app.build_date, 'utf_8')))
+    fw("Welcome, this document is an API reference for Blender %s, built %s.\n" % (BLENDER_VERSION_DOTS, BLENDER_DATE))
     fw("\n")
 
-    # fw("`A PDF version of this document is also available <blender_python_reference_%s.pdf>`_\n" % BLENDER_VERSION_PDF)
-    fw("`A compressed ZIP file of this site is available <blender_python_reference_%s.zip>`_\n" % BLENDER_VERSION_PDF)
+    # fw("`A PDF version of this document is also available <%s>`_\n" % BLENDER_PDF_FILENAME)
+    fw("`A compressed ZIP file of this site is available <%s>`_\n" % BLENDER_ZIP_FILENAME)
 
     fw("\n")
 
     if not EXCLUDE_INFO_DOCS:
-        fw("============================\n")
-        fw("Blender/Python Documentation\n")
-        fw("============================\n")
-        fw("\n")
-        fw("\n")
+        fw(title_string("Blender/Python Documentation", "=", double=True))
+
         fw(".. toctree::\n")
         fw("   :maxdepth: 1\n\n")
         for info, info_desc in INFO_DOCS:
             fw("   %s <%s>\n\n" % (info_desc, info))
         fw("\n")
 
-    fw("===================\n")
-    fw("Application Modules\n")
-    fw("===================\n")
-    fw("\n")
+    fw(title_string("Application Modules", "=", double=True))
     fw(".. toctree::\n")
     fw("   :maxdepth: 1\n\n")
 
-    app_modules = [
+    app_modules = (
         "bpy.context",  # note: not actually a module
         "bpy.data",     # note: not actually a module
         "bpy.ops",
@@ -1341,37 +1499,33 @@ def write_rst_contents(basepath):
         "bpy.app.handlers",
 
         # C modules
-        "bpy.props"
-    ]
+        "bpy.props",
+        )
+
     for mod in app_modules:
         if mod not in EXCLUDE_MODULES:
             fw("   %s\n\n" % mod)
 
-    fw("==================\n")
-    fw("Standalone Modules\n")
-    fw("==================\n")
-    fw("\n")
+    fw(title_string("Standalone Modules", "=", double=True))
     fw(".. toctree::\n")
     fw("   :maxdepth: 1\n\n")
 
-    standalone_modules = [
+    standalone_modules = (
         # mathutils
         "mathutils", "mathutils.geometry", "mathutils.noise",
         # misc
         "Freestyle", "bgl", "blf", "gpu", "aud", "bpy_extras",
         # bmesh
-        "bmesh", "bmesh.types", "bmesh.utils"
-    ]
+        "bmesh", "bmesh.types", "bmesh.utils",
+        )
+
     for mod in standalone_modules:
         if mod not in EXCLUDE_MODULES:
             fw("   %s\n\n" % mod)
 
     # game engine
     if "bge" not in EXCLUDE_MODULES:
-        fw("===================\n")
-        fw("Game Engine Modules\n")
-        fw("===================\n")
-        fw("\n")
+        fw(title_string("Game Engine Modules", "=", double=True))
         fw(".. toctree::\n")
         fw("   :maxdepth: 1\n\n")
         fw("   bge.types.rst\n\n")
@@ -1382,10 +1536,7 @@ def write_rst_contents(basepath):
         fw("   bge.constraints.rst\n\n")
 
     # rna generated change log
-    fw("========\n")
-    fw("API Info\n")
-    fw("========\n")
-    fw("\n")
+    fw(title_string("API Info", "=", double=True))
     fw(".. toctree::\n")
     fw("   :maxdepth: 1\n\n")
     fw("   change_log.rst\n\n")
@@ -1421,7 +1572,7 @@ def write_rst_bpy(basepath):
 
         title = ":mod:`bpy` --- Blender Python Module"
 
-        write_title(fw, title, "=")
+        fw(title_string(title, "="))
 
         fw(".. module:: bpy.types\n\n")
         file.close()
@@ -1435,8 +1586,7 @@ def write_rst_types_index(basepath):
         filepath = os.path.join(basepath, "bpy.types.rst")
         file = open(filepath, "w", encoding="utf-8")
         fw = file.write
-        fw("Types (bpy.types)\n")
-        fw("=================\n\n")
+        fw(title_string("Types (bpy.types)", "="))
         fw(".. toctree::\n")
         fw("   :glob:\n\n")
         fw("   bpy.types.*\n\n")
@@ -1451,8 +1601,7 @@ def write_rst_ops_index(basepath):
         filepath = os.path.join(basepath, "bpy.ops.rst")
         file = open(filepath, "w", encoding="utf-8")
         fw = file.write
-        fw("Operators (bpy.ops)\n")
-        fw("===================\n\n")
+        fw(title_string("Operators (bpy.ops)", "="))
         write_example_ref("", fw, "bpy.ops")
         fw(".. toctree::\n")
         fw("   :glob:\n\n")
@@ -1470,8 +1619,7 @@ def write_rst_data(basepath):
         filepath = os.path.join(basepath, "bpy.data.rst")
         file = open(filepath, "w", encoding="utf-8")
         fw = file.write
-        fw("Data Access (bpy.data)\n")
-        fw("======================\n\n")
+        fw(title_string("Data Access (bpy.data)", "="))
         fw(".. module:: bpy\n")
         fw("\n")
         fw("This module is used for all blender/python access.\n")
@@ -1536,6 +1684,9 @@ def copy_handwritten_rsts(basepath):
         "bge.constraints",
         "bgl",  # "Blender OpenGl wrapper"
         "gpu",  # "GPU Shader Module"
+
+        # includes...
+        "include__bmesh",
     ]
     for mod_name in handwritten_modules:
         if mod_name not in EXCLUDE_MODULES:
@@ -1590,7 +1741,7 @@ def align_sphinx_in_to_sphinx_in_tmp():
     # remove deprecated files that have been removed
     for f in sorted(sphinx_in_files):
         if f not in sphinx_in_tmp_files:
-            print("\tdeprecated: %s" % f)
+            BPY_LOGGER.debug("\tdeprecated: %s" % f)
             os.remove(os.path.join(SPHINX_IN, f))
 
     # freshen with new files.
@@ -1604,53 +1755,142 @@ def align_sphinx_in_to_sphinx_in_tmp():
                 do_copy = False
 
         if do_copy:
-            print("\tupdating: %s" % f)
+            BPY_LOGGER.debug("\tupdating: %s" % f)
             shutil.copy(f_from, f_to)
 
 
+def refactor_sphinx_log(sphinx_logfile):
+    refactored_log = []
+    with open(sphinx_logfile, "r", encoding="utf-8") as original_logfile:
+        lines = set(original_logfile.readlines())
+        for line in lines:
+            if 'warning' in line.lower() or 'error' in line.lower():
+                line = line.strip().split(None, 2)
+                if len(line) == 3:
+                    location, kind, msg = line
+                    location = os.path.relpath(location, start=SPHINX_IN)
+                    refactored_log.append((kind, location, msg))
+    with open(sphinx_logfile, "w", encoding="utf-8") as refactored_logfile:
+        for log in sorted(refactored_log):
+            refactored_logfile.write("%-12s %s\n             %s\n" % log)
+
+
 def main():
 
-    # dirs preparation
+    # eventually, create the dirs
     for dir_path in [ARGS.output_dir, SPHINX_IN]:
         if not os.path.exists(dir_path):
             os.mkdir(dir_path)
 
+    # eventually, log in files
+    if ARGS.log:
+        bpy_logfile = os.path.join(ARGS.output_dir, ".bpy.log")
+        bpy_logfilehandler = logging.FileHandler(bpy_logfile, mode="w")
+        bpy_logfilehandler.setLevel(logging.DEBUG)
+        BPY_LOGGER.addHandler(bpy_logfilehandler)
+
+        # using a FileHandler seems to disable the stdout, so we add a StreamHandler
+        bpy_log_stdout_handler = logging.StreamHandler(stream=sys.stdout)
+        bpy_log_stdout_handler.setLevel(logging.DEBUG)
+        BPY_LOGGER.addHandler(bpy_log_stdout_handler)
+
+    # in case of out-of-source build, copy the needed dirs
+    if ARGS.output_dir != SCRIPT_DIR:
+        # examples dir
+        examples_dir_copy = os.path.join(ARGS.output_dir, "examples")
+        if os.path.exists(examples_dir_copy):
+            shutil.rmtree(examples_dir_copy, True)
+        shutil.copytree(EXAMPLES_DIR,
+                        examples_dir_copy,
+                        ignore=shutil.ignore_patterns(*(".svn",)),
+                        copy_function=shutil.copy)
+
+        # eventually, copy the theme dir
+        if ARGS.sphinx_theme in SPHINX_THEMES['bf']:
+            if os.path.exists(SPHINX_THEME_DIR):
+                shutil.rmtree(SPHINX_THEME_DIR, True)
+            shutil.copytree(SPHINX_THEME_SVN_DIR,
+                            SPHINX_THEME_DIR,
+                            ignore=shutil.ignore_patterns(*(".svn",)),
+                            copy_function=shutil.copy)
+
     # dump the api in rst files
+    if os.path.exists(SPHINX_IN_TMP):
+        shutil.rmtree(SPHINX_IN_TMP, True)
+
+    rna2sphinx(SPHINX_IN_TMP)
+
     if ARGS.full_rebuild:
         # only for full updates
         shutil.rmtree(SPHINX_IN, True)
-        shutil.rmtree(SPHINX_OUT, True)
-        rna2sphinx(SPHINX_IN_TMP)
         shutil.copytree(SPHINX_IN_TMP,
                         SPHINX_IN,
                         copy_function=shutil.copy)
+        if ARGS.sphinx_build and os.path.exists(SPHINX_OUT):
+            shutil.rmtree(SPHINX_OUT, True)
+        if ARGS.sphinx_build_pdf and os.path.exists(SPHINX_OUT_PDF):
+            shutil.rmtree(SPHINX_OUT_PDF, True)
     else:
-        # write here, then move
-        shutil.rmtree(SPHINX_IN_TMP, True)
-        rna2sphinx(SPHINX_IN_TMP)
+        # move changed files in SPHINX_IN
         align_sphinx_in_to_sphinx_in_tmp()
 
     # report which example files weren't used
     EXAMPLE_SET_UNUSED = EXAMPLE_SET - EXAMPLE_SET_USED
     if EXAMPLE_SET_UNUSED:
-        print("\nUnused examples found in '%s'..." % EXAMPLES_DIR)
-        for f in EXAMPLE_SET_UNUSED:
-            print("    %s.py" % f)
-        print("  %d total\n" % len(EXAMPLE_SET_UNUSED))
-
-    # eventually, copy the theme in the output directory
-    if ARGS.sphinx_theme in SPHINX_THEMES['bf']:
-        if not os.path.exists(SPHINX_THEME_DIR):
-            shutil.copytree(SPHINX_THEME_SVN_DIR,
-                            SPHINX_THEME_DIR,
-                            copy_function=shutil.copy)
+        BPY_LOGGER.debug("\nUnused examples found in '%s'..." % EXAMPLES_DIR)
+        for f in sorted(EXAMPLE_SET_UNUSED):
+            BPY_LOGGER.debug("    %s.py" % f)
+        BPY_LOGGER.debug("  %d total\n" % len(EXAMPLE_SET_UNUSED))
 
-    # eventually, build the docs
+    # eventually, build the html docs
     if ARGS.sphinx_build:
         import subprocess
-        sphinx_build_command = "sphinx-build %s %s" % (SPHINX_IN, SPHINX_OUT)
-        print ('\n%s\n' % sphinx_build_command)
-        subprocess.call(sphinx_build_command, shell=True)
+        subprocess.call(SPHINX_BUILD)
+
+        # sphinx-build log cleanup+sort
+        if ARGS.log:
+            if os.stat(SPHINX_BUILD_LOG).st_size:
+                refactor_sphinx_log(SPHINX_BUILD_LOG)
+
+    # eventually, build the pdf docs
+    if ARGS.sphinx_build_pdf:
+        import subprocess
+        subprocess.call(SPHINX_BUILD_PDF)
+        subprocess.call(SPHINX_MAKE_PDF, stdout=SPHINX_MAKE_PDF_STDOUT)
+
+        # sphinx-build log cleanup+sort
+        if ARGS.log:
+            if os.stat(SPHINX_BUILD_PDF_LOG).st_size:
+                refactor_sphinx_log(SPHINX_BUILD_PDF_LOG)
+
+    # eventually, prepare the dir to be deployed online (REFERENCE_PATH)
+    if ARGS.pack_reference:
+
+        if ARGS.sphinx_build:
+            # delete REFERENCE_PATH
+            if os.path.exists(REFERENCE_PATH):
+                shutil.rmtree(REFERENCE_PATH, True)
+
+            # copy SPHINX_OUT to the REFERENCE_PATH
+            ignores = ('.doctrees', 'objects.inv', '.buildinfo')
+            shutil.copytree(SPHINX_OUT,
+                            REFERENCE_PATH,
+                            ignore=shutil.ignore_patterns(*ignores))
+            shutil.copy(os.path.join(REFERENCE_PATH, "contents.html"),
+                        os.path.join(REFERENCE_PATH, "index.html"))
+
+            # zip REFERENCE_PATH
+            basename = os.path.join(ARGS.output_dir, REFERENCE_NAME)
+            tmp_path = shutil.make_archive(basename, 'zip',
+                                           root_dir=ARGS.output_dir,
+                                           base_dir=REFERENCE_NAME)
+            final_path = os.path.join(REFERENCE_PATH, BLENDER_ZIP_FILENAME)
+            os.rename(tmp_path, final_path)
+
+        if ARGS.sphinx_build_pdf:
+            # copy the pdf to REFERENCE_PATH
+            shutil.copy(os.path.join(SPHINX_OUT_PDF, "contents.pdf"),
+                        os.path.join(REFERENCE_PATH, BLENDER_PDF_FILENAME))
 
     sys.exit()