Bunch of fixes for GLSL display transform

- GLSL shader wasn't aware of alpha predivide option,
  always assuming alpha is straight. Gave wrong results
  when displaying transparent float buffers.

- GLSL display wasn't aware of float buffers with number
  of channels different from 4, crashing when trying to
  display image with different number of channels.

  This required a bit larger changes, namely now it's
  possible to pass format (GL_RGB, GL_RGBAm GL_LUMINANCE)
  to glaDrawPixelsTex, This also implied adding format to
  glaDrawPixelsAuto and modifying all places where this
  functions are called.

  Now GLSL will handle both 3 and 4 channels buffers,
  single channel images are handled by CPU.

- Replaced hack for render result displaying with a bit
  different hack.

  Namely CPU conversion will happen only during render,
  once render is done GLSL would be used for displaying
  render result on a screen.

  This is so because of the way renderer updates parts
  of the image -- it happens without respect to active
  render layer in image user. This is harmless because
  only display buffer is modifying, but this is tricky
  because we don't have original buffer opened during
  rendering.

  One more related fix here was about when rendering
  multiple layers, wrong image would be displaying when
  rendering is done. Added a signal to invalidate
  display buffer once rendering is done (only happens
  when using multiple layers). This solves issue with
  wrong buffer stuck on the display when using regular
  CPU display space transform and if GLSL is available
  it'll make image displayed with a GLSL shader.

- As an additional change, byte buffers now also uses
  GLSL display transform.

  So now only dutehr and RGB curves are stoppers for
  using GLSL for all kind of display transforms.

1 files changed, 12 insertions, 4 deletions

diff --git a/source/blender/imbuf/IMB_colormanagement.h b/source/blender/imbuf/IMB_colormanagement.h
index a758cfa04f0..203d01cab67 100644
--- a/source/blender/imbuf/IMB_colormanagement.h
+++ b/source/blender/imbuf/IMB_colormanagement.h
@@ -152,14 +152,22 @@ void IMB_colormanagement_processor_free(struct ColormanageProcessor *cm_processo
 
 /* Configures GLSL shader for conversion from scene linear to display space */
 int IMB_colormanagement_setup_glsl_draw(const struct ColorManagedViewSettings *view_settings,
-                                        const struct ColorManagedDisplaySettings *display_settings);
-/* Same as above, but color management settings are guessing from a given context */
-int IMB_colormanagement_setup_glsl_draw_from_ctx(const struct bContext *C);
+                                        const struct ColorManagedDisplaySettings *display_settings,
+                                        int predivide);
+/* Same as above, but display space conversion happens from a specified space */
+int IMB_colormanagement_setup_glsl_draw_from_space(const struct ColorManagedViewSettings *view_settings,
+                                                   const struct ColorManagedDisplaySettings *display_settings,
+                                                   struct ColorSpace *colorspace,
+                                                   int predivide);
+/* Same as setup_glsl_draw, but color management settings are guessing from a given context */
+int IMB_colormanagement_setup_glsl_draw_ctx(const struct bContext *C, int predivide);
+/* Same as setup_glsl_draw_from_space, but color management settings are guessing from a given context */
+int IMB_colormanagement_setup_glsl_draw_from_space_ctx(const struct bContext *C, struct ColorSpace *colorspace, int predivide);
 /* Finish GLSL-based display space conversion */
 void IMB_colormanagement_finish_glsl_draw(void);
 
 /* Configures GLSL shader for conversion from space defined by role to scene linear space */
-int IMB_colormanagement_setup_transform_from_role_glsl(int role);
+int IMB_colormanagement_setup_transform_from_role_glsl(int role, int predivide);
 /* Finish GLSL-based color space conversion */
 void IMB_colormanagement_finish_glsl_transform(void);