st/mesa: fix accum buffer allocation in st_renderbuffer_alloc_storage()

If the gallium driver doesn't support PIPE_FORMAT_R16G16B16A16_SNORM
the call to st_choose_renderbuffer_format() would fail and we'd generate
an GL_OUT_OF_MEMORY error.  We'd never get to the subsequent code that
handles software/malloc-based renderbuffers.

Add a special-case check for PIPE_FORMAT_R16G16B16A16_SNORM which is used
for software-based accum buffers.  This could be fixed in other ways but
it would be a much larger patch.  st_renderbuffer_alloc_storage() could
be reorganized in the future.

This fixes accum buffer allocation for the svga driver.

Note: This is a candidate for the 7.11 branch.

Reviewed-by: José Fonseca <jfonseca@vmware.com>

diff --git a/src/mesa/state_tracker/st_cb_fbo.c b/src/mesa/state_tracker/st_cb_fbo.c
index a351d11..1e8ec05 100644 (file)
--- a/src/mesa/state_tracker/st_cb_fbo.c
+++ b/src/mesa/state_tracker/st_cb_fbo.c
@@ -75,8 +75,18 @@ st_renderbuffer_alloc_storage(struct gl_context * ctx,
    enum pipe_format format;
    struct pipe_surface surf_tmpl;
 
-   format = st_choose_renderbuffer_format(screen, internalFormat,
-                                          rb->NumSamples);
+   if (internalFormat == GL_RGBA16_SNORM && strb->software) {
+      /* Special case for software accum buffers.  Otherwise, if the
+       * call to st_choose_renderbuffer_format() fails (because the
+       * driver doesn't support signed 16-bit/channel colors) we'd
+       * just return without allocating the software accum buffer.
+       */
+      format = PIPE_FORMAT_R16G16B16A16_SNORM;
+   }
+   else {
+      format = st_choose_renderbuffer_format(screen, internalFormat,
+                                             rb->NumSamples);
+   }
 
    if (format == PIPE_FORMAT_NONE) {
       return FALSE;