How to manipulate texture content on the fly? How to manipulate texture content on the fly? ios ios

How to manipulate texture content on the fly?


There are at least two fundamentally different approaches:

1. Update pixels (i assume this is what you mean in the question)

The most effective technique to change the pixels in the texture is called Render-to-Texture and can be done in OpenGL/OpenGL ES via FBOs. On desktop OpenGL you can use pixel buffer objects (PBOs) to manipulate pixel data directly on GPU (but OpenGL ES does not support this yet).

On unextended OpenGL you can change the pixels in system memory and then update texture with glTexImage2D/glTexSubImage2D - but this is inefficient last resort solution and should be avoided if possible. glTexSubImage2D is usually much faster since it only updates pixel inside the existing texture, while glTexImage2D creates entirely new texture (as a benefit you can change the size and pixel format of the texture). On the other side, glTexSubImage2D allows to update only parts of the texture.

You say that you want it to work with OpenGL ES, so I would propose to do the following steps:

  • replace glTexImage2D() with glTexSubImage2D() - if you gain enough performance that's it, just let it be;
  • implement render-to-texture with FBOs and shaders - it will requirefar more work to rewrite your code, but will give even betterperformance.

For FBOs the code can look like this:

// setup FBOglGenFramebuffers( 1, &FFrameBuffer );glBindFramebuffer( GL_FRAMEBUFFER, FFrameBuffer );glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, YourTextureID, 0 );glBindFramebuffer( GL_FRAMEBUFFER, 0 );// render to FBOglBindFramebuffer( GL_FRAMEBUFFER, FFrameBuffer );glViewport( 0, 0, YourTextureWidth, YourTextureHeight );your rendering code goes here - it will draw directly into the textureglBindFramebuffer( GL_FRAMEBUFFER, 0 );// cleanupglBindFramebuffer( GL_FRAMEBUFFER, 0 );glDeleteFramebuffers( 1, &FFrameBuffer );

Keep in mind that not all pixel formats can be rendered to. RGB/RGBA are usually fine.

2. Update geometry

You can also change the geometry of the object your texture is mapped on. The geometry should be tesselated enough to allow smooth interaction and prevent artifacts to appear. The deformation of geometry can be done via different methods: parametric surfaces, NURBS, patches.


I have tried two things that might solve your problem. The methods are pretty different, so I suppose your specific usage case will determine which one is appropriate (if any).

First I've done image deformation with geometry. That is, mapping my texture to a subdivided grid and then overlaying this grid with another grid of bezier control points. The user will then move those control points and thus deforming the vertices of the rendered geometry in a smooth manner.

The second method is more similar to what you outline in your question. When creating your texture keep that data around, so you can manipulate the pixels directly there. Then call something like

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageData);

every time you draw. This might be terribly inefficient, since it likely involves re-uploading all the texture data every frame and I'd love to hear if there's a better way to do it - like directly manipulating the texture data on the GPU. In my case, though the performance is adequate.

I hope this helps, even though it's pretty low on detail.


Modifying the texture render target using FBO is tricky, but pretty straighforward.

So, we have:

  1. TW by TH offscreen buffer (associated with the texture) - we call this Dest
  2. New "array of pixels", the IW by IH texture - we call this Src
  3. The desire to put (IW x IH) texture in position (TX,TY) to the Dest texture

The trick to "put" Src to Dest is to

  1. Bind the generated FBO as a render target
  2. Use a dummy 4-vertex quad with trivial vertex coords (TX,TY), (TX+IW,Y), (TX+IW,TY+IH), (TX,TY+IH) and texture coordinates (0,0), (1,0), (1,1), (0,1)
  3. Bind a trivial pixel shader which reads Src texture bound to the first Texture unit and outputs it on the "screen" (a.k.a. render target, Dest)
  4. Render the quad
  5. Unbind the FBO

For the Src to be rendered correctly you have to use orthographic projection and identity camera transform.

The (TX,TY) and (IW,IH) coordinates in my 4-step solution must be divided by TW and TH respectively to get mapped correctly to the [0..1, 0..1] framebuffer size. To avoid these divisions in shader you can just use the appropriate Orthographic projection for [0..TW, 0..TH] viewport.

Hope this solves problems with FBOs.