JavaScript Canvas WebGL Convolutions
Written by Ian Elliot   
Monday, 18 May 2020
Article Index
JavaScript Canvas WebGL Convolutions
Texture Coordinates

Canvas in WebGL mode provides the ability to perform fast pixel operations. In this extract from a chapter in my new book on JavaScript Graphics we look at how to use the GPU to implement a convolution

Now available as a paperback or ebook from Amazon.

JavaScript Bitmap Graphics
With Canvas

largecover360

 

Contents

  1. JavaScript Graphics
  2. Getting Started With Canvas
  3. Drawing Paths
      Extract: Basic Paths
      Extract: SVG Paths
      Extract: Bezier Curves
  4. Stroke and Fill
      Extract: Stroke Properties 
      Extract: Fill and Holes
      Extract: Gradient & Pattern Fills
  5. Transformations
      Extract: Transformations
      Extract: Custom Coordinates 
      Extract  Graphics State
  6. Text
      Extract: Text, Typography & SVG 
      Extract: Unicode
  7. Clipping, Compositing and Effects
      Extract: Clipping & Basic Compositing
  8. Generating Bitmaps
      Extract:  Introduction To Bitmaps
      Extract :  Animation 
  9. WebWorkers & OffscreenCanvas
      Extract: Web Workers
      Extract: OffscreenCanvas
  10. Bit Manipulation In JavaScript
      Extract: Bit Manipulation
  11. Typed Arrays
      Extract: Typed Arrays **NEW!
  12. Files, blobs, URLs & Fetch
      Extract: Blobs & Files
      Extract: Read/Writing Local Files
  13. Image Processing
      Extract: ImageData
      Extract:The Filter API
  14. 3D WebGL
      Extract: WebGL 3D
  15. 2D WebGL
    Extract: WebGL Convolutions

<ASIN:1871962625>

<ASIN:B07XJQDS4Z>

<ASIN:1871962579>

<ASIN:1871962560>

<ASIN:1871962501>

<ASIN:1871962528>

Earlier in the chapter we looked at how 2D graphics could be implemented in WebGL. Not only can you draw new things you can also process bitmaps and in this extract the focus is on how to handle bitmaps in a GPU and on implementing a convolution which is a very basic filtering operation.

Included in the chapter but not in this extract are:

  • 2D WebGL
  • A 2D Vertex Shader
  • Transformations
  • Triangles
  • Rectangles
  • Circles - Triangle Fan
  • Multiple Objects
  • Animation

 

Bitmaps in WebGL

Although WebGL works in terms of vertices and triangles it is possible to load and work with bitmaps. This ability is used in 3D to apply texture to 3D solids by mapping bitmaps to each of the faces. In our case we are going to make use of texture bitmaps as bitmaps, but you should be able to see how to apply the same ideas to 3D texture mapping.

There are some key new ideas and one of them is a varying. This is a variable that is used by the fragment shader to determine the property of a pixel that isn’t positioned at one of the vertices of the triangle. The idea is simply that a varying takes a value that is a weighted average of the values at the vertices that surround the pixel. The weights are simply the distances to each of the vertices. So a varying that is halfway between two vertices will have the average of the value at each vertex. More generally the pixel shown in the diagram below will have a value that is the average of the values at A, B and C weighted by the distance from each of the vertices.

shader

The most common example of a varying is assigning a color to each of the three points and then allowing the fragment shader to color the interior of the fragment in a gradient fill. A more usual practice use is to obtain a co-ordinate for the pixel based on the co-ordinates assigned to the vertices.

For example, if we have a right-angled triangle and assign the co-ordinates as shown, then the pixel equidistant from each vertex has a value (0.5,0.5) and pixels at intermediate positions have a proportional co-ordinate. This is the basic idea of a texture co-ordinate. Notice that the co-ordinates assigned to the vertices are not their actual position.

You can assign texture co-ordinates as you please as they are only used to provide the pixels within the triangle co-ordinates. To keep the distinction between co-ordinates that fix the position of the vertices and assigned texture co-ordinates we usually use x,y for position and u,v for texture co-ordinates.

triangle

What has all this got to do with bitmaps? The answer is that we are going to load a bitmap, usually referred to as a texture map, into the GPU. A GPU bitmap has a standard co-ordinate system (0,0) to (1,1)

text

By applying texture co‑ordinates to the vertices of a fragment, we can sample pixels within the texture bitmap to set the color of the fragment’s pixel. To do this we need new shaders. The vertex shader becomes:

var vsScript = 
 `attribute vec2 a_texCoord;
  varying vec2 v_texCoord;
  attribute vec2 vertexPosition;
  uniform mat3 transform;
  void main(void) {
    vec2 temp= vec2(transform*vec3(vertexPosition,1.0));
           gl_Position = vec4(temp,0.0,1.0);
           v_texCoord = a_texCoord;
          }`;

The new features are the two additional texCoord variables, one an attribute and one a varying. At the end of the shader, the attribute is passed to the varying which is automatically passed to the fragment shader. Vertex shaders can read and write varyings and they are automatically passed to the fragment shader where they are read only:

var fsScript = 
`precision mediump float; uniform vec4 f_color; uniform sampler2D u_image; varying vec2 v_texCoord; void main(void) { gl_FragColor = texture2D(u_image, v_texCoord); }`;

You can see that in the fragment shader we now have an image variable and the textCoord variable which is passed in from the vertex shader. The varying changes its value according to the position of the pixel being processed and the final line samples the color of the pixel in the bitmap at the specified texture co-ordinate.

To make this work we now need to load the bitmap into the GPU and set the texture co-ordinates on a suitable set of vertices. Loading the image into the GPU is a standard operation:

function loadBitmap(gl,img) {        
  var texture = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, texture);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S,
                                     gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T,
                                     gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, 
gl.LINEAR); gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img); }

The first two instructions create the texture. The next three set how it will be handled if it is sampled at different resolutions and outside of its borders. The final instruction associates the image object holding the bitmap with the texture. You can see that you have to specify its color format.



Last Updated ( Tuesday, 19 May 2020 )