Project 5: Xiaomao Ding

README.md

@@ -3,26 +3,66 @@ WebGL Deferred Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Xiaomao Ding
+* Tested on: Windows 8.1, i7-4700MQ @ 2.40GHz 8.00GB, GT 750M 2047MB (Personal Computer)
 
 ### Live Online
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[![](img/thumb.png)](https://xnieamo.github.io/Project5-WebGL-Deferred-Shading-with-glTF/)
 
 ### Demo Video/GIF
 
-[![](img/video.png)](TODO)
+![](img/Demo.gif)
 
-### (TODO: Your README)
+### Intro
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+This repository implements a WebGL based deferred shader. The concept behind deferred shading is to store the geometry information in a GBuffer. Then, further lighting and post-processing passes are executed on this buffer. This organization of shading will allow us to render multiple lights with a lower performance hit. This is because we no longer need to calculate the geometry for each additional light.  
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+#### Features
+The deferred shader in this repository has the following features:
+* Blinn-Phong shading
+* Toon shading
+* Bloom shading
+* 2x 1D Gaussian Bloom shading optimization
+* GBuffer packing optimizations
 
+### Performance Analysis
+#### GBuffer Optimization
+The GBuffer is where we store information relevant to our geometry during the single pass through the scene. The naive implementation stores the color, position, normal, and normal map for each fragment in our shader. We can first optimize this by calculating the modified normal during the GBuffer copy pass, which reduces the number of vectors we need to store from 4 to 3.  Furthermore, the GBuffers are vec4's but both color and position are only vec3's. Since the normals are normalized, we can be clever and store just 2 values of the normal (calculating the third on the fly).  This allows us to further reduce the number of vec4's we need down to 2. The following is the organization of our buffers, where P stands for position, C for color, and N for normals:
+
+[Px][Py][Pz][Nx]
+
+[Cr][Cg][Cb][Ny]
+
+As expected, reducing the size of the GBuffer improves performance. 
+
+<p align="center">
+  <img src="https://github.com/xnieamo/Project5-WebGL-Deferred-Shading-with-glTF/blob/master/img/GBufferOpt.png?raw=true">
+</p>
+
+#### Shaders
+The Blinn-Phong shader is the default in this repository. It combines a diffuse and specular component of the light color in order to fill in the fragments. The toon shader applies a ramp to the color of the fragments. In this implementation, the ramp is only applied to the diffuse color while a hard threshold is set for the specular. Finally, the bloom shader is a post-processing effect that adds blur to the lights in a scene. The result is that lights in a scene now appear to glow.  This can be achieve by applied a 2D Gaussian filter. Using Multiple Rendering Targets, we create a separate color buffer that contains only the lights, selected as fragments that have a brightness exceeding some threshold. Because this buffer contains only the lights, applying the Gaussian filter will not blur the rest of the scene. This repository contains two implementations of the Gaussian filter. The first implementation calculates the entire filter in 1 shader. The second optimizes the process by applying two shaders, one horizontal and the other vertical. The addition of the post-process step is expected to decrease performance, while the optimization should improve it.
+
+
+<p align="center">
+  <img src="https://github.com/xnieamo/Project5-WebGL-Deferred-Shading-with-glTF/blob/master/img/Shaders.png?raw=true">
+</p>
+
+The Bloom optimizations turn out to not be too great. It could possibly be because the filter is too small (9x9) or perhaps that it is because the first pass now needs to allocate a framebuffer so that its results can be saved and sent to the second pass. Furthermore, it may be due to the fact that I kept the weights the same for the Gaussian blurs, but the 2x 1D pass is slightly dimmer than the 2D pass.
+
+#### Toon Shader
+![](img/toon.PNG)
+
+#### Bloom Post Process
+![](img/Bloom.PNG)
+
+### Debug Views
+#### Depth
+![](img/Depth.PNG)
+#### Normals
+![](img/Normals.PNG)
+#### Albedo
+![](img/Albedo.PNG)
 
 ### Credits
 

glsl/copy.frag.glsl

@@ -3,18 +3,43 @@
 precision highp float;
 precision highp int;
 
+#define NUM_GBUFFERS 2
+
 uniform sampler2D u_colmap;
 uniform sampler2D u_normap;
 
 varying vec3 v_position;
 varying vec3 v_normal;
 varying vec2 v_uv;
 
+vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
+    normap = normap * 2.0 - 1.0;
+    vec3 up = normalize(vec3(0.001, 1, 0.001));
+    vec3 surftan = normalize(cross(geomnor, up));
+    vec3 surfbinor = cross(geomnor, surftan);
+    return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor;
+}
+
 void main() {
     // TODO: copy values into gl_FragData[0], [1], etc.
     // You can use the GLSL texture2D function to access the textures using
     // the UV in v_uv.
+    vec4 color = texture2D(u_colmap, v_uv);
+    vec4 normap = texture2D(u_normap, v_uv);
+
+    vec3 nor = normalize(applyNormalMap(v_normal, normap.xyz));
 
     // this gives you the idea
-    // gl_FragData[0] = vec4( v_position, 1.0 );
+    gl_FragData[0] = vec4( v_position, nor[0] );
+
+#if NUM_GBUFFERS == 2
+    gl_FragData[1] = vec4( color.rgb, nor[1]);
+#elif NUM_GBUFFERS == 3
+    gl_FragData[1] = vec4( nor, nor[1]);
+    gl_FragData[2] = color;
+#elif NUM_GBUFFERS == 4
+    gl_FragData[1] = vec4( v_normal, 0.0);
+    gl_FragData[2] = color;
+    gl_FragData[3] = normap;
+#endif
 }

glsl/deferred/ambient.frag.glsl

@@ -3,25 +3,28 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 2
 
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
 
 varying vec2 v_uv;
 
 void main() {
-    vec4 gb0 = texture2D(u_gbufs[0], v_uv);
-    vec4 gb1 = texture2D(u_gbufs[1], v_uv);
+#if NUM_GBUFFERS == 2
+    vec4 gb2 = texture2D(u_gbufs[1], v_uv);
+#elif NUM_GBUFFERS > 2
     vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+#endif
+
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
+    vec3 ambientColor = gb2.rgb * 0.2;
 
     if (depth == 1.0) {
         gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0
         return;
     }
 
-    gl_FragColor = vec4(0.1, 0.1, 0.1, 1);  // TODO: replace this
+    gl_FragColor = vec4(ambientColor, 1);  // TODO: replace this
 }

glsl/deferred/blinnphong-pointlight.frag.glsl

@@ -1,17 +1,23 @@
 #version 100
+#extension GL_EXT_draw_buffers: enable
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 2
+
+uniform vec3 u_cameraPos;
 
 uniform vec3 u_lightCol;
 uniform vec3 u_lightPos;
 uniform float u_lightRad;
+
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
 
 varying vec2 v_uv;
 
+const vec3 lightThresh = vec3(0.2126, 0.7152, 0.0722);
+
 vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     normap = normap * 2.0 - 1.0;
     vec3 up = normalize(vec3(0.001, 1, 0.001));
@@ -20,20 +26,73 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor;
 }
 
+float diffuseLighting(vec3 normal, vec3 lightDir){
+    return clamp(dot(normal, lightDir), 0.001, 1.0);
+}
+
+float specularLighting(vec3 normal, vec3 lightDir, vec3 viewDir) {
+    vec3 H = normalize(lightDir + viewDir);
+    float specAngle = clamp(dot(normal, H), 0.0, 1.0);
+    return pow(specAngle, 0.01) * 0.01;
+}
+
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
+
+#if NUM_GBUFFERS >= 3
     vec4 gb2 = texture2D(u_gbufs[2], v_uv);
+#endif
+
+#if NUM_GBUFFERS == 4
     vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+#endif
+
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
 
+    vec3 pos = gb0.xyz;     // World-space position
+#if NUM_GBUFFERS == 2
+    vec3 colmap = gb1.rgb;  // The color map - unlit "albedo" (surface color)
+#else
+    vec3 colmap = gb2.rgb;
+#endif
+
+#if NUM_GBUFFERS == 4
+    vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
+    vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
+    vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+#elif NUM_GBUFFERS == 3 
+    vec3 nor = gb1.xyz;
+#elif NUM_GBUFFERS == 2
+    float z = 1.0 - (gb0[3]*gb0[3]) - (gb1[3]*gb1[3]);
+    vec3 nor = vec3(gb0[3], gb1[3], z);
+#endif
+
     // If nothing was rendered to this pixel, set alpha to 0 so that the
     // postprocessing step can render the sky color.
     if (depth == 1.0) {
-        gl_FragColor = vec4(0, 0, 0, 0);
+        gl_FragData[0] = vec4(0, 0, 0, 0);
         return;
     }
 
-    gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+    vec3 lightDir = u_lightPos - pos;
+    float distance = length(lightDir);
+    lightDir = lightDir / distance;
+
+    vec3 viewDir = normalize(u_cameraPos - pos);
+
+
+    float attenuation = max(0.0, u_lightRad - distance);
+
+    float diffuse = diffuseLighting(nor, lightDir);
+    float specular = specularLighting(nor, lightDir, viewDir);
+    vec3 color = colmap * u_lightCol * (diffuse + specular) * attenuation;
+
+    gl_FragData[0] = vec4(color, 1);  // TODO: perform lighting calculations
+
+    // Check for the brightness of this fragment. If it meets some threshold, set it 
+    // to be part of the light.
+    if (dot(color, lightThresh) > 1.0)
+        gl_FragData[1] = vec4(color, 1);
 }

glsl/deferred/debug.frag.glsl

@@ -2,9 +2,10 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 2
 
 uniform int u_debug;
+uniform vec3 u_cameraPos;
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
 
@@ -23,30 +24,52 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
+
+#if NUM_GBUFFERS >= 3
     vec4 gb2 = texture2D(u_gbufs[2], v_uv);
+#endif
+
+#if NUM_GBUFFERS == 4
     vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+#endif
+
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
     // These definitions are suggested for starting out, but you will probably want to change them.
     vec3 pos = gb0.xyz;     // World-space position
+#if NUM_GBUFFERS == 2
+    vec3 colmap = gb1.rgb;  // The color map - unlit "albedo" (surface color)
+#else
+    vec3 colmap = gb2.rgb;
+#endif
+
+#if NUM_GBUFFERS == 4
     vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
-    vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
     vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
     vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+#elif NUM_GBUFFERS == 3 
+    vec3 nor = gb1.xyz;
+    vec3 geomnor = nor;
+    vec3 normap = nor;
+#elif NUM_GBUFFERS == 2
+    vec3 nor = vec3(gb0[3], gb1[3], 1.0 - gb0[3]*gb0[3] - gb1[3]*gb1[3]);
+    vec3 geomnor = nor;
+    vec3 normap = nor;
+#endif
 
     // TODO: uncomment
     if (u_debug == 0) {
         gl_FragColor = vec4(vec3(depth), 1.0);
     } else if (u_debug == 1) {
-        // gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
+        gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
     } else if (u_debug == 2) {
-        // gl_FragColor = vec4(abs(geomnor), 1.0);
+        gl_FragColor = vec4(abs(geomnor), 1.0);
     } else if (u_debug == 3) {
-        // gl_FragColor = vec4(colmap, 1.0);
+        gl_FragColor = vec4(colmap, 1.0);
     } else if (u_debug == 4) {
-        // gl_FragColor = vec4(normap, 1.0);
+        gl_FragColor = vec4(normap, 1.0);
     } else if (u_debug == 5) {
-        // gl_FragColor = vec4(abs(nor), 1.0);
+        gl_FragColor = vec4(abs(nor), 1.0);
     } else {
         gl_FragColor = vec4(1, 0, 1, 1);
     }