Awesome

Computer Graphics Resources

This is a curated list of computer graphics tutorials and resources.

• Formative papers in the field
• Introductory resources / tutorials
• Groundbreaking research / state-of-art techniques
• Software packages and tools

This document is a work in progress - some sections have received a lot less love than others and I intend to correct that.

Please suggest papers/articles/resources through Github pull requests. If you believe this list is missing something or has factually inaccurate info, you can also file an issue in the issue tracker.

Contents

• New to Graphics?
• Other Collections
• Course Notes
• Textbooks
• Physically-based (Photorealistic) Rendering
  • Overview and Survey
  • Ray Tracing
  • Vanilla Path Tracing
  • Photon Mapping
  • Bidirectional Path Tracing
  • Metropolis Light Transport
  • Importance Sampling
  • BRDF/BSDF/BSSRDF
  • Volume Rendering
  • Light Field Capture
  • Microfacet Models
  • GPU
  • Hybrid Strategies & Miscellaneous
• 3D Games
• Non-Photorealistic Rendering
• Shader Art
• Fluids and Simulation
• Differential Geometry
• Three Dimensional Displays
• Meshes
• Image-based Editing and Reconstruction
• Virtual Reality
• Computational Photography

Computer Graphics (CG) is a subfield of Computer Science pertaining to "making images with computers".

Fruits of computer graphics:

CG is ubiquitous and highly interdisciplinary; producing just 5 seconds of a Pixar film will utilize techniques from these areas of computer science and mathematics:

• Data Structures and Algorithms
• Numerical Optimization
• Hardware and Systems
• Distributed and High-Performance Computing
• Robotics
• Finite Element Methods
• Fluid Dynamics
• Machine Learning
• Differential Geometry
• Topology
• Statistics
• Optics
• Signal Processing
• Nuclear Fusion (!!)
• Trigonometry (!!)

Exciting stuff. But if you're just getting started, this list can be overwhelming! Where to begin?

1. Make sure you have an introductory background in basic programming, algorithms, and data structures (such as a semester-long introductory CS course). Brush up on your trigonometry (sines, cosines, triangles, projection of vectors and planes).

2. You can play with computer graphics in whatever language you are most comfortable with. Processing or Three.js (JavaScript) are the most painless languages to get your hands dirty with graphics. If you come from an artistic background, many familiar tools (Maya, Photoshop, Houdini) have scripting interfaces that let you build things procedurally.

3. Spend some time learning tools for CG artists (e.g. Maya, Houdini, Photoshop, Blender, ZBrush). Play some Halo 5 and admire how much geometry is in the scene. These will give you a strong intuition of the capabilities and limitations of CG today.

4. Textbooks and notes from university-level graphics courses are a good resource for learning more math-heavy concepts like physically-based rendering and geometry processing.

5. Remember to have fun! There are no rules.

Many items in the list shown here were consolidated from other reading lists, maintained by people who know a lot more graphics than I do. Please check them out!

• Cornell CS663 Reading List
• Ke-Sen Huang's Website
• extremeistan's realtime GI collection
• realtimerendering.com
• Generative Art Links at hvidtfeldts.net

• Stanford CS348b: Image Synthesis
• CS6630 Cornell University - Realistic Image Synthesis
• http://www.cs.cornell.edu/Courses/cs6630/2012sp/schedule.stm
• CMU 15-462/662

• Principles of Digital Image Synthesis
  • Graphics pipelines, CUDA, path tracing are for today, but physics is forever.
• Computer Graphics: Principles and Practice, 3rd Edition, (Hughes 2013)
  • Authoritative computer graphics reference for students and practitioners. Well-written and expansive in both breadth and depth.
• Physically Based Rendering: From Theory to Implementation, 2nd Edition (Pharr 2010)
  • THE Rendering Book. It also won an Academy Award at the Oscars!
• GPU Gems 3, by NVIDIA

Since Jim Kajiya's 1986 paper on "The Rendering Equation", the vast majority of renderers compute images by simulating the physics of light transport in the scene. Within Physically-based Rendering (PBR), there are 2 open challenges: (1) Render as accurately as possible, and (2) Render as fast as possible

PBR is sometimes used interchangeably with "Global Illumination" in literature, since light scattering is coupled to the shading model. In the end, pictures are made by photons moving into a camera, and nothing more.

• PBRT book (see above)
• Robust Monte Carlo Methods for Light Transport Simulation. Eric Veach. PhD Thesis, Stanford University, 1997.
  • Nearly 20 years later, this monster thesis is still relevant when it comes to developing rendering algorithms. Introduces Monte Carlo rendering methods, multiple importance sampling, bidirectional path tracing, Metropolis Light Transport.
• Analytic Methods for Simulated Light Transport. James Avro. PhD Thesis, Yale University, December 1995

Before physically-based rendering theory was developed, 3D rendering was mostly a big bag of tricks that was raytracing. However, raytracing is still widely used today in production films and games, so it's still important to understand.

• Realistic Raytracing, by Zack Waters

We want to estimate the path integral of irradiance arriving at the sensor (eye) in the scene. In a (basic) path tracer, we sample paths by tracing them from the eye into the scene. A sampled path has nonzero radiance if it eventually touches an emitter.

• The Rendering Equation James T. Kajiya. SIGGRAPH 1986.
• Simple PathTracing by Inigo Quilez
• smallpt (pathtracer in 99 lines of code) by Kevin Beason
  • Check out the various ports and extensions at the bottom of the page!

Path tracers converge slowly if the light source is small. Instead, we trace paths from the light source into the scene, and store where photons land. Then, we can use a naive raytracer to simply "gather" these photons at render time. At the expense of extra storage, it's easy to do realtime dynamic viewpoints.

• Global illumination using photon maps. Jensen, H. W. Rendering Techniques 1996.
• Photon Mapping by Zack Waters

Basic idea: combine eye->light tracing and light->eye tracing to increase convergence speed and reduce noise.

• Bidirectional Estimators for Light Transport Veach & Guibas. Proceedings of EGRW 1994.
• [Mathematical Models and Monte Carlo Algorithms for Physically Based Rendering] Lafortune. PhD Thesis, Katholieke Universiteit Leuven, February 1996.

MCMC sampling for light paths.

• Metropolis light transport Veach & Guibas. SIGGRAPH 1997.
• Mitsuba Renderer MLT is notoriously difficult to implement.
• Peter Kutz's BPT+MLT Blog

Similar to photon mapping. Light paths from the light sources are constructed, and hits are converted into point lights (VPLs). Then do multiple passes of raytracing and accumulate contributions from these point lights.

• Instant Radiosity Keller, A. SIGGRAPH 1997.
  • Cornell CS663 Presentation on the Paper

When doing Monte Carlo integration, samples with value 0 are wasted computation. Instead, we want to use what we know about the scene to only sample nonzero path integral samples. There are lots of options - we can importance sample in image-gradient-domain, the BSDF, and even the light field g=L(x) itself.

• Importance Sampling for Production Rendering. Siggraph 2010.
• Monte Carlo Techniques for Direct Lighting Calculations Shirley et al. ACM Transactions on Graphics 1996.
• Optimally Combining Sampling Techniques for Monte Carlo Rendering Veach & Guibas. SIGGRAPH 1995.
• Gradient-domain metropolis light transport. Lehtinen et al. ACM Transactions on Graphics 2013.
• Gradient-domain Path Tracing. Kettunen et al 2015.
• A Machine Learning Approach for Filtering Monte Carlo Noise. SIGGRAPH 2015.

Different materials (metal, wood, skin, leggings) interact with light in different ways due to material properties and geometric differences at a microscopic level. Shading models attempt to capture these behaviors across different materials.

• Chapter 15 of "Principles of Digital Image Synthesis" (Morgan-Kaufman 1995)
• Geometrical Considerations and Nomenclature for Reflectance. National Bureau of Standards. Nicodemus et al. (U.S.) monograph, issued October 1977.
• A microfacet-based BRDF generator. Ashikhmin and Shirley. SIGGRAPH 2000.
• A precomputed polynomial representation for interactive BRDF editing with global illumination Ben-Artzi et al. ACM Transactions on Graphics, 2008.
• A Reflectance Model for Computer Graphics. Cook & Torrance. Computer Graphics. 14(3), pp. 307-316. 1981.
  • Cook-Torrance BRDF model.
• Theory for Off-specular Reflection from Roughened Surfaces. Torrance & Sparrow. Journal of the Optical Society of America 57:9 (1967).
  • Torrance-Sparrow shading model.
• Models of Light Reflection for Computer Synthesized Pictures - James F. Blinn, SIGGRAPH 1977.
  • This paper reviews Torrance-Sparrow, Phong model, and proposes the Blinn-Phong shading model as a computationally efficient alternative.
• Shadowing by Non-Gaussian Random Surfaces. Garry S. Brown, IEEE Transactions on Antennas and Propagaion (1980).
• A data-driven reflectance model Matusik et al. SIGGRAPH 2003.
• Experimental analysis of BRDF models. Ngan et al. Proceedings of the 2005 Eurographics Symposium on Rendering.
• A model for anisotropic reflection. Poulin & Fournier. SIGGRAPH 1990.
• Generalization of the Lambertian Model and Implications for Machine Vision. Oren & Nayar. International Journal of Computer Vision 14:3 (1995).
  • Introduces the Oren-Nayar reflectance model.
• Bidirectional Reflection Distribution Function of Thoroughly Pitted Surfaces. Koenderink et al. International Journal of Computer Vision 31 (1999).
  • BRDF of surfaces that are rough at both macro and micro scale.

• Chapter 12 (Energy Transport) of Principles of Digital Image Synthesis.
  • Easy explanation of the volume rendering equation from the ground up.
• Volume Rendering Techniques. Ikits et al. GPU Gems, Pearson Education, 2004.
• A Survey on Participating Media Rendering Techniques. Cerezo et al. Visual Computer, 2005
• A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure
  • Scattering models usually assume isotropic media. This paper presents a scattering model that supports anisotropic scattering (hair, cloth, skin).
  • Contains nice derivations of the standard isotropic case as well, for the diffusion approximation and the dipole BSSRDF.
• Interactive multiple anisotropic scattering in clouds. Bouthors et al. Proceedings of I3D 2008.
• Unifying points, beams, and paths in volumetric light transport simulation. Křivánek et al. SIGGRAPH 2014.
  • The results are SUUUUPER beautiful
• Scalable and Heterogeneous Rendering of Subsurface Scattering Materials Arbree. PhD Thesis, Cornell University, 2009.
  • Thesis on rendering translucent materials, derviation of dipole approximation.
• Virtual ray lights for rendering scenes with participating media. Novak et al. 2012 SIGGRAPH 2012.
• Ray tracing volume densities. Kajiya & von Herzen.SIGGRAPH 1984, July 1984.
• Semi-Automatic Generation of Transfer Functions for Direct Volume Rendering. Kindlmann & Durkin. IEEE Symposium on Volume Visualization 1998.
• GigaVoxels: ray-guided streaming for efficient and detailed voxel rendering. Crassin et al. I3D 2009.
• Reflection from Layered Surfaces Due to Subsurface Scattering. Hanrahan & Krueger. SIGGRAPH 1993.
• Wave Propagation and Scattering in Random Media. Akira Ishimaru. Chapter 7 in Wave Propagation and Scattering in Random Media, Volume 1. Academic Press, 1978.
• A model for volume lighting and modeling. Kniss et al. IEEE TVCG 9:2 (2003).
• Fast Volume Rendering Using a Shear-Warp Factorization of the Viewing Transformation. Lacroute & Levoy. SIGGRAPH 1994.
• Efficient Simulation of Light Transport in Scene with Participating Media using Photon Maps. Jensen & Christensen. Proceedings of SIGGRAPH 98.
• A Practical Model for Subsurface Light Transport. Jensen et all. Proceedings of ACM SIGGRAPH 2001.
• A Rapid Hierarchical Rendering Technique for Translucent Materials. Jensen & Buhler. SIGGRAPH 2002.
• Multidimensional Transfer Functions for Interactive Volume Rendering. Kniss et al. IEEE TVGG 2002.
• Practical Rendering of Multiple Scattering Effects in Participating Media. Premoze et al. Rendering Techniques 2004: 15th Eurographics Workshop on Rendering.
• Physically-based simulation of rainbows. Sadeghi et al. ACM Transactions on Graphics, 31(1), (2012).

Much camera. Very science. Wow.

• Paul Debevec's Home Page
  • Lots of great publications, resources, and free HDRI maps!
• Acquiring the reflectance field of a human face. Debevec et al. SIGGRAPH 2000.
• A Dual Light Stage. Hawkins et al. Proceedings of EGSR 2005.
• Fast bilateral filtering for the display of high-dynamic-range images. Durand & Dorsey. SIGGRAPH 2002.
• DISCO: acquisition of translucent objects. Goesele et al. ACM Transactions on Graphics. 23(3), pp. 835-844, 2004.
• The Lumigraph. Gortler et al. Siggraph 1996.
• Femto-Photography: Visualizing Photons in Motion at a Trillion Frames Per Second. Raskar et al.
• Gradient domain high dynamic range compression. Fattal et al. ACM SIGGRAPH 2002.

Typically used to render high-frequency spatial information, like the knitting of cloth or the imperfections of skin.

• Microfacet Models for Refraction through Rough Surfaces. Walter et al. Eurographics Symposium on Rendering 2007.

• Building Volumetric Appearance Models of Fabric using Micro CT Imaging. Zhao et al. ACM Transactions on Graphics (SIGGRAPH 2011).

• Structure-aware Synthesis for Predictive Woven Fabric Appearance. Zhao eet al. SIGGRAPH 2012.

• Efficient Rendering of Human Skin. d'Eon et al. Eurographics Symposium on Rendering 2007.

TLDR: GPUs make everything better. But they are hard to program.

• Understanding the Efficiency of Ray Traversal on GPUs. Timo Aila and Samuli Laine. Proc. High-Performance Graphics 2009
  • Kepler and Fermi architecture Addendum. NVIDIA Technical Report 2012.

Modern and proprietary commercial renderers probably implement a combination of techniques (like MLT + BPT + PT or MLT + Photon Mapping + Radiosity).

• Energy Redistribution Path Tracing. Cline et al. 2005. ACM Transactions on Graphics (TOG), 24(3) (2005).
  • Combines MLT with basic path tracing.
  • Implementation Details
• Lightcuts: a scalable approach to illumination Walter et al. ACM Transactions on Graphics 2005.
  • Render time usually scales linearly with number of lights. This method provides sublinear scaling cost.
• Sorted Deferred Shading for Production Path Tracing. Eisenacher et al. EGSR 2013.
  • One of the secret sauces in Disney's Hyperion Renderer: rays are sorted by direction to improve cache locality of scene traversal?
• Light transport simulation with vertex connection and merging. Georgiev et al. SIGGRAPH 2012.
• Reconstructing the indirect light field for global illumination Lehtinen et al. SIGGRAPH 2012.
• Global illumination with radiance regression functions (SIGGAPH 2013)
  • Using neural networks to predict the indirect light field.
  • video
• Temporal light field reconstruction for rendering distribution effects. Lehtinen et al. SIGGRAPH 2011.
• The Secret Life of Photons Light transport tutorial

See Ke-Sen Huang's paper collection of i3d papers (Symposium on Interactive 3D Graphics and Games).

• Tutorial on Spherical Harmonics (2003)
• Interactive Display of Isosurfaces with Global Illumination. Wyman et al. IEEE TVGG 2006.
• John Carmack's Fast Inverse Square Root

• Painterly Rendering for Animation Meier, Barbara. Disney Animation Research, 1996.
• Stylizing Animation by Example. Pixar Animation. 2013.
• Coherent Noise for Non-Photorealistic Rendering

Turns out you can do quite a lot of graphics using only a quad and an OpenGL fragment shader.

• The Book of Shaders by Patricio Gonzalez Vivo
  • The book appears to have been left unfinished, but what has been written so far is pretty good introductory material.
• ShaderToy A big collection of awesome fragment shaders. Many of these use cool tricks to do cheap lighting and imgae-space effects like blur/vignetting.
• Articles by Inigo Quilez
• Generative Art Links

• A material point method for snow simulation
  • an outline of the techniques used in the snow simulation for Disney's Frozen. video demo
  • Since the publication of this paper, Disney appears to invest quite heavily in MPM methods.
• Augmented MPM for phase-change and varied materials . Stomakhin et al. SIGGRAPH 2014.
• Asynchronous Contact Mechanics. Harmon et al. ACM communications 2012.
  • Another active area of research for Disney.

• Keenan Crane's Homepage
  • Excellent Youtube Talk
• Globally Optimal Direction Fields. Knöppel et al. SIGGRAPH 2014.

Another dream of computer graphics:Iron-Man -styled Holograms.

• Three-Dimensional Display Technologies: a Survey. 2013.
• Pixie Dust: Graphics Generated by Levitated and Animated Objects in Computational Acoustic-Potential Field
• Three Dimensional Images in the Air
  • TLDR: Focus laser beam, plasma emission phenomenon produces a burst of light at the focal point. Doing this at a high-enough frequency allows rasterization of plasma dots. Probably not safe to touch.

• Mean Value Coordinates for Closed Triangular Meshes. Ju, Schaefer, Warren. ACM Trans. on Graphics 2005.
• Feature Adaptive GPU Rendering of Catmull-Clark Subdivision Surfaces

• Recovering high dynamic range radiance maps from photographs Debevec & Malik 1997. In SIGGRAPH 97 (August 1997)
• Accurate, Dense, and Robust Multi-View Stereopsis. Furukawa and Ponce 2010.
  • Yasutaka Furukawa
• Structure-aware Hair Capture. Luo, Li, Rusinkiewicz. Siggraph 2013.
• Interactive digital photomontage Agarwala et al. ACM Trans. on Graphics, 23(3):294--302, Aug. 2004
• [Image-based reconstruction of spatial appearance and geometric detail](Image-based reconstruction of spatial appearance and geometric detail). Lensch et all. ACM Transactions on Graphics 2003.
• Light field rendering. Levoy & Hanrahan. Siggraph 1996.
• Photo tourism: Exploring photo collections in 3D. Snavely, Seitz, Szzeliski. Siggraph 2006.
• First Person Hyperlapse Videos Microsoft Research 2014.
• PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing. Barnes et al. Siggraph 2009.
• Efficient Gradient-Domain Compositing Using Quadtrees. ACM Trans. Graph. 2007

• Oliver Kreylos's blog is a good resource to get started learning about VR / stereo rendering.
  • Good Stereo vs. Bad Stereo
• Implementing Stereoscopic 3D in your applications
• John Carmack's Oculus Connect 2015 talk
• Why You Won’t See Hard AR Anytime Soon

• CAVE Laboratory at Columbia