Awesome

doc3D-renderer

Doc3D is the first 3D dataset focused on document unwarping with realistic paper warping and renderings.

Instructions to render your own data:

• Step 1: We need a few assets to start the rendering.

  • a) Meshes
  • b) HDR environment maps for lighting
  • c) Textures For your convenience, we already provide some sample meshes (/obj), HDR maps (/env) and textures (/tex). Each line is a path to a mesh, HDR map and a texture.

• Step 2: List the assets in objs.csv,envs.csv, and texs.csv. laval_env.csv is the example file for Laval HDR environment maps, note that the intensity multiplier is set to 100 to avoid dark images.

• Step 3a : Run the rendering code (for images, UVs, 3D coordinates):

  • blender --background --python render_mesh.py -- <folder-id> <start-mesh> <end-mesh> This command renders the images (/img), 3D coordinates (/wc) and UV (/uv) in folder <folder-id>. <start-mesh> and <end-mesh> refers to line numbers in objs.csv specifying the meshes to be used while rendering. Additionally, render_mesh.py also saves the Blender model (.blend) files for further rendering process (albedo, norm etc.). Toggle it using the save_blend_file=False flag in the code.
  • For faster rendering you can use the multiprocessing code batch_render.py. Run python batch_render.py <folder-id> <start-mesh> <end-mesh>. Remember to modify the processor counts using nproc.

• Step 3b : Run the rendering code (for checkerboards, albedos, depth etc.):

  • Run blendnames.py to list the available .blend files, python blendnames.py <folder-id>. Remember to do this step!
  • Albedos (/alb): blender --background --python render_alb.py -- <folder-id> <start-mesh> <end-mesh>
  • Normals (/norm): blender --background --python render_norm.py -- <folder-id> <start-mesh> <end-mesh>
  • Depths (/dmap): blender --background --python render_dmap.py -- <folder-id> <start-mesh> <end-mesh>
  • Checkerboard (/recon): blender --background --python render_recon.py -- <folder-id> <start-mesh> <end-mesh>

• Step 4: If you want to create the backward mappings from UV:

  • /uv2backwardmap contains the necessary scripts. We use MatLab to do this.
  • python exr2mat.py <folder-id>, converts the .exr files to .mat files.
  • Edit the src_dir and dst_dir accordingly and run fm2bm.m.

Citation:

If you use the dataset or this code, please consider citing our work-

@inproceedings{SagnikKeICCV2019, 
Author = {Sagnik Das*, Ke Ma*, Zhixin Shu, Dimitris Samaras, Roy Shilkrot}, 
Booktitle = {Proceedings of International Conference on Computer Vision}, 
Title = {DewarpNet: Single-Image Document Unwarping With Stacked 3D and 2D Regression Networks}, 
Year = {2019}}   

Acknowlegement:

Thanks to the awesome software, Blender, thanks to it's developers and also to the super awesome community.