3D Sketch Rendering

The 3D sketch rendering module converts Sketch3D Bézier curves into 2D raster images from one or more camera views.

This page documents the implementation in sketchkit/renderer3d and a brief practical usage introduction in tests/unit/test_3Dsketch.py.

Source: sketchkit/renderer3d/renderer3d.py, sketchkit/renderer3d/diffvg_renderer.py

Available Methods

sketchkit.renderer3d.DiffVGRenderer([...])

Project and render 3D sketch curves with selectable renderer backend.

DiffVGRenderer

DiffVGRenderer is the main 3D-to-2D renderer in SketchKit for multi-view sketch visualization.

• Input: a Sketch3D object and one or more Camera views.

• Pipeline:

  1. project 3D Bézier control points into 2D view space,

  2. resolve stroke style (color and width),

  3. rasterize with DiffVG,

  4. return rendered PIL.Image frames.

• Output: a list of RGB images, one image per camera view.

Code

import numpy as np
import torch

from sketchkit.core.camera import Camera
from sketchkit.core.sketch3d import Sketch3D
from sketchkit.renderer3d import DiffVGRenderer
from sketchkit.renderer3d.diffvg_renderer import DiffVGRenderOptions
from sketchkit.utils3d.transforms import wxyz_from_look_at


# Build orbit cameras for a multi-view render test.
# This follows the same idea as tests/test_3Dsketch.py.
def build_orbit_cameras(num_views: int = 60, radius: float = 1.0):
    cameras = []
    for az in np.linspace(0.0, 2.0 * np.pi, num_views, endpoint=False):
        xyz = np.array([radius * np.cos(az), 0.0, radius * np.sin(az)])
        cam = Camera()
        cam.set_xyz(xyz)
        cam.set_wxyz(
            wxyz_from_look_at(
                xyz, np.array([0.0, 0.0, 0.0]), up=np.array([0.0, 1.0, 0.0])
            )
        )
        cam.set_fov(fov=np.deg2rad(60.0))
        cameras.append(cam)
    return cameras


# Load a predefined 3D sketch from JSON.
# In practice, this file can be generated by Sketch3D.to_json(...).
sketch = Sketch3D.from_json("./sketch3d_test_out/3d_sketch_example.json")
cameras = build_orbit_cameras()

# Keep stroke_width=None to let curve-level widths work (including variable widths).
opts = DiffVGRenderOptions(canvas_size=(512, 512), stroke_width=None)
# Typical device selection for local testing.
opts.device = "cuda" if torch.cuda.is_available() else "cpu"

renderer = DiffVGRenderer(render_options=opts)

# Test case 1: single-view smoke test.
single_view = renderer.render(sketch, cameras=[cameras[0]], render_options=opts)[0]
single_view.save("single_view.png")

# Test case 2: multi-view orbit render (used to make GIF/video in tests).
images = renderer.render(sketch, cameras=cameras, render_options=opts)
# Save one representative frame from the orbit sequence.
images[0].save("first_view.png")

Usage Notes

Constructor

• DiffVGRenderer(render_options=None)

  • render_options: optional DiffVGRenderOptions object.

  • Throws RuntimeError if pydiffvg is not installed.

Render Options

• Render3DOptions

  • canvas_size: output image size (width, height).

  • background_color: RGB background color.

  • stroke_color: optional global RGB stroke override.

  • stroke_width: optional global stroke width override.

• DiffVGRenderOptions(Render3DOptions)

  • device (str): execution device, e.g., "cpu" or "cuda".

Render Method

• render(sketch3d, cameras, render_options=None)

  • sketch3d: input Sketch3D object.

  • cameras: list of Camera objects (one image is generated per camera).

  • render_options: optional dataclass or dict override.

  • Returns: list[PIL.Image.Image].

Sketch3D

Sketch3D is the geometric container consumed by the 3D renderer.

• Hierarchy

  • Sketch3D contains multiple Path3D objects.

  • Each Path3D contains one or more cubic Curve3D objects.

  • Each Curve3D has 4 control points: p_start, p_ctrl1, p_ctrl2, p_end.

• Point and vertex semantics

  • Point3D stores (x, y, z) and optional thickness.

  • Vertex3D extends Point3D with optional color and opacity.

• Tensor interface

  • Curve3D.as_tensor() returns shape [4, 3].

  • Path3D.stack_tensor() returns shape [C, 4, 3].

  • Sketch3D.stack_tensor() returns shape [N, 4, 3] for rendering.

• Serialization

  • Sketch3D.to_json(...) and Sketch3D.from_json(...) are used in tests/workflows to persist and reload 3D sketches.

Current Limitations

• Depth ordering approximation: Curve depth is currently approximated by the mean camera-space depth of each curve. This can be unstable for long or looped curves with large within-curve depth variation, and may lead to imperfect occlusion ordering in some views.

• Dense-scene CUDA fallback: For very large curve counts (or highly compressed projected geometry), DiffVG GPU rendering may be forced or switched to CPU to avoid CUDA illegal memory access errors. This improves rendering robustness but can noticeably increase render time.