Image-to-Scene Sketch Translation

Table of Contents

• About
• Setup
• Authors

---

About

Publication

https://ieeexplore.ieee.org/document/9288276

Description

Image generation from sketch is a popular and well-studied computer vision problem. However, the inverse problem image-to-sketch (I2S) synthesis still remains open and challenging, let alone image-to-scene sketch (I2S 2 ) synthesis, especially when full-scene sketch generations are highly desired. This research investigates a framework for generating full-scene sketch representations from natural scene images, aiming to generate outputs that approximate hand-drawn scene sketches.

Implementation

The base architecture for this project is a pytorch implementation of CycleGAN.

Exploratory work includes:

• Designed Canny edge detection pytorch module for additional input
• Implemented Holistically-Nested Edge Detection architecture for additional input
• Implemented Wasserstein loss metric as alternative to least squares and cross entropy
• Experimented with learning rate decay and other scheduling

Canny Edge Detection:

Works referenced in this project

CycleGAN: Project | Paper | Torch

Pix2pix: Project | Paper | Torch

HED: Project | Paper | Torch

Back to Top

Setup

Required modules for training/inference

• PyTorch 0.41+

Required modules for scripts

• opencv-python

Anaconda environment

See environment.yml

Back to Top

Authors

---

• Dan McGonigle dpmcgonigle

Back to Top