iOS 15.4 opened up the LiDAR sensor to AVFoundation. In this demo MIDI notes played on a paired synthesizer are interpreted by Metal shaders to produce augmented reality visualizations. This demo builds on parts of Capturing Depth Using the LiDAR Camera, AudioKit, and Linnstrument Helper.
The demo with sound can be viewed at this tweet.
