For this series, I wanted to create a simple interface for “drawing” melodies on the screen. I started with a mouse interaction where clicking and dragging the mouse drew a sequence of linked nodes along the path of the cursor movement. It worked much like how you would expect any drawing app to work. The melody would start playing as soon as you release your mouse. The nodes were sequentially highlighted as their corresponding notes were played.

In order to add variations in note durations, I first mapped the duration to the distance between neighboring nodes. This turned out to appear counter-intuitive as moving mouse faster resulted in longer notes. In later revisions, I decided to map the note durations in a way that roughly reflected how long each nodes were drawn.

I landed on a color theme of muted colors for the nodes and used the blur filter to create a glowing effect for active nodes. The pitch of each note depended on the y axis.

To add a more magical effect, instead of mouse interaction, I utilized the hand pose model in ml5.js to let people draw melodies with their hand. More specifically, pinching your thumb and index finger would trigger the drawing mode while spreading them out would clear all nodes.

I also experimented with a pie-shaped status indicator, but it stood out too much against the overall minimalistic aesthetic.