Image to Algorithm to Music: Generating Music from the Pixels

Published on October 28, 2025 by h | 40 views

Who doesn't relate soundtracks to photos? At the end it is part of the human experience to match memories with senses or vice-versa - but what if we let the machine do it for us?

 

I love algorithms and I am trying hard to learn more on how to design and improve them. Also, while working on Tormes I have been diving deeper into PageRank in order to better implement a relevance system in the search engine. That's how I ended up stumbling upon the Markov Chains Algorithm last week and getting to know in a very, VERY superficial way how it works for now.

 

 

The main thing about Markov Chains is that it describes a process that moves from one state to another and the probability of switching the state - but ONLY the current state matters and the past, well, you leave it in the past lol

 

You can only move from one stone to another, and where you go next depends only on the color you're standing on now — not on how you got there.

 

That's the key idea: The future depends only on the present, not the past.

 

For example:

 

  • If you're on a red stone, there's a 70% chance you'll step to blue next, and a 30% chance to green.
  • If you're on blue, maybe you go to red 50% of the time and stay on blue 50%.

 

If you repeat this process over and over, you get a Markov chain - a system where things change step by step, and each step depends only on where you are right now.

 

At the end, a Markov chain is not about knowing exactly the next move, but more about the behavior of switching states and capturing it correctly. It can be a good algorithm to apply when generating music from an image!

 

How Do We Reflect This When Generating Music?

 

I won't say simple, but, yeah, 'unsimple' - but let's turn you, my fellow reader, a machine:

 

> You start painting the first pixel of an image which has only 10 colors. It is the photo of a sunset, and its first pixel is painted with #FF6B35 (a vibrant orange)

 

> Then you paint the next pixel, which is #FFA07A (some peachy salmon color)

 

> But then, wow, you have to paint in vibrant orange / #FF6B35 again

 

> After that it is #4B0082, which is indigo

 

> ...and so on and so on

 

By taking a good look we can see a pattern hence we establish our matrix which goes like this:

 

| Current Color            | Likely Next Color                                                |
| ------------------------ | ---------------------------------------------------------------- |
| #FF6B35 (vibrant orange) | #FFD700 (golden yellow) — 60%<br>#FF4500 (deep orange-red) — 40% |
| #FFD700 (golden yellow)  | #FF1493 (hot pink) — 80%<br>#FF6B35 (vibrant orange) — 20%       |
| #FF1493 (hot pink)       | #8B4789 (purple) — 70%<br>#FF8C69 (coral) — 30%                  |
| #8B4789 (purple)         | #191970 (midnight blue) — 90%<br>#FF1493 (hot pink) — 10%        |
| #191970 (midnight blue)  | #FF6B35 (vibrant orange) — 100%                                  |

 

So far so good - thus we will leave the machine persona and return to our human one.

 

We can assign notes after group the colors by similarity:

 

| Color                     | Group   | Note |
| ------------------------- | ------- | ---- |
| #FF6B35 (vibrant orange)  | Color 0 | C |
| #FFD700 (golden yellow)   | Color 1 | D |
| #FF4500 (deep orange-red) | Color 2 | E |
| #FF1493 (hot pink)        | Color 3 | F |
| #8B4789 (purple)          | Color 4 | G |
| #191970 (midnight blue)   | Color 5 | A |
| #4B0082 (indigo)          | Color 6 | B |

 

This is how we get the musical piece! If the most common color is #FF6B35 (vibrant orange) / C and we start with it, the process could go:

  • From #FF6B35 (vibrant orange) → likely #FFD700 (golden yellow) → D
  • From #FFD700 (golden yellow) → likely #FF1493 (hot pink) → F
  • From #FF1493 (hot pink) → likely #8B4789 (purple) → G
  • From #8B4789 (purple) → likely #191970 (midnight blue) → A

 

We get this melody:

C → D → F → G → A → C → D → F → G → A...

 

Beethoven would be proud!!

 

You can test it using the 'Pixel Music Generator' widget at the home page, the last option in the second row. I will be adapting the code soon to make it available on GitHub. For the moment it only has 7 groups of colors, which are correlated to the 7 musical notes, but my plan is to add at some point Majors, Minors, Sharps, etc.

 

Keep creating!

 

Further reading / watching:

← Back to Blog