The neural network was taught to "hear" colors and "see" music

"Synesthesia" is a rare psychophysiological phenomenon in which different sensory sensations are mixed. For example, a person develops the ability to hear colors or taste words. Usually, synesthesia occurs when there are serious malfunctions in the nervous system, when substances are used to expand consciousness, or when creative experiments are carried out deliberately. Now this rare skill is available to artificial intelligence as well.

Belgian machine learning researcher and lecturer Xander Steenbrugge has developed a new type of neural network that can "turn on" synesthesia on command. It's called "Neural Synesthesia". At the moment, the neural network selects visual forms for musical compositions, using ready-made artistic solutions and images generated by it itself.

According to Steenbrugge, this project is an attempt to create new approaches in the development of audio-visual solutions based on the use of AI. The operator does not know and does not even imagine the end result, he only sets some criteria to the AI ​​and gives instructions on which databases and which content should be used. Further, the neural network itself generates a visual series, which can be accepted by the operator or sent for revision. This is not pure creativity - rather, an attempt to train AI to perceive music in such a way that it can express it in a different sensory format.