Application US20200201435

End-To-End Deep Neural Network For Auditory Attention Decoding
[+] Show Details

US20200201435

1. A method comprising: (6) (4)

receiving neural data responsive to a listener's auditory attention;

receiving an acoustic signal responsive to a plurality of acoustic sources;

for each of the plurality of acoustic sources: generating, from the received acoustic signal, audio data comprising one or more features of the acoustic source,

forming combined data representative of the neural data and the audio data, and

providing the combined data to a classification network configured to calculate a similarity score between the neural data and the acoustic source using one or more similarity metrics; and

using the similarity scores calculated for each of the acoustic sources to identify, from the plurality of acoustic sources, an acoustic source associated with the listener's auditory attention.

2. The method of claim 1, comprising separating the acoustic signal into a plurality of candidate signals, wherein generating audio data for an acoustic source comprises generating audio data using one of the plurality of candidate signals associated with the acoustic source. (0)

3. The method of claim 1, comprising: (1) (2)

5. The method of claim 1, wherein the neural data comprises multi-channel electroencephalogram (EEG) data. (0)

6. The method of claim 1, wherein forming the combined data comprises generating a matrix comprising the neural data and the audio data. (0)

7. The method of claim 1, wherein the classification network is a convolutional deep neural network (DNN). (1) (0)

9. The method of claim 1, wherein using the similarity scores calculated for each of the acoustic sources to identify the acoustic source associated with the listener's auditory attention comprises identifying an acoustic source having the highest similarity score. (0)

10. An apparatus comprising: (5) (3)

a neural sensor interface configured to receive one or more neural signals responsive to a listener's auditory attention;

an audio input configured to receive an acoustic signal responsive to a plurality of acoustic sources; and

a processor configured to: process the one or more neural signals to generate multi-channel neural data;

for each of the plurality of acoustic sources: generate, from the received acoustic signal, audio data comprising one or more features of the acoustic source,

form combined data representative of the neural data and the audio data, and

provide the combined data to a classification network configured to calculate a similarity score between the neural data and the acoustic source using one or more similarity metrics; and

identify, from the plurality of acoustic sources, an acoustic source associated with the listener's auditory attention based on the calculated similarity scores.

11. The apparatus of claim 10, wherein the processor is configured to separate the acoustic signal into a plurality of candidate signals, and to generate audio data for an acoustic source using one of the plurality of candidate signals associated with the acoustic source. (1) (0)

13. The apparatus of claim 10, wherein the neural sensor interface is configured to receive multi-channel electroencephalogram (EEG) measurements. (0)

14. The apparatus of claim 10, wherein the processor is configured to form the combined data comprises as a matrix comprising the neural data and the audio data. (0)

15. The apparatus of claim 10, wherein the classification network is a convolutional deep neural network (DNN). (2) (0)

18. The apparatus of claim 10, wherein the processor is configured to identify the acoustic source associated with the listener's auditory attention by identifying an acoustic source having the highest similarity score. (0)

19. A non-transitory computer-readable medium storing program instructions that are executable to: (0) (4)

receive neural data responsive to a listener's auditory attention;

receive an acoustic signal responsive to a plurality of acoustic sources;

for each of the plurality of acoustic sources: generate, from the received acoustic signal, audio data comprising one or more features of the acoustic source,

form combined data representative of the neural data and the audio data, and

provide the combined data to a classification network configured to calculate a similarity score between the neural data and the acoustic source using one or more similarity metrics; and

use the similarity scores calculated for each of the acoustic sources to identify, from the plurality of acoustic sources, an acoustic source associated with the listener's auditory attention.

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Application US20200201435

End-To-End Deep Neural Network For Auditory Attention Decoding

Patent Matrix® Search