Patent US9451883

Apparatus and method for decoding sensory and cognitive information from brain activity
[+] Show Details

US9451883

1. A method for decoding and reconstructing a subjective perceptual or cognitive experience, the method comprising: (10) (8)

acquiring a first set of brain activity data from a first subject, wherein said first set of brain activity data is acquired using a first brain imaging device, and wherein said first set of brain activity data is produced in response to a first set of brain activity stimuli;

converting said first set of brain activity data into a corresponding set of encoding model parameter values for each of one or more linearizing feature spaces;

acquiring a second set of brain activity data from said first subject or a second subject, wherein said second set of brain activity data is acquired using a second brain imaging device, and wherein said second set of brain activity data is produced in response to a second set of brain activity stimuli;

creating a decoding database comprising one or more items;

generating a corresponding set of predicted brain activity signals for each of the one or more items in the decoding database using the set of encoding model parameter values corresponding to each of the one or more linearizing features spaces;

determining a probability for each of the one or more items that said second set of brain activity data corresponds to said set of predicted brain activity signals corresponding to the item;

selecting at least one item from the decoding database based on the probability determined for each of the one or more items in the decoding database; and

producing a reconstructed set of brain activity stimuli based on the selected at least one item.

2. A method as recited in claim 1, wherein said first set of brain activity data or said second set of brain activity data is acquired using a brain imaging technique selected from the group consisting of EEG, MEG, fMRI, fNIRS, SPECT and ECoG. (0)

3. The method of claim 1, wherein said first set of brain activity stimuli is selected from the group consisting of sensory stimuli, motor stimuli, and cognitive stimuli, or any combination thereof. (0)

4. The method of claim 1, wherein the selected at least one item comprises a plurality of selected items, and (0) (1)

5. The method of claim 1, wherein said second set of brain activity stimuli is selected from the group consisting of sensory stimuli, motor stimuli, and cognitive stimuli, or any combination thereof. (0)

6. A method as recited in claim 1, wherein determining the probability for each of the one or more items comprises constructing a decoding distribution, and (1) (2)

said decoding distribution is calculated using a formula that comprises:

wherein α represents a value of said second set of brain activity stimuli, r is a measurement of said second set of brain activity data, s is said second set of brain activity stimuli, and h is the set of encoding model parameter values corresponding to each of the one or more linearizing feature spaces.

7. A method as recited in claim 6, (0) (4)

wherein constructing said decoding distribution further comprises:

evaluating said r, said h, and said s of said decoding distribution by constructing a portion of the formula, the portion being p(r|s,h);

integrating over all possible parameter values with respect to p(h); and

integrating over all possible stimuli with respect to p(s|α).

8. A method as recited in claim 1, wherein the encoding model parameter values corresponding to each of the one or more linearizing feature spaces are part of an encoding model that describes a relationship between said first set of brain activity data and said first set of brain activity stimuli, (0) (2)

9. A method as recited in claim 1, wherein the encoding model parameter values corresponding to each of the one or more linearzing feature spaces are part of an encoding model that describes a relationship between said first set of brain activity data and said first set of brain activity stimuli, (0) (2)

10. A method as recited in claim 1, wherein said first set of brain activity stimuli is associated with a first cognitive state, and (0) (1)

11. A method as recited in claim 1, wherein the second brain imaging device is the first brain imaging device. (0)

12. A method as recited in claim 1, wherein the selected at least one item is the second set of brain activity stimuli; and (0) (1)

13. An apparatus for decoding and reconstructing a subjective perceptual or cognitive experience, the apparatus comprising: (6) (10)

(a) a processor: and

(b) programming executable on the processor for performing a method comprising:

converting said first set of brain activity data into a corresponding set of encoding model parameter values for each of one or more linearizing feature spaces;

obtaining a plurality of stimulus items;

generating a corresponding set of predicted brain activity signals for each of the plurality of stimulus items using the set of encoding model parameter values corresponding to each of the one or more linearizing feature spaces;

determining a probability for each of the plurality of stimulus items that said second set of brain activity data corresponds to said set of predicted brain activity signals corresponding to the stimulus item;

selecting one or more of the plurality of stimulus items based on the probability determined for each of the plurality of stimulus items; and

producing a reconstructed set of brain activity stimuli based on the selected one or more stimulus items.

14. An apparatus as recited in claim 13, wherein the reconconstructed set of brain activity stimuli is produced by computing an average or weighted average of the selected one or more stimulus items. (0)

15. An apparatus as recited in claim 13, wherein determining the probability for each of the one or more stimulus items comprises constructing a decoding distribution, and (1) (2)

said decoding distribution is calculated using a formula that comprises:

16. An apparatus as recited in claim 15, wherein constructing said decoding distribution comprises: (0) (3)

17. An apparatus as recited in claim 13, (0) (3)

18. An apparatus as recited in claim 13, wherein the encoding model parameter values corresponding to each of the one or more linearizing feature spaces are part of an encoding model that describes a relationship between said first set of brain activity data and said first set of brain activity stimuli, (0) (2)

19. An apparatus as recited in claim 13, wherein the second brain imaging device is the first brain imaging device. (0)

20. An apparatus as recited in claim 13, wherein said first set of brain activity stimuli is associated with a first cognitive state, and (0) (1)

21. A method comprising: (7) (10)

obtaining training data comprising one or more training stimuli and training brain activity data, the training brain activity data representing brain activity elicited in at least one first subject by each of the one or more training stimuli;

transforming each of the one or more training stimuli into one or more training feature spaces to obtain a set of training feature spaces;

calculating a plurality of encoding model parameter values that model a linear relationship between the set of training feature spaces and the training brain activity data;

obtaining measured brain activity data representing brain activity elicited in at least one second subject by a particular stimulus;

obtaining a plurality of candidate stimulus items;

transforming each of the plurality of candidate stimulus items into a corresponding feature space set comprising one or more candidate feature spaces;

generating predicted brain activity data for each of the plurality of candidate stimulus items using the feature space set that corresponds to the candidate stimulus item and the set of encoding model parameter values;

comparing the measured brain activity data to the predicted brain activity data generated for each of the plurality of candidate stimulus items;

selecting one or more of the plurality of candidate stimulus items based on the comparison; and

producing a reconstructed stimulus based on the selected one or more candidate stimulus items.

22. The method of claim 21, wherein selecting the one or more of the plurality of candidate stimulus items based on the comparison comprises: (0) (1)

23. The method of claim 21, wherein selecting the one or more of the plurality of candidate stimulus items based on the comparison comprises: (0) (1)

24. The method of claim 21, wherein the reconstructed stimulus is produced by computing an average or weighted average of the selected one or more candidate stimulus items. (0)

25. The method of claim 21, wherein the selected one or more candidate stimulus items is the particular stimulus, and (0) (1)

26. The method of claim 21, wherein transforming each of the one or more training stimuli into the one or more training feature spaces comprises passing each of the one or more training stimuli through a bank of nonlinear Gabor wavelet filters, and (1) (1)

28. The method of claim 21, wherein the at least one second subject is the at least one first subject. (0)

29. The method of claim 21, wherein each of the one or more training stimuli is associated with a cognitive state, and (0) (1)

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Patent US9451883

Apparatus and method for decoding sensory and cognitive information from brain activity

Patent Matrix® Search