Application US20200151519

Intelligent Health Monitoring
[+] Show Details

US20200151519

1. A method comprising: (3) (9)

obtaining a content audio data;

obtaining a semantic audio data;

obtaining a generated audio data;

extracting a content audio feature from the content audio data;

extracting a semantic audio feature from the semantic audio data;

extracting a generated audio feature from the generated audio data;

feeding the extracted semantic, content and generated audio features into a neural network;

propagating the features through a neural network iteratively and updating the generated audio feature until convergence; and

upon convergence, outputting the generated feature.

2. The method of claim 1, wherein the neural network is trained by minimizing a content cost function, a semantic cost function and a combination of the content and semantic cost functions. (0)

3. The method of claim 1, wherein the content audio feature represents an audible respiratory symptom and the semantic audio feature represents an utterance from a different person, the same person in a different disease state, an acoustic environment, a recording device or any combination thereof. (0)

4. The method of claim 1, wherein the content cost function is the cost function that minimizes a distance between the content audio feature and the generated audio feature and the semantic cost function minimizes a distance between the semantic audio feature and the generated audio feature. (2) (0)

5. A method comprising: (5) (7)

obtaining, using one or more processors, first data corresponding to a first desired audio data;

obtaining, using the one or more processors, second data corresponding to one or more anchored audio events;

obtaining, using the one or more processors, third data corresponding to one or more negative audio events;

feeding, using the one or more processors, the first, second and third data into a neural network that is trained on an aggregate cost function of two or more cost functions;

generating, using the neural network, a feature for the first desired audio data by employing one or more anchor class features and one or more negative class features;

training, using the one or more processors, a classifier using the generated feature and the anchor class features as one class and the negative class features as another class; and

predicting, using the one or more processors, a class for an arbitrary audio event using the trained classifier.

6. The method of claim 5, wherein the aggregate cost function minimizes a weighted average distance between the desired audio event and the anchored audio event while maximizing a weighted average distance between the desired audio event and the one or more negative audio events. (0)

7. The method of claim 5, wherein the anchored audio event is similar to the desired audio event and the one or more negative audio events differ from the desired audio event and the anchored audio event. (0)

8. The method of claim 5, wherein the classifier is used to detect symptoms for each patient in a multi-patient room using a microphone by iteratively assigning each patient's audible symptoms as the desired audio event and the anchored audio event and the other patient's audible symptoms as the negative audio events. (0)

9. The method of claim 5, wherein the classifier can be used to reject false alarms by assigning similar audible symptoms as the desired audio events or as the anchored audio events and assigning false alarms as the negative audio events. (0)

23. The method of claim 5, wherein anchor and negative audio events are selected and updated over time to implement a classifier that is robust to one or more changes in the data collection device or an acoustical environment, new audible symptoms or new patients. (0)

10. A method comprising: (4) (4)

obtaining, using one or more processors of a device, audio data that contains two or more audio events with overlapping;

feeding the audio data into a convolutional neural network (CNN), where the CNN is trained on two or more analysis windows;

determining, using the one or more processors, boundaries of each audio event in a time-frequency representation of the audio data; and

classifying, using the one or more processors, a category of the previously localized audio data.

11. The method of claim 10, wherein the two or more analysis windows include two-dimensional windows with different heights and widths configured to detect specified audio events in the desired time-frequency representation of the audio data. (0)

12. The method of claim 10, wherein the classification is implemented using one of k-nearest neighbor, support vector machine, k-means, gaussian mixture model, a decision tree and its variations or one or more neural networks. (0)

24. The method of claim 10, wherein the time-frequency representation of the audio data is compressed to fit predetermined grid boxes, wherein each grid box corresponds to labels of audio events in the audio data. (0)

25. The method of claim 10, wherein a neural network previously trained on a set of audio events is used to extract descriptive features by propagating the desired audio events through the neural network and assigning the values from one or more layers of the network as the feature. (0)

13. A method of describing the content of an audio recording, comprising: (2) (3)

extracting features from a plurality of audio recordings;

feeding the features to a pre-trained, recurrent neural network; and

generating, using the recurrent neural network, a sentence describing the content of the recording.

14. The method of claim 13, wherein the pre-trained recurrent network is trained by minimizing a cost function rolled over time with a sequence of audio features extracted from the audio recordings as the input to the neural network and their corresponding descriptive sentences as the output. (0)

15. The method of claim 13, wherein the neural network is constructed from convolutional cells followed by a recurrent neural network with Long Short-Term Memory (LSTM), or a Gatted Recurrent Unit (GRU) and softmax layers. (0)

16. (canceled) (0)

17. A system comprising: (2) (2)

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising: obtaining a content audio data;

obtaining a semantic audio data;

obtaining a generated audio data;

extracting a content audio feature from the content audio data;

extracting a semantic audio feature from the semantic audio data;

extracting a generated audio feature from the generated audio data;

feeding the extracted semantic, content and generated audio features into a neural network;

propagating the neural network iteratively and updating the generated audio feature until convergence; and

upon convergence, outputting the generated feature that includes the content audio feature and the semantic audio feature.

26. The system of claim 17, wherein the content cost function considers one or more middle layers of the neural network as the target for the content of the generated audio feature when the true content audio data is propagated through the neural network. (0)

27. The system of claim 17, wherein the semantic cost function measures a similarity of the semantic audio data from one or more layers with the generated audio feature by calculating a gram matrix. (1) (0)

18. A system comprising: (0) (2)

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising: obtaining, using one or more processors, first data corresponding to a first desired audio data;

obtaining, using the one or more processors, second data corresponding to one or more anchored audio events;

obtaining, using the one or more processors, third data corresponding to one or more negative audio events;

feeding, using the one or more processors, the first, second and third data into a neural network that is trained on an aggregate cost function of two or more cost functions;

generating, using the neural network, a feature for the first desired audio data by employing one or more anchor class features and one or more negative class features;

training, using the one or more processors, a classifier using the generated feature and the anchor class features as one class and the negative class features as another class; and

predicting, using the one or more processors, a class for an arbitrary audio event using the trained classifier.

19. A system comprising: (2) (3)

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising: obtaining, using one or more processors of a device, audio data that contains two or more audio events with or without overlapping;

feeding the audio data into a convolutional neural network (CNN), where the CNN is trained on two or more analysis windows;

determining, using the one or more processors, boundaries of each audio event in a time-frequency representation of the audio data; and

classifying, using the one or more processors, a category of the audio data.

29. The system of claim 19, wherein the time-frequency representation of the audio data is compressed to fit predetermined grid boxes, wherein each grid box corresponds to labels of audio events in the audio data. (0)

30. The system of claim 19, wherein a neural network previously trained on a set of audio events is used to extract descriptive features by propagating the desired audio events through the neural network and assigning the values from one or more layers of the network as the feature. (0)

20. A system comprising: (0) (4)

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

extracting features from a plurality of audio recordings; feeding the features to a pre-trained, recurrent neural network; and

generating, using the recurrent neural network, a sentence describing the content of the recording;

extracting features from a plurality of audio recordings; feeding the features to a pre-trained, recurrent neural network; and

generating, using the recurrent neural network, a sentence describing the content of the recording.

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Application US20200151519

Intelligent Health Monitoring

Patent Matrix® Search