Application US20200151516

Methods for Explainability of Deep-Learning Models
[+] Show Details

US20200151516

1. A method of interpreting an artificial intelligence (AI) model prediction, comprising: (4) (9)

feeding a first set of input features to the AI model;

obtaining, using one or more processors, a first set of raw output predictions from the model;

determining, using the one or more processors, a first set of impact scores for one or more of the input features fed into the model;

training a neural network with the first set of impact scores as input to the network and pre-determined sentences describing the model's behavior as output;

feeding a second set of input features to the AI model;

obtaining, using one or more processors, a second set of raw output predictions from the model;

determining a second set of impact scores based on the second set of output predictions;

feeding the second set of impact scores to the neural network; and

generating a sentence describing the AI model's behavior on the second set of input features.

2. The method of claim 1, further comprising backtracking through the neural network by tracing activation values from each neuron in the neural network through all hidden layers in the neural network back to each input feature. (0)

7. The method of claim 1, wherein the sentence is generated by looking up the sentence in a look-up table of predetermined sentences indexed by the neurons that have the most impact on the prediction. (0)

8. The method of claim 1, wherein the sentence is generated one letter at a time by a recurrent neural network, long short term memories (LSTMs), or gated recurrent units (GRUs). (0)

9. The method of claim 1, wherein the neural network is trained on impact scores as input and predetermined sentences as labels interpreting the neural network. (0)

3. A system comprising: (4) (2)

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising: feeding a first set of input features to an artificial intelligence (AI) model;

obtaining a first set of raw output predictions from the AI model;

determining a first set of impact scores for one or more of the input features fed into the AI model;

training a neural network with the first set of impact scores as input to the network and pre-determined sentences describing the model's behavior as output;

feeding a second set of input features to the AI model;

obtaining a second set of raw output predictions from the model;

determining a second set of impact scores based on the second set of output predictions;

feeding the second set of impact scores to the neural network; and

generating a sentence describing the AI model's behavior on the second set of input features.

4. The system of claim 3, the operations further comprising backtracking through the neural network by tracing activation values from each neuron in the neural network through all hidden layers in the neural network back to each input feature. (2) (0)

5. The method of claim 4, wherein the backtracking is done by tracking a weighted average of activation functions from selected previous input layers and picking the largest value corresponding to the feature for each layer. (1) (0)

10. The system of claim 4, wherein the backtracking is done by tracking a weighted average of activation functions from selected previous input layers and picking the largest value corresponding to the feature for each layer. (1) (0)

12. The system of claim 3, wherein the sentence is generated by looking up the sentence in a look-up table of predetermined sentences indexed by the neurons that have the most impact on the prediction. (0)

13. The system of claim 3, wherein the sentence is generated one letter at a time by a recurrent neural network, long short term memories (LSTMs), or gated recurrent units (GRUs). (0)

14. The system of claim 3, wherein the neural network is trained on impact scores as input and predetermined sentences as labels interpreting the neural network. (0)

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Application US20200151516

Methods for Explainability of Deep-Learning Models

Patent Matrix® Search