Interactive Heat map demo
Drag mouse (from the top to the right) to mark a new region for inference.
Use the A,S,D,W keys to translate and J,K,L,I to enlarge bounding box (hold the Shift key
to move x10 faster). This region from the source image (see its position,
width and height in console) is scaled into 224x224 image and is shown to the right.
Open a new browser window (or tab) and Copy / Paste(Ctrl+V) images from
image.google.com,
unsplash.com, ...
This application shows which areas of the image contribute
the most to the "daisy" prediction.
The brightness of the image is proportional to exp(0.1 (Aij - max)),
where Aij - activations in the "daisy" class on 7x7 grid
(see the Aij maximum and average values in console),
max - is the maximum Aij value or is equal to "actMax" (if checked).
You can set new Heat map Class and actMax values.
Unfortunately it is not clear what features give activations.
Class Activation Mapping and CNN surgery
As you can see below ("mobilenet.summary()" is used),
the 1024-features map (or embeddings or activations)
of the MobileNet_v1_1.0_224 is made by the Global Average Pooling (GAP) over
the 7×7 space grid of the preceding [7,7,1024]-features map.
See also MobileNet v1 Layers model analysis or
You can use e.g. "JSON Viewer Awesome" Chrome extension to explore
model.json
file.
conv_pw_13_relu [null,7,7,1024]
____________________________________________
global_average_pooling2d_1 [null,1024]
Therefore, if we cut off the GAP layer, we will get localization of activations
in the "daisy" channel on the 7×7 space grid. You can see above activations
in bilinear interpolated and nearest modes. See also
"Daisy" Class Activation Mapping
and one more application
by Chao Wang.
TFjs notes
updated 11 Jan 2020