Let’s say we have an array of 3 x 1 x 6 x 8. Then we can quickly make some assertions and inferences: 1. The “smallest” level elements of this MD array are size 8.
When we do broadcasting, we are just doing expand_dims operations, and then working on two huge cubes that are the same size!
When we do unsqueezing, we can get a real flavour of how Torch/numpy actually stores the values in the underlying memory addresses.
Additionally: if we perform an expand_dims operation. It does it explicitly in the memory-layout view as we know from before! In particular, 2 x 1 x 3, and then doing an expand to ((2,3,3)) means that we will get 3x3 arrays. The 1x3 arrays are simply copied.
For instance: [ # 2 elements [ # 1 element [ # 3 elements 1,2,3]], [[4,5,6]] ]
Goes to: [
[[1,2,3],[1,2,3],[1,2,3] ],
[[4,5,6],[4,5,6],[4,5,6]] #nice, an expand is just a duplicate in the approriate dimension]
This explains why when we expand/unsqueeze in the right most dimension, we get the lengthening of the “internal”/lowest list elements.