Interpretation of Einstein notation for matrix multiplication
Andrew Henderson 
Consider the matrix product $C = AB$ where $A \in \mathbb{R}^{m \times n}, B \in \mathbb{R}^{n \times p}$. The Einstein summation notation for this is$$ c_{ik} = a_{ij}b_{jk}. $$Is there any example from math, physics, engineering, statistics etc. where each term in the sum $a_{ij}b_{jk}$ has a meaningful interpretation?
Equivalently, I guess: suppose we did not use Einstein summation notation, and instead defined $D \in \mathbb{R}^{m \times n \times p}$ as:$$ D_{ijk} = A_{ij}B_{jk}. $$Would the elements of $D$ mean anything with respect to $A, B, C$ or the linear transformations represented by these matrices? Obviously $C_{ik} = \sum_{j} D_{ijk}$, but what does this mean?
Or suppose, instead of summing over the second index of $D$ to get $C$, we instead summed over the first index of $D$ to get $E \in \mathbb{R}^{n \times p}$:$$ E_{jk} = \sum_{i} D_{ijk}. $$Obviously this cannot be represented in Einstein notation. So does this not have meaning as a tensor contraction, and unlikely to have a physical interpretation?
Edit: For the powers of adjacency matrices (eg $C = A^2 = AA$), the terms do have an interpretation. Iff $a_{ij}b_{jk} = 1$ (not using Einstein notation), then there is a path from node $i$ to node $k$ through node $j$.
$\endgroup$2 Answers
$\begingroup$Define a third order tensor whose components are equal to zero unless all three indices are equal $${\cal H}_{ijk} = \begin{cases} 1 \quad{\rm if}\; i\!=\!j=\!k \\ 0 \quad{\rm otherwise} \\ \end{cases} $$Then you can use Einstein notation to write$${\cal D}_{ijk} = A_{ip}{\cal H}_{pjs}B_{sk}$$This tensor is a useful addition to standard matrix algebra.
It can be used to generate a diagonal matrix $A$ from a vector $a$ (using a single-dot product) $$A = \operatorname{Diag}(a) = {\cal H}\cdot a \quad\implies A_{ij} = {\cal H}_{ijk}\,a_k$$or to create a vector $b$ from the main diagonal of a matrix $B$ (using a double-dot product)$$b = \operatorname{diag}(B) = {\cal H}:B \quad\implies b_{i} = {\cal H}_{ijk}\,B_{jk}$$or simply as a way to write ${\cal D}$ without resorting to index notation$${\cal D} = A\cdot{\cal H}\cdot B$$
$\endgroup$ $\begingroup$One way to think about this is as if you're taking the scalar/inner/dot product of the vectors $a_i$ and $b_k$ by indexing these two vectors with j. In this way, each entry of the matrix $c_{ik}$ is exactly the inner product in question.
$\endgroup$
