We provide a full characterisation of all of the possible group equivariant neural networks whose layers are some tensor power of ℝ^n for three symmetry groups that are missing from the machine learning literature: O(n), the orthogonal group; SO(n), the special orthogonal group; and Sp(n), the symplectic group. In particular, we find a spanning set of matrices for the learnable, linear, equivariant layer functions between such tensor power spaces in the standard basis of ℝ^n when the group is O(n) or SO(n), and in the symplectic basis of ℝ^n when the group is Sp(n).