I am becoming more interested in rigorizing my knowledge of mathematics.
A group is a set of elements $$G$$ and an operation $$\bigotimes$$. It satisfies the following four properties:
- Closure
- Associativity
- Neutral Element
- Inverse Element
From Groups to Vector Spaces: If we enhance our group with an outer operation, then we get a vector space! Roughly, this outer operation takes an element from inside our set, and an external element from a field (such as the reals) and then gives us a way of combining the inner element with the external element.
Equipping with an inner product: We can equip our vector space with an inner product. If the equipment is the dot product, then we have an Euclidean vector space. The inner product induces geometry on our vector space, even when we cannot visualize it.
Why it matters: Vector spaces underlie many of the operations we do in machine learning. When we are doing matrix multiplication, this forms a group. And it explains why we have associativity.
How to interpret the sup. The sup is the least upper bound. This means, it is the smallest element that bounds the expression. In effect, this is like saying the max, but where the max might be outside the set. Unlike the max, it is NOT saying that our particular element (inside the max/sup expression is the LUB). But instead, we are saying that we have an LUB over all the elements in the expression!
Dual norm: the dual norm is the