Ogg is a transport bitstream designed to provide framing, error protection and seeking structure for higher-level codec streams.
See the xiph.org wiki and specification documents for more information:
These audio and video codecs can be encapsulated in Ogg: