How Video Containers and Codecs Actually Work

Why does converting a MOV to MP4 take two seconds, but converting a video to GIF takes two minutes? The answer lies in the distinction between containers and codecs — two concepts that most people (and even some developers) confuse constantly.

Containers vs codecs

A container (also called a wrapper) is the file format — the .mp4, .mov, or .webm extension. It's a box that holds multiple streams of data: video, audio, subtitles, metadata, and chapter markers.

A codec (coder-decoder) is the algorithm that compresses and decompresses the actual video and audio streams inside the container. H.264 is a codec. AAC is a codec. The container just organizes them into a single file.

Think of it like a shipping box:

Container (MP4)     = The cardboard box
Video codec (H.264) = The product inside
Audio codec (AAC)   = The instruction manual
Subtitles (SRT)     = The packing slip
Metadata            = The shipping label

Same product, different box:
  video.mp4  → MP4 container + H.264 video + AAC audio
  video.mov  → MOV container + H.264 video + AAC audio
  video.mkv  → MKV container + H.264 video + AAC audio

Same box, different product:
  video.mp4  → MP4 container + H.264 video
  video.mp4  → MP4 container + H.265 video
  video.mp4  → MP4 container + AV1 video

Common containers

Common video codecs

H.264 (AVC)

The most widely supported video codec on the planet. Every phone, browser, TV, and game console can play H.264. It offers good compression (typically 5–10 Mbps for 1080p) and fast encoding. If you need maximum compatibility, H.264 is the safe choice.

H.265 (HEVC)

Roughly 40–50% more efficient than H.264 at the same visual quality. The catch: hardware support is spottier (older Android devices, some browsers), and licensing costs have limited adoption. iPhone recordings often use H.265.

VP9

Google's royalty-free competitor to H.265. Similar compression efficiency. Used extensively by YouTube — most YouTube streams are VP9 in a WebM container. Supported in Chrome, Firefox, and Edge.

AV1

The newest contender, developed by an alliance including Google, Mozilla, and Netflix. About 30% more efficient than H.265 and royalty-free. Encoding is very slow, but hardware decoders are appearing in newer chips. YouTube and Netflix are already serving AV1 to supported devices.

Remuxing vs re-encoding

Understanding this distinction explains why some conversions are instant and others take forever:

• Remuxing (container swap) — Copies the compressed video and audio streams into a new container without modification. Takes seconds. No quality loss. Example: MOV (H.264) to MP4 (H.264).
• Re-encoding (transcoding) — Decodes the video completely, then re-compresses it with a different codec or settings. Takes minutes to hours depending on length and resolution. Each re-encode adds generational quality loss.

Speed comparison (1 minute, 1080p video):

MOV → MP4 (remux, same codec)     →  ~2 seconds
MP4 → WebM (re-encode to VP9)     →  ~3 minutes
MP4 → GIF (re-encode, no codec)   →  ~2 minutes
MP4 → MP4 (H.264 → H.265)        →  ~5 minutes

Why GIF is terrible for video

GIF was designed in 1987 for simple animations, not video. It has crippling limitations:

• 256 color limit — Each frame can only use 256 colors from the full 16.7 million color palette. This causes visible banding in gradients and photographs.
• No inter-frame compression — Each frame is stored independently. Video codecs like H.264 store only the differences between frames, which is vastly more efficient.
• No audio — GIF has no audio track support at all.

A 10-second 720p clip might be 2 MB as an MP4 but 20 MB as a GIF. For web animations, WebP or short MP4 loops are almost always better choices.

The container is the envelope, the codec is the letter inside. Most video problems come from confusing the two — or re-encoding when a simple remux would do.