Multimedia Processing with FFMPEG

• Academia / Research
• Tips and Tricks
• Tutorials

FFMPEG is a set of libraries and a command line tool for encoding and decoding audio and video in many different formats. It is a free software project for manipulating/processing multimedia data. Many open source media players are based on FFMPEG libraries.

FFMPEG is developed under Linux but it can be compiled under most operating systems including Mac OS, Microsoft Windows. For more details about FFMPEG please refer here

For windows operating system, it is preferred to download the pre-built binary from here. It is preferred to download the static build. For Linux operating system, we can either build from the source by following the instruction given here or can install on Ubuntu using the following command

    sudo apt-get install ffmpeg

YUV
There are lot of color spaces available like RGB, CMYK, HSV, YUV, CIELAB etc., The one that is widely used for video processing is YUV color space. YUV was invented when engineers wanted color television in a black and white infrastructure. They needed a signal transmission method that was compatible with black-and-white TV being able to add color. The luma component already existed as the black-and-white signal, they added the UV signal to this as a solution. More details here.

Inputs to the Image and Video Processing will be generally YUV files. Input video will be usually in YUV 4:2:0 chroma subsampled format. More details here.

Sample YUV sequences can be downloaded from here.

Video Processing with FFMPEG
The below command resizes the input YUV video of resolution 1920x1080 in YUV 4:2:0 format to 1280x720 YUV video

ffmpeg -s:v 1920x1080 -i input.yuv -vf scale=1280:720 -c:v rawvideo -pix_fmt yuv420p output.yuv

If you want to resize to some other resolution change the width and height assigned to the parameter scale.

To compress the YUV file of resolution 1280x720, frame rate 30 fps in H.264 video format at 5 Mbps

ffmpeg -s:v 1280x720 -i input.yuv -vcodec h264 -r 30 -pix_fmt yuv420p -b:v 5000k output.h264

To compress the above video in MPEG4 video format

ffmpeg -s:v 1280x720 -i input.yuv -vcodec mpeg4 -r 30 -pix_fmt yuv420p -b:v 5000k output.m4v

where the parameters present next to the following parameters
-s:v represents input resolution
-i represents input file
-vcodec represents output codec type
-r represents frame rate of the output video
-pix_fmt represents input video YUV chroma subsampling format
-b:v represents bitrate of the output video

Image Processing with FFMPEG
The following command decompresses the JPEG image file and stores it in YUV format

ffmpeg -i image.jpg -pix_fmt yuv420p image.yuv

The following command can be used to resize the JPEG image

ffmpeg -i image.jpg -vf scale=width:height output.jpg
(or)
ffmpeg -i image.jpg -vf scale=1280:720 output.jpg

The following command can be used to convert JPG image to BMP image format

ffmpeg -i image.jpg image.bmp

To make individual still images from the compressed video

ffmpeg -i inputfile %03d.jpg

To make individual still images from the YUV file

ffmpeg -s:v 1280x720 -pix_fmt yuv420p -i inputyuvfile %03d.jpg

To make individual still images from the YUV file with good quality

ffmpeg -s:v 1280x720 -pix_fmt yuv420p -i inputyuvfile -q 1 %03d.jpg

The parameter -q van vary from 1 to 100. Lower the value higher the quality.

Audio Processing with FFMPEG
FFMPEG can also be used to convert/transcode audio files compressed with MP3 codec to AAC using the following command

ffmpeg -i input.mp3 -c:a libvo_aacenc -b:a 128k output.m4a
(or)
ffmpeg -i input.mp3 -acodec aac -ab 128k -strict experimental output.m4a

where the parameters present next to the following parameters
-c:a or -acodec represents output codec type
-i represents input file
-b:a or -ab represents bitrate of the output audio

To decompress the audio file and store it as PCM samples use the following command

ffmpeg -i input.mp3 output.wav

To compress the uncompressed raw PCM audio samples in wav file to compressed MP3 & AAC format use the below commands

ffmpeg -i input.wav -c:a mp3 -b:a 128k output.mp3
ffmpeg -i input.wav -c:a libvo_aacenc -b:a 128k output.m4a
(or)
ffmpeg -i input.wav -acodec aac -ab 128k -strict experimental output.m4a

Combining Audio and Video files
Suppose if we have video and audio in elementary stream format and we can use the following command to mix them

ffmpeg -i videofile -i audiofile -c:v copy -c:a copy -strict experimental output.mp4

The above command does not perform any processing on the audio and video samples when we use -c:a copy and -c:v copy. It simply copies the contents and packs them in the MP4 container.

TV not playing audio for certain files
Often we would have seen certain TV's while playing videos fail to play audio. Use the following command to fix this issue. Almost all of the TV support AAC audio format and the following command converts the audio to AAC format, stores both the video and AAC audio in MP4 container format.

ffmpeg -i inputfile -vcodec copy -acodec aac -ab 128k -strict experimental outfile.mp4

Capture uncompressed videos from webcam with FFMPEG
ffmpeg -f video4linux2 -r 30 -s 640x480 -i /dev/video0 output.avi

If the above command is executed on Laptop /dev/video0 by default selects the Integrated webcam. To capture from external webcam on Laptop change it to /dev/video1.

Conclusion
This article just provides an overview of FFMPEG usage. It can be used at most of the places where we deal with multimedia data. FFMPEG is swiss army knife of multimedia processing. I will provide more insights in the next articles.

Sign in