General

Batch Conversion: Processing Thousands of Files Efficiently

Published Mar 19, 2026 6 min read By ChangeThisFile Team

Quick Answer

Batch conversion means automating format conversion across hundreds or thousands of files using tools like FFmpeg, ImageMagick, LibreOffice headless, and Pandoc with command-line scripting. The key to reliable batch processing: preserve originals, handle errors gracefully, name outputs predictably, and monitor progress.

Converting one file is a click. Converting a thousand files is an engineering problem. You have a photo archive of 5,000 RAW images that need JPEG versions for web delivery. Or a folder of 200 DOCX reports to convert to PDF for archival. Or a video library of 300 AVI files to transcode to MP4. Doing this one file at a time would take days.

Batch conversion is the solution, and it's built on four command-line powerhouses: FFmpeg (audio/video), ImageMagick (images), LibreOffice (documents), and Pandoc (markup/text). Each can be scripted to process an entire directory tree, handle errors, log results, and produce consistent output. This guide covers the practical scripts and strategies for each tool.

FFmpeg: Audio and Video Batch Processing

FFmpeg processes virtually any audio or video format. It's the engine behind ChangeThisFile's server-side audio and video conversions.

Common Batch Scripts

Convert all AVI to MP4 (H.264):

for f in *.avi; do
  ffmpeg -i "$f" -c:v libx264 -crf 23 -c:a aac -b:a 128k "${f%.avi}.mp4"
done

Convert all FLAC to MP3 (320kbps):

for f in *.flac; do
  ffmpeg -i "$f" -c:a libmp3lame -b:a 320k "${f%.flac}.mp3"
done

Extract audio from all video files:

for f in *.mp4; do
  ffmpeg -i "$f" -vn -c:a copy "${f%.mp4}.m4a"
done

Convert all MKV to MP4 (stream copy, no re-encoding):

for f in *.mkv; do
  ffmpeg -i "$f" -c copy "${f%.mkv}.mp4"
done

Stream copy (-c copy) is dramatically faster than re-encoding because it simply repackages the existing streams. Use it whenever you're changing container but not codec. MKV to MP4 and MOV to MP4 often work with stream copy.

FFmpeg Batch Tips

Overwrite control: Add -n to skip existing outputs, or -y to overwrite. Default behavior is to ask, which hangs batch scripts.
Error handling: Check exit code with $?. FFmpeg returns 0 on success, non-zero on failure.
Recursive processing: Use find . -name '*.avi' -exec ffmpeg ... or find . -name '*.avi' | while read f; do ... done for subdirectories.
Metadata preservation: FFmpeg copies metadata by default. Add -map_metadata 0 explicitly if metadata is disappearing, or -map_metadata -1 to strip it.

ImageMagick: Image Batch Processing

ImageMagick's mogrify command is designed specifically for batch operations — it converts files in place (or to a new directory).

Common Batch Scripts

Convert all PNG to JPG (quality 85):

mogrify -format jpg -quality 85 *.png

Convert all images to WebP, save in a different directory:

mkdir -p webp_output
mogrify -format webp -path webp_output *.{jpg,png,bmp}

Resize all images to max 1920px wide:

mogrify -resize 1920x\> *.jpg

The \> flag means "only resize if larger" — images under 1920px are left untouched.

Convert and resize in one pass:

mogrify -format webp -resize 1200x -quality 80 -path output/ *.png

For higher performance, consider using sharp (Node.js/libvips) or vips directly — they're 5-10x faster than ImageMagick for bulk operations. PNG to WebP and JPG to WebP conversions are especially fast with libvips.

LibreOffice Headless: Document Batch Processing

LibreOffice's headless mode converts documents without a GUI, making it ideal for server-side and batch operations.

Convert all DOCX to PDF:

libreoffice --headless --convert-to pdf --outdir output/ *.docx

Convert all XLSX to CSV:

libreoffice --headless --convert-to csv --outdir output/ *.xlsx

Convert all PPT to PPTX:

libreoffice --headless --convert-to pptx --outdir output/ *.ppt

LibreOffice Batch Limitations

Single-instance bottleneck: LibreOffice headless runs one conversion at a time. A second instance attempting to start will either queue or fail. For large batches, this means conversions are sequential — 1,000 DOCX files at 3 seconds each means 50 minutes. Don't try to parallelize with xargs -P or GNU Parallel — it will cause lock conflicts.

Font dependency: Output quality depends on installed fonts. If the DOCX uses Calibri and your system doesn't have it, LibreOffice substitutes, causing layout shifts. Install Microsoft core fonts before batch converting: apt install ttf-mscorefonts-installer on Ubuntu.

ChangeThisFile uses LibreOffice headless for server-side document conversions: DOCX to PDF, XLSX to CSV, PPTX to PDF, and more.

Pandoc: Markup and Text Batch Processing

Pandoc converts between markup formats: Markdown, HTML, LaTeX, reStructuredText, DOCX, EPUB, and more.

Convert all Markdown to HTML:

for f in *.md; do
  pandoc "$f" -o "${f%.md}.html" --standalone
done

Convert all Markdown to PDF (via LaTeX):

for f in *.md; do
  pandoc "$f" -o "${f%.md}.pdf" --pdf-engine=xelatex
done

Convert all HTML to DOCX:

for f in *.html; do
  pandoc "$f" -o "${f%.html}.docx"
done

Pandoc is fast — it processes text-based formats in milliseconds per file. A batch of 1,000 Markdown files converts to HTML in under a minute. Related: Markdown to HTML, HTML to Markdown, Markdown to PDF.

Parallel Processing with GNU Parallel

For CPU-bound conversions (FFmpeg encoding, ImageMagick processing), parallel execution dramatically reduces total time. GNU Parallel distributes work across CPU cores:

Convert AVI to MP4 using all CPU cores:

find . -name '*.avi' | parallel -j$(nproc) \
  ffmpeg -i {} -c:v libx264 -crf 23 -c:a aac {.}.mp4

Convert PNG to WebP with 4 parallel workers:

find . -name '*.png' | parallel -j4 \
  convert {} -quality 80 {.}.webp

How many parallel jobs? For FFmpeg video encoding, 2-4 parallel jobs (each FFmpeg process uses multiple threads internally). For ImageMagick image conversion, match the number of CPU cores. For LibreOffice, only 1 (single-instance limitation). For Pandoc, match CPU cores (it's single-threaded per invocation).

Error Handling and Logging

Batch conversion without error handling means you discover 50 failed files after the entire batch finishes — and you don't know which ones. Always log results:

#!/bin/bash
LOG="conversion_$(date +%Y%m%d_%H%M%S).log"
FAILED=0
SUCCESS=0

for f in *.avi; do
  echo "Converting: $f" | tee -a "$LOG"
  if ffmpeg -y -i "$f" -c:v libx264 -crf 23 -c:a aac "${f%.avi}.mp4" 2>>"$LOG"; then
    echo "  SUCCESS" | tee -a "$LOG"
    ((SUCCESS++))
  else
    echo "  FAILED" | tee -a "$LOG"
    ((FAILED++))
  fi
done

echo "\nComplete: $SUCCESS succeeded, $FAILED failed" | tee -a "$LOG"

Key practices:

Log everything. Redirect stderr to a log file. FFmpeg's error messages explain exactly why a conversion failed.
Count successes and failures. A summary at the end tells you immediately if something went wrong.
Don't stop on first error. Use conditional execution (if/then) rather than set -e. One corrupted file shouldn't abort the remaining 999.
Verify outputs. After batch conversion, check that output file count matches input, and spot-check a few files for quality.

Naming Conventions and Directory Structure

Three strategies for batch output organization:

Same directory, new extension: photo.png → photo.webp. Simplest, but mixes originals and conversions.
Output subdirectory: originals/photo.png → converted/photo.webp. Clean separation. Use -path (mogrify) or -outdir (LibreOffice).
Suffix-based: photo.png → photo_web.webp. Keeps files together while distinguishing originals from derivatives.

The cardinal rule: never overwrite originals. Keep source files untouched until you've verified all conversions succeeded. An interrupted batch that overwrites files in place leaves you with neither the original nor the complete conversion.

Batch conversion is where command-line tools pay for themselves. A 10-line Bash script replacing 5,000 manual clicks isn't premature optimization — it's basic sanity. The investment in learning FFmpeg, ImageMagick, or LibreOffice headless syntax pays dividends every time you face a folder full of files in the wrong format.

For one-off conversions, use ChangeThisFile's web converter. For bulk operations, take the scripts in this guide, adapt them to your file types, and let the machine do the repetitive work. That's what computers are for.

Key Takeaways

FFmpeg, ImageMagick, LibreOffice headless, and Pandoc are the four workhorses of batch conversion
GNU Parallel speeds up CPU-bound conversions by distributing across cores — except LibreOffice (single instance only)
Always log results and count successes/failures — don't discover errors after a 2-hour batch completes
Never overwrite originals during batch conversion; use output directories or different extensions
FFmpeg's stream copy (-c copy) repackages without re-encoding — 100x faster when you only need to change containers
LibreOffice headless processes one file at a time; plan for sequential throughput (~3 seconds per document)

Frequently Asked Questions

How fast is batch image conversion?

ImageMagick converts approximately 10-50 images per second depending on resolution and target format. Libvips (used by sharp) is 5-10x faster — 50-200 images per second. A batch of 10,000 photos (12MP JPG to WebP) takes roughly 3-15 minutes with ImageMagick or 30 seconds to 3 minutes with libvips. Parallel processing across multiple cores scales nearly linearly for image conversion.

Can I batch convert documents on a server without a GUI?

Yes. LibreOffice headless mode (--headless flag) runs without any display server. It works on Linux servers without X11, making it ideal for Docker containers and headless servers. Install LibreOffice, run 'libreoffice --headless --convert-to pdf *.docx', and it processes everything without a GUI. ChangeThisFile's conversion server uses this exact approach.

Why does LibreOffice only process one file at a time?

LibreOffice uses a lock file and inter-process communication that prevents multiple instances from running simultaneously. Starting a second instance while one is converting will either queue it or produce errors. This is a known architectural limitation. The workaround for higher throughput is to use multiple LibreOffice profiles (--env:UserInstallation=...) on different ports, but this is complex and fragile.

How do I batch convert files in subdirectories recursively?

Use 'find' to locate files recursively and pipe them to your conversion tool. For example: 'find /path -name "*.png" | while read f; do convert "$f" "${f%.png}.webp"; done'. Or with GNU Parallel: 'find /path -name "*.png" | parallel convert {} {.}.webp'. The find command handles any directory depth, and the parameter expansion ({.} or ${f%.png}) strips the extension for the output filename.

What if some files fail during batch conversion?

This is expected — batch conversion over hundreds of files will likely encounter some corrupted or unusual files. Design your script to log failures and continue processing. After the batch completes, review the log for failed files and handle them individually. Never let one failure abort the entire batch. The error handling script in this guide shows the pattern: try each file, log the result, count successes and failures.

Can I limit how much CPU or memory batch conversion uses?

Yes. GNU Parallel's -j flag limits concurrent jobs. FFmpeg's -threads flag limits threads per process. For memory: FFmpeg uses memory proportional to video resolution, and ImageMagick's memory can be controlled with -limit memory 1GiB. On Linux, you can also use 'nice' (lower priority) and 'cpulimit' (throttle CPU percentage) to prevent batch jobs from starving other processes.

Should I re-encode video or stream copy for batch operations?

If you're only changing the container (MKV to MP4, MOV to MP4) and the target container supports the existing codecs, use stream copy (-c copy). It's 100x faster because it just repackages the data. If you need to change codec (H.265 to H.264), bitrate, or resolution, you must re-encode. A common pattern: try stream copy first, fall back to re-encode if it fails.

How much disk space do I need for batch conversion?

At minimum, enough for all input files plus all output files simultaneously — you need both until verification. For video re-encoding, temporary files can add 10-50% overhead. As a rule: have 2x your input size available. If converting 100GB of video, ensure 200GB free. Monitor with 'df -h' during long batches. If space is tight, process in smaller batches and delete verified outputs before continuing.

Ready to convert your files?

Use ChangeThisFile to convert between 600+ formats — free, fast, and private.

Start Converting