gallery-dl

Overview

gallery-dl is a command-line tool for downloading image galleries and collections from over 100 websites — including Instagram, Twitter/X, Reddit, Pixiv, DeviantArt, Tumblr, Flickr, ArtStation, Imgur, and dozens of image boards. It handles authentication, pagination, metadata extraction, and flexible file naming. Think of it as yt-dlp but for images and galleries.

Instructions

Step 1: Installation

# pip (recommended)
pip install gallery-dl

# With optional dependencies for broader format support
pip install gallery-dl[yt-dlp]    # adds video download support via yt-dlp

# Standalone binary (Linux)
curl -L https://github.com/mikf/gallery-dl/releases/latest/download/gallery-dl.bin -o /usr/local/bin/gallery-dl
chmod +x /usr/local/bin/gallery-dl

# macOS
brew install gallery-dl

# Update
pip install -U gallery-dl

# Verify
gallery-dl --version

Step 2: Basic Downloads

# Download from a gallery/post URL
gallery-dl "https://www.instagram.com/p/POST_ID/"
gallery-dl "https://twitter.com/user/status/123456789"
gallery-dl "https://www.reddit.com/r/EarthPorn/top/?t=month"
gallery-dl "https://www.pixiv.net/en/artworks/12345678"
gallery-dl "https://imgur.com/a/ALBUM_ID"

# Download entire user profile
gallery-dl "https://www.instagram.com/username/"
gallery-dl "https://twitter.com/username/media"
gallery-dl "https://www.deviantart.com/username"
gallery-dl "https://www.pixiv.net/en/users/USER_ID"

# Download with custom output directory
gallery-dl -d ./downloads "URL"

# Download with verbose output
gallery-dl -v "URL"

Step 3: Output Templates and Organization

gallery-dl uses a powerful template system for organizing downloaded files into directories.

# Custom filename template
gallery-dl -o "filename={category}_{id}_{title}.{extension}" "URL"

# Organize by site and user
gallery-dl -o "directory=['{category}', '{subcategory}']" "URL"

# Custom directory per site
gallery-dl -o "base-directory=./media" -o "directory=['{category}', '{user}', '{date:%Y-%m}']" "URL"

Default directory structure: ./gallery-dl/{category}/{subcategory}/

Step 4: Authentication

Many sites require login for full access (age-restricted content, private profiles, higher rate limits).

# Username + password
gallery-dl -u "username" -p "password" "URL"

# Cookie-based authentication (recommended for most sites)
# Export cookies from your browser using a browser extension (e.g., "Get cookies.txt")
gallery-dl --cookies cookies.txt "URL"

# Browser cookie extraction (auto-extracts from installed browser)
gallery-dl --cookies-from-browser firefox "URL"
gallery-dl --cookies-from-browser chrome "URL"

# OAuth (for Tumblr, DeviantArt, Reddit, etc.)
gallery-dl oauth:reddit          # opens browser for OAuth flow
gallery-dl oauth:deviantart
gallery-dl oauth:tumblr

Step 5: Filtering and Selection

# Download only images (skip videos)
gallery-dl --filter "extension in ('jpg', 'jpeg', 'png', 'gif', 'webp')" "URL"

# Download only videos
gallery-dl --filter "extension in ('mp4', 'webm', 'mov')" "URL"

# Minimum image dimensions
gallery-dl --filter "width >= 1920 and height >= 1080" "URL"

# Date range
gallery-dl --filter "date >= datetime(2024, 1, 1)" "URL"

# Limit number of downloads
gallery-dl --range "1-50" "URL"       # first 50 items
gallery-dl --range "10-20" "URL"      # items 10 through 20

# Skip files that already exist
gallery-dl --no-skip "URL"    # download even if exists (default: skip)

Step 6: Configuration File

gallery-dl supports a JSON configuration file at ~/.config/gallery-dl/config.json (or ~/.gallery-dl.conf).

{
  "extractor": {
    "base-directory": "./gallery-dl/",
    "archive": "~/.gallery-dl/archive.sqlite3",

    "instagram": {
      "cookies-from-browser": "firefox",
      "directory": ["instagram", "{user}"],
      "filename": "{date:%Y%m%d}_{shortcode}_{num:>02}.{extension}",
      "posts": {
        "include": ["image", "video", "sidecar"]
      }
    },

    "twitter": {
      "cookies-from-browser": "chrome",
      "directory": ["twitter", "{user[name]}"],
      "filename": "{date:%Y%m%d}_{tweet_id}_{num:>02}.{extension}",
      "retweets": false,
      "text-tweets": false
    },

    "reddit": {
      "directory": ["reddit", "{subreddit}", "{id}"],
      "comments": 0,
      "morecomments": false
    },

    "pixiv": {
      "directory": ["pixiv", "{user[name]}"],
      "filename": "{id}_p{num:>02}.{extension}",
      "ugoira": true
    }
  },

  "downloader": {
    "rate": "2M",
    "retries": 3,
    "timeout": 30.0,
    "part": true,
    "part-directory": "/tmp/.gallery-dl/"
  },

  "output": {
    "log": {
      "level": "info",
      "format": "{message}"
    }
  },

  "postprocessor": [
    {
      "name": "metadata",
      "mode": "json"
    }
  ]
}

Step 7: Archive and Deduplication

The archive feature tracks downloaded files in a SQLite database, preventing re-downloads on subsequent runs.

# Use archive file (skip already downloaded)
gallery-dl --download-archive archive.sqlite3 "URL"

# Clear archive for specific URL
gallery-dl --clear-archive archive.sqlite3 "URL"

In config:

{
  "extractor": {
    "archive": "~/.gallery-dl/archive.sqlite3"
  }
}

Step 8: Post-Processing

gallery-dl can run post-processors on downloaded files — metadata export, conversion, classification.

{
  "postprocessor": [
    {
      "name": "metadata",
      "mode": "json",
      "directory": "metadata/"
    },
    {
      "name": "exec",
      "command": ["convert", "{_path}", "-resize", "1920x1080>", "{_path}"]
    },
    {
      "name": "zip",
      "compression": "store",
      "extension": "cbz"
    }
  ]
}

Step 9: Supported Sites (Partial List)

Major platforms supported by gallery-dl:

• Social media: Instagram, Twitter/X, Reddit, Tumblr, TikTok
• Art platforms: Pixiv, DeviantArt, ArtStation, Newgrounds
• Image hosting: Imgur, Flickr, SmugMug, Google Photos
• Image boards: Danbooru, Gelbooru, Safebooru, E621, Konachan
• Manga/comics: MangaDex, Webtoon, Tapas, Dynasty Scans
• Stock/photography: Unsplash, 500px
• Other: Patreon, Kemono, Cohost, Mastodon, Bluesky, Pillowfort

Full list: gallery-dl --list-extractors

Step 10: Batch Downloads

# Download from URL list file
gallery-dl -i urls.txt

# Combine with rate limiting
gallery-dl -i urls.txt --rate-limit 2M --sleep 3-5

# Download with metadata export
gallery-dl -i urls.txt --write-metadata

Examples

Example 1: Archive an artist's complete portfolio from multiple platforms

User prompt: "Download all artwork from this artist — they post on Pixiv, Twitter, and DeviantArt. I want everything organized by platform with metadata saved."

The agent will:

1. Set up a config with per-platform directory organization: {category}/{user}/{date:%Y-%m}/
2. Enable archive tracking to avoid duplicates: --download-archive artist_archive.sqlite3
3. Run gallery-dl for each platform URL with metadata post-processor enabled.
4. The archive database ensures running the same command later only downloads new posts.
5. Report total files downloaded per platform and total disk usage.

Example 2: Build an image dataset from Reddit

User prompt: "I need to collect high-resolution landscape photos from r/EarthPorn for a machine learning dataset. Only images above 1920x1080, from the last year, save metadata as JSON."

The agent will:

1. Configure gallery-dl with dimension filter: --filter "width >= 1920 and height >= 1080"
2. Add date filter for the last year and metadata post-processor.
3. Run: gallery-dl --filter "width >= 1920 and height >= 1080" --download-archive dataset_archive.sqlite3 -o "postprocessor=[{'name': 'metadata', 'mode': 'json'}]" "https://www.reddit.com/r/EarthPorn/top/?t=year"
4. The JSON metadata files include title, author, score, dimensions, and original URL — useful for dataset labeling.

Guidelines

• Always use --download-archive when downloading from profiles or feeds that update over time; it prevents re-downloading previously saved files on subsequent runs.
• Set up authentication (cookies or OAuth) before downloading from platforms that gate content behind login — Instagram, Pixiv, Patreon, and Twitter all limit unauthenticated access.
• Use --cookies-from-browser instead of manual cookie export when possible — it auto-extracts fresh cookies and handles session tokens.
• Apply rate limiting ("rate": "2M" and "sleep": "3-5" in config) to avoid triggering anti-bot measures.
• Test with --range "1-5" first to verify your configuration and filename templates before running a full download.
• gallery-dl pairs well with yt-dlp — gallery-dl handles images and galleries, yt-dlp handles video. Together they cover most content download scenarios.