site

files for beauhilton.com
git clone https://git.beauhilton.com/site.git
Log | Files | Refs

yt-dlp.md (6370B)


      1 # yt-dlp Nix config
      2 
      3 <time id="post-date">2023-10-18</time>
      4 
      5 <p id="post-excerpt">
      6 yt-dlp is an incredibly useful and customizable tool 
      7 for downloading online videos
      8 from YouTube and elsewhere.
      9 Here's my setup 
     10 (declared in Nix, but easily generalizable).</p>
     11 
     12 If you just want the config, scroll to the bottom
     13 (ignore the ramblings under the asterisk).
     14 
     15 ## yt-dlp
     16 
     17 youtube-dl was, for years, the de facto standard 
     18 for retrieving videos from YouTube 
     19 (and other websites - e.g. Vimeo) for local use, 
     20 but eventually it got old and crufty.
     21 
     22 [yt-dlp](https://github.com/yt-dlp/yt-dlp) is a fork of youtube-dl 
     23 that is more actively maintained,
     24 with many upgrades.
     25 
     26 It has sane defaults, but I have a few things I change to make it suit my uses.
     27 
     28 I mostly use it to download instructional videos so they're easier to use:
     29 no getting interrupted by ads 
     30 whenever I scrub back and forth to find or rewatch a particular section,
     31 and I can proceed without an internet connection.
     32 
     33 It's also a boon for archival of any video you use frequently for any reason,
     34 e.g. in an educational setting - 
     35 if you have a lesson plan that depends on a clip from an online video,
     36 it is wise to have your own copy of the video,
     37 both to reduce dependence on finicky wifi
     38 and to make it more likely you'll have the vid
     39 for every time you repeat this lesson in the future.
     40 Videos go offline for strange, sundry, and unpredictable reasons.
     41 
     42 (I maintain that YouTube and related video sites, 
     43 taken as a whole,
     44 are the single greatest source of procedural knowledge 
     45 ever created by humankind\* -
     46 the problems that threaten their utility are,
     47 first, curation, and second, durability)
     48 
     49 
     50 ## configuring
     51 
     52 yt-dlp accepts a config file, 
     53 usually placed at `~/.config/yt-dlp/config`,
     54 so you don't have to have a long string of flags and options 
     55 in your shell command to yt-dlp
     56 (you can override your config file with shell options, 
     57 so it's reasonable to set your defaults to your most common use 
     58 and not worry about messing up some divergent situation that may come up).
     59 
     60 For example, 
     61 to download the whole 
     62 [JohnWatsonRooney YouTube channel](https://www.youtube.com/c/JohnWatsonRooney), 
     63 all I had do was:
     64 
     65 ```sh
     66 cd youtube-downloads/JohnWatsonRooney
     67 yt-dlp "https://www.youtube.com/c/JohnWatsonRooney"
     68 ```
     69 
     70 And it did something like this:
     71 
     72 ![yt-dlp downloading example](/images/yt-dlp-downloading-example.png)
     73 
     74 Resulting in:
     75 
     76 ![yt-dlp filename example](/images/yt-dlp-filename-example.jpg)
     77 
     78 In the second screenshot you can see that some files are still in process -
     79 when all the parts are downloaded,
     80 yt-dlp smushes them into an mkv container,
     81 so the videos/thumbnails/subtitle files are clean and accessible to most video players.
     82 
     83 ## config
     84 
     85 Here's my config file, with some annotation.
     86 
     87 I'm using Nix with Home Manager,
     88 but you can translate this into the usual yt-dlp config file syntax with ease.
     89 
     90 One of the great things about Nix 
     91 is that it lets me group installation and configuration 
     92 for programs that make sense together -
     93 here, I want to use aria2 as the downloader,
     94 so I declare the aria2 options to pass to yt-dlp,
     95 and also make sure aria2 is installed 
     96 (it's ok if you installed aria2 in another part of your setup - 
     97 if a program is declared for installation in multiple places,
     98 Nix is smart enough to install it only once,
     99 so it's often wise to err on the side of bundling).
    100 
    101 If I take this config to any other machine with Nix installed,
    102 I'll get the whole setup including dependencies.
    103 This example is very simple, just adding another program,
    104 but the principle scales to arbitrarily complex setups.
    105 
    106 ```nix
    107 {
    108   programs.yt-dlp = {
    109     enable = true;
    110     settings = {
    111       embed-chapters = true; # embed all the things
    112       embed-metadata = true;
    113       embed-thumbnail = true;
    114       convert-thumbnail = "jpg"; 
    115       # so every file manager can show the thumbnail - webp support is not quite universal
    116       embed-subs = true;
    117       sub-langs = "all"; 
    118       # subtitle files are very small, 
    119       # and sometimes language names are declared badly, 
    120       # so worth it to grab them all
    121       downloader = "aria2c";
    122       downloader-args = "aria2c:'-c -x16 -s16 -k2M'"; 
    123       # -c is resume if interrupted ("continue"), 
    124       # -x is max connections to a server, 
    125       # -s is number of connections used for download of a specific file, 
    126       # -k is size of chunks
    127       download-archive = "yt-dlp-archive.txt"; 
    128       # writes a file to the current directory specifying which files have already been downloaded - 
    129       # nice for updating your collection of a channel's videos 
    130       # (just run the download command again and it will grab only what you're missing)
    131       restrict-filenames = true; # disallow spaces, weird characters, etc.
    132       output = "%(upload_date>%Y-%m-%d)s--%(uploader)s--%(title)s--%(id)s.%(ext)s"; 
    133       # I like to be able to sort by date 
    134       # and have enough info in the filename 
    135       # so I don't need to open it to find out what it is, 
    136       # so I include the:
    137       # - ISO 8601-style date
    138       # - uploader's name
    139       # - title of the video
    140       # - video ID (for easy copy pasta if I ever want to find it online, 
    141       # e.g. to see the comment section or show notes.)
    142     };
    143   };
    144   programs.aria2 = {
    145     enable = true;
    146   };
    147 }
    148 ```
    149 
    150 ## side note
    151 
    152 \* Contrary to popular belief, 
    153 things that go online do *not* stay online forever:
    154 [digital obsolescence](https://en.wikipedia.org/wiki/Digital_obsolescence) is real, 
    155 and is [probably going to get worse](https://blog.archive.org/2023/03/25/the-fight-continues/) 
    156 before it gets better ("?better"). 
    157 This section is under an asterisk and at the end because it is a side thought, 
    158 just an idea that tools like yt-dlp 
    159 may have import that goes beyond helping me write web scrapers,
    160 build saunas, etc. -
    161 maybe they have the primitives to help build a 
    162 videographic Whole Earth Catalog for the ages.
    163 I don't think everything is going to explode any time soon,
    164 but wouldn't it be interesting to have a digital archive with videos,
    165 rather than just text, of how to do the basic things (and some not-so-basic things)
    166 that make life possible and enjoyable? 
    167 Some kind of all-in-one system with redundant storage 
    168 and a power-supply (hand crank? solar?),
    169 durable screen and speakers,
    170 to help you bootstrap the physical needs of a society?
    171 I'm sure the raw content is out there.