index.html (10563B)
1 <!DOCTYPE html> 2 <html lang="en"> 3 <head> 4 <link rel="stylesheet" href="/style.css" type="text/css"> 5 <meta charset="utf-8"> 6 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 7 <meta name="viewport" content="width=device-width, initial-scale=1.0"> 8 <link rel="stylesheet" type="text/css" href="/style.css"> 9 <link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>🏕️</text></svg>"> 10 <title></title> 11 </head> 12 <body> 13 <div id="page-wrapper"> 14 <div id="header" role="banner"> 15 <header class="banner"> 16 <div id="banner-text"> 17 <span class="banner-title"><a href="/">beauhilton</a></span> 18 </div> 19 </header> 20 <nav> 21 <a href="/about">about</a> 22 <a href="/now">now</a> 23 <a href="/thanks">thanks</a> 24 <a class="nav-active" href="/posts">posts</a> 25 <a href="https://notes.beauhilton.com">notes</a> 26 <a href="https://talks.beauhilton.com">talks</a> 27 <a href="https://git.beauhilton.com">git</a> 28 <a href="/contact">contact</a> 29 <a href="/atom.xml">rss</a> 30 </nav> 31 </div> 32 <main> 33 <h1> 34 How to add a mirror to a single ZFS disk 35 </h1> 36 <p> 37 <time id="post-date">2023-10-19</time> 38 </p> 39 <p id="post-excerpt"> 40 tl;dr: <code>zpool attach data /dev/disk/by-partlabel/zfs-3a1xx /dev/sdx0</code>. Adjust for your own pool and disks. 41 </p> 42 <h2> 43 A mirror is a Very Good Thing 44 </h2> 45 <p> 46 I got a second identical disk to add as a ZFS mirror to my existing 47 setup. 48 </p> 49 <p> 50 Redundancy FTW. ZFS is king and queen. Needs at least two disks to 51 reap the full benefit (though a single-disk setup is still rad - CoW 52 with snapshots and send/receive are worth it, and NixOS makes it easy to 53 set up). 54 </p> 55 <h2> 56 It’s all about the disk path 57 </h2> 58 <p> 59 The basic command is (as a super user, or using sudo): 60 </p> 61 <pre tabindex="0"><code class="language-sh">zpool attach name<span class="hl kwb">-of-the-pool</span> disk<span class="hl kwb">-you-already-had</span> disk<span class="hl kwb">-you-want-to-add</span> 62 </code></pre> 63 <p> 64 On Linux, with a new drive, no need for formatting or any other disk 65 preparation. Just plug it in. 66 </p> 67 <p> 68 But how to specify which disks to use in the command? I saw several 69 guides that just used <code>/dev/sdx</code>, but it didn’t work. 70 </p> 71 <p> 72 I don’t know what it is about my setup that differs from the guides I 73 found online (if you have a hint, <a href="beauhilton.com/contact">lmk</a>), so YMMV, but below is some of 74 what I tried and what eventually worked, mostly for my own 75 reference. 76 </p> 77 <p> 78 In my setup, <code>/dev/sdb</code> was my existing drive and 79 <code>/dev/sda</code> is the one I was trying to add (these names are 80 determined at boot time and can vary from boot to boot - ZFS takes care 81 of finding things again on the back end after you’ve added a drive, 82 regardless of the label you used to add it, so using the 83 <code>/dev/sdx</code> name is just fine in many situations). 84 </p> 85 <p> 86 However, using 87 </p> 88 <pre tabindex="0"><code class="language-sh">zpool attach data <span class="hl opt">/</span>dev<span class="hl opt">/</span>sdb dev<span class="hl opt">/</span>sda 89 </code></pre> 90 <p> 91 gave me: 92 </p> 93 <pre tabindex="0"><code class="language-sh">cannot attach <span class="hl opt">/</span>dev<span class="hl opt">/</span>sda to <span class="hl opt">/</span>dev<span class="hl opt">/</span>sdb<span class="hl opt">:</span> no such device <span class="hl kwa">in</span> pool 94 </code></pre> 95 <p> 96 I tried a bunch of different things, from just the lsblk name with 97 the partition (“sdb1”), to by-id (“ata-ST1400…”), to full path 98 combinations, etc. 99 </p> 100 <p> 101 What finally worked was the by-partlabel identifier for the existing 102 disk, combined with the /dev/sdx0 name of the new one. 103 </p> 104 <p> 105 A little <code>zdb -l</code> on the existing drive gave me the path I 106 wanted: 107 </p> 108 <pre tabindex="0"><code class="language-sh">zdb <span class="hl kwb">-l</span> <span class="hl opt">/</span>dev<span class="hl opt">/</span>sdb1 109 <span class="hl kwb">------------------------------------</span> 110 LABEL <span class="hl num">0</span> 111 <span class="hl kwb">------------------------------------</span> 112 version<span class="hl opt">:</span> <span class="hl num">5000</span> 113 name<span class="hl opt">:</span> <span class="hl sng">'data'</span> 114 state<span class="hl opt">:</span> <span class="hl num">0</span> 115 txg<span class="hl opt">:</span> <span class="hl num">170113</span> 116 pool_guid<span class="hl opt">:</span> <span class="hl num">8237228336559358688</span> 117 errata<span class="hl opt">:</span> <span class="hl num">0</span> 118 hostid<span class="hl opt">:</span> <span class="hl num">3236406100</span> 119 hostname<span class="hl opt">:</span> <span class="hl sng">'dell7050'</span> 120 top_guid<span class="hl opt">:</span> <span class="hl num">11182661003591634341</span> 121 guid<span class="hl opt">:</span> <span class="hl num">11182661003591634341</span> 122 vdev_children<span class="hl opt">:</span> <span class="hl num">1</span> 123 vdev_tree<span class="hl opt">:</span> 124 <span class="hl kwb">type</span><span class="hl opt">:</span> <span class="hl sng">'disk'</span> 125 id<span class="hl opt">:</span> <span class="hl num">0</span> 126 guid<span class="hl opt">:</span> <span class="hl num">11182661003591634341</span> 127 path<span class="hl opt">:</span> <span class="hl sng">'/dev/disk/by-partlabel/zfs-3a1e459c75dc9b74'</span> 128 whole_disk<span class="hl opt">:</span> <span class="hl num">1</span> 129 metaslab_array<span class="hl opt">:</span> <span class="hl num">128</span> 130 metaslab_shift<span class="hl opt">:</span> <span class="hl num">34</span> 131 ashift<span class="hl opt">:</span> <span class="hl num">12</span> 132 asize<span class="hl opt">:</span> <span class="hl num">14000504438784</span> 133 is_log<span class="hl opt">:</span> <span class="hl num">0</span> 134 DTL<span class="hl opt">:</span> <span class="hl num">3865</span> 135 create_txg<span class="hl opt">:</span> <span class="hl num">4</span> 136 features_for_read<span class="hl opt">:</span> 137 com.delphix<span class="hl opt">:</span>hole_birth 138 com.delphix<span class="hl opt">:</span>embedded_data 139 labels <span class="hl opt">=</span> <span class="hl num">0 1 2 3</span> 140 </code></pre> 141 <p> 142 So I tried: 143 </p> 144 <pre tabindex="0"><code class="language-sh">zpool attach data <span class="hl opt">/</span>dev<span class="hl opt">/</span>disk<span class="hl opt">/</span>by<span class="hl kwb">-partlabel</span><span class="hl opt">/</span>zfs<span class="hl kwb">-3a1e459c75dc9b74</span> <span class="hl opt">/</span>dev<span class="hl opt">/</span>sda1 145 </code></pre> 146 <h2> 147 Great success 148 </h2> 149 <pre tabindex="0"><code class="language-conf">->> zpool status 150 pool<span class="hl opt">:</span> data 151 state<span class="hl opt">:</span> ONLINE 152 <span class="hl kwa">status</span><span class="hl opt">:</span> One or more devices is currently being resilvered<span class="hl opt">.</span> The pool will 153 continue to function<span class="hl opt">,</span> possibly in a degraded state<span class="hl opt">.</span> 154 <span class="hl kwa">action</span><span class="hl opt">:</span> Wait for the resilver to complete<span class="hl opt">.</span> 155 scan<span class="hl opt">:</span> resilver in progress since Thu Oct <span class="hl kwb">19 17</span><span class="hl opt">:</span><span class="hl kwb">05</span><span class="hl opt">:</span><span class="hl kwb">26 2023</span> 156 <span class="hl kwb">2.33</span>T scanned at <span class="hl kwb">1.90</span>G/s<span class="hl opt">,</span> <span class="hl kwb">252</span>G issued at <span class="hl kwb">205</span>M/s<span class="hl opt">,</span> <span class="hl kwb">3.77</span>T total 157 <span class="hl kwb">252</span>G resilvered<span class="hl opt">,</span> <span class="hl kwb">6.53</span>% done<span class="hl opt">,</span> <span class="hl kwb">04</span><span class="hl opt">:</span><span class="hl kwb">59</span><span class="hl opt">:</span><span class="hl kwb">42</span> to go 158 <span class="hl kwa">config</span><span class="hl opt">:</span> 159 160 NAME STATE READ WRITE CKSUM 161 data ONLINE <span class="hl kwb">0 0 0</span> 162 mirror-<span class="hl kwb">0</span> ONLINE <span class="hl kwb">0 0 0</span> 163 zfs-<span class="hl kwb">3</span>a1e459c75dc9b74 ONLINE <span class="hl kwb">0 0 0</span> 164 sda1 ONLINE <span class="hl kwb">0 0 0</span> (resilvering) 165 166 <span class="hl kwa">errors</span><span class="hl opt">:</span> No known data errors 167 </code></pre> 168 <p> 169 …and the next day, after resilvering completed (and a reboot to make 170 sure the disk came online): 171 </p> 172 <pre tabindex="0"><code class="language-conf">->> zpool status 173 pool<span class="hl opt">:</span> data 174 state<span class="hl opt">:</span> ONLINE 175 scan<span class="hl opt">:</span> resilvered <span class="hl kwb">3.77</span>T in <span class="hl kwb">05</span><span class="hl opt">:</span><span class="hl kwb">13</span><span class="hl opt">:</span><span class="hl kwb">53</span> with <span class="hl kwb">0</span> errors on Thu Oct <span class="hl kwb">19 22</span><span class="hl opt">:</span><span class="hl kwb">19</span><span class="hl opt">:</span><span class="hl kwb">19 2023</span> 176 <span class="hl kwa">config</span><span class="hl opt">:</span> 177 178 NAME STATE READ WRITE CKSUM 179 data ONLINE <span class="hl kwb">0 0 0</span> 180 mirror-<span class="hl kwb">0</span> ONLINE <span class="hl kwb">0 0 0</span> 181 zfs-<span class="hl kwb">3</span>a1e459c75dc9b74 ONLINE <span class="hl kwb">0 0 0</span> 182 zfs-e2937e0fc8ebe95a ONLINE <span class="hl kwb">0 0 0</span> 183 184 <span class="hl kwa">errors</span><span class="hl opt">:</span> No known data errors 185 </code></pre> 186 <p> 187 (notice that what was <code>sda1</code> is now referred to by 188 partlabel) 189 </p> 190 </main> 191 <div id="footnotes"></div> 192 <footer></footer> 193 </div> 194 </body> 195 </html>