A while ago my lab got itself into a properly broken state, and the quickest way out was to reinstall ESX on all the hosts and start fresh. So I did exactly that – wiped the OS, reinstalled ESX 9.1, and went straight into preparing the hosts for a new VMware Cloud Foundation deployment. And that’s where I ran into a wall: the fresh install refused to play nicely with the old vSAN disks. Errors like “vSAN partition found”, “found zero SSD devices for SSD cache tier” and disks stuck at “Ineligible for use by VSAN” kept showing up.
In this blog post, I’ll demonstrate how to quickly remove old vSAN data from an ESX host.
First, a short excursion why reinstalling ESX fails in my scenario: It ESX only rewrites the boot device. It never touches the data disks. So every former vSAN cache and capacity disk still carries its old vSAN partition table and disk-group metadata from before.
This is especially painful with VCF, because the installer insists that vSAN-eligible disks be completely raw – no partitions, no disk-group metadata, no lingering storage-pool claims. Commissioning simply refuses hosts whose disks aren’t clean, and the error messages don’t always make it obvious that stale metadata is the culprit.
In my case this was a three-host vSAN cluster that I was scrapping to make room for a fresh VCF deployment. All three siblings were still running, but every one of them was getting reinstalled – the whole cluster was going away, and there was no data on the vSAN worth keeping. Normally, we would cleanly remove a disk group kicks off a resync to keep objects accessible on the remaining nodes.
The first step now is to check each disk for vSAN eligibility using the vdc command.
vdq -qOn a freshly reinstalled former vSAN host, the output looks like this:
[
{
"Name": "naa.50000f0072487070",
"VSANUUID": "529a26a4-f236-debb-95b1-96d9f22e7c67",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "0",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.50000f00724875d0",
"VSANUUID": "52280f22-c79a-7db6-4f64-22e77b05aae2",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "1",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.58ce38ee210a948d",
"VSANUUID": "52730171-18c6-ca99-7db3-d35dd5535c8c",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "1",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.58ce38ee210a94cd",
"VSANUUID": "",
"State": "Ineligible for use by VSAN",
"Reason": "Has partitions",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Has partitions",
"IsSSD": "1",
"IsCapacityFlash": "0",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.58ce38ee210a9499",
"VSANUUID": "52cf7964-0128-175e-a543-10dae3e5684f",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "1",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.58ce38ee210ad07d",
"VSANUUID": "526ede9a-f9d7-7af5-34be-3bf12a6f8f24",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "1",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.50000f00724871b0",
"VSANUUID": "52172240-16f1-2fc8-e19a-2c916fcbb7ae",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "1",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
},
{
"Name": "naa.50000f007231e010",
"VSANUUID": "529fd0fc-28d0-2761-383a-12517c8c9bc9",
"State": "Ineligible for use by VSAN",
"Reason": "Not mounted on this host",
"StoragePoolState": "Ineligible for use by Storage Pool",
"StoragePoolReason": "Disk in use by disk group",
"IsSSD": "1",
"IsCapacityFlash": "1",
"IsPDL": "0",
"Size(MB)": "3662830",
"FormatType": "512e",
"IsVsanDirectDisk": "0"
}
]A populated VSANUUID together with “Disk in use by disk group” and “Not mounted on this host” means the disk still holds a disk group belonging to a vSAN identity this reinstalled host no longer owns. A disk with an empty VSANUUID that just says “Has partitions“, on the other hand, is usually a plain VMFS datastore – often one we want to keep.
The IsCapacityFlash field tells us the role within an OSA disk group: 0 marks the flash cache device that owns the group, 1 marks a capacity device. In my eight-disk hosts I found exactly one cache disk, six capacity disks in the same stale group, and one leftover partitioned disk.
Since this is a fresh install with no data to preserve, we’ll now clear each former vSAN disk at the block level: reset the partition table, then zero the head and tail to erase the primary and backup GPT along with any vSAN signatures.
Let’s take one disk from my example – the cache device naa.50000f0072487070 from the vdq output above – and walk through it:
DISK=naa.50000f0072487070
partedUtil setptbl /vmfs/devices/disks/$DISK gpt
dd if=/dev/zero of=/vmfs/devices/disks/$DISK bs=1M count=100 conv=notrunc
dd if=/dev/zero of=/vmfs/devices/disks/$DISK bs=1M seek=3662730 count=100 conv=notrunc
partedUtil setptbl /vmfs/devices/disks/$DISK gpt The seek value for the tail wipe is roughly ten megabytes short of the disk size in MiB (mine were 3,662,830 MB disks, hence 3662730) which lands us on the backup GPT near the end of the device. A partition-table rewrite on its own usually clears vSAN OSA metadata, but the head-and-tail zeroing guarantees it for the stubborn cases, including the ESA “stale COMPUTE mode” claim that survives a plain table rewrite.
Once we’ve confirmed that works on the one disk, we can wrap it in a loop over the rest of the stale disks on the host. I like to list each disk explicitly, so there’s no chance of accidentally catching the boot device or a datastore I mean to keep (in my case I’m skipping disk naa.58ce38ee210a94cd:
for DISK in \
naa.50000f0072487070 \
naa.50000f00724875d0 \
naa.58ce38ee210a948d \
naa.58ce38ee210a9499 \
naa.58ce38ee210ad07d \
naa.50000f00724871b0 \
naa.50000f007231e010 ; do
echo "Wiping $DISK ..."
partedUtil setptbl /vmfs/devices/disks/$DISK gpt
dd if=/dev/zero of=/vmfs/devices/disks/$DISK bs=1M count=100 conv=notrunc
dd if=/dev/zero of=/vmfs/devices/disks/$DISK bs=1M seek=3662730 count=100 conv=notrunc
partedUtil setptbl /vmfs/devices/disks/$DISK gpt
doneAfter that, the disks are eligible for a fresh vSAN installation. Every wiped disk should flip to "Eligible for use by VSAN" with an empty reason. We can check it by running vdc once more:
vdq -q | grep -E '"Name"|"State"|"Reason"'That’s it for today 🙂
Leave a Reply