Our scheduled cron backup as failed 2 consecutive times since we’ve made a change to ESX on a VM. We’ve moved the drive from a smaller SSD to a spinning disk and assigned this disk 1TB which is thin provisioned (only 200 odd GB in use).
Failure 1:
2019-04-16 19:35:15.063526 INFO COMMAND_FAILURE Failed to run command ‘/bin/vim-cmd vmsvc/snapshot.removeall 1’: No space left on device
2019-04-16 19:35:15.063734 ERROR COMMAND_FAILURE Failed to run command '/bin/vim-cmd vmsvc/snapshot.removeall 1': No space left on device
Failure 2:
2019-04-17 19:32:14.623791 INFO COMMAND_FAILURE Failed to run command ‘/bin/vim-cmd vmsvc/power.getstate 2’: No space left on device
2019-04-17 19:32:14.624127 ERROR COMMAND_FAILURE Failed to run command '/bin/vim-cmd vmsvc/power.getstate 2': No space left on device
Vertical runs from the spinning disk which according to ESX has a capacity of 1.82TB, 216.97GB provisioned and 1.61TB free. This then backs up to a Synology box (SYN-01) which has 1.48TB free as per screenshot below.
I manually run the backup yesterday which worked fine so I don’t see why the cron backup would be failing. Any ideas or suggestions?!
P.s I’ve also run an Inodes check on both Syn-01 and /
[icfadmin@hv:/vmfs/volumes/4c2e0dc0-a2afa1cc-16eb-0025903c61d8/verticalbackup] stat -f /
File: “/”
ID: 100000000 Namelen: 127 Type: visorfs
Block size: 4096
Blocks: Total: 342606 Free: 185852 Available: 185852
Inodes: Total: 655360 Free: 649045
[icfadmin@hv:/vmfs/volumes/4c2e0dc0-a2afa1cc-16eb-0025903c61d8/verticalbackup] stat -f /vmfs/volumes/syn-01/
File: “/vmfs/volumes/syn-01/”
ID: 4280421435b97038 Namelen: 127 Type: nfs
Block size: 4096
Blocks: Total: 719884784 Free: 410421987 Available: 410396387
Inodes: Total: 182845440 Free: 180655669