Multiple snapshots cause inconsistent parent UUID in vdi causing VER_FILE_NOT_FOUND error

[jbrown10]

Problem: Snapshotting and deleting of old snapshots in a running VM results in the retention of a reference to a deleted snapshot, inhibiting the resume of saved state.

Setup and process of snapshotting and snapshot removal:

I have a situation where I need to run a long set of analytics in a guest VM. I constructed a single base VM that installed all the software and base set of data. I then created 4 linked clones of that base VM. In each clone, I then configure the specific parameters of an algorithm. The execution of each set of analysis is approximately 1 month.

To protect myself from unexpected problems that require a complete restart of the analysis, I began to use snapshots twice a day to capture the current state so if the VM crashed for any reason (e.g. power outage or the host becoming frozen), I have a state that I can revert to and resume, losing at most 12 hours of analysis.

After a couple of days, I decided to clean up a few of the older snapshots (using the UI to remove old snapshots in the order of oldest to newest until I had only one snapshot left and the current state for each clone).

I performed the snapshot operation one each of the running clones and I conducted the snapshot removal against the running clones. In other words, as the VM was running, I began with the oldest snapshot and used the GUI to do the removal (merge and delete). All worked well. The VM continued to run and I was able to take another snapshot and then remove the older snapshot. However, when I decided to do maintenance to the host, the problem emerged.

I saved the state of each running clone, conducted my host maintenance (basic patching and rebooted the host). When I went to restart the 4 clones, each reports that there is a missing snapshot. The snapshot is the immediate parent which is the snapshot just prior to the current snapshot. Error message below.

Error:
Could not open the medium '/home/jpb/VirtualBox VMs/OT-Strat-Dev Clone1/Snapshots/{fe2f4eca-c178-4a2d-9092-8c82d6bab0b0}.vdi'.
VD: error VERR_FILE_NOT_FOUND opening image file '/home/jpb/VirtualBox VMs/OT-Strat-Dev Clone1/Snapshots/{fe2f4eca-c178-4a2d-9092-8c82d6bab0b0}.vdi' (VERR_FILE_NOT_FOUND).


Result Code: 
NS_ERROR_FAILURE (0x80004005)
Component: 
MediumWrap
Interface: 
IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda}

The missing snapshot that it cannot find I believe to be the most recently removed snapshot from the running VM prior to saving the state.

I've attached 2 views from the Virtual Media Manager. View 1 shows the child of Clone1 which represents the current state (I believe). It also shows its parent "missing". View 2 is the same but I highlighted the "missing" snapshot.

The third attachment is from the VBox Manager showing the snapshots for Clone1.

Doing a showmediuminfo command against the lowest level snapshot (current state), it shows that the Parent UUID as the missing snapshot (which was deleted via the GUI after it merged it with the base).

jpb@explorer:~/VirtualBox VMs/OT-Strat-Dev Clone1/Snapshots$ vboxmanage showmediuminfo f1a42411-6f9a-44ae-be07-00ae7de93f89
UUID:           f1a42411-6f9a-44ae-be07-00ae7de93f89
Parent UUID:    fe2f4eca-c178-4a2d-9092-8c82d6bab0b0
State:          created
Type:           normal (differencing)
Auto-Reset:     off
Location:       /home/jpb/VirtualBox VMs/OT-Strat-Dev Clone1/Snapshots/{f1a42411-6f9a-44ae-be07-00ae7de93f89}.vdi
Storage format: VDI
Format variant: differencing default
Capacity:       81920 MBytes
Size on disk:   678 MBytes
Encryption:     disabled
In use by VMs:  OT-Strat-Dev Clone1 (UUID: 640b91f3-4748-480a-8d8e-51c576201b5e)

However, when I use the dumphdinfo, I get a different value for the uuidParent - which is actually the higher (and correct) UUID of the surviving and recent snapshot.

jpb@explorer:~/VirtualBox VMs/OT-Strat-Dev Clone1/Snapshots$ vboxmanage internalcommands dumphdinfo {f1a42411-6f9a-44ae-be07-00ae7de93f89}.vdi
--- Dumping VD Disk, Images=1
Dumping VD image "{f1a42411-6f9a-44ae-be07-00ae7de93f89}.vdi" (Backend=VDI)
Dumping VDI image "{f1a42411-6f9a-44ae-be07-00ae7de93f89}.vdi" mode=r/o uOpenFlags=9 File=0x00563f4bf61680
Header: Version=00010001 Type=4 Flags=0 Size=85899345920
Header: cbBlock=1048576 cbBlockExtra=0 cBlocks=81920 cBlocksAllocated=676
Header: offBlocks=1048576 offData=2097152
Header: Geometry: C/H/S=1024/255/63 cbSector=512
Header: uuidCreation={f1a42411-6f9a-44ae-be07-00ae7de93f89}
Header: uuidModification={eff43f9a-8441-4e7b-9956-17fad78601a5}
Header: uuidParent={3f80396b-40b9-4c17-aa02-8013bf5a3ea6}
Header: uuidParentModification={44e04488-2274-4b00-92ac-1d0c0a02d180}
Image:  fFlags=00020000 offStartBlocks=1048576 offStartData=2097152
Image:  uBlockMask=000FFFFF cbTotalBlockData=1048576 uShiftOffset2Index=20 offStartBlockData=0

I haven't found the way to change the uuid that shows up in showmediuminfo to match the uuidParent of the dumphdinfo. I believe this would resolve my problem and allow me to restore the saved state of my clones.

My snapshot directory also interestingly has multiple saved state files which I assume happens as part of the snapshotting of a running VM since the date and time stamp corresponds to the snapshotting events. So, these do not look to be cleaned up by the snapshot process.

jpb@explorer:~/VirtualBox VMs/OT-Strat-Dev Clone1/Snapshots$ ls -l
total 11186860
-rw------- 1 jpb jpb  905360735 May 10 16:03 2017-05-10T21-03-44-944351000Z.sav
-rw------- 1 jpb jpb  872413499 May 20 07:54 2017-05-20T12-53-55-899688000Z.sav
-rw------- 1 jpb jpb  970377870 May 20 17:29 2017-05-20T22-28-49-589899000Z.sav
-rw-r--r-- 1 jpb jpb 7647264768 May 20 07:55 {3f80396b-40b9-4c17-aa02-8013bf5a3ea6}.vdi
-rw------- 1 jpb jpb  352321536 May 17 09:55 {7b0b9120-8a8d-46a1-a82b-fec4448e9b9f}.vdi
-rw------- 1 jpb jpb  710934528 May 20 17:28 {f1a42411-6f9a-44ae-be07-00ae7de93f89}.vdi

My current option is to revert to the snapshot, discarding the current state and reprocess the work from the last 12 hours. I'll save clone1 as is incase you need more information.