When preparing a System Center 2016 Virtual Machine Manager (4.0.1662.0) environment for bare-metal deployment I walked into the following issue when adding the PXE server. This VMM installation is highly available, in other words: Clustered.
Issue
Error in VMM:
Error (21150)
Failed to customize WinPE Image.
Recommended Action
Verify that Virtual Machine Manager and the Windows Assessment and Deployment Kit are installed correctly.
Failed to customize WinPE Image.
Recommended Action
Verify that Virtual Machine Manager and the Windows Assessment and Deployment Kit are installed correctly.
Error when enabling debug logging:
4
2
private\product\engine\PxeServer\PxeServerTask.cs
164
CarmineException [ex#5d3] caught by RunSubtask Failed to add PXE server win4870vm.domain.local, attempt removal (catch Exception) [[(CarmineException#[3fc]) { Microsoft.VirtualManager.Utils.CarmineException: Failed to customize WinPE Image.
Verify that Virtual Machine Manager and the Windows Assessment and Deployment Kit are installed correctly. ---> System.InvalidOperationException: Windows cannot verify the digital signature for this file. A recent hardware or software change might have installed a file that is signed incorrectly or damaged, or that might be malicious software from an unknown source
--- End of inner exception stack trace ---
at Microsoft.VirtualManager.Engine.PxeServer.CustomWinPE.Mount()
at Microsoft.VirtualManager.Engine.PxeServer.CustomWinPE.Customize(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.PxeServer.PublishWindowsPE.RunSubtask(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.PxeServer.AddPxeServerTask.RunSubtask(IVmmDbConnection dbConnection)
*** Carmine error was: PublishWinPECustomizationError (21150)
[s#ed3ca0] Task`1.SubtaskRun(this: (Task`1<AddPxeServerTask>#[c]) { TBD }, TaskID: (guid) 603832e3-9ac4-4196-833e-44008cd6afae) in t:\btvsts\2811\private\product\engine\TaskRepository\Task.cs:line 254
[s#92070a] VMRefresherBase.UpdateFullVMObjectToCarmine(this: (VMRefresherBase#[32]){ TBD }, vmComputer: (VirVMComputerSystem#[1ed]) { tbd }, vmData: (VMData#[10f3452]) { id = 8dd6f1c6-7c95-4428-b0aa-d31e4933d691, LastUpdatedTimestamp = 11/12/2017 11:02:50 PM, objectType = VM, ObjectState = 0, name = "WIN4814VM", Enabled = True, Accessibility = Public, description = "", CreationTime = 10/23/2017 9:49:36 AM, modifiedTime = 11/12/2017 11:02:50 PM, GrantedToList = Ø, vmHostId = f1f79ca2-a14e-4930-9bdb-4ded54f0da72, hostType = None, Location = "C:\ProgramData\Microsoft\Windows\Hyper-V", hardwareConfigData = (HardwareConfigData#[10f3453]) { id = f1a843ab-50e0-428b-9f5e-1c51e113ccb6, LastUpdatedTimestamp = 11/14/2017 1:03:30 AM, objectType = HardwareProfile, Enabled = True, Accessibility = Public, CreationTime = 11/14/2017 1:03:30 AM, modifiedTime = 11/14/2017 1:03:30 AM, GrantedToList = Ø }, SANCloneCapable = False, VirtualDiskDrives = Ø, StartAction = AlwaysAutoTurnOnVM, StopAction = SaveVM, RunGuestCredentialInput = False, DelayStart = (Int32?) 0 (0x0), OSId = (guid?) b808453f-f2b5-451f-894f-001c49db255a, VMAddition = (string) "10.0.14393", QuotaPoint = 1, Tag = "(none)", VsvVirtualServerVersion = (Version#a00396ea) 10.0.14393.1770, Custom = (String[][9]#3780bda) { (null), (null), (null), (null), (null), (null), (null), (null), (null) }, customPropertyIDs = Ø, UseHardwareAssist = (bool?) True, sanStatus = (List`1<ErrorInfo>#20b5da0) { n: 1[4] (ErrorInfo#38cd172) DeployLUNNotFoundUsingStorageProvider (26207) }, VirtualDiskDriveIds = (List`1<Guid>#3b5714e) { n: 1[4] c237cccf-f6fd-447d-92c6-6a735b5558cf }, VMCheckpointIds = Ø, VMShapshotIds = Ø, vmResource = (ClusterResourceData#[3f5d17]) { type = Unknown, status = ClusterResourceStateUnknown }, vmConfigResource = (ClusterResourceData#[3f5d18]) { type = Unknown, status = ClusterResourceStateUnknown }, ApplicationIds = Ø, MostRecentTaskID = c31993a5-0f65-4671-951c-68ce8dc7e963, MostRecentTaskUIState = Completed, RefresherErrorByteData = (Byte[][202]#3847a7b) { [0..15] 64, 16, 65, 114, 114, 97, 121, 79, 102, 69, 114, 114, 111, 114, 73, 110 }, refresherErrors = Ø, VirtualMachineState = Running, TotalSize = 18761121792, TotalMaxSize = 32212254720, CurrentAvailableMemoryPercent = 89, AvailableMemoryBuffer = 4211, HasPassthroughDisk = False, VirtualizationPlatform = HyperV, ParentId = "", RootVMId = 00000000-0000-0000-0000-000000000000, VMId = "36D0B984-090F-4394-9FEC-138283B1B7F8", VmcLocation = "C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\36D0B984-090F-4394-9FEC-138283B1B7F8.vmcx", cpuUtilization = 0, perfCPUUtilization = -1, perfMemory = -1, perfDiskBytesRead = -1, perfDiskBytesWrite = -1, perfNetworkBytesRead = -1, perfNetworkBytesWrite = -1, cpuTieredPerfCounterID = 713c82ec-7f6b-4074-a7ed-344b9707d480, memoryTieredPerfCounterID = 0fb780ef-1ec7-4d4e-a755-03ef8412c546, diskBytesReadTieredPerfCounterID = 6e0d025e-cb1f-4392-ba74-224a69f53a5d, diskBytesWriteTieredPerfCounterID = a37b4168-eaac-4567-accc-bf2f844c8aed, networkBytesReadTieredPerfCounterID = 8b7ac0df-12fc-41a0-bb5e-5a7c3376c9f4, networkBytesWriteTieredPerfCounterID = 8ad9f833-caf7-471f-bf70-44280de1b4f6, SrcObjectType = Job, OSShutdownEnabled = True, TimeSynchronizationEnabled = False, DataExchangeEnabled = True, HeartbeatEnabled = True, BackupEnabled = True, CheckpointLocation = "C:\ProgramData\Microsoft\Windows\Hyper-V", LastRestoredCheckpointID = "36D0B984-090F-4394-9FEC-138283B1B7F8", isUndergoingLiveMigration = False, IsFaultTolerant = False, IsCloned = False, IsModifiedAfterExport = False, DRState = Disabled, IsPrimaryVM = (bool) False, IsRecoveryVM = (bool) False, IsExtendedRecoveryVM = (bool) False, IsTestReplicaVM = (bool) False }, updateRequired: UpdateRequired.UpdateObject, vmObjectHasChanged: (Boolean#0) False, dbConnection: (TransactableDbContext#[882fe]) { constructionCallSite = (CallSite) { SqlContext.Connect }, constructionIdentity = (WindowsIdentity#2069def) System.Security.Principal.WindowsIdentity }) in t:\btvsts\2811\private\product\engine\BitBoS\VmRefresher.cs:line 3313
Full call stack from when the exception was thrown: at Microsoft.VirtualManager.Engine.PxeServer.PublishWindowsPE.RunSubtask(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.PxeServer.AddPxeServerTask.RunSubtask(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.DB.SqlContext.Connect(Action`1 action)
at Microsoft.VirtualManager.Engine.TaskRepository.Task`1.SubtaskRun(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
; error = (ErrorInfo#299beb7) PublishWinPECustomizationError (21150) }]]
EV_RenderedValue_5.00
2
private\product\engine\PxeServer\PxeServerTask.cs
164
CarmineException [ex#5d3] caught by RunSubtask Failed to add PXE server win4870vm.domain.local, attempt removal (catch Exception) [[(CarmineException#[3fc]) { Microsoft.VirtualManager.Utils.CarmineException: Failed to customize WinPE Image.
Verify that Virtual Machine Manager and the Windows Assessment and Deployment Kit are installed correctly. ---> System.InvalidOperationException: Windows cannot verify the digital signature for this file. A recent hardware or software change might have installed a file that is signed incorrectly or damaged, or that might be malicious software from an unknown source
--- End of inner exception stack trace ---
at Microsoft.VirtualManager.Engine.PxeServer.CustomWinPE.Mount()
at Microsoft.VirtualManager.Engine.PxeServer.CustomWinPE.Customize(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.PxeServer.PublishWindowsPE.RunSubtask(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.PxeServer.AddPxeServerTask.RunSubtask(IVmmDbConnection dbConnection)
*** Carmine error was: PublishWinPECustomizationError (21150)
[s#ed3ca0] Task`1.SubtaskRun(this: (Task`1<AddPxeServerTask>#[c]) { TBD }, TaskID: (guid) 603832e3-9ac4-4196-833e-44008cd6afae) in t:\btvsts\2811\private\product\engine\TaskRepository\Task.cs:line 254
[s#92070a] VMRefresherBase.UpdateFullVMObjectToCarmine(this: (VMRefresherBase#[32]){ TBD }, vmComputer: (VirVMComputerSystem#[1ed]) { tbd }, vmData: (VMData#[10f3452]) { id = 8dd6f1c6-7c95-4428-b0aa-d31e4933d691, LastUpdatedTimestamp = 11/12/2017 11:02:50 PM, objectType = VM, ObjectState = 0, name = "WIN4814VM", Enabled = True, Accessibility = Public, description = "", CreationTime = 10/23/2017 9:49:36 AM, modifiedTime = 11/12/2017 11:02:50 PM, GrantedToList = Ø, vmHostId = f1f79ca2-a14e-4930-9bdb-4ded54f0da72, hostType = None, Location = "C:\ProgramData\Microsoft\Windows\Hyper-V", hardwareConfigData = (HardwareConfigData#[10f3453]) { id = f1a843ab-50e0-428b-9f5e-1c51e113ccb6, LastUpdatedTimestamp = 11/14/2017 1:03:30 AM, objectType = HardwareProfile, Enabled = True, Accessibility = Public, CreationTime = 11/14/2017 1:03:30 AM, modifiedTime = 11/14/2017 1:03:30 AM, GrantedToList = Ø }, SANCloneCapable = False, VirtualDiskDrives = Ø, StartAction = AlwaysAutoTurnOnVM, StopAction = SaveVM, RunGuestCredentialInput = False, DelayStart = (Int32?) 0 (0x0), OSId = (guid?) b808453f-f2b5-451f-894f-001c49db255a, VMAddition = (string) "10.0.14393", QuotaPoint = 1, Tag = "(none)", VsvVirtualServerVersion = (Version#a00396ea) 10.0.14393.1770, Custom = (String[][9]#3780bda) { (null), (null), (null), (null), (null), (null), (null), (null), (null) }, customPropertyIDs = Ø, UseHardwareAssist = (bool?) True, sanStatus = (List`1<ErrorInfo>#20b5da0) { n: 1[4] (ErrorInfo#38cd172) DeployLUNNotFoundUsingStorageProvider (26207) }, VirtualDiskDriveIds = (List`1<Guid>#3b5714e) { n: 1[4] c237cccf-f6fd-447d-92c6-6a735b5558cf }, VMCheckpointIds = Ø, VMShapshotIds = Ø, vmResource = (ClusterResourceData#[3f5d17]) { type = Unknown, status = ClusterResourceStateUnknown }, vmConfigResource = (ClusterResourceData#[3f5d18]) { type = Unknown, status = ClusterResourceStateUnknown }, ApplicationIds = Ø, MostRecentTaskID = c31993a5-0f65-4671-951c-68ce8dc7e963, MostRecentTaskUIState = Completed, RefresherErrorByteData = (Byte[][202]#3847a7b) { [0..15] 64, 16, 65, 114, 114, 97, 121, 79, 102, 69, 114, 114, 111, 114, 73, 110 }, refresherErrors = Ø, VirtualMachineState = Running, TotalSize = 18761121792, TotalMaxSize = 32212254720, CurrentAvailableMemoryPercent = 89, AvailableMemoryBuffer = 4211, HasPassthroughDisk = False, VirtualizationPlatform = HyperV, ParentId = "", RootVMId = 00000000-0000-0000-0000-000000000000, VMId = "36D0B984-090F-4394-9FEC-138283B1B7F8", VmcLocation = "C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\36D0B984-090F-4394-9FEC-138283B1B7F8.vmcx", cpuUtilization = 0, perfCPUUtilization = -1, perfMemory = -1, perfDiskBytesRead = -1, perfDiskBytesWrite = -1, perfNetworkBytesRead = -1, perfNetworkBytesWrite = -1, cpuTieredPerfCounterID = 713c82ec-7f6b-4074-a7ed-344b9707d480, memoryTieredPerfCounterID = 0fb780ef-1ec7-4d4e-a755-03ef8412c546, diskBytesReadTieredPerfCounterID = 6e0d025e-cb1f-4392-ba74-224a69f53a5d, diskBytesWriteTieredPerfCounterID = a37b4168-eaac-4567-accc-bf2f844c8aed, networkBytesReadTieredPerfCounterID = 8b7ac0df-12fc-41a0-bb5e-5a7c3376c9f4, networkBytesWriteTieredPerfCounterID = 8ad9f833-caf7-471f-bf70-44280de1b4f6, SrcObjectType = Job, OSShutdownEnabled = True, TimeSynchronizationEnabled = False, DataExchangeEnabled = True, HeartbeatEnabled = True, BackupEnabled = True, CheckpointLocation = "C:\ProgramData\Microsoft\Windows\Hyper-V", LastRestoredCheckpointID = "36D0B984-090F-4394-9FEC-138283B1B7F8", isUndergoingLiveMigration = False, IsFaultTolerant = False, IsCloned = False, IsModifiedAfterExport = False, DRState = Disabled, IsPrimaryVM = (bool) False, IsRecoveryVM = (bool) False, IsExtendedRecoveryVM = (bool) False, IsTestReplicaVM = (bool) False }, updateRequired: UpdateRequired.UpdateObject, vmObjectHasChanged: (Boolean#0) False, dbConnection: (TransactableDbContext#[882fe]) { constructionCallSite = (CallSite) { SqlContext.Connect }, constructionIdentity = (WindowsIdentity#2069def) System.Security.Principal.WindowsIdentity }) in t:\btvsts\2811\private\product\engine\BitBoS\VmRefresher.cs:line 3313
Full call stack from when the exception was thrown: at Microsoft.VirtualManager.Engine.PxeServer.PublishWindowsPE.RunSubtask(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.PxeServer.AddPxeServerTask.RunSubtask(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.Engine.TaskRepository.SubtaskBase.Run(IVmmDbConnection dbConnection)
at Microsoft.VirtualManager.DB.SqlContext.Connect(Action`1 action)
at Microsoft.VirtualManager.Engine.TaskRepository.Task`1.SubtaskRun(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
; error = (ErrorInfo#299beb7) PublishWinPECustomizationError (21150) }]]
EV_RenderedValue_5.00
Solution
Disable “Secure boot” on the VMM VMs, something is blocked that doesn’t need to be blocked
Happy PXE’ing! 🙂