SCDPM Troubles: "The destination to store the backup cannot be one of the source volumes"

By YellowOnline on Friday 8 November 2013 16:49 - Comments (3)
Category: SCDPM, Views: 7.523

How great to work with SCDPM, a product almost no one uses. It's not a bad product, but it isn't mature yet, not wide-spread and not on Microsoft's list of high-priority products. I hope someday the application will break out of this vicious circle.

The problem with a little used application is that there isn't as much information around on the net to help you with your problems. Ironically, most of the "Click here for more information" links in the product itself forward to a 404 on the Microsoft site. Ugh.

Anyway: today I dived into a problem I should've solved a long time ago already and I actually managed to find a solution for it. One of my DPM servers couldn't get backed up by my Disaster Recovery DPM.

The problem has actually little to do with DPM itself but rather with Windows Backup, so this is SCDPM version independent.


So anyway, I'm looking at it from a DPM Admin point of view and will reconstruct the search starting from DPM. It all starts with this annoying message in the DPM console (I use 2010 here).


http://tweakers.net/ext/f/4nnonSaXq7wSTEYFgfvFgRar/full.jpg



code:
1
2
3
Type:       Consistency check
Status:     Failed
Description:DPM cannot create a backup because Windows Server Backup (WSB) on the protected computer encountered an error (WSB Event ID: 546, WSB Error Code:  0x80780040). (ID 30229 Details: Internal error code: 0x809909FB)



Let's google for WSB Error Code 0x80780040: 33 results, all threads on fora, none explaining what the error code means. Bad start?

Let's google for DPM Error Code 0x809909FB: 51 results, pretty much the same problem as the previous search. In hindsight, there was some valuable information to be found here I must admit.

Right now though, the errors DPM throws aren't bringing us anywhere. The next obvious step is to take a look on the protected machine. DPM uses the VSS writer from Windows if Windows Backup Features are installed. I did a test here to backup to a local path and found out the problem can be replicated without any DPM involvement, so it isn't really a DPM problem. Still, we need to find a solution if we want to backup. We can't just say to the server admins "'well, it's your server, let me know when you found a solution". I can't do that in any case. It's a matter of attitude I guess.


http://tweakers.net/ext/f/bQpeAOpPO7gewQEWqS9OXlaN/full.jpg


Yup, it definitely didn't work out. When we look at the details, I just love this not resizeable window that obscures the error.


http://tweakers.net/ext/f/OCdGo16fx0lwqDet1N6MR5YR/full.jpg


Fortunately a MouseOver does show the error.


http://tweakers.net/ext/f/mYshXCRqcBEJiFyc0Wohmm9I/full.jpg



code:
1
2
The operation failed. Retry the operation. If the issue persists, delete and then recreate scheduled backup settings. 
The destination to store the backup cannot be one of the source volumes.



Hits on Google (or Bing or ...): 0. Yay.

There's only one thing left to check: the Event Log. First you need to know where Windows Backup places its events.


http://tweakers.net/ext/f/Xt7sl70F3sx0rffSByvrxeut/full.jpg



code:
1
Event Viewer/Applications and Services Logs/Microsoft/Windows/Backup/Operational



And what do we see now: another error code!



code:
1
The backup operation attempted at '‎2013‎-‎11‎-‎08T13:44:04.855867200Z' has failed to start, error code '2155348032'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.



Now we'll run this new error code through our favourite search engine. This time 37 hits, and the 5 first are bingo, even though only the 5th one (in my case in any case) refers to an official Microsoft KB 944530: Error message when you try to perform a system state backup in Windows Server 2008 and Windows Server 2008 R2.

It's all about having a DWORD called AllowSSBToAnyVolume with a value set to 1. I checked my other DPM servers (remember, I'm backing up DPM servers with a DR DPM) and that value indeed existed on all servers except on the one that had trouble, even though I had done a reinstall of the Windows Backup Features.

tl;dr
So the solution for problem codes 0x80780040, 0x809909FB and 2155348032 is (probably)... go to regedit, expand to...



code:
1
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\wbengine



Add a key SystemStateBackup if it wouldn't exist and create a DWORD AllowSSBToAnyVolume with the value 1.


http://tweakers.net/ext/f/jzMDdU534ddNkaCJ2pbCxquW/full.jpg


Now try your backup again. No need to restart a service (it's a manual one) or rebooting the server. If all goes well, you have no more trouble locally...


http://tweakers.net/ext/f/V6mFKOILdRXHQA6TtDAccQpr/full.jpg


http://tweakers.net/ext/f/tgsWWhrm2ybJbU1SSGVNiTKj/full.jpg


... and after retrying to create a Recovery Point or a Consistency Check DPM will be ahppy too...


http://tweakers.net/ext/f/AaGPBRwsC1F8Sj2AAxrG5fRA/full.jpg


I hope this could be useful to someone. :)

Volgende: YellowOnline @ TheDailyWTF 11-'13 YellowOnline @ TheDailyWTF
Volgende: Adtadmin "Access Denied" when setting up SCOM 2012SP1 ACS 09-'13 Adtadmin "Access Denied" when setting up SCOM 2012SP1 ACS

Comments


By Tweakers user Monkeybrains, Friday 8 November 2013 19:45

Great blog and good to see someone else is using SCDPM and is running into issues as well :)

The biggest problem I have with DPM is the fact that sometimes, for no apparent reason, backups fail.It simply isn't reliable most of the times.

Usually a consistency check solves problems but sometimes you have to drop the protected resource (thus deleting all your backups) and add them again to a protection group.

I have 5 Hyper-V hosts, 3 with Windows Server 2008 R2, 2 with Windows Server 2012 (which is a huge improvement for Hyper-V). 2 of the 2008 R2 hosts and the 2012 hosts can be backupped without any issues, but 1 2008 R2 host works for about a month and then refuses to create recovery points for the VM's on the server. Sometimes a reboot of the Hyper-V host helps, sometimes dropping and adding the VM to the protection group.

Oh well, that particular 2008 R2 host is due to be upgraded to Windows Server 2012 after we migrate the VM's running on it to a new 2012 Hyper-V host. I hope that solves the backup issues.

By Tweakers user YellowOnline, Friday 8 November 2013 20:30

@Monkeybrains
I backup a lot with DPM: 500+ servers BMR/SS, 50+ SQL instances clustered and unclustered, 1000+ datasources, 1 DAG consisting of 8 mailbox servers, 4 Sharepoint farms,... but no Hyper-V unfortunately :) Our infrastructure is VMware and I don't think I'll ever convince the VM-team to put the ESX hypervisor on top of the Hyper-V one (which is possible as far as I know). So unfortunately I can't help you with Hyper-V issues.

But...

I have had a similar issue with Sharepoint by the way. When you remove the protected object from a Protection Group, just retain the data. When you add it again, it will find its previous backups and after a consistency check things should be all-right again. Did you try this yet?

By Tweakers user Monkeybrains, Friday 8 November 2013 21:28

@YellowOnline

I tried that as well, worked for about a month and the problem returned :( Even upgrading DPM from 2010 to 2012 didn't help. I am convinced there is something on the 2008 R2 host that causes this problem but I haven't been able to figure it out yet.

Thanks for the suggestion.

Comments are closed