Crashplan Pro Issue



  • Ran into a little issue with Crashplan Pro (or Business, whatever it is called now) today.

    I have a user who has not been backed up in 9 days. After confirming the machine was online, I logged into his machine remotely to take a look. The CPP client was not connecting to their server.

    I live chatted with them, and found this out...

    CPP: "Thanks. Give me a minute to look into it from my end. I am thinking this is related to an issue with ojne of our storage servers, but I want to confirm.
    Yes, it looks like this is on our end. This archive is stored on https://dqh-sea.crashplanpro.com. We are in the process of moving all archives off of this serrver. This process will still take some time to complete, but once the archive has moved, the backup would resume."

    ME (Paraphrased): Uh, that's ridiculous for a real-time backup service. What if there is data loss?

    CPP: "I agree. I would recommend setting up a local backup while we are having this issue. https://support.code…acking_Up_To_An_External_Drive

    ME: Ranting about how bad this is for users backing up data.

    CPP: "We are in the process of moving all archives on that storage node to other servers. This process takes time to complete.
    I can understand your frustration. If I had a way to move this process along faster, I would.
    I can understand that. It did appear that this storage node was causing issues for many other users as well.

    ME: "Surely there was an e-mail or something to alert users that they have no good backups for over a week now?"

    CPP: "This is an issue that is still under investigation, and no email was sent"

    So anyway, if you use CPP, and you are having backup issues, you might want to give them a call.



  • Whoa. Having issues is fine, taking a while is fine... but not alerting people!!



  • @scottalanmiller said in Crashplan Pro Issue:

    Whoa. Having issues is fine, taking a while is fine... but not alerting people!!

    "Taking a while" <> 9 days



  • Umm checking my backups now.....



  • @Minion-Queen said in Crashplan Pro Issue:

    Umm checking my backups now.....

    Might just be that one server.

    My other users were fine.



  • @BRRABill said in Crashplan Pro Issue:

    @Minion-Queen said in Crashplan Pro Issue:

    Umm checking my backups now.....

    Might just be that one server.

    My other users were fine.

    Yup I am fine. PHEW!!!! Thanks for letting us know though.



  • Hm.



  • Yikes. I remember having to wait out some kind of repository maintenance on their end at one point but do not remember how long it lasted to be honest. Good luck, and I think we're all hoping for no data loss.



  • wow - moving users to another server? shouldn't that be as easy as - hey client, you're now backing up to this server again - you get to do a full download again, have a nice day.



  • Now that I think about it, I tried restoring a file for him the other day, and it wouldn't load, so we gave up.

    I bet that is a related problem.

    Really just terrible...



  • @BRRABill said in Crashplan Pro Issue:

    Now that I think about it, I tried restoring a file for him the other day, and it wouldn't load, so we gave up.

    I bet that is a related problem.

    Of course it is, the repository is offline, how would you even think that it could download a file?



  • @JaredBusch said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    Now that I think about it, I tried restoring a file for him the other day, and it wouldn't load, so we gave up.

    I bet that is a related problem.

    Of course it is, the repository is offline, how would you even think that it could download a file?

    Well I was unaware of that news at the time I was trying to restore.



  • @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    Now that I think about it, I tried restoring a file for him the other day, and it wouldn't load, so we gave up.

    I bet that is a related problem.

    Of course it is, the repository is offline, how would you even think that it could download a file?

    Well I was unaware of that news at the time I was trying to restore.

    I was mostly replying to

    I bet that is a related problem



  • @JaredBusch said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    Now that I think about it, I tried restoring a file for him the other day, and it wouldn't load, so we gave up.

    I bet that is a related problem.

    Of course it is, the repository is offline, how would you even think that it could download a file?

    Well I was unaware of that news at the time I was trying to restore.

    I was mostly replying to

    I bet that is a related problem

    Oh, I forgive you then.

    Carry on...



  • @BRRABill And that is why I tend to not recommend CPP. Far from the first time I've heard of something like this, hopefully everything transfers ok, I've heard some horror storied about it not working and the entire archive being lost.



  • @travisdh1 said in Crashplan Pro Issue:

    @BRRABill And that is why I tend to not recommend CPP. Far from the first time I've heard of something like this, hopefully everything transfers ok, I've heard some horror storied about it not working and the entire archive being lost.

    Links and proof instead of hearsay and conspiracy theories.



  • @BRRABill said in Crashplan Pro Issue:

    ME: "Surely there was an e-mail or something to alert users that they have no good backups for over a week now?"

    CPP: "This is an issue that is still under investigation, and no email was sent"

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    This has nothing to do with Code42 choosing or not choosing to send an email out. This is enabled by default and also I believe it is set to 7 days be default. I do not recall as I always set it to 1 and 3 as below.

    0_1485909645173_upload-e26fa378-b76f-4cf1-a720-dba89b4e9153

    0_1485909714372_upload-f3095464-32ee-4b4d-9e02-57f6b1c762a1



  • @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.



  • @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.

    Guess you need to change that setting to every one day then.



  • @Dashrender said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.

    Guess you need to change that setting to every one day then.

    It doesn't change the point you can't do backups or restores.

    What difference does it make if I get 9 days of e-mails telling me the service is broken?



  • @BRRABill said in Crashplan Pro Issue:

    @Dashrender said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.

    Guess you need to change that setting to every one day then.

    It doesn't change the point you can't do backups or restores.

    What difference does it make if I get 9 days of e-mails telling me the service is broken?

    Because you know something failed immediately and can take action as required by how critical the information is.



  • @BRRABill said in Crashplan Pro Issue:

    @Dashrender said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.

    Guess you need to change that setting to every one day then.

    It doesn't change the point you can't do backups or restores.

    What difference does it make if I get 9 days of e-mails telling me the service is broken?

    Things break - that's fine. and while you work to repair them, you decide how else you're going to protect them, if there's an option for that.

    For example, if you were make aware 9 days ago, you could have told you users to backup daily to USB stick until the problem is resolved, or you could have moved them to another service. Instead, you're now out a restore opportunity, and still have a broken system.



  • @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.

    If you did get this, then why did you try to restore something when you knew it was not working? You make completely no sense here.

    And it is nothing like a house burning down. This is a backup failure. It affects nothing under normal circumstances. That is why it is a backup.



  • @Dashrender said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    @Dashrender said in Crashplan Pro Issue:

    @BRRABill said in Crashplan Pro Issue:

    @JaredBusch said

    While this is not an optimal situation, you should have received an email from your system informing you that it had not reached the backup target in X days.

    I did, and that is why I contacted them.

    However, that's like getting an e-mail that your house burned down. Not particularly helpful and nothing can be done about it.

    Guess you need to change that setting to every one day then.

    It doesn't change the point you can't do backups or restores.

    What difference does it make if I get 9 days of e-mails telling me the service is broken?

    Things break - that's fine. and while you work to repair them, you decide how else you're going to protect them, if there's an option for that.

    For example, if you were make aware 9 days ago, you could have told you users to backup daily to USB stick until the problem is resolved, or you could have moved them to another service. Instead, you're now out a restore opportunity, and still have a broken system.

    The CrashPlan application has settings for this even in the home version. He could have easily set it to backup to a local media the first day a notice was received.



  • You two are both crazy.

    Not sure how you can defend this.

    Has nothing to do with what I could or could not have done.

    They should have alerted their customers, and to be honest this never should happen in the first place.



  • @BRRABill said in Crashplan Pro Issue:

    You two are both crazy.

    Not sure how you can defend this.

    Has nothing to do with what I could or could not have done.

    They should have alerted their customers, and to be honest this never should happen in the first place.

    I'll give you that once they made the decision to move customers to new servers, YES, they should have contacted you - but contact as soon as the failure happened, nope, can't give that to you. Because things happen, they just need to be fixed. And as JB said, this is backup, not production, but after a few day of downtime, considering the default is no notice until 9 days of failed backup - OK they should have notified you on day 3 if not day two.



  • It's not 9 days.

    I got alerts after 3 days. This is a remote user who doesn't always have his computer on. Wasn't anything way out of the norm until I noticed it a few days in a row, and decided to check in on it.

    And to be honest, it would be more an issue for restores.

    Which to be honest I am not even sure is related. But it sure makes sense.



  • @BRRABill said in Crashplan Pro Issue:

    You two are both crazy.

    Not sure how you can defend this.

    Has nothing to do with what I could or could not have done.

    They should have alerted their customers, and to be honest this never should happen in the first place.

    Whether or not they should have alerted their customers is honestly a completely separate issue. Your issue is completely dumping on them when the mitigation is still on all you for not getting handled.

    Even had it been announced, most customers would have just bitched and waited it out because they sure as fuck were not going to reseed (i suspect) terabytes of data or more, if they could even figure out how to force their data to a new server themselves in the first place.

    Should they have made some kind of notice? Yes. Does not change the prior assumption from almost certainly being a fact.

    If it was my application, I would have designed a pop up notice into it that I can trigger for such events.

    From the outside, I also wonder why data is down for such a long time, this is true. To my understanding my data is stored in some encrypted form. So, to me, that means the worst thing that should happen is that the server is dead and users are offline while their data is moved and will come back online once their blob has moved.
    Maybe they just executed an rsync and there is no prioritization. I would have done some kind of little guy to largest guy scenario in order to keep the longer impact to the smallest number of users.

    But again, all of that said, you still are bitching about not performing your own job of handling a backup failure.



  • @BRRABill said in Crashplan Pro Issue:

    It's not 9 days.

    I got alerts after 3 days. This is a remote user who doesn't always have his computer on. Wasn't anything way out of the norm until I noticed it a few days in a row, and decided to check in on it.

    And to be honest, it would be more an issue for restores.

    Which to be honest I am not even sure is related. But it sure makes sense.

    OMFG how can you even say that you are not sure a restore is related? Seriously?



  • @JaredBusch said

    But again, all of that said, you still are bitching about not performing your own job of handling a backup failure.

    It's a free world. Thanks for your opinion.