Support

Akeeba Backup for Joomla!

#28764 Automated backups keep failing

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by dlb on Tuesday, 28 November 2017 12:07 CST

derekkoopowitz
Please look at the bottom of this page (under Support Policy Summary) for our support policy summary, containing important information regarding our working hours and our support policy. Thank you!

EXTREMELY IMPORTANT: Please attach a ZIP file containing your Akeeba Backup log file in order for us to help you with any backup or restoration issue. If the file is over 2Mb, please upload it on your server and post a link to it.

Description of my issue:

I've noticed over the past month or so that the automated backups are failing more often than not. I ran a manual backup the other day and it worked ok. I've attached one of the smaller log files for you to look over. I looked at the other log files that failed, and they all seem to fail when it tries to "load from the database".

Any help on this would be appreciated. I have a large site (over 5gb of files to backup)...

Best regards,
-Derek

derekkoopowitz
In fact, it looks like that as soon as the file size went above 5gb (for the backup), the backups started to fail more often than not. When it was below 5gb, the backups worked every day. I'm not sure if this is a reason or not, but it seems suspicious to me.

dlb
Derek,

Do you have multiple backup profiles set up? The log you posted has a part size of 500 Mb, the backup can't create a part larger than that. If you do have a part larger than that, then we have a problem with the backup reading the profile.

If you mean the total of all the parts is larger then 5 Gb, then you're running out of disk space. There is no such thing as an unlimited disk, so there is no such thing as unlimited disk space. I don't care what your host tells you. You should be able to see a disk space error at the end of the failed logs if that is happening.

Do you post process the backup archive to cloud storage or leave it on the site?


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
Hi Dale,

I only have one backup profile. The part size is configured at 512mb - the overall size of the backup is over 5gb.

If you think I'm running out of disk space then I'll contact my host and find out - they do say unlimited space for the website, so we'll see how unlimited the unlimited space is.

I don't post-process the backup to cloud - I leave it on my host.

I'll let you know what they say about the storage.

Thanks for the tip.

-Derek

dlb
Derek,

They may allocate disk space in chunks, so when you run out, they allocate another chunk. But the backup is already dead at that point.

You may also find an inode limit in the fine print. That is a limit on the number of files and folders that you can have on your site. There has to be a limit somewhere. The inode limit is widely used to limit disk space and advertise "unlimited" space. Technically it is unlimited.

We might be able to bypass the problem by moving the backup to cloud storage in post processing.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
Hi Dale,

I contacted support last night and she told me my public_html folder was almost full. I asked how it could be full when there is unlimited storage - she didn't answer me. As you suggested in your last message, I changed the backup to move the files to the cloud. I have an iDrive account with 2tb of storage. The backup did not complete again last night. It only backed up about 480mb and then died. I had deleted a bunch of older archives prior to that but it still failed.

I'm running a manual backup now to see if the backup to the cloud will work.

-Derek

dlb
At least we know the problem. Another trick is to check the setting to upload each part immediately instead of waiting to upload all of them at the end. This will reduce the total storage requirement since you only store one part of the backup at a time. That setting is in the post processing setup.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
The manual backup isn't working either. I've attached the log file for you below... this is supposed to send the files to my iDrive account. We have a ton of PDF files that I'm trying to move to the cloud so I don't have to back them up - that should alleviate the pressure on the backup with large files. I've also tried moving old archives to the cloud under Manage Backups using the "transfer archive" button but I've not had any success.

-Derek

dlb
The log file didn't come through, can you try again please?


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
Here you go...

dlb
It's still not coming through. Can you throw it up on a file service like Dropbox, Google Drive, etc. and send me the link?


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
https://www.dropbox.com/s/vks2ptrz1eaewdp/Akeeba%20Backup%20Debug%20Log.txt.zip?dl=0

dlb
Derek,

It is stopping when it tries to upload the first part of the archive. Can you please double check the settings for your post processing?


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
I modified it yesterday to go to my OneDrive account and I also excluded all the PDFs that get backed up. I ran it manually and it worked - it uploaded all the files to OneDrive. I ran it twice just to make sure and it worked both times. Last night during the automated backup cycle it didn't work again. Unfortunately I had turned of the log information thinking that I'd solved the issue. I've enabled logging again and lets see what happens tonight.

-Derek

derekkoopowitz
Ok, so it failed again last night. I've tried eliminating backing up any large files and I've got the overall backup size down now to less than 1gb. I've attached the log file to the same Dropbox link that I sent you earlier. When I run this manually it works just fine.

dlb
In your OneDrive setup, please make sure that "Enable chunk upload" is checked. If it is already checked, reduce the "Chunk size" field to 5 Mb. We're getting a timeout on the upload, but it isn't giving me any feedback about what is wrong.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
Hi Dale,

The Enable Chunk Upload was checked already so I reduced the chunk size to 5mb as you suggested. The backup runs just fine when I do it manually, but not automated. It failed again last night. I ran the log analysis and nothing was displayed. The size of the backed up file is now 787mb (from the original 5.01gb).

I put the log file on Dropbox again if you'd like to see it.

-Derek

dlb
Derek,

Usually when it runs from the back end but not from CRON, we look at the elapsed time on the log. Many host companies will kill a CRON job that they think is running too long. Your's is going 12+ minutes, so that's probably not what's happening.

It is still timing out on the upload to OneDrive. We're not getting any connection messages or any feedback from OneDrive, just the timeout. I'm about out of ideas, if the smaller chunk size doesn't help I'm going to have to fire up the Bat Signal on Monday.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
Hi Dale,

Last night it crapped out almost immediately. I've put the log file on Dropbox. I'm not sure what to do next.

-Derek

derekkoopowitz
I changed over to use the akeeba-backup.php script - this did not work a while back when I had the same issues. I suspect it was because of the backup size. Anyway, after I changed back over to use this script, the automated backup started working again. It has now worked 2 nights in a row so I think I'm halfway there. Obviously the wget backup is still having problems - I've run it in parallel with the script and it still fails (it is running at a different time of course).

Let's see how it goes over the next week or so and if it is still completing successfully then I'll close the ticket.

dlb
Nicholas took a look at it this morning, here's what he told me:
The timeout does not come from us

It looks like _his server_ kills the backup process 30 to 40 seconds into the upload

This is too fast even for a 5M chunk size

OneDrive etc don't support smaller chunk sizes than that

I think he needs to contact his host

I don't know what they are using to apply that limit

Maybe something monitoring resource usage?

If akeeba-backup.php or akeeba-backupalt.php is working, that's the best solution. If not, we need to try to beat it into submission. Please let me know.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

derekkoopowitz
So far so good... it has now worked 3 nights in a row. I want to give it another 4 or 5 nights before I say its good to go. Thanks for the help!

dlb
You're welcome! If you need any more help, just holler.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!