Akeeba Backup for Joomla!

Wednesday, 28 March 2018 23:40 CDT

cooltide

Description of our issue:

Hi.

We are successfully using Akeeba Back Up without issue on our Azure installations
Our akeeba Back Ups successfully back up to a Dropbox folder... All good, and has been for some time.

Our question is this: Can we configure Akeeba Back Up for Joomla to have a separate (different) path for the temporary J01 files created during the back up, to the final back up path for the JPA file?

The reason: Akeeba is creating the temporary J01 files and Dropbox kicks in and syncs immediately. Of course, we don't want this to happen. We are only interested in the final JPA file created at the end.

When we look at the processing and the data transfer across the entire web server, where there are multiple back ups, it can have an impact on the services, both performance and financial.

Request: Can we configure Akeeba to save the temp J01 files to a folder outside of Dropbox and then have the final JPA file written to a different path... The Dropbox folder?

I understand that we could back up outside the Dropbox folder and then write a Powershell script to move the back ups but that doesn't seem very elegant.

Maybe we have missed a way of doing this?

Thanks for your help

Kind regards

Colin

Thursday, 29 March 2018 01:17 CDT

nicholas

YOU ARE CORRUPTING YOUR BACKUPS, MAKING THEM IMPOSSIBLE TO RESTORE! DO NOT DELETE THE .j01, .j02 ETC FILES.

Akeeba Backup takes backups in multiple parts. The .jpa file and all the .j01, .j02 etc files are part of the same backup archive set and are all required to extract it. It's the same thing as WinZip's .zip, .z01, .z02 etc or WinRAR's .rar, .r01, .r02 etc. This convention for naming part files of compressed backup archives has been around since the 1980s; we didn't invent it.

Actually, .j01 is the first file of the backup set which contains the header of the backup archive set, telling as how to extract it, and at least part of your database data. The .jpa file is the last file of the backup set. If you delete the .j01, .j02 etc part files of the backup set it is impossible to extract your backup archive for the obvious reason: you can't extract data which no longer exists.

This is information you can easily find in the documentation, the tooltips of the Configuration page and when you run Kickstart to restore a backup.

Speaking of which, do remember to test your backups regularly. An untested backup is worth as no backup at all. Test regularly by restoring to a test/dev server or even a local server. Forewarned is forearmed. You definitely don't want to go troubleshooting, if it's at all possible, when your site is down and you can't restore your backup.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 29 March 2018 10:52 CDT

cooltide

Hi Nick

Thanks for the speedy reply.

I think I didn't made our issue clear...

We are not deleting the J01 files. They disappear automatically when the Akeeba back up is complete.

We have been successfully backing up and restoring from .JPA files, created by Akeeba, for many years, using Kickstart.
We have never had a corrupt .JPA file? We have restored many, many times, and indeed used .JPAs to migrate sites as well.

Our experience is this:

The Akeeba Scheduler starts the back up and a J01 file is created in the folder (This happens to be a Dropbox folder)
When the Back Up is complete the J01 file disappears and the JPA is created. It's always been like this?

This has been happening for years like this. When we need to restore, we simply kick start the latest JPA and successfully restore, or migrate.

Note: The only time j01 files appear in the folder is "during" the back up process. They disappear "after" it completes.
The only exception to this is when the back up, very occasionally, fails. When this happens we end up with a j01 file and no JPA file.

So, effectively all of our sites are backed up this way.

What we are trying to achieve is to stop Dropbox syncing the temp J01 files during the back up? That's all.

If I understand your reply the J01, J02 are created and make up a back up set. Am I correct in thinking that when the set is complete they disappear? I also assume from your reply that the J01,J02 files will be created in the same directory? Just like zip files.

Solution: I have now realised that I can do this using Akeeba existing "Post Processing Engine" feature "back up to Dropbox" after the back up.

We are now saving the back up within the site path and using Akeeba to then transfer to Dropbox. Rather than doing the back process within a Dropbox folder and thus causing Dropbox to sync the temp J01 files.

I thought I'd just reply in case this helps others who might be trying the same thing.

Kind regards

Colin

Friday, 30 March 2018 03:54 CDT

nicholas

Hm, I believe that we are talking about different things.

When the backup begins, the backup archive is created with a .j01 extension in the backup output folder of your site. This is NOT a temporary file, it's the actual backup archive and it is NOT uploaded to your Dropbox.

If the entire backup archive fits inside a single part, the file is renamed to .jpa. Then and only then is it uploaded to Dropbox. In this case (entire backup fits in a single part) there is no .j01 upload.

If the backup does NOT fit inside a single part and one or more additional parts are required what happens next depends on your settings.

If you have enabled the "Process each part immediately" option the .j01 file is NOT uploaded yet. The reason is that the first 19 bytes of the file are the JPA file header which contain the number of part files the backup archive consists of, a detail we cannot know until the backup finished. So this part stays on your server. The next part is created with a .j02 extension. If a third part is needed the .j02 file is uploaded to Dropbox (that's the first file being uploaded to Dropbox!) and then we create the .j03 file. And so on and so forth until the backup is complete. When the backup is complete, the current part file (e.g. the .j03) is renamed to .jpa before being uploaded to Dropbox. Finally, the .j01 file's header is updated and then uploaded to Dropbox. Therefore the upload order of the files in this case is .j02, .j03, ..., .jpa, .j01. The files are uploaded in order except the first part which is uploaded last. None of them is a temporary file, they are all part files of the same backup set.

If you have NOT enabled the "Process each part immediately" option we continue creating the next part file .j02 in the backup output directory. And so on and so forth until the backup is complete. When the backup is complete, the current part file (e.g. the .j03) is renamed to .jpa. Then the .j01 file's header is updated. At this point the post-processing steps begin. Midway through the post-processing steps we have the upload to Dropbox. The files are uploaded in the order they were created: .j01, .j02, .j03, ..., .jpa. None of them is a temporary file, they are all part files of the same backup set.

There is no way that Akeeba Backup uploads a "temporary" .j01 file to Dropbox and then makes it disappear because a. .j01 etc files are not temporary and b. the upload code cannot possibly kick in until the backup part is finalized.

HOWEVER if your backup output directory is synchronized to Dropbox EXTERNALLY (not by Akeeba Backup but by the Dropbox agent running on your server computer) then You Are Doing It Wrong :) The backup output directory is also used as a temporary area. This is where we write the SQL files before putting them to the backup archive. This is where the backup archives (all their parts) are written while the backup is in progress. The fact that the archive has a .j01 extension before being renamed to .jpa is actually irrelevant and happens purely out of necessity (you cannot know how many part files you need before you take the backup, obviously). Well, if that's the case you should have a backup output directory outside the Dropbox synchronization root. But I think that's an overkill and completely unnecessary since we don't delete and create afresh the backup archive. We simply rename .j01 to .jpa which means that Dropbox uploads less than 1KB of command data to effect the rename. It does not re-upload the entire backup archive. Dropbox is very smart. I can swear that what I described is accurate, I have tried it with massive files (7GB exported virtual machines over a 512KBps shared ADSL uplink; the file rename took seconds, not hours).

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Sunday, 29 April 2018 17:17 CDT

System Task

system

This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support