Support

Akeeba Backup for Joomla!

#20465 Restore Off-site Directories

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Thursday, 10 July 2014 03:22 CDT

nyocca
EXTREMELY IMPORTANT: Please attach a ZIP file containing your Akeeba Backup log file in order for us to help you with any backup or restoration issue. If the file is over 2Mb, please upload it on your server and post a link to it.

Description of my issue: I was using kickstart 3.8.0, and the restore hung during the routine in which the files from the external or offsite directories are restored, at my option, to the directory on my site from which they came. I interrupted the process and I could access my site. I apologize that I did not preserve the log file. I made another Akeeba backup in the meantime, and it was too late when I realized that there might be something interesting happening.

Not only had Akeebabackup hung, afterward I found a new directory tree of empty file folders that appeared under the directory where the external files were to be restored.

While those files were not restored, what was instead was in fact restored there were the directory folders of my Joomla site. For your notes I will also mention that I find these folders to be undeletable and inaccessible, which may have relevance to confirming their origin. Those folders had already been restored to my site before its external files restoring routine ran.

I had never tried that routine before, although I noticed it in version 3.8.0 of kickstart before.

As usual, I want to again take this opportunity to offer my highest praise and congratulations on establishing and maintaining the greatest, without peer, software that I use today. I also hope for your continued success and the greatest of fortune for you at Akeeba. It is a great privilege to use your works, to learn from you, and to receive the great customer care you truly provide. Your honesty is always refreshing. Akeeba truly deserves its growing renown as the best in class.

nicholas
Akeeba Staff
Manager
I need a clarification to better help you.

Kickstart's only job is to extract the backup archive. The off-site directories will be extracted in a folder inside your site, by default called external_files. When Kickstart finishes it will launch ANGIE, the installation script. Part of its job is to help you move the off-side directories to their proper location.

Which part fails? Kickstart or ANGIE? This is critical for me to help you as it means two entirely different things.

I believe that your problem is when ANGIE tries to copy the files to off-site directories. This is NOT Kickstart and the version of Kickstart used is completely irrelevant. The critical factors for the success of this operation are:
  • You must create the off-site directories in advance
  • The directories must have the proper ownership and permissions for ANGIE to be able to copy the files in them (i.e. the web server user must be able to write into these directories – you may want to give them 0777 permissions while moving files into them)
  • You must make sure that the absolute path is correct, something which you can verify using the file browser button in ANGIE before moving the contents of each external directory.


For your notes I will also mention that I find these folders to be undeletable and inaccessible, which may have relevance to confirming their origin.


Absolutely not. When extracting a backup archive Kickstart cannot retain the ownership or the permissions of the files. That would require root access which is not given to the web server for obvious security reasons. You are extracting those directories using Kickstart in direct file writes mode. As noted in the Security Information chapter of our documentation this means that the FTP user doesn't have access to them. Please read that chapter, it will help you understand what is going on.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nyocca
I would like to clarify that I am describing something that occurred during the ANGIE script, which presented me the choice whether to copy from the site subdirectory where /excluded_files were initially restored, to the off-site directory of my choice. I was using the default choice, the original directory of the files, which was presented by ANGIE.

The offsite folder had 777 permissions. Its new subdirectories have 341 permissions.

Thank you. I will be interested to read how it works.

Nicholas, I greatly appreciate your valuable time.

nicholas
Akeeba Staff
Manager
341 permissions? o_O Wow, these permissions are extremely strange. 341 permissions mean: read files and list contents for the owner user, read files for the owning group, list contents for everyone else. These permissions make no sense. Is it possible that the server somehow forces permissions for off-site directories? I can't explain this otherwise :s

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nyocca
Dear Nicholas and Team:

I suppose my interrupting of ANGIE processing had lead the server to give these aberrant permissions to the folders and files when I interrupted the process. That's only the first error for which I must definitely take full responsibility. Also I did not check the source and target directory information in ANGIE.

I need to be clear that the off-site folders that were populated by ANGIE contained a restored version of the main site backup, and not any of the off-site files as was intended. What I mean is that the main site backup was already restored, perfectly, in the site folder, by kickstart.php but then ANGIE (which did not copy anything from /external_files to the off-site directory) performed or initiated another restore of the main site database into that off-site directory.

At least this is how it appears, but honestly the most likely reason is user error on my part. It is very possible I made a user error, especially if no one has reported an issue of this kind. If not only operator error, then the other likely possibility is an unusual complication. In this case the complication is that the main Joomla site stored its log files in a subdirectory of the same off-site directory, and so the log files of the Joomla site were restored to this off-site subdirectory by kickstart.php. Maybe then ANGIE also sees this and then, as a result, it may restart the restoration of the whole main site to that folder instead of copying the external files folder to this offsite folder. That's very conjectural on my part and probably of no help. I really apologize for troubling you about this, and I am simply grateful.

Again, I want to mention that your support is superb, your products are magnificent, you are phenomenal and I wish you nothing but the best of good fortunes.

Nick Yocca

nyocca
Initiating this ticket, I made a skrivener's error and put my Joomla version in the PHP version box. Sorry. They are Joomla 3.3.1 and PhP 5.3.28.

Also, when I wrote my first message, I did not consciously recall the prior occasion of my own in which a set of main-site directories had been restored, that time to a different off-site folder, with basically the same fact pattern as this time where my intention as user was to allow ANGIE to particularly copy the restored /external_files to where they belonged but somehow got teh full main site restore directed to that offsite directory.

Thirdly, I just said above that a "database" was copied to the off-site directory. I don't know why. I meant the files that were in the Akeebabackup of the main site were restored to the external folder. While it seemed to have that effect, from my user perspective it looked like I was just waiting for ANGIE.

Further to my previous mention of the permission states of files I found, I cannot add much of my own, but I am informed by some luck Googling that someone running PhP 5.2.5 said that he tried to upload something and by mistake got files with these weird 341 permissions. But the situation had really nothing else I could discern in common with this, except it contained a possible solution to a skrivener's error of another kind, one that to see it you gotta look hard.

The fellow who had the issue and created files with 341 permissions like mine wrote this code which contains Linux commands something like this
if(!mkdir($folder,"0777",true)){[/code]

There was a little explanation that shed another light on how the files really get their permissions.
It mentioned a Linux command "unmask" was mentioned as follows: "The real permissions depend on the parameter to mkdir and the umask. The umask is subtracted from the permissions given to mkdir. Try setting your umask to 0 before doing the mkdir." After thinking about that, I can say of my first-hand knowledge that the Linux file system assigns file permissions through a step-wise process in which the permissions are applied that equals the difference between two parameters. If either parameter is unreadable by Linux, the consequences can be weird.

But the reason I mentioned that guy's code is that there happened to be a very simple fix provided to him by another guy who also would not come close to being in a class to even mention in the same paragraph as you when it comes to knowledge. But it quickly solved his Linux problem, which I am sure you saw, which was that 0777 does not require quotes because it is an octal rather than a text string, he said in this article I saw in StackOverflow via Google.
subject- was-php-creates-folder-with-341-permissions

His user input was nothing much like mine. First my input, if you recall, was to terminate the program, go onto my admin side and my front end, find everything perfect as usual. But I must admit to you and everyone that at that point I did in fact hit every darn button I could think of to terminate dear ANGIE, and I say that with all due respect to your beautiful baby, who serves me so well time and again.

My conclusion is that when Linux fails to read the necessary input parameter's value, it might result in the weird 341 permissions depending on the two parameters that were in place and whether Linux could read only one of the two parameters needed for its rules to set the correct permissions. If the uploading routine has an unreadable mkdir parameter then I think an unmask octal still gets subtracted, but since it is subtracted from from zero rather than subtracting it from a 777 parameter value it came out weird, which might depend on whether or how Linux handles negative numbers.

nicholas
Akeeba Staff
Manager
I see what is going on. My developer used the string '0755' instead of numerical 0755 when writing this feature. I will rectify this issue and warn him about the obvious difference between a string (which is read as a base 10 integer) and an octal.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nyocca
Nicholas:

I wanted to express my gratitude to you again, and I mean it from my heart, for the fine products you build and the business that you built to build those fine products. It's amazing. I am a fan. You are offering the world's greatest backup solution now to more platforms (WP etc.) . Great move. You are a genius in business too.

Thank you again. I still remember several years ago when you personally spoke with me on the phone, taking time to help me on a day, I believe, when you were at Joomla Day in Athens.

My brother the fighter pilot taught me a phrase I will sign off with. They would say it when two pilots were flying side by side and it was time to say so long for now,

"I'm a dot."

Nick Yocca

nicholas
Akeeba Staff
Manager
You're welcome, Nick and thank you for the heads up! It's always a great pleasure knowing that we have clients who really care for our products deeply and will go out of their way to help us identify a fix a bug :)

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!