Support

Akeeba Solo

#29295 remote transfer server error

Posted in ‘Akeeba Solo (standalone)’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

PHP version
n/a
Akeeba Solo version
n/a

Latest post by on Friday, 30 March 2018 17:17 CDT

Mittelcom
Hi, I'm asking you just to limit the scope of my search, I'm trying a remote transfer using all the options other than SFTP (the only one not working), all give the same result: uploading working (checked on receiving side) until 19% for 2048MB split size, until 8,9% with 512MB, then an
AJAX Loading Error
HTTP Status: 500 (Internal Server Error)
Internal status: error
XHR ReadyState: 4
Raw server response:
Do you think it could be only the receiving server's fault (or sender) or could it be anything else?
Too bad I can't give you now server data, I have to switch back the dns...
So please just give me any hint you can.
Thanks and regards,
Guido

nicholas
Akeeba Staff
Manager
The log file you sent me has no post-processing and it has completed successfully. Therefore we know that the actual backup does work.

What gives you the problem is transferring huge files over SFTP because it takes some time and that causes PHP to time out. Actually, we have documented why this happens. Please read the paragraphs below the image on that page.

So the answer to your question is that it's nobody's fault. The timeouts are there for a very important reason (to prevent buggy pages caught in an infinite loop or an unrealistically long calculation from bringing down the entire server). It's just how web servers work.

There are a few ways to go about it.

1. Recommended. Run your backup through the provided CLI script. PHP does not have a timeout limit in CLI mode. Moreover, you don't have Apache, with its own internal timeouts, in the way. Therefore any size part file, even a 2G one, can be transferred through SFTP without a problem. This is what I do for all my sites.

2. If you really need to run your backups through the web interface and use SFTP to upload the only solution is to use a smaller part size. Typically something in the range 10M to 50M works but you need to experiment since the value depends on the speed of both servers (web server and the SFTP target). Start high (100M) and keep decreasing the part size until you find the sweet spot.

3. Use a remote storage which allows chunked uploading such as Amazon S3, Google Storage, BackBlaze, Dropbox, Google Drive or OneDrive. All of them allow you to have a large part size e.g. 2G but they can transfer your backup archives in smaller chunks of 5M to 20M. Since the work item is a chunk we can transfer just as many chunks as possible without triggering a timing out, then continue the upload in the next page load, repeat until the entire archive is transferred. This method is the best for transferring large files but requires the remote storage to support the reconstruction of a big file out of smaller chunks. SFTP does not have such a feature, that's why it's not possible to do that with SFTP.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

System Task
system
This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!