Support

Site Restoration

#13737 Amazon S3 Kickstart import is extremely slow

Posted in ‘Site restoration’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

PHP version
n/a
CMS Type
Other
CMS Version
n/a
Backup Tool Version
n/a
Kickstart version
n/a

Latest post by nicholas on Thursday, 11 October 2012 02:27 CDT

webcoast
Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes
Have I searched the tickets before posting? Yes
Have I read the documentation before posting (which pages?)? Yes
Joomla! version: J1.5.26
PHP version: 5.3.13
MySQL version: 5.1.65
Host: Rochen
Akeeba Backup version which took the backup: 3.2.7
Kickstart version used to extract the backup: Kickstart Pro svn1675(Alpha)

Description of my issue:
I am attempting to install a copy of a website on another server by using kickstart and copying the backup files from Amazon S3. Initially I received an error about object out of range, so I searched the forum and found a solution to try Kickstart svn1675(alpha). Using svn1675 has allowed the restoration to proceed, however, like another thread in the forum, the restoration is unacceptably slow. The initial screen progress bar showed 1.6% progress and in 30 minutes it has not updated again.
The screen warning says do not close the window - how can I close it safely and resume the kickstart by uploading the files via ftp, or is there a work around for this very slow Amazon S3 importing?
Thanks, Nicola

nicholas
Akeeba Staff
Manager
Please try the new dev release I just uploaded, svn1732, which does fix the slow extraction issue. If it works for you too I'll publish a new Kickstart release later this week.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

webcoast
Hi Nicholas, thanks for the update. I tried svn1732, but came across a new problem... after entering Amazon public and private keys, and selecting bucket, there was no list of archive files. The display had <ROOT> which appeared to be a link, but went nowhere, "Folders" written on the left underneath <ROOT>, and "Files" written on the right. No archive files were listed.

See attached image.
Regards, Nicola

nicholas
Akeeba Staff
Manager
There seems to be some browser issue. I just tried on the following browsers on my Mac OS X Mountain Lion machine:
- Google Chrome 22.0
- Firefox 15.0.1
- Opera 12.01 and 12.02
- Safari 6.0.1
In all cases the contents of the bucket were listed just fine.

Please make sure that the user you are connecting with has full access to the bucket and the files written to it. Also try disabling any browser extensions or application which may be interfering with Javascript. Finally I have to remind you that the total of changes between the two dev releases you tried is three lines, none of which has anything to do with the way bucket contents are being listed.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

webcoast
Hi Nicholas,
I am using Firefox 15.0.1. Today when I tried it the archive files displayed correctly. Why that is different to yesterday when I tried twice I don't know, mystery.
Anyway, once I selected the archive file for importing, the screen still seems to freeze like it is not progressing with the import. Here is a screenshot after 15 mins - had been no progress.
Regards, Nicola

nicholas
Akeeba Staff
Manager
How long does it take between progress bar refreshes, before it gets stuck? My educated guess is that it takes too long for the data to be downloaded from Amazon S3, causing a timeout error on the server. If that's not the case (e.g. the previous refreshes happen within 2-3 seconds, not 30-40 seconds) we are talking about a network issue between your PC and the server. This also explains why the script wasn't working yesterday.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

webcoast
Hi Nicholas, as far as I can tell the import progress page loads and has that 1.5% slither of a progress bar on it from the start (either that or the first refresh happens so quickly that it appears to be the starting screen), and then is never refreshed again. The progress bar never moves from it's original state.
The website is hosted with Rochen in Dallas USA. Not sure where Amazon S3 is physically located, but I think it is USA also. When doing the Akeeba backup procedure on this website, the configuration is set to 10meg split archive file size chunks, with disable multipart uploads ticked, otherwise timeout errors occur. Does Kickstart need some similar parameters perhaps??
Thanks for your continual help with this.
Regards, Nicola

nicholas
Akeeba Staff
Manager
The website is hosted with Rochen in Dallas USA.

That's strange. When I replied to you earlier yesterday I had run my tests on a test server hosted with Rochen, in Dallas. For what is worth, I tested again. It still works. I tested with two buckets, one stored in US East and on in EU Amazon S3 locations. They both work like a charm.

Does Kickstart need some similar parameters perhaps??

It already does. It downloads each file in 1Mb chunks, then stitches them together. It keeps on trying until 5-10 seconds have elapsed. This avoids server timeouts; you wouldn't expect me to overlook something as fundamental as that, right? Dodging server timeouts is what made Akeeba Backup and Kickstart what they are. That's why I asked how long it takes for the first chunk. Some hosts are desperately slow and take upwards 30 seconds to transfer 1Mb from S3, leading to timeouts.

Based on your information I have to conclude that the problem lies somewhere in your computer. Using the same Kickstart version, on the same host (now, that's one heck of a coincidence!) works fine from my computer.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!