Support

Akeeba Backup for Joomla!

#25240 Ref. #8927 - ZIP compression on SQL file dump

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by on Sunday, 26 June 2016 17:20 CDT

silke1
Description of my issue:

Dear Nicholas,

I would like to revive the feature request for zipping the DB snapshop SQL file created by the "Main site database only (SQL file)" backup type.

With respect to size a zip file is much more economic to store, archive and/or transmit and especially to extract or restore traditionally (e.g. for table with db extracts and comparison).

The alternative compact type of "All configured databases (archive file) " does include a complex file and folder structure including an SQL file truncation that do not come in handy for plain SQL file operations.

Any chance that you would pick up that feature request again?

Best regards,
Silke
Silke Schäfers
my-track Portal GmbH

Fabrikstr. 3
48599 Gronau
Germany

VAT Number: DE247563722

nicholas
Akeeba Staff
Manager
We cannot do that for one simple reason: memory. If you have a SQL dump that's 30Mb you need at least 80Mb of PHP memory limit to compress it. If you have a SQL dump of 300Mb you need a massive 620Mb of PHP memory limit which makes no sense. Since PHP doesn't have built in support for streaming GZip compression (unless you manually enable it – therefore not available on most hosts) we can't overcome this issue. I have seen scripts producing a big SQL file and then calling the command line zip command but that's even worse: it's insecure, it doesn't work on more than two thirds of servers and is bound to give you problems.

On top of that, restoring such a SQL file is deeply problematic. You can only import it successfully using phpMyAdmin running under cPanel's / Plesk's special privileged web server without time and memory limits.

Quite simply, if you want a massive dump in a zip file you should use mysqldump and tar or zip. Restoring it is a simple matter of using tar/unzip and mysql on the command line. If, however, you want ease of use then use Akeeba Backup's feature to backup all databases (it even includes a restoration script). Remember, Akeeba Backup is designed to make things simple, not possible ;)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

silke1
Hmm, so if I want this, you suggest to use the traditional way of a commandline dump+tar.gz (shell script), which supports streaming of big files (and/or large numbers of files).
Such a script mostly imposes the need to hard-code the db password, a fact that I do not embrace.

I guess a pass by would be to zip or tar.gz the dump files created by akeeba backup, but then akeeba will loose control of these files, as the uncompressed sql-dump should be removed after zipping to achieve the desired storage reduction.

Yes, I recognize the benefit of your restorable archive.
Out of curiosity (sorry if it is noted otherwise, but I did not find anything on it by searching): Are thes restorable database archives also allowing for single table restoration? (I do not presume so.)

BR,
Silke
Silke Schäfers
my-track Portal GmbH

Fabrikstr. 3
48599 Gronau
Germany

VAT Number: DE247563722

nicholas
Akeeba Staff
Manager
Such a script mostly imposes the need to hard-code the db password, a fact that I do not embrace.


Considering that your database password and all of the other connection information is hard coded in configuration.php for Joomla! to work, that's not actually a problem. Think about it.

On one hand, your web accessible PHP application with arbitrary third party, non audited for security code running on it has the keys to your database. A vulnerability in any of your public facing code could allow an attacker to gain this information.

On the other hand you have hardcoded your database connection information in the CRON command line. The CRON configuration is NOT available on public facing parts of your site. It is only visible to the operating system and the control panel software (which is audited for security and has far stricter controls than your public site). Obtaining this information would require a full compromise of the server, in which case the point is thoroughly moot: you can do whatever you want on a compromised server so you don't even need to read the db connection information from the CRON command line or configuration.php...

I guess a pass by would be to zip or tar.gz the dump files created by akeeba backup, but then akeeba will loose control of these files, as the uncompressed sql-dump should be removed after zipping to achieve the desired storage reduction.


Yet another moot point. As I explained, when you're taking a single file .sql backup of your site's database you cannot restore it with Akeeba Backup. A massively huge file requires at the very least using phpMyAdmin inside your host's cPanel, so you are NOT bound by memory and time limits, or in many cases command line (SSH). Therefore it doesn't matter if Akeeba Backup lists this file in the back-end or not: you are still unable to use it inside Akeeba Backup.

Out of curiosity (sorry if it is noted otherwise, but I did not find anything on it by searching): Are thes restorable database archives also allowing for single table restoration? (I do not presume so.)


Not yet. It's on my to-do list. I have a few ideas on how to make it happen but I cannot promise I will implement it on a specific timeline. It's still on my todo list as a research and development task :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

System Task
system
This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!