Support

Akeeba Backup for Joomla!

#20043 Chinese filenames don't backup

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Tuesday, 13 May 2014 01:46 CDT

tabletguy
EXTREMELY IMPORTANT: Please attach a ZIP file containing your Akeeba Backup log file in order for us to help you with any backup or restoration issue. If the file is over 2Mb, please upload it on your server and post a link to it.

Description of my issue:

It appears to fail when trying to read file names that have Chinese as part of the file name.

The file has exactly the same "permissions" as all other files in the same directory.

I get this message (example) when I try and create a backup.
Unreadable file F:/wamp/www/hyccchina3/images/photo/?? DSC00331.JPG. Check permissions

Actual filename is "倍仢 DSC00331.JPG"

I got a LOT of these warning messages, across many subfolders within the image directory.

I don't recall getting this message before, with other (live) sites, running Joomla 2.5.

I hope it's just a stupid misconfiguration on my WAMP server...

I've attached the log file

nicholas
Akeeba Staff
Manager
Please make sure that your PHP's default character set is set to UTF-8. The only bug which was affecting international characters in filenames was resolved five years ago. I would have noticed if it broke; I am using Greek (α, β, γ, ...), German (ß, ü) and Nordic (ø) characters in filenames of my test sites which don't throw an error.

FWIW, if you want a serious local development environment throw XAMPP and WAMP off the window and install Zend Server. After the first seven days it automatically switches to the free, developer's version which is more than fine for testing, debugging and developing sites locally. I've had all sorts of strange issues with XAMPP, WAMP, MAMP and the like, but no issue with Zend Server. It's the difference between some random guys having a side project and the company behind PHP's parser putting their full weight behind a kick-ass, tested, prepackaged server environment.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tabletguy
Thanks for the tip on default_charset. I'd not heard of this before, and I never saw the problem until this website (first one with Joomla 3.3. It also happens to be a new version of WAMP that was just released in May.

For anyone else, stackoverflow has a good summary of what to change and recommendations for "complete" support of utf-8 here:

http://stackoverflow.com/questions/1605760/how-to-best-configure-php-to-handle-a-utf-8-website

I ended up using their recommendations of the following settings:
default_charset = "utf-8"
mbstring.internal_encoding=utf-8
mbstring.http_output=UTF-8
mbstring.encoding_translation=On
mbstring.func_overload=6

I will definitely check out Zend. Thanks very much for that tip as well.

Feel free to close this ticket yourself if you wish. (I didn't close in case you want to respond again).

Stephan

tabletguy
Quick note:

I found that "mbstring.func_overload=6" causes problems so I removed it.

And,,, doing a new backup (should have done that before posting a reply) is still failing the same way.

nicholas
Akeeba Staff
Manager
mbstring.func_overload isn't supported by Joomla!. It's an installation blocker, actually. I would also disable the encoding translation and set the internal encoding to neutral. It's a bad idea having mbstring convert character encodings on the fly, it causes the exact problem you're currently having.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tabletguy
I removed all but the single

default_charset = "utf-8"

but it still fails. I've restarted WAMP a couple of times.

I could not find any documentation on the setting "neutral", but I assume that's the same as no setting (empty) The Php documentation says: "Since PHP 5.6.0, if this setting is empty, default_charset is used. "

I'm using Php 5.5.4, and I have the mbstring.internal_encoding commented out now but it's still failing to backup those files.

I'm downloading a copy of Zend Server to try out. I'm also wondering if this is specific to the newest WAMP, since it was just released (with Php 5.5) this month.


nicholas
Akeeba Staff
Manager
I can't reproduce it on any other server. I've tried Zend Server on two major OS (Mac OS X Mavericks and Ubuntu Linux/Server 14.04). I've also tested using MAMP and MNPP on my Mac. I haven't tested WAMPServer as I'm not at home this weekend and only have my Macbook with me (I left the Windows laptop at home). I did test with various subminor versions of PHP 5.3, 5.4 and 5.5. I tried backing up files with filenames containing Chinese, Greek, German, Nordic and Slavic characters. I still can't reproduce this issue.

Trying to squeeze my brain about it, I believe that the problem is with Windows itself. I remember that some years ago (probably on Windows XP) I had a similar issue. It turned out that I had to change the regional settings on Windows. For the life of me I can't remember if I had to set the encoding to Greek or UTF-8. I also can't remember anyone having this issue with Windows 7 or 8.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tabletguy
I installed Zend, and it doesn't happen there.

I also uninstalled WAMP 64 and installed WAMP 32 (Windows 7 Pro 64bit), but it still fails. I also tried setting the MySql default encoding (during installation of MySql) to utf-8, and of course the database itself is utf-8.

You would be welcome to connect to my desktop if you wished, via TeamViewer, perhaps. I am in Minnesota, (CDT - Central Daylight Time).

The PHP default encoding is already UTF-8

The newest WAMP (May 1st) doesn't have any other versions of PHP available (other than 5.5) so I can't try that. The older version of WAMP had 5.4, without any issues.

I've gone through and made sure any services (from ZEND install) were stopped before trying to run WAMP, after a clean reboot.

I've read over your documentation on this error several times as well as surrounding pages.
https://www.akeebabackup.com/documentation/troubleshooter/abunreadablewarnings.html

It's not an urgent issue, but I would like to get it working at some point with WAMP.

The previous version of WAMP didn't have this issue. Same Win 7. Same day, in fact (switching from old WAMP to new one to get PHP 5.5)

nicholas
Akeeba Staff
Manager
This is a problem with the configuration of WAMP. Unfortunately we can't provide support for issues on local servers. Once I am back at the office I will have to install WAMP and take a look at what is going on. However, my recommendation for local servers is to always use Zend Server when possible. It's THE reference PHP-capable server setup by the company which has written the brains of PHP.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tabletguy
I agree that it's a WAMP problem, not a Windows 7 problem. I wasn't trying to get extra "support" by offering to let you connect, just trying to offer help if you wanted to speed things along.

nicholas
Akeeba Staff
Manager
I was out of office, without my regular gear and a slow Internet connection :) Yesterday I got back to the office and tried installing the latest WAMP on Windows 8. I am now absolutely certain that the problem lies with how PHP was compiled in WAMP. When you ask PHP to list the contents of the directory it always returns corrupt filenames if they contain any character which can't be mapped directly to the ISO-8859-1 character space. This is an enormous bug in WAMP.

My suggestion is to not use WAMP. It's broken beyond repair.

FYI: I wanted to make sure that the problem REALLY is in WAMP and not my code. Even though I tested the low-level PHP API (DirectoryIterator) directly I wanted to be 100% sure I am not doing something silly. So I tried creating a few files with international characters in the images directory and use Joomla!'s Media Manager. No surprise, I got mangled filenames in Joomla!'s Media Manager as well. Therefore I am certain beyond any reasonable doubt that WAMP is broken beyond repair and MUST NOT be used to restore or develop sites which contain filenames with international characters beyond those which can be represented in the ISO-8859-1 code page.

I hope that helps!

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!