Support

UNiTE, Remote CLI, eXtract Wizard

#39282 Unite certificate property error

Posted in ‘UNiTE and Remote CLI’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

PHP version
8.1
Tool
UNiTE
Tool version
5.0.2

Latest post by nicholas on Tuesday, 08 August 2023 11:03 CDT

carcam

Hi,

I'm findin a weird issue with Akeeba Unite. Whenever I try to use it I get the following error:

Β 

Property getSignature not found in Akeeba\UNiTE\DataShape\DataShapeObject

Β 

This is the full prompt:

[INFO ] Parsing /var/www/html/amascotados.xml
[INFO ] Validating the source data
[ERROR ] Property getSignature not found in Akeeba\UNiTE\DataShape\DataShapeObject

In DataShapeObject.php line 182:

Property getSignature not found in Akeeba\UNiTE\DataShape\DataShapeObject



Weirdest thing is that I think I have used this release with no issues some weeks ago...

Any idea?

thanks!!

nicholas
Akeeba Staff
Manager

This is strange. I do not see any references toΒ getSignature anywhere.

Can you please re-run this with --verbose and give me the debug backtrace?

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nicholas
Akeeba Staff
Manager

Can you please use this dev release instead?Β https://www.akeeba.com/download/unite-dev/5-1-0-dev202308020828-rev6dfb0ab/unite-package-5-1-0-dev202308020828-rev6dfb0ab-zip.zip. I believe this will address the issue you are facing.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

carcam

Hi Nicholas,

thank you very much for your reply.

That release fixes the certificate problem but it outputs a new error in the prompt:

[INFO ] S3: using custom endpoint s3.eu-central-1.wasabisys.com
[ERROR ] Unable to display the estimated time if the maximum number of steps is not set.

Unable to display the estimated time if the maximum number of steps is not set.

Any idea?

Thanks!!

Β 

nicholas
Akeeba Staff
Manager

That's probably because you are giving it an Access / Secret Key combination which does not allow the S3 listObjects API call.

When you are trying to import from S3 you only give it the name of a single backup archive, the one with the .jpa, .jps, or .zip extension. UNiTE needs to know if this is a split archive i.e. if it has parts named .j01, .j02, … / .z01, .z02, … To do that, it needs to list the objects (files) in the bucket. This requires calling listObjects. If your user β€”as defined by the Access and Secret Key combinationβ€” does not have the permission to do that UNiTE cannot know what to download, so it errors out instead of doing something potentially stupid.

You would also get the same error if the access / secret keys are wrong, the region is wrong (for v4 signatures), or the signature type is wrong. In all cases, the call to listObjects fails and you get an error in return.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

carcam

Hi Nicholas,

thank you very much for your detailed explanation. I feel that I need to add more context to show where I see the issue.

I'm using legacy v2 because I'm connecting to Wasabi to retrieve the backup files and the Credentials I'm using are the same I was using a couple of months ago, before the certificate error and this is a new issue.

In any case I have checked Wasabi keys and roles and permissions and they seem to have full access to the folders. Actually they are the same credentials I use to upload the backup from Akeeba Solo and to manage the remote quota, and it does it pretty well, removing old backups.

So I think this is a new issue. Running the command with the --verbose option provides the attached error.

nicholas
Akeeba Staff
Manager

The trace actually confirms what I said.

Note that the code we use to connect to the S3 API has not changed. It'sΒ https://github.com/akeeba/s3 which we also use in Akeeba Backup itself. However, we don't list files in Akeeba Backup. We only write and delete files. This is why you never noticed a problem; that part of the code does not run there.

However, you're right in saying that you didn't observe an error in the past. It was a bug that was fixed :) Previous versions of UNiTE ignored the problem of the S3 API returning nothingΒ  and tried to download just the one file (.jpa, .jps, or .zip) you had specified in the job XML file.Β I suspect that since you had single-part archives you never noticed this being a problem.

In this version we use the progress bar from the Symfony Console package. We pass the total file size read from the S3 API for all part files as its max length argument. When the S3 API fails to return a file listing this is zero and causes the progress bar to emit an error. The root cause is not the S3 API code (which hasn't changed and used in Akeeba Backup), nor the Symfony progress bar; it's that the S3 API returns no file listing.

For what it's worth, it's not a problem in UNiTE's logic either. I have tested this with a multi-part (10 parts) archive using Amazon S3 proper and it works.

It would appear that something's up with Wasabi. Can you try using your credentials with s3cmd using its ls command to list the contents of your bucket? s3cms uses boto3, the official S3 API implementation for Python and is maintained by Amazon Web Services itself. This will tell us if the problem is with the Wasabi credentials, or at least their implementation of the S3 API.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nicholas
Akeeba Staff
Manager

Side notes:

Using Wasabi with S3:Β https://knowledgebase.wasabi.com/hc/en-us/articles/115001757791-How-do-I-use-s3cmd-with-Wasabi-

You can totally use v4 signatures with Wasabi. The regions are the part between s3 and wasabisys.com in the URLs found in https://knowledgebase.wasabi.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions-. I have confirmed this with another user and long-time friend a few months ago.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

carcam

Hi Nicholas,

thank you very much for your detailed explanation. It totally makes sense, so I have installed s3cmd and after configuring it with my Wasabi API credentials, I have tested the command to list the bucket:

Β 

s3cmd ls s3://hepta-clients/Amascotados/

Β 

It worked with no issues providing the listing of all the files in the bucket as you may see in the attached screenshot, so I think this is not an issue with my credentials. On a side note, Cyberduck also works pefectly with those credentials, and it basically lists all the items in a bucket.

I have switchedΒ <signature> tag to v4 and tried again with no avail. Any idead of what could be causing this issue? For multipart backups, Akeeba Unite is my best tool as it gets all the files directly from Wasabi easily.

Best!

Β 

nicholas
Akeeba Staff
Manager

Stupid question before I pay Wasabi for an account. Are you using the same credentials, signature, region, and endpoint in UNiTE as Akeeba Backup?

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nicholas
Akeeba Staff
Manager

So, I did try with Wasabi and found it to be working. Here are the relevant settings from my YAML file:

siteInfo:
package: myfile.jpa
s3:
accesskey: MYACCESSKEY
secretkey: secretSECRETsecretSECRET
signature: v2
bucket: mybucket
endpoint: s3.eu-central-2.wasabisys.com
filename: myfile.jpa
ssl: 1

This is for a bucket in the eu-central-2 region (Germany) of Wasabi using the v2 signature.

While troubleshooting this I noticed that it would never apply the setting for a v4 signature if you were using a custom endpoint. Moreover, Wasabi has a non-compliant version of the v4 signature. Unlike Amazon S3, they do not want us to put the Date header into the string to sign (whereas Amazon S3 proper requires it, that's why I am saying it's non-compliant). I added a workaround for Wasabi's broken v4 signature calculation and fixed the S3 v4 signature feature to work with custom endpoints as well. You will find a new dev release in https://www.akeeba.com/download/unite-dev/5-1-0-dev202308081335-rev3fe24b1.html. Using that dev release you can use the following settings in the S3 section, properly using v4 signatures:

s3:
accesskey: MYACCESSKEY
secretkey: secretSECRETsecretSECRET
signature: v4
region: eu-central-2
bucket: mybucket
endpoint: s3.eu-central-2.wasabisys.com
filename: myfile.jpa
ssl: 1

So, I am not sure what you are doing in your configuration file. Is it possible you're using the wrong endpoint or bucket name? I know I already asked you to check the endpoint, but I am not sure if you did.

Another thing, is that if your bucket name contains dots it might cause problems with the way SSL works on Wasabi's (and Amazon's) end. If you are using such a bucket, please instead use a bucket which has dashes instead of dots.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

carcam

Hi Nicholas,

thank you very much for looking into this with such detailed!!

I rechecked everything and tested it with no success, but then I noticed you were testing the file stored directly in your bucket, and in my case the file is stored in a folder inside the bucket. Then I saw I was adding a "/" before the folder name like this:

/FOLDER/backup.jpa

So I tested removing the "/", and it worked!!

I am using my own the same unite XML file structure I have used in the past, and at some point I guess I changed this and never took notes on it.

I tested it with the first dev release you provided and "v2" and it worked, and I have also tested it with latest dev release and "v4" and it worked beautifully.

Likewise, I think we can call this issue as closed.

Thank you very much for your patience and thoroughness.

Best!!

nicholas
Akeeba Staff
Manager

You're welcome!

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!