Vfrdtyky

I've got PKCS#7 Der formatted file called p7
and an x509 certificate file called mroot.der.cer which matches the root certificate of p7 chain.

I'd like to verify my p7 certificate chain using openssl using the following commands :

First - convert my mroot trusted cert file to pem format. 

openssl x509 -in mroot.der.cer -inform der -outform PEM -out mroot.pem.cer



Second - verify the root chain using mroot.pem.cer

openssl  smime -verify -CAfile mroot.pem.cer -in p7 -inform DER -out blabla

However, I got the following error :

Verification failure
140735569544136:error:21075075:PKCS7 routines:PKCS7_verify:certificate verify error:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.50.2/libressl/crypto/pkcs7/pk7_smime.c:343:Verify error:unable to get local issuer certificate

I also tried to add the last command, the -noverify flag but got a different error.

Verification failure
140735569544136:error:21071065:PKCS7 routines:PKCS7_signatureVerify:digest failure:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.50.2/libressl/crypto/pkcs7/pk7_doit.c:1084:
140735569544136:error:21075069:PKCS7 routines:PKCS7_verify:signature failure:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.50.2/libressl/crypto/pkcs7/pk7_smime.c:412:

The pkcs7 structure should be fine as I've extracted it from the PE file iexlorer.exe and from its chain I've extracted the root certificate, and refer to it as a trusted one.

What am I doing wrong here ?

P.s.
To observe the same failures I did, I've uploaded the files to the following links :

https://ufile.io/vrqpt

https://ufile.io/ajgex

Your uploaded example files have a few properties which prevent it from being verified.

First, the certificate of the signer in the p7 file has expired on Apr 24 22:33:39 2014 GMT. You will have to disable checking of the expiration date if you want to verify the chain. This is programmatically done with the verify flag X509_V_FLAG_NO_CHECK_TIME, or the option -no_check_time for the OpenSSL smime -verify tool.

Then, your "root of trust", found in the mroot.pem.cer file is not the right one. You extracted the Microsoft Time-Stamp PCA certificate whereas the signer of the p7 file chains up to the Microsoft Code Signing PCA certificate.

Let's say that you extract that correct certificate to a file called trust.pem.cer. That certificate is not self-signed: its issuer is the Microsoft Root Certificate Authority. You will have to indicate that you are using a so-called partial chain if you want such a certificate to be at the end of the chain. This is programmatically done with the verify flag X509_V_FLAG_PARTIAL_CHAIN, or the option -partial_chain for the OpenSSL smime -verify tool.

Also, it looks like the OpenSSL implementation of the PKCS7 verification requires your certificate to include the extended key usage of S/MIME signing, which your certificate does not include. It looks like this can be worked around by setting a code-signing purpose for the OpenSSL X509_STORE. The OpenSSL smime -verify tool does not expose this kind of setting so you will have to do this programmatically by setting the XKU_CODE_SIGN purpose. XKU stands for eXtended Key Usage and, as a caveat, the OpenSSL documentation around that is virtually non-existent. You have to thoroughly test this if you decide to use it.

The piece of code below (return code checking omitted) successfully verifies the certificate chain in your p7 file, but not the signature:

BIO *bio_p7 = BIO_new_file("p7", "r");

PKCS7 *p7 = d2i_PKCS7_bio(bio_p7, NULL);

X509_STORE *store = X509_STORE_new();

X509_LOOKUP *lookup = X509_STORE_add_lookup(store, X509_LOOKUP_file());

X509_LOOKUP_load_file(lookup, "trust.pem.cer", X509_FILETYPE_PEM);

X509_STORE_set_purpose(store, XKU_CODE_SIGN); /* see caveat above */

X509_VERIFY_PARAM_set_flags(

    X509_STORE_get0_param(store),

    X509_V_FLAG_NO_CHECK_TIME | X509_V_FLAG_PARTIAL_CHAIN);

int retcode = PKCS7_verify(p7, NULL, store, NULL, NULL, PKCS7_NOSIGS);

For a verification of the signature itself: the function verify_pe_pkcs7() in osslsigncode.c gives example code to do that. Its PKCS7_verify() invocation does not verify the certificate chain, but it does check the signature. This requires extraction of a hash which is stored in a Microsoft-specific element in the p7 of a type called SpcIndirectDataContent, as pointed out by @dave_thompson_085 below. It is possible to verify the signature that was taken over that hash. For a complete verification, you will also need to re-calculate the hash over the PE file itself and compare it to the hash value found in the p7.

This answer is based on OpenSSL 1.1.1. Just now, I realize that you are using libressl, which is based on a (much) older version of OpenSSL. It might not work in your case. For example, for my version of libressl, the smime -verify tool does not support the partial_chain and no_time_check options because those were introduced in the OpenSSL 1.1.0 branch.