Using OpenSSL tool to check SSL certificates for expiration dates

I am sure many of the Application or System administrators might have encoutered the issue of SSL certificates getting expired in the middle of the day causing application outages. The part of the reason is that the application server environments are getting complex day by day in terms of number of systems it's interacting over SSL like LDAP, WebServices, WebSever plugin, Siebel and even database connectivity in highly secured environments. It further complicates as these are disparate systems maintained by different groups in the enterprise having different expiration dates and different formats of keystore and truststore. So unless you have good enterprise wide policy of checking SSL expiration dates well ahead, there is a very good chance that you might run into application failures.

Being an Websphere Application Administrator, you will be the first point of contact when application goes down, even the SSL certs expired in one of the system that it's intreacting with like the LDAP wich is used for security authentication. In order to diagnose , identify and notify the repsective system owner, openSSL tool comes handy in such a way that you just need to know the hostname and the port where the services SSL port is listening on and you will be able to find the expiration dates without having to know the password for the keystore or the format or on how to access and view the keystore. The tool usually gets installed by default in unix systems and in windows you can download and install it.


Steps to check the SSL Certificate expiratio using openSSL tool:

1) openssl s_client -connect hostname:port > cert - this command will get the certificate and redirect it to the file.
2) openssl x509 -in cert -noout -enddate - show the expiration date of the downloaded certificate.


(e.g) To check the expiration for www14.software.ibm.com webserver host

$openssl s_client -connect www14.software.ibm.com:443 > cert

Loading 'screen' into random state - done
depth=1 /C=US/O=Equifax/OU=Equifax Secure Certificate Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0

$openssl x509 -in cert -noout -enddate

notAfter=Dec 8 13:00:22 2009 GMT

Note not only this works with webserver it will work with any service with SSL turned on.

You can also check other information about the certificates without redirecting to the file. Also you can check my other article for SSL Certificates expiration monitoring for WebSphere or any java based application server using java keystore as well.

$openssl s_client -connect www14.software.ibm.com:443

Loading 'screen' into random state - done
CONNECTED(000006DC)
depth=1 /C=US/O=Equifax/OU=Equifax Secure Certificate Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/C=US/ST=New York/L=Armonk/O=IBM/CN=www14.software.ibm.com
i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
1 s:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC1TCCAj6gAwIBAgIDCiE+MA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT
MRAwDgYDVQQKEwdFcXVpZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkwHhcNMDgxMTA3MTMwMDIyWhcNMDkxMjA4MTMwMDIy
WjBgMQswCQYDVQQGEwJVUzERMA8GA1UECBMITmV3IFlvcmsxDzANBgNVBAcTBkFy
bW9uazEMMAoGA1UEChMDSUJNMR8wHQYDVQQDExZ3d3cxNC5zb2Z0d2FyZS5pYm0u
Y29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC4tUCL4Ar8Sx8pNEbRb+XQ
7CMa1fgh1k2p0zucagZeX/C8e3DMefU5UH3pJKjYZoYte6PWB92jwtw673Rj0Oq6
oDBljHbdtYo0H58C6QqSkjMLFEKyNBHOAZ1SGws11H5WCz+xiOZ1c5Xq11Jjl/BM
nu6PKbfKUUrBwIdTfK/heQIDAQABo4GuMIGrMA4GA1UdDwEB/wQEAwIE8DAdBgNV
HQ4EFgQUuB0DNKiqXdoH87tWzq+71OYK7FUwOgYDVR0fBDMwMTAvoC2gK4YpaHR0
cDovL2NybC5nZW90cnVzdC5jb20vY3Jscy9zZWN1cmVjYS5jcmwwHwYDVR0jBBgw
FoAUSOZo+SvSspXXR9gjIBBPM5iQn9QwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
AQUFBwMCMA0GCSqGSIb3DQEBBQUAA4GBAExSdcrIMiwQqDOAelP4C6QiUeXgis2g
8ePc17lrt2NjRe1aRoSmMYq2U7Rc3203L3WFQ/SOFzfUSKWwBjx98IPFGOuKTMza
DYe1xknf+jjg8IwzhAtcg9Y06jabJBwxMXebSH2lazqFSD8ztBedBjLM7/zQ/Ttz
a7e4t98R0/fX
-----END CERTIFICATE-----
subject=/C=US/ST=New York/L=Armonk/O=IBM/CN=www14.software.ibm.com
issuer=/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
No client certificate CA names sent
---
SSL handshake has read 1688 bytes and written 322 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : AES256-SHA
Session-ID: 00038028A696989D0973A59DEF091113DCCB885C585858584A80A84C0000298D
Session-ID-ctx:
Master-Key: 6A694E8816CE422DB3AD280BEC469ACBDBE1EB7BF116C5E3C600A1A68CC71B7864AE39D8A59CC1F07263C1AB1664238D
Key-Arg : None
Start Time: 1249945678
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)

SSL Certificates expiration monitoring for WebSphere or any java based application server using java keystore

If you are a WebSphere Administrator or any Application Server (WebLogic, Tomcat) administrator you might already know that managing the ssl certificates in a large complex environments becomes hectic and troublesome because of the different expiration dates of the certificates that websphere uses and also the ssl certificates of the external systems (like SAP , Siebel) that websphere applicaton server might connect to using a secure connection, multiple administrators in an organization renewing it and not keeping tracking of the expiration dates. The problem is , SSL certificate might expire resulting in servers will not initialize and your running servers will stop operating, becomes unresposive if not properly renewed on time. Hence this article will explain how you can monitor the expiration of the ssl certificates using a simple command and proactively monitor the expiration dates, setup your calendar and renew it on time to prevent any downtimes.

If you are using the latest version (as on Dec 2007) of WebSphere Application Server 6.1 and using default self signed there are provisions in the application server itself to effectively monitor the certificate expiration, notifying you and renewing it automatically. If you are using a perosnal self signed certificate or a personal certificate signed by a certificate authority (CA) like VeriSign or Thawte , which is most likely in all of your production environments then you might want to use the below command to find the expiration date and renew it.

If you are using a java keystore ( DummyServerKeyFile.jks , DummyServerTrustFile.jks , jssecerts , cacerts) use the below find command along with keytool or websphere's ikeyman command line option to find the expiration of the keys.

LINUX:


Using Ikeyman cmdline utility (ikeyman.sh):

find PATH -name *.jks | xargs -i bash -c '$WAS_HOME/java/bin/java -classpath $WAS_HOME/java/jre/lib/ext/ibmjceprovider.jar:$WAS_HOME/AppServer/java/jre/lib/ext/ibmjcefw.jar:$WAS_HOME/AppServer/java/jre/lib/ext/US_export_policy.jar:$WAS_HOME/AppServer/java/jre/lib/ext/local_policy.jar:$WAS_HOME/AppServer/java/jre/lib/ext/ibmpkcs.jar:$WAS_HOME/AppServer/java/jre/lib/ext com.ibm.gsk.ikeyman.ikeycmd -cert -list all -expiry -db {} -type jks -pw <password> | grep "Not After:.*yyy[y,y]" -B 3 && echo {}'

(e.g)
find /usr/IBM/WebSphere/AppServer/profiles/ -name *.jks | xargs -i bash -c '/usr/IBM/WebSphere/AppServer/java/bin/java -classpath /usr/IBM/WebSphere/AppServer/java/jre/lib/ext/ibmjceprovider.jar:/usr/IBM/WebSphere/AppServer/java/jre/lib/ext/ibmjcefw.jar:/usr/IBM/WebSphere/AppServer/java/jre/lib/ext/US_export_policy.jar:/usr/IBM/WebSphere/AppServer//java/jre/lib/ext/local_policy.jar:/usr/IBM/WebSphere/AppServer/java/jre/lib/ext/ibmpkcs.jar:/usr/IBM/WebSphere/AppServer/java/jre/lib/ext com.ibm.gsk.ikeyman.ikeycmd -cert -list all -expiry -db {} -type jks -pw WebAS | grep "Not After:.*200[7,8]" -B 3 && echo {}'

Using Java Keytool utility:

find $WAS_HOME -name '*.jks' | xargs -i bash -c "echo {}; $WAS_HOME/java/jre/bin/keytool -list -v -keystore {} -storepass password | grep 'until: .*/.*/Y[Y,Y]' -A 3 -B 7"

(e.g)
find /usr/IBM/WebSphere/AppServer -name '*.jks' | xargs -i bash -c "echo {};/usr/IBM/WebSphere/AppServer/java/jre/bin/keytool -list -v -keystore {} -storepass WebAS | grep 'until: .*/.*/0[7,8]' -A 3 -B 7"

Output:

/usr/IBM/WebSphere/AppServer/profiles/etc//WWWStageClientKey.jks
Alias name: verisign class 1 ca individual subscriber-persona not validated
Creation date: Dec 18, 2006
Entry type: trustedCertEntry

Owner: CN=VeriSign Class 1 CA Individual Subscriber-Persona Not Validated, OU="www.verisign.com/repository/RPA Incorp. By Ref.,LIAB.LTD(c)98", OU=VeriSign Trust Network, O="VeriSign, Inc."
Issuer: OU=Class 1 Public Primary Certification Authority, O="VeriSign, Inc.", C=US
Serial number: d8b4feeaad218df5bf4756a9d29e17ffb
Valid from: 5/12/98 12:00 AM until: 5/12/08 11:59 PM
Certificate fingerprints:
         MD5:  CA:66:3C:FC:71:2B:BA:41:92:71:DD:72:AD:E5:65:65
         SHA1: 12:51:9A:E9:CD:77:7A:56:01:84:F1:FB:D5:42:15:22:2E:95:E7:1F
--
Creation date: Dec 18, 2006
Entry type: keyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=wadm.stage.WWW.com, OU=IT, O=WWW, L=Fremont, ST=CA, POSTALCODE=94089, C=US
Issuer: CN=wadm.stage.WWW.com, OU=IT, O=WWW, L=Fremont, ST=CA, POSTALCODE=94089, C=US
Serial number: 458709ef
Valid from: 12/18/06 9:36 PM until: 12/18/07 9:36 PM
Certificate fingerprints:
         MD5:  C2:4F:CC:SS:19:DC:E4:88:B0:2A:78:76:69:4D:DC:EF:47
         SHA1: 53:AB:D0:13:45:45:12:59:64:F6:C9:38:41:F2:C8:E3:37:05:73:95:F3:83

/usr/IBM/WebSphere/AppServer/profiles/etc/wwwStageServerKey.jks

Alias name: www stage websphere server ca
Creation date: Dec 8, 2006
Entry type: keyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=wadm.stage.www.com, OU=IT, O="www, Inc.", L=Fremont , ST=California, C=US
Issuer: OU=Secure Server Certification Authority, O="RSA Data Security, Inc.", C=US
Serial number: 1ea5f21b33ca65e551440d662399dffdfga382
Valid from: 12/5/06 12:00 AM until: 12/15/07 11:59 PM
Certificate fingerprints:
         MD5:  C2:22:E4:12:12:6D:3F:22:67:4E:37:48:12:3F:F3:1B:FE:26
         SHA1: CA:81:37:A7:E9:12:22:19:27:D9:3F:C5:79:E9:25:C8:1A:3C:1E:5C:17:02

As you can see in the above output the command will list the file name (/usr/IBM/WebSphere/AppServer/profiles/etc/wwwStageServerKey.jks) , certificate label (www stage websphere server ca) , Entry Type (keyEntry - personal cert or trustedCertEntry - Signer Cert), and the expiration date (12/5/06 12:00 AM until: 12/15/07 11:59 PM) of
all the java keystore (*.jks) in the given directory that expired in 2007 (07) or 2008 (08). By which you shoule be able to indentify the certs, setup your calendat atleast 10 days before the expiration and renew it to proactively to preveny any problems.

Note you can also change the above command by passing differnt filename like cacerts and jssecerts and also pass different year infomration to find expirations during those years.

Sometimes you might get no output which means either there are no expiring ceritificates for the years that you passed or if you are expecting any expiring certificate to show and if not then the password for the keystore might be wrong, so run without the grep portion in the above command which will list all the certificates irrespective of the expiration date or it will complain if the password is wrong as below.

find /usr/IBM/WebSphere/AppServer -name '*.jks' | xargs -i bash -c "echo {};/usr/IBM/WebSphere/AppServer/java/jre/bin/keytool -list -v -keystore {} -storepass WebAS

/usr/IBM/WebSphere/AppServer/profiles/etc/wwwStageServerKey.jks
java.io.IOException: Keystore was tampered with, or password was incorrect


Also you can use the same command by modifying the -type parameter to "-type cms" for the webserver such as IBM HttpServer where it's using cms database ( kdb file).


find /usr/IBM/HttpServer -name *.kdb | xargs -i bash -c '/usr/IBM/WebSphere/Plugins/java/bin/java -classpath /usr/IBM/WebSphere/Plugins/java/jre/lib/ext/ibmjceprovider.jar:/usr/IBM/WebSphere/Plugins/java/jre/lib/ext/ibmjcefw.jar:/usr/IBM/WebSphere/Plugins/java/jre/lib/ext/US_export_policy.jar:/usr/IBM/WebSphere/Plugins//java/jre/lib/ext/local_policy.jar:/usr/IBM/WebSphere/Plugins/java/jre/lib/ext/ibmpkcs.jar:/usr/IBM/WebSphere/Plugins/java/jre/lib/ext com.ibm.gsk.ikeyman.ikeycmd -cert -list all -expiry -db {} -type cms -pw WebAS | grep "Not After:.*200[7,8]" -B 3 && echo {}'