Debugging SSL

Currently we are in the process of upgrading some components in our cyberinfrastructure (catchy word huh..) in a move towards more standardised solutions. Our existing system is more or less home grown and has it's own customized SSL layer based on puretls to handle the mutual authentication based on PEM encoded X509 certificats. We are currently moving towards Tomcat based solutions for some parts of our infrastructure. Part of our challenge is to figure out how to setup SSL mutual authentication on Tomcat. Other half of the challenge is to make that work witht he existing clients of other components. 

Soon it proved to be a very rough ride, taking weeks to debug, especially since I'm new to SSL stuff. If you are also new to SSL and want to debug a SSL setup, chances are high that you'll also go on the same path as I went. We tried to use the various clients (existing puretls based, jsse) to debug and figure out what's going on, hoping the stacktraces will give us a clue.. Oh...wait... We are dealing with security.. Even the error messages are encrypted (or is it security by obscurity), as any error would translate to a very small set of generic error messages. This is when OpenSSL command line tool came to rescue us..

Openssl command line program provides use with tools to verify certificates, ssl clients to test SSL enabled servers and even supports setting up a temporary SSL enables server to test your clients. 

  • Testing the mutual authenticaton enabled server SSL setup
[me@home tmp]$ openssl s_client -connect localhost:8443 -cert client_cert.pem -CAfile server_ca.pem -state -key client_key.pem

Above command will use client_cert.pem to authenticate himself to the server and will use the certificate of the trusted CA, server_ca.pem , to authenticate the server. "-state" will give more detailed debug information. Believe me, the error codes I got from this are very specific and quickly lead us to the issue we had. Above command hinted use that something is wrong in our client certificate. This is something we never thought of earlier, as our existing system works well with these certificates. Before this we mainly thought that the error occured due to lack of interoperability between mod_ssl & much older puretls implementation.

  •  Verifying the certificate
[me@home tmp]$ openssl verify -CAfile ca.pem -purpose sslclient client_cert.pem

The above command threw an error mentioning that the "purpose" of our client certificate is not sslclient, which in fact was the bug responsible for many of my fallen hair during the last week. Eventually we found that our existing system which is based on Puretls does not validate this certificate extensions, which made our exisiting system to work even with this bug.

All and all I found openssl command line program to be a very usefull/helpfull extensive tool which comes very handy when debuging SSL setups. 

Facebook photo storage

Wow... It's massive...

Stats from facebook engineering blog..
  • 2-3 Terabytes of photos are being uploaded to the site every day
  • Just over one petabyte of photo storage
  • Serve over 15 billion photo images per day
  • Photo traffic now peaks at over 300,000 images served per second
Wonder how the facebook software stack looks like.. There must be lot of innovation & research going on to make their platform's scale with the above numbers, especially given that photo storage is just a small part of there offerings. Fortunately they have already put out some of the components as open source projects.

In my earlier post I mentioned how to setup tomcat to support SSL (https) using the PEM-encoded X509 certificate files. This post will be a followup to that specifying how to add SSL client authentication to tomcat using the PEM-encoded X509 certificates.

<!-- Define a SSL HTTP/1.1 Connector on port 8443 -->
<connector port="8443" maxhttpheadersize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" disableUploadTimeout="true"
acceptCount="100" scheme="https" secure="true"
sslProtocol="TLS"
SSLEngine="on"
SSLVerifyClient="require"
SSLCertificateFile="/path/yourcert.pem"
SSLCACertificateFile="/path/your_trusted_cas.pem"

SSLCertificateKeyFile="/path/yourkey.pem"
SSLPassword="yourpasswordtothekey"/>


One important point to keep in mind is that mod_ssl does not honor the the tomcat default "clientAuth" parameter. You need to make sure to use "SSLVerifyClient" parameter instead of clientAuth.

Also you need to provide the certificates of the certification authority(CA), who issued the certificates to your clients. You can do it either by providing a single file and using the "SSLCACertificateFile" as above or providing a directory containing the certificates and using the "SSLCACertificatePath" attribute.

Comment Moderation Turned On!!!

Thanks to a bunch of comment spammers, I had to switch "on" the comment moderation on this blog..

Hope everybody comes here would bare with it.. I'll make sure to publish the comments ASAP, unless it's a pure spam..

I came across this interesting post about photographers hating Creative Commons license. One of the points discussed was that, what if somebody uses one of the photographs you put in to public domain to a purpose you don't accept. It can be a corrupted political movements or a company with an evil mission... I really don't think I personally would be comfortable with the situation, so might be the society around me. This gets much worse if you have your self, somebody else or a personal property appearing in the photo. The solution may be to add a clause to the license declaration that requires the users to get written permission from the owner of the photo before using it. I'm not sure whether Creative Commons supports it or not.. But this might ruin the motive and the advantages you get by making it open at the first place.

Luckily or unluckily I do not think my photographs are worth reusing, and I do not worry about making them available online.. But what really strike me was whether the same issue applies to the open source developments we are doing. "What if somebody uses the open source software we wrote/writing for evil motives that we do not want to contribute at all?". Been a strong promoter of open development for years, it's surprising that this did not come to my mind. May be because the usages are not so clear as in the case of using photographs. I sincerely hope nobody will use open source software for evil causes. But I'm still thinking....................

Where the hell is Matt?.

This video is a must see :)...



I wish someday I'll be able to visit at least half the countries he has been to..

Real Time Flight Monitoring

Wow.. This is super cool and pretty usefull, specially when your loved one's are on air. Check it out at http://www.flightstats.com/.

10 Tips to pass the Driving Test

California DMV has done a great work by coming up with a set of videos depicting top 10 reasons for failing a driving test and guiding how to get through them correctly. I'll be having my driving test soon and luckily I found them today. The information in the video's helped me to clear out some of the unclear situations I had in my mind. Specially the things like "do not drive too slowly when you are under normal conditions , not slow than 10 Mph than the given speed limit" would not come to the mind of many test takers.

I'm sure these are helpful not only to pass the driving test but also as a great source of guidance. If you are a international driver who thinks of driving in USA using an International Driving Permit, I strongly recommend you to go through these videos irrespective of the driving experience you have on your country.

Cudo's to California DMV for creating them and posting them to YouTube.



#2
#3
#4
#5
#6
#7
#8
#9
#10

Google Code University

Check it out...
http://code.google.com/edu/

Currently they have courses for Ajax, Web Security, Distributed systems & languages. Interesting stuff..

Go Mora!!!

Wow. This is super cool. University of Moratuwa,where I did my undergrad studies had the most number of Google Summer of Code 2008 applicants and the most number of accepted proposals. Congratulations to all the summer of code award winners from Mora and to the driving forces behind, Dr. Sanjiava Weerawarana & Ms. Vishaka Nanayakkara. According to the sources 18 out of 24 students are from my old department .

At this moment I'm very proud to be among the first set of Sri Lankan GSOC 2005 winners, together with Dinesh, Chathura and Indika. All four of us continued our contributions to OSS and became Apache committers and went further to become PMC members in various Apache projects giving a proof to the long term positive effects of GSOC. After the first GSOC we tried our best to do our share of promoting OSS & GSOC in Sri Lanka by word of mouth as well as by organising promotion meetings. It soooo nice when we hear that our younger generation is continuing the tradition. Keep up the great work !!!!

For those of you who wonder, Sri Lanka is a pioneer in OSS contributions in Asia. Sri Lanka has produced a huge number Apache committers, which I believe is the most from a country outside US & Europe. Most of these committers are from the University of Moratuwa. Apart from Apache, we can see Sri lankan contributors in many other OSS projects including our own Sahana. Full credit for this large number of OSS contributions in Sri Lanka should go to LSF, WSO2 and their founder Dr.Sanjiva Weerawarana.



Copyright 2006| Blogger Templates by GeckoandFly modified and converted to Blogger Beta by Blogcrowds.