Introduction
When the web was in it’s infancy, data security was of little concern. Businesses were still trying to figure out what the web could be used for beyond a presence on a new emerging technology they had little idea about. Amazon entered the scene in 1994 and over time e-commerce developed as a new way of selling things. The data security technology of the time was vastly undercooked and misunderstood at best, requiring each vendor to jimmy up their own solution using CGI-BIN or similar. This may have partially addressed the authentication issue (if at all), but the gate for data in transit was left wide open. An industry standard needed to be developed to fill the vacuum.
SSL (secure Sockets Layer) was devised by the browser company of the day, Netscape Communications. Necessity being the mother of invention, Nestscape also gave us scripting on the browser through the poorly named ‘Javascript’ language and cookies to make an inherently stateless protocol stateful.
For data encryption and authentication, Netscape realised that something needed to be hooked in to the server side and browser in concert to ensure that websites could be a) trusted by the user and b) data could be sent confidentially. Both issues being critical to the uptake and acceptance of e-commerce.
Netscape disappeared after a few years being a victim of the browser wars of the late 1990’s, being usurped by Internet Explorer and other browsers. They left the world with the blueprints to internet security – and programmers many bad days trying to understand it all.
This article will briefly touch on SSL but spend more time on Mutual SSL with a practical example in Java SpringBoot. Mutual SSL extends the concept of SSL outside of the simple client(/browser) server(/web host) scenario. Taking what was built for the web and applying it to a business setting where geographically disparate systems can communicate sensitive data over a public internet.
HTTPS is an industry standard method for encrypting traffic between the client and server in transit. It thwarts anyone listening to the traffic from being able to piece together the flow of data (aka Man in the middle attack) by performing encryption at the session layer. It makes for a good use case for authentication and data security in a re-adapted form.
Before discussing mutual SSL, a refresher on Single SSL. The concepts will help to learn mutual SSL
SSL primer (skip if you know this already)
To explain, lets take a look at the case of a user visiting a website https://example.com
When a user visits this site with a standard web browser, the page is served and the browser displays a padlock to signify that the page is secure. Behind the scenes the client and server are performing a number of operations that are opaque to the user. This includes swapping available cipher lists, setting a symmetric session key for data encryption, performing public key operations on signed certificates to ensure that the website is legitimate. More detail on these operations follows.
Firstly, the HTTPS server sends a certificate back to the user. This certificate is essentially a public key wrapped up with some additional information. This additional information is in the form of an X509 standard. This is a very old format existed before the common web as we know it and was developed for the LDAP protocol . It contains a number of standard fields. The most important being the CN (Common Name). This has been retrofitted for the web host to store the server’s domain name instead. The name of the organisation, it’s address, email address and other metadata including the expiry of the certificate are stored in this record.
An example certificate is shown below:
Essentially a certificate is just a public key with some additional describing metadata enclosing it.
When the browser has received the certificate from the server, a check is made to determine whether it should trust or deny the connection based on the certificate. It does this by looking at it’s trust store.
A trust store is a set of 90 or so certificates that are pre-loaded in to the browser. It might also be referred to as CA Certs or root certificates. Java has it’s own version of trust store with a number of different vendors. How do we know that these certificates are trustworthy? We just take it axiomatically that there are well known organisations out there that have been granted the privilege to vet companies and perform these actions. (There are means to revoke certificates that are deemed to be compromised, but that’s another story best not told)
The appropriate certificate (let’s say it’s signed by Verisign) is loaded by the browser. The public key of the CA certificate is used to “unlock” the server certificate that has been sent to the browser. It can be determined if the server’s certificate was signed by Verisign (in this example) by performing a public key decryption on it.
If the certificate is successfully verified, nothing special happens. The web page is loaded normally, and that is a good outcome. If the certificate cannot be verified, then a prompt will display the error condition that the certificate cannot be trusted. Most people will have come across this at least once.
Certificate Authority setup
At some point in time Example.com will have generated a certificate on it’s own. A public key would be randomly generated and the details (domain name, physical address and so on) of the X509 certificate typed in.
To give more credibility (and avoid browser security errors) the web host pays a fee to a CA to sign their certificate. It is their job to vouch for the credentials found in a certificate. They may use old school ways of doing this. Checking the address, telephone number, website domain records etc.
Once satisfied, a CA can choose to SIGN (encrypt with their CA private key) the certificate and send it back to the requestor.
This is known as a CSR (Certificate Signing Request). It’s a once-off backoffice function. The CA will send the certificate back to the requestor by some means.
Now at this point the web host has a certificate that has been authenticated – the same way that the government may a vouch for a citizen with a passport. It’s an extra level of assurance. In PKI lingo this is known as a “trust-chain”. On any website secured with HTTPS, one can view a trust chain. There is the option for a certificate to be signed by another certificate which in turn is (down the line) trusted by a ROOT certificate.
To recap: The CA already has it’s own counterpart public key stored in a set of well known trusted certificates. These are pre-loaded in to the browser and are implicitly trusted. The private key is held by the CA and never disclosed!
The web host has a public key in a certificate that is ultimately sent back to the web browser.
The browser is able to use the public key of the browser ca’s to correctly unlock the signed certificate (PubKa match with PrivKa). The public key is used to encrypt a shared symmetric session key. Everything works and this is how a browser can now trust an HTTPS connection.
In a browser centric view, single SSL is sufficient since we only require the server to authenticate itself to the browser. In a business setting a requirement may be for applications to authenticate and encrypt traffic each other. Typically an application may act as both a server and a client. Therein lies a challenge!
Mutual SSL allows for both endpoints to authenticate each other, as if the browser becomes a server and a server becomes a browser. The extra SSL handshake is performed in one call with a few more steps taken.
The stand out benefit of mutual SSL over other forms of authentication is that it is a non-interactive password-less solution. Two applications running on separate hosts can authenticate and swap traffic confidentially all by virtue of each host configuring each other’s trust-store as a one time setup.
Code
I will first show a SpringBoot application that serves a simple JSON response, configure it to run on SSL and configure it’s keys. Next, a client will be coded to show the mutual authentication taking place. This will be verified by use of a protocol analyser to show the frames of data on the wire.
First things first: The tricky aspect of setting up the keys. Both the client and server require the keys to be generated in a specific way. This can be done with utilities such as keytool or openssl. For this tutuorial, we wil use keytool which is shipped with Java.
These steps will generate the necessary keys for client and server:
#create server certificate. For "First and last name" [CN] type "localhost". For Orginizational Unit [OU] type "Yin"
keytool -genkeypair -alias server-keypair -keyalg RSA -keysize 2048 -validity 3650
-keypass password -keystore server-keystore.jks -storepass password
#export keys(public) from above
keytool -exportcert -alias server-keypair -file server-public-key.cer -keystore
server-keystore.jks -storepass password
#import above key in the client trust store and give to client
keytool -importcert -keystore client-truststore.jks -alias server-public-key -file
server-public-key.cer -storepass password -noprompt
# Reverse the procedure to create client certificate
# For "First and last name" [CN] type "localhost". For Orginizational Unit [OU] type "Yang"
# generate client keystore
keytool -genkeypair -alias client-keypair -keyalg RSA -keysize 2048 -validity 3650
-keypass password -keystore client-keystore.jks -storepass password
# export keys public from above
keytool -exportcert -alias client-keypair -file client-public-key.cer -keystore
client-keystore.jks -storepass password
# import it into the server keystore and give to server
keytool -importcert -keystore server-truststore.jks -alias client-public-key -file
client-public-key.cer -storepass password -nopromptWe now have the keys setup. The intermediate certs (step 2 and 5) can be deleted.
A useful tool to inspect these files is keytool explorer (https://keystore-explorer.org/).
Opening the keystore we can see that the identity key has a private and public key. The truststore has a certificate only – which is the public key that was extracted out from the keystore.
Java Springboot application
The application can be pulled from github or the zip file can be downloaded.
Set up a SpringBoot application; and configure the properties as so:
security.require-ssl=true
server.ssl.key-store-type=JKS
server.ssl.key-store=classpath:server-keystore.jks
server.ssl.key-store-password=password
server.ssl.key-alias=server-keypair
server.ssl.trust-store=classpath:server-truststore.jksserver.ssl.trust-store-password=passwordserver.ssl.trust-store-type=JKSserver.ssl.client-auth=want
The keystore contains the private/public key pair of the server itself
The truststore contains the certificate of the client
Model
@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonPropertyOrder({
"image",
"desc",
"title"
})
public class Response implements Serializable
{
@JsonProperty("image")
private String image;
@JsonProperty("desc")
private String desc;
@JsonProperty("title")
private String title;
@JsonIgnore
@Valid
private Map<String, Object> additionalProperties = new HashMap<String, Object>();
private final static long serialVersionUID = 8531467032914953885L;
/**
* No args constructor for use in serialization
*
*/
//import com.fasterxml.jackson.annotation.JsonProperty;
public Response() {
}
/**
*
* @param title
* @param image
* @param desc
*/
public Response(String image, String desc, String title) {
super();
this.image = image;
this.desc = desc;
this.title = title;
}
@JsonProperty("image")
public String getImage() {
return image;
}
@JsonProperty("image")
public void setImage(String image) {
this.image = image;
}
public Response withImage(String image) {
this.image = image;
return this;
}
@JsonProperty("desc")
public String getDesc() {
return desc;
}
@JsonProperty("desc")
public void setDesc(String desc) {
this.desc = desc;
}
@JsonProperty("title")
public String getTitle() {
return title;
}
@JsonProperty("title")
public void setTitle(String title) {
this.title = title;
}
@JsonAnyGetter
public Map<String, Object> getAdditionalProperties() {
return this.additionalProperties;
}
@Override
public int hashCode() {
return new HashCodeBuilder().append(title).append(additionalProperties).append(image).append(desc).toHashCode();
}
@Override
public boolean equals(Object other) {
if (other == this) {
return true;
}
if ((other instanceof Response) == false) {
return false;
}
Response rhs = ((Response) other);
return new EqualsBuilder().append(title, rhs.title).append(additionalProperties, rhs.additionalProperties).append(image, rhs.image).append(desc, rhs.desc).isEquals();
}
}
Rest endpoints
There will be 3 endpoints set up, named bronze, silver and gold.
Bronze – will deny all connections
Silver – will allow all connections
Gold – will only allow access to a client that has mutually authenticated
The code for the REST endpoints is as follows:
Bronze
@PostMapping(value = "/bronze/person", produces = "application/json")
@ResponseBody
public ResponseEntity personMethodTwo(@RequestBody Person person) {
preamble(person);
ResponseContainer responseContainer = new ResponseContainer();
List response = Collections.singletonList(new Response("Image2", "Basketball", "The Title2"));
responseContainer.setResponse(response);
return new ResponseEntity<>(responseContainer, HttpStatus.OK);
}
Silver
@PostMapping(value = "/silver/person", produces = "application/json")
@ResponseBody
public ResponseEntity personMethodThree(@RequestBody Person person) {
preamble(person);
ResponseContainer responseContainer = new ResponseContainer();
List response = Collections.singletonList(new Response("SoccerBall.jpg", "Soccer ball", "The Title3"));
responseContainer.setResponse(response);
return new ResponseEntity<>(responseContainer, HttpStatus.OK);
}
Gold
@PostMapping(value = "/gold/person", produces = "application/json")
@ResponseBody
public ResponseEntity personMethod(@RequestBody Person person) {
logger.info("Incoming info (displayed and discarded)");
logger.info("Name {}", person.getName());
logger.info("AGE {}", person.getAge());
SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
logger.info("Hit the service at {}", format.format(new java.util.Date()));
ResponseContainer responseContainer = new ResponseContainer();
List responseList = new ArrayList<>();
Response response1 = new Response();
Response response2 = new Response();
response1.setDesc("Beachball");
response1.setImage("ImageBeachBall.jpg");
response1.setTitle("Title 1");
response2.setDesc("Basketball");
response2.setImage("ImageBasketball.jpg");
response2.setTitle("Title 2");
responseList.add(response1);
responseList.add(response2);
responseContainer.setResponse(responseList);
return new ResponseEntity<>(responseContainer, HttpStatus.OK);
}
A lot of the boilerplate code is out of the way. Now to draw attention to the facilities that Spring provides for securing endpoints.
Antmatchers
Antmatcher is a regex-like way of specifying patterns and its heritage comes from ant build tool.
The first consideration is whether the entire application is to be secured via mutual SSL auth. This is controlled by the flagserver.ssl.client-auth=want
The flag can also be set to “need” which will require all connections to be mutually authenticated. For this application, we will set it to want as we still want other endpoints to function without needing mutual SSL.
To provide this configuration extend the WebSecurityAdapter class.
@Value("${X509SubjectMatch}")
private String valueToMatch;
@Override
protected void configure(HttpSecurity http) throws Exception {
String x509Subject = String.format("CN=localhost(.*%s.*)",valueToMatch);
X509AuthenticatedUserDetailsService x509UserDetails = new X509AuthenticatedUserDetailsService();
x509UserDetails.setMatch(valueToMatch);
http.csrf().disable()
.authorizeRequests()
.antMatchers("/bronze/person").denyAll()
.antMatchers("/silver/person").permitAll()
.and().authorizeRequests()
.antMatchers("/gold/person").authenticated()
.and().x509().subjectPrincipalRegex(x509Subject).authenticationUserDetailsService(x509UserDetails);
}
The bronze and silver endpoints are controlled by the simple predicates denyall() and permitAll() respectively. We will use these as our base cases for testing.
For mutual SSL endpoint, we will need a way to selectively apply certificate inspection of the client. This is controlled by the following code
.and().authorizeRequests()
.antMatchers("/gold/person").authenticated()
.and().x509().subjectPrincipalRegex("CN=localhost(.*something.*)").authenticationUserDetailsService(new X509AuthenticatedUserDetailsService())
What is happening here is that the incoming certificate from the Client (which will be built and explained later) is being sent to the server. The server will read the subject line of the certificate. If it matches a value that we expect to see, then both endpoints will be mutually authenticated. If the subject doesn’t match in some regard, the connection is denied.
A curious aspect is that the x509 predicate requires a UserDetailsService to be added. In this case there isn’t really a notion of a user or a password. We create this user class all the same, and perform additional checks. The password field can be left blank.
@Component
public class X509AuthenticatedUserDetailsService implements AuthenticationUserDetailsService<PreAuthenticatedAuthenticationToken> {
private final org.slf4j.Logger logger = LoggerFactory.getLogger(this.getClass());
private String partialSubject;
@Override
public UserDetails loadUserDetails(PreAuthenticatedAuthenticationToken token)
throws UsernameNotFoundException {
logger.info("Entered loadUserDetails(...)");
X509Certificate certificate = (X509Certificate)token.getCredentials();
if (!certificate.getSubjectX500Principal().getName().contains(partialSubject)) {
logger.warn("This is an unknown / unexpected cert");
}
Collection<GrantedAuthority> authorities = Collections.EMPTY_LIST;
return new User(certificate.getSubjectX500Principal().getName(), "(un-necessary for certificate validation)", authorities);
}
public void setMatch(String partialSubject) {
this.partialSubject = partialSubject;
}
}
The incoming client certificate can be inspected and it’s values read:
Next, we need to build a client to test it. Postman can be used, though you will need to configure it to send certificates (not explained in this article). Instead a small homegrown client will be built to show how certificates can be sent across in an HTTPS call. Remember, for normal HTTPS calls (for instance connecting to a website), this is not normally a concern for the user. All that would be required is changing HTTP to HTTPS.
For mutual authentication SSL, we will require the client to send the certificate to the server, so that the server can validate it.
Client
The client will make use of ApacheHTTP Client for making outbound calls to the ReST service.
The example JSON payload will look something like:
public static final String ONE_JSON = "{\n" +
"\t\"name\" : \"tom\",\n" +
"\t\"age\" : \"23\"\n" +
"}";
package com.alphastar.service;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.PrivateKeyDetails;
import org.apache.http.ssl.PrivateKeyStrategy;
import org.apache.http.ssl.SSLContexts;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.io.ClassPathResource;
import org.springframework.stereotype.Component;
import com.alphastar.statics.JsonFiles;
import javax.annotation.PostConstruct;
import javax.net.ssl.SSLContext;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Socket;
import java.security.*;
import java.security.cert.CertificateException;
import java.util.Map;
@Component
public class HttpClientSend {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
@PostConstruct
public void go() {
try {
String CERT_ALIAS = "client-keypair", CERT_PASSWORD = "password";
// use classpath to avoid filesystem. File should be in resources folder root dir
InputStream clientKeystore = new ClassPathResource(
"client-keystore.jks").getInputStream();
InputStream clientTruststore = new ClassPathResource(
"client-truststore.jks").getInputStream();
KeyStore identityKeyStore1 = KeyStore.getInstance("jks");
identityKeyStore1.load(clientKeystore, CERT_PASSWORD.toCharArray());
KeyStore trustKeyStore1 = KeyStore.getInstance("jks");
trustKeyStore1.load(clientTruststore, CERT_PASSWORD.toCharArray());
SSLContext sslContext = null;
try {
sslContext = SSLContexts.custom()
// load identity keystore
.loadKeyMaterial(identityKeyStore1, CERT_PASSWORD.toCharArray(), new PrivateKeyStrategy() {
@Override
public String chooseAlias(Map<String, PrivateKeyDetails> aliases, Socket socket) {
return CERT_ALIAS;
}
})
// load trust keystore
.loadTrustMaterial(trustKeyStore1, null)
.build();
} catch (UnrecoverableKeyException e) {
e.printStackTrace();
}
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext,
new String[]{"TLSv1.2", "TLSv1.1"},
null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory(sslConnectionSocketFactory)
.build();
logger.info("[GOLD] MUTUAL AUTH SSL ONLY");
String ncwurl = "https://localhost:8443/gold/person";
HttpPost post = new HttpPost(ncwurl);
StringEntity ncwparams = new StringEntity(JsonFiles.ONE_JSON);
setPostAttrs(post,ncwparams);
HttpResponse response = client.execute(post); // object reused intentionally
printResponse(response);
logger.info("[BRONZE] FORBIDDEN BY ANTMATCHER");
post = new HttpPost("https://localhost:8443/bronze/person");
setPostAttrs(post, ncwparams);
response = client.execute(post);
printResponse(response);
logger.info("[SILVER] ALLOW ALL BY ANTMATCHER");
post = new HttpPost("https://localhost:8443/silver/person");
setPostAttrs(post, ncwparams);
response = client.execute(post);
printResponse(response);
} catch (NoSuchAlgorithmException | KeyStoreException | KeyManagementException | IOException e) {
throw new RuntimeException(e);
} catch (CertificateException e) {
e.printStackTrace();
logger.error("Error caught", e);
}
}
private void setPostAttrs(HttpPost post, StringEntity ncwparams) {
post.setHeader("Accept", "application/json");
post.setHeader("Content-type", "application/json");
post.setEntity(ncwparams);
}
private void printResponse(HttpResponse response) throws IOException {
logger.info("Response Code: " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while (true) {
try {
if (!((line = rd.readLine()) != null)) break;
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(line);
}
}
}
To test the service, run the Server Application in Eclipse or IntelliJ. Then start the Client service and inspect. The client will return data for the gold() antMatcher using mutual authentication.
The results should look similar to this:
The packet trace in Wireshark will look similar to this:
The next article will extend this program to add API token for selected endpoints while keeping mutual SSL for others. Stay tuned…