Wednesday, November 20, 2013

Building a data entitlements ecosystem with WSO2 IS, ESB, and DSS

Introduction 

Most enterprises have a tendency to consider data security as a function attached to or supported by the business logic layer.  A common practice is to use a key identifier such as user roles or permissions to be part of the data filtering criteria often combined with other business specific filtering.  Traditionally this practice has evolved with the comfort which system designers took in making sure that business components and data components would reside within the same deployment and system administration boundaries.

Things have evolved.  With enterprises adapting more loosely coupled and SOA driven architectures, the need for federated security has become a point of discussion across all application layers, including data.  Commonly known as data entitlements, the idea is to use enterprise-wide security models at the point of producing and storing data in and out of the data providers so that the applications will not have to handle filtering based on entitlements.

So why is this important?  I think the answer can be found if you take a closer look at some typical issues seen on enterprises related to data duplication and consolidation nightmares, overlapping security policies and policy administration overheads.

Let me try to explain the problem through a common use case.

'SomeCo' is a premium advertising company with multiple business units to handle a large client base.  They have a divided sales organization specialized for banking & financial, manufacturing & retail, food and beverages.  The company maintains a pool of applications that needs to comply with data entitlements policies across the enterprise.  The platform needs to make sure that sales information related to a particular domain is only made visible to the members of that particular sales group as well as anyone with an organizational role above a senior manager.  This policy needs to be enforced across all applications that present sales information to the users.




Let’s take a look at a typical ecosystem that WSO2 can provide for SomeCo’s enterprise data entitlements need.  For this solution, I am going to leverage the WSO2 ESB, IS and DSS.   I believe the purpose of using DSS should be quite familiar to most… and that is to expose SomeCo’s sales data as services.   We are going to use WSO2 IS’s attribute based authorization service built on XACML to define entitlements policies for our sample service call.   The idea is, based on some user identifier (i.e. UsernameToken), we will query the WSO2 IS for a set of claims through a XACML request and built our dynamic query based on claims received as part of the response before calling the DSS service.


Solution




- Our aim is to provide a unified interface to the consumer applications to access Sales data so that applications do not need to handle data filtering based on users’s security attributes (such as user roles).

- Assume the scenario where the users of these different sales organizations need to query sales details through different applications (i.e. Sales forecasting, revenue predictions etc).    To achieve this, our solution needs to expose a service API (i.e. let’s say an API called getSalesInfo), which typically is an ESB endpoint which these applications can connect to.

- Each application needing to consume this API will send its request payload with the authentication headers (in this sample a UsernameToken) to the ESB endpoint.

- After the ESB authenticates the request (usually against the same user store which the applications are connected to), the ESB flow needs to acquire a set of claims this user is entitled for.  WSO2 ESB provides an entitlements mediator for this purpose.  The primary role of the entitlements mediator is to create and send a XACML request to WSO2 IS, and branch out the API flow based on the decision (i.e. Permit, Deny, indeterminate, Not Applicable).  In addition to the decision, this mediator can also receive claims (in the form of advices) back from IS which is really what we are interested in to build the dynamic DSS query.

- As an example, we can receive user’s roles as advices inside the XACML response so that we can build the DSS service’s filter criteria based on roles.

- As you might have noticed, the caller applications didn’t have to worry at all about filtering for data entitlements and it was all handled by the service API we have created using the WSO2 components.


ESB Mediation Flow



The heart of the mediation flow is the part where we extract claims out of the XACML response to build our dynamic query.  In many instances, the filter we might need to construct would usually be quite specific to transnational data; hence we may need to translate those claims into a filter query which the DSS service will be able to successfully map to SQL.   Well, we can think of many possible options for doing this, but for simplicity I implemented a simple mediator to build the filter query.  

Please refer [R2] for the Java code I used to read the advices from the Message context, and then create the filter query for the DSS service.

Resources

[R1] Sample XACML Policy with AdviceExpressions for returning claims to the caller (marked in red). 
Note the XACML policy defines that user's role is returned with the decision. The entitlements mediator can grab this and pass this on to the ESB sequence which can construct the dynamic query based on these claims.  In this example, we are using the user's role, but it can be any attribute that is in the user store and retrieved by Policy Information Point (PIP)

<Policy xmlns="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17"  PolicyId="CustomerServiceSales" RuleCombiningAlgId="urn:oasis:names:tc:xacml:1.0:rule-combining-algorithm:first-applicable" Version="1.0">
   <Target></Target>
   <Rule Effect="Permit" RuleId="Rule1">
      <Target>
         <AnyOf>
            <AllOf>
               <Match MatchId="urn:oasis:names:tc:xacml:1.0:function:string-regexp-match">
                  <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">customerproxy</AttributeValue>
                  <AttributeDesignator AttributeId="urn:oasis:names:tc:xacml:1.0:resource:resource-id" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource" DataType="http://www.w3.org/2001/XMLSchema#string" MustBePresent="true"></AttributeDesignator>
               </Match>
               <Match MatchId="urn:oasis:names:tc:xacml:1.0:function:string-equal">
                  <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">read</AttributeValue>
                  <AttributeDesignator AttributeId="urn:oasis:names:tc:xacml:1.0:action:action-id" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:action" DataType="http://www.w3.org/2001/XMLSchema#string" MustBePresent="true"></AttributeDesignator>
               </Match>
               <Match MatchId="urn:oasis:names:tc:xacml:1.0:function:string-equal">
                  <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">sales</AttributeValue>
                  <AttributeDesignator AttributeId="http://wso2.org/claims/role" Category="urn:oasis:names:tc:xacml:1.0:subject-category:access-subject" DataType="http://www.w3.org/2001/XMLSchema#string" MustBePresent="true"></AttributeDesignator>
               </Match>
            </AllOf>
         </AnyOf>
      </Target>
   </Rule>
   <AdviceExpressions>
      <AdviceExpression AdviceId="customerService" AppliesTo="Permit">
         <AttributeAssignmentExpression AttributeId="employee.role">
            <AttributeDesignator AttributeId="http://wso2.org/claims/role" Category="urn:oasis:names:tc:xacml:1.0:subject-category:access-subject" DataType="http://www.w3.org/2001/XMLSchema#string" MustBePresent="true"></AttributeDesignator>
         </AttributeAssignmentExpression>
      </AdviceExpression>
   </AdviceExpressions>
</Policy> 



[R2] Mediator code to build filter query

@Override
public boolean mediate(MessageContext ctx) {

  try {
  // Retrieve advices from the message context
  String rawXML = ctx.getProperty("adviceXml").toString();

   
  OMElement advice = AXIOMUtil.stringToOM(rawXML);

  Iterator<OMElement> advicesIter = advice
.getFirstChildWithName(new QName("", "Result"))
.getFirstChildWithName(new QName("", "AssociatedAdvice"))
.getFirstChildWithName(new QName("", "Advice"))
.getChildren();

   String sqlfilter = " ";

   // Build filter query.  In this sample we will use the role names as they are.
   while (advicesIter.hasNext()) {

 OMElement elem = advicesIter.next();

 String filterColumnName = elem.getAttribute(
 new QName("", "AttributeId")).getAttributeValue();
         sqlfilter += filterColumnName + "='" + elem.getText() + "'";

 if (advicesIter.hasNext()) {

 sqlfilter += " OR ";
 }
}

ctx.setProperty("DSSFilter", sqlfilter);

    } catch (XMLStreamException e) {
    
    }

  return true;

}



[R3[ ESB flow

<proxy name="getSalesInfo"
  transports="https http"
  startOnLoad="true"
  trace="disable">
  
  <description/>
  
  <target inSequence="InSeq" outSequence="OutSeq">
     <endpoint>
     <address uri="http://localhost:9765/services/customerservice"/>
     </endpoint>
   </target>
   </proxy>

   <sequence name="OutSeq">
      <send/>
      <log level="full"/>
   </sequence>
   
<sequence name="InSeq">
      <entitlementService remoteServiceUrl="https://localhost:9444/services/" remoteServiceUserName="admin" remoteServicePassword="enc:kuv2MubUUveMyv6GeHrXr9il59ajJIqUI4eoYHcgGKf/BBFOWn96NTjJQI+wYbWjKW6r79S7L7ZzgYeWx7DlGbff5X3pBN2Gh9yV0BHP1E93QtFqR7uTWi141Tr7V7ZwScwNqJbiNoV+vyLbsqKJE7T3nP8Ih9Y6omygbcLcHzg=">
         <onReject>
            <makefault version="soap12">
               <code xmlns:soap12Env="http://www.w3.org/2003/05/soap-envelope"
                     value="soap12Env:Receiver"/>
               <reason value="UNAUTHORIZED"/>
               <node/>
               <role/>
               <detail>XACML Authorization failed</detail>
            </makefault>
         </onReject>
         <onAccept>
            <send>
               <endpoint>
                  <address uri="http://localhost:9765/services/customerservice"/>
               </endpoint>
            </send>
         </onAccept>
         <obligations/>
         <advice/>
      </entitlementService>

   </sequence>

References

[a] Blog : Guide to write XACML policies in WSO2 Identity Server 2.0
http://blog.facilelogin.com/2009/06/guide-to-write-xacml-policies-in-wso2.html

[b] Managing Entitlement
http://docs.wso2.org/display/IS450/Managing+Entitlement

Friday, November 8, 2013

Integrating HornetQ with WSO2 ESB

Introduction

HornetQ is a JMS compliant open source asynchronous messaging project from JBoss. WSO2 ESB provides a simplified configuration model to integrate any JMS compliant messaging system.  Recently I came across few discussion threads looking for a sample configuration between HorenetQ and WSO2 ESB, hence this is a short article to outline the steps. 

Sample Scenario

In this example, I am going to expose an ESB proxy service which accepts sample SOAP messages and push them to a JMS queue configured in HornetQ.  This ESB proxy will only execute an outward operation and will not return anything back to the caller.

Steps

1.       First, create a sample queue by editing $HORNET_HOME/config/stand-alone/non-clustered/hornetq-jms.xml.

NOTE : There are multiple execution modes supported by HornetQ, but I am only going to use the stand-alone configuration for this example.

<queue name="wso2">
      <entry name="/queue/mySampleQueue"/>
</queue>


 2.       Put the following 2 connection factory entries that will be required if the ESB needs to act as a JMS consumer.  However, this example will only cover the scenario of WSO2 ESB acting as a producer of which the incoming payload will simply be pushed to a queue.

<connection-factory name="QueueConnectionFactory">
      <xa>false</xa>
      <connectors>
         <connector-ref connector-name="netty"/>
      </connectors>
      <entries>
         <entry name="/QueueConnectionFactory"/>
      </entries>
</connection-factory>



<connection-factory name="TopicConnectionFactory">
      <xa>false</xa>
      <connectors>
         <connector-ref connector-name="netty"/>
      </connectors>
      <entries>
         <entry name="/TopicConnectionFactory"/>
      </entries>

</connection-factory>

 3.       Copy HoenetQ client JARs into $ESB_HOME/repository/components/lib.


NOTES :
[a.] There is a current limitation on the ESB that whenever multiple non-OSGi JARs having the same package names are copied, the process to convert them as OSGi bundle may drop these packages on subsequent JARs.  Hence I had to assemble those multiple client side JARs into a single JAR.  There are multiple ways to do that (i.e Maven JAR assembly plugin, etc.), and for the convenience I am attaching the assembled JAR for HornetQ 2.3.0.  You just have to copy this hornet-all.jar into $ESB_HOME/repository/components/lib if you decide to use this already assembled JAR.

Download the jar from here


[b.] If you opt to pack the JARs yourself, please make sure the remove the javax.jms package from this assembled JAR to avoid the carbon runtime from picking this implementation of JMS over the bundled-in distribution.

 4.       Uncomment the following line to enable JMS transportSender on axis2 core @ $ESB_HOME/repository/conf/axis2/axis2.xml

<transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender"/>


 5.       Let’s configure a simple proxy service as below which accept a simple SOAP message and push the SOAP envelope into the mySampleQueue. 

<proxy xmlns="http://ws.apache.org/ns/synapse" name="toJMSProxy" transports="https,http" statistics="disable" trace="disable" startOnLoad="true">
   <target>
      <inSequence>
         <property name="Accept-Encoding" scope="transport" action="remove"/>
         <property name="Content-Length" scope="transport" action="remove"/>
         <property name="Content-Type" scope="transport" action="remove"/>
         <property name="User-Agent" scope="transport" action="remove"/>
         <property name="OUT_ONLY" value="true"/>
         <send>
            <endpoint>
               <address uri="jms:/queue/mySampleQueue?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory&java.naming.provider.url=jnp://localhost:1099&transport.jms.DestinationType=queue"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <publishWSDL uri="file:repository/samples/resources/proxy/sample_proxy_1.wsdl"/>
   <description>HornetQ-WSO2 ESB sample</description>
</proxy>


NOTES :
[a] We are creating a proxy for the SimpleStockQuoteService shipped with the ESB samples.  You don’t really need to have the actual service running for this example, as our plan is to push the incoming payload into the JMS queue (and not calling the actual back-end service). 

[b] Extra HTTP headers are removed to present only the XML content for validation. 

[c] This is only a one way operation pushing the payload into the queue, hence OUT_ONLY is set to true. 

[d] You may need to change the hostname, port etc. on the JMS endpoint string matching to your environment. 

6.       Start the WSO2 ESB server and use a client application of your choice to call the Proxy service endpoint.  Here I am using soapUI to create a request calling the placeOrder operation of the proxied SimpleStockQuoteService.




7.       Multiple ways to check if the payload was successfully pushed to the JMS queue in HornetQ, but I used a simple Java client to pull the message out from mySampleQueue.

public class HornetQJMSTestClient {

       public static void main(String[] args) throws Throwable {

              // Step 1. Create an initial context to perform the JNDI lookup.              Hashtable<String, String> env = new Hashtable<String, String>();
              env.put(Context.PROVIDER_URL, "jnp://localhost:1099");
              env.put(Context.INITIAL_CONTEXT_FACTORY,
                           "org.jnp.interfaces.NamingContextFactory");
              env.put(Context.URL_PKG_PREFIXES,
                           "org.jboss.naming:org.jnp.interfaces  ");
              Context ctx = new InitialContext(env);

              // Step 2. Lookup the connection factory              ConnectionFactory cf = (ConnectionFactory) ctx
                           .lookup("/ConnectionFactory");

              // Step 3. Lookup the JMS queue              Queue queue = (Queue) ctx.lookup("/queue/mySampleQueue ");

              // Step 4. Create the session
              Connection connection = cf.createConnection();
              Session session = connection.createSession(false,
                           Session.AUTO_ACKNOWLEDGE);

              // Step 5. Create a JMS Message Consumer to receive message
              MessageConsumer messageConsumer = session.createConsumer(queue);

              // Step 6. Start the Connection so that the server
              connection.start();

              // Step 7. Receive the message              TextMessage messageReceived = (TextMessage) messageConsumer
                           .receive(5000);
              System.out.println("Received message: " + messageReceived.getText());

              // clean up all the JMS resources              connection.close();

       }
 
} 

NOTE :If you need to configure the WSO2 ESB as a JMS consumer, you will need to enable the transportReceiver block with HornetQ configuration parameters as follows on the axis2.xml file.


<transportReceiver name="jms"
class="org.apache.axis2.transport.jms.JMSListener">
<parameter name="myTopicConnectionFactory" locked="false">
<parameter name="java.naming.factory.initial" locked="false">org.jnp.interfaces.NamingContextFactory</parameter>
<parameter name="java.naming.factory.url.pkgs" locked="false">org.jboss.naming:org.jnp.interfaces</parameter>
<parameter name="java.naming.provider.url" locked="false">jnp://localhost:1099</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName"
locked="false">TopicConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType"
locked="false">topic</parameter>
</parameter>
<parameter name="myQueueConnectionFactory" locked="false">
<parameter name="java.naming.factory.initial" locked="false">org.jnp.interfaces.NamingContextFactory</parameter>
<parameter name="java.naming.factory.url.pkgs" locked="false">org.jboss.naming:org.jnp.interfaces</parameter>
<parameter name="java.naming.provider.url" locked="false">jnp://localhost:1099</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName"
locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType"
locked="false">queue</parameter>
</parameter>
<parameter name="default" locked="false">
<parameter name="java.naming.factory.initial" locked="false">org.jnp.interfaces.NamingContextFactory</parameter>
<parameter name="java.naming.factory.url.pkgs" locked="false">org.jboss.naming:org.jnp.interfaces</parameter>
<parameter name="java.naming.provider.url" locked="false">jnp://localhost:1099</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName"
locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType"
locked="false">queue</parameter>
</parameter>

</transportReceiver>