The focus of this blog post is on Entity Aggregation (EA). It looks at the problem context and situations that require integration of information present in different systems and covers WSO2 technologies – WSO2 Enterprise Service Bus and WSO2 Data Services - that may be used to build an EA solution.
The Integration Problem
Businesses continue to grow and as they expand over the years the demand for IT systems also increase. Mergers and acquisitions became the standard for growing businesses and the demands for new IT systems skyrocketed. As a result, today's enterprises are composed of multiple IT systems that are independent of each other. The result is hundreds, if not thousands, of application islands. For the business to sail smoothly, all the data in these isolated applications must be easily accessible to each other. This usually creates an integration nightmare.
What are These Particular Integration Problems?
- Organizations find themselves struggling with significant problems of data integration among a wide variety of data sources.
- The mismatch between the data representation that one application provides and the data representation that another application expects forms the core of this long-lived problem.
- This data integration problem is particularly troublesome for SOA implementations because it limits the loose coupling between service providers and consumers.
Integration Needs Can Fall into These Categories:
- Standard Integration – This is the straightforward need for integrating applications and systems for data sharing.
- Entity Aggregation – This is enforcing consistency between systems sharing same entity information by maintaining a holistic view that is concerned with the whole instead of just the parts.
- Standards Compliance - Adherence to standards insures the loose coupling and composition of services and data exchange mechanisms with a guarantee that the data exchanged with an individual service will be universally comprehensible e.g. HL7 in healthcare information systems.
What is Entity Aggregation?
There are essentially three reasons for the aggregation of entities in different systems:
- Single View of Entity—having a single view of all common entities spread across enterprise systems. E.g. a unified view of a Customer entity, named Customer Information Repository (CIR), is required to build useful solutions in many enterprises, especially in the banking sector.
- Horizontal Partitions—there are situations where entity information is distributed across many different services, based on factors such as geographical boundaries.
- Cross-Join scenarios—these situations require cross-join among entities present in different services.
Goal of an Aggregation Service
We can use an aggregation service to embed the transformation logic used to connect the disparate systems and expose a unified view of the entity data. This transformation service, loosely-coupled in a SOA implementation, accepts incoming data in one format, and outputs data with the same semantic meaning in another format, which will be the holistic view that is required from an entity aggregation service. The use of a data transformation service or an aggregation service in this case, improves loose coupling by separating the concerns of service providers and consumers.
Some goals of having an aggregation service:
- Acts as a unified source of entities.
- Provides a holistic view of an entity.
- Provides a holistic view of the entity model—entities and their relationships with other entities
- Enriches the combined data model across systems—the whole being better than the sum of its parts.
- (Kothandraman, 2004)
Reference Architecture: Entity Aggregation
In this particular organization, information of the Customer entity resides in two systems – in their Customer Relationship Management and Enterprise Resource Planning systems. As shown in the diagram, the systems share the same entity (Customer) while the data models are slightly different. Some fields are duplicated while some others are additional information that one system lacks. An Entity Aggregation solution can be designed to not only act as a single point in accessing information that exists in multiple systems, but it also provides a holistic view of an entity and the entity model. The entity aggregation service communicates with the CRM and ERP systems and creates a whole new data model using some transformation logic (in this case it uses XSLT transformation). This holistic view can be exposed as an API to parties interested in retrieving all the information related to a customer in this organization.
WSO2 Solution Architecture: Entity Aggregation
A solution to act as the data aggregation service can be implemented with the WSO2 Enterprise Service Bus or WSO2 Data Services Server or a combination of both depending on the systems that need to be integrated.
If both the CRM and ERP need to be integrated at the application level, the WSO2 ESB can integrate the systems and expose a unified view as a REST API or a proxy service which is a SOAP service. The WSO2 Enterprise Service Bus supports rich transformations using the transformation mediators belonging to its comprehensive mediator library such as XSLT, FastXSLT, XQuery, Enrich and Payloadfactory mediators.
If both the CRM and ERP need to be integrated at the data level, where the information needs to be extracted from two databases and then transformed to a RESTful or SOAP service, the WSO2 Data Services Server can be leveraged. While the Data Services Server can expose data from various data sources – e.g. RDBMS data sources, Excel spreadsheets, CSV, Google spreadsheets etc. – it also supports transformation using XSLT transformation, where the user can define the transformation in XSLT.
If one system needs to be integrated at the application level and another needs to be integrated at the data level, which is the case in the above diagram, the ESB alone or a combination of the ESB and Data Services Server products can be used.
Part II
The implementation details of the above scenario using the WSO2 ESB and WSO2 Data Services Server will be covered in the 2nd part of this blog post.
References
Kothandraman, R. 2004. SOA Challenges: Entity Aggregation, http://msdn.microsoft.com/en-us/library/ms954596.aspx
No comments:
Post a Comment