WSO2 Governance Registry 5.x.x FAQs

The WSO2 Governance Registry (G-Reg) has gone through some major transformations, starting from G-Reg 5.0.0 (the current version as of this writing is 5.1.0). In addition to the new and enhanced registry and repository features, G-Reg now comes with multiple views for different roles, i.e. publishers, consumers/subscribers and administrators. This is a significant change from the previous versions which just included one view for all the users. Before understanding the nuts and bolts of G-Reg let's first understand what a registry is and its purpose. What does it do? Why use one?

If your business is SOA-enabled, you need to keep track of your services and who consumes them. Furthermore, as businesses undergo change, including mergers and acquisitions, the number of platforms, consumers, services, and exposed APIs can increase rapidly. SOA Governance is needed to provide full visibility into existing assets; without it, businesses lack the tools to govern and manage assets consistently.It's all about ensuring and validating that assets and artifacts within the architecture are acting as expected and maintaining a certain level of quality. This is where the registry comes into the picture to facilitate SOA governance. The registry can act as a central database that includes artifacts for all services planned for development, in use and retired. Essentially, it's a catalog of services which are searchable by service consumers and providers. The WSO2 Governance Registry is more than just a SOA registry, because in addition to providing end-to-end SOA governance, it can also store and manage any kind of enterprise asset including but not limited to: services, APIs, policies, projects, applications, people. 

Now that we understand what a registry is, here is a compilation of some FAQs and answers related to WSO2 G-Reg to understand what it offers and how it behaves. 

What is the WSO2 Governance Registry ?

The WSO2 Governance Registry (G-Reg) is a SOA-integrated registry-repository for storing and managing data or metadata related to service artifacts and other artifacts. It provides a rich set of features including SOA governance, lifecycle management, and a strong framework for governing anything. For more information on the features and functionality of WSO2 Governance Registry, go to WSO2 Governance Registry.

WSO2 Governance Registry’s main functionality falls under the following two categories.

Content repository

Governance framework

WSO2 Governance Registry provides three main web based user interfaces to facilitate the features and functionality as follows. 

G-Reg Publisher - an end-user, collaborative web interface  for governance artifacts providers to publish artifacts, manage them, show their dependencies, and gather feedback on quality and usage of them. 

G-Reg Store - an end-user, collaborative Web interface for consumers to self-register, discover governance artifact functionality, subscribe to artifacts, evaluate them and interact with artifact publishers.

G-Reg Management Console - a Web interface for administrators to perform admin tasks. 

How does a service consumer use the solution to find a service and implement a service client?

The service consumers can use the G-Reg Store to self-register, discover and search for SOAP/REST services. G-Reg offers configuration options such as tags, categories, comments, properties, ratings and descriptions for a resource. It is important to plan the use of these configurations, to facilitate discovering services and enabling correct SOA Governance. Resources for service discovering tremendously help in service reuse. In fact, it's one of the major functions of a registry-repository product. G-Reg provides enhanced search capabilities to facilitate search based on tags and other advanced criteria.

How does a service provider register a new service?

G-Reg allows service providers to register services through the G-Reg Publisher. Users can choose either to enter service details manually  or to import service information using a WSDL/WADL url. The G-Reg Publisher facilitates artifact providers/creators to publish artifacts, manage them, show their dependencies, and gather feedback on quality and usage of them.

How does a service provider use the solution when making a change to a service specification or endpoint ?

A service provider can use the G-Reg publisher to perform changes such as editing or versioning an existing service.  G-Reg provides tools for asset comparison, dependency management and visualizing service descriptions. It also supports WS-Eventing-based subscriptions and notifications that can be used to govern changes made to individual resources as well as to the lifecycle to which it belongs.

How does the solution facilitate service governance?

Service reuse is the heart of SOA. Before implementing a new service, a service provider can search in the registry for existing implementations. This helps the provider to use an existing service either as it is or by developing a new service associating the existing service. Furthermore, registry-repositories help discover associations among services. This helps to get a better idea of any impacts when changing a particular service. And services in a registry undergoes lifecycle states of create, test, deploy and deprecate.

G-Reg can perform the following functions:

• Enforce policies during transitions of the states of create, test, deploy and deprecate.
• Define "who can access what?" of services. Access to certain services may differ depending on the user, user group or state of the service lifecycle.
• Send notifications to relevant users once a change to a service artifact has been made.

As more and more services are introduced and reused, it is necessary to keep track of dependencies of each service in an organization. G-Reg makes life easier by keeping inter-service dependency information as relationships among service information artifacts. For example, such relationships can be Contains, Implements, Uses, Depends, etc.

Service artifacts evolve over time due to reasons such as fulfilling new requirements and yielding to different versions of the same service. G-Reg provides versioning capabilities that can enable automatic version control of artifacts stored. Additionally. G-Reg keeps older versions of artifacts to allow users to migrate smoothly from one version to another.

In summary, G-Reg provides the following capabilities to facilitate SOA Governance:

• Record information on services
• Add service/API information manually or import WSDLs/WADLs
• Discover services using scheduled tasks and discovery agents
• Search for an existing service for reuse
• Search using tags/categories. Supported via SOLR.
• Discover associations and dependencies of a service
• Service lifecycle management
• Lifecycle-based asset management
• In-built and custom lifecycle executors
• User access control
• Automatic version control
• Notification support (email, UI etc.)
• An SDK for registry-repository extensibility

How is the solution used at design time vs. runtime?

G-Reg can be used during design-time to record service information and govern the service lifecycle .
If needed, using lifecycle executors, services can be deployed/undeployed in relevant servers based on the lifecycle state transition. For example, a Jenkins job can be triggered during a state transition by a custom lifecycle executor - the executor will invoke a remote API of Jenkins which will, for example, build and and deploy service(s) in production environment when promoted from testing to production.

Run-time policy enforcement can be done when associating a WS-Policy with a SOAP service. G-Reg can apply these policies using Handlers (Handlers provide the basis for extending the WSO2 Governance Registry functionality). This is an extension feature as G-Reg only creates an association out-of-the-box.

How does the solution manage multiple endpoints for a service (e.g. Dev, QA, Production)?

Endpoints can be added manually to a service via G-Reg Publisher.  When importing WSDLs, only one endpoint will be added; however, more endpoints (QA, Prod) can be added manually.

How does the solution manage multiple versions of a service?

G-Reg provides support to version existing services, view all versions of a service and restore to a previous version. It is possible to compare different versions of governance artifacts via the Publisher, provided that the comparison is between versions of the same artifact type. If required, resources can be automatically versioned when they are added or updated. But this feature is disabled by default.

How does the solution manage service deprecation and discontinuation?

Currently, G-Reg only changes the state of the service to Deprecated in the default service lifecycle and can be configured to notify subscribers of that service  However, if any actions need to take place as a result of the lifecycle state being changed to Deprecated, a lifecycle executor can be configured to implement such tasks.

Resources

https://docs.wso2.com/display/Governance510/About+Governance+Registry
http://wso2.com/library/webinars/2015/11/wso2-product-release-webinar-wso2-governance-registry-5.1/
https://www.youtube.com/watch?v=e_qESIQTRzI
http://wso2.com/library/articles/2015/07/wso2-governance-registry-governance-framework-extension-points/

WSO2 API Manager vs WSO2 ESB: Should the API Manager replace the ESB?

In the `good old days’ the aim of the Enterprise Service Bus (ESB) was to integrate applications inside the organisation. Many years later, the need for integration still continues to grow, and the integration scenarios tend to exponentially diversify. These days it’s absolutely critical to be able to connect with partners and customers outside the organisation in much more flexible and agile ways. This is where API Managers (or API Gateways as offered by some vendors) are predominantly used. Does this mean that the ESB is old-fashioned and should be replaced by an API Manager? 

There used to be only one way to integrate applications: exchanging files. Then application transaction calls were added followed by asynchronous exchanges using queuing. Now, mobile, Internet of Things, and machine-to-machine (M2M) technologies introduce multiple additional scenarios using APIs and streaming integration styles. Nevertheless, even with these new scenarios, older technologies remain in use. So, when all these integration scenarios are combined, it becomes hard for developers to decide which should be used over the other or when to use both with respect to Integration Solutions (ESB) and API Managers.

So, back to the question: Should ESBs be replaced by API Managers? In most cases, it will not be an either/or situation; ESBs and API Gateway Services can coexist and complement each other. However, depending on the integration requirement and cost implications (using both technologies is expensive), one might have to be chosen over the other. Let’s take a look at some examples of real-world integration requirements and see how they should be addressed by using either the ESB or the API Manager or both. 

Scenario 1 - A loosely coupled integration solution

In an organization, there are numerous heterogeneous systems (as a result of a merger between several companies), from database applications to legacy ERP systems to cloud applications like Salesforce. There are also some SOAP web services and some recently developed REST APIs. The need is to seamlessly connect them through a centralized integration layer so that tight coupling between the applications can be avoided.  The solution should be on-premise and provide application connectors, message queueing, transformation, service orchestration and monitoring. Ability to add security and other policies to enable security, throttling etc. are useful features. What can this organization use?

Solution: ESB 

An ESB can be defined as:

- An architectural construct that provides fundamental services to complex architectures

- An entity that acts as a hub connecting many diverse systems

- A central driver that facilitates Enterprise Application Integration (EAI)

Some of these basic services of an ESB:

- Message passing, routing and filtering

- Message transformation

- Protocol conversion

- QoS enforcement (security, reliable delivery etc)

- Logging, auditing and monitoring

The WSO2 ESB can be used to integrate a large number of heterogeneous systems in an enterprise setting. It is currently used in hundreds of production deployments all around the world to connect various applications, implemented using various technologies (both open source and proprietary) running on various platforms (Windows, Linux, .NET, J2EE, LAMP, cloud). In other words, it is used as a centralized driver that facilitates EAI. The configuration model of the WSO2 ESB is so agile and powerful that practically any EAI pattern can be implemented on top of them. It has a very flexible enterprise messaging model. This is achieved through its ability to support for any wire level protocol or any message format that can be easily implemented on top of this model and can be deployed as a separate pluggable module. 

. Figure 1 - WSO2 ESB as a centralized integration layer

Figure 1 shows how the ESB can act as a centralized integration layer which contains all integration logic to connect to heterogeneous applications and systems. It can be integrated with other WSO2 middleware products such as the WSO2 Identity Server to implement security scenarios; it can publish events to the WSO2 Data Analytics Server (DAS) to perform batch, real-time, predictive and interactive analytics. 

Scenario 2 - A centrally-owned but locally-managed integration solution

An IT services provider has a closed group of clients, who have identical business interests. The clients want to be able to make use of some common integration services as well as a few others that are specific to each client to be able to communicate over domain specific standards and protocols. In order to perform information retrieval and write their own applications, they want these services to be provided by the said IT services provider. For example, a national data registry or database needs to be accessed over a custom protocol by different municipalities in a country to write their apps. Both the clients and the service provider prefer a cloud service as opposed to an on-premise solution at each site due to ease of management and to have a single point of control - for example if there are global changes or updates, they can be easily applied at a single place. (Check  [5] for a real case study on a connected government system). What can the IT services provider use?

Solution: A multi-tenanted ESB in private/public cloud

WSO2 middleware products inherently support multi-tenancy. In this example, each client’s integration needs can be configured in separate tenants in the ESB. The service provider will have to create an adapter(s) for the ESB to integrate with the data registries and create integration services for each client. 

Scenario 3 - Basic API management

An organization already has ready-to-use REST APIs and SOAP services; they don’t want to change anything (no transformations required); they want to become API providers. Some APIs will be only for internal use to enable business agility and customer engagement. Some others will be B2B APIs to optimize value chain relationships and processes. And the rest will be public APIs for access by thousands of developers across the open Web.  It is important to make it easy for API users — developers, whether internal or external to the API provider’s organization, who write applications that leverage an API — to access and understand how to use the API; to be able to secure and throttle the access of and API, and know who is using an API, typically by having them register for an API key; and ensure that API users have the support necessary to solve any problems they may have, whether that support comes from the API provider or from other API users (e.g., through community discussion forums). What does this organization need?

Solution: API Manager

,

An API manager provides developer portal frameworks with pre-built capabilities for the above mentioned requirements and more. It can enforce agreements on API usage and security. An API key is often only the first element of an API provider’s control over API use. API management products enforce usage parameters as agreed to between an API provider and API users in a variety of ways: the use of a secure sockets layer (SSL) or digital signatures for added security; the use of OAuth to allow the API provider’s customers to authorize access to their data; or quotas and rate limits for how many API calls an API user can make. 

Figure 2 - WSO2 API management platform

The WSO2 API Manager is a complete solution for designing and publishing APIs, creating and managing a developer community, and for scalably routing API traffic. It leverages proven, production-ready integration, security, and governance components from the WSO2 Enterprise Service Bus, WSO2 Identity Server, and WSO2 Governance Registry. In addition, it leverages the WSO2 Data Analytics Server for Big Data analytics, giving you instant insight into APIs behavior.

Scenario 4 - API management with simple mediation

An organization has several REST and SOAP APIs already and needs the same capabilities as mentioned in the 3rd scenario. However, the APIs are not ready-to-use and there are ‘minor’ transformations required. For example, a mobile application can only send and receive JSON messages but the backend APIs/services process only XML. Should the organization use the ESB instead of the API Manager or should it use both?

Solution: API Manager

Apart from routing, enforcing security and throttling, the WSO2 API Manager's gateway includes in-built minor payload and protocol transformation capabilities. In fact, the API Manager's Gateway is a basic stripped-down version of the WSO2 ESB and is not intended to be used as a fully-fledged ESB. For minor transformations, dependance on the ESB can be reduced or avoided altogether. For this scenario the API Manager would suffice. 

Scenario 5 - API management with complex integrations

There is a mix of APIs, services and applications (including home-grown applications). In addition to the ready-made APIs and services, a service/API layer must be introduced to access the applications. Some applications communicate over proprietary protocols while some others support standard protocols. Service chaining/orchestration may also need to take place. All services must be exposed centrally to external and internal parties. Management and monitoring of all these APIs are absolutely necessary. Obviously this is far more complex than the previous scenario; so should both the API Manager and ESB be used in this case?

Solution: API Facade Pattern (The combination of the API Manager and the ESB)

API facade is an interface between API consumers (application developers) and the backend systems of the organization. API facade would hide the complexity of the backend services from the application developers and provide a unified layer that developers can use for their applications. 

Because of the heterogeneous nature and the complexity of the backend services, they cannot be exposed directly as APIs to application developers. These backend services require a certain level of mediation; this highlights the need to have a separate mediation layer behind the facade layer. Coupling these two layers increase the complexity in the facade layer, create architectural and scalability limitations, and would require a significant level of mediation logic to be outside the firewall if the APIs are to be exposed to external developers. The solution to this problem is the use of the API Facade pattern.

Figure 3 - API facade pattern 

Implementation of the API facade pattern can be done by using the WSO2 API Manager to build a facade layer and the WSO2 ESB to build the mediation layer (add in products like the WSO2 Data Analytics Server for analytics) and connect to the existing services/applications. 

Other Frequently Implemented Patterns with the API Manager

WSO2’s open source platform provides many opportunities to extend API management. Aside from being the only fully open source solution, the unique feature for WSO2 is the ability to extend API management features . 

Scenario 6 - External key manager

An organization already has an OAuth2 authorization server which issues and validates OAuth2 tokens and want to use that instead of the in-built Key Manager of the WSO2 API Manager. Can this be done?

Absolutely! With the revamped architecture of the API Manager (having started from version 1.9.0), all integration points with the key manager are made to be extensible, thereby enabling the plugging of an external OAuth2 authorization server. See [7] for more information. 

Scenario 7 - External API gateway

What if an organization already has an existing API gateway or ESB component (non-WSO2), which handles token generation and validation on its own or in combination with a Key Manager, and what they are looking for are API store and publisher portals, and their API management capabilities (such as versioning, lifecycle management, notifications etc.). Can they still use the WSO2 API Manager?

This pattern can be implemented using WSO2 Governance Registry 5.0.0 [8].  The Governance Registry can provide store and publisher interfaces. It can be used to write executors to push API definitions to the external gateway (in the required format) once an API’s lifecycle state changes from ‘created’ to ‘published’ state in the publisher (and removed/disabled when the state changes to ‘deleted’). 

Resources - 

[1] - https://www.forrester.com/TechRadar+Integration+Technologies+Q2+2015/fulltext/-/E-RES116227

[2] - http://techfeast-hiranya.blogspot.com/2012/09/busting-synapse-and-wso2-esb-myths.html

[3] - http://wso2.com/whitepapers/the-evolution-of-Integration-a-comprehensive-platform-for-a-connected-business/

[4] - http://wso2.com/blogs/architecture/2013/05/a-pragmatic-approach-to-the-api-faade-pattern/

[5] - http://wso2.com/whitepapers/connected-government-cloud-enabling-public-services/

[6] - http://wso2.com/library/blog-post/2014/10/blog-post-revamping-wso2-api-manager-key-management-architecture-around-open-standards/

[7] - https://docs.wso2.com/display/AM191/Configuring+a+Third-Party+Key+Manager

[8] - http://wso2.com/products/governance-registry/