In today’s digital age, effective collaboration is increasingly achieved via an online workplace which provides a holistic set of workplace tools, platforms and environments, delivered in a coherent, usable and productive way.
Online workplaces help to empower individuals, teams, colleagues, clients and partners to share, communicate and collaborate with little to no friction, delay or challenge; allowing all these stakeholders to perform their jobs more effectively.
One of our clients recently expressed their need for an online workspace to keep track of business process documentations, data sheets, process workflows, media files etc. As systems integration experts, we ran this project knowing that we could effectively build the workspace using Apache Solr and WSO2 ESB.
Because we love to share knowledge, this ‘How-To-Guide’ outlines how Apache Solr can be successfully integrated for search within applications, using WSO2 ESB.
Apache Solr is an open source search platform built upon a Java library called Lucene. It is a popular search platform for web sites because it can index and search multiple sites and return recommendations for related content based on the search query’s taxonomy. Apache Solr is also a popular search platform for enterprise search because of its ability to index, search documents, and email attachments.
It has a range of versatile features which include:
● Advanced full-text search capabilities
● Standards based open interfaces – XML, JSON and HTTP
● Comprehensive administration interfaces
● Easy monitoring
● Highly scalable and fault tolerant
● Flexible and adaptable with easy configuration
● Near real-time indexing
● Extensible plugin architecture
● Supports indexing and searching multiple sites
● Indexing attachments (Email, PDF)
● It is also optimised for high volume traffic.
Companies like Apple, AOL, Digg, Cnet, Disney and Netflix use Apache Solr for their search feature.
WSO2 Enterprise Service Bus (WSO2 ESB) is a lightweight, high performance, comprehensive ESB that is 100% open source. WSO2 ESB effectively addresses integration standards and supports all integration patterns, enabling interoperability among various heterogeneous systems and business applications.
Its intuitive visually supported tools offer users a smooth experience in development environments and production deployments. Designed for customisation, WSO2 ESB incorporates ESB specific analytics dashboards to gain better insights into system performance.
Equipped with drill-down capabilities, it also allows users to instantly understand and trace mediation flows for better analysis.
WSO2 ESB provides greater flexibility and agility to meet growing enterprise demands.
Integrating WSO2 ESB with Apache Solr Server
We took our first project step by building and implementing an Atlassian Confluence-like software application to hold documentation for the client’s business processes. We built it using the Apache Solr Server to enable content publishing, the creation of indexes, and full text searches.
The application we created allows a logged user to go to a particular library within the system, look for relevant business process documents and read or print them. When a user types a word or phrase into the system to access a particular document, the search results appear in a list – similar to Google’s search results – with highlighted texts.
We also added some additional functionalities. These included:
● Frequent database syncing
● Tenant wise content filtering
● Role based content filtering.
With regards to publishing dynamic properties and indexing in the Solr server, we faced some challenges. This is because we initially decided to use an NOSQL Graph Database (Neo4J) to store documents in the system.
Using an NOSQL database can lead to problems because, unlike the RDBMS table, different properties can exist in the database for the same entity type, thus, the Indexer needs to be able to Index unique dynamic properties based on both Tenant and Roles.
It was time to introduce WSO2 ESB to the project!
Content Search using WSO2 ESB
Once the indexer was created to publish, we knew that WSO2 ESB middleware would be the best content search solution, as the search requests coming from the web can be intercepted by the ESB and modified according to user privileges. They are then submitted to the Solr server through the Apache Solr REST API, and structured in such a way that they populate back into the web in a similar fashion to Google search results.
The image below provides an overview of the solution we developed: