InfoTrack: Powering the search for data-driven innovation with Elasticsearch Service

A simple way to explain InfoTrack is that it is Google for legal and associated information that lawyers and their clients need to get business done.  It amalgamates numerous digital searching, filing and signing features onto one platform with a single UI and workflow so that individuals and businesses can easily search for and retrieve information from thousands of third parties and file documents with government agencies. 

Throughout its history, InfoTrack’s developer-led culture and use of the latest technologies has helped it to differentiate itself in the market by continuing to push out new features all while providing customers with premium service. It has developers worldwide and a dedicated Innovation team that operates as an incubator for new ideas, products, and processes — defining what the company will look like in the next three to five years and beyond.  By leveraging the Elastic Stack and managing their deployments on Elasticsearch Service, InfoTrack’s developers and Innovation team are able to provide customers with blazingly fast search and easily create and deploy new products on the fly.

Ashim Joshi, Head of Innovation, and Nick Carton, Technical Lead, say InfoTrack’s move from on-premise systems to the cloud over the past four years has helped foster this innovation by making it easier to explore and deploy new technologies. “It is very difficult to scale a business when you’re restricted by on-premise systems,” said Carton. “The freedom and flexibility that came with moving to the cloud, and ultimately becoming cloud-native, really launched our developer-led culture and allowed us to scale up very quickly.”

The InfoTrack platform saves customers time and hassle in gathering information and documents they need to carry out a range of legal and property transactions. For example, in the Australian Market, if an individual were to purchase property, they would require documents like a Certificate of Title from the local council or land registry office. Engaging a conveyancer to retrieve these in person can take up to 10 days, but using InfoTrack, the documents can be searched for online in an instant. 

Of course, it’s one thing to be able to get those documents for folks in a way that wasn’t possible before. It’s a whole other thing to continue to pull these documents from disparate data sources and structures and serve them up at near instant speeds, all while the amount of requests continue to grow. That’s why logging and monitoring are now integral to InfoTrack’s every day operations. This Elastic Stack-powered concept allows developers to keep track of how applications are performing and get to the root cause of errors to address them right away. 

Logging was initially managed manually using a file log tool, but this approach became too time consuming as InfoTrack grew and didn’t allow developers to trace and debug applications. So InfoTrack searched for an alternative solution and briefly considered building its own. However, this would have taken months and required a dedicated resource to manage activities like horizontal scaling and the separation of logs between applications. The search for a solution continued until early 2017 when the CEO heard about the Elastic Stack and asked the Innovation team to test it out. 

InfoTrack first implemented Amazon Elasticsearch Service. The Innovation team thought that it would be a quick and easy solution, but they quickly ran into security and performance issues. One of the issues was that the Amazon Elasticsearch Service ran an older version of the Elastic Stack. This version only allowed security using IP restriction and restricted developers from tuning the underlying Java Virtual Machine (JVM). Additionally, in using Amazon Elasticsearch Service, the InfoTrack team lacked access to the additional features of the Stack, including APM, Index Lifecycle Management (ILM), and Graph. Without these, the team could not reach full potential in architecting their system for long-term success and stability, and ensuring they were getting the most value out of the data available to them. 

With great support and documentation from Elastic, the migration was effortless and gave InfoTrack access to the latest version of the Elastic Stack and a wider set of capabilities. This included more robust, role-based security controls and added flexibility to spin up new internal customers and projects. In fact, it’s so simple now that all developers need to do in order to scale up or add a new cluster is click on a slider bar in the Elasticsearch Service Console. 

InfoTrack is now using Filebeat and Metricbeat to ship data from thousands of applications into Logstash where it is transformed into a readable format. The data can then be searched and visualized in real time using Elasticsearch and Kibana. 

Carton said that in addition to improving the speed of search, the Elasticsearch Service provided greater flexibility when it came to hooking into different data streams and managing index life cycle, which it plans to take advantage of in the future by implementing a hot-warm-cold architecture to better balance performance and cost. 

“We’ve now centralised all of our logs and metrics and are monitoring everything in Kibana. This makes it really easy to spot application errors and understand what’s going on so we can minimize any downtime,” said Carton.

Over time, InfoTrack has continued to find more value from the Elastic Stack by using it to monitor other aspects of the business and inform ideas for new products and service improvements. The Innovation team has also more recently used the Elastic Stack to design a new pyramid-shaped ‘product story’ which has revolutionized searching on its platform. At the base of the pyramid is InfoTrack’s data lake and a data mart layer powered by the Elastic Stack; InfoTrack’s API layer and applications sit on top. This allows developers to architect products based on these data marts rather than spinning up and maintaining different databases. 

The Innovation team is now in the process of tagging the metadata of all 55+ million document orders on InfoTrack’s platform to provide a smarter and faster search mechanism across its products. With the help of the data team, they have also set up an ETL pipeline that tags and transforms documents on the platform into a lightweight format for efficient full text searching. 

“Establishing these data marts and enabling more granular key word search with the Elastic Stack has promoted this culture where our developers are asking what else can we do with this data, and how can we extend the power of this search to our customers,” said Joshi. 

InfoTrack’s new Data Lake Search is a result of this innovation. It is a smart search solution which allows customers to reorder, visualise, and discover connections across multiple documents from different third parties. For example, a lawyer running a search on a potential client’s name, can surface information related to that person’s properties, businesses and current or past legal dealings - all in one place. 

This transforms the speed of customer research and allows conflict of interest searches to be completed in one day as opposed to weeks. 

The use of the Elastic Stack and the Elasticsearch Service has delivered wide-ranging benefits, starting with the time saved on managing logs and resolving application errors and leading to a trusted customer experience. "Our customers choose InfoTrack because we offer a premium service. Elastic helps us deliver that by allowing us to spot application issues early on and resolve support tickets a lot faster. Our help desk NPS has improved from 70 to more than 80+ as a result," said Joshi. 

Other metrics like growth and customer retention are also set to rise with the use of Kibana helping to reduce the time spent investigating issues and giving developers more time to focus on pushing out quality code. They’re able to deploy fixes and features daily and release innovative solutions like InfoTrack’s Data Lake Search which can’t be found elsewhere. What’s more, the business has more data than ever before, helping it to  identify cross-selling opportunities and prevent customer churn. 

InfoTrack is now focused on maturing its development process and adding on new capabilities of the Elastic Stack which is as easy to do as clicking and dragging on a slider in the Elastic Cloud console. Its future plans include using APM to trace points of failure in its document delivery pipeline, so whether the failure originates from its own technology stack or a partner’s, it can resolve the issue faster. InfoTrack is also excited about having easy access to SIEM within the Elastic Cloud and is exploring how it can take advantage of this to monitor threats and drive security operations and avoid building in-house solutions to  do the same. 

“Our goal is to release an entire solution every four months, and Elastic gives us the flexibility and efficiency to achieve that,” said Joshi.