Profit.co scales search and log management with Elasticsearch to deliver exceptional user experiences

Profit.co builds OKR management software that helps leaders execute and measure progress. With brands from startups to Fortune 500, Profit.co’s platform provides a singular view to manage business goals, resources, and performance so its customers can achieve desired results.

How did Profit.co help increase productivity for its customers and deliver features to drive engagement? The company needed a full-text search solution that could scale. This is why Profit.co chose Elasticsearch, the powerful engine behind Elastic products and solutions. Profit.co was able to dramatically improve its search experience. After the initial success with search, the Profit.co team also moved their centralized log management to Elasticsearch.

Today, Profit.co uses Elasticsearch to create positive search experiences and streamline log management, continually optimizing performance to meet end user needs. Learn more on how Elastic solved Profit.co’s challenges.

In the beginning, Profit.co used a variety of solutions. According to Kumar Devarakonda, vice president and head of engineering at Profit.co, "We started with MySQL, and then later we moved to Postgres, because it had some additional features compared to MySQL."

But Postgres struggled to scale. Profit.co's users need to search across multiple fields, get results faster, and sort the results based on relevance. To help its users, Profit.co realized it needed a full-text search solution. Devarakonda explains, "We used to have Postgres combined on every table where we wanted to query. When we are storing the object, we used to capture all the fields from our data from all the fields, and then store it in that single column. But it was very slow. It also didn't give accurate results, like scoring relevance."

As the Profit.co team searched, they discovered most solutions were very slow and lacked the accuracy Profit.co customers required. While exploring solutions, they came across Elasticsearch and in 2014, they switched.

"Elastic has helped us in terms of the application's scalability," according to Devarakonda. It was a slow transition to Elastic. He continues, "We started with full text search, and we used to maintain dual storage in Postgres and Elasticsearch. We stored the record in Postgres and used to push the data to Elasticsearch as well."

Then Devarakonda realized, "Why do we need to maintain dual storage? Why can't we go directly into Elasticsearch?" By storing everything in Elasticsearch, Profit.co was able to gain speed and streamline the experience for not only end users, but also internal teams.

Need for speed

Speed was the primary reason for switching. But Devarakonda states, "We started looking for a solution that could provide scoring relevance." He continues, "Without Elastic, we obviously would have continued to have frustrated customers always searching for the information. And they wouldn't get accurate information."

The slowness with Postgres had impacted all the resources down the line. Users were waiting. Engineering teams also experienced delays when trying to implement changes. But with Elastic, speeds improved. A search query would take more than 10 seconds with a traditional database. When teams used Elastic, complex queries (search on multiple fields) had a returned response below 100 milliseconds.

Inability to scale

Along with improved search experiences, the move to Elastic also helped Profit.co scale. According to Devarakonda, "We struggled with the limit on the number of connections with Postgres. The horizontal scalability of Elastic allows any number of clients to connect, based on infrastructure resource availability, rather than a predetermined number of connections."

Now, Profit.co can have multiple clusters without worrying about separating the users into clusters. Elastic also provides support for the nested fields. Profit.co can store any data in the document, even a large document. And for Devarakonda's teams, "We don't have to analyze the data prior to storing." This saves precious development time so teams can focus on more high-value tasks.

Why Elastic

Besides speed and scale, why did Profit.co choose Elastic? Devarakonda needed certain functionality that could help his teams excel. "Querying on the nested fields is also one of the reasons we chose Elastic. Plus the support for integrations, is why we ultimately went with Elastic."

For team members, it was a simple switch. "Elastic is easy to learn from the developer perspective. Onboarding is very quick," Devarakonda explains. Plus, Elastic is secure enough to not only meet but exceed expectations. Elastic gives Devarakonda strict controls on coding, unlike with regular SQL. He continues, "We don't need to write any custom queries. Everything is available as an API, which creates fewer errors during coding." Fewer errors means better features and a more positive user experience.

Centralized log management

Once Profit.co started using Elastic for search, the team realized the additional benefits it could have for their organization. Elastic + Google Cloud helped them streamline their log monitoring capabilities. "Having a centralized logging system is something we were looking for before we moved to the Google Cloud." Devarakonda continues, "With Elastic, we could check easily with an automated script and find any log files that are required for the debugging, etc."

Profit.co CEO and founder Bastin Gerald also saw immense value. “Everything that goes out and comes in is logged into that database in a separate Elastic cluster that we use to check.” Now teams can use Elastic to identify data loss, prevention, and more across approximately 2 million requests per day. Using Elasticsearch also allows Profit.co teams to save approximately 90% of the total cost. The logs ingested from all application nodes and microservices contribute to just 5GB of storage per day.

Gerald explains, "In terms of monitoring the requests and responses that go out from us and come into us, we log all of them in a separate cluster. If a particular client’s performance is bad or the data size is large, these are the sort requests we analyze." Being able to get a bird’s eye view of performance helps to address customers’ concerns before they become a bigger problem.

Future of Profit.co

Profit.co currently supports around 500 customers. As its customer base continues to expand, Elastic will grow with it.

Learn more about Elasticsearch and deploy it on your preferred cloud in minutes. You can try Elasticsearch free for 14-days. Sign up today.