BAI Communications: Powering the Search for Real-Time Business Analytics and Data-Driven Sales & Marketing Campaigns

In New York and Toronto, BAI Communications, in conjunction with their sister company Transit Wireless, offers free connectivity to commuters and uses its infrastructure to sell advertising services to businesses. This includes online advertisements shown to users at the point of connection, as well as physical, digital signage in the actual subway stations. With the potential to reach tens of thousands of commuters hourly, it's valuable advertising space. The challenge for BAI Communications, though, was that it lacked the real-time metrics to prove this value. The sales team had static spreadsheets reporting on Wi-Fi use, but they lacked the detail to respond to specific questions from leads and customers.

"If an advertiser wanted to know how many commuters were typically connected to the Wi-Fi at a certain time or in a particular location, our developers would need to write a script and run a new query. It would take a minimum of 28 hours and sometimes as long as a week to generate a response," said Jeremy Foran, Technology Specialist at BAI Communications.

This was not only time consuming for developers, but also meant that by the time a question was answered, the sales opportunity may have been lost. The business needed the ability to answer questions the day they were asked and also wanted to delve deeper into the Wi-Fi data to find new insights on user behaviour that previously weren't seen by their team. This was particularly important with advertisers seeking to be more targeted in their approach and reach consumers in specific locations.

The business needed an analytics solution that was versatile, fast, and wouldn't slow down the accessibility of the Wi-Fi network. The average time users spent connected to the network was less than two minutes per session, so to maximise their experience and returns for advertisers, the time to log on needed to be less than ten seconds.

At the same time and on a parallel track, Foran was building a centralised log system for BAI using the Elastic Stack. They had previously stored logs on a third-party Linux server, but when they were forced to take on the maintenance of that server themselves, it became unsustainable.

Foran's research into alternatives led him to the Elastic Stack. "Among other things, we were impressed with the anomaly detection of the machine learning feature. It's like having an algorithm that mirrors someone with forty or fifty years’ experience," said Foran. "It allows us to spot issues right away whereas in the past you basically had to be a Linux ninja to go through the server logs and hone in on a problem."

"So when the business put forth its requirements, I thought, 'why don’t we use Elastic Stack for both logging and analytics?'"

The IT team spun up a proof of concept and gave sales reps access to Kibana where they could explore data on their own. They could respond to advertisers' specific questions right away, including how many users were connected to the Wi-Fi at any given time and at any given location. Also, with instant insights on user behaviour — including where and when they logged on to the Wi-Fi — reps could proactively tailor the conversations with potential customers by showing them exactly where and how their money would be best spent.

BAI Communications' initial production environment included a number of Elastic products on version 2.x. This included Logstash and Filebeat for ingesting and shipping logs into Elasticsearch, and Kibana for all kinds of visualizations. There are dashboards for Sales and Operations and, in the company's New York office, there's a wall-sized display that brings them altogether.

The business can monitor and run instant reports on Wi-Fi usage across the network and narrow in to review metrics for specific times and locations. By combining usage data with machine learning, it can also detect anomalies, including those related to the number of users. For instance, if it's 9am on a weekday and there are 50% fewer users than usual, Foran and his team know there's a problem and can work quickly to fix it.

A recent upgrade to Elastic Stack 6.0 has given BAI Communications access to more features, including the full screen dashboard. It has also provided advancements in compression, allowing the business to increase data retention from 12 to 16 months without increasing their IT infrastructure. The business can report and search across 2.5 billion documents and its average Elasticsearch CPU usage is 2%.

"One of the great things about the Elastic Stack is that it has given us more features and more bandwidth and all we needed to do was upgrade. It didn't cost us anything more," said Foran.

"A seamless experience for commuters and new opportunities for business."

The Elastic Stack helps BAI Communications keep commuters connected as they move through the underground rail networks. The IT team can spot anomalies faster and fix small issues before the impact becomes large. More importantly, the business has a better understanding of how and when commuters are using the Wi-Fi.

This makes it easier to sell existing advertising space and identify new premium opportunities that take advantage of stations increased traffic. Sales, operations and management teams can use Kibana to explore the data on their own and also receive daily updates. These reports once took a minimum of 30 minutes to prepare and are now instant, saving hours of administrative work each week.

Looking to the future, Foran said the Wi-Fi usage data holds potential value for rail operators as well. With additional modelling, the number of users and how long they were connected can indicate how many commuters were in the station at a particular time and how long they waited for a train. These insights could be used to make predictions about passenger numbers and wait times, and inform decisions to manage congestion. This could include adding more trains or cars where the Wi-Fi data suggests passengers are waiting too long.

"We have millions of data points coming in each hour, and being able to analyse these and detect anomalies is quite powerful. It helps us increase the efficiency and monetisation of our own infrastructure, but opens up new possibilities for our customers as well," said Foran.