Swedish Public Employment Service brings job seekers and employers together with Elastic

Arbetsförmedlingen, the Swedish Public Employment Service, makes a vital contribution to the economy by matching employers and job seekers as efficiently as possible. Arbetsförmedlingen provides information, analysis, and employment forecasts to create the best conditions for a well-functioning labor market. It has about 12,000 employees including 1,000 IT specialists.

The organization must adapt to annual changes to governmental guidelines that determine its responsibilities. To meet these objectives, Arbetsförmedlingen deployed flexible IT infrastructure that supports more efficient services, including the digitalization of citizen and business communications.

Observability, search, and application performance monitoring (APM) all play a critical role in this innovative technology ecosystem. Over the years, the organization accumulated technical debt through the deployment of solutions from multiple vendors, which made it difficult to integrate processes, increase efficiency, and reduce costs.

Tobias Ström, Product Owner, Operation Center, Arbetsförmedlingen, explains some of the challenges facing the Logging Platform team. "Our observability solution had long response times and was out of date. The APM platform was expensive and complicated, and we wanted to consolidate several search tools into one solution."

Ronny Bergh, IT specialist at Arbetsförmedlingen, researched the market and spoke to teams using a mix of legacy platforms. In 2019, the organization was using ArcSight foundation but needed to expand to more users and disk space. However, they soon realized that ArcSight's platform was not flexible enough and would be costly to expand.

Bergh discovered that some Java developers had moved to an open-source version of Elastic for system logging. Impressed by its small but effective deployment, Arbetsförmedlingen decided to incorporate Elastic into their general solutions for monitoring and logging. As a result, they went from having trouble with only two users to hundreds working without any issues.

This evolved into a five-server cluster that attracted the attention of other Arbetsförmedlingen development teams. "At this stage, acquiring an Elastic Enterprise License made sense to accommodate the growing needs of our teams and products. With our log platform relying on Elasticsearch, we required 24/7 support for internal application and system support. As the license for the initial servers expired and our outdated legacy search platform 'siteseeker' needed to be replaced, we decided to move to Elastic Cloud Enterprise (ECE) for our log platform," says Bergh.

The adoption of Elastic Observability is now underway across all Arbetsförmedlingens product development teams (60+). This is driven by three teams, Stefan Jonsson, Product Owner, Arbetsförmedlingen leads the deployment of Elastic Cloud Enterprise, Ström leads the APM deployment while Bergh heads up logging activities.

Logging and APM are already well established, and the Arbetsförmedlingen leads are also exploring the potential of Elasticsearch for its development teams who are currently in the ITIL service strategy phase.

So far, the organization has 12 servers in its production environment and three in testing. This adds up to 73 Elastic Enterprise Resource Units (ERUs) that monitor approximately 800 systems and applications, storing a total of 130 terabytes of data.

Jonsson and his team also chose Elastic Cloud Enterprise (ECE) because it gave them the opportunity to deploy on premises with the same specifications and features as a hosted cloud offering. "Swedish and EU regulations mean that regulated or sensitive data cannot leave our internal network. With ECE, we get the power and scalability we need while complying with these requirements," says Jonsson.

Arbetsförmedlingen has seen significant improvements in productivity and a reduction of cost associated with multiple vendors since replacing its legacy logging and search tools with Elastic.

Take the example of development teams who have implemented APM in the test and production environments. They can now identify and fix errors earlier in the development process. This is critical for Arbetsförmedlingen’s journey towards a modern DevOps culture, as it increases efficiency in the software development process and improves product quality.

Another good example of logging in action is the process which prevents job seekers from being assigned to staff who are close family members. "To ensure fairness, we must check back three generations to make sure that they are not related. With Elastic, this check is automated, saving us a lot of man hours."

The ability to set up a bespoke Kibana dashboard to visualize and monitor logs also benefits employees who may not have the skills to configure complex systems. Bergh says, "In most cases, you can set up a dashboard in a matter of minutes. We now have about 400, providing employees with actionable insights that boost the performance of their teams and the wider organization."

Elastic enables Arbetsförmedlingen to reduce storage costs by 75% through the efficient use of data tiers. "The use of searchable snapshots capability means that we can search infrequently accessed and read-only data in the frozen data tier which reduces our dependency on expensive solid-state storage," says Jonsson.

The organization uses Elasticsearch to create a modern search experience for employers and employees throughout Sweden and enhance its extranet. "A great search experience is fundamental for all of our stakeholders," says Jonsson. "Our goal is to offer a complete package that includes search components at the front end and Elasticsearch at the back end for our application development teams in the near future." A growing number of teams are testing Elastic APM for the identification and resolution of root cause issues with correlated traces, logs, metrics, and metadata. "With Elastic we get deeper visibility into our applications with support for all our programming languages and distributed tracing. We are also planning on using machine learning to analyze infrastructure and application data to determine outliers and trends, and identify abnormal behavior."

Arbetsförmedlingen also benefits from the expertise of the local Elastic Professional Services team working out of Stockholm. "We have a great relationship with Elastic Consulting, and they provide us with a lot of assistance, whatever the question. This is quite a wide-ranging deployment, so their technical knowledge makes an enormous difference," says Jonsson.

The deployment of Elastic means that Arbetsförmedlingen is now ahead of the IT benchmark for Sweden's public sector and is paving the way for other organizations who want to boost the performance of their observability, search and APM platforms. "As more teams adopt Elastic in Arbetsförmedlingen, we hope to share our experience with other government organizations," says Jonsson.