Behavior Correlation using Elasticsearch: Cause Award Honoree NoSchoolViolence.org

The intersection of sociology and technology is where Paul Privateer envisions the future of his work. Privateer, who holds a PhD in informatics and narrative theory, is the founder of NoSchoolViolence.org (NSV), an organization on a mission to reduce school violence by identifying predictive behavioral patterns in adolescents. NSV was one of our Elastic Cause Award honorees at Elastic{ON} 2017.

Using Elasticsearch, the NSV team created The Lantern: a tool that helps researchers, parents, school administrators, and mental health professionals make connections between specific observable behaviors and potential violence. Rather than focusing on strategies solely related to coping with school violence, this app gives community members the resources to proactively help at risk adolescents before they act out.

The beginnings of this project stem from personal exposure. After experiencing student tragedies first-hand as a graduate-level professor, Privateer grew wary of hearing the response, “There was no way we could have known.” Was there really no way to predict any of these cases?

Wondering how he could reach out to students before a tragedy, Privateer began making mental notes of shared behaviors like slipping grades, antisocial tendencies, unjustified hostility, and more. What if he could find a pattern that would serve as an alert to parents and teachers? He knew he was onto something when he discovered that “the CDC, the World Health Organization, Department of Justice, FBI, the UN, all recognize 12 [risk factors of violence] all the way from cyber-bullying, sexual harassment, weapons use, fighting, drug use,” Privateer explained.

The discovery of this correlation compelled Privateer to take action. He assembled a small team to start researching these connections further, combing through police state records, FBI records, and academic journals. Over time he became worried about how they would identify relationships. “If it’s SQL…in the old form of relational, how are we going to even begin to deal with that? That would take eons to be able to look through all of those grids and start looking at those graphs and then correlating the implications of it.” For the project to be valuable, the NSV team had to find a realistic way to interact with their data.

Based on a recommendation from his student, Privateer started using Elasticsearch - and the seemingly untamable database became a beacon of hope. By putting Elasticsearch on top of a Python flask tool, The Lantern app can structure the correlating frequency of a specific term to the possible form of violence. Users of the search function can type in a string, and have the frequency of the associated type of violence returned. Their research became accessible and their app could give concerned parents and administrators direct insight regarding these behavioral correlations.

The forecast for Privateer and his team involves a more sophisticated use of Elasticsearch. Going beyond single use case searches, he sees the technology being used to create pattern frameworks that help “create a much broader intervention program that deals with children in a much more precise and accurate way.” As the tools grow more robust, the scope of their focus will likely expand into other areas including health issues and academic performance.

Privateer gave a detailed presentation during the closing keynote at Elastic{ON} 2017, in addition to an on-camera interview during the event (below).