Support horror stories, cluster sizing, Graph, and what the heck is BM25? Counting down the top 10 videos from Elastic{ON}16.

<html><head></head><body>Elastic{ON}16 was a rad experience. This was my first time at our annual user conference. It was so much fun and an amazing opportunity for meeting other users, <a href="/blog/ama-booth-elasticon-tour-2015-recap">asking questions</a> of our amazing dev team, and learning more about the Elastic Stack.
For those of you who attended, I can&apos;t read all of your minds. So I can&apos;t say for sure that it was as much of an overwhelming and cool experience for you, but I did have a unique vantage point as one of the coordinators of the conference&apos;s <a href="http://doubledutch.me/">official mobile app</a>.
One thing that stood out in the mobile app comment feed again and again is that there were many great presentations happening across three stages, plus the Spotlight Theater, and the Birds of a Feather community chats in the lunch area.&#xA0;It was often hard to choose which talk to attend. We have proudly shared videos of all of the great <a href="/elasticon/conf/2016/sf">Elastic{ON}16 presentations</a>, and now want to provide you with a roundup of the&#xA0;most popular talks.
We had two criteria for selecting videos for this top 10 list: 1) top views on Elastic.co and 2) top-rated video (1 to 5 stars) by users from our mobile app. A handful even achieved the impressive average rating of 5 out of 5 &#x2014; there are diamonds in the rough here!
So we narrowed the list down to the following 10 vids. We hope you find a presentation or two that you haven&apos;t watched, or maybe didn&apos;t even realize existed. Without further delay, enjoy!
1) <a href="/elasticon/conf/2016/sf/dive-deep-with-logstash-from-pipelines-to-persistent-queues?storm=blog-contents&amp;elektra=blog">Dive Deep with Logstash &#x2013; From Pipelines to Persistent Queues</a>
Logstash&apos;s creator Jordan Sissel and team lead Suyog Rao wowed the crowd when they presented &#x201C;<a href="/elasticon/conf/2016/sf/whats-the-latest-in-logstash?storm=blog-contents&amp;elektra=blog">The Latest in Logstash</a>,&#x201D; but this later talk &#x2014; a technical deep dive with core developers Colin Surprenant and Andrew Cholakian &#x2014; has been very popular on demand, earning a place on our list.
2) <a href="/elasticon/conf/2016/sf/geospatial-data-structures-in-elasticsearch-and-apache-lucene?storm=blog-contents&amp;elektra=blog">Geospatial Data Structures in Elasticsearch and Apache Lucene</a>
&#x201C;Space&#x2026; the final frontier.&#x201D; Oh wait, this isn&apos;t about that awesome Star Trek vs. Star Wars string wall at Elastic{ON}16? My bad. This geo aggregations and visualizations talk is all about geo and Elasticsearch, and was given by core Elasticsearch developer Nick Knize, who self-describes himself as our &#x201C;spatial agent.&#x201D;
3) <a href="/elasticon/conf/2016/sf/graph-capabilities-in-the-elastic-stack?storm=blog-contents&amp;elektra=blog">Graph Capabilities in the Elastic Stack</a>
Probably the most anticipated and talked about new feature announced at Elastic{ON}16, <a href="/products/graph?storm=blog-contents&amp;elektra=blog">Graph</a> is now available for <a href="/downloads/graph?storm=blog-contents&amp;elektra=blog">download</a>. Elastic&apos;s Mark Harwood and Steve Kearns revealed Graph&apos;s power to discover and explore the relevant connections in your data sets.
<img style="display: block; margin: auto;" alt="elasticon-16-string-wall.jpg" data-sys-asset-uid="blt244bd714982246c9" src="https://api.contentstack.io/v2/assets/575e4d57d8edd48f7693804d/download?uid=blt3522809d1c21392c?uid=blt3522809d1c21392c">The amazing string wall. 4) <a href="/elasticon/conf/2016/sf/how-to-build-your-own-kibana-plugins?storm=blog-contents&amp;elektra=blog">How to Build Your Own Kibana Plugins</a>
Another superstar among on-demand recordings! Elastic software engineer Spencer Alger and Director of Product Management Tanya Bragin discussed Kibana&apos;s extensibility, the goals of Kibana plugins, and resources like the plugin generator.
5) <a href="/elasticon/conf/2016/sf/improved-text-scoring-with-bm25?storm=blog-contents&amp;elektra=blog">Improved Text Scoring with BM25</a>
&quot;Here be math.&quot; The first talk on this list that was rated 5 stars out of 5 by you, attendees of Elastic{ON}16! Elasticsearch software engineer Britta Weber spoke about BM25, the scoring algorithm of Apache Lucene 6 that will soon be the new default scoring algorithm in Elasticsearch.
6) <a href="/elasticon/conf/2016/sf/ingest-node-enriching-documents-within-elasticsearch?storm=blog-contents&amp;elektra=blog">Ingest Node: Enriching Documents within Elasticsearch</a>
Enter Ingest Node: a new node type that will allow you to enrich documents before indexing. This is another talk &#x2014; featuring Elastic&apos;s Martijn van Groningen, Tal Levy, and Jim Unger &#x2014; that attendees rated 5/5 stars!
7) <a href="/elasticon/conf/2016/sf/quantitative-cluster-sizing?storm=blog-contents&amp;elektra=blog">Quantitative Cluster Sizing</a>
Also rated 5/5 is this gem of a talk on choosing the best size and configuration of your cluster. Presented by Elastic solution architect Christian Dahlqvist and Elastic technical trainer Ryan Schneider, this features the answers to important questions that so many of you have about sizing your cluster.
8) <a href="/elasticon/conf/2016/sf/securing-elasticsearch?storm=blog-contents&amp;elektra=blog">Securing Elasticsearch</a>
The first talk on our list that took place in the smaller Spotlight Theater location at Elastic{ON}16 is all about securing Elasticsearch. Software engineer Igor Motov and security engineer Jay Modi discuss common pitfalls for securing data, document- and field-level security, and why <a href="/downloads/x-pack">X-Pack</a> will make integrated security across the Elastic Stack so grand.
9) <a href="/elasticon/conf/2016/sf/stories-from-elastic-support-top-problems-and-solutions?storm=blog-contents&amp;elektra=blog">Stories from Support: Top Problems and Solutions</a>
You&apos;re not alone. That&apos;s the message that was sent loud and clear by Elastic&apos;s Chris Earle and Mark Walkom as they presented on the most interesting trials and tribulations our customers have shared with our <a href="/subscriptions">support team</a>. This talk was given five stars by in-app voting!
10) <a href="/elasticon/conf/2016/sf/whats-evolving-in-elasticsearch?storm=blog-contents&amp;elektra=blog">What&apos;s Evolving in Elasticsearch</a>
In one of the most popular talks from the first day of Elastic{ON}16, Elastic&apos;s Clinton Gormley and Simon Willnauer walked the audience through the breaking news and big developments for Elasticsearch, including <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html">pipeline aggregations</a>, the <a href="/blog/elasticsearch-2-2-0-and-2-1-2-and-1-7-5-released#query-profiler">query profiler</a>, and much more.
We&apos;re very grateful that we were able to hang out with so many of our friends and colleagues at Elastic{ON} this year. If you haven&apos;t heard, we&apos;re doing it all again at Pier 48 next year. Hope to see all of you in San Francisco&#xA0;at Elastic{ON}17, March 7-9, 2017! <a href="/elasticon/updates?storm=blog-contents&amp;elektra=blog">Sign up</a>&#xA0;for notifications.</body></html>

top-10-elasticon-16-presentations.jpg

BM25-slide.png

Five-Star Videos and More: Top 10 Elastic{ON}16 Presentations

USAA’s old SIEM wasn’t giving them the same bang for the buck that the Elastic Stack does today. Elastic quickly became part of USAA's cyber threat prevention.

<html><head></head><body>Is your company getting the most out of its SIEM and log management solutions? Allow us to share a story with a happy ending. It&apos;s about a company that needed the best way to analyze their ever-growing data, in order to protect personal information and other sensitive files. The analysts now enjoy faster, easier data management, while predicting and averting cyber threats all along the way &#x2026; all while USAA saves money.
<img src="https://api.contentstack.io/v2/assets/575e4d5243e9adc538715bce/download?uid=blt9ae15080fcbf0339?uid=blt9ae15080fcbf0339" data-sys-asset-uid="blt1cd331ef8a036e3b" alt="Neelsen Cyrus - USAA" style="float: left; width: 296px; height: 296px; margin:0 20px 20px 0;">USAA is a financial institution serving the U.S. military community, and Neelsen &quot;Nelly&quot; Cyrus has been a part of that institution for almost two decades. As a senior security analyst in the company&apos;s Cyber Threat Operations Center (CTOC), his primary focus is infrastructure support. At <a href="/elasticon/conf/2016/sf">Elastic{ON}16</a> in San Francisco, we were honored to host Nelly and five other attendees from the USAA CTOC.USAA&apos;s 93-year history stretches back to a group of U.S. Army officers who saw a need for auto insurance when other insurers had classified military officers as &quot;high risk.&quot; Today, <a href="https://www.usaa.com" target="_blank">USAA</a> employs over 26,000 people, boasts a multi-billion dollar annual net income, and has been consistently named one of the 100 Best Companies to Work For over the past 11 years by <a href="http://fortune.com/best-companies/" target="_blank">Fortune</a>.
At Elastic{ON}16, Nelly presented on USAA&apos;s transition to the Elastic Stack from a security information and event management (SIEM) solution &#x2014; a transition that saved the company money and improved productivity among the company&apos;s security analysts.
The presentation included a recipe for &quot;hunting&quot; &#x2014; the practice of information security analysts proactively seeking out malicious activity and vulnerabilities before harm is done. Hunters have to think like attackers and block off routes before they can be exploited.
As the volume of these attacks is ever-increasing, analysts like Nelly and the team at USAA CTOC rely on technology like the Elastic Stack for effective logging and constant monitoring for malicious activity.
<img src="https://api.contentstack.io/v2/assets/575e4d529985d58976378b1a/download?uid=blt842d83f2f4323807?uid=blt842d83f2f4323807" data-sys-asset-uid="bltb310d9bdc482c5e4" alt="USAA - Elastic - Quote" style="float:right; width: 418px; height: 209px; margin:0 0 20px 20px;">USAA traditionally invested in large enterprise solutions. Advocating the Elastic Stack, an open source product, up the chain of command for use at the size and scale needed by the CTOC team was challenging. However, the <a href="/subscriptions">subscription support</a> available from Elastic&apos;s world-class engineers made all the difference.
&quot;We know that they&apos;re there, and they&apos;ve proven it time and time again,&quot; Nelly said. One manager assured Nelly that he made the right move. The cost was easily justifiable based on seeing analyst productivity improvements.In production, USAA&apos;s Elastic Stack deployment has grown to seven clusters, grouped by feed type &#x2014; feeds change often but include &quot;almost all of the major security appliances,&quot; Unix and Windows server events, etc. &#x2014; after they broke up their single, monolithic cluster (and upgraded <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html">to Elasticsearch 2.0</a>) about two months before Elastic{ON}16. They send 24 feeds into Elasticsearch, with between 2 billion and 4 billion security events daily and an average of about 52,700 events per second. They have 53.11 billion documents in their store.
<a href="https://www.elastic.co/elasticon/conf/2016/sf/all-quiet-digital-front-security-analytics-usaa"><img src="https://api.contentstack.io/v2/assets/575e4d520342dfd7382653a4/download?uid=blt6d7ba7e4930f82a4?uid=blt6d7ba7e4930f82a4" data-sys-asset-uid="blt82d0aee5acda0a2b" alt="Neelsen Cyrus, USAA - at Elastic{ON}16"></a>To watch the full USAA&#xA0;presentation, click the image above or just follow this&#xA0;<a href="/elasticon/conf/2016/sf/all-quiet-digital-front-security-analytics-usaa">link</a>.
About moving to the multi-cluster setup, Nelly noted that they cut the time it takes to create a snapshot of their data from 20+ hours to 10 hours, with the snapshots executed in parallel. It&apos;s important for all companies to back up data, but keenly important for those in financial and military spaces like USAA. The great thing about Elasticsearch&apos;s snapshot API is that after the first backup process, subsequent snapshots save the delta change between the existing snapshots and new data. Transmitting far less data means snapshots takes less valuable time away from CTOC personnel.Bottom line: USAA&apos;s old SIEM and old log management solution weren&apos;t giving them the same bang for the buck that the Elastic Stack does today. Elastic quickly became an integral part of USAA&apos;s cyber threat prevention process, and the speed and scale of Elastic helps these analysts ask questions (and find answers) that they couldn&apos;t ask before&#xA0;&#x2014; and if this interests you, definitely watch the recording and hear all about Nelly&apos;s eight steps for hunting success.And if you want to check out&#xA0;more Elastic{ON} security analytics presentations, we suggest <a href="/elasticon/conf/2016/sf/tapping-out-security-threats-at-fireeye">Tapping Out Security</a> with FireEye, <a href="/elasticon/conf/2016/sf/hunting-the-hackers-how-cisco-talos-is-leveling-up-security">Hunting the Hackers</a> by Cisco&apos;s Talos, <a href="/elasticon/tour/2015/los-angeles/keeping-your-data-from-getting-swiped-right-away-security-analytics-at-tinder">Tinder: Keeping Your Data From Getting Swiped Right Away</a> from last year&apos;s Elastic{ON} Tour &#x2014; Los Angeles, and <a href="/elasticon/tour/2015/washington-dc/cyber-security-log-analytics-at-decision-lab">Cyber Security Log Analytics</a> with Decision Lab at Elastic{ON} Tour &#x2014; Washington, D.C.
Nelly is also active in the San Antonio DevOps&#xA0;Meetup. If you are in the San Antonio, Texas, area <a href="http://www.meetup.com/SanAntonioDevOps/events/230169400/">on April 25, 2016</a>, attend to learn&#xA0;about the USAA use case and more.</body></html>

USAA-Elasticon-16-full-bleed.jpg

usaa-ctoc-mission-board-elasticsearch-upgrade.jpg

USAA, Security Analytics, and a Journey to the Elastic Stack

Learn why Elastic Cloud — the official hosted Elasticsearch service — is the best choice for your company, organization, or project. Plus, infrastructure info.

<html><head></head><body><img src="https://api.contentstack.io/v2/assets/575e4d450342dfd738265138/download?uid=blte5e2dff4ce491991?uid=blte5e2dff4ce491991" data-sys-asset-uid="blt7593c169fd93296f" alt="Elastic Cloud" style="float: left; width: 509px; margin: 0px; max-width:100%;" height="246" width="509">
Allow me to ask you a couple of questions that might seem silly. Do you only eat food that you&apos;ve grown? No? Did you build your own home or apartment? No? Why not? Was it easier to simply buy or rent a place that suitably addresses your needs for shelter and comfort?
Sometimes it&apos;s wise to just take the path of least complexity. That&apos;s pretty much what makes <a href="https://www.elastic.co/cloud">Elastic Cloud</a> &#x2014; the official hosted Elasticsearch service &#x2014; a perfect choice for many companies, organizations, and individuals. Choosing us for your Elasticsearch needs simply makes good sense. We understand that every team and system has a set of unique requirements. However, if you think about hosting the Elastic Stack on your own and consider the time needed to deploy, setup, get started, and maintain &#x2014; you or your boss may be thinking, &quot;How can I get this taken care of with a minimum of added stress?&quot;
That&apos;s where Elastic Cloud comes in. It&apos;s the only hosted Elasticsearch offering from the creators of Elasticsearch, Kibana, Beats, and Logstash. Whether your team needs 1GB of memory or 4TB of storage, Elastic Cloud is the best hosted Elasticsearch experience. Fully hosted and providing the real-time search and analytics magics of <a href="/products/elasticsearch">Elasticsearch</a>, Elastic Cloud is always running the most current version of Elasticsearch and incorporates all of the latest and greatest <a href="/cloud/features">features</a>. You can also get Elastic&apos;s world-class tech <a href="/cloud/support">support</a> and security, alerting, and monitoring. And you can go try it out right now with a free <a href="/cloud/signup">14-day trial</a>. Don&apos;t take my word for it, experience it for yourself.
For <a href="/elasticon/conf/2016/sf">Elastic{ON}16</a>, over 1,800 users converged on San Francisco&apos;s Pier 48 to hear from the creators and engineers of the Elastic Stack. This included a fantastic presentation on the architecture and history of Elastic Cloud by Software Engineer Njal Karevoll and Site Reliability Engineer Erik Redding, as well as a live demo.
 In the recording below, watch Erik and Njal discuss the history and early architectures of what is now Elastic Cloud, a fascinating section about &quot;living with systems that fail&quot;, an in-depth look at Elastic Cloud&apos;s current architecture, and the demo, in which they install <a href="https://info.elastic.co/cloud-enterprise?storm=blog-text&amp;elektra=blog" target="_blank">Elastic Cloud Enterprise</a> on three servers, all while shining a big flashlight to illuminate how it all works.<a href="https://www.elastic.co/elasticon/conf/2016/sf/how-cloud-hosted-and-managed-elasticsearch-is-built"><img src="https://api.contentstack.io/v2/assets/575e4d4543e9adc538715991/download?uid=blt2865e24c626c6681?uid=blt2865e24c626c6681" data-sys-asset-uid="blt057a528daddc1a52" alt="erik-redding-njal-karevoll-cloud-hosted-elasticsearch-how-its-built.jpg" style="display: block; margin: auto;"></a>
If you enjoyed this presentation, you may also want to check out the Elastic{ON}16 <a href="/elasticon/conf/2016/sf/opening-keynote">Keynote</a>, in which Njal and Elastic Cloud Team Lead Michael Basnight presented a demo of Elastic Cloud Enterprise, beginning at the 85-minute mark.
For even more Elastic Cloud coolness:
<ul>
	<li>Check out Software Engineer Konrad Beiske&apos;s recent article on <a href="/blog/cluster-cloning-in-the-cloud">cluster cloning</a>.</li>
	<li>Read about how <a href="/use-cases/docker">Docker uses Elastic Cloud</a> to power its high-performance search.</li>
	<li>Read the official blog post from Elastic{ON}16 <a href="/blog/introducing-elastic-cloud-and-elastic-cloud-enterprise">announcing Elastic Cloud Enterprise</a>.</li>
	<li>Read the announcement that <a href="/blog/hosted-elasticsearch-monitoring-available-on-found">monitoring has joined</a> with alerting and security as Elastic Cloud features.</li>
</ul></body></html>

thumbs-up-for-elastic-cloud.jpg

clouds-over-lake-elastic-cloud.jpg

Why Elastic Cloud is the Best Choice for Your Hosted Elasticsearch Needs

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Jason Dickson

Register to Attend

Register to Watch

Надсилаючи цю форму ви погоджуєтеся з <a href="/legal/terms-of-use">Умовами надання послуг Elastic</a>. Ваші особисті дані будуть опрацьовані відповідно з <a href="/legal/privacy-statement">Положеннями про конфіденційність Elastic</a>.

Newsletter GDPR Text - Ukrainian

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Intentionally Blank

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

提交即表示您確認已詳閱並同意我們的<a href="/legal/elastic-cloud-account-terms" target="_blank">服務條款</a>，且同意 Elastic 使用您在上方提供的個人資料<a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">與您聯絡</a>，為您提供相關產品和服務的資訊。如需詳細說明或隨時想停止收到產品和服務資訊，請參閱 <a href="/legal/privacy-statement/" target="_blank">Elastic 的隱私聲明</a>。

Newsletter GDPR Text - TW

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.

GDPR Text + Google Cloud

Inviando la registrazione (1) esprimi il consenso ai <a href="/legal/terms-of-use">termini e condizioni di Elastic</a> e <a href="/legal/privacy-statement">all’informativa sulla privacy</a>, nonché (2) a ricevere saltuarie e-mail.

Newsletter GDPR Text - Italian

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Load more

Overview

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content.

You'll also receive an email with related content.

You'll also receive an email with related content

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

Author

Articles by Jason Dickson

Five-Star Videos and More: Top 10 Elastic{ON}16 Presentations

USAA, Security Analytics, and a Journey to the Elastic Stack

Why Elastic Cloud is the Best Choice for Your Hosted Elasticsearch Needs