Selecting fields to return

When indexing a document, by default, Elasticsearch stores the originally sent JSON document in a special field called _source. Documents returned from a search query are materialized from the _source field returned from Elasticsearch for each hit.

It is also possible to store a field from the JSON document separately within Elasticsearch by using store on the mapping. Why would you ever want to do this? Well, you may disable _source so that the source is not stored and select to store only specific fields. Another possibility is that the _source contains a field with large values, for example, the body of a blog post, but typically only another field is needed, for example, the title of the blog post. In this case, we don’t want to pay the cost of Elasticsearch deserializing the entire _soure just to get a small field.

When storing fields in this manner, the individual field values to return can be specified using .StoredFields on the search request

var searchResponse = _client.Search<Project>(s => s
    .StoredFields(sf => sf
        .Fields(
            f => f.Name,
            f => f.StartedOn,
            f => f.Branches
        )
    )
    .Query(q => q
        .MatchAll()
    )
);

And retrieving them is possible using .Fields on the response

foreach (var fieldValues in searchResponse.Fields)
{
    var document = new 
    {
        Name = fieldValues.ValueOf<Project, string>(p => p.Name),
        StartedOn = fieldValues.Value<DateTime>(Infer.Field<Project>(p => p.StartedOn)),
        Branches = fieldValues.Values<Project, string>(p => p.Branches.First())
    };
}

This works when storing fields separately. A much more common scenario however is to return only a selection of fields from the _source; this is where source filtering comes in.

Only some of the fields of a document can be returned from a search query using source filtering

var searchResponse = _client.Search<Project>(s => s
    .Source(sf => sf
        .Includes(i => i 
            .Fields(
                f => f.Name,
                f => f.StartedOn,
                f => f.Branches
            )
        )
        .Excludes(e => e 
            .Fields("num*") 
        )
    )
    .Query(q => q
        .MatchAll()
    )
);

With source filtering specified on the request, .Documents will now contain partial documents, materialized from the source fields specified to include

var partialProjects = searchResponse.Documents;

It’s possible to exclude _source from being returned altogether from a query with

searchResponse = _client.Search<Project>(s => s
    .Source(false)
    .Query(q => q
        .MatchAll()
    )
);