Fluent mapping

Fluent mapping POCO properties to fields within an Elasticsearch type mapping offers the most control over the process. With fluent mapping, each property of the POCO is explicitly mapped to an Elasticsearch type field mapping.

To demonstrate, we’ll define two POCOs

Company, which has a name and a collection of Employees
Employee which has various properties of different types and has itself a collection of Employee types.

public class Company
{
    public string Name { get; set; }
    public List<Employee> Employees { get; set; }
}

public class Employee
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public int Salary { get; set; }
    public DateTime Birthday { get; set; }
    public bool IsManager { get; set; }
    public List<Employee> Employees { get; set; }
    public TimeSpan Hours { get; set; }
}

Manual mapping

edit

To create a mapping for our Company type, we can use the fluent API and map each property explicitly

var createIndexResponse = _client.Indices.Create("myindex", c => c
    .Map<Company>(m => m
        .Properties(ps => ps
            .Text(s => s
                .Name(n => n.Name)
            )
            .Object<Employee>(o => o
                .Name(n => n.Employees)
                .Properties(eps => eps
                    .Text(s => s
                        .Name(e => e.FirstName)
                    )
                    .Text(s => s
                        .Name(e => e.LastName)
                    )
                    .Number(n => n
                        .Name(e => e.Salary)
                        .Type(NumberType.Integer)
                    )
                )
            )
        )
    )
);

Here, the Name property of the Company type has been mapped as a text datatype and the Employees property mapped as an object datatype. Within this object mapping, only the FirstName, LastName and Salary properties of the Employee type have been mapped.

The json mapping for this example looks like

{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "employees": {
        "type": "object",
        "properties": {
          "firstName": {
            "type": "text"
          },
          "lastName": {
            "type": "text"
          },
          "salary": {
            "type": "integer"
          }
        }
      }
    }
  }
}

Manual mapping in this way is powerful but can become verbose and unwieldy for large POCOs. The majority of the time you simply want to map all the properties of a POCO in a single go without having to specify the mapping for each property, particularly when there is inferred mapping from CLR types to Elasticsearch types.

This is where the fluent mapping in conjunction with auto mapping comes in.

Auto mapping with fluent overrides

edit

In most cases, you’ll want to map more than just the vanilla datatypes and also provide various options for your properties, such as the analyzer to use, whether to enable doc_values, etc.

In this case, it’s possible to use .AutoMap() in conjunction with explicitly mapped properties.

Here we are using .AutoMap() to automatically infer the mapping of our Company type from the CLR property types, but then we’re overriding the Employees property to make it a nested datatype, since by default .AutoMap() will infer the List<Employee> property as an object datatype

var createIndexResponse = _client.Indices.Create("myindex", c => c
    .Map<Company>(m => m
        .AutoMap()
        .Properties(ps => ps
            .Nested<Employee>(n => n
                .Name(nn => nn.Employees)
            )
        )
    )
);

{
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "employees": {
        "type": "nested"
      }
    }
  }
}

.AutoMap() is idempotent therefore calling it before or after manually mapped properties will still yield the same result. The next example generates the same mapping as the previous

createIndexResponse = _client.Indices.Create("myindex", c => c
    .Map<Company>(m => m
        .Properties(ps => ps
            .Nested<Employee>(n => n
                .Name(nn => nn.Employees)
            )
        )
        .AutoMap()
    )
);

Auto mapping overrides down the object graph

edit

Just as we were able to override the inferred properties from auto mapping in the previous example, fluent mapping also takes precedence over Attribute Mapping. In this way, fluent, attribute and auto mapping can be combined. We’ll demonstrate with an example.

Consider the following two POCOS

[ElasticsearchType(RelationName = "company")]
public class CompanyWithAttributes
{
    [Keyword(NullValue = "null", Similarity = "BM25")]
    public string Name { get; set; }

    [Text(Name = "office_hours")]
    public TimeSpan? HeadOfficeHours { get; set; }

    [Object(Store = false)]
    public List<EmployeeWithAttributes> Employees { get; set; }
}

[ElasticsearchType(RelationName = "employee")]
public class EmployeeWithAttributes
{
    [Text(Name = "first_name")]
    public string FirstName { get; set; }

    [Text(Name = "last_name")]
    public string LastName { get; set; }

    [Number(DocValues = false, IgnoreMalformed = true, Coerce = true)]
    public int Salary { get; set; }

    [Date(Format = "MMddyyyy")]
    public DateTime Birthday { get; set; }

    [Boolean(NullValue = false, Store = true)]
    public bool IsManager { get; set; }

    [Nested]
    [PropertyName("empl")]
    public List<Employee> Employees { get; set; }
}

Now when mapping, AutoMap() is called to infer the mapping from the POCO property types and attributes, and inferred mappings are overridden with fluent mapping

var createIndexResponse = _client.Indices.Create("myindex", c => c
    .Map<CompanyWithAttributes>(m => m
        .AutoMap() 
        .Properties(ps => ps 
            .Nested<EmployeeWithAttributes>(n => n
                .Name(nn => nn.Employees)
                .AutoMap() 
                .Properties(pps => pps 
                    .Text(s => s
                        .Name(e => e.FirstName)
                        .Fields(fs => fs
                            .Keyword(ss => ss
                                .Name("firstNameRaw")
                            )
                            .TokenCount(t => t
                                .Name("length")
                                .Analyzer("standard")
                            )
                        )
                    )
                    .Number(nu => nu
                        .Name(e => e.Salary)
                        .Type(NumberType.Double)
                        .IgnoreMalformed(false)
                    )
                    .Date(d => d
                        .Name(e => e.Birthday)
                        .Format("MM-dd-yy")
                    )
                )
            )
        )
    )
);

	Automap company
	Override company inferred mappings
	Automap nested employee type
	Override employee inferred mappings

{
  "mappings": {
    "properties": {
      "employees": {
        "type": "nested",
        "properties": {
          "birthday": {
            "format": "MM-dd-yy",
            "type": "date"
          },
          "empl": {
            "properties": {
              "birthday": {
                "type": "date"
              },
              "employees": {
                "properties": {},
                "type": "object"
              },
              "firstName": {
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                },
                "type": "text"
              },
              "hours": {
                "type": "long"
              },
              "isManager": {
                "type": "boolean"
              },
              "lastName": {
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                },
                "type": "text"
              },
              "salary": {
                "type": "integer"
              }
            },
            "type": "nested"
          },
          "first_name": {
            "fields": {
              "firstNameRaw": {
                "type": "keyword"
              },
              "length": {
                "analyzer": "standard",
                "type": "token_count"
              }
            },
            "type": "text"
          },
          "isManager": {
            "null_value": false,
            "store": true,
            "type": "boolean"
          },
          "last_name": {
            "type": "text"
          },
          "salary": {
            "ignore_malformed": false,
            "type": "double"
          }
        }
      },
      "name": {
        "null_value": "null",
        "similarity": "BM25",
        "type": "keyword"
      },
      "office_hours": {
        "type": "text"
      }
    }
  }
}

As demonstrated, by calling .AutoMap() inside of the .Nested<Employee> mapping, it is possible to auto map the Employee nested properties and again, override any inferred mapping from the automapping process, through manual mapping

Mapping runtime fields

edit

A runtime field is a field that is evaluated at query time. Runtime fields may be defined in the mapping of an index.

In this example, we’ll define a CompanyRuntimeFields class with a single property which we may then use in the strongly-typed runtime field mapping.

public class CompanyRuntimeFields
{
    public string BirthDayOfWeek { get; set; }
}

var createIndexResponse = _client.Indices.Create("myindex", c => c
        .Map<Company>(m => m
            .RuntimeFields<CompanyRuntimeFields>(rtf => rtf 
                .RuntimeField(f => f.BirthDayOfWeek, FieldType.Keyword, f => f.Script("emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))"))) 
        )
    );

	Use the `CompanyRuntimeFields` class as the generic argument
	Use the `BirthDayOfWeek` property as the runtime field name

{
  "mappings": {
    "runtime": {
      "birthDayOfWeek": {
        "type": "keyword",
        "script": {
          "source": "emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))"
        }
      }
    }
  }
}

It’s not necessary to define a type for the runtime field mapping. Runtime fields can optionally be defined by providing a string name.

createIndexResponse = _client.Indices.Create("myindex", c => c
    .Map<Company>(m => m
        .RuntimeFields(rtf => rtf
            .RuntimeField("birthDayOfWeek", FieldType.Keyword, f => f.Script("emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))")))
    )
);

One may also include and use parameters in the script.

createIndexResponse = _client.Indices.Create("myindex", c => c
    .Map<Company>(m => m
        .RuntimeFields(rtf => rtf
            .RuntimeField("birthDayOfWeek", FieldType.Keyword, f => f
                .Script(s => s
                    .Source("emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT) + params.suffix)")
                    .Params(p => p.Add("suffix", " with a suffix."))
                    )))
    )
);

{
  "mappings": {
    "runtime": {
      "birthDayOfWeek": {
        "type": "keyword",
        "script": {
          "source": "emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT) + params.suffix)",
          "params": {
            "suffix": " with a suffix."
          }
        }
      }
    }
  }
}

« Attribute mapping Applying conventions through the Visitor pattern »