Logging

elasticsearch-hadoop uses commons-logging library, same as Hadoop, for its logging infrastructure and thus it shares the same configuration means. Out of the box, no configuration is required - by default, elasticsearch-hadoop logs relevant information about the job progress at INFO level. Typically, whatever integration you are using (Map/Reduce, Cascading, Hive, Pig), each job will print in the console at least one message indicating the elasticsearch-hadoop version used:

16:13:01,946  INFO main util.Version - Elasticsearch Hadoop v2.0.0.BUILD-SNAPSHOT [f2c5c3e280]

Configuring logging for Hadoop (or Cascading, Hive and Pig) is outside the scope of this documentation, however in short, at runtime, Hadoop relies on log4j 1.2 as an actual logging implementation. In practice, this means adding the package name of interest and its level logging the log4j.properties file in the job classpath. elasticsearch-hadoop provides the following important packages:

The default logging level (INFO) is suitable for day-to-day use; if troubleshooting is needed, consider switching to DEBUG but be selective of the packages included. For low-level details, enable level TRACE however do remember that it will result in a significant amount of logging data which will impact your job performance and environment.

To put everything together, if you want to enable DEBUG logging on the Map/Reduce package make sure the log4j.properties (used by your environment):

log4j.category.org.elasticsearch.hadoop.mr=DEBUG