Path parameters

• id string Required

  Identifier for the data frame analytics job. If you do not specify this option, the API returns information for the first hundred data frame analytics jobs.

Query parameters

• allow_no_match boolean

  Specifies what to do when the request:

  1. Contains wildcard expressions and there are no data frame analytics jobs that match.
  2. Contains the _all string or no identifiers and there are no matches.
  3. Contains wildcard expressions and there are only partial matches.

  The default value returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

• from number

  Skips the specified number of data frame analytics jobs.

• size number

  Specifies the maximum number of data frame analytics jobs to obtain.

• verbose boolean

  Defines whether the stats response should be verbose.

Responses

• 200 application/json
  Hide response attributes Show response attributes object

  • count number Required
  • data_frame_analytics array[object] Required

    An array of objects that contain usage information for data frame analytics jobs, which are sorted by the id value in ascending order.

    Hide data_frame_analytics attributes Show data_frame_analytics attributes object

    • analysis_stats object

      Additional properties are allowed.

      Hide analysis_stats attributes Show analysis_stats attributes object

      • classification_stats object

        Additional properties are allowed.

        Hide classification_stats attributes Show classification_stats attributes object

        • hyperparameters object Required

          Additional properties are allowed.

          Hide hyperparameters attributes Show hyperparameters attributes object

          • alpha number

            Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

          • lambda number

            Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

          • gamma number

            Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

          • eta number

            Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

          • eta_growth_rate_per_tree number

            Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

          • feature_bag_fraction number

            Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

          • downsample_factor number

            Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

          • max_attempts_to_add_tree number

            If the algorithm fails to determine a non-trivial tree (more than a single leaf), this parameter determines how many of such consecutive failures are tolerated. Once the number of attempts exceeds the threshold, the forest training stops.

          • max_optimization_rounds_per_hyperparameter number

            Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

          • max_trees number

            Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

          • num_folds number

            The maximum number of folds for the cross-validation procedure.

          • num_splits_per_feature number

            Determines the maximum number of splits for every feature that can occur in a decision tree when the tree is trained.

          • soft_tree_depth_limit number

            Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

          • soft_tree_depth_tolerance number

            Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • iteration number Required

          The number of iterations on the analysis.

        • timestamp number

          Time unit for milliseconds

        • timing_stats object Required

          Additional properties are allowed.

          Hide timing_stats attributes Show timing_stats attributes object

          • elapsed_time
          • iteration_time
        • validation_loss object Required

          Additional properties are allowed.

          Hide validation_loss attributes Show validation_loss attributes object

          • fold_values array[string] Required

            Validation loss values for every added decision tree during the forest growing procedure.

          • loss_type string Required

            The type of the loss metric. For example, binomial_logistic.

      • outlier_detection_stats object

        Additional properties are allowed.

        Hide outlier_detection_stats attributes Show outlier_detection_stats attributes object

        • parameters object Required

          Additional properties are allowed.

          Hide parameters attributes Show parameters attributes object

          • compute_feature_influence boolean

            Specifies whether the feature influence calculation is enabled.

          • feature_influence_threshold number

            The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1

          • method string

            The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

          • n_neighbors number

            Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

          • outlier_fraction number

            The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

          • standardization_enabled boolean

            If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

        • timestamp number

          Time unit for milliseconds

        • timing_stats object Required

          Additional properties are allowed.

          Hide timing_stats attributes Show timing_stats attributes object

          • elapsed_time
          • iteration_time
      • regression_stats object

        Additional properties are allowed.

        Hide regression_stats attributes Show regression_stats attributes object

        • hyperparameters object Required

          Additional properties are allowed.

          Hide hyperparameters attributes Show hyperparameters attributes object

          • alpha number

            Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

          • lambda number

            Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

          • gamma number

            Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

          • eta number

            Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

          • eta_growth_rate_per_tree number

            Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

          • feature_bag_fraction number

            Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

          • downsample_factor number

            Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

          • max_attempts_to_add_tree number

            If the algorithm fails to determine a non-trivial tree (more than a single leaf), this parameter determines how many of such consecutive failures are tolerated. Once the number of attempts exceeds the threshold, the forest training stops.

          • max_optimization_rounds_per_hyperparameter number

            Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

          • max_trees number

            Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

          • num_folds number

            The maximum number of folds for the cross-validation procedure.

          • num_splits_per_feature number

            Determines the maximum number of splits for every feature that can occur in a decision tree when the tree is trained.

          • soft_tree_depth_limit number

            Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

          • soft_tree_depth_tolerance number

            Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • iteration number Required

          The number of iterations on the analysis.

        • timestamp number

          Time unit for milliseconds

        • timing_stats object Required

          Additional properties are allowed.

          Hide timing_stats attributes Show timing_stats attributes object

          • elapsed_time
          • iteration_time
        • validation_loss object Required

          Additional properties are allowed.

          Hide validation_loss attributes Show validation_loss attributes object

          • fold_values array[string] Required

            Validation loss values for every added decision tree during the forest growing procedure.

          • loss_type string Required

            The type of the loss metric. For example, binomial_logistic.

    • assignment_explanation string

      For running jobs only, contains messages relating to the selection of a node to run the job.

    • data_counts object Required

      Additional properties are allowed.

      Hide data_counts attributes Show data_counts attributes object

      • skipped_docs_count number Required

        The number of documents that are skipped during the analysis because they contained values that are not supported by the analysis. For example, outlier detection does not support missing fields so it skips documents with missing fields. Likewise, all types of analysis skip documents that contain arrays with more than one element.

      • test_docs_count number Required

        The number of documents that are not used for training the model and can be used for testing.

      • training_docs_count number Required

        The number of documents that are used for training the model.

    • id string Required
    • memory_usage object Required

      Additional properties are allowed.

      Hide memory_usage attributes Show memory_usage attributes object

      • memory_reestimate_bytes number

        This value is present when the status is hard_limit and it is a new estimate of how much memory the job needs.

      • peak_usage_bytes number Required

        The number of bytes used at the highest peak of memory usage.

      • status string Required

        The memory usage status.

      • timestamp number

        Time unit for milliseconds

    • progress array[object] Required

      The progress report of the data frame analytics job by phase.

      Hide progress attributes Show progress attributes object

      • phase string Required

        Defines the phase of the data frame analytics job.

      • progress_percent number Required

        The progress that the data frame analytics job has made expressed in percentage.

    • state string Required

      Values are started, stopped, starting, stopping, or failed.