SQLMetric — Physical Operator Metric

SQLMetric is a accumulator metric used to monitor performance of physical operators.

Note
 Use Details for Query page in SQL tab in web UI to see the metrics of a structured query.

SQLMetric takes a metric type and an initial value when created.

Table 1. Metric Types and Corresponding Create Methods (in alphabetical order)
Metric Type Create Method Failed Values Counted? Description

size

createSizeMetric

no

Used when…​

sum

createMetric

no

Used when…​

timing

createTimingMetric

no

Used when…​

Posting Driver-Side Updates to SQLMetrics — postDriverMetricUpdates Method

postDriverMetricUpdates(
  sc: SparkContext,
  executionId: String,
  metrics: Seq[SQLMetric]): Unit

postDriverMetricUpdates posts a SparkListenerDriverAccumUpdates to a SparkListenerBus when executionId is specified.

Note
 postDriverMetricUpdates used in BroadcastExchangeExec, FileSourceScanExec and SubqueryExec physical operators.

results matching ""

    No results matching ""