BasicOperators Execution Planning Strategy

BasicOperators is an execution planning strategy (of SparkPlanner) that in general does simple conversions from logical operators to their physical counterparts.

Table 1. BasicOperators' Logical to Physical Operator Conversions
Logical Operator Physical Operator

RunnableCommand

ExecutedCommandExec

MemoryPlan

LocalTableScanExec

DeserializeToObject

DeserializeToObjectExec

SerializeFromObject

SerializeFromObjectExec

MapPartitions

MapPartitionsExec

MapElements

MapElementsExec

AppendColumns

AppendColumnsExec

AppendColumnsWithObject

AppendColumnsWithObjectExec

MapGroups

MapGroupsExec

CoGroup

CoGroupExec

Repartition (with shuffle enabled)

ShuffleExchange

Repartition

CoalesceExec

SortPartitions

SortExec

Sort

SortExec

Project

ProjectExec

Filter

FilterExec

TypedFilter

FilterExec

Expand

ExpandExec

Window

WindowExec

Sample

SampleExec

LocalRelation

LocalTableScanExec

LocalLimit

LocalLimitExec

GlobalLimit

GlobalLimitExec

Union

UnionExec

Generate

GenerateExec

OneRowRelation

RDDScanExec

Range

RangeExec

RepartitionByExpression

ShuffleExchange

ExternalRDD

ExternalRDDScanExec

LogicalRDD

RDDScanExec

BroadcastHint

PlanLater

Tip
Confirm the operator mapping in the source code of BasicOperators.
Note
BasicOperators expects that Distinct, Intersect, and Except logical operators are not used in a logical plan and throws a IllegalStateException if not.

results matching ""

    No results matching ""