val dataset = Seq(1).toDF
scala> dataset.explain(true)
== Parsed Logical Plan ==
LocalRelation [value#216]
== Analyzed Logical Plan ==
value: int
LocalRelation [value#216]
== Optimized Logical Plan ==
LocalRelation [value#216]
== Physical Plan ==
LocalTableScan [value#216]
LocalRelation Logical Query Plan
LocalRelation
is a leaf logical plan that allow functions like collect
or take
to be executed locally, i.e. without using Spark executors.
Note
|
When Dataset operators can be executed locally, the Dataset is considered local.
|
LocalRelation
represents Datasets
that were created from local collections using SparkSession.emptyDataset or SparkSession.createDataset methods and their derivatives like toDF.
It can only be constructed with the output attributes being all resolved.
The size of the objects (in statistics
) is the sum of the default size of the attributes multiplied by the number of records.
When executed, LocalRelation
is translated to LocalTableScanExec physical operator.