val q1 = spark.read.option("header", true).csv("../datasets/people.csv")
scala> println(q1.queryExecution.logical.numberedTreeString)
00 Relation[id#72,name#73,age#74] csv
val q2 = sql("select * from `csv`.`../datasets/people.csv`")
scala> println(q2.queryExecution.optimizedPlan.numberedTreeString)
00 Relation[_c0#175,_c1#176,_c2#177] csvLogicalRelation Logical Operator — Adapter for BaseRelation
LogicalRelation is a leaf logical operator that acts as an adapter for BaseRelation in a logical query plan.
LogicalRelation is created when:
-
DataFrameReaderloads data from a data source that supports multiple paths (through SparkSession.baseRelationToDataFrame) -
DataFrameReaderloads data from an external table using JDBC (through SparkSession.baseRelationToDataFrame) -
TextInputCSVDataSourceandTextInputJsonDataSourceare requested to infer schema -
ResolveSQLOnFileconverts a logical plan -
FindDataSourceTableconverts a logical plan -
RelationConversionsconverts a logical plan -
CreateTempViewUsingrunnable command is executed -
Structured Streaming’s
FileStreamSourcecreates batches of records
|
Note
|
|