// explode (that uses Generate operator) does not support codegen
val ids = Seq(Seq(0,1,2,3)).toDF("ids").select(explode($"ids") as "id")
val query = spark.range(1).join(ids, "id")
scala> query.explain
== Physical Plan ==
*Project [id#150L]
+- *BroadcastHashJoin [id#150L], [cast(id#147 as bigint)], Inner, BuildRight
:- *Range (0, 1, step=1, splits=8)
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)))
+- Generate explode(ids#143), false, false, [id#147]
+- LocalTableScan [ids#143]
InputAdapter Unary Physical Operator
InputAdapter
is a unary physical operator (with CodegenSupport) that is an adapter to generate code for the single child operator that does not support whole-stage code generation but participates in such code generation for a structured query.
InputAdapter
is created exclusively when CollapseCodegenStages
inserts a WholeStageCodegenExec operator into a physical plan.
InputAdapter
removes the star from a stringified tree representation of a physical plan (that WholeStageCodegenExec adds), e.g. for explain operator.
doProduce
Method
doProduce(ctx: CodegenContext): String
doProduce
generates a Java source code that consumes internal row of a single input RDD
one at a time (in a while
loop).
Note
|
doProduce supports one input RDD only (that the single child generates when executed).
|
Internally, doProduce
generates two terms input
and row
and uses the code from consume code generator.
Note
|
doProduce is a part of CodegenSupport Contract to generate a Java source code.
|