ResultStage
— Final Stage in Job
A ResultStage
is the final stage in a job that applies a function on one or many partitions of the target RDD to compute the result of an action.
Figure 1. Job creates ResultStage as the first stage
The partitions are given as a collection of partition ids (partitions
) and the function func: (TaskContext, Iterator[_]) ⇒ _
.
Figure 2.
ResultStage
and partitions
Tip
|
Read about TaskContext in TaskContext.
|
func
Property
Caution
|
FIXME |
setActiveJob
Method
Caution
|
FIXME |
removeActiveJob
Method
Caution
|
FIXME |
activeJob
Method
activeJob: Option[ActiveJob]
activeJob
returns the optional ActiveJob
associated with a ResultStage
.
Caution
|
FIXME When/why would that be NONE (empty)?
|