private[spark] trait JobListener {
def taskSucceeded(index: Int, result: Any)
def jobFailed(exception: Exception)
}
JobListener
Spark subscribes for job completion or failure events (after submitting a job to DAGScheduler) using JobListener
trait.
The following are the job listeners used:
-
JobWaiter waits until DAGScheduler completes a job and passes the results of tasks to a
resultHandler
function. -
ApproximateActionListener
…FIXME
An instance of JobListener
is used in the following places:
-
In
ActiveJob
as a listener to notify if tasks in this job finish or the job fails. -
In
JobSubmitted
JobListener
Contract
JobListener
is a private[spark]
contract with the following two methods:
A JobListener
object is notified each time a task succeeds (by taskSucceeded
) and when the whole job fails (by jobFailed
).