JobListener

Spark subscribes for job completion or failure events (after submitting a job to DAGScheduler) using JobListener trait.

The following are the job listeners used:

  1. JobWaiter waits until DAGScheduler completes a job and passes the results of tasks to a resultHandler function.

  2. ApproximateActionListener…​FIXME

An instance of JobListener is used in the following places:

  • In ActiveJob as a listener to notify if tasks in this job finish or the job fails.

  • In JobSubmitted

JobListener Contract

JobListener is a private[spark] contract with the following two methods:

private[spark] trait JobListener {
  def taskSucceeded(index: Int, result: Any)
  def jobFailed(exception: Exception)
}

A JobListener object is notified each time a task succeeds (by taskSucceeded) and when the whole job fails (by jobFailed).

results matching ""

    No results matching ""