TaskInfo

TaskInfo is information about a running task attempt inside a TaskSet.

TaskInfo is created when:

Note
 Back then, at the commit 63051dd2bcc4bf09d413ff7cf89a37967edc33ba, when TaskInfo was first merged to Apache Spark on 07/06/12, TaskInfo was part of spark.scheduler.mesos package — note "Mesos" in the name of the package that shows how much Spark and Mesos influenced each other at that time.
Table 1. TaskInfo’s Internal Registries and Counters
Name Description

finishTime

Time when TaskInfo was marked as finished.

Used when…​FIXME

Creating TaskInfo Instance

TaskInfo takes the following when created:

  • Task ID

  • Index of the task within its TaskSet that may not necessarily be the same as the ID of the RDD partition that the task is computing.

  • Task attempt ID

  • Time when the task was dequeued for execution

  • Executor that has been offered (as a resource) to run the task

  • Host of the executor

  • TaskLocality, i.e. locality preference of the task

  • Flag whether a task is speculative or not

TaskInfo initializes the internal registries and counters.

Marking Task As Finished (Successfully or Not) — markFinished Method

markFinished(state: TaskState, time: Long = System.currentTimeMillis): Unit

markFinished records the input time as finishTime.

markFinished marks TaskInfo as failed when the input state is FAILED or killed for state being KILLED.

Note
 markFinished is used when TaskSetManager is notified that a task has finished successfully or failed.

results matching ""

    No results matching ""