Jobs Tab
The Jobs Tab shows status of all Spark jobs in a Spark application (i.e. a SparkContext).
The Jobs tab is available under /jobs
URL, i.e. http://localhost:4040/jobs.
The Jobs tab consists of two pages, i.e. All Jobs and Details for Job pages.
Internally, the Jobs Tab is represented by JobsTab
class that is a custom SparkUITab with jobs
prefix.
Note
|
The Jobs tab uses JobProgressListener to access statistics of job executions in a Spark application to display. |
Showing All Jobs — AllJobsPage
Page
AllJobsPage
is a page (in Jobs
tab) that renders a summary, an event timeline, and active, completed, and failed jobs of a Spark application.
Tip
|
Jobs (in any state) are displayed when their number is greater than 0 .
|
AllJobsPage
displays the Summary section with the current Spark user, total uptime, scheduling mode, and the number of jobs per status.
Note
|
AllJobsPage uses JobProgressListener for Scheduling Mode .
|
Under the summary section is the Event Timeline section.
Note
|
AllJobsPage uses ExecutorsListener to build the event timeline.
|
Active Jobs, Completed Jobs, and Failed Jobs sections follow.
Jobs are clickable, i.e. you can click on a job to see information about the stages of tasks inside it.
When you hover over a job in Event Timeline not only you see the job legend but also the job is highlighted in the Summary section.
The Event Timeline section shows not only jobs but also executors.
Tip
|
Use Programmable Dynamic Allocation (using SparkContext ) to manage executors for demo purposes.
|
Details for Job — JobPage
Page
When you click a job in AllJobsPage page, you see the Details for Job page.
JobPage
is a custom WebUIPage
that shows statistics and stage list for a given job.
Details for Job page is registered under /job
URL, i.e. http://localhost:4040/jobs/job/?id=0
and accepts one mandatory id
request parameter as a job identifier.
When a job id is not found, you should see "No information to display for job ID" message.
JobPage
displays the job’s status, group (if available), and the stages per state: active, pending, completed, skipped, and failed.
Note
|
A job can be in a running, succeeded, failed or unknown state. |