rootPool: Pool
Schedulable Builders
SchedulableBuilder
is a contract of schedulable builders that operate on a pool of TaskSetManagers (from an owning TaskSchedulerImpl).
Schedulable builders can build pools and add new Schedulable entities to the pool.
Note
|
A SchedulableBuilder is created when TaskSchedulerImpl is being initialized. You can select the SchedulableBuilder to use by spark.scheduler.mode setting.
|
Spark comes with two implementations of the SchedulableBuilder Contract:
-
FIFOSchedulableBuilder - the default
SchedulableBuilder
Note
|
SchedulableBuilder is a private[spark] Scala trait. You can find the sources in org.apache.spark.scheduler.SchedulableBuilder.
|
SchedulableBuilder Contract
Every SchedulableBuilder
provides the following services:
-
It manages a root pool.
-
It can build pools.
Root Pool (rootPool method)
rootPool
method returns a Pool (of Schedulables).
This is the data structure managed (aka wrapped) by SchedulableBuilders
.
Build Pools (buildPools method)
buildPools(): Unit
Note
|
It is exclusively called by TaskSchedulerImpl.initialize. |
Adding Schedulable (to Pool) (addTaskSetManager method)
addTaskSetManager(manager: Schedulable, properties: Properties): Unit
addTaskSetManager
registers the manager
Schedulable (with additional properties
) to the rootPool.
Note
|
addTaskSetManager is exclusively used by TaskSchedulerImpl to submit a TaskSetManager for a stage for execution.
|