AddWebUIFilter(
filterName: String,
filterParams: Map[String, String],
proxyBase: String)
AMEndpoint — ApplicationMaster RPC Endpoint
onStart Callback
When onStart
is called, AMEndpoint
communicates with the driver (the driver
remote RPC Endpoint reference) by sending a one-way RegisterClusterManager
message with a reference to itself.
After RegisterClusterManager
has been sent (and received by YarnSchedulerEndpoint) the communication between the RPC endpoints of ApplicationMaster (YARN) and YarnSchedulerBackend (the Spark driver) is considered established.
RPC Messages
AddWebUIFilter
When AddWebUIFilter
arrives, you should see the following INFO message in the logs:
INFO ApplicationMaster$AMEndpoint: Add WebUI Filter. [addWebUIFilter]
It then passes the AddWebUIFilter
message on to the driver’s scheduler backend (through YarnScheduler RPC Endpoint).
RequestExecutors
RequestExecutors(
requestedTotal: Int,
localityAwareTasks: Int,
hostToLocalTaskCount: Map[String, Int])
When RequestExecutors
arrives, AMEndpoint
requests YarnAllocator
for executors given locality preferences.
If the requestedTotal
number of executors is different than the current number, resetAllocatorInterval is executed.
In case when YarnAllocator
is not available yet, you should see the following WARN message in the logs:
WARN Container allocator is not ready to request executors yet.
The response is false
then.
resetAllocatorInterval
When RequestExecutors message arrives, it calls resetAllocatorInterval
procedure.
resetAllocatorInterval(): Unit
resetAllocatorInterval
requests allocatorLock
monitor lock and sets the internal nextAllocationInterval
attribute to be initialAllocationInterval
internal attribute. It then wakes up all threads waiting on allocatorLock
.
Note
|
A thread waits on a monitor by calling one of the Object.wait methods.
|