BlockStatusListener Spark Listener

BlockStatusListener is a SparkListener that tracks BlockManagers and the blocks for Storage tab in web UI.

Table 1. BlockStatusListener Registries
Registry Description

blockManagers

The lookup table for a collection of BlockId and BlockUIData per BlockManagerId.
Caution
 FIXME When are the events posted?
Table 2. BlockStatusListener Event Handlers
Event Handler Description

onBlockManagerAdded

Registers a BlockManager in blockManagers internal registry (with no blocks).

onBlockManagerRemoved

Removes a BlockManager from blockManagers internal registry.

onBlockUpdated

Puts an updated BlockUIData for BlockId for BlockManagerId in blockManagers internal registry.

Ignores updates for unregistered BlockManagers or non-StreamBlockIds.

For invalid StorageLevels (i.e. they do not use a memory or a disk or no replication) the block is removed.

results matching ""

    No results matching ""