ShuffleClient

ShuffleClient is an interface (abstract class) for reading shuffle files.

ShuffleClient Contract

Every ShuffleClient can do the following:

  • It can be init. The default implementation does nothing by default.

    public void init(String appId)
  • fetchBlocks fetches a sequence of blocks from a remote node asynchronously.

    public abstract void fetchBlocks(
      String host,
      int port,
      String execId,
      String[] blockIds,
      BlockFetchingListener listener);

ExternalShuffleClient

Caution
FIXME

Register Block Manager with Shuffle Server (registerWithShuffleServer method)

Caution
FIXME

results matching ""

    No results matching ""