ShuffleWriter

Caution
FIXME

ShuffleWriter Contract

abstract class ShuffleWriter[K, V] {
  def write(records: Iterator[Product2[K, V]]): Unit
  def stop(success: Boolean): Option[MapStatus]
}
Note
ShuffleWriter is a private[spark] contract.
Table 1. ShuffleWriter Contract
Method Description

write

Writes a sequence of records (for a RDD partition) to a shuffle system when a ShuffleMapTask writes its execution result.

stop

Closes a ShuffleWriter and returns MapStatus if the writing completed successfully.

Used when a ShuffleMapTask finishes execution with the input success flag to match the status of the task execution.

results matching ""

    No results matching ""