bytesWritten: Long
ShuffleWriteMetrics
ShuffleWriteMetrics
is a collection of accumulators that represents task metrics about writing shuffle data.
ShuffleWriteMetrics
tracks the following task metrics:
Note
|
Accumulators allow tasks (running on executors) to communicate with the driver. |
Name | Description |
---|---|
Accumulator to track how many shuffle bytes were written in a shuffle task. Used when NOTE: |
|
Accumulator to track shuffle write time (as 64-bit integer) of a shuffle task. Used when NOTE: |
|
Accumulator to track how many shuffle records were written in a shuffle task. Used when NOTE: |
decRecordsWritten
Method
Caution
|
FIXME |
decBytesWritten
Method
Caution
|
FIXME |
writeTime
Method
Caution
|
FIXME |
recordsWritten
Method
Caution
|
FIXME |
Returning Number of Shuffle Bytes Written — bytesWritten
Method
bytesWritten
represents the shuffle bytes written metrics of a shuffle task.
Internally, bytesWritten
returns the sum of _bytesWritten internal accumulator.
Note
|
|
Incrementing Shuffle Bytes Written Metrics — incBytesWritten
Method
incBytesWritten(v: Long): Unit
incBytesWritten
simply adds v
to _bytesWritten internal accumulator.
Note
|
|
Incrementing Shuffle Write Time Metrics — incWriteTime
Method
incWriteTime(v: Long): Unit
incWriteTime
simply adds v
to _writeTime internal accumulator.
Note
|
|
Incrementing Shuffle Records Written Metrics — incRecordsWritten
Method
incRecordsWritten(v: Long): Unit
incRecordsWritten
simply adds v
to _recordsWritten internal accumulator.
Note
|
|