NewHadoopRDD

NewHadoopRDD is an RDD of K keys and V values.

NewHadoopRDD is created when:

  • SparkContext.newAPIHadoopFile

  • SparkContext.newAPIHadoopRDD

  • (indirectly) SparkContext.binaryFiles

  • (indirectly) SparkContext.wholeTextFiles

Note
 NewHadoopRDD is the base RDD of BinaryFileRDD and WholeTextFileRDD.

getPreferredLocations Method

Caution
 FIXME

Creating NewHadoopRDD Instance

NewHadoopRDD takes the following when created:

  • SparkContext

  • HDFS' InputFormat[K, V]

  • K class name

  • V class name

  • transient HDFS' Configuration

NewHadoopRDD initializes the internal registries and counters.

results matching ""

    No results matching ""