Hadoop源码详解之Job 类

Hadoop源码详解之​​Job​​类

1. 源码

• • 
    
• 包：​​org.apache.hadoop.mapreduce​​
• 继承的接口有：​​AutoCloseable​​​，​​JobContext​​​，​​org.apache.hadoop.mapreduce.MRJobConfig​​
  • 
    

The job submitter’s view of the Job.
It allows the user to configure the job, submit it, control its execution, and query the state. The set methods only work until the job is submitted, afterwards they will throw an IllegalStateException.
Normally the user creates the application, describes various facets of the job via Job and then submits the job and monitor its progress.

作业提交者层次上的作业视图。

它允许用户配置job，提交它，并且控制它的运行，然后查询状态。​​set​​ 方法 仅仅工作直到job被提交，否则会抛出​​IllegalStateException​​。

正常情况下，用户创建一个应用，描述工作的各个方面，通过​​Job​​ 类，并且提交job，然后监测它的进度。

下面给出一个示例关于如何使用 ​​Job​​ 类去提交一个job。

// Create a new Job
     Job job = Job.getInstance();
     job.setJarByClass(MyJob.class);

     // Specify various job-specific parameters     
     job.setJobName("myjob");

     job.setInputPath(new Path("in"));
     job.setOutputPath(new Path("out"));

     job.setMapperClass(MyJob.MyMapper.class);
     job.setReducerClass(MyJob.MyReducer.class);

     // Submit the job, then poll for progress until the job is complete
     job.waitForCompletion(true);

2. 方法详解

2.1 构造器

以前的构造器全部建议不再使用，转而使用​​getInistance(...)​​这个方法。

2.2 ​​waitForCompletion​​

• 方法释义

Submit the job to the cluster and wait for it to finish.
提交job到集群，并且等待它完成。

• 方法源码

/**
   * @param verbose print the progress to the user
   * @return true if the job succeeded
   * @throws IOException thrown if the communication with the JobTracker is lost
   */
  public boolean waitForCompletion(boolean verbose
                                   ) throws IOException, InterruptedException,
                                            ClassNotFoundException {
    if (state == JobState.DEFINE) {
      submit();
    }
    if (verbose) {
      monitorAndPrintJob();
    } else {
      // get the completion poll interval from the client.
      int completionPollIntervalMillis = 
        Job.getCompletionPollInterval(cluster.getConf());
      while (!isComplete()) {
        try {
          Thread.sleep(completionPollIntervalMillis);
        } catch (InterruptedException ie) {
        }
      }
    }
    return isSuccessful();
  }