Hadoop MapReduce 错误: Illegal partition for 1660,1998-01-10 (-16) 解决方法

修改代码中的错误提示可以更清晰地指出问题所在。根据当前的错误提示，可以看出是在map函数中写出了非法的输出分区。可能的原因是输出的key值不符合分区规则，导致无法正确分配到相应的分区。建议检查输出key值的格式和内容，确保符合分区规则。同时，可以查看分区函数的实现，以确保其正确性。另外，还可以考虑调整分区数，避免出现分区不足的情况。

示例代码

package com.mystudy.pxkDazuoye;

import java.io.IOException;

import javax.security.auth.login.Configuration;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Mapper.Context;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

import com.google.common.collect.Table.Cell;


public class mapper extends Mapper<LongWritable, Text, Text, Text> {
    // ...

    @Override
  
    	 protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
    	        if (key.get() > 0) { // 排除表头
    	            String[] fields = value.toString().split(',');
    	            String firstName = fields[1];
    	            String lastName = fields[2];
    	            String emailAddress = fields[3];
    	            Text outputKey = new Text(firstName + ',' + lastName);
    	            Text outputValue = new Text(fields[0] + ';' + firstName + ' ' + lastName + ';' + emailAddress);
    	            context.write(outputKey, outputValue);
    	        }
    	    }
    // ...
}
java.lang.Exception: java.io.IOException: Illegal partition for 1660,1998-01-10 (-16)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552)
Caused by: java.io.IOException: Illegal partition for 1660,1998-01-10 (-16)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1096)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:727)
	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
	at com.mystudy.pxkDazuoye.mapper.map(mapper.java:31)
	at com.mystudy.pxkDazuoye.mapper.map(mapper.java:18)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

解决方案

1. 检查输出key值的格式和内容，确保符合分区规则。
2. 查看分区函数的实现，以确保其正确性。
3. 调整分区数，避免出现分区不足的情况。