ChatGLM2-6B 训练调试报错: SchemaInferenceError 和 DatasetGenerationError

在使用 ChatGLM2-6B 进行训练时，可能会遇到以下错误：/n/n/nTraceback (most recent call last):/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//builder.py/', line 1949, in _prepare_split_single/n num_examples, num_bytes = writer.finalize()/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//arrow_writer.py/', line 600, in finalize/n raise SchemaInferenceError('Please pass `features` or at least one example when writing data')/ndatasets.arrow_writer.SchemaInferenceError: Please pass `features` or at least one example when writing data/n/nThe above exception was the direct cause of the following exception:/n/nTraceback (most recent call last):/n File /'D://work//test//ChatGLM2-6B//ptuning//main.py/', line 411, in <module>/n main()/n File /'D://work//test//ChatGLM2-6B//ptuning//main.py/', line 98, in main/n raw_datasets = load_dataset(/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//load.py/', line 2133, in load_dataset/n builder_instance.download_and_prepare(/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//builder.py/', line 954, in download_and_prepare/n self._download_and_prepare(/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//builder.py/', line 1049, in _download_and_prepare/n self._prepare_split(split_generator, **prepare_split_kwargs)/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//builder.py/', line 1813, in _prepare_split/n for job_id, done, content in self._prepare_split_single(/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//datasets//builder.py/', line 1958, in _prepare_split_single/n raise DatasetGenerationError('An error occurred while generating the dataset') from e/ndatasets.builder.DatasetGenerationError: An error occurred while generating the dataset/nERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 11320) of binary: C:/Users//Admin//AppData//Local//Programs//Python//Python310//python.exe/nTraceback (most recent call last):/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//runpy.py/', line 196, in _run_module_as_main/n return _run_code(code, main_globals, None,/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//runpy.py/', line 86, in _run_code/n exec(code, run_globals)/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//Scripts//torchrun.exe//__main__.py/', line 7, in <module>/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//torch//distributed//elastic//multiprocessing//errors//__init__.py/', line 346, in wrapper/n return f(*args, **kwargs)/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//torch//distributed//run.py/', line 794, in main/n run(args)/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//torch//distributed//run.py/', line 785, in run/n elastic_launch(/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//torch//distributed//launcher//api.py/', line 134, in __call__/n return launch_agent(self._config, self._entrypoint, list(args))/n File /'C://Users//Admin//AppData//Local//Programs//Python//Python310//lib//site-packages//torch//distributed//launcher//api.py/', line 250, in launch_agent/n raise ChildFailedError(/ntorch.distributed.elastic.multiprocessing.errors.ChildFailedError:/n/n/n这个错误是由于在生成数据集时缺少'features'或至少一个示例导致的。具体来说，数据集生成器在写入数据时需要提供'features'参数或至少一个示例。您需要检查代码中生成数据集的部分，确保提供了正确的'features'参数或至少一个示例。/n/n另外，此错误还导致了分布式训练中的一个进程退出。您可能还需要检查分布式训练的设置和环境，确保配置正确并且环境准备就绪。/n/n如果您需要更详细的帮助，请提供相关的代码片段和更多上下文信息。/n