数据分类:将文本数据集拆分为训练集、验证集和测试集
import\ random\n\n#\ Read\ the\ dataset\ from\ the\ txt\ file\ndata\ =\ []\nwith\ open('dataset.txt',\ 'r')\ as\ file:\n\tfor\ line\ in\ file:\n\t\tdata.append(line.strip().split(','))\n\n#\ Split\ the\ data\ into\ train,\ validation\ and\ test\ sets\ for\ each\ class\ntrain_set\ =\ []\nvalidation_set\ =\ []\ntest_set\ =\ []\nclass_counts\ =\ [0]\ *\ 8\n\nfor\ sample\ in\ data:\n\tlabel\ =\ int(sample[-1])\n\tif\ class_counts[label]\ <\ 1000:\n\t\ttrain_set.append(sample)\n\t\tclass_counts[label]\ +=\ 1\n\telif\ class_counts[label]\ <\ 1100:\n\t\tvalidation_set.append(sample)\n\t\tclass_counts[label]\ +=\ 1\n\telif\ class_counts[label]\ <\ 1200:\n\t\ttest_set.append(sample)\n\t\tclass_counts[label]\ +=\ 1\n\n#\ Shuffle\ the\ train,\ validation\ and\ test\ sets\nrandom.shuffle(train_set)\nrandom.shuffle(validation_set)\nrandom.shuffle(test_set)\n\n#\ Save\ the\ train\ set\ to\ a\ txt\ file\nwith\ open('train_set.txt',\ 'w')\ as\ file:\n\tfor\ sample\ in\ train_set:\n\t\tfile.write(','.join(sample)\ +\ '\n')\n\n#\ Save\ the\ validation\ set\ to\ a\ txt\ file\nwith\ open('validation_set.txt',\ 'w')\ as\ file:\n\tfor\ sample\ in\ validation_set:\n\t\tfile.write(','.join(sample)\ +\ '\n')\n\n#\ Save\ the\ test\ set\ to\ a\ txt\ file\nwith\ open('test_set.txt',\ 'w')\ as\ file:\n\tfor\ sample\ in\ test_set:\n\t\tfile.write(','.join(sample)\ +\ '\n')
原文地址: https://www.cveoy.top/t/topic/jN1h 著作权归作者所有。请勿转载和采集!