【自学小模型day1】——验证小模型

文章目录

• 进入yolo环境
• 现有资源
• • 如何使用这些.pt文件
• 验证小模型对数据集的分割效果——predict
• 验证小模型对数据集的分割效果——val
• • 现有资源
  • 前期工作
  • 开始验证
  • 结果分析
  • • 先算四种像素统计量
  • 异常数据

进入yolo环境

source~/yolov8_env/bin/activate

现有资源

• 同事训练的小模型

• 同事提供的数据集

• ubuntu20.04+搭建yolo运行环境

• best.pt epoch10.pt epoch20.pt epoch30.pt epoch40.pt epoch5.pt epoch0.pt epoch15.pt epoch25.pt epoch35.pt epoch45.pt last.pt

• best.pt 是什么
  表示：在整个训练过程中，在验证集上表现最好的那个模型。通常 Ultralytics 会根据验证指标来判断“最好”，对分割任务一般重点看验证集上的综合指标，比如 mAP。
  所以：它不一定是最后一次训练得到的，但通常是最推荐拿来推理 / 验证 / 部署的

• last.pt 是什么
  表示：训练结束时最后一个 epoch 的模型。
  也就是说，无论最后效果是不是最好，训练停下来的那一刻保存的就是 last.pt。
  它适合：继续训练（resume），看最终训练停在哪个状态，和 best.pt 做比较

• epoch0.pt / epoch5.pt / epoch10.pt … 是什么
  这些就是：训练过程中某些特定 epoch 的检查点（checkpoint）。
  比如：
  epoch0.pt：第 0 轮训练后保存
  epoch5.pt：第 5 轮后保存
  epoch10.pt：第 10 轮后保存
  …
  epoch45.pt：第 45 轮后保存

如何使用这些.pt文件

• 看效果 / 做推理：优先用 best.pt 怀疑后期过拟合
• 想对比：拿best.pt、last.pt、epoch20.pt、epoch45.pt 分别试
• 想继续训练：一般用 last.pt 或某个epochXX.pt
• `yolo task=segment mode=val \

验证小模型对数据集的分割效果——predict

• 【得到每张图片的分割效果图】

yolotask=segmentmode=predict\model=/home/xxx/AI/segment/train/weights/best.pt\source=/home/xxx/AI/dataset\save=True

• 观察以下文件夹里的图像的分割是否合理

/home/xxx/AI/dataset/runs

验证小模型对数据集的分割效果——val

现有资源

• 同事的数据集和对应的只有二值的png图片

• bianjie1_1.png 和 ./label/bianjie1_1.png

• 经过验证， ./label/bianjie1_1.png是二值mask图片，只有0和1，其中1代表grass

前期工作

• 需要数据集配置文件 xxx.yaml文件，配置yaml文件中的# Classes 和 path 和 train:
• 需要已经手动标定的mask和被验证的数据集
• 因为mask不是标准的yolo格式，所以需要自己写代码进行验证

开始验证

frompathlibimportPathimportnumpyasnpfromPILimportImagefromultralyticsimportYOLO# =========================# Config# =========================MODEL_PATH="/home/xxx/AI/segment/train/weights/best.pt"IMAGE_DIR=Path("/home/xxx/AI/dataset/bianjie1")LABEL_DIR=IMAGE_DIR/"label"IMAGE_SUFFIX=".png"# According to your model names:# 0: grassGRASS_CLASS_ID=0# Prediction confidence thresholdCONF_THRES=0.25# =========================# Helpers# =========================defload_gt_mask(label_path:Path)->np.ndarray:""" Ground truth: 1 = grass 0 = non-grass Returns boolean mask of shape (H, W), True for grass. """gt=np.array(Image.open(label_path))ifgt.ndim!=2:raiseValueError(f"GT mask is not single-channel:{label_path}")returngt==1defresize_bool_mask(mask:np.ndarray,target_hw:tuple[int,int])->np.ndarray:""" Resize boolean mask to target size using nearest-neighbor. target_hw = (H, W) """h,w=target_hw img=Image.fromarray(mask.astype(np.uint8)*255)img=img.resize((w,h),Image.NEAREST)arr=np.array(img)returnarr>0defbuild_pred_grass_mask(result,target_hw:tuple[int,int])->np.ndarray:""" From one Ultralytics result, merge all predicted masks with class == GRASS_CLASS_ID into one binary mask. Returns boolean mask of shape target_hw. """h,w=target_hw pred_mask=np.zeros((h,w),dtype=bool)ifresult.masksisNoneorresult.boxesisNone:returnpred_mask cls_arr=result.boxes.cls.cpu().numpy().astype(int)conf_arr=result.boxes.conf.cpu().numpy()masks_arr=result.masks.data.cpu().numpy()# shape: [N, Hm, Wm]fori,(cls_id,conf)inenumerate(zip(cls_arr,conf_arr)):ifcls_id!=GRASS_CLASS_ID:continueifconf<CONF_THRES:continuem=masks_arr[i]>0.5ifm.shape!=(h,w):m=resize_bool_mask(m,(h,w))pred_mask|=mreturnpred_maskdefcalc_metrics(pred:np.ndarray,gt:np.ndarray)->dict:""" pred, gt are boolean masks of same shape """tp=np.logical_and(pred,gt).sum()fp=np.logical_and(pred,~gt).sum()fn=np.logical_and(~pred,gt).sum()tn=np.logical_and(~pred,~gt).sum()precision=tp/(tp+fp)if(tp+fp)>0else0.0recall=tp/(tp+fn)if(tp+fn)>0else0.0iou=tp/(tp+fp+fn)if(tp+fp+fn)>0else0.0f1=2*precision*recall/(precision+recall)if(precision+recall)>0else0.0return{"tp":int(tp),"fp":int(fp),"fn":int(fn),"tn":int(tn),"precision":float(precision),"recall":float(recall),"iou":float(iou),"f1":float(f1),}# =========================# Main# =========================defmain():model=YOLO(MODEL_PATH)image_paths=sorted([pforpinIMAGE_DIR.glob(f"*{IMAGE_SUFFIX}")ifp.is_file()])ifnotimage_paths:raiseFileNotFoundError(f"No images found in{IMAGE_DIR}")total_tp=total_fp=total_fn=total_tn=0per_image_metrics=[]print(f"Found{len(image_paths)}images.")print(f"Using model:{MODEL_PATH}")print(f"Evaluating grass class id ={GRASS_CLASS_ID}")print("-"*80)foridx,img_pathinenumerate(image_paths,start=1):label_path=LABEL_DIR/img_path.nameifnotlabel_path.exists():print(f"[WARN] Missing label for{img_path.name}, skip.")continue# Load GTgt_mask=load_gt_mask(label_path)h,w=gt_mask.shape# Predict one imageresults=model.predict(source=str(img_path),task="segment",conf=CONF_THRES,verbose=False,save=False)result=results[0]pred_mask=build_pred_grass_mask(result,(h,w))m=calc_metrics(pred_mask,gt_mask)total_tp+=m["tp"]total_fp+=m["fp"]total_fn+=m["fn"]total_tn+=m["tn"]per_image_metrics.append(m)print(f"[{idx:03d}/{len(image_paths)}]{img_path.name}| "f"IoU={m['iou']:.4f}, P={m['precision']:.4f}, "f"R={m['recall']:.4f}, F1={m['f1']:.4f}")print("-"*80)# Global metrics from total pixelsglobal_precision=total_tp/(total_tp+total_fp)if(total_tp+total_fp)>0else0.0global_recall=total_tp/(total_tp+total_fn)if(total_tp+total_fn)>0else0.0global_iou=total_tp/(total_tp+total_fp+total_fn)if(total_tp+total_fp+total_fn)>0else0.0global_f1=(2*global_precision*global_recall/(global_precision+global_recall)if(global_precision+global_recall)>0else0.0)# Mean per-image metricsifper_image_metrics:mean_iou=np.mean([x["iou"]forxinper_image_metrics])mean_precision=np.mean([x["precision"]forxinper_image_metrics])mean_recall=np.mean([x["recall"]forxinper_image_metrics])mean_f1=np.mean([x["f1"]forxinper_image_metrics])else:mean_iou=mean_precision=mean_recall=mean_f1=0.0print("Global pixel-level metrics:")print(f" IoU :{global_iou:.6f}")print(f" Precision:{global_precision:.6f}")print(f" Recall :{global_recall:.6f}")print(f" F1 :{global_f1:.6f}")print("\nMean per-image metrics:")print(f" IoU :{mean_iou:.6f}")print(f" Precision:{mean_precision:.6f}")print(f" Recall :{mean_recall:.6f}")print(f" F1 :{mean_f1:.6f}")print("\nConfusion totals:")print(f" TP:{total_tp}")print(f" FP:{total_fp}")print(f" FN:{total_fn}")print(f" TN:{total_tn}")if__name__=="__main__":main()

• 验证结果

[001/293] bianjie1_1.png | IoU=0.9418, P=0.9950, R=0.9462, F1=0.9700
[002/293] bianjie1_10.png | IoU=0.9825, P=0.9977, R=0.9847, F1=0.9912
[003/293] bianjie1_100.png | IoU=0.9597, P=0.9735, R=0.9854, F1=0.9794
[004/293] bianjie1_101.png | IoU=0.9663, P=0.9958, R=0.9703, F1=0.9829
[005/293] bianjie1_102.png | IoU=0.9739, P=0.9952, R=0.9786, F1=0.9868
[006/293] bianjie1_103.png | IoU=0.9765, P=0.9879, R=0.9883, F1=0.9881
[007/293] bianjie1_104.png | IoU=0.0000, P=0.0000, R=0.0000, F1=0.0000
[008/293] bianjie1_105.png | IoU=0.0000, P=0.0000, R=0.0000, F1=0.0000
[009/293] bianjie1_106.png | IoU=0.9671, P=0.9964, R=0.9705, F1=0.9833
[010/293] bianjie1_107.png | IoU=0.9510, P=0.9717, R=0.9781, F1=0.9749
[011/293] bianjie1_108.png | IoU=0.9736, P=0.9927, R=0.9807, F1=0.9866
[012/293] bianjie1_109.png | IoU=0.9758, P=0.9952, R=0.9804, F1=0.9878
[013/293] bianjie1_11.png | IoU=0.9739, P=0.9908, R=0.9828, F1=0.9868
[014/293] bianjie1_110.png | IoU=0.9639, P=0.9987, R=0.9650, F1=0.9816
[015/293] bianjie1_111.png | IoU=0.9778, P=0.9934, R=0.9842, F1=0.9888
[016/293] bianjie1_112.png | IoU=0.9739, P=0.9866, R=0.9870, F1=0.9868
[017/293] bianjie1_113.png | IoU=0.9653, P=0.9977, R=0.9675, F1=0.9823
[018/293] bianjie1_114.png | IoU=0.9302, P=0.9880, R=0.9408, F1=0.9638
[019/293] bianjie1_115.png | IoU=0.9703, P=0.9941, R=0.9760, F1=0.9849
[020/293] bianjie1_116.png | IoU=0.9699, P=0.9834, R=0.9860, F1=0.9847
[021/293] bianjie1_117.png | IoU=0.9652, P=0.9913, R=0.9735, F1=0.9823
[022/293] bianjie1_118.png | IoU=0.0000, P=0.0000, R=0.0000, F1=0.0000
[023/293] bianjie1_119.png | IoU=0.9754, P=0.9989, R=0.9765, F1=0.9876
[024/293] bianjie1_12.png | IoU=0.9714, P=0.9869, R=0.9841, F1=0.9855
[025/293] bianjie1_120.png | IoU=0.9494, P=0.9801, R=0.9680, F1=0.9740
[026/293] bianjie1_121.png | IoU=0.0000, P=0.0000, R=0.0000, F1=0.0000
[027/293] bianjie1_122.png | IoU=0.9655, P=0.9973, R=0.9681, F1=0.9825
[028/293] bianjie1_123.png | IoU=0.9679, P=0.9801, R=0.9873, F1=0.9837
[029/293] bianjie1_124.png | IoU=0.9737, P=0.9959, R=0.9776, F1=0.9867
[030/293] bianjie1_125.png | IoU=0.9533, P=0.9978, R=0.9553, F1=0.9761
[031/293] bianjie1_126.png | IoU=0.9562, P=0.9882, R=0.9673, F1=0.9776
[032/293] bianjie1_127.png | IoU=0.9598, P=0.9739, R=0.9851, F1=0.9795
[033/293] bianjie1_128.png | IoU=0.9472, P=0.9923, R=0.9542, F1=0.9729
[034/293] bianjie1_129.png | IoU=0.9744, P=0.9973, R=0.9770, F1=0.9870
[035/293] bianjie1_13.png | IoU=0.9716, P=0.9990, R=0.9725, F1=0.9856
[036/293] bianjie1_130.png | IoU=0.9639, P=0.9772, R=0.9860, F1=0.9816
[037/293] bianjie1_131.png | IoU=0.9761, P=0.9978, R=0.9782, F1=0.9879
[038/293] bianjie1_132.png | IoU=0.0000, P=0.0000, R=0.0000, F1=0.0000
[039/293] bianjie1_133.png | IoU=0.9491, P=0.9628, R=0.9852, F1=0.9739
[040/293] bianjie1_134.png | IoU=0.9550, P=0.9987, R=0.9562, F1=0.9770
[041/293] bianjie1_135.png | IoU=0.9732, P=0.9941, R=0.9788, F1=0.9864
[042/293] bianjie1_136.png | IoU=0.9482, P=0.9990, R=0.9491, F1=0.9734
[043/293] bianjie1_137.png | IoU=0.0000, P=0.0000, R=0.0000, F1=0.0000
[044/293] bianjie1_138.png | IoU=0.9606, P=0.9996, R=0.9610, F1=0.9799
[045/293] bianjie1_139.png | IoU=0.9667, P=0.9985, R=0.9681, F1=0.9831
[046/293] bianjie1_14.png | IoU=0.9373, P=0.9846, R=0.9513, F1=0.9677
[047/293] bianjie1_140.png | IoU=0.9795, P=0.9940, R=0.9853, F1=0.9896
[048/293] bianjie1_141.png | IoU=0.9625, P=0.9986, R=0.9639, F1=0.9809
[049/293] bianjie1_142.png | IoU=0.9702, P=0.9956, R=0.9744, F1=0.9849
[050/293] bianjie1_143.png | IoU=0.9797, P=0.9944, R=0.9851, F1=0.9897

结果分析

先算四种像素统计量

tp=np.logical_and(pred,gt).sum()fp=np.logical_and(pred,~gt).sum()fn=np.logical_and(~pred,gt).sum()tn=np.logical_and(~pred,~gt).sum()

• 含义：

TP：预测是 grass，真值也是 grass
FP：预测是 grass，但真值不是 grass
FN：预测不是 grass，但真值是 grass
TN：预测不是 grass，真值也不是 grass

• Precision

precision = tp / (tp + fp)
模型预测成 grass 的地方，有多少是真的 grass。

• Recall

recall = tp / (tp + fn)
真值里的 grass，有多少被模型找到了。

• IoU

iou = tp / (tp + fp + fn)
预测区域和真值区域的交并比。

• F1

f1 = 2 * precision * recall / (precision + recall)
Precision 和 Recall 的综合指标。

异常数据

发现在黄草区域的草的识别有问题，尤其是光线不好的时候，会和附近的暗灰色水泥地混合识别成路面，后续需要改善