Pārlūkot izejas kodu

删除 backend/model_assistant_generalized:逻辑与文档已迁入 models,清理陈旧引用

18922397810 3 nedēļas atpakaļ
vecāks
revīzija
5bd3872d7d

+ 0 - 1
.gitignore

@@ -16,7 +16,6 @@ backend/LabelingPreTraining/inputdatas/
 *.log
 .backup_old_*/
 cache/
-backend/model_assistant_generalized/datasets_*/
 # 生成的模型数据(可由 python models/... 复现,不入库)
 models/datas/hourly/
 models/datas/pressure/

+ 0 - 73
backend/model_assistant_generalized/README.md

@@ -1,73 +0,0 @@
-# PKS 泛化模型(7/8/9 号机,机组相对特征)
-
-本目录是 7/8/9 号机**压力**与**振动**的泛化模型,两者**互相独立**:分别建特征、分别训练、分别评估,互不影响。
-
-核心思路:绝对油压/振动值各机组不同,直接混合训练会把一台机组的正常水平当成另一台机组的异常。因此去掉绝对特征,改用机组相对(泛化)特征:
-
-- **压力**:窗口内相对 + 相对本机周期基线(周期前 3 天)+ 相对近期 24h 基线 + 回撤/连续下降 + 机组 z 分数。
-- **振动**:相对本机近期 7×24 小时稳健基线(与扫描器口径一致,跨停机周期取历史)的偏离与比例 + 机组 z 分数 + 报警比例。
-- 机组基准(`*_z` 的 median/MAD)只用 train 划分的正常窗口计算,存 `models/*_unit_baselines.json`;每台机组在 train 中必须有正常事件组,否则报错。
-
-详见 `RESULTS_789_GENERALIZED.md`(压力)、`RESULTS_789_VIBRATION.md`(振动)、`跨机组泛化与样本扩充方案.md`(方案)、`实时联合诊断方案.md`(实时接入)。
-
-## 目录
-
-- `build_datasets.py`:样本构建,支持 `--window-hours`、`--model-types`、`--units`、`--hourly-csv`(复用中间表不查库)。
-- `split_datasets.py`:按事件组切分,支持 `--split-ratios`、`--seed`、`--positive-basis`。
-- `generalized_features.py`:压力泛化特征。
-- `generalized_features_vibration.py`:振动泛化特征。
-- `train_model.py`:训练并保存机组基准(按 `model_type` 选择特征模块)。
-- `evaluate_model.py`:留出测试,输出两类指标与分机组误报率。
-- `infer_and_combine.py`:规则联合辅助输出。
-- `data/hourly.csv`:7/8/9 小时级中间表(含油压与振动列),供 `--hourly-csv` 复用。
-- `datasets_789/`、`models_789/`:压力 3h;`datasets_789_6h/`、`models_789_6h/`:压力 6h 对照。
-- `datasets_vibration_789/`、`models_vibration_789/`:振动 3h;`datasets_vibration_789_6h/`、`models_vibration_789_6h/`:振动 6h 对照。
-
-## 压力复现
-
-```bash
-# 1. 重建 3h 样本(重叠即异常,复用中间表不查库)
-python backend/model_assistant_generalized/build_datasets.py \
-  --window-hours 3 --model-types pressure \
-  --hourly-csv backend/model_assistant_generalized/data/hourly.csv \
-  --output-dir backend/model_assistant_generalized/datasets_789
-
-# 2. 按事件组切分(6.5/2/1.5,固定种子)
-python backend/model_assistant_generalized/split_datasets.py \
-  --datasets-dir backend/model_assistant_generalized/datasets_789 \
-  --model-types pressure --split-ratios 6.5 2 1.5 --seed 20260921
-
-# 3. 训练
-python backend/model_assistant_generalized/train_model.py pressure \
-  --datasets-dir backend/model_assistant_generalized/datasets_789 \
-  --models-dir backend/model_assistant_generalized/models_789 \
-  --all-normal --rounds 800 --early-stopping-rounds 50
-
-# 4. 封存测试
-python backend/model_assistant_generalized/evaluate_model.py pressure \
-  --datasets-dir backend/model_assistant_generalized/datasets_789 \
-  --models-dir backend/model_assistant_generalized/models_789 \
-  --output-dir cache/pks_scanner/model_assistant_generalized_789
-
-# 5. 规则联合辅助输出
-python backend/model_assistant_generalized/infer_and_combine.py pressure \
-  --datasets-dir backend/model_assistant_generalized/datasets_789 \
-  --models-dir backend/model_assistant_generalized/models_789 \
-  --output-dir cache/pks_scanner/model_assistant_generalized_789
-```
-
-## 振动复现
-
-振动与压力流程相同,把 `--model-types pressure` 换成 `vibration`,目录换成 `datasets_vibration_789` / `models_vibration_789`,输出目录换成 `cache/pks_scanner/model_assistant_generalized_vibration_789`。切分时振动建议加 `--positive-basis groups`(每个异常事件等权,避免个别长事件整组落入同一划分)。
-
-## 6 小时窗口对照
-
-把上面的 `datasets_*`、`models_*` 换成 `_6h` 版本,并在 `build_datasets.py` 传 `--window-hours 6` 即可。
-
-## 压力结果摘要(3h)
-
-测试 2,253 窗口:异常查准率 95.12%、召回率 98.11%、F1 96.59%、PR-AUC 99.61%;正常类 F1 99.74%;事件召回 10/10。分机组正常误报率:7 号机 0.32%、8 号机 1.13%、9 号机 0.00%。
-
-## 振动结果摘要(3h)
-
-测试 2,180 窗口:异常查准率 71.43%、召回率 71.43%、F1 71.43%、PR-AUC 82.70%;正常类 F1 99.72%;事件召回 4/4。分机组正常误报率:7 号机 0.00%、8 号机 0.71%、9 号机 0.13%。

+ 0 - 131
backend/model_assistant_generalized/RESULTS_789_GENERALIZED.md

@@ -1,131 +0,0 @@
-# 7/8/9 号机压力泛化模型结果(机组相对 + 周期相对特征)
-
-本实验在 `backend/model_assistant_generalized/` 下独立进行,不改动原 `backend/model_assistant/` 的算法、模型与结果。
-
-目标:把 7/8/9 号机数据全部纳入,用**机组相对特征**训练共享压力模型,解决原全量模型“7 号机正常窗口被大量误报”的跨机组问题。
-
-当前版本:`generalized_cycle_relative_v2`(在 v1 机组相对特征基础上,加入扫描器口径的周期/近期相对与趋势形态特征)。
-
-## 一、方法
-
-### 1. 特征演进
-
-**v1(已弃用)**:去掉绝对油压,只用窗口内相对 + 机组归一化(`oil_*_z`)。
-
-**v2(当前)**:v1 基础上加入周期/近期相对特征,共 22 维:
-
-- 窗口内相对:`oil_last_relative_to_window`、`oil_range_rel`、`oil_slope_rel`、`oil_min_dev`、`oil_max_dev`、`oil_low_ratio_mean`、`oil_low_low_ratio_mean`、`running_coverage_mean`、`running_coverage_min`
-- **周期/近期相对(新增)**:`oil_rel_cycle_mean/min/last`(窗口油压 / 周期基线 − 1)、`oil_rel_recent_mean/last`(窗口油压 / 近 24h 基线 − 1)、`oil_slope_rel_cycle`、`oil_decline_hours`、`oil_drawdown_cycle`、`oil_range_rel_cycle`
-- 机组归一化:`oil_mean_z`、`oil_min_z`、`oil_last_z`、`oil_slope_z`
-
-周期基线按扫描器口径计算:周期**前 3 个自然日**每日油压中位数,再取其中位数;近期基线用窗口前 24 个有效小时的稳健基线。二者都只用窗口之前/周期开始的数据,无泄漏。
-
-特征重要度(3h v2)显示周期相对特征主导:
-
-| 特征 | 重要度 |
-|---|---:|
-| oil_rel_cycle_min | 46.77% |
-| oil_drawdown_cycle | 22.66% |
-| oil_rel_cycle_mean | 10.25% |
-| oil_rel_cycle_last | 8.24% |
-| 其余 18 维合计 | 12.08% |
-
-### 2. 机组基准
-
-机组 `median/MAD` 只用该机组 train 划分的正常窗口计算,存 `models_*/pressure_unit_baselines.json`。仅用于 `oil_*_z`。训练正常窗口少于 24 个或 MAD=0 直接报错。
-
-### 3. 数据与切分
-
-- 窗口:3 小时与 6 小时各一套;标签:重叠即异常(含 1 小时“轻微”阶段,用前后小时补足窗口,不做过渡窗口降权)。
-- 数据来源:复用 `datasets/pressure/hourly.csv`(`--hourly-csv`,不查库);阶段来源 `cache/pks_scanner/evolution/润滑油压力小时阶段段.csv`。
-- 切分:6.5 / 2 / 1.5,seed 20260921,事件组整体不拆,按(机组、标签)分层,并保证每机组 train 都有正常事件组。
-
-## 二、数据范围
-
-| 版本 | 候选(正常/异常) | train | validation | test | 事件组泄漏 |
-|---|---|---:|---:|---:|---:|
-| 3h | 13,805(12,789/1,016) | 9,095(8,439/656) | 2,457(2,256/201) | 2,253(2,094/159) | 0 |
-| 6h | 13,710(12,507/1,203) | 8,646(7,868/778) | 2,841(2,596/245) | 2,223(2,043/180) | 0 |
-
-异常来源:8 号机 986、7 号机 24、9 号机 6(3h)。
-
-## 三、训练
-
-| 版本 | 采样 | 轮数 | 早停 | 最佳轮次 | 最佳验证 logloss |
-|---|---|---|---:|---|---:|---:|
-| 3h | `--all-normal` | 请求 800 | 151 轮 | 101 | 0.03169 |
-| 6h | `--all-normal` | 请求 800 | 150 轮 | 100 | 0.03327 |
-
-## 四、验证集
-
-| 版本 | 阈值 | 查准率 | 召回率 | F1 | PR-AUC |
-|---|---|---:|---:|---:|---:|
-| 3h | 0.862553 | 93.90% | 99.50% | 96.62% | 99.65% |
-| 6h | 0.389191 | 92.45% | 100.00% | 96.08% | 97.41% |
-
-## 五、封存测试集
-
-### 3 小时窗口
-
-测试集 2,253 窗口(正常 2,094、异常 159)。
-
-| 类别 | 查准率 | 召回率 | F1 | 样本数 |
-|---|---:|---:|---:|---:|
-| 正常 | 99.86% | 99.62%(特异度) | 99.74% | 2,094 |
-| 异常 | 95.12% | 98.11% | 96.59% | 159 |
-
-综合:准确率 99.51%、平衡准确率 98.87%、macro-F1 98.17%、PR-AUC 99.61%、**ROC-AUC 0.9997**;事件召回 10/10。
-
-分机组:
-
-| 机组 | 正常窗口 | 正常误报率 | 异常窗口 | 异常召回 | 异常查准 |
-|---|---:|---:|---:|---:|---:|
-| 7 号机 | 952 | 0.32% | 10 | 70.00% | 70.00% |
-| 8 号机 | 442 | 1.13% | 149 | 100.00% | 96.75% |
-| 9 号机 | 700 | 0.00% | 0 | — | — |
-
-### 6 小时窗口
-
-测试集 2,223 窗口(正常 2,043、异常 180)。
-
-| 类别 | 查准率 | 召回率 | F1 | 样本数 |
-|---|---:|---:|---:|---:|
-| 正常 | 100.00% | 98.92%(特异度) | 99.46% | 2,043 |
-| 异常 | 89.11% | 100.00% | 94.24% | 180 |
-
-综合:准确率 99.01%、平衡准确率 99.46%、macro-F1 96.85%、PR-AUC 99.45%、**ROC-AUC 0.9995**;事件召回 14/14。
-
-分机组:
-
-| 机组 | 正常窗口 | 正常误报率 | 异常窗口 | 异常召回 | 异常查准 |
-|---|---:|---:|---:|---:|---:|
-| 7 号机 | 937 | 0.85% | 6 | 100.00% | 42.86% |
-| 8 号机 | 415 | 3.13% | 174 | 100.00% | 93.05% |
-| 9 号机 | 691 | 0.14% | 0 | — | — |
-
-## 六、规则联合辅助输出
-
-- 3h:`cache/pks_scanner/model_assistant_generalized_789/压力模型辅助确认.csv`,13,805 行
-  - 模型辅助确认异常 1,012、保留规则异常人工复核 4、继续观察 122、正常 12,667
-- 6h:`cache/pks_scanner/model_assistant_generalized_789_6h/压力模型辅助确认.csv`,13,710 行
-  - 模型辅助确认异常 1,203、继续观察 113、正常 12,394
-
-v2 下“继续观察”从 v1 的 11,000 降到约 120,说明模型对正常窗口的疑似标记大幅减少。
-
-## 七、版本对比
-
-| 版本 | 特征 | 测试查准率 | 测试召回率 | F1 | PR-AUC | 8 号机正常误报率 |
-|---|---|---:|---:|---:|---:|---:|
-| v1 3h(机组相对) | 13 | 50.91% | 70.44% | 59.10% | 57.92% | 14.93% |
-| **v2 3h(+周期相对)** | 22 | **95.12%** | 98.11% | **96.59%** | **99.61%** | **1.13%** |
-| v2 6h(+周期相对) | 22 | 89.11% | **100.00%** | 94.24% | 99.45% | 3.13% |
-| 旧全量 6h(绝对油压) | 10 | 11.44% | 43.40% | 18.11% | 10.71% | 约 3.6%(170 个) |
-
-**结论**:加入扫描器口径的周期相对特征后,跨机组误报和异常召回同时大幅改善。3h 与 6h 都接近完美,3h 的 F1(96.59%)略高于 6h(94.24%),6h 召回 100% 但误报更多。**推荐 3h v2**。
-
-## 八、限制
-
-- 周期基线依赖“周期开始前 3 天正常”的假设,与扫描器一致;若周期起始即异常,基线会偏。
-- 周期相对特征本质上接近规则核心信号,模型更偏向“跨机组复现并校准规则”,而非发现规则之外的新异常,符合“辅助确认”定位。
-- 7 号机测试异常只有 10 个窗口(来自 1 小时“轻微”阶段),其召回/查准数字样本弱,不作结论;9 号机测试无异常。
-- 标签仍来自扫描器规则阶段,未经维修记录或人工确认,指标代表对规则标签的泛化,不是物理故障诊断能力。

+ 0 - 117
backend/model_assistant_generalized/RESULTS_789_VIBRATION.md

@@ -1,117 +0,0 @@
-# 7/8/9 号机振动泛化模型结果(机组相对特征)
-
-本实验在 `backend/model_assistant_generalized/` 下独立进行,与压力模型完全独立,互不影响。数据来源为 `cache/pks_scanner/evolution/振动异常段.csv`。
-
-目标:把 7/8/9 号机振动数据全部纳入,用**机组相对(泛化)特征**训练一个共享振动模型,解决原振动模型“绝对振动值导致跨机组误报、召回不稳”的问题。
-
-## 一、方法
-
-### 1. 泛化特征(去掉绝对振动值)
-
-振动规则的核心信号是“相对本机**近期稳健基线**(前 7×24 = 168 个有效小时)的偏离”。因此特征以近期基线相对量为主,机组编号只用于选基准、不进模型输入。
-
-删除的绝对特征:各侧 `*_avg_mean/max/range`、`*_peak_mean/max/range`、`*_slope`。
-
-使用的特征(每侧 coupling、chain):
-
-- 近期基线偏离(z 分数):`{side}_avg_dev`、`{side}_peak_dev`
-- 近期基线相对比例:`{side}_avg_rel_last`、`{side}_peak_rel_last`、`{side}_avg_rel_mean`、`{side}_peak_rel_max`
-- 机组归一化:`{side}_avg_z`、`{side}_peak_z`、`{side}_avg_slope_z`(train 正常窗口 median/MAD)
-- 报警比例:`{side}_high_ratio_mean`、`{side}_high_high_ratio_mean`
-- 共享:`both_sides_rising`、`running_coverage_mean/min`
-
-特征重要度(3h)显示“相对比例 + 偏离”主导:`chain_peak_rel_max` 32.9%、`chain_peak_dev` 19.1%、`coupling_peak_rel_max` 8.1%、`coupling_avg_dev` 7.8%。
-
-### 2. 基线口径(与扫描器一致)
-
-振动基线用机组**完整有效历史**在窗口前 **7×24 小时**内的稳健基线,**跨停机周期**取历史、排除重启后首个有效小时,与 `scan_pks_fault_evolution.py` 完全一致。
-
-### 3. 数据与切分
-
-- 窗口:3 小时与 6 小时各一套;标签:重叠即异常(含 1~2 小时短阶段,用前后小时补足);重启后首个有效小时不作样本。
-- 数据来源:复用 `data/hourly.csv`(`--hourly-csv`,不查库)。
-- 切分:6.5 / 2 / 1.5,seed 20260921,事件组不拆,按(机组、标签)分层,保证每机组 train 有正常事件组。
-- 异常事件分配用 `--positive-basis groups`(每个事件等权),避免个别长事件整组落入同一划分。
-
-## 二、数据范围
-
-- 候选窗口:14,139(正常 13,908、异常 231)。
-- 异常事件 25 个:7 号机 8、8 号机 8、9 号机 9(三台机组都有异常)。
-
-| 版本 | train | validation | test |
-|---|---|---|---|
-| 3h | 9,302(正常 9,112/异常 190,16 事件) | 2,657(2,637/20,5 事件) | 2,180(2,159/21,4 事件) |
-| 6h | 8,951(8,731/220,16 事件) | 2,625(2,599/26,5 事件) | 2,332(2,299/33,4 事件) |
-
-跨集合事件组泄漏:0。
-
-## 三、训练
-
-| 版本 | 采样 | 轮数 | 早停 | 最佳轮次 | 最佳验证 logloss |
-|---|---|---|---:|---|---:|---:|
-| 3h | `--all-normal` | 请求 800 | 359 轮 | 309 | 0.02262 |
-| 6h | `--all-normal` | 请求 800 | 382 轮 | 332 | 0.03350 |
-
-## 四、验证集
-
-| 版本 | 阈值 | 查准率 | 召回率 | F1 | PR-AUC |
-|---|---|---:|---:|---:|---:|
-| 3h | 0.916630 | 70.00% | 70.00% | 70.00% | 70.40% |
-| 6h | 0.864419 | 72.41% | 80.77% | 76.36% | 77.21% |
-
-## 五、封存测试集
-
-### 3 小时窗口
-
-测试集 2,180 窗口(正常 2,159、异常 21)。
-
-| 类别 | 查准率 | 召回率 | F1 | 样本数 |
-|---|---:|---:|---:|---:|
-| 正常 | 99.72% | 99.72%(特异度) | 99.72% | 2,159 |
-| 异常 | 71.43% | 71.43% | 71.43% | 21 |
-
-综合:准确率 99.45%、平衡准确率 85.58%、PR-AUC 82.70%;事件召回 4/4。
-
-分机组:
-
-| 机组 | 正常窗口 | 正常误报率 | 异常窗口 | 异常召回 | 异常查准 |
-|---|---:|---:|---:|---:|---:|
-| 7 号机 | 676 | 0.00% | 9 | 77.78% | 100.00% |
-| 8 号机 | 702 | 0.71% | 12 | 66.67% | 61.54% |
-| 9 号机 | 781 | 0.13% | 0 | — | — |
-
-### 6 小时窗口
-
-测试集 2,332 窗口(正常 2,299、异常 33)。
-
-| 类别 | 查准率 | 召回率 | F1 | 样本数 |
-|---|---:|---:|---:|---:|
-| 正常 | 99.14% | 99.91%(特异度) | 99.52% | 2,299 |
-| 异常 | 86.67% | 39.39% | 54.17% | 33 |
-
-综合:准确率 99.06%、平衡准确率 69.65%、PR-AUC 86.40%;事件召回 4/4。
-
-分机组:7 号机误报率 0.00%、召回 60.00%;8 号机误报率 0.00%、召回 22.22%;9 号机误报率 0.22%。
-
-## 六、版本对比
-
-| 版本 | 测试查准率 | 测试召回率 | F1 | PR-AUC | 事件召回 |
-|---|---:|---:|---:|---:|---:|
-| 旧振动模型(6h,绝对特征) | 24.15% | 100.00% | 38.91% | 75.66% | 1/1 |
-| **振动泛化 3h** | **71.43%** | **71.43%** | **71.43%** | 82.70% | 4/4 |
-| 振动泛化 6h | 86.67% | 39.39% | 54.17% | **86.40%** | 4/4 |
-
-**结论**:泛化特征显著改善了振动模型的查准率(旧模型 24% → 3h 71%)。3h 的 F1 与召回更均衡(71/71),6h 查准率更高但召回只有 39%。振动故障确认以“不漏”优先,**推荐 3h**。
-
-## 七、规则联合辅助输出
-
-- 3h:`cache/pks_scanner/model_assistant_generalized_vibration_789/振动模型辅助确认.csv`,14,139 行
-  - 模型辅助确认异常 219、保留规则异常人工复核 12、继续观察 13、正常 13,895
-- 6h:`cache/pks_scanner/model_assistant_generalized_vibration_789_6h/振动模型辅助确认.csv`,13,908 行
-
-## 八、限制
-
-- 振动异常阶段短(多为 1~2 小时),测试异常窗口仅 21~33 个、4 个事件,指标波动较大。
-- 9 号机测试集没有异常事件,其正常误报率可评估,但异常召回不作结论。
-- 标签来自扫描器规则阶段,未经维修记录或人工确认,指标代表对规则标签的泛化。
-- 模型定位为规则辅助确认,不替代扫描器的阶段划分。

+ 0 - 1
backend/model_assistant_generalized/__init__.py

@@ -1 +0,0 @@
-"""PKS rule-assistance model dataset and evaluation tools."""

+ 0 - 406
backend/model_assistant_generalized/build_datasets.py

@@ -1,406 +0,0 @@
-#!/usr/bin/env python3
-"""Create reviewable configurable-window PKS samples from scanner hourly data.
-
-This module deliberately imports the scanner's read and aggregation helpers.
-It must not create an independent interpretation of PKS timestamps, coverage,
-thresholds, or batch-to-unit mapping.
-"""
-from __future__ import annotations
-
-import argparse
-import csv
-import json
-import math
-import shutil
-import sys
-from collections import Counter, defaultdict
-from datetime import datetime, timedelta
-from pathlib import Path
-from statistics import median
-
-ROOT = Path(__file__).resolve().parents[2]
-BACKEND = ROOT / "backend"
-if str(BACKEND) not in sys.path:
-    sys.path.insert(0, str(BACKEND))
-
-import scan_pks_fault_evolution as scanner  # noqa: E402
-
-WINDOW_HOURS = 6
-ISOLATION_HOURS = 12
-RESTART_GAP_HOURS = 12
-DEFAULT_START = "2025-01-01 00:00:00"
-DEFAULT_END = "2026-09-01 00:00:00"
-
-
-def parse_dt(value: str) -> datetime:
-    return datetime.strptime(value, "%Y-%m-%d %H:%M:%S")
-
-
-def finite(value) -> float | None:
-    if value is None or value == "":
-        return None
-    try:
-        number = float(value)
-    except (TypeError, ValueError):
-        return None
-    return number if math.isfinite(number) else None
-
-
-def mean(values: list[float | None]) -> float | None:
-    valid = [value for value in values if value is not None]
-    return sum(valid) / len(valid) if valid else None
-
-
-def value_range(values: list[float | None]) -> float | None:
-    valid = [value for value in values if value is not None]
-    return max(valid) - min(valid) if valid else None
-
-
-def slope(values: list[float | None]) -> float | None:
-    return scanner.linear_slope(values)
-
-
-def ratio(count, running) -> float:
-    return float(count or 0) / max(float(running or 0), 1.0)
-
-
-def csv_rows(path: Path) -> list[dict]:
-    with path.open(encoding="utf-8-sig", newline="") as handle:
-        return list(csv.DictReader(handle))
-
-
-def stage_ranges(path: Path, stage_field: str) -> list[dict]:
-    ranges = []
-    for index, row in enumerate(csv_rows(path), 1):
-        ranges.append({
-            "stage_id": f"{stage_field}_{row['机组']}_{row['批次']}_{index:04d}",
-            "unit": row["机组"],
-            "batch": int(row["批次"]),
-            "level": row[stage_field],
-            "start": parse_dt(row["开始小时"]),
-            "end": parse_dt(row["结束小时"]),
-            "confirmed": row.get("是否形成确认等级", ""),
-        })
-    return ranges
-
-
-def fetch_hourly(start: datetime, end: datetime, chunk_days: int) -> tuple[list[dict], dict[int, dict]]:
-    batch_to_unit = scanner.load_pks_batches()
-    configs = scanner.load_alarm_config(batch_to_unit)
-    ranges = scanner.load_batch_ranges(sorted(batch_to_unit), start, end)
-    grouped = defaultdict(list)
-    for batch in sorted(ranges):
-        grouped[batch_to_unit[batch]].append(batch)
-    rows = []
-    for unit, batches in sorted(grouped.items(), key=lambda item: scanner._unit_sort_key(item[0])):
-        unit_start = min(ranges[batch][0] for batch in batches)
-        unit_end = max(ranges[batch][1] for batch in batches)
-        rows.extend(scanner.fetch_hourly(unit, batches, unit_start, unit_end, chunk_days, configs[batches[0]]))
-    rows.sort(key=lambda row: (scanner._unit_sort_key(row["机组"]), row["批次"], row["_dt"]))
-    return rows, configs
-
-
-def load_hourly_csv(path: Path) -> list[dict]:
-    """Rebuild the hourly row structure from a saved hourly.csv without querying the DB."""
-    result = []
-    for row in csv_rows(path):
-        item = {
-            "机组": row["机组"],
-            "批次": int(row["批次"]),
-            "小时": row["小时"],
-            "_dt": parse_dt(row["小时"]),
-            "样本数": int(row["样本数"]),
-            "运行样本数": int(row["运行样本数"]),
-            "运行覆盖率": float(row["运行覆盖率"]),
-        }
-        for field in scanner.NUMERIC_HOURLY_FIELDS:
-            item[field] = finite(row.get(field))
-        result.append(item)
-    result.sort(key=lambda item: (scanner._unit_sort_key(item["机组"]), item["批次"], item["_dt"]))
-    return result
-
-
-def assign_run_cycles(rows: list[dict], model_type: str) -> list[dict]:
-    grouped = defaultdict(list)
-    for row in rows:
-        if row["运行覆盖率"] < scanner.MIN_HOUR_COVERAGE:
-            continue
-        if model_type == "pressure" and row["oil_avg"] is None:
-            continue
-        if model_type == "vibration" and (row["coupling_avg"] is None or row["chain_avg"] is None):
-            continue
-        grouped[(row["机组"], row["批次"])].append(row)
-
-    result = []
-    for (unit, batch), series in grouped.items():
-        series.sort(key=lambda row: row["_dt"])
-        cycle = 0
-        previous = None
-        for row in series:
-            restart = previous is None or row["_dt"] - previous > timedelta(hours=RESTART_GAP_HOURS)
-            if restart:
-                cycle += 1
-            item = dict(row)
-            item["run_cycle_id"] = f"{unit}_B{batch}_R{cycle:04d}"
-            item["restart_hour"] = restart
-            result.append(item)
-            previous = row["_dt"]
-    return sorted(result, key=lambda row: (scanner._unit_sort_key(row["机组"]), row["批次"], row["_dt"]))
-
-
-def previous_baseline(series: list[dict], index: int, field: str, hours: int) -> tuple[float | None, float | None]:
-    now = series[index]["_dt"]
-    prior = [row[field] for row in series[:index] if row["_dt"] >= now - timedelta(hours=hours) and row[field] is not None]
-    return scanner.robust_baseline(prior)
-
-
-def source_hour(row: dict) -> dict:
-    running = row["运行样本数"]
-    return {
-        "hour": row["小时"],
-        "running_samples": running,
-        "running_coverage": row["运行覆盖率"],
-        "oil_avg": row["oil_avg"], "oil_min": row["oil_min"], "oil_max": row["oil_max"],
-        "oil_low_ratio": ratio(row["oil_low_count"], running),
-        "oil_low_low_ratio": ratio(row["oil_low_low_count"], running),
-        "coupling_avg": row["coupling_avg"], "coupling_max": row["coupling_max"],
-        "chain_avg": row["chain_avg"], "chain_max": row["chain_max"],
-        "coupling_high_ratio": ratio(row["coupling_high_count"], running),
-        "coupling_high_high_ratio": ratio(row["coupling_high_high_count"], running),
-        "chain_high_ratio": ratio(row["chain_high_count"], running),
-        "chain_high_high_ratio": ratio(row["chain_high_high_count"], running),
-    }
-
-
-def cycle_baseline(series: list[dict], days: int = scanner.BASELINE_DAYS) -> float | None:
-    """Median of daily oil medians over the first `days` calendar days of a cycle.
-
-    Mirrors the scanner's cycle baseline: the baseline is taken from the start of
-    the run cycle, so it never uses information from the window being scored.
-    """
-    by_day: dict = {}
-    for row in series:
-        if row["oil_avg"] is None:
-            continue
-        day = row["_dt"].date()
-        if day not in by_day and len(by_day) >= days:
-            break
-        by_day.setdefault(day, []).append(row["oil_avg"])
-    daily = [scanner.median_value(values) for _, values in sorted(by_day.items())[:days]]
-    daily = [value for value in daily if value is not None]
-    return scanner.median_value(daily)
-
-
-def decline_hours(series: list[dict], end_index: int) -> int:
-    """Consecutive valid hours of oil-pressure decline ending at end_index."""
-    count = 0
-    for index in range(end_index, 0, -1):
-        current, previous = series[index]["oil_avg"], series[index - 1]["oil_avg"]
-        if current is None or previous is None or current >= previous:
-            break
-        count += 1
-    return count
-
-
-def pressure_features(series: list[dict], end_index: int, window: list[dict]) -> dict:
-    oil = [row["oil_avg"] for row in window]
-    coverages = [row["运行覆盖率"] for row in window]
-    baseline = scanner.median_value([row["oil_avg"] for row in window[:-1]])
-    current = oil[-1]
-    oil_mean, oil_min, oil_max = mean(oil), min(oil), max(oil)
-    oil_range = value_range(oil)
-    oil_slope = slope(oil)
-    cycle_base = cycle_baseline(series)
-    recent_center, _ = previous_baseline(series, end_index, "oil_avg", 24)
-    return {
-        "oil_mean": oil_mean, "oil_min": oil_min, "oil_max": oil_max, "oil_range": oil_range,
-        "oil_slope": oil_slope,
-        "oil_last_relative_to_window": (current / baseline - 1) if baseline else None,
-        "oil_low_ratio_mean": mean([ratio(row["oil_low_count"], row["运行样本数"]) for row in window]),
-        "oil_low_low_ratio_mean": mean([ratio(row["oil_low_low_count"], row["运行样本数"]) for row in window]),
-        "running_coverage_mean": mean(coverages), "running_coverage_min": min(coverages),
-        "oil_rel_cycle_mean": (oil_mean / cycle_base - 1) if cycle_base else None,
-        "oil_rel_cycle_min": (oil_min / cycle_base - 1) if cycle_base else None,
-        "oil_rel_cycle_last": (current / cycle_base - 1) if cycle_base else None,
-        "oil_rel_recent_mean": (oil_mean / recent_center - 1) if recent_center else None,
-        "oil_rel_recent_last": (current / recent_center - 1) if recent_center else None,
-        "oil_slope_rel_cycle": (oil_slope / cycle_base) if cycle_base else None,
-        "oil_decline_hours": decline_hours(series, end_index),
-        "oil_drawdown_cycle": ((cycle_base - oil_min) / cycle_base) if cycle_base else None,
-        "oil_range_rel_cycle": (oil_range / cycle_base) if cycle_base else None,
-    }
-
-
-def vibration_features(series: list[dict], end_index: int, window: list[dict]) -> dict:
-    features = {
-        "running_coverage_mean": mean([row["运行覆盖率"] for row in window]),
-        "running_coverage_min": min(row["运行覆盖率"] for row in window),
-    }
-    for side in ("coupling", "chain"):
-        averages = [row[f"{side}_avg"] for row in window]
-        peaks = [row[f"{side}_max"] for row in window]
-        avg_center, avg_scale = previous_baseline(series, end_index, f"{side}_avg", scanner.VIBRATION_BASELINE_DAYS * 24)
-        peak_center, peak_scale = previous_baseline(series, end_index, f"{side}_max", scanner.VIBRATION_BASELINE_DAYS * 24)
-        current_avg, current_peak = averages[-1], peaks[-1]
-        features.update({
-            f"{side}_avg_mean": mean(averages), f"{side}_avg_max": max(averages), f"{side}_avg_range": value_range(averages),
-            f"{side}_avg_slope": slope(averages), f"{side}_peak_mean": mean(peaks), f"{side}_peak_max": max(peaks),
-            f"{side}_peak_range": value_range(peaks), f"{side}_peak_slope": slope(peaks),
-            f"{side}_avg_baseline_deviation": ((current_avg - avg_center) / avg_scale) if avg_center is not None and avg_scale else None,
-            f"{side}_peak_baseline_deviation": ((current_peak - peak_center) / peak_scale) if peak_center is not None and peak_scale else None,
-            f"{side}_avg_rel_last": (current_avg / avg_center - 1) if avg_center else None,
-            f"{side}_peak_rel_last": (current_peak / peak_center - 1) if peak_center else None,
-            f"{side}_avg_rel_mean": (mean(averages) / avg_center - 1) if avg_center else None,
-            f"{side}_peak_rel_max": (max(peaks) / peak_center - 1) if peak_center else None,
-            f"{side}_high_ratio_mean": mean([ratio(row[f"{side}_high_count"], row["运行样本数"]) for row in window]),
-            f"{side}_high_high_ratio_mean": mean([ratio(row[f"{side}_high_high_count"], row["运行样本数"]) for row in window]),
-        })
-    features["both_sides_rising"] = int(features["coupling_avg_slope"] > 0 and features["chain_avg_slope"] > 0)
-    return features
-
-
-def window_status(window: list[dict], stages: list[dict]) -> tuple[int | None, str, str | None, str | None]:
-    start, end = window[0]["_dt"], window[-1]["_dt"]
-    overlapping = [stage for stage in stages if stage["start"] <= end and stage["end"] >= start]
-    if overlapping:
-        stage = min(overlapping, key=lambda item: (item["start"], item["stage_id"]))
-        return 1, "rule_stage", stage["stage_id"], None
-    nearby = [stage for stage in stages if stage["start"] - timedelta(hours=ISOLATION_HOURS) <= end and stage["end"] + timedelta(hours=ISOLATION_HOURS) >= start]
-    if nearby:
-        return None, "excluded", None, "anomaly_isolation_period"
-    return 0, "stable_normal", None, None
-
-
-def build_samples(rows: list[dict], model_type: str, stages: list[dict], window_hours: int) -> list[dict]:
-    cycle_rows = assign_run_cycles(rows, model_type)
-    by_cycle = defaultdict(list)
-    for row in cycle_rows:
-        by_cycle[(row["机组"], row["批次"], row["run_cycle_id"])].append(row)
-    # The scanner's vibration baseline is the robust baseline over the device's
-    # full valid history in the prior 7*24 hours, across restarts, excluding
-    # restart transition hours.  Reproduce that here so features match the rule.
-    full_by_batch: dict[tuple, list[dict]] = {}
-    index_by_batch: dict[tuple, dict] = {}
-    if model_type == "vibration":
-        grouped = defaultdict(list)
-        for row in cycle_rows:
-            if not row["restart_hour"]:
-                grouped[(row["机组"], row["批次"])].append(row)
-        for key, series in grouped.items():
-            series.sort(key=lambda row: row["_dt"])
-            full_by_batch[key] = series
-            index_by_batch[key] = {row["_dt"]: index for index, row in enumerate(series)}
-    samples = []
-    sequence = 0
-    for (unit, batch, cycle_id), series in sorted(by_cycle.items()):
-        series.sort(key=lambda row: row["_dt"])
-        for end_index in range(window_hours - 1, len(series)):
-            window = series[end_index - window_hours + 1:end_index + 1]
-            if any(window[index]["_dt"] - window[index - 1]["_dt"] > timedelta(hours=RESTART_GAP_HOURS) for index in range(1, len(window))):
-                continue
-            if any(row["restart_hour"] for row in window):
-                exclude_reason = "contains_restart_transition_hour"
-                label, source, stage_id = None, "excluded", None
-            else:
-                scoped_stages = [stage for stage in stages if stage["unit"] == unit and stage["batch"] == batch]
-                label, source, stage_id, exclude_reason = window_status(window, scoped_stages)
-            sequence += 1
-            sample_id = f"{model_type}_{unit}_B{batch}_{window[0]['_dt']:%Y%m%d%H}_{window[-1]['_dt']:%Y%m%d%H}_{sequence:06d}"
-            event_group_id = stage_id or f"{cycle_id}_normal"
-            if model_type == "pressure":
-                features = pressure_features(series, end_index, window)
-            else:
-                key = (unit, batch)
-                full_series = full_by_batch[key]
-                full_index = index_by_batch[key][window[-1]["_dt"]]
-                features = vibration_features(full_series, full_index, window)
-            samples.append({
-                "sample_id": sample_id, "model_type": model_type, "unit": unit, "batch": batch,
-                "run_cycle_id": cycle_id, "event_group_id": event_group_id,
-                "start_hour": window[0]["小时"], "end_hour": window[-1]["小时"],
-                "valid_hour_count": window_hours,
-                "natural_duration_hours": int((window[-1]["_dt"] - window[0]["_dt"]).total_seconds() / 3600) + 1,
-                "label": label, "label_name": "anomaly" if label == 1 else "normal" if label == 0 else "excluded",
-                "label_source": source, "label_confidence": "initial_rule" if label is not None else "not_usable",
-                "rule_stage_id": stage_id, "exclude_reason": exclude_reason,
-                "features": features, "source_hours": [source_hour(row) for row in window],
-            })
-    return samples
-
-
-def write_hourly(path: Path, rows: list[dict]) -> None:
-    fields = ["机组", "批次", "小时", "样本数", "运行样本数", "运行覆盖率", *scanner.NUMERIC_HOURLY_FIELDS]
-    with path.open("w", encoding="utf-8-sig", newline="") as handle:
-        writer = csv.DictWriter(handle, fieldnames=fields)
-        writer.writeheader()
-        for row in rows:
-            writer.writerow({field: row.get(field) for field in fields})
-
-
-def write_samples(base: Path, samples: list[dict]) -> Counter:
-    for folder in ("candidates", "excluded"):
-        (base / folder).mkdir(parents=True, exist_ok=True)
-    manifest_fields = ["sample_id", "model_type", "unit", "batch", "run_cycle_id", "event_group_id", "start_hour", "end_hour", "label", "label_name", "label_source", "label_confidence", "rule_stage_id", "exclude_reason", "split"]
-    counts = Counter()
-    with (base / "manifest.csv").open("w", encoding="utf-8-sig", newline="") as handle:
-        writer = csv.DictWriter(handle, fieldnames=manifest_fields)
-        writer.writeheader()
-        for sample in samples:
-            folder = "excluded" if sample["label"] is None else "candidates"
-            with (base / folder / f"{sample['sample_id']}.json").open("w", encoding="utf-8") as output:
-                json.dump(sample, output, ensure_ascii=False, indent=2, allow_nan=False)
-            writer.writerow({field: sample.get(field, "") for field in manifest_fields})
-            counts[(folder, sample["label_name"], sample.get("exclude_reason") or "")] += 1
-    return counts
-
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description="从现有PKS扫描器构建可配置小时窗口的模型辅助样本")
-    parser.add_argument("--start", default=DEFAULT_START)
-    parser.add_argument("--end", default=DEFAULT_END)
-    parser.add_argument("--chunk-days", type=int, default=14)
-    parser.add_argument("--window-hours", type=int, default=WINDOW_HOURS)
-    parser.add_argument("--output-dir", default=str(Path(__file__).resolve().parent / "datasets"))
-    parser.add_argument("--evolution-dir", default=str(ROOT / "cache" / "pks_scanner" / "evolution"))
-    parser.add_argument("--hourly-csv", default=None, help="复用已保存的 hourly.csv,不查询数据库")
-    parser.add_argument("--model-types", nargs="+", choices=("pressure", "vibration"), default=["pressure", "vibration"])
-    parser.add_argument("--units", nargs="+", default=None, help="只构建指定机组,例如 8号机")
-    args = parser.parse_args()
-    start, end = parse_dt(args.start), parse_dt(args.end)
-    if end <= start or args.chunk_days < 1 or args.window_hours < 1:
-        parser.error("时间范围或 chunk-days 无效")
-    output = Path(args.output_dir)
-    evolution = Path(args.evolution_dir)
-    for name in ("润滑油压力小时阶段段.csv", "振动异常段.csv"):
-        if not (evolution / name).is_file():
-            parser.error(f"缺少扫描器结果:{evolution / name}")
-    if output.exists():
-        shutil.rmtree(output)
-    output.mkdir(parents=True)
-    if args.hourly_csv:
-        hourly_path = Path(args.hourly_csv)
-        if not hourly_path.is_file():
-            parser.error(f"缺少 hourly.csv:{hourly_path}")
-        print(f"复用小时中间表:{hourly_path}")
-        rows = load_hourly_csv(hourly_path)
-    else:
-        print("只读模式:复用 scan_pks_fault_evolution.py 的小时聚合和数据口径")
-        rows, _ = fetch_hourly(start, end, args.chunk_days)
-    pressure_stages = stage_ranges(evolution / "润滑油压力小时阶段段.csv", "压力阶段")
-    vibration_stages = stage_ranges(evolution / "振动异常段.csv", "振动阶段")
-    selected_units = set(args.units or [])
-    for model_type in args.model_types:
-        stages = pressure_stages if model_type == "pressure" else vibration_stages
-        scoped_rows = [row for row in rows if not selected_units or row["机组"] in selected_units]
-        scoped_stages = [stage for stage in stages if not selected_units or stage["unit"] in selected_units]
-        base = output / model_type
-        base.mkdir()
-        write_hourly(base / "hourly.csv", scoped_rows)
-        counts = write_samples(base, build_samples(scoped_rows, model_type, scoped_stages, args.window_hours))
-        print(f"{model_type} samples: {dict(counts)}")
-    print(f"小时中间数据:{len(rows):,} 行")
-    print(f"样本输出目录:{output}")
-
-
-if __name__ == "__main__":
-    main()

Failā izmaiņas netiks attēlotas, jo tās ir par lielu
+ 0 - 40537
backend/model_assistant_generalized/data/hourly.csv


+ 0 - 138
backend/model_assistant_generalized/evaluate_model.py

@@ -1,138 +0,0 @@
-#!/usr/bin/env python3
-"""Evaluate a fixed PKS assistant model against the sealed test directory."""
-from __future__ import annotations
-
-import argparse
-import csv
-import importlib
-import json
-from collections import Counter, defaultdict
-from pathlib import Path
-
-import joblib
-import numpy as np
-from sklearn.metrics import average_precision_score, confusion_matrix, precision_recall_fscore_support
-
-
-def feature_module(model_type: str):
-    """Pressure and vibration use independent feature modules."""
-    return importlib.import_module(
-        "generalized_features" if model_type == "pressure" else "generalized_features_vibration"
-    )
-
-
-def load_samples(directory: Path) -> list[dict]:
-    return [json.loads(path.read_text(encoding="utf-8")) for path in sorted(directory.glob("*.json"))]
-
-
-def matrix(feature_dicts: list[dict], names: list[str]) -> np.ndarray:
-    return np.asarray([[item.get(name) for name in names] for item in feature_dicts], dtype=float)
-
-
-def metrics(y: np.ndarray, predicted: np.ndarray, probabilities: np.ndarray) -> dict:
-    labels = [0, 1]
-    per_precision, per_recall, per_f1, per_support = precision_recall_fscore_support(
-        y, predicted, labels=labels, zero_division=0
-    )
-    tn, fp, fn, tp = confusion_matrix(y, predicted, labels=labels).ravel()
-    samples = int(len(y))
-    normal_support, anomaly_support = int(per_support[0]), int(per_support[1])
-    specificity = float(tn / (tn + fp)) if (tn + fp) else 0.0
-    accuracy = float((tp + tn) / samples) if samples else 0.0
-    return {
-        "samples": samples, "normal_samples": normal_support, "anomaly_samples": anomaly_support,
-        "tp": int(tp), "fp": int(fp), "fn": int(fn), "tn": int(tn),
-        "precision": float(per_precision[1]), "recall": float(per_recall[1]), "f1": float(per_f1[1]),
-        "pr_auc": float(average_precision_score(y, probabilities)) if len(set(y)) == 2 else None,
-        "specificity": specificity, "accuracy": accuracy,
-        "balanced_accuracy": float((per_recall[0] + per_recall[1]) / 2),
-        "macro_f1": float((per_f1[0] + per_f1[1]) / 2),
-        "weighted_f1": float((per_f1[0] * normal_support + per_f1[1] * anomaly_support) / samples) if samples else 0.0,
-        "class_metrics": {
-            "normal": {"precision": float(per_precision[0]), "recall": float(per_recall[0]), "f1": float(per_f1[0]), "support": normal_support},
-            "anomaly": {"precision": float(per_precision[1]), "recall": float(per_recall[1]), "f1": float(per_f1[1]), "support": anomaly_support},
-        },
-        "confusion_matrix_counts": {"tn": int(tn), "fp": int(fp), "fn": int(fn), "tp": int(tp)},
-        "confusion_matrix_normalized": {
-            "normal": {"predicted_normal": float(tn / normal_support) if normal_support else 0.0, "predicted_anomaly": float(fp / normal_support) if normal_support else 0.0},
-            "anomaly": {"predicted_normal": float(fn / anomaly_support) if anomaly_support else 0.0, "predicted_anomaly": float(tp / anomaly_support) if anomaly_support else 0.0},
-        },
-    }
-
-
-def event_metrics(samples: list[dict], predicted: np.ndarray) -> dict:
-    groups = defaultdict(list)
-    for sample, value in zip(samples, predicted):
-        groups[sample["event_group_id"]].append((sample, int(value)))
-    positives = [values for values in groups.values() if values[0][0]["label"] == 1]
-    negatives = [values for values in groups.values() if values[0][0]["label"] == 0]
-    tp = sum(any(value for _, value in values) for values in positives)
-    fn = len(positives) - tp
-    fp = sum(any(value for _, value in values) for values in negatives)
-    return {
-        "anomaly_events": len(positives), "normal_events": len(negatives),
-        "detected_anomaly_events": tp, "missed_anomaly_events": fn, "false_positive_normal_events": fp,
-        "event_recall": tp / len(positives) if positives else None,
-    }
-
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description="仅对封存PKS测试集做最终模型评估")
-    parser.add_argument("model_type", choices=("pressure", "vibration"))
-    parser.add_argument("--datasets-dir", default=str(Path(__file__).resolve().parent / "datasets"))
-    parser.add_argument("--models-dir", default=str(Path(__file__).resolve().parent / "models"))
-    parser.add_argument("--output-dir", default=str(Path(__file__).resolve().parents[2] / "cache" / "pks_scanner" / "model_assistant_generalized"))
-    args = parser.parse_args()
-    gf = feature_module(args.model_type)
-    bundle = joblib.load(Path(args.models_dir) / f"{args.model_type}_model.joblib")
-    metadata = json.loads((Path(args.models_dir) / f"{args.model_type}_metadata.json").read_text(encoding="utf-8"))
-    samples = load_samples(Path(args.datasets_dir) / args.model_type / "test")
-    if not samples or not any(sample["label"] == 1 for sample in samples):
-        parser.error("封存测试集必须包含至少一个异常样本")
-    x = bundle["imputer"].transform(matrix(gf.feature_matrix(samples, bundle["unit_baselines"]), bundle["feature_names"]))
-    y = np.asarray([sample["label"] for sample in samples])
-    probabilities = bundle["model"].predict_proba(x)[:, 1]
-    predicted = (probabilities >= metadata["threshold"]).astype(int)
-    distances, neighbors = bundle["knn"].named_steps["knn"].kneighbors(bundle["knn"].named_steps["scale"].transform(x), n_neighbors=1)
-    rows = []
-    for index, sample in enumerate(samples):
-        rows.append({
-            "sample_id": sample["sample_id"], "unit": sample["unit"], "batch": sample["batch"],
-            "event_group_id": sample["event_group_id"], "start_hour": sample["start_hour"], "end_hour": sample["end_hour"],
-            "true_label": sample["label"], "anomaly_probability": probabilities[index], "predicted_label": int(predicted[index]),
-            "nearest_training_sample_index": int(neighbors[index][0]), "nearest_training_distance": float(distances[index][0]),
-            "nearest_training_similarity": float(1 / (1 + distances[index][0])),
-        })
-    by_unit = {}
-    for unit in sorted({sample["unit"] for sample in samples}):
-        indices = [index for index, sample in enumerate(samples) if sample["unit"] == unit]
-        by_unit[unit] = metrics(y[indices], predicted[indices], probabilities[indices])
-    report = {
-        "model_type": args.model_type, "evaluation_set": "sealed_test", "threshold": metadata["threshold"],
-        "window_metrics": metrics(y, predicted, probabilities), "event_metrics": event_metrics(samples, predicted),
-        "by_unit": by_unit,
-        "warning": "Labels originate from current scanner rules unless manually updated; metrics measure held-out rule-label agreement, not confirmed physical-fault diagnosis.",
-    }
-    output = Path(args.output_dir)
-    output.mkdir(parents=True, exist_ok=True)
-    with (output / f"{args.model_type}_test_predictions.csv").open("w", encoding="utf-8-sig", newline="") as handle:
-        writer = csv.DictWriter(handle, fieldnames=list(rows[0]))
-        writer.writeheader()
-        writer.writerows(rows)
-    (output / f"{args.model_type}_test_evaluation.json").write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
-    summary = report["window_metrics"]
-    print(f"测试集:{summary['samples']} 窗口(正常 {summary['normal_samples']}、异常 {summary['anomaly_samples']}),阈值 {report['threshold']:.6f}")
-    print(f"{'类别':<6}{'查准率':>10}{'召回率':>10}{'F1':>10}{'样本数':>10}")
-    for name, label in (("正常", "normal"), ("异常", "anomaly")):
-        item = summary["class_metrics"][label]
-        print(f"{name:<6}{item['precision']:>10.4f}{item['recall']:>10.4f}{item['f1']:>10.4f}{item['support']:>10}")
-    print(
-        f"准确率 {summary['accuracy']:.4f}  平衡准确率 {summary['balanced_accuracy']:.4f}  "
-        f"macro-F1 {summary['macro_f1']:.4f}  加权F1 {summary['weighted_f1']:.4f}  "
-        f"特异度 {summary['specificity']:.4f}  PR-AUC {summary['pr_auc']:.4f}"
-    )
-    print(json.dumps(report, ensure_ascii=False, indent=2))
-
-
-if __name__ == "__main__":
-    main()

+ 0 - 141
backend/model_assistant_generalized/generalized_features.py

@@ -1,141 +0,0 @@
-#!/usr/bin/env python3
-"""Unit-relative (generalized) pressure features for cross-unit PKS models.
-
-Absolute oil-pressure levels differ between units, so a shared model trained on
-raw values treats one unit's normal level as another unit's anomaly.  These
-features express each window relative to its own window and its own unit's
-normal baseline (median/MAD), so the same model can be shared across units.
-
-Baselines must be computed from training-split normal windows only.
-"""
-from __future__ import annotations
-
-from collections import defaultdict
-from statistics import median
-
-MIN_BASELINE_SAMPLES = 24
-
-# Absolute oil-pressure features dropped from the generalized feature set.
-ABSOLUTE_FEATURES = ("oil_mean", "oil_min", "oil_max", "oil_range", "oil_slope")
-
-# Fixed feature order for the shared model.
-FEATURE_NAMES = (
-    "oil_last_relative_to_window",
-    "oil_range_rel",
-    "oil_slope_rel",
-    "oil_min_dev",
-    "oil_max_dev",
-    "oil_low_ratio_mean",
-    "oil_low_low_ratio_mean",
-    "running_coverage_mean",
-    "running_coverage_min",
-    "oil_rel_cycle_mean",
-    "oil_rel_cycle_min",
-    "oil_rel_cycle_last",
-    "oil_rel_recent_mean",
-    "oil_rel_recent_last",
-    "oil_slope_rel_cycle",
-    "oil_decline_hours",
-    "oil_drawdown_cycle",
-    "oil_range_rel_cycle",
-    "oil_mean_z",
-    "oil_min_z",
-    "oil_last_z",
-    "oil_slope_z",
-)
-
-# Features computed in build_datasets from the run cycle and passed through as-is.
-CYCLE_RELATIVE_FEATURES = (
-    "oil_rel_cycle_mean",
-    "oil_rel_cycle_min",
-    "oil_rel_cycle_last",
-    "oil_rel_recent_mean",
-    "oil_rel_recent_last",
-    "oil_slope_rel_cycle",
-    "oil_decline_hours",
-    "oil_drawdown_cycle",
-    "oil_range_rel_cycle",
-)
-
-
-def feature_names() -> list[str]:
-    return list(FEATURE_NAMES)
-
-
-def _safe_ratio(numerator, denominator):
-    if numerator is None or denominator is None or denominator == 0:
-        return None
-    return numerator / denominator
-
-
-def _last_hour_oil(sample: dict):
-    hours = sample.get("source_hours") or []
-    if not hours:
-        return None
-    return hours[-1].get("oil_avg")
-
-
-def compute_unit_baselines(samples: list[dict]) -> dict[str, dict]:
-    """Compute per-unit normal oil-pressure median/MAD from training normal windows.
-
-    Raises if a unit has fewer than MIN_BASELINE_SAMPLES training normal windows
-    or a degenerate MAD, because the normalization would be unstable.
-    """
-    by_unit = defaultdict(list)
-    for sample in samples:
-        if sample["label"] != 0:
-            continue
-        value = sample["features"].get("oil_mean")
-        if value is not None:
-            by_unit[sample["unit"]].append(value)
-    if not by_unit:
-        raise RuntimeError("训练集没有正常窗口,无法计算机组基准")
-    baselines = {}
-    for unit, values in sorted(by_unit.items()):
-        if len(values) < MIN_BASELINE_SAMPLES:
-            raise RuntimeError(
-                f"{unit} 训练正常窗口仅 {len(values)} 个,少于 {MIN_BASELINE_SAMPLES} 个,无法计算稳定基准"
-            )
-        center = float(median(values))
-        mad = float(median([abs(value - center) for value in values]))
-        if mad <= 0:
-            raise RuntimeError(f"{unit} 正常油压 MAD=0,无法归一化")
-        baselines[unit] = {"median": center, "mad": mad, "normal_samples": len(values)}
-    return baselines
-
-
-def generalized_features(sample: dict, baselines: dict[str, dict]) -> dict:
-    features = sample["features"]
-    unit = sample["unit"]
-    if unit not in baselines:
-        raise KeyError(f"缺少 {unit} 的机组基准")
-    center = baselines[unit]["median"]
-    mad = baselines[unit]["mad"]
-    oil_mean = features.get("oil_mean")
-    oil_min = features.get("oil_min")
-    oil_max = features.get("oil_max")
-    oil_range = features.get("oil_range")
-    oil_slope = features.get("oil_slope")
-    last_oil = _last_hour_oil(sample)
-    result = {
-        "oil_last_relative_to_window": features.get("oil_last_relative_to_window"),
-        "oil_range_rel": _safe_ratio(oil_range, oil_mean),
-        "oil_slope_rel": _safe_ratio(oil_slope, oil_mean),
-        "oil_min_dev": _safe_ratio(oil_min - oil_mean, oil_mean) if oil_min is not None and oil_mean is not None else None,
-        "oil_max_dev": _safe_ratio(oil_max - oil_mean, oil_mean) if oil_max is not None and oil_mean is not None else None,
-        "oil_low_ratio_mean": features.get("oil_low_ratio_mean"),
-        "oil_low_low_ratio_mean": features.get("oil_low_low_ratio_mean"),
-        "running_coverage_mean": features.get("running_coverage_mean"),
-        "running_coverage_min": features.get("running_coverage_min"),
-        "oil_mean_z": (oil_mean - center) / mad if oil_mean is not None else None,
-        "oil_min_z": (oil_min - center) / mad if oil_min is not None else None,
-        "oil_last_z": (last_oil - center) / mad if last_oil is not None else None,
-        "oil_slope_z": oil_slope / mad if oil_slope is not None else None,
-    }
-    for name in CYCLE_RELATIVE_FEATURES:
-        result[name] = features.get(name)
-    return result
-
-
-def feature_matrix(samples: list[dict], baselines: dict[str, dict]) -> list[dict]:
-    return [generalized_features(sample, baselines) for sample in samples]

+ 0 - 118
backend/model_assistant_generalized/generalized_features_vibration.py

@@ -1,118 +0,0 @@
-#!/usr/bin/env python3
-"""Unit-relative (generalized) vibration features for cross-unit PKS models.
-
-Absolute vibration levels differ between units, so a shared model trained on raw
-values treats one unit's normal level as another unit's anomaly.  These features
-express each window relative to its own unit's recent robust baseline (the same
-7*24-hour baseline the scanner rule uses) and to its own unit's normal
-distribution (median/MAD), so one model can be shared across units.
-
-Baselines must be computed from training-split normal windows only.
-"""
-from __future__ import annotations
-
-from collections import defaultdict
-from statistics import median
-
-MIN_BASELINE_SAMPLES = 24
-SIDES = ("coupling", "chain")
-
-# Absolute vibration features dropped from the generalized feature set.
-ABSOLUTE_FEATURES = tuple(
-    f"{side}_{kind}_{stat}"
-    for side in SIDES
-    for kind in ("avg", "peak")
-    for stat in ("mean", "max", "range")
-) + tuple(f"{side}_{kind}_slope" for side in SIDES for kind in ("avg", "peak"))
-
-# Fixed feature order for the shared model.
-FEATURE_NAMES = tuple(
-    name
-    for side in SIDES
-    for name in (
-        f"{side}_avg_dev",
-        f"{side}_peak_dev",
-        f"{side}_avg_rel_last",
-        f"{side}_peak_rel_last",
-        f"{side}_avg_rel_mean",
-        f"{side}_peak_rel_max",
-        f"{side}_avg_z",
-        f"{side}_peak_z",
-        f"{side}_avg_slope_z",
-        f"{side}_high_ratio_mean",
-        f"{side}_high_high_ratio_mean",
-    )
-) + ("both_sides_rising", "running_coverage_mean", "running_coverage_min")
-
-
-def feature_names() -> list[str]:
-    return list(FEATURE_NAMES)
-
-
-def compute_unit_baselines(samples: list[dict]) -> dict[str, dict]:
-    """Compute per-unit normal median/MAD for each side from training normal windows.
-
-    Raises if a unit has fewer than MIN_BASELINE_SAMPLES training normal windows
-    or a degenerate MAD, because the normalization would be unstable.
-    """
-    by_unit = defaultdict(list)
-    for sample in samples:
-        if sample["label"] != 0:
-            continue
-        by_unit[sample["unit"]].append(sample["features"])
-    if not by_unit:
-        raise RuntimeError("训练集没有正常窗口,无法计算机组基准")
-    baselines = {}
-    for unit, features in sorted(by_unit.items()):
-        if len(features) < MIN_BASELINE_SAMPLES:
-            raise RuntimeError(
-                f"{unit} 训练正常窗口仅 {len(features)} 个,少于 {MIN_BASELINE_SAMPLES} 个,无法计算稳定基准"
-            )
-        unit_baseline = {"normal_samples": len(features)}
-        for side in SIDES:
-            for kind in ("avg", "peak"):
-                key = f"{side}_{kind}_mean"
-                values = [item[key] for item in features if item.get(key) is not None]
-                if len(values) < MIN_BASELINE_SAMPLES:
-                    raise RuntimeError(f"{unit} {key} 有效正常窗口不足,无法计算稳定基准")
-                center = float(median(values))
-                mad = float(median([abs(value - center) for value in values]))
-                if mad <= 0:
-                    raise RuntimeError(f"{unit} {key} 的 MAD=0,无法归一化")
-                unit_baseline[f"{side}_{kind}"] = {"median": center, "mad": mad}
-        baselines[unit] = unit_baseline
-    return baselines
-
-
-def generalized_features(sample: dict, baselines: dict[str, dict]) -> dict:
-    features = sample["features"]
-    unit = sample["unit"]
-    if unit not in baselines:
-        raise KeyError(f"缺少 {unit} 的机组基准")
-    unit_baseline = baselines[unit]
-    result = {}
-    for side in SIDES:
-        avg = unit_baseline[f"{side}_avg"]
-        peak = unit_baseline[f"{side}_peak"]
-        avg_mean = features.get(f"{side}_avg_mean")
-        peak_mean = features.get(f"{side}_peak_mean")
-        avg_slope = features.get(f"{side}_avg_slope")
-        result[f"{side}_avg_dev"] = features.get(f"{side}_avg_baseline_deviation")
-        result[f"{side}_peak_dev"] = features.get(f"{side}_peak_baseline_deviation")
-        result[f"{side}_avg_rel_last"] = features.get(f"{side}_avg_rel_last")
-        result[f"{side}_peak_rel_last"] = features.get(f"{side}_peak_rel_last")
-        result[f"{side}_avg_rel_mean"] = features.get(f"{side}_avg_rel_mean")
-        result[f"{side}_peak_rel_max"] = features.get(f"{side}_peak_rel_max")
-        result[f"{side}_avg_z"] = (avg_mean - avg["median"]) / avg["mad"] if avg_mean is not None else None
-        result[f"{side}_peak_z"] = (peak_mean - peak["median"]) / peak["mad"] if peak_mean is not None else None
-        result[f"{side}_avg_slope_z"] = avg_slope / avg["mad"] if avg_slope is not None else None
-        result[f"{side}_high_ratio_mean"] = features.get(f"{side}_high_ratio_mean")
-        result[f"{side}_high_high_ratio_mean"] = features.get(f"{side}_high_high_ratio_mean")
-    result["both_sides_rising"] = features.get("both_sides_rising")
-    result["running_coverage_mean"] = features.get("running_coverage_mean")
-    result["running_coverage_min"] = features.get("running_coverage_min")
-    return result
-
-
-def feature_matrix(samples: list[dict], baselines: dict[str, dict]) -> list[dict]:
-    return [generalized_features(sample, baselines) for sample in samples]

+ 0 - 90
backend/model_assistant_generalized/infer_and_combine.py

@@ -1,90 +0,0 @@
-#!/usr/bin/env python3
-"""Generate retrospective model-assistance outputs without changing rule CSVs."""
-from __future__ import annotations
-
-import argparse
-import csv
-import importlib
-import json
-from pathlib import Path
-
-import joblib
-import numpy as np
-from sklearn.neighbors import NearestNeighbors
-
-
-def feature_module(model_type: str):
-    """Pressure and vibration use independent feature modules."""
-    return importlib.import_module(
-        "generalized_features" if model_type == "pressure" else "generalized_features_vibration"
-    )
-
-
-def load_samples(directory: Path) -> list[dict]:
-    return [json.loads(path.read_text(encoding="utf-8")) for path in sorted(directory.glob("*.json"))]
-
-
-def matrix(feature_dicts: list[dict], names: list[str]) -> np.ndarray:
-    return np.asarray([[item.get(name) for name in names] for item in feature_dicts], dtype=float)
-
-
-def combined_result(rule_label: int, probability: float, anomaly_similarity: float, threshold: float) -> str:
-    model_support = probability >= threshold or anomaly_similarity >= 0.70
-    if rule_label == 1 and model_support:
-        return "模型辅助确认异常"
-    if rule_label == 1:
-        return "保留规则异常,模型意见不一致,人工复核"
-    if model_support:
-        return "规则正常但模型疑似异常,继续观察"
-    return "正常"
-
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description="生成PKS规则与模型的历史联合辅助确认结果")
-    parser.add_argument("model_type", choices=("pressure", "vibration"))
-    parser.add_argument("--datasets-dir", default=str(Path(__file__).resolve().parent / "datasets"))
-    parser.add_argument("--models-dir", default=str(Path(__file__).resolve().parent / "models"))
-    parser.add_argument("--output-dir", default=str(Path(__file__).resolve().parents[2] / "cache" / "pks_scanner" / "model_assistant_generalized"))
-    args = parser.parse_args()
-    gf = feature_module(args.model_type)
-    dataset = Path(args.datasets_dir) / args.model_type
-    bundle = joblib.load(Path(args.models_dir) / f"{args.model_type}_model.joblib")
-    metadata = json.loads((Path(args.models_dir) / f"{args.model_type}_metadata.json").read_text(encoding="utf-8"))
-    samples = load_samples(dataset / "candidates")
-    training = load_samples(dataset / "train")
-    baselines = bundle["unit_baselines"]
-    x = bundle["imputer"].transform(matrix(gf.feature_matrix(samples, baselines), bundle["feature_names"]))
-    x_train = bundle["imputer"].transform(matrix(gf.feature_matrix(training, baselines), bundle["feature_names"]))
-    probabilities = bundle["model"].predict_proba(x)[:, 1]
-    scaled = bundle["knn"].named_steps["scale"].transform(x)
-    train_scaled = bundle["knn"].named_steps["scale"].transform(x_train)
-    anomaly_indexes = [index for index, sample in enumerate(training) if sample["label"] == 1]
-    if not anomaly_indexes:
-        parser.error("训练集没有异常样本,无法建立异常相似性索引")
-    anomaly_neighbors = NearestNeighbors(n_neighbors=1).fit(train_scaled[anomaly_indexes])
-    distances, nearest_indexes = anomaly_neighbors.kneighbors(scaled)
-    rows = []
-    for index, sample in enumerate(samples):
-        nearest = training[anomaly_indexes[int(nearest_indexes[index][0])]]
-        anomaly_similarity = float(1 / (1 + distances[index][0]))
-        rule_label = int(sample["label"])
-        rows.append({
-            "机组": sample["unit"], "批次": sample["batch"], "窗口开始小时": sample["start_hour"], "窗口结束小时": sample["end_hour"],
-            "样本编号": sample["sample_id"], "现有规则结果": "异常" if rule_label else "正常",
-            "规则阶段编号": sample.get("rule_stage_id") or "", "模型异常概率": round(float(probabilities[index]), 6),
-            "最相似历史异常样本": nearest["sample_id"], "历史异常相似度": round(anomaly_similarity, 6),
-            "联合判断": combined_result(rule_label, probabilities[index], anomaly_similarity, metadata["threshold"]),
-        })
-    rows.sort(key=lambda row: (int(row["机组"][0]), row["窗口结束小时"], row["批次"]))
-    output = Path(args.output_dir)
-    output.mkdir(parents=True, exist_ok=True)
-    name = "压力模型辅助确认.csv" if args.model_type == "pressure" else "振动模型辅助确认.csv"
-    with (output / name).open("w", encoding="utf-8-sig", newline="") as handle:
-        writer = csv.DictWriter(handle, fieldnames=list(rows[0]))
-        writer.writeheader()
-        writer.writerows(rows)
-    print(f"{args.model_type} 联合辅助结果:{len(rows):,} 行,输出:{output / name}")
-
-
-if __name__ == "__main__":
-    main()

+ 0 - 185
backend/model_assistant_generalized/split_datasets.py

@@ -1,185 +0,0 @@
-#!/usr/bin/env python3
-"""Split PKS candidate samples by whole event groups without leakage."""
-from __future__ import annotations
-
-import argparse
-import csv
-import json
-import random
-import shutil
-from collections import Counter, defaultdict
-from pathlib import Path
-
-SPLIT_NAMES = ("train", "validation", "test")
-
-
-def normalize_splits(values: list[float]) -> tuple[tuple[str, float], ...]:
-    total = sum(values)
-    if total <= 0:
-        raise ValueError("切分比例之和必须大于 0")
-    return tuple((name, value / total) for name, value in zip(SPLIT_NAMES, values))
-
-
-def load_manifest(path: Path) -> list[dict]:
-    with path.open(encoding="utf-8-sig", newline="") as handle:
-        return list(csv.DictReader(handle))
-
-
-def allocate(groups: list[tuple], splits: tuple[tuple[str, float], ...], assignments: dict, weight=None) -> None:
-    """Assign whole groups so one stratum approaches the target split ratios.
-
-    `weight` maps a group's members to its contribution to the target counts.
-    Defaults to member count; pass a constant to balance by group count instead.
-    """
-    if weight is None:
-        weight = len
-    weights = [weight(members) for _, members, _, _ in groups]
-    total = sum(weights)
-    targets = {name: total * ratio for name, ratio in splits}
-    counts = Counter()
-    for (group_id, members, _, _), group_weight in zip(groups, weights):
-        # Put each group in the currently most under-filled partition.
-        split = min(
-            (name for name, _ in splits),
-            key=lambda name: (counts[name] - targets[name], counts[name], name),
-        )
-        assignments[group_id] = split
-        counts[split] += group_weight
-
-
-def assign_groups(
-    rows: list[dict], splits: tuple[tuple[str, float], ...], seed: int, positive_basis: str = "samples"
-) -> dict[str, str]:
-    """Distribute whole event groups by seeded random sampling, stratified by label and unit.
-
-    Groups are the indivisible unit.  A one-sample event remains one group, even
-    when that means exact file-level target ratios are impossible.  Positive and
-    per-unit normal strata are sampled independently so every split keeps a
-    proportional share, and every split is guaranteed at least one independent
-    positive event group.
-
-    `positive_basis` controls how positive events are balanced: "samples" weights
-    each event by its window count, "groups" weights every event equally (useful
-    when a few long events would otherwise dominate a split).
-    """
-    grouped = defaultdict(list)
-    for row in rows:
-        grouped[row["event_group_id"]].append(row)
-    positives = []
-    normal_by_unit = defaultdict(list)
-    for group_id, members in grouped.items():
-        first = min(members, key=lambda row: row["start_hour"])
-        item = (group_id, members, first["start_hour"], first["unit"])
-        if first["label"] == "1":
-            positives.append(item)
-        else:
-            normal_by_unit[first["unit"]].append(item)
-
-    if len(positives) < 3:
-        raise RuntimeError("独立异常事件少于3个,无法同时建立训练、验证和测试正例")
-
-    assignments = {}
-    rng = random.Random(seed)
-
-    # Positive event groups are sampled globally: a unit with only a few events
-    # cannot independently supply all three partitions.
-    rng.shuffle(positives)
-    positive_weight = (lambda members: 1) if positive_basis == "groups" else None
-    allocate(positives, splits, assignments, weight=positive_weight)
-    by_split = defaultdict(list)
-    for group_id, members, _, _ in positives:
-        by_split[assignments[group_id]].append((group_id, members))
-    for name, _ in splits:
-        if by_split[name]:
-            continue
-        donor = max((other for other, _ in splits), key=lambda other: len(by_split[other]))
-        group_id, members = by_split[donor].pop()
-        assignments[group_id] = name
-        by_split[name].append((group_id, members))
-
-    for unit in sorted(normal_by_unit):
-        groups = normal_by_unit[unit]
-        rng.shuffle(groups)
-        allocate(groups, splits, assignments)
-        if not any(assignments[group_id] == "train" for group_id, _, _, _ in groups):
-            raise RuntimeError(f"{unit} 的正常事件组没有进入训练集,无法计算该机组基准")
-    return assignments
-
-
-def write_split(dataset: Path, splits: tuple[tuple[str, float], ...], seed: int, positive_basis: str) -> dict:
-    manifest = load_manifest(dataset / "manifest.csv")
-    candidates = [row for row in manifest if row["label"] in {"0", "1"}]
-    assignments = assign_groups(candidates, splits, seed, positive_basis)
-    for split, _ in splits:
-        target = dataset / split
-        if target.exists():
-            shutil.rmtree(target)
-        target.mkdir()
-
-    for row in candidates:
-        split = assignments[row["event_group_id"]]
-        source = dataset / "candidates" / f"{row['sample_id']}.json"
-        target = dataset / split / source.name
-        target.hardlink_to(source)
-        row["split"] = split
-
-    fields = list(manifest[0])
-    with (dataset / "manifest.csv").open("w", encoding="utf-8-sig", newline="") as handle:
-        writer = csv.DictWriter(handle, fieldnames=fields)
-        writer.writeheader()
-        writer.writerows(manifest)
-
-    summary = defaultdict(Counter)
-    group_summary = defaultdict(Counter)
-    group_first = {}
-    for row in candidates:
-        split = row["split"]
-        summary[split][f"samples_{row['label_name']}"] += 1
-        summary[split]["samples_total"] += 1
-        group_first.setdefault(row["event_group_id"], row)
-    for row in group_first.values():
-        group_summary[row["split"]][f"groups_{row['label_name']}"] += 1
-        group_summary[row["split"]]["groups_total"] += 1
-    return {
-        "dataset": dataset.name,
-        "sample_counts": {split: dict(summary[split]) for split, _ in splits},
-        "group_counts": {split: dict(group_summary[split]) for split, _ in splits},
-        "excluded_samples": sum(row["label"] == "" for row in manifest),
-        "target_ratios": {name: round(ratio, 4) for name, ratio in splits},
-        "seed": seed,
-        "positive_event_rule": "positive event groups are sampled globally with a fixed seed; every split keeps at least one independent positive event",
-        "rule": "event_group_id is never split across train, validation, and test",
-    }
-
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description="按完整事件组切分 PKS 模型样本")
-    parser.add_argument("--datasets-dir", default=str(Path(__file__).resolve().parent / "datasets"))
-    parser.add_argument("--model-types", nargs="+", choices=("pressure", "vibration"), default=["pressure", "vibration"])
-    parser.add_argument(
-        "--split-ratios", nargs=3, type=float, default=[75.0, 15.0, 10.0],
-        metavar=("TRAIN", "VALIDATION", "TEST"),
-        help="训练/验证/测试目标比例,任意正数,内部按和归一化(默认 75 15 10)",
-    )
-    parser.add_argument("--seed", type=int, default=20260921, help="事件组随机分配种子")
-    parser.add_argument(
-        "--positive-basis", choices=("samples", "groups"), default="samples",
-        help="异常事件分配依据:samples 按窗口数(默认),groups 每个事件等权",
-    )
-    args = parser.parse_args()
-    if any(value <= 0 for value in args.split_ratios):
-        parser.error("切分比例必须为正数")
-    try:
-        splits = normalize_splits(args.split_ratios)
-    except ValueError as error:
-        parser.error(str(error))
-    datasets = Path(args.datasets_dir)
-    result = {name: write_split(datasets / name, splits, args.seed, args.positive_basis) for name in args.model_types}
-    report = datasets / "split_report.json"
-    report.write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
-    print(json.dumps(result, ensure_ascii=False, indent=2))
-    print(f"切分报告:{report}")
-
-
-if __name__ == "__main__":
-    main()

+ 0 - 194
backend/model_assistant_generalized/train_model.py

@@ -1,194 +0,0 @@
-#!/usr/bin/env python3
-"""Train one XGBoost PKS assistant model without reading the test set."""
-from __future__ import annotations
-
-import argparse
-import importlib
-import json
-from collections import defaultdict
-from pathlib import Path
-
-import joblib
-import numpy as np
-from sklearn.impute import SimpleImputer
-from sklearn.metrics import average_precision_score, precision_recall_curve
-from sklearn.neighbors import KNeighborsClassifier
-from sklearn.pipeline import Pipeline
-from sklearn.preprocessing import StandardScaler
-from xgboost import XGBClassifier
-from xgboost.callback import TrainingCallback
-
-
-def feature_module(model_type: str):
-    """Pressure and vibration use independent feature modules."""
-    return importlib.import_module(
-        "generalized_features" if model_type == "pressure" else "generalized_features_vibration"
-    )
-
-
-class ProgressCallback(TrainingCallback):
-    """Print round, learning rate, and train/validation loss for each boosting round."""
-
-    def __init__(self, rounds: int, learning_rate: float) -> None:
-        self.rounds = rounds
-        self.learning_rate = learning_rate
-
-    def after_iteration(self, model, epoch: int, evals_log):
-        losses = []
-        for dataset, metrics in evals_log.items():
-            for name, values in metrics.items():
-                if values:
-                    losses.append(f"{dataset}-{name}={values[-1]:.5f}")
-        print(
-            f"[round {epoch + 1:>4}/{self.rounds}] lr={self.learning_rate:.4f} " + " ".join(losses),
-            flush=True,
-        )
-        return False
-
-
-def load_samples(directory: Path) -> list[dict]:
-    return [json.loads(path.read_text(encoding="utf-8")) for path in sorted(directory.glob("*.json"))]
-
-
-def matrix(feature_dicts: list[dict], names: list[str]) -> np.ndarray:
-    return np.asarray([[item.get(name) for name in names] for item in feature_dicts], dtype=float)
-
-
-def stratified_normal_sample(samples: list[dict], max_ratio: int, seed: int) -> list[dict]:
-    positives = [sample for sample in samples if sample["label"] == 1]
-    normals = [sample for sample in samples if sample["label"] == 0]
-    limit = len(positives) * max_ratio
-    if len(normals) <= limit:
-        return samples
-    buckets = defaultdict(list)
-    for sample in normals:
-        month = sample["start_hour"][:7]
-        buckets[(sample["unit"], sample["batch"], month)].append(sample)
-    rng = np.random.default_rng(seed)
-    selected = []
-    # Round-robin quotas preserve unit/month coverage rather than favoring the
-    # longest uninterrupted running cycle.
-    keys = sorted(buckets)
-    quota, remainder = divmod(limit, len(keys))
-    for index, key in enumerate(keys):
-        bucket = buckets[key]
-        take = min(len(bucket), quota + (1 if index < remainder else 0))
-        selected.extend(bucket[index] for index in rng.choice(len(bucket), size=take, replace=False))
-    if len(selected) < limit:
-        leftovers = [sample for sample in normals if sample not in selected]
-        need = min(limit - len(selected), len(leftovers))
-        selected.extend(leftovers[index] for index in rng.choice(len(leftovers), size=need, replace=False))
-    return positives + selected
-
-
-def choose_threshold(y: np.ndarray, probabilities: np.ndarray) -> tuple[float, dict]:
-    precision, recall, thresholds = precision_recall_curve(y, probabilities)
-    f1 = 2 * precision * recall / np.maximum(precision + recall, 1e-12)
-    index = int(np.nanargmax(f1[:-1])) if len(thresholds) else 0
-    threshold = float(thresholds[index]) if len(thresholds) else 0.5
-    return threshold, {
-        "threshold_selection": "max_validation_f1",
-        "threshold": threshold,
-        "validation_precision": float(precision[index]),
-        "validation_recall": float(recall[index]),
-        "validation_f1": float(f1[index]),
-        "validation_pr_auc": float(average_precision_score(y, probabilities)),
-    }
-
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description="训练 PKS 压力或振动模型,不读取测试集")
-    parser.add_argument("model_type", choices=("pressure", "vibration"))
-    parser.add_argument("--datasets-dir", default=str(Path(__file__).resolve().parent / "datasets"))
-    parser.add_argument("--models-dir", default=str(Path(__file__).resolve().parent / "models"))
-    sampling = parser.add_mutually_exclusive_group()
-    sampling.add_argument("--normal-ratio", type=int, default=None)
-    sampling.add_argument("--all-normal", action="store_true", help="使用全部正常训练样本,不下采样、不复制")
-    parser.add_argument("--seed", type=int, default=20260921)
-    parser.add_argument("--rounds", type=int, default=250)
-    parser.add_argument("--early-stopping-rounds", type=int, default=30)
-    args = parser.parse_args()
-    if args.normal_ratio is not None and args.normal_ratio < 1:
-        parser.error("--normal-ratio 必须大于等于 1")
-    if args.rounds < 1 or args.early_stopping_rounds < 1:
-        parser.error("--rounds 和 --early-stopping-rounds 必须大于等于 1")
-    ratio = args.normal_ratio if args.normal_ratio is not None else (4 if args.model_type == "pressure" else 5)
-    gf = feature_module(args.model_type)
-    dataset = Path(args.datasets_dir) / args.model_type
-    train_all = load_samples(dataset / "train")
-    validation = load_samples(dataset / "validation")
-    if not train_all or not validation:
-        parser.error("训练或验证样本为空")
-    if not any(sample["label"] == 1 for sample in train_all) or not any(sample["label"] == 1 for sample in validation):
-        parser.error("训练和验证集都必须包含异常样本")
-    train = train_all if args.all_normal else stratified_normal_sample(train_all, ratio, args.seed)
-    baselines = gf.compute_unit_baselines(train_all)
-    names = gf.feature_names()
-    x_train, y_train = matrix(gf.feature_matrix(train, baselines), names), np.asarray([sample["label"] for sample in train])
-    x_validation, y_validation = matrix(gf.feature_matrix(validation, baselines), names), np.asarray([sample["label"] for sample in validation])
-    imputer = SimpleImputer(strategy="median")
-    x_train = imputer.fit_transform(x_train)
-    x_validation = imputer.transform(x_validation)
-    scale_pos_weight = float(np.sum(y_train == 0) / max(np.sum(y_train == 1), 1))
-    learning_rate = 0.05
-    print(
-        f"training: rounds={args.rounds}, learning_rate={learning_rate:.4f}, "
-        f"early_stopping_rounds={args.early_stopping_rounds}, samples={len(train):,}",
-        flush=True,
-    )
-    model = XGBClassifier(
-        n_estimators=args.rounds, max_depth=3, learning_rate=learning_rate,
-        subsample=0.8, colsample_bytree=0.8, min_child_weight=3,
-        objective="binary:logistic", eval_metric="logloss", scale_pos_weight=scale_pos_weight,
-        random_state=args.seed, n_jobs=4, early_stopping_rounds=args.early_stopping_rounds,
-        callbacks=[ProgressCallback(args.rounds, learning_rate)],
-    )
-    eval_set = [(x_train, y_train), (x_validation, y_validation)]
-    model.fit(x_train, y_train, eval_set=eval_set, verbose=False)
-    rounds_run = len(model.evals_result()["validation_0"]["logloss"])
-    stopped_early = rounds_run < args.rounds
-    print(
-        f"training complete: rounds_run={rounds_run}/{args.rounds}, "
-        f"best_iteration={model.best_iteration + 1}, best_validation_logloss={float(model.best_score):.5f}, "
-        f"early_stopped={'yes' if stopped_early else 'no'}",
-        flush=True,
-    )
-    validation_probabilities = model.predict_proba(x_validation)[:, 1]
-    threshold, validation_metrics = choose_threshold(y_validation, validation_probabilities)
-    similarity = Pipeline([("scale", StandardScaler()), ("knn", KNeighborsClassifier(n_neighbors=min(5, len(train)), weights="distance"))])
-    similarity.fit(x_train, y_train)
-    models = Path(args.models_dir)
-    models.mkdir(parents=True, exist_ok=True)
-    # The progress callback is defined in this script (module __main__); drop it
-    # before serialization so the artifact can be loaded by other entry points.
-    model.callbacks = []
-    joblib.dump(
-        {"model": model, "imputer": imputer, "knn": similarity, "feature_names": names, "unit_baselines": baselines},
-        models / f"{args.model_type}_model.joblib",
-    )
-    (models / f"{args.model_type}_unit_baselines.json").write_text(
-        json.dumps(baselines, ensure_ascii=False, indent=2), encoding="utf-8"
-    )
-    metadata = {
-        "model_type": args.model_type, "algorithm": "XGBoost binary classifier + KNN similarity",
-        "feature_version": "generalized_cycle_relative_v2",
-        "feature_names": names,
-        "dropped_absolute_features": list(gf.ABSOLUTE_FEATURES),
-        "unit_baselines": baselines,
-        "normal_sampling_strategy": "all_normal" if args.all_normal else "stratified_max_ratio",
-        "normal_to_anomaly_training_ratio_limit": None if args.all_normal else ratio,
-        "actual_normal_to_anomaly_training_ratio": float(np.sum(y_train == 0) / max(np.sum(y_train == 1), 1)),
-        "training_samples_before_sampling": len(train_all), "training_samples_after_sampling": len(train),
-        "training_normal": int(np.sum(y_train == 0)), "training_anomaly": int(np.sum(y_train == 1)),
-        "scale_pos_weight": scale_pos_weight, "validation_samples": len(validation),
-        "test_set_read": False, **validation_metrics,
-        "rounds_requested": args.rounds, "early_stopping_rounds_requested": args.early_stopping_rounds,
-        "rounds_run": rounds_run, "early_stopped": stopped_early,
-        "best_iteration": int(model.best_iteration), "best_score": float(model.best_score),
-    }
-    (models / f"{args.model_type}_metadata.json").write_text(json.dumps(metadata, ensure_ascii=False, indent=2), encoding="utf-8")
-    print(json.dumps(metadata, ensure_ascii=False, indent=2))
-
-
-if __name__ == "__main__":
-    main()

+ 4 - 3
models/README.md

@@ -1,8 +1,9 @@
 # PKS models —— 压力/振动算法与模型重构
 
-本目录把原先合并在一起的 PKS 故障算法与泛化模型拆开、规范化重排。原
-`backend/scan_pks_fault_evolution.py` 与 `backend/model_assistant_generalized/`
-保留不动,作为回退参照。
+本目录把原先合并在一起的 PKS 故障算法与泛化模型拆开、规范化重排,是本项目
+压力/振动算法与模型的唯一实现。原 `backend/model_assistant_generalized/` 已删除,
+其逻辑与文档已全部迁入本目录;`backend/scan_pks_fault_evolution.py`(合并版扫描器)
+保留作为历史参照。
 
 ## 目录
 

+ 3 - 3
models/common/modeling.py

@@ -1,8 +1,8 @@
 """Generic XGBoost training / evaluation / inference for both model types.
 
-Ports ``train_model.py``, ``evaluate_model.py`` and ``infer_and_combine.py`` from
-``backend/model_assistant_generalized``.  The only model-specific input is the
-feature module (``features_pressure`` or ``features_vibration``).
+Ports the original generalized model pipeline's training, evaluation and
+inference logic.  The only model-specific input is the feature module
+(``features_pressure`` or ``features_vibration``).
 """
 from __future__ import annotations
 

+ 2 - 2
models/common/splitting.py

@@ -1,7 +1,7 @@
 """Event-group dataset splitting shared by both models.
 
-Ported from ``backend/model_assistant_generalized/split_datasets.py`` so the
-held-out split is identical for a fixed seed.
+Ported from the original generalized model pipeline so the held-out split is
+identical for a fixed seed.
 """
 from __future__ import annotations
 

+ 3 - 3
models/common/windowing.py

@@ -1,8 +1,8 @@
 """Shared window construction for the pressure and vibration models.
 
-These helpers reproduce ``backend/model_assistant_generalized/build_datasets.py``
-and only expose the parts that are identical for both model types.  The model
-specific feature extraction is injected via the ``feature_fn`` argument.
+These helpers reproduce the original generalized model pipeline's dataset
+building and only expose the parts that are identical for both model types.
+The model specific feature extraction is injected via the ``feature_fn`` argument.
 """
 from __future__ import annotations
 

+ 2 - 0
models/markdown/README.md

@@ -7,5 +7,7 @@
 | `振动算法说明.md` | 振动异常段算法(`models/vibration/vibration_algorithm.py`) |
 | `振动模型结果.md` | 振动 XGBoost 泛化模型方法与 3h/6h 结果 |
 | `模型辅助确认方案.md` | 压力/振动共用的窗口、标签与规则联合判断约定 |
+| `跨机组泛化与样本扩充方案.md` | 跨机组泛化特征与样本扩充方案 |
+| `实时联合诊断方案.md` | 实时联合诊断与模型/规则联合输出方案 |
 
 原合并文档 `PKS润滑油压力与振动算法说明.md` 仍保留在项目根目录。

+ 2 - 2
models/markdown/压力模型结果.md

@@ -107,9 +107,9 @@
 
 ## 六、规则联合辅助输出
 
-- 3h:`cache/pks_scanner/model_assistant_generalized_789/压力模型辅助确认.csv`,13,805 行
+- 3h:`models/results/pressure/压力模型辅助确认.csv`,13,805 行
   - 模型辅助确认异常 1,012、保留规则异常人工复核 4、继续观察 122、正常 12,667
-- 6h:`cache/pks_scanner/model_assistant_generalized_789_6h/压力模型辅助确认.csv`,13,710 行
+- 6h:本轮未重新生成(`models/generalized/run_pipeline.py` 保留 `--window-hours 6` 参数化路径)
   - 模型辅助确认异常 1,203、继续观察 113、正常 12,394
 
 v2 下“继续观察”从 v1 的 11,000 降到约 120,说明模型对正常窗口的疑似标记大幅减少。

+ 2 - 2
backend/model_assistant_generalized/实时联合诊断方案.md

@@ -30,14 +30,14 @@
 ## 3. 特征一致性(必须)
 
 - 把 `build_datasets.py` 的压力特征函数(周期基线、近期 24h 基线、回撤、连续下降小时数等)抽成**共享模块**,训练与实时共用,避免口径漂移。
-- 机组基准(`*_z` 的 median/MAD)从 `models_*/pressure_unit_baselines.json` 加载。
+- 机组基准(`*_z` 的 median/MAD)从 `models/datas/artifacts/pressure/pressure_unit_baselines.json` 加载。
 - 周期基线按扫描器口径:周期前 3 个自然日每日油压中位数,再取其中位数。
 
 ## 4. 输出
 
 - 规则输出不变:`cache/pks_scanner/evolution/润滑油压力小时阶段段.csv`。
 - 模型确认动作:当前阶段先**写 CSV**(后续再讨论改为数据库字段/通知等形式),建议:
-  - 文件:`cache/pks_scanner/model_assistant_generalized_789/压力实时确认.csv`
+  - 文件:`models/results/pressure/压力实时确认.csv`
   - 唯一键:`机组 + 批次 + 窗口结束小时`;15 分钟 tick 内更新同一行,不重复追加。
   - 字段:机组、批次、窗口开始/结束小时、规则小时等级、阶段编号(若已形成)、模型异常概率、最相似历史异常样本、相似度、联合判断、原因。
   - 原因 = 规则触发依据 + 相对周期基线变化/回撤/连续下降小时数 + 模型概率 + 最相似异常样本。

+ 2 - 2
models/markdown/振动模型结果.md

@@ -107,9 +107,9 @@
 
 ## 七、规则联合辅助输出
 
-- 3h:`cache/pks_scanner/model_assistant_generalized_vibration_789/振动模型辅助确认.csv`,14,139 行
+- 3h:`models/results/vibration/振动模型辅助确认.csv`,14,139 行
   - 模型辅助确认异常 219、保留规则异常人工复核 12、继续观察 13、正常 13,895
-- 6h:`cache/pks_scanner/model_assistant_generalized_vibration_789_6h/振动模型辅助确认.csv`,13,908 行
+- 6h:本轮未重新生成(`models/vibration/run_pipeline.py` 保留 `--window-hours 6` 参数化路径)
 
 ## 八、限制
 

backend/model_assistant_generalized/跨机组泛化与样本扩充方案.md → models/markdown/跨机组泛化与样本扩充方案.md