这一页不是宣传材料。每张卡上的判决,都是本仓库里的案件文件送进 GateEngine 跑出来的原始结果——含命中的规则编号、等级、理由原文。你可以在本地用一条命令重跑,得到与页面逐字相同的结果。
python benchmarks/build_demo_page.py · 校验 python benchmarks/build_demo_page.py --check(不一致则 CI 失败)对照物不是商用产品,是一个"最可能被写出来的偷懒实现":JSON 里出现 banned 关键词就 reject、出现 requires_submission 就 hold、其余放行。下面是 benchmarks/adversarial_baseline.py 此刻的真实输出(逐字)。
| 实现 | 与预期一致的探针 | 说明 |
|---|
朴素法错在两个相反方向:该拦的放行(空证据、未知变更类型、字符串 "yes" 被当成等价声明)、该放的拦住(PCCP 绿色案件因为 JSON 里出现了键名 affects_intended_use(值为 false)而被误判 hold)。这正是"键名匹配"与"语义判定"的差别。
判定器给结论不点条款,就和"模型的意见"无法区分。下表把每条规则落到依据上,并分列两种来源:LAW 是我们读过原文的法条/法规/MDCG·CAMD 通知/FDA 指南;DOCTRINE 是本家规则(L2 证据门、单辖区声明),明示为家法,不许冒充法条。家法占比由测试硬性封顶,防止它悄悄长成多数。
| 判定器 | 规则 | 条款依据 | 类型 |
|---|
复核方式:python benchmarks/clause_matrix.py 双向检查——有规则无依据会失败,有依据但对应规则不存在也会失败(死引用正是引用表烂掉的方式)。可读版见 docs/methods/Clause_Matrix.md。
没写进规则的条款,如果不列出来,就和"没想到"无法区分。下面每一条都是明知存在、故意没编码的,附类型与理由;数据在 docs/methods/coverage_gaps.json,由测试校验字段、状态枚举、理由长度与三辖区覆盖。
| 辖区 | 条款 | 状态 | 理由 |
|---|---|---|---|
| NMPA | 47号令 第七十九条 — 其他变化,30 日内备案 | out_of_scope | The table judges which route applies, not whether the clock was met. No days_since_change field exists, so a late filing cannot be detected — adding one would turn a routing judge into a compliance clock, and the caller's timestamps would h… |
| NMPA | 47号令 第七十九条 第一款 — 实质性变化且可能影响安全有效 | deferred | The substantive judgement is carried by the caller's safety_effectiveness_assessed flag; the table accepts the conclusion, not the argument behind it. |
| NMPA | 各省对一类器械备案的实施细则 | out_of_scope | Provincial implementing rules differ and are outside the central-law basis we verified; encoding them would mean picking provinces. |
| EU | MDR Art.120(3) as amended by Regulation (EU) 2023/607 — 2027-12-31 / 2028-12-31 endpoints | out_of_scope | The table judges significance, not transitional validity. Validity depends on certificate dates the case JSON does not carry. |
| EU | MDCG 2020-3 rev.1 — software change decision support | deferred | Chart C is guidance on how to reach the significance conclusion; the rules encode the criteria (design/intended-purpose, cumulative), not the flowchart. |
| FDA | FD&C §515C(a)(3)/(b)(3) — labeling, notification if the device does not function as intended, performance requirements | optional_by_law | The statute says the Secretary *may* require these; a plan without a notification clause is not thereby unlawful. |
| FDA | FDA PCCP guidance (4 Dec 2024) — continuous learning / automatic update Q-Sub discussion | optional_by_law | A recommendation, not an obligation; making it a rule would manufacture a requirement the guidance does not impose. |
| FDA | 21 CFR 892.2055 之外的 PCCP 器械分类特殊控制 | deferred | Each classification regulation carries its own special controls; only 892.2055 has been read end to end so far. |
| FDA | 510(k) 变更分类之外的判据(如 510(k) 摘要与 SE 论证细则) | out_of_scope | This table answers 'is a new submission needed', not 'is the submission adequate'. |
复核方式:python -m unittest tests.test_contract.TestCoverageGaps -v。注意本条的限制:没列出的缺口与没被承认的缺口,在页面上无从区分。
这 18 个案件不是外部质疑,是我们自己造的反例,而且每一个都成功过:它们曾经拿到 go。共同形状只有一句——规则等在某个中间产物上,于是把这个产物的"缺席"读成了"许可"。等的东西可以是一个布尔标记、一个词表取值、一个下游分类,或者干脆是一个没人填的字段。
| 域 | 案件 | 文件 | 判决 | 现在拦它的规则 |
|---|---|---|---|---|
| 510(k) 变更分类 | FDA-510K-RED-01 | demo1/cases/510k_red_team_label_expansion.json | hold | R-MAJ-09-label-expansion-new-claim |
| 510(k) 变更分类 | DEMO1-RED-01 | demo1/cases/case01_red_pma_process.json | reject | R-CRIT-01-class1-recall |
| 510(k) 变更分类 | DEMO1-RED-02 | demo1/cases/case02_red_material.json | reject | R-CRIT-01-class1-recall |
| 上市后监测处置 | PM-RED-01 | demo1/cases_postmarket/pm01_class1_software.json | reject | PM-CRIT-01-class1-recall |
| 上市后监测处置 | FDA-PM-RED-01 | demo1/cases_postmarket/pm04_red_team_death_without_recall.json | hold | PM-MAJ-04-reportable-event-without-recall |
| PCCP 变更范围 | PCCP-RED-03 | demo1/cases_pccp/pccp03_red_intended_use.json | reject | P-CRIT-02-intended-use-change |
| PCCP 变更范围 | PCCP-RED-04 | demo1/cases_pccp/pccp04_red_criteria_not_met.json | reject | P-CRIT-03-acceptance-criteria-not-met |
| PCCP 变更范围 | PCCP-RED-06 | demo1/cases_pccp/pccp06_red_se_to_predicate_broken.json | reject | P-CRIT-04-se-to-predicate-broken |
| PCCP 变更范围 | PCCP-RED-07 | demo1/cases_pccp/pccp07_red_modified_version_as_predicate.json | reject | P-CRIT-05-modified-version-used-as-predicate |
| PCCP 变更范围 | FDA-PCCP-RED-01 | demo1/cases_pccp/pccp08_red_team_absent_special_controls.json | hold | P-MAJ-03-validation-not-documented |
| EU MDR 重大变更 | MDR-RED-04 | demo1/cases_mdr/mdr04_red_open_fsca.json | reject | M-CRIT-01-change-during-open-fsca |
| EU MDR 重大变更 | MDR-RED-05 | demo1/cases_mdr/mdr05_red_declared_non_significant.json | reject | M-CRIT-02-significant-declared-non-significant |
| EU MDR 重大变更 | MDR-RED-01 | demo1/cases_mdr/mdr08_red_team_acceptance_scope_expansion.json | hold | M-MAJ-01-intended-purpose |
| EU MDR 重大变更 | MDR-RED-02 | demo1/cases_mdr/mdr09_red_team_acceptance_scope_classification.json | hold | M-MAJ-03-safety-or-performance |
| NMPA 变更注册 | NMPA-RED-04 | demo1/cases_nmpa/nmpa04_red_licence_as_filing.json | reject | N-CRIT-02-licence-item-filed-as-registration |
| NMPA 变更注册 | NMPA-RED-05 | demo1/cases_nmpa/nmpa05_red_level1_recall.json | reject | N-CRIT-03-level1-recall-open |
| NMPA 变更注册 | NMPA-RED-01 | demo1/cases_nmpa/nmpa08_red_team_class1_registration_route.json | hold | N-MAJ-15-class1-no-registration-route |
| NMPA 变更注册 | NMPA-RED-02 | demo1/cases_nmpa/nmpa09_red_team_class1_first_registration.json | hold | N-MAJ-15-class1-no-registration-route |
复核方式:每个文件都带 expected_verdict,--all 在修复被回退时直接变红;逐条复跑与"过去会怎样"的说明在 docs/methods/RedTeam_Probes.md。三种能继续用的攻击形状也写在那份文档里:删字段而不是设为否定值、用词表认识的值而不是词表拒绝的值、喂规则所键的中间产物而不是它要证明的事实。
对外说"类目都怎样"很容易,说过就算的代价是外人无法验证。下表每行都写清:我们这侧用什么命令settle、对方那侧我们到底查没查。只有 observed 且给出链接的,才能当作对他人的事实陈述;其余是推断或未查。
| 维度 | 我们的主张(可复核) | 证据强度 |
|---|---|---|
| does the verdict call a model | Verdicts are pure functions: no network, no model, byte-identical across reruns.benchmarks/verdict_cost.py | inferred— |
| does each rule name its legal basis | All 60 rules carry a LAW or DOCTRINE label, with DOCTRINE capped at 12.benchmarks/clause_matrix.py | observedsource |
| is the evaluation report generated or written | The report is emitted by the evaluator and a check fails when its numbers disagree with the repo.benchmarks/eval_method_note.py | unverified— |
| are coverage gaps published | Nine intentionally unencoded clauses are published with a status and a reason.docs/methods/coverage_gaps.json | observedsource |
| can an outsider re-run the claims | Twenty checks run from a fresh clone with no credentials.benchmarks/eval_method_note.py | unverified— |
| are the claims enforced in public CI | 58 contract assertions run on every push, including the three laws.tests/test_contract.py | unverified— |
| is the scope limited to enumerated categories | Anything outside the tables returns hold; coverage is the bottleneck, stated as such.lgd_engine/judges/base.py | inferred— |
复核方式:python benchmarks/edge_claims.py;可读版 docs/methods/Competitive_Edges.md。若 observed 占比过半,测试直接失败——那意味着我们在宣称一场没做过的调研。