How should clinical engineers evaluate AI and review its results?

Conceptual AI-generated illustration: robotic arm and human hand represent technology and human review.

Cover: conceptual AI-generated illustration following the Arkmeds identity. It does not depict an actual test or measurement.

AI can support clinical engineering in organizing records, drafting text and prioritizing work. To assess that support, define the task, how its output will be checked and who owns the decision. Fluent text or a high accuracy percentage alone does not establish that a tool is suitable for your operation.

Are measurement, integration and AI the same thing?

Imagine a work order combining analyzer readings, inventory information and a suggested conclusion. These have different origins. Identifying the origin helps investigate the right error instead of treating every populated field as a measurement.

OriginOutputWhat to check
Test instrumentResult of a defined measurement or simulationAsset, instrument, procedure, configuration and unit
Software integrationTransfer or association of dataDestination field, asset record and service
Classification modelCategory or priority according to a modelReference labels, errors by category and local relevance
Generative AISuggested text or summaryFidelity to the record, sources and added information

Review depends on the task. Transferring a reading and suggesting a conclusion are different steps. The applicable technical procedure still governs testing; software evaluation does not replace physical assessment.

Can AI invent a reference to a standard or study?

A well-formatted citation may describe a work that does not exist. Walters and Wilder (2023) checked 636 references in 84 reviews generated on 42 topics in the first week of April 2023. Table 3 reported 55% fabricated references for GPT-3.5 and 18% for GPT-4, with different denominators.

Fabricated references in the April 2023 study: GPT-3.5, 55% of 222 references; GPT-4, 18% of 414.
Own plot from Table 3 of Walters and Wilder (2023). Published rounded percentages from a historical bibliography task, not a current clinical engineering benchmark.
Model assessed in April 2023References examinedPublished fabricated percentage
GPT-3.522255%
GPT-441418%

These data concern a historical bibliography task, not current AI errors in clinical engineering or Arkmeds performance. The practical implication for technical documentation is to open the original source before incorporating a standard, procedure or tolerance. Confirm its existence, edition and relevance to the work. A real publication can also be cited with incorrect information.

The 2024 NIST AI 600-1 profile addresses confidently generated false content and recommends checking sources and citations and evaluating capabilities empirically. Asking the same tool to confirm its answer is not an independent check.

How can 95% accuracy miss every high-priority case?

Consider a wholly fictional example, involving no real devices or customers. A dataset contains 1,000 records: 950 previously assigned low priority and 50 high priority. A simplistic rule answers “low priority” for every record.

Wholly fictional matrix: all 950 low-priority records predicted low; all 50 high-priority records predicted low. Overall accuracy 95%; high-priority recall 0%.
Wholly fictional editorial example. Rows: reference category; columns: rule output. The percentages have different denominators, stated in the table and text.
Fictional reference categoryAnswer: lowAnswer: highTotal
Low priority9500950
High priority50050
Total1,00001,000

The rule gets 950 of 1,000 records right: overall accuracy is 95%. Yet it identifies none of the 50 high-priority records: recall for that category is 0%. Since it predicts no high-priority cases, precision for that category cannot be calculated with this denominator; a value should not be invented.

The example explains why a confusion matrix and results by category matter. Case frequency affects the overall metric. It is not a maintenance acceptance threshold: the institution must define criteria appropriate to the task and the consequences of errors.

What does a 2026 hospital study establish?

Ye and colleagues analyzed 9,924 devices at a hospital in Wenzhou, China: 8,967 in the preventive maintenance dataset and 957 in the corrective dataset. Models classified urgency using labels derived from statistical clustering. Published test accuracies were 93.79% and 86.46%, respectively.

Matching these labels does not establish agreement with an external clinical reference. The authors explicitly acknowledge that limitation. A subsequent quasi-experimental comparison involved 830 devices in the intervention campus and 849 in the control campus during 2025. Potential confounding prevents interpreting associations as causal proof or a savings promise. The study assessed neither Arkmeds nor an LLM drafting technical reports.

A useful question for a supplier is: “What reference does your percentage compare against, for which devices and period?”. A result can be valid for reproducing a training category while still not answering your team’s operational question.

How can a report suggestion be tested against team records?

Choose a limited task, such as identifying gaps or summarizing documented work. Preserve the original information and include cases with outstanding issues, missing fields and units that must not be changed. Use data authorized for that environment and record the model and configuration assessed.

In this fictional case WO-FIC-042, the record says “battery replaced; performance test outstanding”. A suggestion says “equipment passed and released”. It adds an unsupported conclusion. Review should retain the outstanding task and refer the conclusion to the technical owner, without supplying missing evidence through assumption.

Evaluation record fieldAdaptable example
Task and inputSummarize WO-FIC-042; original record version
Output and discrepancySuggestion added a pass without documented testing
TreatmentReject the conclusion; retain the outstanding task
OwnerReviewer and evidence consulted
Follow-upCorrect input or configuration and repeat affected cases

Measure omitted issues, unsupported added facts, unit or asset changes, rejected suggestions and review time. Include review in total time, not just generation. Preserve denominators: suggestions with at least one discrepancy / suggestions examined differs from total discrepancies because one suggestion may contain several.

How does CMMS help verify records?

An organized database makes completed work retrievable and software suggestions reviewable. The Arkmeds CMMS platform brings work orders and asset history together. Its mobile application supports field work without internet access and synchronizes records when connectivity returns, according to the features and configuration used.

The relevant benefit is retrieving evidence alongside the service record, preserving the distinction between measured results, transferred data and suggestions. Assess availability after synchronization, association with the correct asset and follow-up of corrections. Document organization features do not inherit the cited studies’ results or establish an AI model’s validity by themselves.

Which references help organize a pilot?

The 2023 NIST AI RMF 1.0 calls for assessment under conditions similar to deployment, documented measurements and operational monitoring. It is a voluntary, cross-sector framework; it does not certify clinical engineering software. On 6 October 2026, NIST states that version 1.0 is being revised.

The WHO’s 2025 inventory and maintenance guide addresses consistent data entry, migration validation and training during CMMS implementation (§7.5). These support preparation of the database. The evaluation plan and case presented here are adaptable editorial proposals, not official WHO or NIST forms.

What should happen when the model or workflow changes?

Record changes to version, configuration or data source and reassess affected cases. Compare equivalent periods, including service complexity and review time. Preserve ways to reject suggestions and correct records. Improvement on known examples warrants investigation; everyday use requires evidence of usefulness and continuing error monitoring.

Where can you explore data in clinical engineering further?

The Arkmeds channel’s Portuguese-language video, O papel da Engenharia Clínica na Era dos dados e da Inteligência Artificial, is additional discussion material. The data and conclusions in this article are tied to the primary sources below; they were not extracted from the presentation.

Sources and further reading

  1. Walters & Wilder (2023) — Fabrication and errors in the bibliographic citations generated by ChatGPT
  2. Ye et al. (2026) — A data-driven priority assessment and deployment framework for medical equipment maintenance in a tertiary hospital
  3. NIST AI 600-1 (July 2024) — Generative Artificial Intelligence Profile
  4. NIST AI RMF 1.0 (January 2023) — Artificial Intelligence Risk Management Framework
  5. NIST — AI Risk Management Framework, current status
  6. WHO (2025) — Inventory and maintenance management information system for medical devices
  7. Arkmeds — CMMS maintenance management platform
  8. Arkmeds — Palestra | O papel da Engenharia Clínica na Era dos dados e da Inteligência Artificial (Portuguese)

Research and source review: 6 October 2026. Own plots; matrix and WO-FIC-042 wholly fictional.

Original archive date: 2026-10-04. Editorial date reorganized on 8 October 2026.