What Hospitals Ask AI Device Vendors For: The Buyer Evidence Pack

A 510k answers FDA's question for a defined intended use. A hospital AI governance committee asks a different one. Will this tool work here, in our workflow and on our patients, and who keeps watching it after go-live? In 2026, several buyer groups wrote their questions down. Radiology now has an adopted practice parameter for imaging AI. A health system collaborative published a vendor disclosure framework in NEJM AI. The Coalition for Health AI (CHAI) released draft intake controls, and the Health Sector Coordinating Council (HSCC) proposed contract terms for third-party AI.

‍

Below is what these documents ask AI device vendors for, where the answers usually already sit in your design history file and technical documentation, and the five gaps to close first.

‍

Who is asking

• ACR-SIIM Practice Parameter for Imaging AI (adopted 2026). Sets out how radiology practices select, test locally, monitor and retire AI tools, and what they should get from manufacturers.
• HAIP AI Vendor Disclosure Framework (NEJM AI, April 2026). A Health AI Partnership guide listing what health systems should request and vendors should disclose, across five domains. It comes with a downloadable vendor assessment form.
• CHAI AI Intake Playbook (October 2026). 57 draft intake controls in 11 domains, now going to pilot.
• HSCC Third-Party AI Risk and Supply Chain Transparency Guide (April 2026). A procurement lifecycle from use-case justification to end of life, with proposed contract terms.
• Joint Commission and CHAI guidance on the Responsible Use of AI in Healthcare (September 2025). Asks hospitals to review vendor testing, validation and bias assessments and to test tools on local data. Since June 2026 it underpins a voluntary Joint Commission certification of healthcare organizations, not of products.

‍

You will not be in the room

The ACR-SIIM parameter expects governance members to be impartial and says "a manufacturer or developer of an AI solution would not be expected to participate in governance discussions related to the deployment of their product at the practice." The committee reads your materials without you. A pack that states its limitations up front reads better than one where the committee finds them itself.

‍

What they ask, and where the answer usually lives

Grouped by the question a committee is trying to answer. Most of these already exist in some form in your design history file, risk file, V&V reports, labeling and postmarket procedures.

‍

What the tool is

• Intended use, users, limitations, inclusion and exclusion criteria. Asked by HAIP and ACR-SIIM. Lives in your instructions for use and labeling.
• Regulatory status. HAIP also asks, for an unregulated tool, why it is not a device. Lives in your clearance letter or your regulatory determination.

‍

How well it works, and for whom

• Training data summary. Asked by ACR-SIIM and CHAI. Lives in your data management documentation.
• Internal and external evaluation results. Asked by ACR-SIIM and HAIP. Lives in your standalone and clinical performance reports.
• Subgroup performance and bias assessment. Asked by ACR-SIIM, HAIP, CHAI and the Joint Commission guidance. Lives in your performance reports and risk management file.
• Known risks and failure modes. Asked by HAIP and HSCC. Lives in your risk management file and labeling.

‍

Will it work here

• Support for local acceptance testing. ACR-SIIM asks practices to evaluate performance on local data against "Predefined metrics for acceptable AI performance," and for pixel-based AI to use data "from a variety of machine manufacturers, models and protocols." CHAI asks the vendor to validate performance "on a representative dataset for the customer's intended use prior to deployment." Your V&V protocols are the starting point.
• Integration, staffing and total cost. HAIP asks for interoperability, infrastructure, personnel needs and "complete cost estimates, including up-front, operational, and potential hidden expenses." This usually lives with sales and implementation, not regulatory.

‍

How it changes and who watches it

• Release notes, versioning and PCCP scope. Asked by ACR-SIIM, CHAI and HSCC. Lives in your change control records and PCCP. Our post on the PCCP as a procurement document covers this in detail.
• Monitoring thresholds and degradation response. Asked by HAIP, CHAI and HSCC. Lives in your postmarket surveillance plan, if you have a field monitoring plan at all.
• Adverse event reporting. HAIP asks for the protocol for reporting to the vendor, the hospital and regulators, and CHAI asks for a two-way process. Lives in your complaint handling and medical device reporting procedures.

‍

Data, security and contract terms

• Data use, retention, ownership and business discontinuation planning. Asked by HAIP, HSCC and the Joint Commission guidance.
• Security and an AI bill of materials. HSCC asks for an SBOM and an AI-BOM. Your cybersecurity documentation covers the first. Our SBOM guide covers what goes in it.
• Contract terms. HSCC proposes update notice periods, incident notice, audit rights and 12 to 18 months' end-of-life notice.

‍

Five gaps to close first

1. PPV at the buyer's prevalence. Committees turn sensitivity and specificity into positive predictive value at their own prevalence, and into expected flags per day. Do it for them at two or three realistic prevalences, and say which operating point the device uses.

‍

2. Failures in the denominator. Report how often the tool cannot produce a result (ungradable, rejected or failed inputs) and include those cases in the headline numbers or next to them. A July 2026 evidence review of autonomous diabetic retinopathy AI, prepared for New York Medicaid, noted that studies excluding images the AI judged to be of insufficient quality "risk inflating estimates of device accuracy."

‍

3. Subgroup tables that match the buyer. Break results down by the groups a hospital will check, such as age bands, sex, and scanner manufacturer, model and protocol. If a group is too small to report, say so rather than leaving it out.

‍

4. A ready-to-run local acceptance test. Give sites a short protocol with suggested metrics and thresholds, the case mix to sample, and a data extraction template. If a site wants a prospective or silent-mode evaluation, settle the IRB and investigational device questions before it starts.

‍

5. A completed model card, plus what it leaves out. In CHAI's draft playbook, a completed Applied Model Card fully satisfies 12 of the 57 controls and partly addresses 29 more. The HAIP authors note that existing transparency tools, including the Model Facts Label, do not fully meet the information needs of vendor engagement teams. Pair the card with the cost, integration and lifecycle answers above.

‍

What this means for developers

You already have most of it. The work is translation. Pull the answers from the design history file, risk file and V&V reports into one pack written for a reader who was not in your submission.

‍

Build it once and version it. Keep the pack in document control and update it with each release, so the performance numbers, subgroup tables and PCCP summary match the version a hospital runs.

‍

Fill in the HAIP form yourself. HAIP publishes its vendor assessment form. Completing it before a buyer sends it shows you where your gaps are.

‍

Keep claims inside the cleared indications. A pack written for sales can drift into workflow or outcome claims your evidence and clearance do not support. Have regulatory review it like labeling.

‍

What to watch out for

• None of this is a regulation. These are a practice parameter, guides, a published framework and draft controls. Each hospital asks in its own way.
• Imaging leads. The ACR-SIIM parameter is specific to radiology practices. Other specialties are moving more slowly, though the general frameworks apply to any clinical AI.
• Protect what is proprietary. Summaries usually serve. Share more under a confidentiality agreement when a committee needs it.
• A good pack informs the decision. It does not make it. Committees weigh clinical need, workflow and cost alongside your evidence.

‍

The bigger picture

Clearance, purchase and payment are three separate evidence gates. FDA's draft guidance on AI-enabled devices emphasizes independent test data and human-device team performance. Buyers now ask for local testing and monitoring. Payers ask whether care changes. CMS's proposed RAPID coverage pathway states that "FDA market authorization alone does not entitle that technology to Medicare coverage." Our post on the CPT 2027 AI codes covers the coding side. This post is part of a series on what each gate asks for.

‍

How Cosm Can Help

Cosm helps AI device teams turn their design history file and technical documentation into a buyer evidence pack, including performance summaries at realistic prevalence, subgroup tables, local acceptance test protocols, monitoring plans and PCCP summaries. Talk to Cosm about what your next customer will ask.