THE FASCINATING ART AND SCIENCE OF ASSESSING CLINICAL JUDGMENT

Jun 28, 2022, 18:16 PM

<4-min. read> Assessing clinical judgment is the reason behind the creation of the Next Generation NCLEX. And it’s why ATI takes the topic very seriously.

COMPARING STUDENTS: HOW TO DEVELOP AND ASSESS THEIR CLINICAL JUDGMENT SKILLS

Student taking notes from lecture is less effective than interactive learning. ATI relies on a scientific approach to develop and maintain the highest-quality assessments for every subject category. But the assessment of clinical judgment has obviously gotten a great deal of attention in recent years based on the expected launch of the Next Generation NCLEX in April 2023.

ATI applies 4 basic processes in the development of all its assessments:

Applying scientific principles and research to create rigorous and trustworthy products and solutions
Instituting processes that meet or exceed testing-industry, publishing, and ethical standards for product and assessment development
Insisting on exceptional caliber originality, accuracy, and consistency
Setting rigorous expectations that ensure legal defensibility of products and accreditation status.

NOTE: During the National Simulation Study (2014), the NCSBN selected ATI ’s Content Mastery Series® exams as their measurement tool for student nursing knowledge.

Stay up-to-date on the Next Generation NCLEX.

We rely on a team of experts with decades of experience in five key disciplines:

RESEARCH & PSYCHOMETRICS
Master’s- and PhD-level experts who ensure the science behind ATI assessments.
CONTENT DEVELOPMENT
Master’s- and doctorate-prepared nurses with years of content-specific experience and teaching who have mastered item writing and a strong network of diverse external experts who develop creative and original content.
ASSESSMENT & LEARNING PRODUCTS
Test developers, editors, and designers who create products and assessments that meet an array of specifications.
MEDIA DEVELOPMENT
Skilled professionals who engage every type of learner with thoughtful applications using a variety of innovative media tools that allow access to those with disabilities.
TEST SECURITY
Experts who actively protect the validity and intellectual property of our products through preventive and correction efforts.

AN OVERVIEW: ASSESSMENT DEVELOPMENT & CLINICAL JUDGMENT

ATI Process of item development ATI maintains a rigorous process to establish and support the reliability, validity, and trustworthiness of its clinical judgment assessments, as well as its assessments on all other nursing-related topics. Establishing the validity of an assessment requires an ongoing collection of procedural and empirical evidence throughout the life of the product. It starts early, and it continues throughout the ATI Assessment Development Process:

Test-plan development
Item-level development
Test-level development
Test maintenance
Outcomes assessment.

Item development is a meticulous process. It consists of:

CONTINUOUS ITEM DEVELOPMENT

The steps involved include:

A strong network of diverse internal and external content experts
Content experts who consistently apply evidence-based practices to ensure content is valid and defensible
ATI item-writing guidelines that mirror NCLEX style of items.

ITEM REVIEW MEETINGS CONDUCTED BY TEST DEVELOPERS AND EXPERIENCED NURSE EDUCATORS

These meetings include:

A review of each item by two content experts
An analysis that ensures all content is relevant and tied to a blueprint task
Confirmation that all items meet the item-writing guidelines
A determination whether proctored items are ready for pretesting
A final review of items that didn’t fall into the statistical range originally set for them.

Three key steps align job analysis and the NCLEX test blueprint for both the NCSBN and ATI:

THE NCSBN (NATIONAL COUNCIL OF STATE BOARDS OF NURSING)

Performs a Job Analysis Study every three years (a task that showed the organization that clinical judgment was a key skill that new nurses perform daily)
Develops the NCLEX-RN and NCLEX-PN test plans
Researches aspects of nursing to aid in improved client care, such as the range and number of tasks expected of entry-level nurses.

ATI NURSING EDUCATION

Conducts a Test-Plan Development Meeting as needed.
Performs a crosswalk of the new NCSBN test plan and adjusts our current test plans to align.
Finalizes the ATI test plan.

Testing & equating

Is one student’s test harder than another’s? Why equating is important.

Total scores are adjusted between different assessment versions to compensate for (small) differences in difficulty.
Most ATI proctored assessments are equated, meaning that different versions of the assessment and their scores can be used interchangeably.

After an item is approved, it will then be pretested to ensure it is psychometrically sound.

Assessments are often comprised of both operational items (scored) and pretest items (unscored), such as the Special Research Section of the current version of the NGN. Operational items have already completed the full development process, are known to perform to an acceptable standard, and are used to determine the candidate’s score on the assessment.

Pretest items, however, do not contribute to a candidate’s score. Candidates are not able to differentiate between operational and pretest items.

This process allows ATI to collect high-quality administration data. Once we have this data, we examine the items for their psychometric performance. For example, we review their difficulty and their ability to discriminate between high-performing and low-performing candidates. This step has become increasingly important with the introduction of new item types designed by the NCSBN to assess clinical judgment.

Items that don’t perform well are reviewed and potentially revised.

Test security for proactive protection

ATI employs a variety of technical controls and test designs

EMBEDDED CONTROLS

Built within proctored assessments, this preventive measure alerts the proctor if a candidate attempts to navigate away from the assessment, which could indicate exam misconduct or compromise of exam items.

APPLICATION SECURITY REVIEWS

Web application security reviews identify security risks associated with high-severity vulnerabilities found by an independent third-party (professional hacker) in our environment.

PROPRIETARY TECHNOLOGY FOR DETECTING IP (INTELLECTUAL PROPERTY)

This technology mitigates risk associated with content theft and assists in easier identification of ATI content with quick determination whether the content has been compromised. As the Next Generation NCLEX item types have been introduced, content theft has become increasingly common.

ENHANCED ASSESSMENT DISTRIBUTION

ATI randomly distributes a set of items nationwide to decrease item exposure and lessen predictability.

DO YOUR STUDENTS UNDERSTAND THE IMPORTANCE OF ASSESSMENTS AS THEY PROGRESS THROUGH NURSING SCHOOL? SHARE IN THE COMMENTS BELOW.

Educator Blog