What should design students know about AI?
Researchers in Guangzhou developed a 23-item scale covering technical skills, tool use, ethics and originality, perceived value, and independent evaluation of AI output. Two Chinese student samples supported the five-factor structure, but the instrument still needs cross-cultural and outcome validation.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1The researchers turned 12 interviews into an initial item pool, then used exploratory factor analysis with 294 design students and confirmatory factor analysis with a separate sample of 325.
- 2The final 23-item instrument has five proposed dimensions: concepts and technical skills, tool proficiency, ethics and originality, recognition of AI's value in design, and independent evaluation of AI outputs.
- 3The study validates a measurement structure, not a curriculum or learning gain. Both samples were convenience-recruited from two Chinese public universities, and one subscale's reliability was marginal.
Research topic
Development and early psychometric validation of a discipline-specific AI-literacy scale for university design students
The study asked what AI literacy means inside design practice
Generic AI-literacy frameworks usually combine technical understanding, practical use, evaluation and ethics. The authors argue that design education adds distinctive pressures: students use generative systems during ideation and production, must judge output rather than merely retrieve information, and face questions about originality, authorship, copyright and over-reliance. Their research question was therefore not whether AI improves design education. It was whether a defensible, discipline-specific set of self-report items could represent the capabilities design students need and behave consistently as a measurement scale.
That distinction matters for educators. A scale can help organize curriculum or identify areas where students report less confidence, but it does not itself teach those abilities or demonstrate them. The proposed instrument—the Artificial Intelligence Literacy Scale for Design Students, or AILS-DS—contains 23 agreement statements. Its five dimensions combine knowledge and technical operation, selection and use of tools, ethical and originality awareness, recognition of AI's value in design, and independent evaluation of generated outputs.[1]
Interviews shaped the first item pool
The researchers began with semi-structured interviews involving 12 people selected through purposive snowball sampling. The group included participants in AI development and research, design educators and design students who frequently used AI tools. Two researchers independently reviewed transcripts and reconciled differences. The process produced 110 phrases or statements, which were consolidated into 40 candidate items. Two domain experts then reviewed relevance and wording, leaving 31 items rated on a five-point agreement scale.
This qualitative phase gives the instrument contact with real design practice, including prompting, image-generation parameters, local or cloud deployment, copyright, disclosure of AI assistance and resistance to automation bias. Yet 12 interviews and two expert reviewers cannot establish that the item pool covers every design discipline or cultural context. Graphic, product, fashion, service and interaction design may place different weight on model training, image tools, client communication, accessibility and material constraints. Content validity remains a programme of research, not a completed property.[1]
Separate samples were used to discover and test the structure
For exploratory factor analysis, the team collected 320 questionnaires in June and July 2025 and retained 294 eligible design-student responses. Participants came from a prominent fine-arts academy and a comprehensive public university in China, reached through WeChat groups with course instructors' assistance. The team used principal-axis factoring with an oblique rotation and parallel analysis. Eight weak or cross-loading items were removed. The resulting five-factor, 23-item solution explained 50.54% of the variance and produced an overall Cronbach's alpha of 0.838.
A second, independent convenience sample was collected in April and May 2026. Of 354 responses, 325 remained after 29 failed attention checks. Confirmatory factor analysis using an estimator suited to ordinal responses produced an RMSEA of 0.064, CFI of 0.915, TLI of 0.902 and SRMR of 0.061—values the authors interpret as acceptable fit. Standardized item loadings ranged from 0.518 to 0.794. Using separate discovery and validation samples is a meaningful strength because it reduces the risk of declaring a structure valid in the same data that created it.[1]
Five factors make an educational claim, not a universal definition
The first factor combines concepts with hands-on technical skills, such as understanding model logic, interpreting image-generation parameters, prompting, small-sample training and deployment. The second concerns selecting and coordinating tools in a workflow. The third focuses on copyright risk, originality, ethical debate and disclosure of AI assistance. The fourth measures whether students recognize value in designer-led, AI-assisted work. The fifth isolates independent evaluation: maintaining judgment, drawing on experience and resisting over-reliance on generated output.
That fifth factor is the study's most useful curricular provocation. Tool competence and critical evaluation are not the same. A student may produce polished output quickly while failing to notice factual, cultural, legal or aesthetic problems. Still, factor analysis does not reveal a natural law. The structure depends on the proposed items and responses from this setting. Other cohorts might merge, split or reject dimensions, especially as models and design practice change. Educators should treat the five factors as a testable framework rather than a universal checklist.[1]
Reliability was encouraging overall but uneven by subscale
In the confirmatory sample, overall internal consistency was 0.852. Subscale alpha values ranged from 0.697 to 0.752. The concepts-and-technical-skills factor sat at 0.697, just below the conventional 0.70 rule of thumb, so small differences on that subscale should be interpreted cautiously. Item-discrimination estimates ranged from 0.95 to 2.69. Those results support further use in research, but reliability is not validity: a set of questions can produce consistent scores without measuring real competence or predicting responsible design performance.
The instrument is also self-report. Agreement with statements about tool skill, ethics or independent judgment may reflect confidence, aspiration or socially desirable responding. The study did not compare scores with a practical design task, expert assessment, portfolio quality, copyright decisions, detection of flawed output or performance after instruction. Nor did it establish test-retest stability or sensitivity to genuine learning. Before universities use scores for progression or accreditation, they need evidence that the scale measures more than students' perceptions of themselves.[1]
The sample limits international and institutional claims
Both quantitative samples were recruited by convenience through student WeChat groups at two Chinese public universities, with instructors helping distribute the link. Students were told participation was anonymous and would not affect grades, and they received 10 yuan. The authors acknowledge that instructor involvement may have created perceived pressure or social-desirability bias. The samples are adequate for the reported analyses but modest for scale development, and the paper does not establish equivalence across gender, course, year, institution or design specialization.
The Guangzhou Academy of Fine Arts ethics committee approved the study and participants gave online consent. Chinese education and Guangdong research programmes funded the work; the authors declared no competing financial or personal interests. Those disclosures are reassuring but do not solve external validity. Translation, curriculum, tool availability, copyright law, studio culture and labour-market expectations can reshape what students understand by an item. A direct score comparison across countries would be premature without careful translation and measurement-invariance testing.[1]
What educators can do now—and what would change the assessment
The framework is most defensible as a low-stakes diagnostic conversation. A programme could map teaching against the five dimensions, ask students where confidence is low, and pair self-report with studio critique or practical tasks. It could ensure that technical instruction is not separated from attribution, copyright, originality and independent evaluation. Pre- and post-course scores might generate hypotheses about change, but the authors' paper does not yet show that a score increase represents better work, safer practice or lasting learning.
The assessment would strengthen through preregistered replication in varied countries, institutions and design disciplines; cognitive interviews testing how students interpret each item; measurement-invariance analysis; test-retest evidence; and comparison with observable performance. Studies should examine whether scores predict the ability to challenge hallucinated or derivative output, document AI assistance, choose an appropriate tool, defend a design decision and preserve human originality. Because tools evolve quickly, the item set will also need planned revision without making year-to-year results meaningless.[1]
What this means for people
- Students could receive more precise support than a single generic AI-literacy score if programmes use the five dimensions diagnostically.
- Educators can connect tool training with originality, disclosure and independent judgment rather than treating prompting as the whole curriculum.
- High-stakes use would be premature: self-report scores should not determine progression, hiring or competence without performance validation.
Global context
The scale was developed in Guangzhou within Chinese public higher education, where tool access, curriculum and professional norms differ from those elsewhere. Its focus on designer-led judgment, originality and disclosure has wider relevance, but international use requires more than translation. Researchers must test whether the same factor structure holds and whether scores mean the same thing across languages, institutions and design disciplines.
What the evidence does not yet show
- Both samples were convenience-recruited from two public universities in China, limiting generalization across institutions, countries and design disciplines.
- The scale measures self-reported agreement, not observed design skill, learning, originality, ethical conduct or labour-market performance.
- One subscale had Cronbach's alpha of 0.697, slightly below the conventional 0.70 threshold, and the samples were modest for scale development.
- Instructor-assisted recruitment may have introduced perceived pressure or socially desirable answers despite anonymity and voluntary consent.
- The study did not report cross-cultural measurement invariance, test-retest stability or sensitivity to verified educational change.
What to watch next
- Independent replication across countries, languages, institutions and specific design disciplines.
- Validation against practical studio tasks, expert ratings, error detection, attribution and copyright decisions.
- Whether scores change after a course and whether those changes persist or predict better work.
- Transparent updating as generative tools and professional expectations evolve.
Living evidence record
Impact record IAI-1K4M065
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
5 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what International Journal of Technology and Design Education published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 5 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
Education
Does the way students use AI matter?
A peer-reviewed survey of 713 Saudi undergraduates found that supportive and creative AI use was associated with stronger self-reported creativity and metacognitive awareness, while substitutive use showed negative associations. The one-time self-report design cannot establish what AI caused—or whether skills improved.
8 min · 1 source
Education
What should teachers learn about AI? UNESCO reviews 36 competency frameworks
A new UNESCO IITE analysis compares national, institutional and academic frameworks for teacher AI competence. It offers a policy checklist, but does not test whether any framework improves classroom outcomes.
5 min · 2 sources
Education
PISA analysis separates frequent AI use from effective learning
New OECD analysis of PISA evidence explores how students use AI and digital tools alongside reading skills, warning that access or frequency alone does not establish deeper understanding.
2 min · 1 source
The Impact Brief
Keep the evidence trail, not the noise.
Get the most consequential AI developments with direct sources and clear limits.
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.